1. Big Picture

1.1 Levels and Layers of Abstraction in a Linux System

  • The kernel is software residing in memory that tells the CPU what to do. The kernel manages the hardware and acts primarily as an interface between the hardware and any running program.
  • Processes - the running programs that the kernel manages - collectively make up the system’s upper level, called user space.
  • The kernel runs in kernel mode, and the user processes run in user mode. Code runnning in kernel mode has unrestricted access to the processor and main memory.
  • The act of one process giving up control of the CPU to another process is called a context switch. The kernel is responsible forcontext switching.

1.3 The Kernel

1.3.1 Context Switching

  1. The CPU (the actual hardware) interrupts the current process based on an internal timer, switches into kernel mode, and hands control back to the kernel.
  2. The kernel records the current state of the CPU and memory, which will be essnetial to resuming the process that was just interrupted.
  3. The kernel performs ans tasks that might have come up during the preceding time slice (such as collecting data from input and output, or I/O, operations).
  4. The kernel is now ready to let another process run. The kernel analyzes the list of processes that are ready to run and choose one.
  5. The kernel prepares the memory for this new process, and then prepares the CPU.
  6. The kernel tells the CPU how long the time slice for the new process will last.
  7. The kernel switches the CPU into user mode and hands control of the CPU to the process.
  • Kernel runs between process time slices during a context switch.

1.3.4 System Calls and Support

  • fork(): When a process calls fork(), the kernel creates a nearly identical copy of the process.
  • exec(): When a process calls exec(program), the kernel starts program, replacing the current process.
  • Other than init(), all user processes on a Linux system start as a result of fork(), and most of the time, you also run exec() to start a new program instead of running a copy of an existing process.

1.5 Users

  • A user is an entity that can run processes and own files. A user is associated with a username. Kernel identifies users by userids instead of usernames.
  • Users exists primarily to support permissions and boundaries.
  • The root user is an exception to the preceding rules because root may terminate and alter another user’s processes and read any file on the local system. For this reason, root is known as the superuser. A person who can operate as root is said to have root access and is an administrator on a traditional Unix system.
  • As powerful as the root user is, it still runs in the operating system’s user mode, not kernel mode.

2. Basic Commands and Directory Hierachy

2.2 Using the Shell

  • cat outputs the contents of one or more files.
$ cat file1 file2 ...
  • Unix processes use I/O streams to read and write data. Processes read data from input streams and write data to output streams.
  • Ctrl-D stops the current standard input entry from the terminal (sometimes results in program termination), while Ctrl-C terminates a program.

2.3 Basic Commands

  • cp copies files, mv rename/move files.
$ cp file1 file2
$ cp file1 ... fileN dir
$ mv file1 file2
$ mv file1 ... fileN dir
  • touch command creates a file. If the file already exists, touch does not change it, but it does update the file’s modification time stamp printed with the ls -l command.
$ touch file
  • echo command prints its arguments to the standard output (useful for finding expansions of shell globs and variables).

2.4 Nagivating Directories

  • rmdir can remove an empty directory, use rm -rf to remove a non-empty directory.
  • Use globbing (wildcard) for pattern matching. * matches any number (including 0) of arbitary characters, ? matches exactly one arbitary character.
  • If no files match a glob, the shell performs no expansion, and the command runs with literal characters such as *.
  • If you don’t want the shell to expand a glob in a command, enclose the glob in single quotes ('').
$ echo Desk*top
Desktop

$ echo Desk'*'top
Desk*top

$ echo some*randomText
some*randomText

$ echo Desk?op
Desktop

$ echo Desk?top
Desk?top
  • It is important to remember that the shell performs expansions before running commands, and only then. Therefore, if a * makes it to a command without expanding, the shell will do nothing more with it; it’s up to the command to decide what it wants to do.

2.5 Intermediate Commands

  • The grep command prints the lines from a file or input steam that match an expression.
# Print the lines in the /etc/passwd file that contain the text "root"
$ grep root /etc/passwd

# Check every file in /etc that contains the word "root"
$ grep root /etc/*
  • grep -i performs case-insensitive match, grep -v invert the search (lines don’t match).

  • Use less to view large files (paged).

  • When inside less, use /word to forward search ‘word’ inside the file, use ?word to backward search; use n to continue searching.

# Use less to view result of grep
$ grep ap /usr/share/dict/words | less
  • Use diff [-u] file1 file2 to check the differences between two files.
  • file file1 shows the format of file1.
  • find dir -name file -print finds file in dir.
  • You must enclose special pattern-matching characters such as * in single quotes to protect the special characters from the shell’s own globbing feature.
$ find Developer -name tree'*'.py -print
Developer/trees/util/tree_alg.py
Developer/trees/tree/tree_node.py
  • Use head or tail to view the first/last 10 lines of a file.
# Show first 10 lines of file1.
$ head file1

# Show last 5 lines of file1.
$ tail -n 5 file1
  • The sort command quickly puts the lines of a text file in alphanumeric order. Use -n to sort in numerical order, and -r to reverse the order.
# Display the first 20 lines in /etc/passwd in descending order
$ head -n 20 /etc/passwd | sort -r

2.6 Changing Your Password and Shell

  • Use passwd to change current user’s password.
  • Use chsh to change shell.

2.8 Environment and Shell Variables

  • The shell can store temporary variables, called shell variables, containing the values of text strings.
  • An environment variable is like a shell variable, but it’s not specific to the shell. The operating system passes all of your shell’s environment variables to programs that the shell runs, whereas shell variables cannot be accessed in the commands that you run. Assign an environment variable with the shell’s export command.
# Set shell variable "DB_USER" to have value "shuyang".
$ DB_USER=shuyang

# Assign "DB_USER" as an environment variable.
$ export DB_USER

2.9 The Command Path

  • Command path (path for short) is a list of system directories that the shell searches when trying to locate a command.
  • If programs with the same name appear in several directories in the path, the shell runs the first matching program.
  • Use PATH=dir:$PATH to insert dir at the beginning of PATH.

2.11 Command-Line Editing

Keystroke Action
Ctrl - B Move the cursor left one letter
Alt - B Move the cursor left one word
Ctrl - F Move the cursor right one letter
Alt - F Move the cursor right one word
Ctrl - P View the previous command
Ctrl - N View the next command
Ctrl - A Move the cursor to the beginning of the line
Ctrl - E Move the cursor to the end of the line
Ctrl - W Erase the preceding word
Alt - D Erase the following word
Ctrl - U Erase from cursor to beginning of line
Ctrl - K Erase from cursor to end of line
Ctrl - Y Paste erased text (of example, from Ctrl - U)

2.14 Shell Input and Output

  • $ command > file sends the output of command to file. If the files does not exist, it creates the file; if the file already exists, it erases the original content.

  • $ command >> file appends the output of command to file.

  • $ command1 | command2 (the pipe character) sends the standard output of command1 to the standard input of command2.

  • Use stream ID to redirect standard error.

# Redirects ls' standard output to stdout.txt, and standard error to stderr.txt.
# 2 is standard error's stream id.
$ ls /fffffffff > stdout.txt 2> stderr.txt

# Send standard error to the same place as stdout with the `>&` notation.
$ ls /fffffffff > stdout.txt 2>&1

Common File Commands

Command Description Directory?
cmp Compare two files to see if they are the same. [ ]
du Display disk usage for files. [x]
file Analyze file type. [ ]
find Search file in directory tree, then process results. [ ]
whereis Find files associated with a command. [ ]