01. Review of Common Linux Utilities

Working With The Shell

• The shell is case sensitive
• To quickly repeat a previous command:
  • Use the up arrow key to step back the list of commands you’ve run
  • Use the down arrow key to step forward the list of commands you’ve run
  • At the command you want to repeat, hit the enter key
• Use tab to complete a long filename
• Use control-u to quickly erase a long command line
• If a command line is too long, use \ before the enter key. This escapes the newline and the shell will interpret both lines of text as one command line
• The default secondary prompt is >
  • When you see the secondary prompt, it means the shell is expecting more input on the command line
• When the shell interprets the command line, it uses 1 or more whitespaces as a delimiter:
  • The first word on the command line is interpreted to be a utility name or an executable file (a compiled binary file or a shell script)
  • All subsequent words preceded by – or + are interpreted as an option or list of options
  • All subsequent words without – or + are interpreted as arguments
  • If the arguments have wildcard characters, the shell expands the wildcards to match existing filenames
  • All options and arguments are passed to the executable without further error checking
  • If there are redirection, pipe, tee, or job control metacharacters, the shell takes the appropriate action with the input or output, and how it runs the executable

Files on Linux

• A file can mean any of the 7 types of files on Linux, including the 3 types used in this notes: regular files, directories, links
• Each file has a unique inode number to identify it to the system. Therefore, filenames that are hard linked together have the same inode number
• Filenames should be descriptive
  • Filenames should not contain any of the following shell metacharacters :
    • & ; | * ? ~ ! $ ^ # / \ ‘ “ ` [ ] ( ) { } < > and space
    • (after this review note, you should be able to explain why each of the characters should not be used for filenames)
• Wildcards can be used as arguments, in place of or in addition to filenames
  • Wildcard characters are: * ? [ ]

General Purpose Utilities

• Change a password: passwd
• Stop the current process (useful for any utility that doesn’t have a quit option, and useful for logging out of the system): exit , control-d
• Abort (kill) the current process (useful ‘emergency brake’ for cases such as an infinite loop that keeps printing out to screen in a foreground process): control-c
• Information on users on the system: who, whoami, w, finger
• Information about your system and your terminal: tty, stty, uname, clear
• Recording all screen output to a file: script
• Print a hard copy: lpr
• Get more information on a utility: man, info
• Get current date / time: date, cal
• Print a line of text to screen: echo
• Spawn a new shell: sh, bash, tcsh, ksh, …

File Utilities

• Show content of a file: cat, more, less, head, tail
• List a directory: ls
• Commonly used options: a, l, F, d, i
• Create a new file or update the time stamp of an existing file: touch
• Copy a file: cp
• Rename or move a file: mv
• Delete a file: rm
• To change the access rights of a file: chmod
• To change default access rights of files: umask
• To recursively find files that match certain criteria: find
  • To do work on files that match the criteria, use the –exec option of find or the xargs utility
• To archive multiple files into one archive file: tar
• To compress a file: gzip, gunzip

Directory Utilities and Link Utilities

• There are 2 types of directory paths:
  • absolute path (starts at root)
  • relative path (starts from the current directory)
• Some special directory symbols: . .. ~ ~userID
• Show current directory: pwd
• Change directory: cd
• Create a new directory: mkdir
• Show the location of a utility: whereis
• Show the actual utility that runs when typing a command name: which
• There are 2 types of links:
  • Hard link (actual location of file on hard disk)
  • Symbolic link (logical path to the file location)
• To create a link: ln

vim

• Native text editor of Linux
• 2 modes: command mode to issue command, and insert mode to type in text
  • To go from insert mode to command mode: escape
• vim is a useful utility for this class since your shell scripts are created by using vim
• For the voyager system vim has been aliased to vi

Basic commands:

• To move: arrow keys, h, j, k, l, 0, $, control-f, control-b, G, nG (where n is a number)
• To add text: i, I, a, A, o, O
• To delete text: x, dw, dd, D
• To replace text: r, cw, cc
• To find and substitute: :s
• To search: / or ?, then n or N
• To copy and paste: yw or yy, then p or P
• To cut and paste: dw or dd, then p or P
• To join 2 lines: J
• To repeat the previous command: .
• To run a command n times: n<command>
• To undo: u
• To bring in another file: :r
• To save: :w
• To quit: :q

mail

• Native email utility of Linux
• 2 modes: send mode and receive mode
• The mail utility is useful for this class because you will receive feedback of your lab assignment through voyager mail
• Send mode
  • Command to get into send mode: mail email_addr
  • Command to quit send mode:
    • control-d send
    • control-c exit without send
  • Commands in send mode:
    • ~v use vi to edit the message
    • ~m n n is a number, bring in an existing message n
    • ~r filename bring in a text file with name filename
• Receive mode
  • Command to get into receive mode: mail
  • Commands in receive mode
    • (n is the optional message number, if there is no message number, the current message is the default message)
    • n enter Show message n
    • R n reply to message n
    • d n delete message n
    • u n undo delete of message n
    • s n file save message n to file
    • q quit out of mail

Features of the Shell

• Standard input comes the keyboard, standard output and error go to screen
• To redirect input, output, error from/to a file, use: < (from file) and > or >> (to file)
• To send output of one command to the input of a second command, use the pipe: |
• To send the output of one command to both a file and another command, use a tee: tee
• Multiple commands on a command line can be grouped with ( ) or be separated with ;
• To get the literal meaning of shell metacharacters, use single quotes ‘ ’, double quotes “ ”, or back slash \
• To do command substitution, use back single quotes `command` or use $(command)
• To run a job in the background, add & at the end of the command line
• A job or process can be in 1 of 4 states: running in foreground, stopped, running in background, or terminated
  • To switch from foreground to background: control-z, then bg
  • To switch from background to foreground: fg
  • To stop a job running in foreground: control-z
  • To terminate a job: control-c (job in foreground), kill (job in background)
• To customize a command: alias
• To create a variable for temporary data storage: varName=value
• To access variable: $varName
• To see environment variables: env

Filters

• To see content of a file: more, less, head, tail, cut, paste
• To filter consecutive identical lines: uniq
• To count lines, words, characters in a file: wc
• To see difference in 2 files: diff, cmp
• To change characters in a file: tr
• To sort lines of a file: sort
• To select certain lines of a file: fgrep, grep, egrep
• To modify manipulate input text lines: sed , awk
  • sed is useful when you want to reformat parts of a line
  • awk is useful when you need to manipulate fields of a line by using programming constructs (such as loops, if statements, arrays…
    • Shell variables can be passed to awk by using the –v option for each variable you need to pass to awk:
      • num=5 # set shell variable num to 5
      • awk –v n=$num ‘{ print n }’ fileA
      • # set awk variable n to the same value as num
      • # awk will print 5 for each line of fileA

Regular Expression

• Used for pattern matching by certain utilities
• The common ones are: egrep, sed, awk, and vim substitute
• Most characters in a regex have literal meaning, but some are metacharacters, as shown:
  • [ ] any one character within the [ ]
  • [^ ] any one character not within the [ ]
  • . any one character, except newline
  • ^ begin of line anchor
  • $ end of line anchor
  • {n} n of the previous character
  • {n,m} n to m of the previous character
  • ? 0 or 1 of the previous character
  • + 1 or more of the previous character
  • * 0 or more of the previous character
  • | alternate (or)
  • ( ) group
  • \ take literal meaning of the next character