01. Review of Common Linux Utilities

Working With The Shell

• The shell is case sensitive

• To quickly repeat a previous command:

  • Use the up arrow key to step back the list of commands you’ve run

  • Use the down arrow key to step forward the list of commands you’ve run

  • At the command you want to repeat, hit the enter key

• Use tab to complete a long filename

• Use control-u to quickly erase a long command line

• If a command line is too long, use \ before the enter key. This escapes the newline and the shell will interpret both lines of text as one command line

• The default secondary prompt is >

  • When you see the secondary prompt, it means the shell is expecting more input on the command line

• When the shell interprets the command line, it uses 1 or more whitespaces as a delimiter:

  • The first word on the command line is interpreted to be a utility name or an executable file (a compiled binary file or a shell script)

  • All subsequent words preceded by – or + are interpreted as an option or list of options

  • All subsequent words without – or + are interpreted as arguments

  • If the arguments have wildcard characters, the shell expands the wildcards to match existing filenames

  • All options and arguments are passed to the executable without further error checking

  • If there are redirection, pipe, tee, or job control metacharacters, the shell takes the appropriate action with the input or output, and how it runs the executable

Files on Linux

• A file can mean any of the 7 types of files on Linux, including the 3 types used in this notes: regular files, directories, links

• Each file has a unique inode number to identify it to the system. Therefore, filenames that are hard linked together have the same inode number

• Filenames should be descriptive 

  • Filenames should not contain any of the following shell metacharacters : 

    • &  ;  |  *  ?  ~  !  $  ^  #  /  \  ‘  “  `  [  ]  (  )  {  }  <  >  and space

    • (after this review note, you should be able to explain why each of the characters should not be used for filenames)

• Wildcards can be used as arguments, in place of or in addition to filenames

  • Wildcard characters are:   *    ?    [ ]

General Purpose Utilities

• Change a password: passwd

• Stop the current process (useful for any utility that doesn’t have a quit option, and useful for logging out of the system): exit , control-d

• Abort (kill) the current process (useful ‘emergency brake’ for cases such as an infinite loop that keeps printing out to screen in a foreground process): control-c

• Information on users on the system: who, whoami, w, finger

• Information about your system and your terminal: tty, stty, uname, clear

• Recording all screen output to a file: script

• Print a hard copy: lpr

• Get more information on a utility: man, info

• Get current date / time: date, cal

• Print a line of text to screen: echo

• Spawn a new shell: sh, bash, tcsh, ksh, …

File Utilities

• Show content of a file: cat, more, less, head, tail

• List a directory: ls

• Commonly used options: a, l, F, d, i

• Create a new file or update the time stamp of an existing file: touch

• Copy a file: cp

• Rename or move a file: mv

• Delete a file: rm

• To change the access rights of a file:  chmod

• To change default access rights of files:  umask

• To recursively find files that match certain criteria: find

  • To do work on files that match the criteria, use the –exec option of find or the xargs utility

• To archive multiple files into one archive file: tar

• To compress a file: gzip, gunzip

Directory Utilities and Link Utilities

• There are 2 types of directory paths: 

  • absolute path (starts at root)

  • relative path (starts from the current directory)

• Some special directory symbols:   .     ..      ~     ~userID

• Show current directory:  pwd 

• Change directory:  cd

• Create a new directory:  mkdir

• Show the location of a utility:  whereis

• Show the actual utility that runs when typing a command name:  which

• There are 2 types of links: 

  • Hard link (actual location of file on hard disk)

  • Symbolic link (logical path to the file location)

• To create a link: ln

vim

• Native text editor of Linux

• 2 modes: command mode to issue command, and insert mode to type in text

  • To go from insert mode to command mode: escape

• vim is a useful utility for this class since your shell scripts are created by using vim

• For the voyager system vim has been aliased to vi

Basic commands:

• To move: arrow keys, h, j, k, l, 0, $, control-f, control-b, G, nG (where n is a number)

• To add text: i, I, a, A, o, O

• To delete text: x, dw, dd, D

• To replace text: r, cw, cc

• To find and substitute: :s

• To search: / or ?, then n or N

• To copy and paste: yw or yy, then p or P

• To cut and paste: dw or dd, then p or P

• To join 2 lines: J

• To repeat the previous command:  .

• To run a command n times:  n<command>

• To undo:  u

• To bring in another file:  :r

• To save:  :w

• To quit:  :q

mail

• Native email utility of Linux

• 2 modes: send mode and receive mode

• The mail utility is useful for this class because you will receive feedback of your lab assignment through voyager mail

• Send mode

  • Command to get into send mode:    mail  email_addr

  • Command to quit send mode: 

    • control-d send

    • control-c exit without send

  • Commands in send mode:

    • ~v use vi to edit the message

    • ~m  n n is a number, bring in an existing message n

    • ~r filename bring in a text file with name filename

• Receive mode

  • Command to get into receive mode:    mail

  • Commands in receive mode

    • (n is the optional message number, if there is no message number, the current message is the default message)

    • n  enter Show message n

    • R  n reply to message n

    • d  n delete message n

    • u  n undo delete of message n

    • s  n file save message n to file

    • q quit out of mail

Features of the Shell

• Standard input comes the keyboard, standard output and error go to screen

• To redirect input, output, error from/to a file, use:    <   (from file)   and     >   or    >>   (to file)

• To send output of one command to the input of a second command, use the pipe:  |

• To send the output of one command to both a file and another command, use a tee:  tee

• Multiple commands on a command line can be grouped  with ( ) or be separated with ;

• To get the literal meaning of shell metacharacters, use single quotes ‘ ’, double quotes “ ”, or back slash  \

• To do command substitution, use back single quotes `command` or use $(command)

• To run a job in the background, add & at the end of the command line

• A job or process can be in 1 of 4 states: running in foreground, stopped, running in background, or terminated

  • To switch from foreground to background: control-z, then bg

  • To switch from background to foreground: fg

  • To stop a job running in foreground: control-z

  • To terminate a job: control-c (job in foreground), kill (job in background)

• To customize a command: alias

• To create a variable for temporary data storage:  varName=value

• To access variable: $varName

• To see environment variables: env

Filters

• To see content of a file: more, less, head, tail, cut, paste

• To filter consecutive identical lines: uniq

• To count lines, words, characters in a file: wc

• To see difference in 2 files: diff, cmp

• To change characters in a file: tr

• To sort lines of a file: sort

• To select certain lines of a file: fgrep, grep, egrep

• To modify manipulate input text lines: sed , awk

  • sed is useful when you want to reformat parts of a line

  • awk is useful when you need to manipulate fields of a line by using programming constructs (such as loops, if statements, arrays…

    • Shell variables can be passed to awk by using the –v option for each variable you need to pass to awk:

      • num=5 # set shell variable num to 5

      • awk –v n=$num ‘{ print n }’ fileA 

      • # set awk variable n to the same value as num

      • # awk will print 5 for each line of fileA

Regular Expression

• Used for pattern matching by certain utilities

• The common ones are: egrep, sed, awk, and vim substitute

• Most characters in a regex have literal meaning, but some are metacharacters, as shown:

  • [ ] any one character within the [ ]

  • [^ ] any one character not within the [ ]

  •  .   any one character, except newline

  • ^ begin of line anchor

  • $ end of line anchor

  • {n} n of the previous character

  • {n,m}    n to m of the previous character  

  • ? 0 or 1 of the previous character

  • + 1 or more of the previous character

  • *     0 or more of the previous character

  •  | alternate (or)

  • ( ) group 

  •  \ take literal meaning of the next character