Overview
- During this topic you will learn how to write shell scripts (programs)
- The prerequisites to learn shell scripting are:
- Introduction to Unix/Linux notes
- Common Unix/Linux utilities
- Shell interaction with the commands, such as pipe, redirection, job control
- More advanced utilities such as egrep, sed, awk, find
- Any Programming Language
- Basic programming constructs, such as loops, if else, functions and argument passing
- Basic data storage concept, such as variables, arrays, strings
- Basic testing and debugging techniques
Working With The Shell
- The shell is case sensitive
- To quickly repeat a previous command:
- Use the up arrow key to step back the list of commands you’ve run
- Use the down arrow key to step forward the list of commands you’ve run
- At the command you want to repeat, hit the enter key
- Use tab to complete a long filename
- Use control-u to quickly erase a long command line
- If a command line is too long, use \ before the enter key. This escapes the newline and the shell will interpret both lines of text as one command line
- The default secondary prompt is >
- When you see the secondary prompt, it means the shell is expecting more input on the command line
- When the shell interprets the command line, it uses 1 or more whitespaces as a delimiter:
- The first word on the command line is interpreted to be a utility name or an executable file (a compiled binary file or a shell script)
- All subsequent words preceded by – or + are interpreted as an option or list of options
- All subsequent words without – or + are interpreted as arguments
- If the arguments have wildcard characters, the shell expands the wildcards to match existing filenames
- All options and arguments are passed to the executable without further error checking
- If there are redirection, pipe, tee, or job control metacharacters, the shell takes the appropriate action with the input or output, and how it runs the executable
Files on Linux
- A file can mean any of the 7 types of files on Linux, including the 3 types used in this notes: regular files, directories, links
- Each file has a unique inode number to identify it to the system. Therefore, filenames that are hard linked together have the same inode number
- Filenames should be descriptive
- Filenames should not contain any of the following shell metacharacters :
- & ; | * ? ~ ! $ ^ # / \ ‘ “ ` [ ] ( ) { } < > and space
- (after this review note, you should be able to explain why each of the characters should not be used for filenames)
- Wildcards can be used as arguments, in place of or in addition to filenames
- Wildcard characters are: * ? [ ]
General Purpose Utilities
- Change a password: passwd
- Stop the current process (useful for any utility that doesn’t have a quit option, and useful for logging out of the system): exit , control-d
- Abort (kill) the current process (useful ‘emergency brake’ for cases such as an infinite loop that keeps printing out to screen in a foreground process): control-c
- Information on users on the system: who, whoami, w, finger
- Information about your system and your terminal: tty, stty, uname, clear
- Recording all screen output to a file: script
- Print a hard copy: lpr
- Get more information on a utility: man, info
- Get current date / time: date, cal
- Print a line of text to screen: echo
- Spawn a new shell: sh, bash, tcsh, ksh, …
File Utilities
- Show content of a file: cat, more, less, head, tail
- List a directory: ls
- Commonly used options: a, l, F, d, i
- Create a new file or update the time stamp of an existing file: touch
- Copy a file: cp
- Rename or move a file: mv
- Delete a file: rm
- To change the access rights of a file: chmod
- To change default access rights of files: umask
- To recursively find files that match certain criteria: find
- To do work on files that match the criteria, use the –exec option of find or the xargs utility
- To archive multiple files into one archive file: tar
- To compress a file: gzip, gunzip
Directory Utilities and Link Utilities
- There are 2 types of directory paths:
- absolute path (starts at root)
- relative path (starts from the current directory)
- Some special directory symbols: . .. ~ ~userID
- Show current directory: pwd
- Change directory: cd
- Create a new directory: mkdir
- Show the location of a utility: whereis
- Show the actual utility that runs when typing a command name: which
- There are 2 types of links:
- Hard link (actual location of file on hard disk)
- Symbolic link (logical path to the file location)
- To create a link: ln
vim
- Native text editor of Linux
- 2 modes: command mode to issue command, and insert mode to type in text
- To go from insert mode to command mode: escape
- vim is a useful utility for this class since your shell scripts are created by using vim
- For the voyager system vim has been aliased to vi
Basic commands:
- To move: arrow keys, h, j, k, l, 0, $, control-f, control-b, G, nG (where n is a number)
- To add text: i, I, a, A, o, O
- To delete text: x, dw, dd, D
- To replace text: r, cw, cc
- To find and substitute: :s
- To search: / or ?, then n or N
- To copy and paste: yw or yy, then p or P
- To cut and paste: dw or dd, then p or P
- To join 2 lines: J
- To repeat the previous command: .
- To run a command n times: n<command>
- To undo: u
- To bring in another file: :r
- To save: :w
- To quit: :q
mail
- Native email utility of Linux
- 2 modes: send mode and receive mode
- The mail utility is useful for this class because you will receive feedback of your lab assignment through voyager mail
- Send mode
- Command to get into send mode: mail email_addr
- Command to quit send mode:
- control-d send
- control-c exit without send
- Commands in send mode:
- ~v use vi to edit the message
- ~m n n is a number, bring in an existing message n
- ~r filename bring in a text file with name filename
- Receive mode
- Command to get into receive mode: mail
- Commands in receive mode
- (n is the optional message number, if there is no message number, the current message is the default message)
- n enter Show message n
- R n reply to message n
- d n delete message n
- u n undo delete of message n
- s n file save message n to file
- q quit out of mail
Features of the Shell
- Standard input comes the keyboard, standard output and error go to screen
- To redirect input, output, error from/to a file, use: < (from file) and > or >> (to file)
- To send output of one command to the input of a second command, use the pipe: |
- To send the output of one command to both a file and another command, use a tee: tee
- Multiple commands on a command line can be grouped with ( ) or be separated with ;
- To get the literal meaning of shell metacharacters, use single quotes ‘ ’, double quotes “ ”, or back slash \
- To do command substitution, use back single quotes `command` or use $(command)
- To run a job in the background, add & at the end of the command line
- A job or process can be in 1 of 4 states: running in foreground, stopped, running in background, or terminated
- To switch from foreground to background: control-z, then bg
- To switch from background to foreground: fg
- To stop a job running in foreground: control-z
- To terminate a job: control-c (job in foreground), kill (job in background)
- To customize a command: alias
- To create a variable for temporary data storage: varName=value
- To access variable: $varName
- To see environment variables: env
Filters
- To see content of a file: more, less, head, tail, cut, paste
- To filter consecutive identical lines: uniq
- To count lines, words, characters in a file: wc
- To see difference in 2 files: diff, cmp
- To change characters in a file: tr
- To sort lines of a file: sort
- To select certain lines of a file: fgrep, grep, egrep
- To modify manipulate input text lines: sed , awk
- sed is useful when you want to reformat parts of a line
- awk is useful when you need to manipulate fields of a line by using programming constructs (such as loops, if statements, arrays…
- Shell variables can be passed to awk by using the –v option for each variable you need to pass to awk:
- num=5 # set shell variable num to 5
- awk –v n=$num ‘{ print n }’ fileA
- # set awk variable n to the same value as num
- # awk will print 5 for each line of fileA
Regular Expression
- Used for pattern matching by certain utilities
- The common ones are: egrep, sed, awk, and vim substitute
- Most characters in a regex have literal meaning, but some are metacharacters, as shown:
- [ ] any one character within the [ ]
- [^ ] any one character not within the [ ]
- . any one character, except newline
- ^ begin of line anchor
- $ end of line anchor
- {n} n of the previous character
- {n,m} n to m of the previous character
- ? 0 or 1 of the previous character
- + 1 or more of the previous character
- * 0 or more of the previous character
- | alternate (or)
- ( ) group
- \ take literal meaning of the next character
|
