Shell Programming Notes

The Bourne Shell

The Bourne shell is one of a number of Unix shells (C shell, Bash shell etc.). Like the others, it is both a command language and a programming language. (A shell is frequently called a command interpreter.) As a command language it provides a user interface to Unix/Linux. It executes commands entered by the user or from a file. Files containing commands allow users build their own commands thus tailoring the system to their own needs. Such files are called: shell scripts, shell programs, or command files. These commands have access to the command line parameters and have the same status as other Unix commands. As a programming language, each shell provides I/O, variables, conditionals, loops, and switches. The syntax is aimed at ease of use at a terminal so that strings for example do not have to be

Shell Notes. Joe Carthy

quoted. Each shell (Bourne, C, Bash etc) has its own syntax. Shell programs are typically used to develop: "User-friendly'' commands System administration utilities Application utilities A common criticism of Unix or the shell in particular, is that it is unfriendly. Shell programs can be written to prompt the user for parameters, check them and so on, thus cushioning the user from the powerful shell, and making the system as user-friendly as required. Experienced users often get fed up of such prompting and usually prefer using the normal shell commands. System administrators can build shell programs to add new users, to shutdown the system and so on. Some examples are given in these notes. Finally useful applications can be developed with the shell e.g. a phone lookup system is briefly described. The development of applications often requires
Shell Notes. Joe Carthy

some knowledge of other Unix utilities such as grep, tr, ed and awk. A knowledge of shell programming is essential for the serious Unix user, and will more than reward the few hours invested in exploring the shell, in time saved developing new commands and utilities. This document describes the Bourne shell but similar features are available in the other Unix shells, with notational differences.

Shell Programs
Shell commands may be stored in a file. There are a number of ways to execute the commands in such a file. One is to use the `sh' command with the file as a command line parameter e.g. prog.sh is a file containing 3 Unix commands Usage:

$ sh prog.sh
Code in prog.sh: #!/bin/sh pwd ls -l

Shell Notes. Joe Carthy

date Commands may be separated using newlines as above or using semicolons: pwd ; ls -l ; date Any file name may be used, the extension `.sh' has no significance and is not a Unix convention. The second and perhaps more natural way is to use the `chmod' command to make the file "executable''. Then the file name can be entered directly, just like any Unix command. $ chmod +x prog.sh Make file executable - only done once

$ prog.sh

Comments Shell

in

Bourne

The shell ignores text following the # character up to the end of the line, so it used for commenting shell scripts.

Bourne Shell Variables String variables are the only type of variable provided by the Bourne shell. Variable names
4

Shell Notes. Joe Carthy

begin with a letter and are composed of letters, digits and underscores. They do not have to be declared. They are given values by assignment or by using `read' to read a user value. The following are some examples. These can be entered interactively at your terminal or in a shell program.

mybin=/usr/user/joe/jbin name=`Bill Bloggs'

Note that there are no spaces around the '=' symbol. Strings containing blanks or tabs must be quoted.

Input read FILENAME

Read is a built-in command (i.e. part of the shell program) which reads from the standard input into the variable(s) specified. If more than one variable is specified, it breaks the input into words and assigns each word to the variables specified. If there are not enough variables,
Shell Notes. Joe Carthy

the last variable will contain the rest of the line e.g.

read word2 rest

with input

word1

bill jack john tom mary

will give word1 the value 'bill', word2 the value 'jack' and rest the value 'john tom mary'.

Output
The echo command is used to display messages and the values of variables. The program echo simply prints its command line parameters on the standard output. To access the value of a variable you must precede the variable name with a '$' symbol e.g.

echo "Hello Second Science" echo Directory name $mybin is

echo $name echo Filename is:

6

Shell Notes. Joe Carthy

$FILENAME
echo has one option '-n' which suppresses printing a newline after the output. This is useful for prompting for input e.g.

echo -n "Enter a file name: ''

Variables may be used as a convenient short-hand for long strings. So to move a file to /usr/user/joe/jbin you can enter:

$ $mybin

cp

prog

The shell simply substitutes the value of the variable for each occurrence of the variable name. So you can use the variable value as part of a string:

$ $mybin/prog

ls

-l

Shell Notes. Joe Carthy

Variables are local to your shell unless you 'mark' them for export. Program contains: prog1.sh

export mybin mybin=/usr/user/joe/jbin current=/usr/user/joe prog2.sh # Call prog2.sh echo "Back in prog1.sh'' echo "Value of mybin'' $mybin echo "Value of current'' $current

Program contains

prog2.sh

echo "Prog2.sh: echo "Value of mybin'' $mybin echo "Value of current'' $current current=/usr/user mybin=/bin echo
Shell Notes. Joe Carthy

"Value
8

of mybin'' $mybin
Usage:

$ prog1.sh
Output:

Prog2.sh: Value of mybin /usr/user/joe/jbin Value current Value mybin /bin Back prog1.sh Value of mybin /usr/user/joe/jbin Value of current /usr/user/joe
Exported variables can be accessed in shell programs (commands) executed by this shell. Note: A shell program cannot change the environment in which it was called. A subcommand can change the value of exported variables locally, but any changes made have no effect in the calling shell. Using export is similar in effect to 'call by value' in parameter passing in C. So prog2.sh can change the variable
Shell Notes. Joe Carthy

of of in

"mybin'' locally, but back in prog1.sh, the value of "mybin'' is unchanged. Also, the variable "current'' has no value in prog2.sh, since it was not marked for export in prog1.sh. The above are all examples of user-defined variables.

Bourne Shell System Variables

The shell also provides a number of system variables i.e. variables which are given values by the system. When you create a shell program, the command line parameters are made available in the variables $0, $1, $2, $3,...... $0 is the program file name, $1, $2 etc. refer to parameters 1, 2 and so on. The variable $# contains the number of parameters on the command line, which can be zero (unlike C) if no arguments are specified i.e. the command name is not counted. It is usually used to check
Shell Notes. Joe Carthy

10

that the right number of parameters have been provided. Another useful system variable is $$. It is the process number of the current shell, which will be unique among all processes. Typically, it used to create temporary file names e.g.

date /tmp/record$$ ps -a /tmp/record$$

> >>

The file will have a name like /tmp/record102373. A special parameter $* is used to stand for all the command line parameters except $0. So echo $* is equivalent to echo $1, $2, $3 ....... e.g.

Assume that the file prog.sh contains the line

echo $*
Then executing prog.sh:

$ prog.sh arg1 arg2 word3 abc

outputs
11

Shell Notes. Joe Carthy

arg1 word3 abc

arg2

The shift command is used to promote the command line parameters so that $2 becomes $1, $3 becomes $2 and so on. Parameter $1 is lost. Example:

$ prog word1 word2 word3

Code:

echo $1 shift echo $1 shift echo $1

Output:

word1 word2 word3

Shell Notes. Joe Carthy

12

Other variables having a special meaning to the shell are: HOME Home directory -- default argument for cd command. Set up in the password file /etc/passwd. PATH Search path for commands. A list of directories that is searched by shell for command binary files. By default it is the current directory, /bin and /usr/bin. The directory names are separated by a : e.g. PATH=.:/bin:/usr/bin: $HOME/bin This PATH would cause the current, /bin, /usr/bin and $HOME/bin directories to be searched, in that order. PS1 Primary shell prompt, by default '$' (for Bourne shell). PS2 Secondary shell prompt, by default '> ' (for Bourne shell). It is displayed when a command such as a loop is being entered over a number of lines.
13

Shell Notes. Joe Carthy

Any of the above variables can be changed to suit user requirements. These changes would normally be made in your login command-file '.profile', which is executed once by the shell when you login. The values of these variables may be displayed with echo:

$ $PATH $ $HOME

echo echo

Shell Notes. Joe Carthy

14

Control Flow
The shell provides the usual control flow mechanisms associated with structured programming: for, while, until, case, if-then, and ifthen-else structures.

The for Structure

This structure allows a set of commands to be executed once for each word in a list supplied. General Format:

for word1 word3....... do

var in word2

commands done
Example:

for screen.c compare.c do "File : " $i

i in menu.c echo

lpr $i done

Shell Notes. Joe Carthy

15

This example simply displays the name of each file in the list and sends the file to the printer. The commands between 'do' and 'done' are executed each time around the loop. ('do' and 'done' delimit the body of the loop and are also used in the while and until structures.) It is very common to use the for structure to process the command line parameters. For example, to build a create command which creates empty files, the file names being specified on the command line:

Shell Notes. Joe Carthy

16

Usage:

$ create file1 file2 # Create is a shell program # files

Code of create:

to

create

for File in $* do >$File echo $File created done

The notation >filename causes the shell to create an empty file or truncate an existing one. Because the above loop is so common, an abbreviated form is available where the "in $*'' is omitted:

for File do >$File echo $File created done

Shell Notes. Joe Carthy

17

loops over the command line arguments as in the previous example.

Shell Notes. Joe Carthy

18

A useful example to show some of the power of a very simple shell program is to develop a program called, say, tel which searches a file called phone.dat in the home directory, which has the form:

Dept.

Joe Bloggs of Physics 6767 Fred Smith 5678

The tel program can search for phone numbers by name or any text in the file.

Usage:

$ tel joe fred

Output:

Dept.

Joe Bloggs Of Physics 6767 Fred Smith 5678

$

or

$ tel 6767
Output:

Shell Notes. Joe Carthy

19

Dept.

Joe Bloggs Of Physics 6767

$

Code:

# tel program for i in $* do grep i $i $HOME/phone.dat done

It simply loops over the command line arguments and uses the 'grep' program to search the file phone.dat in the user's home directory. The i option tells grep to ignore case i.e. treat uppercase and lowercase as the same. Grep is just one of the many Unix utilities that allow you build very powerful shell programs, quickly. In fact, to get the most out of shell programming, you need to know what utilities are available and what they can do. Important ones are grep, sort, tr, unique, sed, ed, and awk. Remember that the shell also has the ability to generate file names using special characters called shell metacharacters or
Shell Notes. Joe Carthy

20

wildcard characters. (These allow you build regular expressions for pattern matching.) The commonest are * and ?. The * matches any sequence of characters, and on its own the shell will expand it into the list of filenames, in alphabetical order, that are in the current directory. Try:

echo * echo *.c

It is important to realise that the shell generates this list of file names before running the echo command. As far as echo is concerned, the user may well have typed the list of files on the command line. The ? character matches any single character. Ranges of characters may be specified by enclosing characters in [] e.g. [A-Z] specifies any character in the range A to Z (uppercase). This facility to generate lists of filenames is often used with the for structure. For example to print and then backup all your C programs:
21

Shell Notes. Joe Carthy

for FILE in *.c do lpr $FILE cp $FILE $HOME/backup done

Shell Notes. Joe Carthy

22

Another example of developing a useful utility using the for structure is a shutdown procedure to shutdown the operating system, but give users a warning beforehand, so that they may save their files and logout: Usage:

$ shutdown
Code:

for i in 5 4 3 2 1 do echo "Going down in $i minutes'' | /etc/wall sleep 60 done /etc/wall <<! System Going Down Now Bye ! kill -1 1
The program goes around the for loop 5 times, sleeping for 60 seconds during each pass. 'sleep' is a program that simply causes your program to be suspended for the number of seconds specified i.e. it does not waste CPU time. The program '/etc/wall'
23

Shell Notes. Joe Carthy

writes a message to all terminals, from its standard input. Two methods of providing the standard input to '/etc/wall' are shown. In the first, the output of echo is simply piped into the '/etc/wall' program. In the second case, a "here document'' is used. This is the text enclosed by "<<!......text.......!''. This provides a mechanism for providing input to any program from within a shell program. It is most often used to provide input for the ed editor, and the mail program. You can use any character instead of ! to start and terminate the text. The terminating character, !, in this case must be at the start of a new line. The "here document'' is to be preferred for multi-line messages. Finally, the 'kill' command is used to terminate a process, in this case, we send signal number 1 to process 1, which is the ancestor of all processes in the system (This may vary for different versions of Unix/Linux). By terminating process 1 we shutdown the
24

Shell Notes. Joe Carthy

system. Only the superuser can do this.

The while Structure

Unlike the for structure, while, until and if are conditional structures, similar to the corresponding structures in an ordinary programming language. In shell programming, the exit status of a command (program) determines the action of these conditional structures. An exit status of 0 (returned by the exit system call from a C program) is interpreted as true, non-zero exit status is interpreted as false. There are two programs called 'true' and 'false' which exit with 0 and non-zero respectively. General format:

while command(s) do command(s) done

The command after "while'' is executed and if it returns true (exit status 0) then the command(s) forming the
Shell Notes. Joe Carthy

25

loop body are executed. An endless loop may be constructed, e.g. the asp program (anti-sleep program) to prevent the your PC from going to sleep:

#asp program-you could get bitten for using it ! while true do echo "Sleeping....... sleep 240 # Sleep for 4 minutes done

The

program sends a message to your terminal every 4 minutes and may be halted by an interrupt (Ctrl/C). Until is the opposite to while, looping as long as the condition returns false.

until false # Endless loop do echo "Sleeping..........'' sleep 240 # Sleep for 4 minutes
Shell Notes. Joe Carthy

26

done

Another example of a useful utility is the watchfor program which waits for a user to login and writes a message to his terminal and also echos a message to your terminal: Usage:

$ tom tom logged on

Code:

watchfor has Watchfor who |

# program until grep $1 do

120 # every 2 mins done # User has logged on write $1 <<! Hello $1, Can you contact me ! echo $1 "has logged on''
The pipeline " who | grep

sleep check

Shell Notes. Joe Carthy

27

$1 '' causes the output of the who command to become the input for grep, which searches it for the specified user e.g. tom in this example. If grep succeeds in finding tom in the input, then it returns an exit status of 0, which terminates the until loop. The 'write' command is used to send a message to a user's terminal. A "here document'' is used to provide the input for 'write'. In this example, we sleep for 120 seconds if the user has not logged on before going around the loop again.

Shell Notes. Joe Carthy

28

The test command

A standard command (sometimes built-in for efficiency) called test is available for testing various conditions to do with strings and files. It is one of the most common ways to control "while'', "until'' and "if'' structures. General form:

test expression test exp1 -o exp2 test exp1 -a exp2

It returns 0 if expression evaluates to true, non-zero otherwise. The "-o'' is used for to combine two expressions with logical OR, "-a'' for logical AND. Examples:

test -s File True if File exists and is non-empty test -f File True if File exists and is not a directory test -r FileTrue if File is readable (also -w
29

Shell Notes. Joe Carthy

for writeable) test True directory -d if File File is if

test -z Str True Str has zero length

test -n Str True if Str has non-zero length test str1 = str2 True if strings are equal test n1 -eq True if n1, algebraically equal n2 n2

(Also -ne, -gt, -e, -le, -lt for not equals, etc. )
You can invert the above conditions by using ! e.g.

test ! -d true if file is not a directory.

Shell Notes. Joe Carthy

30

Sample Usage:

while test /tmp/lockfile do sleep 5 done

-r

This code waits for a file called '/tmp/lockfile' to disappear. This could be used to implement a crude form of semaphore system, and is used in the program myprint below to provide a mechanism for preventing users from trying to access the printer simultaneously :

Usage: $ myprint file.c file2.c Code: # myprint while test -r /tmp/lockfile # Wait until printer free do sleep 5 done >/tmp/lockfile # Lock printer for i in $* do pr $i > /dev/lp # pr file on print device done rm
31

Shell Notes. Joe Carthy

/tmp/lockfile printer

Free

This is a good example of the shell being used to create a useful "system'' program. Note it is not foolproof, two users could test for the presence of the lockfile at the same time.

Shell Notes. Joe Carthy

32

The Structure

if-then-fi

General format -- there are 3 possibilities: if command then commands fi conditional-

if command

conditional-

then commands else commands fi if command then commands elif conditionalcommand then commands else commands fi Example 1 conditional-

if test $# = 0 then
Shell Notes. Joe Carthy

33

"Usage: username'' fi

echo watchfor exit

Here we check if no command line parameter is entered, displaying an error message if this is so. This check should be included at the start of the "watchfor'' program shown earlier. .

Shell Notes. Joe Carthy

34

Example 2 if test $# -ne 2 then "Invalid 2 expected'' echo arguments exit else echo " 2 parameters entered'' fi
It is often used to test conditions about files. We could use it to modify the create program developed earlier, so that it does not truncate existing files:

# safecreate program for i in $* do if test -f $i then echo $i "already exists'' else >$i done
This program loops over the command line parameters, testing if each one exists, creating a file if it does not already exist.

Shell Notes. Joe Carthy

35

We could make the program interactive, prompting the user if the file already exists, to find out if he wishes to truncate it:

Shell Notes. Joe Carthy

36

# safecreate2 program for i in $* do if test -f $i then echo "already exists'' echo "Truncate it (Y|N)'' read ANS if test $ANS = Y -o $ANS =y then >$i elif test $ANS = N -o $ANS =n then continue # Go around loop again else echo "Enter Y or N'' $i -n

Shell Notes. Joe Carthy

37

safecreate2 $i fi else >$i fi done

Here, we show the use of elif, an abbreviation of else if. You may have as many elifs as required. The continue statement, takes you to the loop test again (as in C), enabling you go around the loop again. Finally, this program illustrates a very important feature, the ability to use a shell program recursively. Here if the file already exists we ask the user for Y or N reply as to whether to truncate it. If the user does not enter one of these characters then we simply let the program call itself with the current file as its command line parameter. Since the file already exists, it will again prompt the user for a Y or N reply and proceed accordingly. This is a useful way of handling such an error condition.

Shell Notes. Joe Carthy

38

This feature, is probably most often used when processing files in directories. Some of the files may be directories themselves and they in turn may contain subdirectories which contain subdirectories and so on. The sample program lsdir is an example. It lists the files in a directory, and lists the contents of subdirectories and so on. Usage:

$ directory-name
Example:

lsdir

$ /usr/user/year3
Code:

lsdir

# program

lsdir

PATH=/bin:/usr/bin:/use r/usr/joe/bin if test $# = 0 then lsdir . # Use current directory elif test ! -d $1 # Check if $1 is a directory
Shell Notes. Joe Carthy

39

then echo $1 "Not a directory'' ls -l $1 # List the file exit else for i in $1/* # Loop over files in $1 do if test -d $i # If it is a directory then echo "Directory: '' $i ( cd $i ; lsdir . ) else ls -l $i ordinary file fi
If no arguments are supplied to lsdir, it lists the current directory by calling itself recursively with parameter ".''. The for structure, takes the first parameter, expands it to a
Shell Notes. Joe Carthy

\#

fi done

40

list of files in that directory and loops over each file in the list. For example lsdir will give i the value '.' and loop over the values './begin.c', './bin', './c' where . contains the files 'begin.c', 'bin' and 'c'. By enclosing shell commands in parentheses, the commands are executed in a subshell, so that the cd command (which is built-in) will execute in a subshell and when that subshell is finished, we will still be in the directory where we started. ( cd $i ; lsdir $i ) This is the one of the commonest uses of parentheses. Note that PATH is set explicitly in 'lsdir', thus preventing the shell executing any other copy of lsdir that might appear in some subdirectory visited. This is a wise precaution, especially for the superuser, where someone might "plant'' their own copy of 'lsdir' to do some dastardly deed !! It may also be necessary, if 'lsdir'

Shell Notes. Joe Carthy

41

is not in a public 'bin' ( i.e. /bin or /usr/bin ) and PATH has not been set in .profile. Another example of using "while'' and "if-then'' is illustrated in the sample monitor command. This command records a list of who logs on and what they are doing, every five minutes. The record is kept in the file '/tmp/monlist'.

Shell Notes. Joe Carthy

42

PATH=/bin:/usr/bin monlist=/tmp/monlist if test ! -r $monlist then >$monlist fi while : do date >> $monlist echo " '' >> $monlist who >> $monlist echo " '' >> $monlist ps -a >> $monlist echo " '' >> $monlist 300 minutes done
It first checks to see if the file '/tmp/monlist' exists, creating it if it is not present. This program uses ":'' after the "while'' which always returns true, but is built-in and so is more efficient than using the 'true' program.

sleep # 5

Shell Notes. Joe Carthy

43

The echo command is used to put blank lines in the file between the output of the various commands. The program could be executed a "daemon'', a command which is always running in the background. Finally, the system variable PATH is set to '/bin' and '/usr/bin' to reduce the amount of searching that the shell performs when looking for commands.

Shell Notes. Joe Carthy

44

The case Structure

Like the for structure, it is based on pattern matching: General Format:

case word in pattern1) command(s) ;; pattern2) command(s) ;; esac

Note the double semicolons terminating each case.

Example 1: Used to check number of command line parameters

case $# in 0) Echo "No arguments supplied'' ; exit ;; 2) Echo "Correct '' ;; *) Echo "Incorrect number of arguments'' ;; esac

Shell Notes. Joe Carthy

45

Example 2: command

An append

case $# in 1) cat >> $1 ;; 2) >> $2 < $1 ;; *) echo 'Usage: append [fromfile] to-file' ;; esac
Usage: $ append thisfile ontothatfile

cat

The append command allows you append two files when 2 arguments are supplied. With 1 argument, it appends from the standard input onto the specified file.

Example 3: Used to check for options on command line

case $1 in -o) echo "-o option entered'' ;; -c) echo "-c option entered'' ;; esac
A range of characters may be enclosed in []
Shell Notes. Joe Carthy

46

[Yy]) echo "y or Y entered'' ;; -[oc]) echo "-o or -c options entered'' ;; You can specify alternative patterns by using | e.g Y|y|yes|Yes) echo "Y or y or yes or Yes entered'' ;; Another version of the create command shows an example.

# safecreate3 program using case for i in $* do if test -f $i then echo "already exists'' echo "Truncate it (Y|N) '' read ANS case $ANS in y|Y) >$i ;; n|N) continue ;; \# Go around loop again $i -n

Shell Notes. Joe Carthy

47

*) "Enter Y or N'' $i ;; esac

echo

safecreate3

else >$i fi done

Now that all the control structures have been discussed, we can see an example using them all, enabling us to build a useful utility program tidy. This program is used to to allow you to perform a number of operations on a set of files e.g. all files in your directory. It displays a menu, displays each filename in turn, reads a user option allowing the user to process the file. Options include display file contents, list file, print it, delete it, skip to next file. As many options as required may be carried out on any file. Usage:

$ tidy f.c f2.c t.c

or

$ tidy *
48

Shell Notes. Joe Carthy

Code: # Tidy program if test $# -eq 0 then echo "No files specified" exit fi # Display menu cat <<! Exit x Next file n Display file t Delete file d List file l Print file p ! # Process command line parameters for i in $* do echo "File : '' $i # Display file name FIN=n # Process each file while test $FIN = n do echo -n "Enter command: ''

Shell Notes. Joe Carthy

49

read COM case $COM in n|N) break ;; x|X) exit ;; t|T) cat $i ;; p|P) lp $i ;; d|D) rm -i $i ;; l|L) ls -l $i ;; *) "Unknown $COM'';; esac echo echo -n "Finished this file (y/n) '' read FIN # Only n is checked done # Next command for this file done # Next file echo command

Note that it is more efficient to display the menu using cat with a "here document'' (the text to be displayed is included "here" in your file) than to use a sequence of

Shell Notes. Joe Carthy

50

echo commands, since only one program has to be loaded when using cat, whereas if we use a list of echo commands, each command will have to be loaded and executed separately. Alternatively, the menu could be stored in a file and cat used to display it, making it easy to add a "show menu'' option to the commands: m) cat menufile ;; Break allows you break out of a loop, as in C. This is a good example of using the shell to build useful utilities. Basically, you can program it to build almost any kind of similar utility that you can think of. As in any programming language, there are numerous variations and combinations of methods to solve a problem. User creativity is the only real limitation.

Command Substitution
This is a very powerful mechanism allowing the output of a command to be used inline e.g.

today='date'
Shell Notes. Joe Carthy

51

current='pwd'
Note that the quotes are grave quotes (') and not the usual single quotes ('). The variable today gets as value the string output by the 'date' command and current the output of the 'pwd' command. So

echo $today echo are in $current'' echo are in 'pwd' ''

yields:

"You "You

Mon Apr 10:20:45 GMT 1996 You /usr/user/joe You /usr/user/joe are are

4 in in

Command substitution allows you build some very useful programs. Suppose you wish to mail a group of users a particular message then if you create a file of user names called namelist:

joe
52

Shell Notes. Joe Carthy

tom year2 year3 .. ..

one method of doing it is:

mail 'cat name-list' < message

The shell executes the command "cat name-list'' and the output is produced inline just as if you entered:

year2 message

mail joe tom year3 ....<

The contents of the file message is sent to users appearing the file namelist. You could also use a "here document'' instead of using the file message:

mail name-list' <<! This text of message ....... ! is

'cat the the

If you want to loop over the files in a directory, in order of last modification, then
Shell Notes. Joe Carthy

53

you can use 'ls -t' to produce the list of files and use it inline, in a for loop:

for i in 'ls -t' do process $i done

Arithmetic The Bourne shell provides no built-in commands for arithmetic operations. However, a command called 'expr' is available which evaluates arithmetic expressions on its command line and outputs the result to the standard output.

$ expr 2 + 2 4
By executing the command inline, we can perform arithmetic on shell variables. This can be used to loop over command a specific number of times. For example, we could rewrite the shutdown command presented earlier, to use $1 as the number of minutes until shutdown time. If no command line parameters are specified then 5 is chosen as the default number of minutes until
54

Shell Notes. Joe Carthy

shutdown Usage:

10 system mins

$ shutdown2 # Shut down in 10

Shell Notes. Joe Carthy

55

Code:

#shutdown2 program if test $# = 0 then count=5 # Default is 5 minutes else count=$1 fi i=1 tleft=$count while test $i -le $count do echo "System going down in $tleft minutes'' | /etc/wall i='expr $i + 1' # increment i tleft='expr $tleft - 1' sleep 60 done echo "System going down now !!!!!'' | /etc/wall kill -1 1 Output: System going down in 10 minutes System going down in 9 minutes .... System going
Shell Notes. Joe Carthy

56

down in 1 minutes System going down now !!!!! $

Count gets the value of $1 ( a string ), i gets the value 1, and tleft the value of count for the time left. Output of the 'expr' command overwrites the values of i and tleft each time around the loop, until i reaches the value of count. The 'expr' program is not special, we could use this method with any C program to perform whatever operations are required, although you will normally find a suitable utility is available for whatever you want to do. Quoting If you want to use shell metacharacters such as *, ?, $ as ordinary characters you must quote them. There are 3 methods of quoting in the shell. 1) The \ character quotes the the next character e.g. echo The star character: \*

Shell Notes. Joe Carthy

57

echo The backslash character: \\ yields The star character: * The backslash character: \ 2) Single quotes ' ' are used to quote a group of characters: echo '***** Warning **** ???? ' displays ***** Warning **** ???? 3) Double quotes are also used but only quote file generation characters * and ? e.g. echo ''$HOME has no filenames with a * in them'' displays /usr/user/j
Shell Notes. Joe Carthy

58

oe has no filenames with a * in them Variable and parameter substitution are carried out inside double quotes, no substitutions are performed inside single quotes. Here Documents We have already seen the use of "here documents''. One very common use is to prepare edit scripts for the ed editor. Instead of using ed interactively, it can be used by placing ed commands in a file and redirecting the input to come from the file or from a here document: ed file.txt < ed-script where ed-script contains editor commands. This is useful where the same commands have to be applied to a number of files:

ed file.* < script

or

ed file.* <<! 1,\$s/Unix/UNIX/ g a These lines are appended to

59

Shell Notes. Joe Carthy

each file specified . w q !

In the above example, the word Unix is replaced by UNIX in all files starting 'file.'. Two lines of text are appended to each file. Note that the dollar character must be quoted so that the shell does not try to interpret it but passes it on to ed directly. The line editor ed is very powerful, and has excellent pattern matching mechanisms for searching and substitution. Often, an ed script can be used to save you writing a C program to process a text file. In ed the caret character ^ matches the start of a line and "$'' matches the end of a line, allowing you specify commands to be executed on every line starting with some string, or finishing with some string. The following script replaces the string "Mr'' on the start of a line with the string "Mister'', deletes all lines
60

Shell Notes. Joe Carthy

containing the string "Jones'' and deletes all blank lines.

ed file <<! 1,\$s/\^Mr/Mister/ g/Jones/d g/\^$/d w q !

The g command instructs ed to perform the command on all lines, the /Jones/ locates line with the string "Jones''. See the ed manual for details. Note that the caret has to be quoted since otherwise it will be interpreted as a shell metacharacter (alternative symbol for pipe) instead of being passed to ed. This is a good example of where knowledge of a Unix utility, ed in this case, can save a lot of work. It is almost always quicker to learn how to use existing utilities to do a job, than to write even a simple C program from scratch. This mechanism can be used to build a system utility to add a user to the system. To add a new user, an entry must be
61

Shell Notes. Joe Carthy

added to the password file /etc/passwd for the user specifying user name, group, number etc. Password entries have the general form: uname:pwd:uid:gid:misc:ho me:shell user: user name pwd: encrypted password, empty if not set uid: gid: user id. number group id number

misc: usually full name or address -- any text home: login directory shell: optional, defaults to Bourne shell e.g. tom:xHyzio89ws:68:10:Tom Thumb X2134:/usr/user/tom:/bin/s h A directory must be created for the user (his home directory), and the user made the owner of this directory. The protection on this directory may be set appropriately, and the user added to the appropriate group. A command called {\bf mkuser} can easily be
Shell Notes. Joe Carthy

62

programmed to accomplish the task. A simple form of 'mkuser' is given below. No checks are carried out to see if the name is already in use and so on, but these could easily be added. It also assumes that users will all have gid (group identifier) 10. Only the superuser can execute 'mkuser'.

Shell Notes. Joe Carthy

63

Usage: $ mkuser jack 92 Code: # mkuser program: expects two parameters if test $# ! = 2 then echo "Usage: mkuser user-name user-id'' exit fi # Add entry password file to

ed /etc/passwd <<! .a $1::$2::10::/usr/us er/$1:/bin/sh . w q ! mkdir /usr/user/$1 chown $1 /usr/user/$1 chgrp user /user/usr/$1 chmod go-w /usr/user/$1
This is a useful command especially since the password file entry is
Shell Notes. Joe Carthy

64

complicated and it would be easy to make a mistake if you edit it by hand. Note the "$'' is not quoted in the "here document'' since we want the shell to replace it by the value of $1.

Signal Handling
In Unix, programs are sent signals by the kernel in the event of a program error, (such as divide by zero, or memory addressing error) or when the user hits the interrupt key (CTRL C), the quit key (CTRL \), logs out (Hangs up) or the user generates a signal with the 'kill' command. When a program receives any signal, the default action is to terminate, which is usually desirable. However, there are times when, control should be returned to the program, where appropriate action can be taken for a given signal. This allows the program delete temporary files before termination, or the program may be restarted at a suitable point, or finally the signal may be ignored and the program continues running. For this reason, the shell allows the user to "trap'' signals and take
Shell Notes. Joe Carthy

65

appropriate action. (This can also be done from a C program via the signal system call.) Each signal has an associated number, there are approximately 19 different signals numbered from 1 upwards. Some of the common ones are: Signal Number Signal Name Generated by 1 2 3 9 -9 pid 10 Hangup logout Interrupt Ctrl/C Quit Ctrl \ (sure) kill kill error

Bus Memory addressing error 15 Software termination kill -15 pid

The trap command allows the user trap the signal ignore it or take action: trap 'rm /tmp/lockfile ; exit' 1 2 3 15 In the 'lpr' command we created a lock file to prevent concurrent access to the printer. However, if the lpr command is interrupted, the lock file will not be deleted, and the

Shell Notes. Joe Carthy

66

printer will be unusable. The above trap command illustrates how to prevent this situation. It can be interpreted as "if any of signals 1, 2, 3, 15 are received, then execute:

rm /etc/lockfile ; exit
thus freeing the printer and terminating lpr. This line should be inserted at the start of the lpr program. Signals can be ignored by:

trap '' 1 2 3

and reset to their default action by:

trap 1 2 3
As an exercise you could rewrite the 'asp' program, to read a user code-word, go into the sleep loop as before, and when interrupted, prompt for the code-word terminating if it matches that previously entered, otherwise returning to the sleep loop. Efficiency of Shell Programs The efficiency of shell programs can be improved in a number of ways. By setting the value of PATH so that the shell only searches the necessary
Shell Notes. Joe Carthy

67

directories, in the right order is one method. The use of built-in commands where possible such as ":'' instead of the program 'true' is another improvement. Another point is that many Unix utilities such as 'cat', 'rm', 'ls' will themselves loop over their command line arguments. Where possible, you should let the utility do this, instead of writing:

for i in $* do cat $i done

you should write:

cat $*
When you use a shell loop, then the shell must locate and load the program each time around the loop. When you write 'cat $*' the 'cat' program is only loaded once, and it performs the argument processing (after the shell has expanded $*). The use of a "here document'' instead of a series of 'echo' commands is also worthwhile. Exec and Commands

(dot)

Shell Notes. Joe Carthy

68

Normally when the shell executes user commands, it forks a subprocess for each command. When the command finishes, the subprocess dies and control returns to the shell. There are two ways to execute programs without creating new subprocesses. A shell program may overlay itself with another program by using the exec command: exec newprog #Never get to here unless cannot run newprog The program 'newprog' simply replaces the program of the current process and is executed as part of the current process. It not used very frequently. A shell program can run another shell program as part of the current shell (i.e. a new process is not created to run it) by using the ".'' command: other shell command(s) .newprog # Execution resumes here Here the calling program is not overlayed. It is similar to calling a subroutine. When the called program
69

Shell Notes. Joe Carthy

'newprog' terminates, control returns to the calling program. This allows the new program to change variables in the current shell. This is why the login command file is called '.profile', it is executed as part of the user's shell so that variables that are set in '.profile' remain set when '.profile' terminates. Debugging Shell Programs Finally, when debugging shell programs, you can invoke them "verbosely'' with sh -v prog which causes the program lines to be printed as they are being read. This is useful for finding syntax errors. You can achieve the same effect by by entering set -v in your program. You can use set -n if you want to switch execution off, to test the procedure without running it. set -x

Shell Notes. Joe Carthy

70

provides an execution trace. All flags may be turned off by: set -

Shell Notes. Joe Carthy

71