z/OS

z/OS UNIX Shell Scripting for the Not So Dumb

Playing the shell game
Session 2917
August 16, 2006

Garth Godfrey z/OS UNIX System Services Development, IBM Poughkeepsie, NY ggodfrey@us.ibm.com http://www.ibm.com/servers/eserver/zseries/zos/unix

2006 IBM Corporation

z/OS

Trademarks
The following are trademarks of the International Business Machines Corporation in the United States and/or other countries. IBM* IBM eServer IBM e(logo)server* IBM logo* Language Environment* On demand business logo OS/390* Parallel Sysplex* RACF* System z9 z/Architecture z/OS* zSeries* * Registered trademarks of IBM Corporation The following are trademarks or registered trademarks of other companies. Java and all Java-related trademarks and logos are trademarks of Sun Microsystems, Inc., in the United States and other countries. Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both. Microsoft, Windows and Windows NT are registered trademarks of Microsoft Corporation. UNIX is a registered trademark of The Open Group in the United States and other countries. SET and Secure Electronic Transaction are trademarks owned by SET Secure Electronic Transaction LLC. * All other products may be trademarks or registered trademarks of their respective companies. Notes:
Performance is in Internal Throughput Rate (ITR) ratio based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput that any user will experience will vary depending upon considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve throughput improvements equivalent to the performance ratios stated here. IBM hardware products are manufactured from new parts, or new and serviceable used parts. Regardless, our warranty terms apply. All customer examples cited or described in this presentation are presented as illustrations of the manner in which some customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics will vary depending on individual customer configurations and conditions. This publication was produced in the United States. IBM may not offer the products, services or features discussed in this document in other countries, and the information may be subject to change without notice. Consult your local IBM business contact for information on the product or services available in your area. All statements regarding IBM's future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only. Information about non-IBM products is obtained from the manufacturers of those products or their published announcements. IBM has not tested those products and cannot confirm the performance, compatibility, or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products. Prices subject to change without notice. Contact your IBM representative or Business Partner for the most current pricing in your geography.

2006 IBM Corporation

z/OS

Session Objectives
Shell basics Variables Shell Language Statements Shell functions Shell script examples Invoking shell scripts

2006 IBM Corporation

z/OS

Shells on z/OS UNIX

z/OS Shell /bin/sh UNIX-standard command interpreter Some Korn shell features C Shell /bin/tcsh

Perl Open Source Software Perl 5.8.7 now available in feature IBM Ported Tools for z/OS: Perl for z/OS bash Open Source Software

New

(not IBM supported)

www.ibm.com/servers/eserver/zseries/zos/unix/bpxa1toy.html

2006 IBM Corporation

z/OS

Shell Script Basics

Contain shell commands Language facilities are built into the shell
if / then / else / fi for / while loops # comments

Invoked like shell commands

+x permission simple command names looked up in PATH run in child shell process unless "sourced" with "dot": . script

2006 IBM Corporation

z/OS

Shell script example

#!/bin/sh # ct script - Count files in specified directory if [[ -d $1 ]] then COUNT=$(ls $1 | wc -w) echo Directory $1 contains $COUNT files exit 0 else echo Error: $1 is not a directory exit 1 fi

2006 IBM Corporation

z/OS

Shell Script Basics

Script that starts with "#!" uses specified command interpretter
#!/bin/sh #!/bin/tcsh #!/usr/bin/Perl Default C shell script Perl script

2006 IBM Corporation

z/OS

Shell Variables
VAR=value export VAR sets a shell variable marks VAR for export to commands / scripts

export VAR=value VAR and value can contain any characters, any case $VAR expands the variable

values are strings of characters (unless typeset) shell sets PWD, OLDPWD shell uses HOME, SHELL, PATH, PS1, ...
2006 IBM Corporation

z/OS

typeset
typeset marks variables as integer makes arithmetic faster may specify base (e.g. octal) uppercase / lowercase left-justified / right-justified others typeset -i TOTAL

2006 IBM Corporation

z/OS

Arithmetic
integer only let "i = $i + 1"

quotes avoid shell special character handling

i=$(( i + 1 ))

arithmetic substitution may be used outside of an assignment supports arithmetic, relational, and logical operators
+-*/%!><=&|...

2006 IBM Corporation

z/OS

if [[ ... ]]
if tests the exit status of any command
if [[ -d $1 ]] then COUNT=$(ls $1 | wc -w) echo Directory $1 contains $COUNT files exit 0 else echo Error: $1 is not a directory exit 1 fi

[[ ... ]] tests expression for true (0) or false (1) -d True if directory -f True if regular file -L True if symlink -Ep True if file has program control extended attribute -Aa True if file has extended access ACL entry [[ string == pattern ]] True if match e.g. [[ $FILE == *.c ]] matches files ending in .c [[ exp1 -eq exp2 ]] True if equal (arithmetic)

2006 IBM Corporation

z/OS

Positional parameters
Arguments passed to the shell script are expanded with "special shell variables"
if [[ -d $1 ]] then COUNT=$(ls $1 | wc -w) echo Directory $1 contains $COUNT files exit 0 else echo Error: $1 is not a directory exit 1 fi

$0 $1 $2 $3 ... ${10} ${11} ... $* "$@"

name of script positional parameters

all positional parameters all positional parameters quoted separately

2006 IBM Corporation

z/OS

Command Substitution
Output from the command is written to stdout
if [[ -d $1 ]] then COUNT=$(ls $1 | wc -w) echo Directory $1 contains $COUNT files exit 0 else echo Error: $1 is not a directory exit 1 fi

Output from "inner" command is substituted into "outer" command

Assignment captures stdout as a string value

2006 IBM Corporation

z/OS

Conditional execution
command1 && command2 if command1 is successful, run command2

&& ||

LOG=/usr/spool/cron/log cp $LOG /bkup/cronlog.$(date +%m%d%Y.%T) &&

>$LOG

copies cron log to new file: cronlog.08012004.17:22:16 if successful, empty the cron log
[[ $FILE == *.c ]] && COUNT=$((COUNT + 1))

command1 || command2 if command1 fails, run command2

cp badfile badfile2 || { print -u2 "Copy failed"; exit 1; }

2006 IBM Corporation

z/OS

for loop
for arg in "$@" do # Process command-line arg echo "Processing $arg" . . . done DIRLIST=$(find dir -type d) for DIR in $DIRLIST do # Process each directory echo "Processing $DIR" ... done

2006 IBM Corporation

z/OS

Prompting for user input

ANS="" while [[ ! $ANS ]] do read ANS?"Continue? Enter yes or no: " case "$ANS" in "") echo "Nothing entered" ;; y*|Y*) echo "Continuing..." ;; n*|N*) echo "Exiting at user request" exit 1 ;; *) echo "Invalid response" ANS="" ;; esac done

while, read, case

What's wrong with the case patterns in this example?

2006 IBM Corporation

z/OS

Prompting for user input

(improved)

typeset -l ANS="" # lower-case assigned values while [[ ! $ANS ]] do read ANS?"Continue? Enter yes or no: " case "$ANS" in "") echo "Nothing entered" ;; y|yes) echo "Continuing..." ;; n|no) echo "Exiting at user request" exit 1 ;; *) echo "Invalid response" ANS="" ;; esac done

2006 IBM Corporation

z/OS

Parsing command options

# Process arguments (-i, -l, -r, -w and -?) while getopts il:rw\? opt do case $opt in i) iflag=1;; l) lflag=1 logfile="$OPTARG";; r) rflag=1;; w) wflag=1;; \?) Usage ;; *) PrintError "$0: unknown option $OPTARG" Usage ;; esac done # shift positional parameters shift OPTIND-1 # Check user entered proper number of arguments. if [ $# -ne 2 ]; then Usage fi

getopts

2006 IBM Corporation

z/OS

Functions
Like C functions (subroutines) Shell parses the function definition once before it's called Stores definition in shell memory for execution Local positional parms
# This function displays the usage message. Usage()

{
print -u2 "Usage: skulker [-irw] [-l logfile] <directory> <days old>" exit 2

2006 IBM Corporation

z/OS

Functions
# This function appends a text string to the user-specified log file # (for use with -l) # The first parameter is the text to be written to the file. # Note that if $logfile exists, previous data is not overwritten. PrintToLog()

{
if [ -n "$lflag" ]; then print "$1" >> $logfile fi

}
# This function handles printing of error messages. # The parameter sent to the function is the error message to be # displayed. This error is also written to the log file # if -l was specified. PrintError()

{
print -u2 "$1" PrintToLog "$1"

2006 IBM Corporation

z/OS

Finding files
# Find files older than specified day. if [ -n "$rflag" ]; then files=$( find $1 -atime +$2 ! -name "$1" -print ) else files=$( find $1 -type f -atime +$2 -level 0 -print ) fi rc=$? # If an error occured, then print error message and quit. if [ $rc -ne 0 ]; then PrintError "find command returned non-zero exit status: $rc" exit $rc fi # If no files are found, then quit. if [ -z "$files" ]; then PrintToLog "No files found older than specified date. exit 1 fi

Nothing removed."

Avoid using the -exec operator of find for best performance

2006 IBM Corporation

z/OS

Extracting fields from output

awk

for file in $files do user=$(ls -l $file | awk ' $0!="" { print $3 } ') messagebody="$file will be deleted because it has not \ been accessed in $2 days or more. \ Please take appropriate action \ if you would like to keep this file." echo $messagebody | mailx -s "WARNING: file $file to be deleted" $user done PrintToLog "Sent warnings on $(date) for files: \n$files"

2006 IBM Corporation

z/OS

Checking for active files

fuser

else # Remove the files (including read-only files.) if [ -x /bin/fuser ]; then PrintToLog "Files removed on $(date):" for file in $files do checks for if [ -z $( /bin/fuser $file 2>/dev/null ) ]; then zero-length output rm -fr $file rc=$? if [ $rc -ne 0 ]; then PrintError "Error occurred during remove of $file. Retcode=$rc." else PrintToLog "$file" fi else PrintError "$file is in use, not removed." fi done

2006 IBM Corporation

z/OS

Debugging shell scripts

set -v set -x outputs every line that the shell reads outputs lines after shell expansion, before execution

==> ct /u/godfrey + [[ -d /u/godfrey ]] + + 1<TMP> /tmp/shCcebGHBeA + ls /u/godfrey + wc -w COUNT= 104 + echo Dir /u/godfrey contains 104 files Dir /u/godfrey contains 104 files

set -x if [[ -d $1 ]] then COUNT=$(ls $1 | wc -w) echo Dir $1 contains $COUNT files exit 0 fi

For line number xtrace:

PS4='[$LINENO]+ '
2006 IBM Corporation

z/OS

Invoking shell scripts from a shell

Make sure it has read and executable permission
chmod +rx script

Put it in a PATH directory

or invoke with a full pathname

Foreground execution

==> skulker -w /tmp/ 30

wait

==>

while getopts il:rw\? opt ... ... ... exit

2006 IBM Corporation

z/OS

Invoking shell scripts from a shell

Background job &

==> skulker -w /tmp/ 30 & [1] 66102 ==> pid1=$! ==> issue other commands ==> wait $pid1 ==> echo $?

while getopts il:rw\? opt ... ... ... exit

$! %1 jobs wait

Process id of last background job Job id 1 Command displays jobs Command waits for jobs to end

2006 IBM Corporation

z/OS

Invoking a shell script from JCL

JCL EXEC statement
//jobname JOB ... //stepname EXEC PGM=BPXBATCH,PARM='SH script' //STDOUT DD PATH='stdout UNIX file path' //STDERR DD PATH='stderr UNIX file path'

BPXBATCH

Must allocate or redirect stdin/out/err, or they default to /dev/null (bit bucket) SH invokes a login shell with profiles (/etc/profile, $HOME/.profile) In z/OS 1.8, supports MVS datasets for STDOUT, STDERR
Available via APAR OA11699 for 1.5 - 1.7

As a TSO command BPXBATCH SH script >/u/userid/batchout OSHELL command - execute a single shell command oshell script displays stdout & stderr on screen

2006 IBM Corporation

z/OS

The Shell Environment

Environment variables PATH, CLASSPATH, STEPLIB, ... Set in profiles, scripts, STDENV, ... Must be exported, inherited across fork/spawn MVS dataset allocations Lost across fork/exec BPXBATCH BPXBATSL spawns shell Local spawn maintains allocations Shell options, function definitions set -o pipecurrent improves performance set -x xtrace Not inherited Set in $ENV run for shell script

2006 IBM Corporation

z/OS

Improving performance
_BPX_SHAREAS=YES _BPX_SPAWN_SCRIPT=YES set -o pipecurrent Use shell functions For shell scripts invoked by many users Consider defining as shell functions export FPATH=shared_function_dir autoloaded on first use only for /bin/sh scripts!

2006 IBM Corporation

z/OS

Session Summary & References

Experiment (on a test system) to learn more 2992 - z/OS UNIX Shell Scripting Hands-On Lab Ask questions http://www.ibm.com/servers/eserver/zseries/zos/unix http://www.marist.edu/htbin/wlvindex?mvs-oe z/OS V1R7.0 UNIX System Services User's Guide (SA22-7801)
Lots of introductory material for shell

Thur 9:30 317

z/OS V1R7.0 UNIX System Services Command Reference (SA22-7802)

The gory details of all shell commands Shell language statements documented under "sh"

2006 IBM Corporation