Sunteți pe pagina 1din 221

Unix

PLMi

Geometric Limited | www.geometricglobal.com

Unix History

Evolved at AT&T Bell Labs in late sixties. Ken Thomson, Rudd Canaday, Doug McIilroy, Joe Ossanna and Dennis Ritchie - writers of Unix. Originally written as OS for PDP-7 and later for PDP-11. Liberal liscensing: Various versions. System V in 1983 - Unification of all variants.

Features of UNIX

Multi-User & Multiprocessor System

Time Sharing Operating System


Written in C (HLL) Consistent File Format - the Byte Stream

Hierarchical File System

Features of UNIX

Simple User Interface

Supports Languages - FORTRAN, BASIC, PASCAL, Ada, COBOL, LISP,


PROLOG, C, C++ etc

Architecture of UNIX System

Shell user Other Application and system software Unix commands Kernel Hardware shell

shell

user

Database Packages

Features of Unix

Services Provided :

Process Management

Creation, Termination, Suspension and

Communication

Features of Unix

Services Provided :

File Management

Aspects related to files like Creation & Deletion, File security etc.

File System

Hierarchical Structure Consistent Treatment of Data: Lack of file format The Treatment of Peripheral Devices as Files Protection of File Data

File System

Root (/)

bin
<Files>

dev
<Files>

etc
<Files>

mnt

tmp
<Files>

unix
bin

usr
Kumar Sharma

<files>

prog <files>

safe <files>

File System

/ bin commonly used UNIX Command like /usr/bin who, ls, cat, wc etc. are stored here /dev - contains device files of all hardware devices /etc - contains those utilities mostly used by system administrator e.g. passwd, chmod, chown

File System

/tmp - used by some UNIX utilities especially vi and by temporary files /usr - contains all the files created by user, including directory /unix - kernel

user to store login

Release V - does not contain / bin - contains / home instead of /usr

File System

File Types

Regular File Directory File Device File

File System

Consists of Sequence of Logical Blocks (Each block: Multiple of 512 k bytes) An Installation may have several physical disk units each containing one or more file system Data from another File System can be Accessed by Mounting that File System

File System Layout

Boot Block Super Block Inode List Data Blocks

File System

Boot Block (numbered 0)

Beginning of File System (1st Block)


Contains Bootstrap Code Every File System has (possibly empty) Boot Block

File System

Super Block (numbered 1)

Size
# of Files that can be Stored Free Space Information

Inode Information

File System

Inode blocks (numbered 2 onwards)

Information like access permissions, data/time, phy.


Addr. etc. about each inode

File System

Data Block : File Data & Administrative Data: stores actual contents of file Inode Table Fields : File Owner Identifier - Owner & Group id File Type - Regular, Directory, Character / Block / Pipe

File System

File Access Permissions rwx rwx rwx user group others Date & Time - Last File/Inode Modified/Accessed No. of Links Disk Addresses File Size

File Permissions

Associated with every file, useful for security. 3 categories of users Owner (u) Group (g) Others (o) Access permissions Read (r) Write (w) Execute (e)

Basic Unix Commands

Logging in & out

Logon name & password required


Successful logon places user in home directory

Changing password with passwd command

file /etc/passwd stores username, password & other login information

Basic Unix commands

On line help with man command

includes brief description, options & examples.

$man <command>

Commands related to Files & directories

Checking current directory - pwd command $ pwd /usr/Kumar

Commands related to Files & directories

Changing directories - cd commands $ pwd /usr/Kumar $ cd Prog

$ pwd
/usr/Kumar/Prog

Commands related to Files & directories

Moving one level up $ cd ..

Switching to home directory $ cd

Commands related to Files & directories

3. Checking login directory - logname command $ cd /usr/Sharma $ logname Kumar

Commands related to Files & directories

Listing directory contents - ls command e.g. $ ls a.out chap1 chap2 test test.c $

Commands related to Files & directories

Options available in ls
Option Description

-x Displays multi columnar output (prior to Release 4) -F Marks executables with *and directories with / -r Sorts files in reverse order (ASCII collating sequence by default) -l The long listing showing seven attributes of a file -d Forces listing of a directory -a Shows all files including ., .. And those beginning with a dot

Commands related to Files & directories

Options available in ls
Option Description

-t -R -u -i -s

Sorts files by modification time Recursive listing of all files in sub-directories Sorts files by access time (when used with the -t option) Shows i-node number of a file Displays number of blocks used by a file

Commands related to Files & directories

E.g. $ ls - l total 8 -rw-rw-rw-rw-rw-rw-rw-rw-rw-

1 Kumar group 44 May 9 09:08 dept.h 1 Kumar group 212 May 9 09:08 dept.q 1 Kumar group 154 May 9 09:08 emp.h

Commands related to Files & directories

Consider first entry Field1 --> mode - rwx rwx rwx --> user permissions --> group permissions --> others permissions

Commands related to Files & directories

File type

d c b

--> regular file --> directory file --> character - read --> block read

Commands related to Files & directories

Field2 Field3 Field4 Field5 Field6 Field7

- indicates number of links - File owner id - Group id - File size in bytes - Date/time last altered - Filename

Commands related to Files & directories

Displaying and Creating Files - cat command $ cat dept.lst 01|accounts|6213 02|admin|5423 : 06|training|1006

Commands related to Files & directories

Scrolling of large files can be freezed by pressing <ctrl s> and resumed by <ctrl q>

$ cat chap1 chap2 - display contents of chap2 immediately after displaying chap1

Commands related to Files & directories

$ cat > test Type here the text contents After typing the entire text Press <ctrl d> to save and exit <ctrl d> $ cat chap1 chap2 - display contents of chap2 immediately after displaying chap1

Commands related to Files & directories

Copies a file or group of files e.g. $ cp chap1 temp/chap2 Option - i (interactive) $cp - i chap1 chap2 cp: overwrite chap2 ?y Option -r (recursive) to copy entire directory $cp - r temp newtemp

Commands related to Files & directories

To delete files $ rm chap1 chap2 chap3 $ rm * Are you sure? y Option - i (interactive delete) $ rm - i chap1 chap2 chap1 : ? y

Commands related to Files & directories

Option - r (recursive delete) $ rm -r * (Warning: Pl. do not use this option)

Commands related to Files & directories

To rename: mv command Used to rename file or group of files; as well as directories $ mv chap1 man1

The destination file, if existing, gets overwritten

Commands related to Files & directories

e.g.

$ mv temp doc

e.g.

$ mv chap1 chap2 chap3 man1 will move chap1, chap2 & chap3 to man1 directory

Commands related to Files & directories

To alter file permissions Format : chmod <category> <operation> <permission> <filenames> Category Operations Attribute u-user +assigns permission r-read g-group -remove permission w-write o-others =assigns absolute permission x-execute a-all

Commands related to Files & directories

e.g.

e.g.

$ chmod u+x note $ ls - l note -rwx r-- r -1 $ chmod ugo+x note $ ls - l note -rwxr-xr-x note

note

Commands related to Files & directories

e.g.

$ chmod u-x, go+r $ chmod u+x note $ chmod o+wx note $ chmod ugo=r note

note note1

note2

Commands related to Files & directories

Octal notation

describes both category and permission similar to =operator (absolute assignment) read permission - assigned value is 4 write permission - assigned value is 2 Execute permission - assigned value is 1
e.g. $ chmod 666 note will assign read & write permission to all

Commands related to Files & directories

e.g.

$ chmod 777 note will assign all permissions to all $ chmod 753 ? note

e.g.

Commands related to Files & directories

To create a directory - mkdir command Creates a directory e.g. $ mkdir doc e.g. $ mkdir doc doc/example doc/data e.g. $ mkdir doc/example doc - will give error - Order important

Commands related to Files & directories

rmdir command Used to remove directory Only empty dir can be deleted More than one dir can be deleted in single command Command should be executed from at least one level above in the hierarchy

Commands related to Files & directories

e.g.

$ rmdir doc $ rmdir doc/example doc $ rmdir doc doc/example - will give error

Some other Unix commands :wc

Counts lines, words and character depending on option

takes one or more filename as arguments


e.g. $ wc infile 3 20 103 infile

Some other Unix commands :wc

$ wc infile test 3 20 10 100 13 120 $ wc - l infile 3 infile $ wc - wl infile 20 3

103 180 283

infile test total

infile

Some other Unix commands : more

Allows user to view a file one screen at a time Syntax : more <options <+linenumber> <+/pattern> <filename>

more continued

: appears after displaying screen <spacebar> or <enter> - next screen q - quit n - switch to next file on command p - switch to previous file f - displays filename and line no. /pattern - to locate line with pattern ! - to run command

more continued

Command k:n k:p

:f
=

Description Skips to kth next file that has been specified in the command line Skips to kth previous file that has been specified in the command line Displays current filename and line number Displays current line number

more continued

Command spacebar ks kf ,

/pattern

Description Displays next screen Skips k lines forward Skips k screens forward Reverts to beginning of file if no search made, or to point where last search was made Searches for pattern forward

more continued

Command n v . ! cmd q

Description Repeats last search forward Starts up the vi editor Repeats previous command Executes the Unix command cmd Exits from more

Some other Unix commands:cmp

To compare two files & echo first mismatch option -l gives detailed list of byte no. and different byte in octal e.g. $ cmp note1 note2 note1 note2 differ : char3, line1 $ cmp - l note1 note2 3 143 145 6 170 167 7 171 170

Some other Unix commands: comm


To compare two sorted files displays 3 - columnar output first column - lines unique to first file second column - lines unique to second file third column - line common to both files option -1, -2, -3 can be used to display specific column only

Some other Unix commands:diff

Display file differences Display which lines in one file have to be changed to make both files identical

Some other Unix commands:od


Lists the octal value (ASCII) of any file content used to display non-printing character options b to display 16 byte of data on each line c to display character above octal value e.g. $ od - bc odfile

Some other Unix commands:od


Lists the octal value (ASCII) of any file content used to display non-printing character options b to display 16 byte of data on each line c to display character above octal value e.g. $ od - bc odfile

Some other Unix commands: banner


Creates poster by blowing up its arguments maximum ten characters in a line e.g. $ banner UNIX $ banner UNIX IS HERE

Some other Unix commands:cal


Displays calendar of a particular month/ year any calendar from year 1 to 9999 e.g. $ cal 1991 $ cal 01 1991

Some other Unix commands:date

Displays system date number of options available option m - month h - month name d - day of month y - last two digit of the year H - hour M - minute S - second

Some other Unix commands:date

option

( cont.. )

T a r
e.g.

- time in hh:mm:ss - day of week (Sun to Sat) - time in AM/PM

$ date Fri Dec 7 15:00:21 EST 1990 $ date +%m 12 $ date +%h%m Dec 12

Some other Unix commands:who


Displays current users of the system displays logname, terminal no., login date and time $ who $ who am i logname of the user

Some other Unix commands

Printing with lp command

Spooling

Job No
$ lp <file>

Some other Unix commands

tr command

translation of characters

squeeze spaces ( -s )
$ tr s < file1 $ tr [a-z] [A-Z] < file1

Some other Unix commands

find command locate files find <path list> <selection criteria> <action> The following command will locate all the .profile files in the system $ find / -name .profile -print

Some other Unix commands

nl command This command elaborates schemes for numbering lines. $ nl file1.txt This command starts numbering lines from 40 increment value is 7 and width of numbers is 2. ( max 6 allowed ) $nl -w2 -v40 -i7 file1.txt

Terminal Related Commands


tty - displays device name of the terminal stty -a - displays current settings stty can also be used to change the setting To use <ctrl c> as interrupt key instead of <Del> key $stty intr \^c

Introduction to Bourne Shell

Shell

- The agency that sits between user and UNIX System

- much more than command processor

Bourne Shell- named after its founder Steve Bourne

- widely used

C Shell - product from Univ. of California, Berkeley

- advanced user interface and enhanced features

Korn Shell

- by David Korn of Bell Lab.

Introduction to Bourne Shell

Executables in /bin directory

sh indicates

- Bourne Shell

csh if present indicates - C Shell


ksh if present indicated - Korn Shell

Working of shell

Continous sleep-waking-waiting cycle Performs following activities:

Issues a $ prompt & waits for user to enter a command After user enters command, shell scans & processes the command. The command is passed on to the Kernel for execution & the shell waits for its conclusion. The $ prompt appears so that the user can enter next command.

Bourne shell metacharacters

* : To match any number of characters ? : To match with a single character [] : Character class; Matching with any single character specified within [] ! : To reverse matching criteria of character class.

Bourne shell metacharacters

\ : To remove special meaning attached to metacharacters ; : To give more than one command at the same prompt

Shell redirections

Every Unix command has access to: Standard input Standard output Standard error

Shell can redirect I/p, o/p or error to any physical file using meta characters <, > & 2>

Shell redirections
Examples: $ ls > temp $ wc < file1.txt > result $ cat nonexistantfile 2> err

Building block primitives

Pipe - allows stream of data to be passed between reader & writer process The data that the first processor writes into the pipe is input to second process $ who | wc-l $ ls | wc -l $ ls | wc -l > fcount

Building Block Primitives

tee - standard output can be saved in a file as - can be placed anywhere in pipeline e.g. $ who | tee user.lst kumar tty01 May 18 09 : 32 sharma tty02 May 18 11 : 18 tiwary tty03 May 18 13 : 21 e.g. $ who | tee user.lst | wc-l 3

well as displayed

Command Substitution

Command enclosed in backquotes (`), the shell executes the command first, and the enclosed command text is replaced by the output of the command e.g.$echo The date today is `date` The date today is Fri 27 00:12:55 EST 1990 $echo The date today is; date - issues command echo and date sequentially

Command Substitution

e.g.

$echo There are `ls | wc - l` files in current directory $cal `date +%m 19%y`

echo Command

Displays argument e.g. $echo GOOD Morning GOOD Morning $echo \* * $echo * *

echo Command

escape sequence to manipulate cursor position \t : tab \f : formfeed \n : newline \c : cursor immediately after argument e.g. $echo Enter Your Name : \c Enter Your Name : $_

echo Command

accepts ASCII Octal values e.g. $echo \007 < beep hear:

Shell Variables

User variables can be defined no type declaration or initialization Format : variable=value Value is of string type No space preceding and following =operator

Shell Variables

Evaluated by prefixing variable name with $ multi word assignment must be quoted e.g.1 $x = 37 $echo $ x 37

Shell Variables

e.g.2 e.g.3

$msg = You have mail have not found $msg = You have mail $echo $msg You have mail

Shell Variables

$msg = You\ have\ mail $echo $msg You have mail $x = aa $y = bb $echo $x$y aabb

Shell Variables

$echo Amount = $1000 $echo Amount = \$1000 - both will display : Amount =$1000 $echo Amount = $1000 Amount=000 - here shell evaluated variable $1 as double quoted

Shell Variables
$p = `pwd` $echo $p $echo ${fname} emp.sh $echo ${fname}x emp.shx

Shell script: an example


$ cat script1.sh date echo calender for the current month is cal date+%m 20%y

Shell script: an example


$ cat script2.sh echo enter your name. read uname echo hi $uname

Shell script: an example


$script2.sh enter your name: xyz Hi xyz

Simple Filters

Central tools of Unix toolkit useful text manipulators both horizontal & vertical filters available

head Command

Displays the beginning of one or more file s e.g. $head -3 emp.lst will display first 3 lines if number not specified (default) 10 lines $head -2 emp.lst dept.lst

tail command

displays the end of a file default - display last 10 lines $tail -3 emp.lst displays last 3 lines $tail +50 emp.lst displays till end of file starting from line no. 50 OR tail -n +2 emp.lst

tail command
$tail -15c emp.lst display last 15 characters

$tail -1b emp.lst block is used as unit of measurement

Simple filters

Cut :

Allows slicing file vertically identifies both column and fields


$cut -c6-22, 24-32 emp.lst

e.g.

cuts the file emp.lst vertically from column 6 to 22 and from column 24 to 32 $cut -c-5, 10-15, 25 - emp.lst implies 1-5 implies 25 - end of file

Simple filters

Cut :

-d option to specify delimiter can be used with -f option which specifies field no. default delimiter is tab cut selects entire line if not properly delimited
$cut -d | -f 2,3 emp.lst $cut -d | -f 1,5- empl.lst | tee temp

Simple filters

Paste :

paste more than one file laterally $paste file1 file2 <file1> <file2> delimiter can be inserted between two files $paste -d | file1 file2

Sorting a file

Reorders a line in a ASCII collating sequence sorting can be done on a field delimiter can be specified, -t option $sort -t | +1 emp.lst

+1indicates sort after skipping first field i.e. on 2nd field Or $sort t | k 2 emp.lst sorting in reverse order -r option $sort -r +1 emp.lst

Sorting a file

output filename can be specified as argument $sort -0 sortedf emp.lst -c (check) option to check if sort successful. No message indicates sort successful char position can be specified $sort -t | +4.6 - 4.9 emp.lst

will sort 7th & 8th column of field 5 Or $sort t | k4.6, 4.9 emp.lst

sorting on multiple fields $sort -t | +2-3 +1 file1

Sorting a file
$sort -m file1 file2 file3

merges sorted files

Numeric Sort $sort -n file1

Uniq Command

removes duplicate records (-d option) $uniq dept.lst $sort dept.lst | unique - uniqlst -u option if only unique records to be listed -d if only one copy of repeated records to be selected

Editors

Three Types Line Editor - ed, ex Screen Editor - vi Stream Editor - sed

vi Editor

Three Modules :

i, I, a, A, r, R, o, O, s, S

Command Mode
: <Enter> sh <Esc>

Input command Mode

<ctrl d> or exit

ex Mode

Input Mode Commands


Command i I a A o O Function Insert text to left of cursor Inserts text at beginning of line Appends text to right of cursor Appends text at the end of line Opens line below Opens line above

Input Mode Commands

Command rch R s

Function Replaces single character under cursor with character ch (no<Esc>) Replaces text from cursor to right Replaces single character under cursor with any number of characters Replaces entire line

vi Editor - Save & Quit

From input mode ZZ From input mode to command mode press <Esc> and from command mode To Save :w To Quit :q To save & quit :wq or :x

vi Editor - Command Mode


Deletion x - single char 4x - deletes 4 characters from cursor to right dd - line delete 4dd - deletes 4 lines

Cursor Movement Commands


Command h or backspace j k I or spacebar ^ Function Moves cursor left Moves cursor down Moves cursor up Moves cursor right Moves cursor to beginning of first word of line (no repeat factor)

Cursor Movement Commands

Command O or l $ b e w

Function Moves cursor to beginning of line (no repeat factor with O) Moves cursor to end of line Moves cursor backwards to beginning of word Moves cursor forward to end of word Moves cursor forward to beginning of word

Paging Functions
Command <Control-f> <Control-b> <Control-d> <Control-u> <Control-l> Function Full page forward Full page backward Half page forward Half page backward Redraw page screen (no repeat factor)

Search and Repeat Commands


Commands /pat ?pat n Functions Searches forward for pat Searches backward for pattern pat Repeats search in the same direction along which the previous search was made (no repeat factor) Repeats search in a direction opposite to that which the previous search was made (no repeat factor)

Search and Repeat Commands

Commands
fch

Functions

Moves cursor forward to first occurrence of character ch in the current line tch Moves cursor forward onto but before first occurrence of character ch in the current line ; Repeats search in the same direction along which the previous search was made with f or t , Repeats search in a direction opposite to that along which the previous search was made with f or t

vi Editor - Other Features

Joining line J - to join current line with next line 4J - to join 4 lines from current line Marking Text m <single char label> - to mark line <single char label> - to locate marked line

vi Editor - Other Features


Repeating last Instruction - . (dot key) Undo last Instruction - u Reverse all changes made to current line - U

A Few Specimen Operator-Command Combinations


Command Function

d$ or D 5dd d/endif
d30G df. c0 c$ or C

Deletes from cursor to end of line Delete five lines Deletes from cursor up to the first occurrence of the string endif in the forward direction Deletes from cursor up to line number 30 Deletes from cursor to first occurrence of a dot Changes from cursor to beginning of line Changes from cursor to end of line

A Few Specimen Operator-Command Combinations


Command 3cw or c3w cc cG y$ or Y 3yw or y3w 5yy y?case !Gsort Function Changes three words Changes current line Changes from cursor to end of file Yanks from cursor to end of line Yanks three words from cursor position Yanks five lines Yanks from cursor up to the first occurrence of the string case in the reverse Sorts all lines from current position to end of file

ex mode commands
Command :r note :w :w note1 :w! note1 Function Reads file note into present cursor location saves buffer and remains in editing mode Creates file note1 and writes buffer contents to it Overwrites contents of file note1 with contents of buffer Appends buffer contents to file note1

:w>> note1

ex mode commands
Command Function :x Saves buffer and quits editing mode :wq Saves buffer and quits editing mode :q Quits editing mode without saving :q! As above but after abandoning changes :e note1 Stops editing current file, and edits file note1 e! note1 As above, but after abandoning changes made to current file

ex mode commands
Command :e! :e # :n :rew Function Loads last saved edition of current file Returns to editing the most recently edited file Edits next file (on the vi command line) Rewinds file list to start editing from first file specified in the command line Cursor at start of 100th line

100

Customising editing environment


set map abbreviate

Set Options Used by ex/vi

Option autoindent autowrite

Abbreviation ai aw

Significance Next line starts at the previous indented level switching files with :n and escaping to the shell with :sh

Writes current file automatically when-ever

ignorecase ic magic

Ignores case while searching for patterns Treats the characters of the regular expression set as special while searching for patterns

Set Options Used by ex/vi

Option number

Abbreviation nu sm

Significance Displays line numbers on screen Shows momentarily the mach to a ) & } Displays a message when vi is in input mode

showmatch showmode

tabstop
wrapscan

ts
ws

Sets tab for display (default : 8 spaces)


Continues pattern search by moving to the other end of a file

Map & umap commands


Map : used to assign undefined key or reassign defined ones. :map v ayy to display current settings :map to cancel a mapping :umap v

Abbreviate

For mapping of characters in input mode ab se standard error

Options to vi

To salvage a file after crash $ vi -r <file> to begin from specified line $ vi +<line> <file> to begin with specified pattern $ vi +/<pattern> <file>

grep Command - Advanced Filter

Searches for a given pattern in the files specified grep <options> <pattern> <filename(s)> Options -c displays count of occurrences -l displays list of filenames only -n displays line number along with lines -v displays all but the line matching pattern

grep Command - Advanced Filter

Options ( cont..) --i ignores case -h omits filename -f fname reads expression/pattern from file (egrep & fgrep option) -x displays lines match in entirety (fgrep option)

grep Command

e.g.

$grep sales emp.lst - lists records with sales as dept $grep sales emp.lst $str = sales $grep -c $str emp?.lst will count no. of directors in all emp?.lst display will be for each file separate

Regular Expressions with grep

Used to specify pattern specified in single quotes

Expression ch* [pqr] [^pqr] - 0 or more occurrence of char ch - matches single char p, q or r - matches single char which is not p, q or r [c1-c2] - match single char within ASCII range specified

Regular Expressions with grep

Expression .(dot) -matches with any single character ^pattern - matches with pattern at beginning of pattern$ - matches with pattern at end of line ch \ {m\} - matches m occurrence of char ch ch \ {m,n\} - matches min. m & max. n occurrence of char ch ch \ {m,\} - matches min. m occurrence

line

Regular expression with grep


Consider following R.E. (i) *cC+ho *wu+dh*ury (ii) *7-9+ $ char [7-9] followed by 3 more chars rep by at end of line (iii) director.**7-9+$ (iv) ^*\t+*$ (v) ^ 03 .\{10\} (vi) [0-9] [0-9] [0-9] [0-9]

egrep Command

egrep

- Extending grep

extends grep pattern matching capabilities


matches 1 or more char ch

Extending R.E. features

ch+

ch?
exp1/exp2 (x1/x2)x3 x1/x2

Matches or 1 char ch
matches exp1 or exp2 expression matches expression x1x3 or x2x3 matches expression x1 or x2

egrep Command

e.g.

$ egrep (sen/das)gupta emp.lst


$ egrep *aA+gg?*ar++wal emp.lst $ egrep -f pat.lst emp.lst

fgrep Command

fgrep

e.g.

- multiple string searching only simple (pattern) string or group of strings can be searched faster than grep and egrep $fgrep -f pat.lst emp.lst $cat pat.lst sales admin

Some limitations of grep family


No options to identify & work with fields Output formatting, computations etc. Not possible Extremely difficult to specify patterns/ R.E always.

Advanced Filter - awk

Report writer : very powerful text manipulation utility Named after Aho Weinberger Kernigham As powerful as any programming language It can access, transform and format individual fields in a record Generally awk programs are slow.

Advanced Filter - awk

Syntax awk <options> line specifier ,action- <files>

Advanced filter awk


Simple awk filtering awk /director/ ,print- emp.lst
line specifier (selects line for processing) Action (as has an effect of printing all selected lines)

$0 variable stores entire line

Advanced Filter - awk

Line specifier and action option are optional, either of them needs to be specified line specifier not specified indicates all lines to be selected {action} omitted indicates print (default) fields are identified by special variable $1, $2, .; default delimiter is contiguous string of spaces

Advanced Filter - awk

Explicit delimiter can be specified using -F option e.g. awk -F | /sales/,print $3, $4- emp.lst Regular expression of egrep can be used to specify pattern Line numbers can be selected using NR built-in variable $awk -F | NR ==3, NR ==6 ,print NR, $0- emp.lst

Advanced Filter - awk

Output can be formatted using printf similar to printf in c but not a function e.g. $awk -F | /*aA+gg?*ar++wal/, >printf %3d%-20s%-12s%d\n, NR, $2,$3,$6- >emp.lst Record no./ Line number will be printed , (comma) to be used as delimiter in printf

Advanced Filter - awk

Logical and Relational operators

Logical operator &&, ||

$awk - F | $3 ==director || $3 == chairman, >printf %-20s,$2- emp.lst Relational Operators : <, <=, ==, !=, >=, >, ~, !~ e.g. $awk -F | $6>7500 ,printf %20s, $2- emp.lst e.g. $awk -F | $3 == director || $6>7500 , >print $0- emp.lst

Advanced Filter - awk

== tries to find perfect match but string may have trailing spaces to overcome this ~ and !~ (match & negate of match) can be used with R.E. $awk -F | $2~/director/||$~/g.m/, >printf $0- emp.lst $3 ~/^g.m/

Advanced Filter - awk

Number Processing

Computation on numbers can be done +, -, *, /, % operator available no type declaration for variables variables initialized to zero

$awk -F | $3 == director || $6>7500 , >kount = kount+1 >printf %3d%-20s\n, kount,$2- emp.lst

awk Command

-f option awk program can be written in a separate file and used in awk e.g. $awk -F | -f emp.awk emp.lst single quoted contents are written in this file without quotes

awk Command

BEGIN and END Section Format: (i) BEGIN {action} (ii) END {action} awk<options> BEGIN ,actionline specifier {action} END ,action- <files>

awk Command

e.g.

$cat emp.awk BEGIN { printf \n\t Employee details \n\n } $6>7500{ # increment sr. no. and sum salary kount++; tot+=$6 printf %d%-20s%d\n, kount, $2, $6 } END { printf \n The Avg. Sal. Is %6d\n, tot/kount } $_ &awk -F | -f emp.awk emp.lst

awk Command

Positional parameters and shell variable

requires entire awk command should be in the shell script to differentiate positional parameter and field identifier, the positional parameter should be single quoted in awk program e.g. $3 > $1

nawk

Enhancements to awk available as new command - nawk. Its possible to use getline statement to accept I/p from the user or from a file. Its possible to use the built-in variables ARGC & ARGV to check arguments passed. Its possible to define own functions. Its possible to use the system function to run unix commands or any other executable within nawk. The ?: operator as in C is available within nawk. getline x < Emp.lst getline x < /dev/tty

Built-in Variable

NR - Cumulative no. of records read FS - The i/p field separator OFS - The output field separator NF - No. of fields in current record FILENAME- The current i/p file ARGC - No. of arguments in command ARGV - List of arguments

line

Built in functions

Functions int(x) sqrt(x) index(s1,s2) length()

- Returns integer value of x - Returns square root of x - Returns the position of string s2 in - Returns length of the argument

s1

Built in functions

Functions substr(s1,s2,s3) - Returns portion of string of length s3,starting position s2 in string s1 split(s,a) - Split string s into array a optionally returns no. of fields

Arrays

Can handle single dimension array index can be anything even string no declaration required, initialized to zero automatically e.g. totl[1] +=$6; tot[2]+=da; tot[3]+=HRA; temp[$3]

Interface with Shell

Redirection > , piping | possible restriction - filenames, commands that follows these symbol must be in double quotes Programming constructs : (I) if then if (conditions is true) <statements> else <statements>

Interface with Shell

Programming constructs : (II) For statement for (i=1; i <= 9; i++) {

Interface with Shell

(iii)

(iv)

(v)

for loop for arrays for (k in array) <commands> While loop while (condition is true) { } break and continue

Shell Variables

System Variables

Set during
- login

- booting

script executed during login time - profile

profile alters operating environment of a user


to display list of system variable $set

Shell Variables

list of variable

PATH - list of dir. In order of precedence that have to be scanned for hunting command
Home - dir. Named after Login name

IFS - string of char which are used as word separator in the command line
MAIL - mail directory

PS1 - Two prompts - primary and secondary


PS2 - (used by sed, awk) TERM - Terminal type

Shell Variables

DATA in the same file - The here document < redirection symbol, redirects the standard input to come from a file << here document symbol signifies data is here rather than in a Separate file

Shell Variables

e.g. $grep director <<mark - delimiter to indicate end of document >1001 | Sanjay | g.m. | 5 >5001 | Rahul | director | 7 : : >mark

Read Statement

Takes input from standard i/p can be used to read one or more variables e.g.echo \n Enter the Pattern : \c read patname echo \n Enter file name : \c read fname grep $patname $fname

Command Line Arguments


Shell procedures arguments in command Arguments are specified along with the name of shell procedure Arguments are assigned to special variables (positional parameters) $1 - First parameters $2 - Second parameters : :

Command Line Arguments

Special parameters also available $0 - Name of executed command $* - Complete set of positional parameters $# - No. of arguments $$ - PID of current shell $! - PID of last background job $? - Exit status of last command $@ - Same as $* except when enclosed in double quotes

Command Line Arguments


E.g.1 if [$# -nc 3] then : : fi $kill - 9 $$ grep $1 $2 grep director emp.lst >/dev/null;echo$? Exit Status indicates successful 1 indicates failure 2 indicates unreadable file

2 3 4

Conditional Execution

Logical Operators && and ||

&& operator delimits two commands, the second command is executed only if first succeeds
|| operator delimits two commands, the second command is executed only if first fails $grep `director` emp.lst && echo patern found $grep `manager` emp.lst || echo pattern not found

e.g.

Conditional Execution

Script Termination - exit statement

premature termination of program


control returned to the calling program
grep $1 $2 || exit 2 echo pattern found Return value default zero

e.g.

if Statement

Format : (i) if <condition is true> then <execute commands> else (execute commands> fi (ii) if <condition is true> then <execute commands> fi

if Statement

(iii)

if <condition is true> then <execute commands> elif <condition is true> then <execute commands> <> <> else <execute commands> fi

if Statement

Condition can be specified either using test or [cond] Relational Operator used
eq

Equal to nc Not equal to gt Greater than gc Greater than or equal to lt Less than lc Less than or equal to

if Statement

File related tests with test

-a <file> -f <file> -r <file> -w <file> -x <file> -d <file> -s <file>

Trace if <file> exists if exists & regular file if exists & readable if exists & writable if exists & executable if exists & is dir file if exists & size > 0

Test - String Comparison

String tests used by test


n str
z str S1 = S2

True, if str not a null string


True, if str is a null string True, if S1 = S2

S1! = S2
str

True, if S1 S2
True, if str is assigned and not null

Logical Operators
a O !

.AND. .OR.

Not

Test - String Comparison

e.g.

If test $x -eq $y

if [$x -eq $y]


e.g. then If [! -f fname]

echo file does not exists


fi

The Case Statement

Format : case <expression> in <pattern 1> ) <execute commands> ;; <pattern 2> ) <execute commands> ;; <> <> esac

The Case Statement

e.g.

echo \n Enter Option : \c read choice case $choice in 1) ls -l ;; 2) ps -f ;; 3) date ;; 4) who ;; 5) exit ;;

esac

The Case Statement


E.g. Case `date | cat -d -fi ` in Mon > <commands> ;; Tue > <commands> ;; : : esac

The Case Statement


E.g. Case $ans in [yY] [eE] [sS] ;; [nN] [oO] exit ;; *) invalid option ;;

While Statement

Format : while <condition is true> do <execute statements> done

While Statement
e.g. while [$x -gt 3] do ps -f sleep 5 done while true do ps -f sleep 5 done

e.g.

While Statement

E.g. Script to edit, compile & execute a program while true do cc $1 case $? In o) echo Compilation Successful echo Executing a.out a.out ; exit ;; *) echo Compilation Error echo Press <Enter> to edit read pause vi $1 ;; esac done

Break & Continue Statement

Continue

: Suspends execution of all statements

following it, and switches control to top of loop for next iteration

Break : causes control to break out of the loop

Break & Continue Statement


e.g. Validation of designation code fname = desig.lst while echo Designation code : \c do read desig case $desig in [0-9+ *0_9+ if grep ^$desig $fname >/dev/null then echo code exists continue fi ;; *) echo invalid code continue ;; esac done

Until Statement

Complement of while stm loop body is executed repeatedly as long as the condition remains false e.g. until false do ps -f sleep 5 done

for Statement

Format : for variable in list do <execute commands> done

for Statement

e.g.

e.g.

e.g.

for x in 1 2 3 do echo The value of x is $x done for var in $PATH $HOME $MAIL do echo $var done for file in *.c do cc $file done

While Statement

E.g. Script to edit, compile & execute a program while true do cc $1 case $? In o) echo Compilation Successful echo Executing a.out a.out ; exit ;; *) echo Compilation Error echo Press <Enter> to edit read pause vi $1 ;; esac done

Break & Continue Statement

Continue

: Suspends execution of all statements

following it, and switches control to top of loop for next iteration

Break : causes control to break out of the loop

Break & Continue Statement


e.g. Validation of designation code fname = desig.lst while echo Designation code : \c do read desig case $desig in [0-9+ *0_9+ if grep ^$desig $fname >/dev/null then echo code exists continue fi ;; *) echo invalid code continue ;; esac done

Until Statement

Complement of while stm loop body is executed repeatedly as long as the condition remains false e.g. until false do ps -f sleep 5 done

for Statement

Format : for variable in list do <execute commands> done

for Statement

e.g.

e.g.

e.g.

for x in 1 2 3 do echo The value of x is $x done for var in $PATH $HOME $MAIL do echo $var done for file in *.c do cc $file done

Process

Instance of program in execution Many processes can run at the same time Every process identified by Process Identifier PID is allocated by kernel

Parent & child processes

On logging to system, a process is set up due to execution of shell Shell is the parent process for every other process setup due to execution of commands Every process has a parent process Parent process waits for death of child process before resuming execution.

Process Status

ps command to display characteristics of a process options -f - full form -u - details of only users processes -a - all processes details -l - detailed listing -e - system processes

Background Process

To run a process in background, use & operator $sort -0 emp.lst emp.lst &

nohup (no hangup) - permits execution of process even if user has logged off $nohup sort emp.lst & sends output to nohup.out

Process Related Commands

wait waits for child process to complete $wait 138 - waits for background job with pid 138 kill $kill 1005 (default signal 15) kills job with pid 1005 $kill -9 1005 sure killing of job $kill 0 - kills all background process

Process Related Commands

nice - job execution with low priority

prefix the command with nice $nice wc -l xxx $nice -15 wc -l xxx
reduce priority by 15 units

Communication & scheduling


News command invoked by user to read any message sent by administrator Messages stored in /usr/news options: -n,-s,-a

Communication & scheduling

motd

message od the day- a text file in /etc/motd readable by all users

Communication & scheduling

write command: allows 2-way communication between users who are logged on. Message occurs on screen anytime it is sent. The mesg command with parameter n prevents disturbances.

Communication & scheduling

mail command: single channel communication. User need not be logged on : message gets saved in users mailbox. Many internal commands available within mail.

Communication & scheduling

wall command used by system administrator to send messages to all logged on users.

Communication & scheduling

Some commands for scheduling:


sleep : To introduce delay in script

at : To schedule job execution at later date/time


batch: To run the job when system load permits. cron : To schedule jobs for periodic execution.

While Statement

E.g. Script to edit, compile & execute a program while true do cc $1 case $? In o) echo Compilation Successful echo Executing a.out a.out ; exit ;; *) echo Compilation Error echo Press <Enter> to edit read pause vi $1 ;; esac done

Break & Continue Statement

Continue

: Suspends execution of all statements

following it, and switches control to top of loop for next iteration

Break : causes control to break out of the loop

Break & Continue Statement


e.g. Validation of designation code fname = desig.lst while echo Designation code : \c do read desig case $desig in [0-9+ *0_9+ if grep ^$desig $fname >/dev/null then echo code exists continue fi ;; *) echo invalid code continue ;; esac done

Until Statement

Complement of while stm loop body is executed repeatedly as long as the condition remains false e.g. until false do ps -f sleep 5 done

for Statement

Format : for variable in list do <execute commands> done

for Statement

e.g.

e.g.

e.g.

for x in 1 2 3 do echo The value of x is $x done for var in $PATH $HOME $MAIL do echo $var done for file in *.c do cc $file done

Program Development

CC Make SCCS

CC Command

The UNIX system compiler for C. cc compiles the file if no errors are found an executable version of the file is created. The default name of the executable file is a.out Invokes a series of programs the preprocessor, the compiler and the linker. Subprograms can be compiled together.

Make Utility

Make utility and makefiles help automate building of an application. Modifying any one or more of the source files requires recompilation of that part of the program and relinking. Automate this process by specifying the interdependencies between files that make up the application along with the commands needed to recompile and relink each piece.

Make Utility

During program development a number of subroutines may be developed separately and linked at the end. However keeping a track of the updated versions is difficult. Further, files that are changed may not be recompiled.

Make Utility
These instructions are placed in make file. The command make is used to execute instructions defined in make file. $ make myprog

Make Utility

A make file consists of a series of entries of the following form: [.....dependencies....] [.....commands........]

[...targetfile..]: [One Tab Space]

Make Utility
main.o: main.c cc -c main.c Makefile and all files referenced by MAKE should be in the same directory. Sees the dependencies upon program.Then checks to see if program is current

SCCS-Source code control system

SCCS is a group of programs that help in controlling the various versions of a program. Multiple copies are not stored. Only incremental changes to the program are stored.

SCCS-Source code control system


The following commands are used with SCCS Admin : used to create initial versions of the file Get : used to retreive the SCCS files Comb : used to combine two or more versions and remove redundant versions Remdel : used to remove a version Prs : removes information about a SCCS file