Documente Academic
Documente Profesional
Documente Cultură
Which is
used to create a file and to modify an existing file and to append text to an existing file.
It has three modes.
Command mode:-In this mode user can enter commands and all the keys pressed by the
user are interpreted to editor commands. For example, if you hit a h the cursor is moved
one position to the left. In command mode the keys that are hit are not displayed on the
screen. This command mode is also called as ESC mode
Insert mode:-This mode permits insertion of new text, editing of existing text or
replacement of existing text. Each of these operations can be performed by only after
changing over from the command mode to insert mode using appropriate commands. The
insertion mode is also known as input-text mode. To move from command mode to input
mode, we have to press i. To move from insert mode to command mode we have to
press ESC key.
The ex command mode: -This mode permits us to give commands at the command line.
The bottom line of the vi editor is called the command line. Vi uses the command line to
display messages and commands.
Example:Let us now create a file ,add text to it,save this file on the disk and then quit out of vi. To
invoke vi type vi and the name of the file you want to create as shown below.
$vi filename
when we type the above command,vi clears the screen and displays a window in which
you can enter and edit text.The _(underscore) on the top line shows the cursor waiting for
you to enter a command here.Every other line is marked with a ~(tilde), the symbol for an
empty line.The screen as it looks like is shown below.
_
~
~
~
~
~
~
~
~
letter[New file]
If we have successfully entered vi, by default we are in command mode and vi is waiting
for your command.How do we create text?Let us now add text to the file letter.
Press the i key to enter the insert mode of vi.(Do not press the enter key).We can now add
text to the file.(the I is not printed on the screen)
Now type text
Press the ESC Key and press :w to save the contents of a file
The :q command is used to exit from the vi editor.
The :wq command is used to save the file and quit from the vi editor.
Ex2:- $ls a
The above command is used to display the names of hidden files and also normal files
and directories in the current directory.
Ex3:- $ls p*
Pack
power
The above command is used to display the names of files and directories whose names
are starting with letter p.Here * refers to one or a group of characters.
Ex4:- $ls ?ain
Gain
Main
Pain
Rain
The above command is used to display the names of files and directories whose names
ends with ain
Ex5:- $ls ?????
This will list all 5 charcter filenames in the current directory
Ex6:- $ls [aeiou]*
The above command is used to display the names of files and directories whose names
are starting with any one of the letters given within the square brackets,and the remaining
can be anything
Ex7:- $ls [!aeiou]*
The above command is used to display the names of files and directories whose first
character is anything other than a vowel.
Ex8:- $ls [a-m]??
This will list all 3 character filenames in the current directoty whose first character is in
the range a to m whereas the second and third are any valid characters.
Ex9:- $ls l
-rw-rw-rw-rw-rw-rwdrw-rw-rw1
1
MREC MREC 12 Feb 13 23:55 sample
1
MREC MREC 30 Feb 24 12:30 sample2
MREC
MREC
21 Feb27 12:40 josh_dir
In the above example the leftmost character(In File Permission) - indicates that the file
is an ordinary file whereas a d indicates that it is a directory. Other possible file types
are given in the following figure
File TypeMeaning
r-Ordinary file
d-Directory file
c-Character special file
b-Block special file
l-Symbolic link
s-Semaphore
p-Named pipe
m-Shared memory file
The above command displays names of files and directories of current directory in long
fashion.That is, file permissions,owner name,group,links,time,size and names.
Weight
4
2
1
Example:$ls l sample
-rwxrw-r-1
MREC MREC 24
Here the permissions for the file sample from the long listing are rwxrw-r--.In the above
example the leftmost character(in the file permission) - indicates that the file is an
ordinary file.Next 9 characters indicates permissions of a file. Of the nine characters
The first three characters decide the permissions held by owner of the file.In the above
example the owner can read ,write as well as execute the file sample.
The next set of three characters specify the permissions for the members of the group to
which the file owner belongs.In above example the members of the group can read and
write the file but cannot execute that file.A indicates that the permission is denied.
While the last set decides the permissions for the users outside the group (Others).In the
above example all others can only read that file.
Permission
Weight
Read(r)
Write(w)
Execute(x)
4+2+0=6
4+0+1=5
No permission
0+0+0=0
The existing file permissions can be changed by the owner of the file or by the superuser.
The chmod command is used to change the existing file permissions.There are two
methods to change file permissions.
1.Absolute mode
2.Symbolic mode
1.Absolute mode:Ex1:-
10
PROCESS UTILITIES:1)ps command:- This command is used to display the status of the running processes at
any instant.
Ex1:- $ps
PID TTY TIME COMMAND
2269 3a
12:23 sh
2421 3a
12:23 ps
Unix assigns a unique number to every process running in memory. This number is called
process ID or simply PID.
Ex2:- $ps -a
It displays the status of the various process that are running for the other users who have
logged in. Here a standing for processes of all the users.
Ex3:- $ps -f
UID PID PPID C
STIME
TTY TIME COMMAND
Icit
288
1
1
9:32
02
0:01 -sh
Icit
513
288
8
9:51
02
0:00 ps f
The above command displays additional information of the status of the processes.
UID(User ID
PID(Process ID
PPID(Parent process ID
Ex4:- $ps t tty3d
This would list all processes which have been executed from terminal tty3d.
2)kill command:-It is used terminate a process
Ex:- $kill 2323
terminated
3)at command:-This command is capable of executing unix commands at a future date
and time.
Ex:- $at 17:00
Clear >/dev/tty3c
Echo its 5 PM! Backup your files and logout
Ctrl d
Job 2324432546.a at wed Apr 14 17:00:00 IST 2006
Once submitted this command ,the message would be displayed on our terminal at
5:00PM sharp.
4)batch command:-It execute a command on its own, when the system is free.
Ex:- $batch
Sort employee.dat | tee address.out
Ctrl d
Job 3213213.b at Fri Apr 12 17:00:00 IST 2006
11
DISK UTILITIES:1)df command(Disk Free command):- This command reports the free as well as the
used disk space for all the file system installed on your machine.
Ex1:- $df
/ (/dev/root):
12970blocks
27857 I-nodes
2)du command(Disk Usage command):- This command reports the disk space used by
specified files and directories.
Ex1:- $du
.
/backup
./dbf
./fa/backup
3)ulimit command:- It stands for user limit and contains a value which signifies the
largest file that can be created by the user in the file system.
Ex:- $ulimit
2097152
This implies that the user cannot create a file whose size is bigger than 2097152 bytes or
2048KB.If you happen to create a file which exceeds this size,its size would be curtailed
to 2048 KB and the program creating this file would be aborted.
A user can reduce this value by saying
$ ulimit 1
Here onwards no file can be created whose size is bigger than 512bytes.
4)mount command:-It is used to mount file systems such as CDROM or floppy
Syntax:-
12
COMMUNICATION COMMANDS:Unix has excellent provision for communicating with other users. The communication
may be within the network of a single main computer or between two or more such
computer networks. The users can easily exchange mail,data,programs through such
networks. We have different network commands(communication commands).
1)write command:- The write command can be used by any user to write something on
someone elses terminal.
Ex:- $write unix ps/1
Hello Student I am JOSEPH
Ctrl d
The above command displays the specified message to the user whose login name is
user2.
2)mesg command:Ex1: $mesg y
The above command is used to give permission for messages to reach his or her terminal
Ex2:- $mesg -n
The above command is used to deny write permission to your terminal.
3)finger command: finger is one command that tells you which users are connected and
which, if any can receive messages. It displays a list of all those who have logged in and
places a * next to those terminals where mesg is set to n.
$ finger -i
user1 tty3a Fri Mar 12 11:08
user2 *tty3c Fri Mar 12 11:02
4)who command:-The who command lists all the users who are currently logged
in.When used with the T option, it places a + next to the users who have allowed
message and a - sign beside others.
ex:- $ who -t
user1 +tty3a Fri Mar 12 11:08
user2 -tty3c Fri Mar 12 11:02
5)w command:-This displays details about the users in addition to what command they
are working now.
Ex1: $w
Ex2:- $w username
The above displays details about the specified user.
6)wall command:- Wall enables the superuser to write to all irrespective of whether the
users have given write permission to their terminals or not.
Ex:- $ wall
All the best
Ctrl d
7)mail command:- It is used to send mail to a particular user or more than one user.
Ex1:- $ mail user2
13
14
LINK FILES:- Unix supports two types of links for files and directories namingly
hard links and symbolic links
Hard links:Ex:- $ln file1 file2
File2 contains the same information of file1. This command is used to establish a link
between file1 and file2. If we delete file1 then we can get same information from file2.
Ex:
$ln l file1
-rw-rw-r-2
MREC MREC 34
Feb 2 12:08 file1
The number of links for file1 is 2.
Symbolic links:Ex:- ln -s file3 fil4
The above command is used to establish symbolic link between file3 and file4. If we
delete file3 then we cant get same information from file4.
$unlink filename.
15
FILTERS:
Text processing utilities:1)wc command:- It counts the number of lines,words and characters in the specified file
or files.It comes with the options l,-w and c which allow the user to obtain the number
of lines,words or characters individually or in any desired combination.
Ex 1:- $wc lc file1 file2
File1 20
571
File2 30
843
Thus, the file file1 constitutes 20 lines and 571 characters. Similarly for the file file2
Ex2:- $wc lw file1 file2
File1 20
144
File2 30
173
Thus the file1 constitutes 20 lines and 144 words. Similarly for the file file2
Ex3:- $wc wc file1 file2
File1 144
571
File2 178
843
Thus the file1 constitutes 144 words and 571 characters. Similarly for the file file2
2)sort command:- sort command can be used for sorting the contents of a file.It can
merge multiple sorted files and store the result in the specified output file. While sorting
the sort command bases its comparisions on the first character in each line in the file. If
the first character of the two lines is same then the second character in each line is
compared and so on.
Ex1:- $sort myfile
This would sort the contents of myfile and display the sorted output on the screen.
Ex2:- $sort file1 file2 file3
The above command is used to sort the contents of three files at a time and it display the
sorted output on the screen
Ex3:- $sort ofile4 file1 file2 file3
The above command sorts the three files file1,file2 & file3 and saves the result in a file
called file4.
Ex4:- $sort -u -o file4 file1 file2 file3
The above command sort the three files file1,file2 & file3 and saves the result in a file
called file4 but it stores only unique lines in the output file
Ex5:- $sort m file1 file2
If the files have already been sorted and we just want to merge them we can use the
above command.
16
17
2:Xerox:78:A
3:Peter:89:A
4:Repeter:47:C
Here the fields are separated by colon(:) .If we want display 2 and 3 field.We have to use
following command.
Ex5:- $cut -f 2,3 -d: student2
Rox:90
Xerox:78
Peter:89
Repeter:47
4)grep command:-grep is an acronym for globally search a regular expression and print
it.The command searches the specified input fully(globally) for a match with the supplied
pattern and displays it.
Ex1: $grep picture sample
This would search the word picture in the file sample and if found the lines containing it
would be displayed on the screen
Ex2:- $grep picture sample sample2
Here the word picture would be searched in the both files sample and sample2 and if
found,the lines containing it would be displayed along with name of the file where it
occurred.
Ex3:- $grep the picture taken i sample
The above command searches for the pattern enclosed within without heeding the
case(-I makes it case insensitive).
Ex4:- $grep [Rr]ebecca myfile
Here grep would search for all occurrences of Rebecca as well as rebecca in myfile
and display the lines which contain one of these words.
Ex5:- $grep b??k myfile
This command would display all four letter words whose first letter is a b and last
letter,a k. The two ? symbols represent one character each.Thus,lines containing words
like book,back,beak etc. would be listed.
Ex6:- $grep -v a* myfile
This command would,display all those lines that do not contain words starting with a .
Ex7:- $grep ^[abc] myfile
This command would display only those lines which begin with a,b,or c.
Ex8:- $grep [s-z]$ myfile
18
This command would display only those lines which end with any character between s
to z.
5)fgrep(fixed grep) command:-fgrep is used to search a group of strings.Here each
string has to be separated from other by a newline.
Ex:- $fgrep rao
>ram
>raju
This command displays those lines having either rao or ram or raju.
6)egrep(Extended grep):-It is also used to search for a group of strings.Here each string
has to be separated from other by pipe( | ).
Ex:- $egrep rao|ram|raju filename
7)head command:-This command is used to display the lines from the beginning of a
file.
Ex1:- $head filename
This command displays first 10 lines of the given file.
Ex2:- $head 15 myfile
This command displays first 15 lines of the given file.
8)tail command:- This command is used to display the lines from the end of a file.
Ex1:- $tail filename
This command displays last 10 lines of the given file.
Ex2:- $tail 5 filename
This command displays last 5 lines of the given file.
9)more command:-This command is used to see the content of the files page by page or
screen by screen fashion.This is very useful if the contains more number number of lines.
Ex1:- $more filename
This displays the contents of the given file in screen by screen fashion.
Ex2:- $more +10 15 filename
This command starts displaying the contents of myfile,15 lines at a time from 10th line
onwords.
10)pg command:-This command is also used to see the content of the files in page by
page fashion.However this is not available in recent versions.Rather more command is in
wide use and is more flexible.
11)nl command:-This is used to display the contents of the file along with line numbers.
Ex:- $cat >file1
India
Ap
Nizamabad
19
MREC
Ex:- $nl file1
1 India
2 Ap
3 Nizamabad
4 MREC
12)uniq command:-This command displays uniq lines of the given files.That is if
successive lines of a file are same then they will be removed.By default output will be on
to the screen.
Ex:-
$cat >file2
MREC
MREC
MREC
HYDERABAD
HYDERABAD
^d
Ex:
$uniq file2
MREC
HYDERABAD
Ex2:- $uniq c file2
3 MREC
2 NIZAMABAD
Here c option is used to display the number of occurrences of the each line of the input
file.
13)paste command:-This is used to join vertically.
Ex:- $cat >file1
India
Ap
Ex:-
$cat >file2
Nizambad
MREC
Ex:- $paste file1 file2
India Nizamabad
Ap
MREC
Ex2:- $paste d| file1 file2
India | Nizambad
Ap | MREC
14)join command:-This is used to join files.Unlike paste it works similar to join
operation of DBMS.
Ex:- $cat >file3
101 Joseph
`
121 Madhukar
142 Sai Shyam
20
Ex:-
153 Sathish
$cat >file4
101 CSE
178 CSIT
142 EIE
204 CSIT
Ex:-
21
22
23
BACKUP UTILITIES:1)tar command:- This command is used to create and extract archives, one file that
contains one or (usually) many other files. The name "tar" itself comes from the phrase
"tape archive", but that's just an old name. I mostly just create archives and then send
them over the wire these days.
To combine multiple files and/or directories into a single file, use the following
command:
Ex: $tar -cvf
filename.tar
inputfile(s)
Creating archives
To create an archive of all files in your current directory, and all subdirectories, use this
command:
$tar cvf my_files.tar *
his command creates a new file named my_files.tar that contains all the files and
directories from your current directory (specified by the * filename metacharacter). It
doesn't remove your existing files, it just creates an archive that also contains your files.
In this command the c option means "create" and the f option lets you specify the
filename for the archive.
Listing files in an archive
To list files in an archive use the t option, like this:
$tar tvf my_files.tar
Extracting archives
To extract all the files from an archive use the x option:
$tar xvf my_files.tar
24
i : (copy in) Reads an archive from the standard input and conditionally extracts
the files contained in it and places them into the current directory tree.
/dev/st0
3)rlogin command:- With the help of this command it is possible to login to remote
machine if we happened to have legal username and password on that machine. When we
do so the current machine becomes terminal to that remote machines. After that whatever
file we create it will be stored in that remote machine.Usually,rlogin is used for GUI
based remote login service unlike telnet service.
Ex:- $ rlogin Ipaddress or machine name -l username
3)telnet command:Syntax:telnet Ipaddress or machinename
The following output appear on the screen
Trying 127.0.0.1
Connected to localhost.localdomain (127.0.0.1).
25
26
I/O REDIRECTION OPERATORS:Output redirection operator(>):- The symbol > implies redirection of output. The
symbol > sends the output of a command to a file or a device such as a printer.
Ex:- $cat <file1 > file2
This makes cat command to take input from the file file1 and write its output to the file
file2
Ex2:- $cat file1 file2 file3 > file4
The above command creates the file4 which contains the contents of all the files
file1,file2 and file3.
Input redirection operators(<):- The symbol <implies redirection of input.The symbol
takes the input needed for a command from a file rather than from the keyboard.
Ex:- $ cat <sample
It takes the contents of file sample and it displays on the screen.
Document operator:-The symbol >> adds output from a command to the end of a file
without deleting the information already in the file
Ex:- $cat >>sample
The first duty of the student is study
Ctrl d
Piping:- The unix piping facility lets us connect commands to other commands. It is
possible to join commands using a pipe concept.
Ex1:- $ ls | wc -l
Here the output of ls command becomes the input to wc which promptly counts the
number of lines it receives as input and displays this count on the screen
Ex2: $ ls l | grep ^d
This command displays details of only the directories of the current working directory.
That is output of ls l command becomes input to grep command which displays only
those lines which starts with d
Ex3:- $ ls l | grep ^d | wc -l
This command displays number of directories in the given file.
27
SHELLS:
The following example shows how to move to other shells
$ bash
#Move to Bash shell
$ ksh
#Move to Korn shell
$ csh
#Move to C shell
1.Login Shell Verification
UNIX contains a system variable, SHELL, that identifies path to our login shell
$ echo $SHELL
/bin/ksh
2.Current Shell Verification
Our current shell may or may not be our login shell. We use the following command to
determine our current shell. However this command works only with Korn and Bash
shells; it does not work with C shell.
$ echo $0
ksh
3.Redirection
Definition: Redirection is the process by which we specify that a file is to be used in
place of one of the standard files. There are three types. They are
Input redirection
Eg: cat < filename
In the above command contents of given input file will become input to the cat
command
Output redirection
$ who >sample
In the above command the output of the who command will be redirected to sample
file
Error redirection:
$whom
Bash: command not found
PIPES
Definition: Pipe is an operator that temporarily saves the output of one command in
buffer that is being used at the same time as the input of the next command. The symbol
for pipe is (|).
28
Example:
$ who | more
In the above command the output of who is the input to more command.
Tee command
Definition: The tee command copies standard input to standard output and at the same
time copies it to one or more files
Eg1:
$ tee sample
This is a sample of the tee command
This is a sample of the tee command
Each line is repeated on the standard output as we press key enter
Each line is repeated on the standard output as we press key enter
$ cat sample
This is a sample of the tee command
Each line is repeated on the standard output as we press key enter
Eg2:
$ who am I | tee f1
Cse
tty01
sept 15 16.34
tty01
sept 15 16.34
$ cat f1
Cse
In the above command the output of who am I is input to tee command where tee
command copies standard input to standard output i.e. monitor and to one file here f1
29
COMMAND EXECUTION:
Sequenced Commands: A sequence of commands can be entered on one line. Each
command must be separated by a semicolon. There is not direct relationship between the
commands.
Example:
$ date ; who
Wed Sept 7 17: 56: 52 IST 2008
Cse
tty01
sept 15 16.34
Cse 1
tty02
sept 15 16.32
Cse 5
tty04
sept 15 16.37
In the above the two commands date and who are simply combined into one line and
executed. First it executes date command then who command respectively.
Grouped commands: In this method commands are grouped by placing them in
parentheses
Example:
$ (date ;who) > f1
In the above command the output of both date and who command is redirected to the file
f1
Hence $ cat f1 gives
Wed Sept 7 17: 56: 52 IST 2008
Cse
tty01
sept 15 16.34
Cse 1
tty02
sept 15 16.32
Cse 5
tty04
sept 15 16.37
NOTE: in this method also there is no direct relationship between the commands
30
Chained commands: This method of combining commands is to pipe them. Here there
is a direct relationship between the commands. The output of the first becomes the input
of the second .
Example:
$ who | tee f1
Cse
tty01
sept 15 16.34
Cse 1
tty02
sept 15 16.32
Cse 5
tty04
sept 15 16.37
Here the output of who command becomes input to tee command and tee command
copies the input to f1 as well as displays on the monitor.
31
QUOTES:
Metacharacters are characters that have a special interpretation. The shell use a selected
set of Metacharacters in commands.
Collectively as quotes they are
1
2
3
Backslash
Double Quotes
Single Quotes
Backslash: The backslash metacharacter (\) changes the interpretation of the character
that follows it converts literal characters into special characters and special characters
into literal characters.
Examples:
1.Using backslash to print double quotation marks
$ echo HELLO WORLD
HELLO WORLD
$ echo \HELLO WORLD\
HELLO WORLD
2.Displaying special characters as literal text
$ echo < > \ $
bash: syntax error : > unexpected
$ echo \ <
\ > \
\ \ \ \ $
< > \ $
Double Quotes: double quotes remove the special interpretation of most metacharacters.
Examples:
$ x=hello
$ echo < > $x y ? &
< > hello y ? &
In the above command we use echo command to display several metacharacters .
Single Quotes: Single quotes operate like double quotes, but there effect is stronger.
They preserve only the meaning of single quotes. Any enclosed metacharacters are
treated as literal characters
Examples
32
33
JOB CONTROL
One of the important features of a shell is job control.
A job is a user task run on the computer. Editing , sorting, and reading mails are all
examples of jobs.
There are namely two jobs
1 Foreground job
2 Background job
Foreground job: A foreground job is any job run under the active supervision of the
user. It is started by the user and may interact with user. While it is running no other jobs
may be started.
$ ls /
^Z
$fg
$fg
^c
Background job: if we know a job will take long time, we may run it as background job.
Jobs run in the background free the keyboard and monitor so we may use them for other
purposes.
Examples
$ ls | more
[1]
1795841
$ stop
%1
[1]
$ bg
%1
[1]
$ kill
ls | more
%1
[1] + Terminated
35
cse1
sept 10
F1
-rw-------
cse1
sept 10
F2
cse1
sept 10
F1
-rw-------
cse1
sept 10
F2
36
cse1
sept 10
F1
-rw-------
cse1
sept 10
F2
cse1
sept 10
F1
-rw-------
cse1
sept 10
F2
LISTING ALIASES
To list all aliases we use the alias command with no arguments. To list specific command
we use the alias command with one argument, the name of the alias command.
$ alias
file =ls -l
dir= ls -l
|more
stop=kill
-STOP $$
$ alias dir
dir= ls -l
|more
REMOVING ALIASES
Aliases are removed by using the unalias command. It has one argument, a list of aliases
to removed. When it is used with the all option (-a), it deletes all aliases hence must be
very careful with this option. It deletes all the aliases even the ones defined by system
administrator.
$ alias dir
dir= ls -l
|more
$ unalias dir
$ alias dir
dir: alias not found
37
VARIABLES
A variable is a location in memory where values can be stored. Each shell allows us to
create, store, and access values in variables. Each shell variable must have a name. The
name of a variable must start with an alphabetic or underscore (_) character. There are
two broad classifications of variables : user-defined and predefined.
Storing Data in Variables
All three shells provide a means to store value in variables.
Assignment in Korn and Bash
Syntax: variable=value
Assignment in C shell
Syntax:
C
cdpath
Explanation
Contains the search path for cd command
when the
38
EDITOR
home(HOME)
path(PATH)
prompt
shell(SHELL)
term(TERM)
autologout
VISUAL
VISUAL
OPTIONS
Shell options are listed below
Global (noglob) The global option controls the expansion of wildcard tokens in the
command tokens in a command. For example, when the global option is off, the list
file(ls) command use wildcards to match the files in a directory. Thus , the following
command list all files that start with file followed by one character
$ ls file?
On the onther hand, when the global option is on, wildcards become text charcters and
are not expanded. In this case , only the file name file? would be listed
Print Commands (verbose and xtrace) There are two print options, verbose and xtrace ,
that are used to print commands before they are executed. The verbose option prints the
command before it is executed. The xtrace option expands the command arguments
before it prints the command.
Command-Line Editor (emacs and vi) To specify that the emacs editor is to be used in
Korn shell, we turn emacs option. To specify vi editor we turn on vi option.
NOTE: These options are valid only in Korn shell.
Ignore End of File (ignoreeof) Normally if end of file (ctrl +d) is entered at the
command line , the shell terminates. To disable this action, we can turn on the ignore end
of file option, ignoreeof. With this option end of file generates an error message rather
than terminating the shell.
39
No Clobber Redirection (noclobber) When output or errors are directed to a file that
already exists, the current file is deleted and replaced by a new file. To prevent this
action, we set the noclobber option
FILTERS:
A filter is any command that gets its input from the standard input stream, manipulates
the input , and then sends the result to the standard output stream.
Filters and Pipes
Filters work naturally with pipes. Because a filter can send its output to the monitor, it
can be used on the left of a pipe; because a filter can receive its input from the keyboard,
it can be used on the right of a pipe. In other words , a filter can be used on the left of a
pipe, between two pipes, and on the right of a pipe.
Concatenating Files
UNIX provides a powerful utility to concatenate commands. It is known as the catenate
command , or cat for short. It combines one or more files by appending them in the order
they are listed in the command. The input can come from the keyboard; the output goes to
the monitor.
Cat has 6 options
-e
-n
number lines
-s
-t
-u
unbuffered output
-v
40
Option:
-N
number of lines
$ head -2 f1
Unix is multitasking os
Unix is multiuser os
The above command displays first 2 lines from the beginning of the file
When multiple files are include in one head command, head displays the name of the file
before output.
$ head
-2 f1 f2
===> f1 <===
Unix is multitasking os
Unix is multiuser os
===> f2 <===
Unix is widely used
It provides best security
tail command:
The tail command also outputs data, only this time from end of the file. Although only
one file can be referenced it has several options
+N
-N
-l
Counts by line(default)
-c
Counts by character
-b
-r
$tail -5 sample
The above command displays last 5 lines from the end of the file
TRANSLATING CHARACTERS:
tr command: it replaces each character in a user-specified set of characters with a
corresponding characters in a second specified set. Each set is specified as a string.
Simple Translate
41
Translate receives its input from standard input and writes its output to standard output. If
no options are specified the text is matched against the string1 set and any matching
characters are replaced with the corresponding characters in the string2 set. Unmatched
characters are unchanged.
Example:
$ tr aeiou AEIOU
It is very easy to use TRANSLATE.
It Is vEry EAsY tO UsE TRANSLATE.
#input
#output
42
$ tr c aeiou *
It is very easy to use TRANSLATE.
***i***e***ea****o*u*e*********
Files with Duplicate Lines
The uniq command deletes duplicate lines, keeping the first and deleting the others. To be
deleted the lines must be adjacent. Duplicate lines that are not adjacent are not deleted. To
delete nonadjacent lines the file must be sorted.
Suppose let file sample contain
Unix
Unix
Mefaf
Mefa
Unix
Mefa
Dld
Now $ uniq sample
Unix
Mefa
Unix
Mefa
Dld
Above it writes all of the nonduplicated lines and the first of a series of duplicated lines.
This is the default result.
Nonduplicated Lines
The nonduplicated lines option is u. it suppresses the output of the duplicated lines and
list only the unique lines in the file.
$ uniq u sample
Unix
Mefa
Dld
43
44
GREP:
grep stands for global regular expression print.
It is a family of programmes that is used to search the input file for all lines that match a
specified regular expression and write them to the standard output file.
The syntax for using grep command is:
$ grep options regular_expression file_list
OPERATION:
For each line in the standard input file(input file or keyboard), grep performs the
following operations:
1. Copies the next input line
2. into the pattern space. The pattern space is a buffer that can hold only one text
line.
3. Applies the regular expression to the pattern space.
4. If there is a match, the line is copied from the pattern space to the standard output.
The grep utilities repeat these three operations on each line in the input.
GREP FAMILY:
There are three utilities in the grep family,
1. grep,
2. fast grep (fgrep) and
3. extended grep (egrep).
All these three utilities examines each line in the file one by one, when a line contains a
matching pattern, the line is output.
There are several options available to the grep family.
grep:
It supports only a limited number of regular expressions. It is the only member of the
grep family that allows saving the result of a match for later use.
Example:
$ grep day f1
This is the day
45
In the above example the grep command searches for lines that contain the expression
day and prints the line. Here in this case there is only one line that matches with the
given regular expression.
fgrep:
It your search criteria requires only sequence expressions, fgrep is the best utility.
Example:
$ fgrep n , f1
1:This is the day,
2:We will rejoice,
In the above example the fgrep command searches for all the lines having a comma in it
and prints the line. The n option gives even the line number of the matching line.
egrep:
egrep is the most powerful of the three grep utilities. It supports most regular expressions
but not all of them.
Examples:
$ egrep ^$ f1
This command selects the lines from the input file f1 that has exactly three characters.
$ egrep f1
This command selects the lines from the input file f1 that has atleast three characters.
$ egrep v ;$ f1
This command selects the lines fron the input file f1 that doesnt end with a semicolon.
The option v means inverse output. It displays the lines that does not match with the
input pattern.
$ egrep e ^$ f1
This command counts the number of blank lines in the input file.
$ egrep b+ file
This command displays the lines which are named with b, bb, bbb and so on. The plus
sign here means one or more occurrence of the given character.
46
47
SED:SED is an acronym for stream editor. It is not a true editor, it does not change anything in
the original file.
SED scans the input file line by lineand applies a list of instructions (sed script) to each
line in the input file.the script which is usually a separate file, can be included in the sed
command line if it is a one-line command.
The syntax for using the sed command is:
$ sed options script file_list
SCRIPTS
In addition to input data, sed also requires one or more instructions that provide editing
criteria. Most of the time, however, instructions are placed in a file known as sed script.
Each instruction in a sed script contains an address and a command.
SCRIPT FORMATS
When the script fits in a few lines, its instructions can be included in the command line.
In this case, the script must be enclosed in quotes.
$ sed e adress command input_file (
inline script )
$ sed f script.sed input_file
(script file)
INSTRUCTION FORMATS
Each instruction consists of an address and a command.
address ! command ( instruction format )
The address selects the line to be processed ( or not processed ) by the command. The
exclamation point is an optional address compliment. When the compliment operator is
present, any line that does not match the address is selected.
COMMENTS
A comment is a script line that documents or explains one or more instructions in a script.
Comment lines begin with a comment token, which is the pound sign ( # ). If the
comment requires more than one line, each line must start with the comment token .
48
OPERATION
Each line in the input file is given a line number by sed. For each line, sed performs the
following operations:
1. Copies an input line to the pattern space. The pattern space is a special buffer
capable of holding one or more text lines for processing.
2. Applies all the instructions in the script, one by one, to all pattern space lines that
match the specified address in the instruction.
3. Copies the contents of the pattern space to the output file unless directed not to by
the n option flag. When all of the commands have been processed, sed repeats
the cycle starting with one.
ADDRESSES
The address in an instruction determines which lines in the input file are to be processed
by the commands in the instruction. Addresses in sed can be one of four types.
1. Single line
2. Set of lines
3. Range of lines
4. Nested addresses
SINGLE LINE ADDRESSES
A single line address specifies one and only one line in the input file.
SET OF LINE AND ADDRESSES
A set of line address is a regular expression that may match zero or more lines, not
necessarily consecutive, in the input file. The regular expression is written between two
slashes. Any line in the input file that matches the regular expression is processed by the
instruction command. When the regular expression is missing, every line is selected.
RANGE ADDRESSES
An address range defines a set of consecutive lines. Its format is start address, comma
with no space, and end address.
start-address,end-address
The start and end address can be a sed line number or a regular expression.
line-number,line-number
line-number,/regexp/
/regexp/,line-number
/regexp/,/regexp/
49
When a line that is in the pattern space matches a start range, it is selected for processig.
Each input line is processed by the instructions command untill the stop address matches
a line. The line that matches the stop address is also processed by the command, but at
that point, the range is no longer active. If at some future line the start range again
matches, the range is again active untill a stop address is formed.
NESTED ADDRESSES
A nested address is an address that is contained within another address. While the outer
address range must be either a set of lines or an address range, the nested addresses may
be either a single line, a set of lines, or another range.
Example:
20,30 {
/^$/d
}
The first command specifies the line range; it is the outer command. The second
command, which is enclosed in braces, contains the regular expression for a blank line.
COMMANDS
There are 25 commands that can be used in an instruction. We group them into 9
categories based on how they perform their task.
LINE NUMBER COMMAND:
The line number command writes the current line number at the beginning of the line
when it writes the line to the output without affecting the pattern space. It is similar to
grep n option.
MODIFY COMMANDS:
Modify commands are used to insert, append, change, or delete one or more whole lines.
The modify command requires that any text associated with them be placed on the next
line in the script. All modify commands apply to the whole line. One cannot modify just
part of a line.
There are five commands in the modify commands.
1.Insert command(i):
Insert adds one or more lines directly to the output before the address. This command
can only be used with single line and set of lines.
Example:
50
51
A cse 19
D cse 07
C cse 10
In the above example we are changing the second line in the input file with the text given
in the script file.
52
Example:
$ sed s/unix/*** & ***/ file1
Substitute Operation
We can use the substitute command to add, delete,or replace part of a line.
When the pattern matches the text, sed first deletes the text and then inserts the
replacement text.
Delete Part Of A Line:
To delete part of a line, we leave the replacement text empty.
Example:
$ sed s/[0-9]//g
Input:
123abc456
321cba654
Output:
abc
cba
in this session we wanted to delete all the digits. Therefore we used the global flag(g) at
the end of the pattern.
Change Part Of Line:
To change only part of a line, we create a pattern that matches the part to be changed and
then place the new text in the replacement expression.
Example:
$ sed s/ /
/g
Input:
Now is the time
For all good students
To come to the aid
Of their college.
Output:
Now
is
For
all
To
come
Of
their
the
good
to
college.
time
students
the
aid
53
#addPart.sed
$/^/ /
$/$/--/
Input:
Now is the time
For all good students
To come to the aid
Of their college.
Output:
Now is the time-For all the students-To come to the aid-Of their college.-TRANSFORM COMMAND
Transform command requires two parallel set of characters. Each character in the first
string represents a value to be changed to its corresponding character in the second string.
Address y / source characters / replacement characters /
Characters that do not match a source character are left unchanged.
Example:
$ sed y/aeiou/AEIOU/
Input:
A good time was had by all
Under the Harest Moon last September.
Output:
A gOOd tIme wAs hAd by All
UndEr thE HarvEst MOOn lAst SeptEmbEr.
This session demonstrates the transform command by translating lowercase vowels to
their upper case forms.
54
1.Next Command(n)
The next command (n) forces sed to read the next line from the input file. Before reading
the next line, however, it copies the current contents of the pattern space to the putput,
deletes the current text in the pattern space, and then refills it with the next input line.
After reading the input line it cintinues processing through the script.
Example:
$ sed f deleteBlankLines.sed file1
#deleteBlankLines.sed
/^[0-9]/{
n
/^$/d
}
Input:
Second Line: Line 1 & line 3 blank
4th line- followed by non blank line
This is line 5
6th line followed by blank line
Last line (#8)
Output:
Second Line: Line 1 7 line 3 blank
4th line-followed by non blank line
This is line 5
6th line followed by blank line
Last line (#8)
Whenever a line that starts with a digit is immediately followed by a blank line, we delete
the blank line. We first locate a line that begins with a digit. To see if the next line is
blank, we first read it using the next command. This command forces the current line to
be written. We then check the new line. If it is blank, we delete it.
2.Append Next Command(N):
The append next command doesnt clear the pattern space before inputting the next line.
It adds the next input line to the current contents of the pattern space.
55
Example:
$ sed f appendLines.sed file1
#appendLines.sed
N
s/\n/ /
Input:
1111first1111
2222second222
3333third3333
4444fourth444
5555last55555
Output:
1111first1111 2222second222
3333third3333 4444fourth444
5555last55555
If we simply append the lines, when they are printed they will revert two separate lines
because there is a new line at the end of first line.
3.Print Command(p):
The print command copies the contents of the pattern space to the standard output file. If
there are multiple lines in the pattern space, they are all copied. The contents of the
pattern space are not deleted by the print command.
Example:
$sed p linesOfNums.dat
Input:
11111111111
22222222222
33333333333
44444444444
5555last5555
Output:
11111111111
11111111111
22222222222
22222222222
33333333333
33333333333
56
44444444444
44444444444
5555last5555
5555last5555
The real use of this command is to let the script control the printing.
4.Print First Line Command(P):
The print first line command prints only the first line. It prints the contents of the pattern
space up to and including a newline character. Any text following the first newline is not
printed.
Example:
$ sed nf printFirstLIne.sed file1
#printFirstLine.sed
$!N
/\n
/P
D
Input:
This is line 1.
This is line 2.
Line 3 starts with a tab.
Line 4 starts with a tab.
This is line 5. Its the last line.
Output:
This is line 2.
Line 3 starts with a tab.
When we search the pattern space for a newline innediately followed by a tab,if we print
this combination we print only the first line. We then delete the first line only. When we
append to the pattern space, we end up with two lines separated by a new line in the
pattern space. The script then uses a newline-tab pattern to determine if the appended line
starts with a tab. If it does, we print the first line in the pattern space. Finally the script
deletes the first line in the pattern space leaving the second line.
5.List Command(l):
The list command converts the unprintable characters to their octal code.
Example:
$ sed n l listfile
57
Input:
Octal
Octal
Octal
0:
9:
21: | |
||
||
Output:
Octal
Octal
Octal
0:
9:
21: |\21|
||
|\t|
FILE COMMANDS
There are two file commands that can be used to read and write files. The basic format
for the read and write command is as
Address r file-name
Address w file-name
(read command)
(write command)
58
BRANCH COMMANDS
The branch commands change the regular flow of the commands in the script file. The
branch commands allow us to skip one or more commands in the script file. There are
two branch commands.
Each branch command must have a target, which is either a label or the last instruction in
the script (a blannk label). A label consists of a line that begins with a colon (:) and is
followed upto seven characters that constitute the label name.
Example:
:comHere
Branch Command(b):
The branch command follows the normal instruction format consisting of an address, the
command, and an attribute that can be used to branch to the end of the script or to a
specific location within the script. If no label is provided, the branch is to the end of the
script, at which point the current contents of the pattern space are copied to the output file
and the script is repeated for the next input file.
Example:
$ sed f branch.sed inputfile
#branch.sed
/(1)/ b
/(2)/ b print2
/(3)/ b print3
b
:print3
p
p
b
:print2
p
input:
print once.
(2)print twice.
(3)print thrice.
(4)print once.
Output:
Print once.
59
(2)print twice.
(2)print twice.
(3)print thrice.
(3)print thrice.
(3)print thrice.
(4)print once.
It prints the lines in a file once, twice, or thrice depending on a print control at the
beginning of the file. The format of the print control is a set of paranthesis containing a
digit. If there is no print control it prints the line only once.
Branch On Substitution Command(t):
We use the branch on substitution command (or the test command) only if a substitution
has been made.
Example:
$ sed f branchsub.sed filename
#branchsub.sed
s/(1)//
t
s/(2)//
t print2
s/(3)//
t print3
b
:print3
p
p
b
:print2
p
input:
(1)Print once.
(2)print twice.
(3)print thrice.
Default: print once.
Output:
Print once.
print twice.
print twice.
print thrice.
60
print thrice.
print thrice.
Default: print once.
If a substitution was made the branch is taken. It the substitution pattern was not found,
the branch is ignored. The print controls have been removed except for the last one,
which is a user input error.
HOLD SPACE COMMAND
It is used tio save the pattern space. There are five commands that are used to move text
back and forth between the pattern space and the hold space.
1.Hold And Destroy Command(h):
The hold and destroy command copies the current contrents of the pattern space to the
hold space and destroys any text currently in the hold space.
2.Hold And Append Command(H):
The hold and append command appends the current contrents of the pattern space to the
hold space.
3.Get And Destroy Command(g):
The get and destroy command copies the text in the hold space to the pattern space and
destroys any text currently in the pattern space.
4.Get And Append Command(G):
The get and append command appends the curent contents of the hold space to the pattern
space.
5.Exchange Command(x):
The exchange command swaps the text in the pattern and holds spaces. That is, the text in
the pattern space is moved to the hold space, and the data were in the hold space are
moved to the pattern space.
Example:
$ sed nf exchange.sed filename
#exchange.sed
h
n
G
P
Input:
Line 1
61
Line 2
Line 3
Line 4
Line 5
Line 6
Output:
Line 2
Line 1
Line 4
Line 3
Line 6
Line 5
It does not work with an odd number of lines. In that case, the last line is printed twice.
The easiest solution is to run a sed script to delete the last line.
GREP AND SED
To demonstrate how we can use sed in place of grep, we look at two general situations.
1.Lines that match a regular expression
If we need to use sed instead of grep to find a line that matches a regular expression, we
use the print(p) command in sed and turn off the automatic output option (-n).
The two commands are as follows.
grep regular expression file1
sed -n /regular expression/p file1
2.Lines that do not match a regular expression
If we want to use sed instead of grep to find lines that do not match a regular expression,
we again use the print command, but this time we complement the address(!) in sed.
Because we are again controlling the printing, we must use the no output option (-n).
The two commands are as follows.
grep v regular expression
sed -n /regular expression/!p
file1
file1
62
AWK command:
The awk utility, which takes its name from the initials of its authors (Alfred V.Aho,Peter
J. Weinberger, and Brian W. Kernighan), is a powerful language. Its behavior is to some
extent like sed. It reads the input file line by line and performs an action on part of or on
the entire line.
awk utility has two options
-F: specifies the input field separator
-f: names the script file
awk requires one or more instructions that provide editing instructions. When they are
only few instructions they can be entered at the command line from the keyboard.
However if there are many lines it is placed in a file known as an awk script. Each
instruction in an awk script contains a pattern and action.
The format for command-line script is:
$ awk pattern{action} input-file
FIELDS AND RECORDS:
The awk utility views a file as a collection of fields and records. Each line in awk is a
record.
Field : A field is a unit of data that has informational content.
Record : A record is a collection of fields treated as unit.
Suppose consider a file student
$ cat student
1
John 20
record 1
2
James 19
record 2
3
Peter 16
record 3
4
David 18
record 4
The above file contains four records , each with three fields namely serial number, name,
marks.
Field Buffers
There are many field buffers as there are fields in the current record of the input file.
Each field buffer has a name, which is dollar sign ($) followed by the field number in the
current record.
For Example
$1 First Field, $2 Second Field, $3 Third Field..
Record Buffer
There is only one record buffer available. Its name is $0. It holds the whole record. In
other words, its content is the concatenation of all field buffers with one field separator
character between each field.
The record buffer ($0) contains the concatenation of all the fields which may be changed
during script processing.
Variables
There are two different types of variables in awk
1.System variables
2.User-defined variables
63
System Variables
Variable
FS
RS
OFS
ORS
NF
NR
FNR
FILENAME
ARGC
ARGV
RLENGTH
RSTART
Function
Input field separator
Input record separator
Output field separator
Output record separator
Number of nonempty fields in current record
Number of records read from all files
Number of records read record number in current
File
Name of the current file
Number of command-line arguments
Command-line argument array
Length of string matched by a built-in string function
Start of string matched by a built-in string function
64
SCRIPTS:
awk scripts are divided into three parts: begin, body and end
BEGIN { Begins Actions}
BEGIN part
Pattern {action}
Pattern {action}
BODY
Pattern {action}
END {Ends Actions}
END part
Simple awk example
$ cat student
1
John 20
19
2
James 19
10
3
Peter 16
10
4
David 8
3
1.Write awk script to find total marks for each student (From STUDENT Table)
# total.awk
BEGIN {print Sno \t Sname \t Sub1 \t Sub2 \t Total}
{
total=$3 + $4
print $0,total
}
END {print End Totals}
Output:
$ awk f total.awk student
Print Total
Sno Sname sub1 sub2 Total
1
John 20
19
39
2
James 19
10
29
3
Peter 16
10
26
4
David 8
3
11
End Totals
PATTERNS:
The pattern identifies which records in the file are to receive an action. The awk utility
can use several different types of pattern. As it executes a script, it evaluates the patterns
against the records found in the file. If the pattern matches the record the action is taken
else skipped.
awk patterns are divided into two categories
1.Simple patterns
BEGIN
END
EXPRESSIONS
NOTHING(No Pattern)
2.Range patterns
65
Simple Patterns :A simple pattern matches one record. When a pattern matches a record
the result is true and actions statement is executed.
BEGIN AND END : BEGIN is true at the beginning of the file before the first record is
read. It is used to initialize the script before passing any data.
Example:
It sets the field separator here we set output field separator (OFS) to tab space.
BEGIN
{
FS = \t
OFS= \t
}
END { }
END is used at the conclusion of the script.
Expressions:
The awk utility supports four expressions: regular, arithmetic, relational, and logical.
Regular expressions: The awk regular expressions are those defined in egrep. Also awk
requires two operators: match (~) and not match (!~).
NOTE: When using a regular expression it must be enclosed in /slashes/.
Example:
$ cat sample
Unix
MEFA
DLD
PS
Unix is a multiuser operating system.
1.Write an awk command that prints the lines which begins with U
$ awk $0 ~/^U/ {print} sample
Unix
Unix is a multiuser operating system.
In the above command the regular expression is $0 ~/^U/ . it displays the records which
begin with U.
2. Write an awk command that prints the lines which does not begins with U
$ awk $0 !~/^U/ {print} sample
MEFA
DLD
PS
In the above command the regular expression is $0 !~/^U/ . it displays the records which
does not begins with U.
Arithmetic expressions: An arithmetic expression is the result of an arithmetic
operation. When the expression is arithmetic, it matches the record when the value is
nonzero, either plus or minus; it does not match the record when it is zero (false).
Example:
We have input file student
$ cat student
66
1
John
20
19
2
James
19
14
3
Jacob
8
9
4
Samuel
19
10
5
David
6
12
3. Write an awk command to find total marks for all student and display
$ awk total= $3 + $4 {print $2, total} student
John
39
James
33
Jacob
17
Samuel
29
David
18
Relational expressions: Relational expressions compare two values and determine if the
first is less than, equal to, or greater than the second. When the two values are numeric an
algebraic comparison is use; when they are strings, string comparison is used.
Relational expressions
Operator
<
<=
==
!=
>
>=
Explanation
Less than
Less than or equal
Equal
Not equal
Greater than
Greater than or equal
Example:
We have input file student
$ cat student
1
John
20
19
2
James
19
14
3
Jacob
8
9
4
Samuel
19
10
5
David
6
12
4. Write an awk command that prints details of the student who have less than 10
marks in subject1
$ awk $3 < 10 {print } student
3
Jacob 8
9
5
David 6
12
The above command searches for records whose third field i.e first subject marks is less
than 10 and display those records only.
Logical expressions: A logical expression uses logical operator to combine two or more
expression.
Operator
Explanation
67
!expr
expr1 && expr2
expr1 || expr2
Not expression
Expression 1 and expression 2
Expression 1 or expression 2
Result of logical AND is true if and only if both expressions are true; it is false if either of
the expression is false.
Result of logical OR is true if either of the expression is true; it is false if and only if both
expressions are false.
The NOT operator complements
For the same student file
5. Write an awk command that prints details of the students who have less than 15
marks in both the subjects
$ awk $3 < 15 && $4 <15 {print} student
3
Jacob 8
9
5
David 6
12
Note:The above command searches for records whose both subject marks are less than
15 and it displays them.
NOTHING
When no address pattern is entered awk applies the action to every line in the input file.
This is the easiest way to specify that all lines are to be processed.
6.Write an awk command that prints the details of all the students.
$ awk {print $0} student
1
John
20
19
2
James
19
14
3
Jacob
8
9
4
Samuel
19
10
5
David
6
12
RANGE PATTERNS
A range pattern is associated with arrange of records or lines. It is made up of two simple
patterns separated by a comma as shown
start-pattern, end-pattern
The range starts with the record that matches the start pattern and ends with the next
record that matches the end pattern. if the start and end patterns are the same, only one
record is in the range.
Example:
7.$ awk NR==2 , NR==4 {print} student
2
James 19
14
3
Jacob 8
9
4
Samuel
19
10
The above command prints lines 2 to 4 of student file using simple expressions that
identify the line number range.
Note : NR means Number of the Record
68
ACTIONS:
Actions are instructions or statements. They are called actions in awk because they act
when the pattern is true.
In awk an action is one or more statement associated with a pattern. There is a one-to-one
relationship between n action and pattern. One action is associated with only one pattern.
The action statements must be enclosed in a set of braces. A set of braces containing
pattern/action pairs or statements is known as block. When an action consists of several
statements they must be separated by a statement separator such as semicolon, a newline,
or a set of braces.
Basic awk statements
Expression
Output
Print
Printf
Sprintf
Statements
Decision
Control
Loop
next
while
for
do-while
getline
exit
69
1
2
3
4
5
A
B
C
D
E
CSE
CSE
ECE
ECE
CSE
10000
16000
15000
12000
13000
8.Write an awk command to find total salary of CSE employees only (From emp
table)?
awk-script (totalcse.awk)
BEGIN{total=0}
If ( $3 = =CSE)
{
total=total + $4
}
END{print TOTAL=total}
$ awk f totalcse.awk emp
TOTAL=39000
LOOPS:
We have the input file(student)
1
John
20
19
2
James
19
14
3
Jacob
8
9
4
Samuel
19
5
David
6
12
10
1.Write an awk script to find total marks for all student using while loop?
awk-script(total.awk)
{
Total=0
I=3
While (I < = NF)
{
Total = Total + $I
I++
}
print ( $0,Total)
}
70
Output:
$ awk f total.awk student
Sno Sname
sub1 sub2 Total
1
John
20
19
39
2
James
19
14
33
3
Jacob
8
9
17
4
Samuel
19
10
29
5
David
6
12
18
Note : NF means Number of fields
2.Write an awk script to find total marks for all student using for loop?
awk-script(total.awk):
{
Total=0
for( i=3; i<=NF; i++)
{
Total = Total + $i
}
print ( $0,Total)
}
Output:
$ awk f total.awk student
Sno Sname
sub1 sub2 Total
1
John
20
19
39
2
James
19
14
33
3
Jacob
8
9
17
4
Samuel
19
10
29
5
David
6
12
18
Note : NF means Number of fields
3.Write an awk script find total marks for all student using do-while loop?
{
total=0
i=2
do
{
total = total +$i
i++
} while ( i<=NF)
print ( $0,total)
}
Output:
$ awk f total.awk student
Sno Sname
sub1 sub2 Total
1
John
20
19
39
2
James
19
14
33
71
3
4
5
Jacob
Samuel
David
8
6
9
19
12
17
10
18
29
ASSOCIATIVE ARRAYS:
An array is a collection of variables that can be referred to individually or as a collection.
An index is used to refer an individual. The array name is used to refer to the whole array
awk uses string for its indexes unlike programming language where integers are used.
Each index entry identifies data that are stored in an array element i.e the index entry is
associated with the array element. For this reason only awk arrays are known as
associative arrays.
Eg:
$ cat emp
1
A
CSE 10000
2
B
CSE 16000
3
C
ECE 15000
4
D
ECE 12000
5
E
CSE 13000
1.Write an awk script to find total salary of employees in department wise?
awk_script(sal.awk):
BEGIN{OFS= \t}
{
dept[$3]=dept[$3]+$4
}
END{
for (item in Dept)
{
print item : Dept[item]
}
}
Output:
$ awk f sal.awk emp
CSE:39000
ECE:27000
72
STRING FUNCTIONS
The awk utility contains several string functions. These are the built-in functions.
1.Length
The length function has the format
length (string)
Length returns the number of characters in the string parameters.
Example:
Consider sample file
$ cat sample
Unix
Mefa
Dld
Ps
Unix is a multiuser operating system.
1.Write an awk script to illustrate length function?
{
print len= length ($0) , \t $0
}
Output:
$ awk f len.awk sample
4
Unix
4
Mefa
3
Dld
2
Ps
36
Unix is a multiuser operating system.
2.Index
The index function returns the first position of a substring within a string. Its format is
Index (string, substring)
The substring can be string constant or variable . if the substring is not found it returns
zero.
Example:
2.Write an awk script to illustrate index function?
{
loc =index($0,cse)
print cse is found at loc position
}
$ awk f index.awk
Mrec cse
cse is found at 6 position
ctrl +d
In the above command input file is not given it waits for user to give input
3.Substring
The substr function extracts a substring from a string. It has two formats:
73
74
SHELL PROGRAMS:
Aim:write a shell program to perform arithmetic operations.
Program:echo enter two values
read a
read b
c=`expr $a + $b`
echo a+b=$c
c=`expr $a - $b`
echo a-b=$c
c=`expr $a \* $b`
echo a*b=$c
c=`expr $a / $b`
echo a/b=$c
c=`expr $a % $b`
echo a%b=$c
OUTPUT:$sh filename
enter two numbers
20
10
a+b=30
a-b=10
a*b=200
a/b=2
a%b=0
75
76
OUTPUT:$sh filename
enter any character
H
It is in upper case
77
OUTPUT:$sh filename
enter any number
5
the factorial of given number=120
78
79
80
Aim: Write a shell script, which deletes all lines containing the word UNIX in the
files supplied as arguments to this shell script.
Program:if [ $# -eq 0 ]
then
echo enter only one file name
exit
fi
if [ -f $1 ]
then
cat $1 | grep UNIX > file
if [ -s file ]
then
cat $1 | grep UNIX -v > file
mv file $1
else
echo The file does not contain word UNIX
fi
else
echo Invalid Filname
fi
OUTPUT:
$cat sample
This is UNIX recard
This is sample file
$sh prograname sample
$cat sample
This is sample file
81
FILE MANAGEMENT:
SYSTEM CALLS FOR FILE MANAGEMENT:
FILE I/O:- Most Unix file I/O can be performed using only five functions: open, read,
write, lseek, and close.
File Discritors:- To the kernel all open files are referred to by file descriptors .A file
descriptor is a non-negative integer. When we open an existing file or create a new file,
the kernel returns a file descriptors to the process.
Open function:- This function is used to open existing file or to create a new file
#include <sys/types.h>
#include<sys/stat.h>
#include<fcntl.h>
int open(const char pathname, int oflag, /* , mode_t mode */ );
Returns: file descriptor if OK, -1 on error
In the open function the pathname is the name of the file to open or create. There are a
multitude of options for this function,which are specified by the oflag argument.
Oflag arguments:O_RDONLY
Open for reading only
O_WRONLY
Open for write only
O_RDWR
Open for reading and writing
One and only one of these three constants must be specified. The following constants are
optional:
O_APPEND:- Which is used to append text to the end of the file.
O_CREAT:- Which is used to create the file if it doesnt exist.This option requires a
third argument to the open function, the mode, which specifies the access permission
bits of the new file
There are nine permission bits for each file, divided into three categories.These are
shown in following fig.
Mode_t mode
Meaning
S_IRUSR
User_read
S_IWUSR
User_write
S_IXUSR
User_execute
S_IRGRP
Group_read
S_IWGRP
Group_write
S_IXGRP
Group_execute
S_IROTH
Other_read
82
S_IWOTH
S_IXOTH
Other_write
Other_execute
83
The interpretation of the offset depends on the value of the whence argument
If whence SEEK_SET, the files offset is set to offset bytes from the beginning of the file
IF whence is SEEK_CUR, then the files offset = its current value + the offset. The offset
can be positive or negative.
If whence is SEEK_END, the files offset = The size of the file + the offset
84