Sed - An Introduction and Tutorial

2/27/2017 SedAnIntroductionandTutorial
SedAnIntroductionand
TutorialbyBruceBarnett
Grymoire Lastmodified:ThuApr2316:37:48EDT2015
Navigation
QuickLinks
Unix/Linux
Quotes
SedCommands
BourneShell
:label #comment {....}Block
CShell
= printlinenumber a\ Append blabel Branch
File
c\change d and D Delete g and G Get
Permissions
Regular h and H Hold i\ Insert l Look
Expressions n and N Next p and P Print q Quit
grep rfilename ReadFile s/..../..../ Substitute tlabel Test
awk wfilename WriteFilename x eXchange y/..../..../ Transform
sed SedPatternFlags
find /g Global
tar /I IgnoreCase
inodes /p Print
Security /wfilename WriteFilename
IPv6
SedCommandLineoptions
Wireless
ShortOption(Longoption) Sedversion
Hardware
n Classic
spam
escript Classic
Deception
fscriptfile Classic
PostScript
escript(expression=script) GNUsed
Halftones
fscriptfile(file=scriptfile) GNUsed
Privacy
BillofRights h(help) GNUsed
References n(quietsilent) GNUsed

Top10reasons V(version) GNUsed
toavoidCSH r(regexpextended) GNUsed
sedChart PDF i[SUFFIX](inplace[=SUFFIX]) GNUsed
awkReference lN(linelength=N) GNUsed
HTML b(binary) GNUsed
Magic s(separate) GNUsed
Search z(nulldata) GNUsed
About u(unbuffered) GNUsed
Donate (followsymlinks) GNUsed
(posix) GNUsed
Google+: iSUFFIX MacOSX,FreeBSD
Bruce
a MacOSX,FreeBSD
Barnett
l MaxOSX,FreeBSD
Twitter: E MacOSX,FreeBSD X
@grymoire r FreeBSD
ISUFFIX FreeBSD
TableofContents
NoteYoucanclickonthetableofcontentssectionstojumpto
thatsection.
http://www.grymoire.com/Unix/Sed.html 1/47
Thenclickonthesectionheaderofanysectiontojumpbacktothe
tableofcontents.
TheAwfulTruthaboutsed
Theessentialcommand:sforsubstitution
Theslashasadelimiter
Using&asthematchedstring
Using\1tokeeppartofthepattern
ExtendedRegularExpressions
SedPatternFlags
/gGlobalreplacement
Issedrecursive?
/1,/2,etc.Specifyingwhichoccurrence
/pprint
Writetoafilewith/wfilename
/IIgnoreCase
Combiningsubstitutionflags
Argumentsandinvocationofsed
Multiplecommandswithecommand
Filenamesonthecommandline
sedn:noprinting
Using'sed/pattern/'
Using'sedn/pattern/p'toduplicatethefunctionofgrep
sedfscriptname
sedinshellscripts
QuotingmultiplesedlinesintheCshell
QuotingmultiplesedlinesintheBourneshell
sedV
sedh
Asedinterpreterscript
SedComments
Passingargumentsintoasedscript
Usingsedinashellhereisdocument
Multiplecommandsandorderofexecution
AddressesandRangesofText
Restrictingtoalinenumber
Patterns
Rangesbylinenumber
Rangesbypatterns
Deletewithd
Printingwithp
Reversingtherestrictionwith!
Relationshipsbetweend,p,and!
Theqorquitcommand
Groupingwith{and}
Operatinginapatternrangeexceptforthepatterns
Writingafilewiththe'w'command
Readinginafilewiththe'r'command
SunOSandthe#CommentCommand
Adding,Changing,Insertingnewlines
Appendalinewith'a'
Insertalinewith'i'
Changealinewith'c'
Leadingtabsandspacesinasedscript
Addingmorethanoneline
Addinglinesandthepatternspace X
Addressrangesandtheabovecommands
MultiLinePatterns
Printlinenumberwith=
Transformwithy
Displayingcontrolcharacterswithal
WorkingwithMultipleLines
Matchingthreelineswithsed
Matchingpatternsthatspanmultiplelines
Usingnewlinesinsedscripts
http://www.grymoire.com/Unix/Sed.html TheHoldBuffer 2/47

TheHoldBuffer
Exchangewithx
ExampleofContextGrep
HoldwithhorH
Keepingmorethanonelineintheholdbuffer
GetwithgorG
FlowControl
Testingwitht
Debuggingwithl
Analternatewayofaddingcomments
Thepoorlydocumented
Passingregularexpressionsasarguments
Insertingbinarycharacters
GNUsedCommandLinearguments
Theposixargument
Theversionargument
ThehHelpargument
ThelLineLengthArgument
ThesSeparateargument
Theiinplaceargument
Thefollowsymlinksargument
ThebBinaryargument
TherExtendedRegularExpressionargument
TheuUnbufferedargument
ThezNullDataargument
FreeBSDExtensions
aordelayedopen
TheIinplaceargument
EorExtendedRegularExpressions
Usingwordboundries
CommandSummary
InConclusion
MoreReferences
Copyright1994,1995BruceBarnettandGeneralElectricCompany
Copyright2001,2005,2007,2011,2013BruceBarnett
Allrightsreserved
Youareallowedtoprintcopiesofthistutorialforyourpersonaluse,and
linktothispage,butyouarenotallowedtomakeelectroniccopies,or
redistributethistutorialinanyformwithoutpermission.
Originalversionwrittenin1994andpublishedintheSunObserver
IntroductiontoSed
Howtousesed,aspecialeditorformodifyingfilesautomatically.Ifyou
wanttowriteaprogramtomakechangesinafile,sedisthetooltouse.
ThereareafewprogramsthataretherealworkhorseintheUNIXtoolbox.
Theseprogramsaresimpletouseforsimpleapplications,yethavearich
setofcommandsforperformingcomplexactions.Don'tletthecomplex
potentialofaprogramkeepyoufrommakinguseofthesimpleraspects.
I'llstartwiththesimpleconceptsandintroducetheadvancedtopicslater
on. X
WhenIfirstwrotethis(in1994),mostversionsofseddidnotallowyouto
placecommentsinsidethescript.Linesstartingwiththe'#'charactersare
comments.Newerversionsofsedmaysupportcommentsattheendofthe
lineaswell.
Onewaytothinkofthisisthattheold,"classic"versionwasthebasisof
GNU,FreeBSDandSolarisverisonsofsed.Andtohelpyouunderstand
whatIhadtoworkwith,hereisthe sed(1)manualpagefrom
Sun/Oracle .
http://www.grymoire.com/Unix/Sed.html
TheAwfulTruthaboutsed 3/47
TheAwfulTruthaboutsed
Sedistheultimatestreameditor.Ifthatsoundsstrange,pictureastream
flowingthroughapipe.Okay,youcan'tseeastreamifit'sinsideapipe.
That'swhatIgetforattemptingaflowinganalogy.Youwantliterature,
readJamesJoyce.
Anyhow,sedisamarvelousutility.Unfortunately,mostpeopleneverlearn
itsrealpower.Thelanguageisverysimple,butthedocumentationis
terrible.TheSolarisonlinemanualpagesforsedarefivepageslong,and
twoofthosepagesdescribethe34differenterrorsyoucanget.Aprogram
thatspendsasmuchspacedocumentingtheerrorsasitdoesdocumenting
thelanguagehasaseriouslearningcurve.
Donotfret!Itisnotyourfaultyoudon'tunderstandsed.Iwillcoversed
completely.ButIwilldescribethefeaturesintheorderthatIlearned
them.Ididn'tlearneverythingatonce.Youdon'tneedtoeither.
Theessentialcommand:sforsubstitution
Sedhasseveralcommands,butmostpeopleonlylearnthesubstitute
command:s.Thesubstitutecommandchangesalloccurrencesofthe
regularexpressionintoanewvalue.Asimpleexampleischanging"day"in
the"old"fileto"night"inthe"new"file:
seds/day/night/<old>new
Oranotherway(forUNIXbeginners),
seds/day/night/old>new
andforthosewhowanttotestthis:
echoday|seds/day/night/
Thiswilloutput"night".
Ididn'tputquotesaroundtheargumentbecausethisexampledidn'tneed
them.Ifyoureadmyearliertutorial onquotes ,youwouldunderstand
whyitdoesn'tneedquotes.However,Irecommendyoudousequotes.If
youhavemetacharactersinthecommand,quotesarenecessary.Andif
youaren'tsure,it'sagoodhabit,andIwillhenceforthquotefuture
examplestoemphasizethe"bestpractice."Usingthestrong(singlequote)
character,thatwouldbe:
sed's/day/night/'<old>new
Imustemphasizethatthesededitorchangesexactlywhatyoutellitto.
Soifyouexecuted
echoSunday|sed's/day/night/'
Thiswouldoutputtheword"Sunnight"becausesedfoundthestring"day"
intheinput.
Anotherimportantconceptisthatsedislineoriented.Supposeyouhave
theinputfile:
onetwothree,onetwothree
fourthreetwoone X
onehundred
andyouusedthecommand
sed's/one/ONE/'<file
Theoutputwouldbe
ONEtwothree,onetwothree
fourthreetwoONE
ONEhundred
Notethatthischanged"one"to"ONE"onceoneachline.Thefirstlinehad
"one"twice,butonlythefirstoccurrencewaschanged.Thatisthedefault
behavior.Ifyouwantsomethingdifferent,youwillhavetousesomeofthe
optionsthatareavailable.I'llexplainthemlater.
Solet'scontinue.
Therearefourpartstothissubstitutecommand:
s Substitutecommand
/../../Delimiter
one RegularExpressionPatternSearchPattern
ONE Replacementstring
Thesearchpatternisonthelefthandsideandthereplacementstringis
ontherighthandside.
We'vecovered quoting and regularexpressions. .That's90%ofthe

effortneededtolearnthesubstitutecommand.Toputitanotherway,you
alreadyknowhowtohandle90%ofthemostfrequentusesofsed.There
area...fewfinepointsthatanyfuturesedexpertshouldknowabout.
(Youjustfinishedsection1.Thereareonly63moresectionstocover.:)
Oh.Andyoumaywanttobookmarkthispage,....justincaseyoudon't
finish.
Theslashasadelimiter
Thecharacterafterthesisthedelimiter.Itisconventionallyaslash,
becausethisiswhated,more,andviuse.Itcanbeanythingyouwant,
however.Ifyouwanttochangeapathnamethatcontainsaslashsay
/usr/local/binto/common/binyoucouldusethebackslashtoquotethe
slash:
sed's/\/usr\/local\/bin/\/common\/bin/'<old>new
Gulp.Somecallthisa'PicketFence'andit'sugly.Itiseasiertoreadifyou
useanunderlineinsteadofaslashasadelimiter:
sed's_/usr/local/bin_/common/bin_'<old>new
Somepeopleusecolons:
sed's:/usr/local/bin:/common/bin:'<old>new
Othersusethe"|"character.
sed's|/usr/local/bin|/common/bin|'<old>new
Pickoneyoulike.Aslongasit'snotinthestringyouarelookingfor,
anythinggoes.Andrememberthatyouneedthreedelimiters.Ifyougeta
"Unterminated`s'command"it'sbecauseyouaremissingoneofthem.
Using&asthematchedstring
Sometimesyouwanttosearchforapatternandaddsomecharacters,like
parenthesis,aroundornearthepatternyoufound.Itiseasytodothisif
youarelookingforaparticularstring:
sed's/abc/(abc)/'<old>new
X
Thiswon'tworkifyoudon'tknowexactlywhatyouwillfind.Howcanyou
putthestringyoufoundinthereplacementstringifyoudon'tknowwhat
itis?
Thesolutionrequiresthespecialcharacter"&."Itcorrespondstothe
patternfound.
sed's/[az]*/(&)/'<old>new
Youcanhaveanynumberof"&"inthereplacementstring.Youcouldalso
doubleapattern,e.g.thefirstnumberofaline:
%echo"123abc"|sed's/[09]*/&&/'
123123abc
Letmeslightlyamendthisexample.Sedwillmatchthefirststring,and
makeitasgreedyaspossible.I'llcoverthatlater.Ifyoudon'twantitto
besogreedy(i.e.limitthematching),youneedtoputrestrictionsonthe
match.
Thefirstmatchfor'[09]*'isthefirstcharacterontheline,asthis
matcheszeroormorenumbers.Soiftheinputwas"abc123"theoutput
wouldbeunchanged(well,exceptforaspacebeforetheletters).Abetter
waytoduplicatethenumberistomakesureitmatchesanumber:
%echo"123abc"|sed's/[09][09]*/&&/'
123123abc
Thestring"abc"isunchanged,becauseitwasnotmatchedbytheregular
expression.Ifyouwantedtoeliminate"abc"fromtheoutput,youmust
expandtheregularexpressiontomatchtherestofthelineandexplicitly
excludepartoftheexpressionusing"(",")"and"\1",whichisthenext
topic.
ExtendedRegularExpressions
Letmeaddaquickcommentherebecausethereisanotherwaytowrite
theabovescript."[09]*"matcheszeroormorenumbers."[09][09]*"
matchesoneormorenumbers.Anotherwaytodothisistousethe"+"
metacharacterandusethepattern"[09]+"asthe"+"isaspecial
characterwhenusing"extendedregularexpressions."Extendedregular
expressionshavemorepower,butsedscriptsthattreated"+"asanormal
characterwouldbreak.Thereforeyoumustexplicitlyenablethisextension
withacommandlineoption.
GNUsedturnsthisfeatureonifyouusethe"r"commandlineoption.So
theabovecouldalsobewrittenusing
%echo"123abc"|sedr's/[09]+/&&/'
123123abc
MacOSXandFreeBSDuses E insteadof r .Formoreinformationon

extendedregularexpressions,see RegularExpressions andthe
descriptionofthercommandlineargument
Using\1tokeeppartofthepattern
Ihavealreadydescribedtheuseof"("")"and"1"inmytutorialon
regularexpressions. Toreview,theescapedparentheses(thatis,
parentheseswithbackslashesbeforethem)rememberasubstringofthe
charactersmatchedbytheregularexpression.Youcanusethistoexclude
partofthecharactersmatchedbytheregularexpression.The"\1"isthe
firstrememberedpattern,andthe"\2"isthesecondrememberedpattern.
Sedhasuptoninerememberedpatterns.
Ifyouwantedtokeepthefirstwordofaline,anddeletetherestofthe
line,marktheimportantpartwiththeparenthesis:
X
sed's/$[az]*$.*/\1/'
Ishouldelaborateonthis.Regularexpressionsaregreedy,andtryto
matchasmuchaspossible."[az]*"matcheszeroormorelowercase
letters,andtriestomatchasmanycharactersaspossible.The".*"
matcheszeroormorecharactersafterthefirstmatch.Sincethefirstone
grabsallofthecontiguouslowercaseletters,thesecondmatchesanything
else.Thereforeifyoutype
echoabcd123|sed's/$[az]*$.*/\1/'
Thiswilloutput"abcd"anddeletethenumbers.
Ifyouwanttoswitchtwowordsaround,youcanremembertwopatterns
andchangetheorderaround:
sed's/$[az]*$$[az]*$/\2\1/'
Notethespacebetweenthetworememberedpatterns.Thisisusedto
makesuretwowordsarefound.However,thiswilldonothingifasingle
wordisfound,oranylineswithnoletters.Youmaywanttoinsistthat
wordshaveatleastoneletterbyusing
sed's/$[az][az]*$$[az][az]*$/\2\1/'
orbyusingextendedregularexpressions(notethat'('and')'nolonger
needtohaveabackslash):
sedr's/([az]+)([az]+)/\2\1/'#UsingGNUsed
sedE's/([az]+)([az]+)/\2\1/'#UsingAppleMacOSX
The"\1"doesn'thavetobeinthereplacementstring(intherighthand
side).Itcanbeinthepatternyouaresearchingfor(inthelefthandside).
Ifyouwanttoeliminateduplicatedwords,youcantry:
sed's/$[az]*$\1/\1/'
Ifyouwanttodetectduplicatedwords,youcanuse
sedn'/$[az][az]*$\1/p'
orwithextendedregularexpressions
sedrn'/([az]+)\1/p'#GNUsed
sedEn'/([az]+)\1/p'#MacOSX
This,whenusedasafilter,willprintlineswithduplicatedwords.
Thenumericvaluecanhaveuptoninevalues:"\1"thru"\9."Ifyou
wantedtoreversethefirstthreecharactersonaline,youcanuse
sed's/^$.$$.$$.$/\3\2\1/'
SedPatternFlags
Youcanaddadditionalflagsafterthelastdelimiter.Youmighthave
noticedIuseda'p'attheendoftheprevioussubstitutecommand.Ialso
addedthe'n'option.Letmefirstcoverthe'p'andotherpatternflags.
Theseflagscanspecifywhathappenswhenamatchisfound.Letme
describethem.
/gGlobalreplacement
MostUNIXutilitiesworkonfiles,readingalineatatime.Sed,bydefault,
isthesameway.Ifyoutellittochangeaword,itwillonlychangethefirst
occurrenceofthewordonaline.Youmaywanttomakethechangeon
everywordonthelineinsteadofthefirst.Foranexample,let'splace
parenthesesaroundwordsonaline.Insteadofusingapatternlike"[AZa
z]*"whichwon'tmatchwordslike"won't,"wewilluseapattern,"[^]*,"
thatmatcheseverythingexceptaspace.Well,thiswillalsomatchanything
because"*"meanszeroormore.ThecurrentversionofSolaris'ssed(as
X
Iwrotethis)cangetunhappywithpatternslikethis,andgenerateerrors
like"Outputlinetoolong"orevenrunforever.Iconsiderthisabug,and
havereportedthistoSun.Asaworkaround,youmustavoidmatchingthe
nullstringwhenusingthe"g"flagtosed.Aworkaroundexampleis:"[^]
[^]*."Thefollowingwillputparenthesisaroundthefirstword:
sed's/[^]*/(&)/'<old>new
Ifyouwantittomakechangesforeveryword,adda"g"afterthelast
delimiterandusetheworkaround:
sed's/[^][^]*/(&)/g'<old>new
Issedrecursive?
Sedonlyoperatesonpatternsfoundintheincomingdata.Thatis,the
inputlineisread,andwhenapatternismatched,themodifiedoutputis
generated,andtherestoftheinputlineisscanned.The"s"commandwill
notscanthenewlycreatedoutput.Thatis,youdon'thavetoworryabout
expressionslike:
sed's/loop/looptheloop/g'<old>new
Thiswillnotcauseaninfiniteloop.Ifasecond"s"commandisexecuted,it
couldmodifytheresultsofapreviouscommand.Iwillshowyouhowto
executemultiplecommandslater.
/1,/2,etc.Specifyingwhichoccurrence
Withnoflags,thefirstmatchedsubstitutionischanged.Withthe"g"
option,allmatchesarechanged.Ifyouwanttomodifyaparticularpattern
thatisnotthefirstoneontheline,youcoulduse"$"and"$"tomark
eachpattern,anduse"\1"toputthefirstpatternbackunchanged.This
nextexamplekeepsthefirstwordonthelinebutdeletesthesecond:
sed's/$[azAZ]*$$[azAZ]*$/\1/'<old>new
Yuck.Thereisaneasierwaytodothis.Youcanaddanumberafterthe
substitutioncommandtoindicateyouonlywanttomatchthatparticular
pattern.Example:
sed's/[azAZ]*//2'<old>new
Youcancombineanumberwiththeg(global)flag.Forinstance,ifyou
wanttoleavethefirstwordalone,butchangethesecond,third,etc.tobe
DELETEDinstead,use/2g:
sed's/[azAZ]*/DELETED/2g'<old>new
I'veheardthatcombiningthenumberwiththegcommanddoesnotwork
onTheMacOS,andperhapstheFreeSBDversionofsedaswell.
Don'tget/2and\2confused.The/2isusedattheend.\2isusedin
insidethereplacementfield.
Notethespaceafterthe"*"character.Withoutthespace,sedwillruna
long,longtime.(Note:thisbugisprobablyfixedbynow.)Thisisbecause
thenumberflagandthe"g"flaghavethesamebug.Youshouldalsobe
abletousethepattern
sed's/[^]*//2'<old>new
butthisalsoeatsCPU.Ifthisworksonyourcomputer,anditdoeson
someUNIXsystems,youcouldremovetheencryptedpasswordfromthe
passwordfile:
sed's/[^:]*//2'</etc/passwd>/etc/password.new
Butthisdidn'tworkformethetimeIwrotethis.Using"[^:][^:]*"asa
workarounddoesn'thelpbecauseitwon'tmatchannonexistent
X
password,andinsteaddeletethethirdfield,whichistheuserID!Instead
youhavetousetheuglyparenthesis:
sed's/^$[^:]*$:[^:]:/\1::/'</etc/passwd>/etc/password.new
Youcouldalsoaddacharactertothefirstpatternsothatitnolonger
matchesthenullpattern:
sed's/[^:]*:/:/2'</etc/passwd>/etc/password.new
Thenumberflagisnotrestrictedtoasingledigit.Itcanbeanynumber
from1to512.Ifyouwantedtoaddacolonafterthe80thcharacterin
eachline,youcouldtype:
sed's/./&:/80'<file>new
Youcanalsodoitthehardwaybyusing80dots:
sed's/^................................................................................/&:/'<file>
/pprint
Bydefault,sedprintseveryline.Ifitmakesasubstitution,thenewtextis
printedinsteadoftheoldone.Ifyouuseanoptionalargumenttosed,
"sedn,"itwillnot,bydefault,printanynewlines.I'llcoverthisandother
optionslater.Whenthe"n"optionisused,the"p"flagwillcausethe
modifiedlinetobeprinted.Hereisonewaytoduplicatethefunctionof
grepwithsed:
sedn's/pattern/&/p'<file
Butasimplerversionisdescribed later
Writetoafilewith/wfilename
Thereisonemoreflagthatcanfollowthethirddelimiter.Withit,youcan
specifyafilethatwillreceivethemodifieddata.Anexampleisthe
following,whichwillwritealllinesthatstartwithanevennumber,followed
byaspace,tothefileeven:
sedn's/^[09]*[02468]/&/weven'<file
Inthisexample,theoutputfileisn'tneeded,astheinputwasnot
modified.Youmusthaveexactlyonespacebetweenthewandthe
filename.Youcanalsohavetenfilesopenwithoneinstanceofsed.This
allowsyoutosplitupastreamofdataintoseparatefiles.Usingthe
previousexamplecombinedwithmultiplesubstitutioncommands
describedlater,youcouldsplitafileintotenpiecesdependingonthelast
digitofthefirstnumber.Youcouldalsousethismethodtologerroror
debugginginformationtoaspecialfile.
/IIgnoreCase
GNUhasaddedanotherpatternflags/I
Thisflagmakesthepatternmatchcaseinsensitive.Thiswillmatchabc,
aBc,ABC,AbC,etc.:
sedn'/abc/Ip'<old>new
Notethataspaceafterthe'/I'andthe'p'(print)commandemphasizes
thatthe'p'isnotamodifierofthepatternmatchingprocess,,buta
commandtoexecuteafterthepatternmatching.
Combiningsubstitutionflags
X
Youcancombineflagswhenitmakessense.Pleasenotethatthe"w"has
tobethelastflag.Forexamplethefollowingcommandworks:
sedn's/a/A/2pw/tmp/file'<old>new
NextIwilldiscusstheoptionstosed,anddifferentwaystoinvokesed.
Argumentsandinvocationofsed
previously,Ihaveonlyusedonesubstitutecommand.Ifyouneedtomake
twochanges,andyoudidn'twanttoreadthemanual,youcouldpipe
togethermultiplesedcommands:
sed's/BEGIN/begin/'<old|sed's/END/end/'>new
Thisusedtwoprocessesinsteadofone.Asedguruneverusestwo
processeswhenonecando.
Multiplecommandswithecommand
Onemethodofcombiningmultiplecommandsistouseaebeforeeach
command:
sede's/a/A/'e's/b/B/'<old>new
A"e"isn'tneededintheearlierexamplesbecausesedknowsthatthere
mustalwaysbeonecommand.Ifyougivesedoneargument,itmustbea
command,andsedwilleditthedatareadfromstandardinput.
Thelongargumentversionis
sedexpression='s/a/A/'expression='s/b/B/'<old>new
Alsosee QuotingmultiplesedlinesintheBourneshell
Filenamesonthecommandline
Youcanspecifyfilesonthecommandlineifyouwish.Ifthereismorethan
oneargumenttosedthatdoesnotstartwithanoption,itmustbea
filename.Thisnextexamplewillcountthenumberoflinesinthreefiles
thatdon'tbeginwitha"#:"
sed's/^#.*//'f1f2f3|grepv'^$'|wcl
Let'sbreakthisdownintopieces.Thesedsubstitutecommandchanges
everylinethatstartswitha"#"intoablankline.Grepwasusedtofilter
out(delete)emptylines.Wccountsthenumberoflinesleft.Sedhasmore
commandsthatmakegrepunnecessary.Andgrepccanreplacewcl.I'll
discusshowyoucanduplicatesomeofgrep'sfunctionalitylater.
Ofcourseyoucouldwritethelastexampleusingthe"e"option:
sede's/^#.*//'f1f2f3|grepv'^$'|wcl
Therearetwootheroptionstosed.
sedn:noprinting
The"n"optionwillnotprintanythingunlessanexplicitrequesttoprintis
found.Imentionedthe"/p"flagtothesubstitutecommandasonewayto
turnprintingbackon.Letmeclarifythis.Thecommand
sed's/PATTERN/&/p'file
actslikethecatprogramifPATTERNisnotinthefile:e.g.nothingis
changed.IfPATTERNisinthefile,theneachlinethathasthisisprinted
twice.Addthe"n"optionandtheexampleactslikegrep:
X
sedn's/PATTERN/&/p'file
Nothingisprinted,exceptthoselineswithPATTERNincluded.
Thelongargumentofthencommandiseither
sedquiet's/PATTERN/&/p'file
or
sedsilent's/PATTERN/&/p'file
Using'sed/pattern/' 10/47
Using'sed/pattern/'
Sedhastheabilitytospecifywhichlinesaretobeexaminedand/or
modified,byspecifying addresses beforethecommand.Iwilljust
describethesimplestversionfornowthe/PATTERN/address.When
used,onlylinesthatmatchthepatternaregiventhecommandafterthe
address.Briefly,whenusedwiththe/pflag,matchinglinesareprinted
twice:
sed'/PATTERN/p'file
AndofcoursePATTERNisanyregularexpression.
Pleasenotethatifyoudonotincludeacommand,suchasthe"p"for
print,youwillgetanerror.WhenItype
echoabc|sed'/a/'
Igettheerror
sed:eexpression#1,char3:missingcommand
Also,youdon'tneedto,butIrecommendthatyouplaceaspaceafterthe
patternandthecommand.Thiswillhelpyoudistinquishbetweenflagsthat
modifythepatternmatching,andcommandstoexecuteafterthepattern
ismatched.ThereforeIrecommendthisstyle:
sed'/PATTERN/p'file
Using'sedn/pattern/p'toduplicatethe
functionofgrep
Ifyouwanttoduplicatethefunctionalityofgrep,combinethen(noprint)
optionwiththe/pprintflag:
sedn'/PATTERN/p'file
sedfscriptname
Ifyouhavealargenumberofsedcommands,youcanputthemintoafile
anduse
sedfsedscript<old>new
wheresedscriptcouldlooklikethis:
#sedcommentThisscriptchangeslowercasevowelstouppercase
s/a/A/g
s/e/E/g
s/i/I/g
s/o/O/g
s/u/U/g
Whenthereareseveralcommandsinonefile,eachcommandmustbeon
aseparateline.
Thelongargumentversionis
sedfile=sedscript<old>new
X
Alsosee hereonwritingascriptthatexecutesseddirectly
sedinshellscripts
Ifyouhavemanycommandsandtheywon'tfitneatlyononeline,youcan
breakupthelineusingabackslash:
sede's/a/A/g'\
e's/e/E/g'\
e's/i/I/g'\
e's/o/O/g'\
e's/u/U/g'<old>new
QuotingmultiplesedlinesintheCshell
Youcanhavealarge,multilinesedscriptintheCshell,butyoumusttell
theCshellthatthequoteiscontinuedacrossseverallines.Thisisdoneby
placingabackslashattheendofeachline:
#!/bin/cshf
sed's/a/A/g\
s/e/E/g\
s/i/I/g\
s/o/O/g\
s/u/U/g'<old>new
QuotingmultiplesedlinesintheBourneshell
TheBourneshellmakesthiseasierasaquotecancoverseverallines:
#!/bin/sh
sed'
s/a/A/g
s/e/E/g
s/i/I/g
s/o/O/g
s/u/U/g'<old>new
sedV
TheVoptionwillprinttheversionofsedyouareusing.Thelong
argumentofthecommandis
sedversion
sedh
Thehoptionwillprintasummaryofthesedcommands.Thelong
sedhelp
Asedinterpreterscript
Anotherwayofexecutingsedistouseaninterpreterscript.Createafile
thatcontains:

#!/bin/sedf
s/a/A/g
s/e/E/g
s/i/I/g
s/o/O/g
s/u/U/g

Clickheretogetfile: CapVowel.sed X
Ifthisscriptwasstoredinafilewiththename"CapVowel"andwas
executable,youcoulduseitwiththesimplecommand:
CapVowel<old>new
Comments
Sedcommentsarelineswherethefirstnonwhitecharacterisa"#."On
manysystems,sedcanhaveonlyonecomment,anditmustbethefirst
lineofthescript.OntheSun(1988whenIwrotethis),youcanhave
severalcommentlinesanywhereinthescript.ModernversionsofSed
supportthis.Ifthefirstlinecontainsexactly"#n"thenthisdoesthesame
thingasthe"n"option:turningoffprintingbydefault.Thiscouldnot
donewithasedinterpreterscript,becausethefirstlinemuststartwith
"#!/bin/sedf"asIthink"#!/bin/sednf"generatedanerror.Itworked
whenIfirstwrotethis(2008).Notethat"#!/bin/sedfn"doesnotwork
becausesedthinksthefilenameofthescriptis"n".However,
"#!/bin/sednf"
doeswork.
Passingargumentsintoasedscript
Passingawordintoashellscriptthatcallssediseasyifyouremembered
mytutorialontheUNIXquotingmechanism. Toreview,youusethe
singlequotestoturnquotingonandoff.Asimpleshellscriptthatusessed
toemulategrepis:

#!/bin/sh
sedn's/'$1'/&/p'

Howeverthereisaproblemwiththisscript.Ifyouhaveaspaceasan
argument,thescriptwouldcauseasyntaxerrorAbetterversionwould
protectfromthishappening:

#!/bin/sh
sedn's/'"$1"'/&/p'

Clickheretogetfile: sedgrep.sed
Ifthiswasstoredinafilecalledsedgrep,youcouldtype
sedgrep'[AZ][AZ]'<file
Thiswouldallowsedtoactasthegrepcommand.
Usingsedinashellhereisdocument
Youcanusesedtoprompttheuserforsomeparametersandthencreate
afilewiththoseparametersfilledin.Youcouldcreateafilewithdummy
valuesplacedinsideit,andusesedtochangethosedummyvalues.A
simplerwayistousethe"hereis"document,whichusespartoftheshell
scriptasifitwerestandardinput:
#!/bin/sh
echon'whatisthevalue?'
readvalue
sed's/XYZ/'$value'/'<<EOF
ThevalueisXYZ
EOF
Whenexecuted,thescriptsays:
whatisthevalue?
Ifyoutypein"123,"thenextlinewillbe:
X
Thevalueis123
Iadmitthisisacontrivedexample."Hereis"documentscanhavevalues
evaluatedwithouttheuseofsed.Thisexampledoesthesamething:
#!/bin/sh
readvalue
cat<<EOF
Thevalueis$value
EOF
However,combining"hereis"documentswithsedcanbeusefulforsome
complexcases.
Notethat

sed's/XYZ/'$value'/'<<EOF

willgiveasyntaxerroriftheusertypesananswerthatcontainsaspace,
like"abc".Betterformwouldbetoputdoublequotesaroundthe
evaluationofthevalue:
#!/bin/sh
readvalue
sed's/XYZ/'"$value"'/'<<EOF
ThevalueisXYZ
EOF
Icoveredthisinmy tutorialonquotationmarks .
Clickheretogetfile: sed_hereis.sed
Multiplecommandsandorderofexecution
Asweexploremoreofthecommandsofsed,thecommandswillbecome
complex,andtheactualsequencecanbeconfusing.It'sreallyquite
simple.Eachlineisreadin.Eachcommand,inorderspecifiedbytheuser,
hasachancetooperateontheinputline.Afterthesubstitutionsaremade,
thenextcommandhasachancetooperateonthesameline,whichmay
havebeenmodifiedbyearliercommands.Ifyoueverhaveaquestion,the
bestwaytolearnwhatwillhappenistocreateasmallexample.Ifa
complexcommanddoesn'twork,makeitsimpler.Ifyouarehaving
problemsgettingacomplexscriptworking,breakitupintotwosmaller
scriptsandpipethetwoscriptstogether.
AddressesandRangesofText
Youhaveonlylearnedonecommand,andyoucanseehowpowerfulsed
is.However,allitisdoingisagrepandsubstitute.Thatis,thesubstitute
commandistreatingeachlinebyitself,withoutcaringaboutnearbylines.
Whatwouldbeusefulistheabilitytorestricttheoperationtocertainlines.
Someusefulrestrictionsmightbe:
Specifyingalinebyitsnumber.
Specifyingarangeoflinesbynumber.
Alllinescontainingapattern.
Alllinesfromthebeginningofafiletoaregularexpression
Alllinesfromaregularexpressiontotheendofthefile.
Alllinesbetweentworegularexpressions.
Sedcandoallthatandmore.Everycommandinsedcanbeproceededby
anaddress,rangeorrestrictionliketheaboveexamples.Therestrictionor
addressimmediatelyprecedesthecommand:
restrictioncommand
Restrictingtoalinenumber
X
Thesimplestrestrictionisalinenumber.Ifyouwantedtodeletethefirst
numberonline3,justadda"3"beforethecommand:
sed'3s/[09][09]*//'<file>new
Patterns
ManyUNIXutilitieslikeviandmoreuseaslashtosearchforaregular
expression.Sedusesthesameconvention,providedyouterminatethe
expressionwithaslash.Todeletethefirstnumberonalllinesthatstart
witha"#,"use:
sed'/^#/s/[09][09]*//'
Iplacedaspaceafterthe"/expression/"soitiseasiertoread.Itisn't
necessary,butwithoutitthecommandishardertofathom.Seddoes
provideafewextraoptionswhenspecifyingregularexpressions.ButI'll
discussthoselater.Iftheexpressionstartswithabackslash,thenext
characteristhedelimiter.Touseacommainsteadofaslash,use:
sed'\,^#,s/[09][09]*//'
Themainadvantageofthisfeatureissearchingforslashes.Supposeyou
wantedtosearchforthestring"/usr/local/bin"andyouwantedtochange
itfor"/common/all/bin."Youcouldusethebackslashtoescapetheslash:
sed'/\/usr\/local\/bin/s/\/usr\/local/\/common\/all/'
Itwouldbeeasiertofollowifyouusedanunderlineinsteadofaslashasa
search.Thisexampleusestheunderlineinboththesearchcommandand
thesubstitutecommand:
sed'\_/usr/local/bin_s_/usr/local_/common/all_'
Thisillustrateswhysedscriptsgetthereputationforobscurity.Icouldbe
perverseandshowyoutheexamplethatwillsearchforalllinesthatstart
witha"g,"andchangeeach"g"onthatlinetoan"s:"
sed'/^g/s/g/s/g'
Addingaspaceandusinganunderscoreafterthesubstitutecommand
makesthismucheasiertoread:
sed'/^g/s_g_s_g'
Er,Itakethatback.It'shopeless.Thereisalessonhere:Usecomments
liberallyinasedscript.Youmayhavetoremovethecommentstorunthe
scriptunderadifferent(older)operatingsystem,butyounowknowhow
towriteasedscripttodothatveryeasily!CommentsareaGoodThing.
Youmayhaveunderstoodthescriptperfectlywhenyouwroteit.Butsix
monthsfromnowitcouldlooklikemodemnoise.Andifyoudon't
understandthatreference,imaginean8montholdchildtypingona
computer.
Rangesbylinenumber
Youcanspecifyarangeonlinenumbersbyinsertingacommabetween
thenumbers.Torestrictasubstitutiontothefirst100lines,youcanuse:
sed'1,100s/A/a/'
Ifyouknowexactlyhowmanylinesareinafile,youcanexplicitlystate
thatnumbertoperformthesubstitutionontherestofthefile.Inthiscase,
assumeyouusedwctofindoutthereare532linesinthefile:
sed'101,532s/A/a/'
Aneasierwayistousethespecialcharacter"$,"whichmeansthelastline
X
inthefile.
sed'101,$s/A/a/'
The"$"isoneofthoseconventionsthatmean"last"inutilitieslikecate,
vi,anded."cate"Linenumbersarecumulativeifseveralfilesareedited.
Thatis,
sed'200,300s/A/a/'f1f2f3>new
isthesameas
catf1f2f3|sed'200,300s/A/a/'>new
Rangesbypatterns
Youcanspecifytworegularexpressionsastherange.Assuminga"#"
startsacomment,youcansearchforakeyword,removeallcomments
untilyouseethesecondkeyword.Inthiscasethetwokeywordsare
"start"and"stop:"
sed'/start/,/stop/s/#.*//'
Thefirstpatternturnsonaflagthattellssedtoperformthesubstitute
commandoneveryline.Thesecondpatternturnsofftheflag.Ifthe"start"
and"stop"patternoccurstwice,thesubstitutionisdonebothtimes.Ifthe
"stop"patternismissing,theflagisneverturnedoff,andthesubstitution
willbeperformedoneverylineuntiltheendofthefile.
Youshouldknowthatifthe"start"patternisfound,thesubstitutionoccurs
onthesamelinethatcontains"start."Thisturnsonaswitch,whichisline
oriented.Thatis,thenextlineisreadandthesubstitutecommandis
checked.Ifitcontains"stop"theswitchisturnedoff.Switchesareline
oriented,andnotwordoriented.
Youcancombinelinenumbersandregularexpressions.Thisexamplewill
removecommentsfromthebeginningofthefileuntilitfindsthekeyword
"start:"
sede'1,/start/s/#.*//'
Thisexamplewillremovecommentseverywhereexceptthelinesbetween
thetwokeywords:
sede'1,/start/s/#.*//'e'/stop/,$s/#.*//'
Thelastexamplehasarangethatoverlapsthe"/start/,/stop/"range,as
bothrangesoperateonthelinesthatcontainthekeywords.Iwillshow
youlaterhowtorestrictacommandupto,butnotincludingtheline
containingthespecifiedpattern.Itisin Operatinginapatternrange
exceptforthepatterns ButIhavetocoversomemorebasicprinciples.
BeforeIstartdiscussingthevariouscommands,Ishouldexplainthat
somecommandscannotoperateonarangeoflines.Iwillletyouknow
whenImentionthecommands.InthisnextsectionIwilldescribethree
commands,oneofwhichcannotoperateonarange.
Deletewithd
Usingrangescanbeconfusing,soyoushouldexpecttodosome
experimentationwhenyouaretryingoutanewscript.Ausefulcommand
deleteseverylinethatmatchestherestriction:"d."Ifyouwanttolookat
thefirst10linesofafile,youcanuse:
sed'11,$d'<file
whichissimilarinfunctiontotheheadcommand.Ifyouwanttochopoff
theheaderofamailmessage,whichiseverythinguptothefirstblank
line,use:
sed'1,/^$/d'<file X
Youcanduplicatethefunctionofthetailcommand,assumingyouknow
thelengthofafile.Wccancountthelines,andexprcansubtract10from
thenumberoflines.ABourneshellscripttolookatthelast10linesofa
filemightlooklikethis:

#!/bin/sh
#printlast10linesoffile
#Firstargumentisthefilename
lines=`wcl$1|awk'{print$1}'`
start=èxpr$lines10`
start=èxpr$lines10`
sed"1,$startd"$1

Clickheretogetfile: sed_tail.sh
Therangefordeletionscanberegularexpressionspairstomarkthebegin
andendoftheoperation.Oritcanbeasingleregularexpression.Deleting
alllinesthatstartwitha"#"iseasy:
sed'/^#/d'
Removingcommentsandblanklinestakestwocommands.Thefirst
removeseverycharacterfromthe"#"totheendoftheline,andthe
seconddeletesallblanklines:
sede's/#.*//'e'/^$/d'
Athirdoneshouldbeaddedtoremoveallblanksandtabsimmediately
beforetheendofline:
sede's/#.*//'e's/[Î]*$//'e'/^$/d'
Thecharacter"Î"isaCTRLIortabcharacter.Youwouldhaveto
explicitlytypeinthetab.Notetheorderofoperationsabove,whichisin
thatorderforaverygoodreason.Commentsmightstartinthemiddleofa
line,withwhitespacecharactersbeforethem.Thereforecommentsare
firstremovedfromaline,potentiallyleavingwhitespacecharactersthat
werebeforethecomment.Thesecondcommandremovesalltrailing
blanks,sothatlinesthatarenowblankareconvertedtoemptylines.The
lastcommanddeletesemptylines.Together,thethreecommandsremove
alllinescontainingonlycomments,tabsorspaces.
Thisdemonstratesthepatternspacesedusestooperateonaline.The
actualoperationsedusesis:
Copytheinputlineintothepatternspace.
Applythefirst
sedcommandonthepatternspace,iftheaddressrestrictionistrue.
Repeatwiththenextsedexpression,again
operatingonthepatternspace.
Whenthelastoperationisperformed,writeoutthepatternspace
andreadinthenextlinefromtheinputfile.
Printingwithp
Anotherusefulcommandistheprintcommand:"p."Ifsedwasn'tstarted
withan"n"option,the"p"commandwillduplicatetheinput.The
command
sed'p'
willduplicateeveryline.Ifyouwantedtodoubleeveryemptyline,use:
sed'/^$/p'
Addingthe"n"optionturnsoffprintingunlessyourequestit.Anotherway
ofduplicatinghead'sfunctionalityistoprintonlythelinesyouwant.This
exampleprintsthefirst10lines: X
sedn'1,10p'<file
Sedcanactlikegrepbycombiningtheprintoperatortofunctiononall
linesthatmatcharegularexpression:
sedn'/match/p'
whichisthesameas:
grepmatch
Reversingtherestrictionwith! 17/47
Reversingtherestrictionwith!
Sometimesyouneedtoperformanactiononeverylineexceptthosethat
matcharegularexpression,orthoseoutsideofarangeofaddresses.The
"!"character,whichoftenmeansnotinUNIXutilities,invertstheaddress
restriction.Yourememberthat
sedn'/match/p'
actslikethegrepcommand.The"v"optiontogrepprintsalllinesthat
don'tcontainthepattern.Sedcandothiswith
sedn'/match/!p'</tmp/b
Relationshipsbetweend,p,and!
Asyoumayhavenoticed,thereareoftenseveralwaystosolvethesame
problemwithsed.Thisisbecauseprintanddeleteareoppositefunctions,
anditappearsthat"!p"issimilarto"d,"while"!d"issimilarto"p."I
wantedtotestthis,soIcreateda20linefile,andtriedeverydifferent
combination.Thefollowingtable,whichshowstheresults,demonstrates
thedifference:
Relationsbetweend,p,and!
Sed Range Command Results
sedn 1,10 p Printfirst10lines
sedn 11,$ !p Printfirst10lines
sed 1,10 !d Printfirst10lines
sed 11,$ d Printfirst10lines
sedn 1,10 !p Printlast10lines
sedn 11,$ p Printlast10lines
sed 1,10 d Printlast10lines
sed 11,$ !d Printlast10lines
sedn 1,10 d Nothingprinted
sedn 1,10 !d Nothingprinted
sedn 11,$ d Nothingprinted
sedn 11,$ !d Nothingprinted
sed 1,10 p Printfirst10linestwice,thennext10linesonce
sed 11,$ !p Printfirst10linestwice,thenlast10linesonce
sed 1,10 !p Printfirst10linesonce,thenlast10linestwice
sed 11,$ p Printfirst10linesonce,thenlast10linestwice
Thistableshowsthatthefollowingcommandsareidentical:
sedn'1,10p'
sedn'11,$!p'
sed'1,10!d'
sed'11,$d'
Italsoshowsthatthe"!"command"inverts"theaddressrange,operating
ontheotherlines.
Theqorquitcommand
X
Thereisonemoresimplecommandthatcanrestrictthechangestoaset
oflines.Itisthe"q"command:quit.thethirdwaytoduplicatethehead
commandis:
sed'11q'
whichquitswhentheeleventhlineisreached.Thiscommandismost
usefulwhenyouwishtoaborttheeditingaftersomeconditionisreached.
The"q"commandistheonecommandthatdoesnottakearangeof
addresses.Obviouslythecommand
sed'1,10q'
cannotquit10times.Instead
sed'1q'
or
sed'10q'
iscorrect.
Groupingwith{and}
Thecurlybraces,"{"and"},"areusedtogroupthecommands.
Hardlyworththebuildup.Allthatproseandthesolutionisjustmatching
squiggles.Well,thereisonecomplication.Sinceeachsedcommandmust
startonitsownline,thecurlybracesandthenestedsedcommandsmust
beonseparatelines.
Previously,Ishowedyouhowtoremovecommentsstartingwitha"#."If
youwantedtorestricttheremovaltolinesbetweenspecial"begin"and
"end"keywords,youcoulduse:

#!/bin/sh
#ThisisaBourneshellscriptthatremoves#typecomments
#between'begin'and'end'words.
sedn'
/begin/,/end/{
s/#.*//
s/[Î]*$//
/^$/d
p
}
'

Clickheretogetfile: sed_begin_end.sh
Thesebracescanbenested,whichallowyoutocombineaddressranges.
Youcouldperformthesameactionasbefore,butlimitthechangetothe
first100lines:

#!/bin/sh
#ThisisaBourneshellscriptthatremoves#typecomments
#between'begin'and'end'words.
sedn'
1,100{
/begin/,/end/{
s/#.*//
s/[Î]*$//
/^$/d
p
}
}
'

Clickheretogetfile: sed_begin_end1.sh
Youcanplacea"!"beforeasetofcurlybraces.Thisinvertstheaddress,
whichremovescommentsfromalllinesexceptthosebetweenthetwo
reservedwords: X

#!/bin/sh
sed'
/begin/,/end/!{
s/#.*//
s/[Î]*$//
/^$/d
p
}
'

Clickheretogetfile: sed_begin_end2.sh
Operatinginapatternrangeexceptforthe
patterns
YoumayrememberthatImentionedyoucandoasubstituteonapattern
range,likechanging"old"to"new"betweenabegin/endpattern:
#!/bin/sh
sed'
/begin/,/end/s/old/new/
'
Anotherwaytowritethisistousethecurlybracesforgrouping:
#!/bin/sh
sed'
/begin/,/end/{
s/old/new/
}
'
Ithinkthismakesthecodeclearertounderstand,andeasiertomodify,as
youwillseebelow.
Ifyoudidnotwanttomakeanychangeswheretheword"begin"occurred,
youcouldsimpleaddanewconditiontoskipoverthatline:
#!/bin/sh
sed'
/begin/,/end/{
/begin/n#skipoverthelinethathas"begin"onit
s/old/new/
}
'
However,skippingoverthelinethathas"end"istrickier.Ifyouusethe
samemethodyouusedfor"begin"thenthesedenginewillnotseethe
"end"tostoptherangeitskipsoverthataswell.Thesolutionistodoa
substituteonalllinesthatdon'thavethe"end"byusing
#!/bin/sh
sed'
/begin/,/end/{
/begin/n#skipoverthelinethathas"begin"onit
/end/!{
s/old/new/
}
}
'
Writingafilewiththe'w'command
Youmayrememberthatthesubstitutecommandcanwritetoafile.Here
againistheexamplethatwillonlywritelinesthatstartwithaneven
number(andfollowedbyaspace):
sedn's/^[09]*[02468]/&/weven'<file
Iusedthe"&"inthereplacementpartofthesubstitutioncommandsothat
X
thelinewouldnotbechanged.Asimplerexampleistousethe"w"
command,whichhasthesamesyntaxasthe"w"flaginthesubstitute
command:
sedn'/^[09]*[02468]/weven'<file
Rememberonlyonespacemustfollowthecommand.Anythingelsewill
beconsideredpartofthefilename.The"w"commandalsohasthesame
limitationasthe"w"flag:only10filescanbeopenedinsed.
Readinginafilewiththe'r'command 20/47
Readinginafilewiththe'r'command
Thereisalsoacommandforreadingfiles.Thecommand
sed'$rend'<in>out
willappendthefile"end"attheendofthefile(address"$)."Thefollowing
willinsertafileafterthelinewiththeword"INCLUDE:"
sed'/INCLUDE/rfile'<in>out
Youcanusethecurlybracestodeletethelinehavingthe"INCLUDE"
commandonit:

#!/bin/sh
sed'/INCLUDE/{
rfile
d
}'

Clickheretogetfile: sed_include.sh
Theorderofthedeletecommand"d"andthereadfilecommand"r"is
important.Changetheorderanditwillnotwork.Therearetwosubtle
actionsthatpreventthisfromworking.Thefirstisthe"r"commandwrites
thefiletotheoutputstream.Thefileisnotinsertedintothepatternspace,
andthereforecannotbemodifiedbyanycommand.Thereforethedelete
commanddoesnotaffectthedatareadfromthefile.
Theothersubtletyisthe"d"commanddeletesthecurrentdatainthe
patternspace.Onceallofthedataisdeleted,itdoesmakesensethatno
otheractionwillbeattempted.Thereforea"d"commandexecutedina
curlybracealsoabortsallfurtheractions.Asanexample,thesubstitute
commandbelowisneverexecuted:

#!/bin/sh
#thisexampleisWRONG
sede'1{
d
s/.*//
}'

Clickheretogetfile: sed_bad_example.sh
TheearlierexampleisacrudeversionoftheCpreprocessorprogram.The
filethatisincludedhasapredeterminedname.Itwouldbeniceifsed
allowedavariable(e.g"\1")insteadofafixedfilename.Alas,seddoesn't
havethisability.Youcouldworkaroundthislimitationbycreatingsed
commandsonthefly,orbyusingshellquotestopassvariablesintothe
sedscript.Supposeyouwantedtocreateacommandthatwouldincludea
filelikecpp,butthefilenameisanargumenttothescript.Anexampleof
thisscriptis:
%include'sys/param.h'<file.c>file.c.new
Ashellscripttodothiswouldbe:
#!/bin/sh X
#watchoutfora'/'intheparameter
#usealternatesearchdelimiter
sede'\_#INCLUDE<'"$1"'>_{
r'"$1"'
d
}'
Letmeelaborate.Ifyouhadafilethatcontains
Testfirstfile
#INCLUDE<file1>
Testsecondfile
#INCLUDE<file2>
youcouldusethecommand
sed_include1.shfile1<input|sed_include1.shfile2
toincludethespecifiedfiles.

Clickheretogetfile: sed_include1.sh
SunOSandthe#CommentCommand
Aswedigdeeperintosed,commentswillmakethecommandseasierto
follow.Theolderversionsofsedonlyallowonelineasacomment,andit
mustbethefirstline.SunOS(andGNU'ssed)allowsmorethanone
comment,andthesecommentsdon'thavetobefirst.Thelastexample
couldbe:

#!/bin/sh
#watchoutfora'/'intheparameter
#usealternatesearchdelimiter
sede'\_#INCLUDE<'"$1"'>_{

#readthefile
r'"$1"'

#deleteanycharactersinthepatternspace
#andreadthenextlinein
d
}'

Clickheretogetfile: sed_include2.sh
Adding,Changing,Insertingnewlines
Sedhasthreecommandsusedtoaddnewlinestotheoutputstream.
Becauseanentirelineisadded,thenewlineisonalinebyitselfto
emphasizethis.Thereisnooption,anentirelineisused,anditmustbe
onitsownline.IfyouarefamiliarwithmanyUNIXutilities,youwould
expectsedtouseasimilarconvention:linesarecontinuedbyendingthe
previouslinewitha"\".Thesyntaxtothesecommandsisfinicky,likethe
"r"and"w"commands.
Appendalinewith'a'
The"a"commandappendsalineaftertherangeorpattern.Thisexample
willaddalineaftereverylinewith"WORD:"

#!/bin/sh
sed'
/WORD/a\
AddthislineaftereverylinewithWORD
'
X
Clickheretogetfile: sed_add_line_after_word.sh
Youcouldeliminatetwolinesintheshellscriptifyouwish:

#!/bin/sh
sed'/WORD/a\
AddthislineaftereverylinewithWORD'

Clickheretogetfile: sed_add_line_after_word1.sh
2/27/2017 Clickheretogetfile: SedAnIntroductionandTutorial
Ipreferthefirstformbecauseit'seasiertoaddanewcommandbyadding
anewlineandbecausetheintentisclearer.Theremustnotbeaspace
afterthe"\".
Insertalinewith'i'
Youcaninsertanewlinebeforethepatternwiththe"i"command:

#!/bin/sh
sed'
/WORD/i\
AddthislinebeforeeverylinewithWORD
'

Clickheretogetfile: sed_add_line_before_word.sh
Changealinewith'c'
Youcanchangethecurrentlinewithanewline.

#!/bin/sh
sed'
/WORD/c\
Replacethecurrentlinewiththeline
'

Clickheretogetfile: sed_change_line.sh
A"d"commandfollowedbya"a"commandwon'twork,asIdiscussed
earlier.The"d"commandwouldterminatethecurrentactions.Youcan
combineallthreeactionsusingcurlybraces:

#!/bin/sh
sed'
/WORD/{
i\
Addthislinebefore
a\
Addthislineafter
c\
Changethelinetothisone
}'

Clickheretogetfile: sed_insert_append_change.sh
Leadingtabsandspacesinasedscript
Sedignoresleadingtabsandspacesinallcommands.Howeverthese
whitespacecharactersmayormaynotbeignorediftheystartthetext
followinga"a,""c"or"i"command.InSunOS,both"features"are
available.TheBerkeley(andLinux)stylesedisin/usr/bin,andtheAT&T
version(SystemV)isin/usr/5bin/.
Toelaborate,the/usr/bin/sedcommandretainswhitespace,whilethe
X
/usr/5bin/sedstripsoffleadingspaces.Ifyouwanttokeepleading
spaces,andnotcareaboutwhichversionofsedyouareusing,puta"\"as
thefirstcharacteroftheline:
#!/bin/sh
sed'
a\
\ Thislinestartswithatab
'
Addingmorethanoneline 23/47
Addingmorethanoneline
Allthreecommandswillallowyoutoaddmorethanoneline.Justend
eachlinewitha"\:"
#!/bin/sh
sed'
/WORD/a\
Addthisline\
Thisline\
Andthisline
'
Addinglinesandthepatternspace
Ihavementionedthepatternspacebefore.Mostcommandsoperateon
thepatternspace,andsubsequentcommandsmayactontheresultsof
thelastmodification.Thethreepreviouscommands,likethereadfile
command,addthenewlinestotheoutputstream,bypassingthepattern
space.
Addressrangesandtheabovecommands
YoumayrememberthatearlierIwarnedyouthatsomecommandscan
takearangeoflines,andotherscannot.Tobeprecise,thecommands"a,"
"i,""r,"and"q"willnottakearangelike"1,100"or"/begin/,/end/."The
documentationstatesthatthereadcommandcantakearange,butIgot
anerrorwhenItriedthis.The"c"orchangecommandallowsthis,andit
willletyouchangeseverallinesintoone:
#!/bin/sh
sed'
/begin/,/end/c\
***DELETED***
'

Ifyouneedtodothis,youcanusethecurlybraces,asthatwillletyou
performtheoperationoneveryline:
#!/bin/sh
#addablanklineaftereveryline
sed'1,${
a\

}'
MultiLinePatterns
MostUNIXutilitiesarelineoriented.Regularexpressionsarelineoriented.
Searchingforpatternsthatcoversmorethanonelineisnotaneasytask.
(Hint:Itwillbeveryshortly.)
Sedreadsinalineoftext,performscommandswhichmaymodifytheline,
andoutputsmodificationifdesired.Themainloopofasedscriptlookslike
this:
Thenextlineisreadfromtheinputfileandplacesitinthepatternspace.Iftheendoffileisfound,
X
andifthereareadditionalfilestoread,thecurrentfileisclosed,thenextfileisopened,andthefirst
lineofthenewfileisplacedintothepatternspace.
Thelinecountisincrementedbyone.Openinganewfiledoesnotresetthisnumber.
Eachsedcommandisexamined.Ifthereisarestrictionplacedonthecommand,andthecurrentline
inthepatternspacemeetsthatrestriction,thecommandisexecuted.Somecommands,like"n"or
"d"causesedtogotothetopoftheloop.The"q"commandcausessedtostop.Otherwisethenext
commandisexamined.
Afterallofthecommandsareexamined,thepatternspaceisoutputunlesssedhastheoptional"n"
argument.
Therestrictionbeforethecommanddeterminesifthecommandis
executed.Iftherestrictionisapattern,andtheoperationisthedelete
command,thenthefollowingwilldeletealllinesthathavethepattern:
/PATTERN/d
Iftherestrictionisapairofnumbers,thenthedeletionwillhappenifthe
linenumberisequaltothefirstnumberorgreaterthanthefirstnumber
andlessthanorequaltothelastnumber:
10,20d
Iftherestrictionisapairofpatterns,thereisavariablethatiskeptfor
eachofthesepairs.Ifthevariableisfalseandthefirstpatternisfound,
thevariableismadetrue.Ifthevariableistrue,thecommandisexecuted.
Ifthevariableistrue,andthelastpatternisontheline,afterthe
commandisexecutedthevariableisturnedoff:
/begin/,/end/d
Whew!Thatwasamouthful.Ifyouhavereadcarefullyuptohere,you
shouldhavebreezedthroughthis.Youmaywanttoreferback,becauseI
coveredseveralsubtlepoints.Mychoiceofwordswasdeliberate.Itcovers
someunusualcases,like:
#whathappensifthesecondnumber
#islessthanthefirstnumber?
sedn'20,1p'file
and
#generatea10linefilewithlinenumbers
#andseewhathappenswhentwopatternsoverlap
yes|head10|catn|\
sedne'/1/,/7/p'e'/5/,/9/p'
Enoughmentalpunishment.Hereisanotherreview,thistimeinatable
format.Assumetheinputfilecontainsthefollowinglines:
AB
CD
EF
GH
IJ
Whensedstartsup,thefirstlineisplacedinthepatternspace.Thenext
lineis"CD."Theoperationsofthe"n,""d,"and"p"commandscanbe
summarizedas:
PatternSpace NextInput Command Output NewPatternSpace NewTextInput

AB CD n <default> CD EF
AB CD d CD EF
AB CD p AB CD EF
The"n"commandmayormaynotgenerateoutputdependinguponthe
existenceofthe"n"flag.
Thatreviewisalittleeasiertofollow,isn'tit?BeforeIjumpintomultiline
patterns,Iwantedtocoverthreemorecommands:
Printlinenumberwith=
X
The"="commandprintsthecurrentlinenumbertostandardoutput.One
waytofindoutthelinenumbersthatcontainapatternistouse:
#addlinenumbersfirst,
#thenusegrep,
#thenjustprintthenumber
catnfile|grep'PATTERN'|awk'{print$1}'
Thesedsolutionis:
sedn'/PATTERN/='file
EarlierIusedthefollowingtofindthenumberoflinesinafile
#!/bin/sh
lines=`wclfile|awk'{print$1}'`
Usingthe"="commandcansimplifythis:
#!/bin/sh
lines=`sedn'$='file`
The"="commandonlyacceptsoneaddress,soifyouwanttoprintthe
numberforarangeoflines,youmustusethecurlybraces:
#!/bin/sh
#Justprintthelinenumbers
sedn'/begin/,/end/{
=
d
}'file

Sincethe"="commandonlyprintstostandardoutput,youcannotprint
thelinenumberonthesamelineasthepattern.Youneedtoeditmulti
linepatternstodothis.
Transformwithy
Ifyouwantedtochangeawordfromlowercasetouppercase,youcould
write26charactersubstitutions,converting"a"to"A,"etc.Sedhasa
commandthatoperateslikethetrprogram.Itiscalledthe"y"command.
Forinstance,tochangetheletters"a"through"f"intotheiruppercase
form,use:
sed'y/abcdef/ABCDEF/'file
Here'sasedexamplethatconversalluppercaseletterstolowercase
letters,likethetrcommand:
sed'y/ABCDEFGHIJKLMNOPQRSTUVWXYZ/abcdefghijklmnopqrstuvwxyz/'<uppercase>lowercase
Ifyouwantedtoconvertalinethatcontainedahexadecimalnumber(e.g.
0x1aff)touppercase(0x1AFF),youcoulduse:
sed'/0x[09azAZ]*/y/abcdef/ABCDEF'file
Thisworksfineifthereareonlynumbersinthefile.Ifyouwantedto
changethesecondwordinalinetouppercase,andyouareusingclassic
sed,youareoutofluckunlessyouusemultilineediting.(HeyIthink
thereissomesortofthemehere!)
However,GNUsedhasauppercaseandlowercaseextension.
Displayingcontrolcharacterswithal
The"l"commandprintsthecurrentpatternspace.Itisthereforeusefulin
debuggingsedscripts.Italsoconvertsunprintablecharactersintoprinting
charactersbyoutputtingthevalueinoctalprecededbya"\"character.I
founditusefultoprintoutthecurrentpatternspace,whileprobingthe
subtletiesofsed.
X
WorkingwithMultipleLines
Therearethreenewcommandsusedinmultiplelinepatterns:"N,""D,"
and"P."Iwillexplaintheirrelationtothematching"n,""d,"and"p"
singlelinecommands.
The"n"commandwillprintoutthecurrentpatternspace(unlessthe"n"
flagisused),emptythecurrentpatternspace,andreadinthenextlineof
input.The"N"commanddoesnotprintoutthecurrentpatternspaceand
doesnotemptythepatternspace.Itreadsinthenextline,butappendsa
newlinecharacteralongwiththeinputlineitselftothepatternspace.
The"d"commanddeletesthecurrentpatternspace,readsinthenextline,
putsthenewlineintothepatternspace,andabortsthecurrentcommand,
andstartsexecutionatthefirstsedcommand.Thisiscalledstartinganew
"cycle."The"D"commanddeletesthefirstportionofthepatternspace,up
tothenewlinecharacter,leavingtherestofthepatternalone.Like"d,"it
stopsthecurrentcommandandstartsthecommandcycleoveragain.
However,itwillnotprintthecurrentpatternspace.Youmustprintit
yourself,astepearlier.Ifthe"D"commandisexecutedwithagroupof
othercommandsinacurlybrace,commandsafterthe"D"commandare
ignored.Thenextgroupofsedcommandsisexecuted,unlessthepattern
spaceisemptied.Ifthishappens,thecycleisstartedfromthetopanda
newlineisread.
The"p"commandprintstheentirepatternspace.The"P"commandonly
printsthefirstpartofthepatternspace,uptotheNEWLINEcharacter.
Neitherthe"p"northe"P"commandchangesthepatternsspace.
Someexamplesmightdemonstrate"N"byitselfisn'tveryuseful.thefilter
sede'N'
doesn'tmodifytheinputstream.Instead,itcombinesthefirstandsecond
line,thenprintsthem,combinesthethirdandfourthline,andprintsthem,
etc.Itdoesallowyoutouseanew"anchor"character:"\n."Thismatches
thenewlinecharacterthatseparatesmultiplelinesinthepatternspace.If
youwantedtosearchforalinethatendedwiththecharacter"#,"and
appendthenextlinetoit,youcoulduse
#!/bin/sh
sed'
#lookfora"#"attheendoftheline
/#$/{
#Foundonenowreadinthenextline
N
#deletethe"#"andthenewlinecharacter,
s/#\n//
}'file
Youcouldsearchfortwolinescontaining"ONE"and"TWO"andonlyprint
outthetwoconsecutivelines:
#!/bin/sh
sedn'
/ONE/{
#found"ONE"readinnextline
N
#lookfor"TWO"onthesecondline
#andprintifthere.
/\n.*TWO/p
}'file
Thenextexamplewoulddeleteeverythingbetween"ONE"and"TWO:"
#!/bin/sh
sed'
/ONE/{
#appendaline
N
#searchforTWOonthesecondline
/\n.*TWO/{
#founditnoweditmakingoneline
s/ONE.*\n.*TWO/ONETWO/ X
}
}'file
Matchingthreelineswithsed
Youcanmatchmultiplelinesinsearches.
Hereisawaytolookforthestring"skip3",andiffound,deletethatline
andthenexttwolines.
#!/bin/sh
sed'/skip3/{
N
N
s/skip3\n.*\n.*/#3linesdeleted/
}'
Notethatitdoesn'tmatterwhatthenexttwolinesare.Ifyouwantedto
match3particularlines,it'salittlemorework.
Thisscriptlooksforthreelines,wherethefirstlinecontains"one",the
secondcontained"two"andthethirdcontains"three",andiffound,
replacethemwiththestring"1+2+3":
#!/bin/sh
sed'
/one/{
N
/two/{
N
/three/{
N
s/one\ntwo\nthree/1+2+3/
}
}
}
'
Matchingpatternsthatspanmultiplelines
Youcaneithersearchforaparticularpatternontwoconsecutivelines,or
youcansearchfortwoconsecutivewordsthatmaybesplitonaline
boundary.Thenextexamplewilllookfortwowordswhichareeitheron
thesamelineoroneisontheendofalineandthesecondisonthe
beginningofthenextline.Iffound,thefirstwordisdeleted:
#!/bin/sh
sed'
/ONE/{
#appendaline
N
#"ONETWO"onsameline
s/ONETWO/TWO/
#"ONE
#TWO"ontwoconsecutivelines
s/ONE\nTWO/TWO/
}'file
Let'susethe
"D"command,andifwefindalinecontaining
"TWO"immediatelyafteralinecontaining
"ONE,"thendeletethefirstline:

#!/bin/sh
sed'
/ONE/{
#appendaline
N
#ifTWOfound,deletethefirstline
/\n.*TWO/D
}'file
X
Clickheretogetfile: sed_delete_line_after_word.sh
Ifwewantedtoprintthefirstlineinsteadofdeletingit,andnotprint
everyotherline,changethe"D"toa"P"andadda"n"asanargumentto
sed:

#!/bin/sh
sedn'
#bydefaultdonotprintanything
/ONE/{
#appendaline
N
#ifTWOfound,printthefirstline
/\n.*TWO/P
}'file

Clickheretogetfile: sed_print_line_after_word.sh
Itisverycommontocombineallthreemultilinecommands.Thetypical
orderis"N,""P"andlastly"D."Thisonewilldeleteeverythingbetween
"ONE"and"TWO"iftheyareononeortwoconsecutivelines:

#!/bin/sh
sed'
/ONE/{
#appendthenextline
N
#lookfor"ONE"followedby"TWO"
/ONE.*TWO/{
# deleteeverythingbetween
s/ONE.*TWO/ONETWO/
# print
P
# thendeletethefirstline
D
}
}'file

Clickheretogetfile: sed_delete_between_two_words.sh
EarlierItalkedaboutthe"="command,andusingittoaddlinenumbers
toafile.Youcanusetwoinvocationsofsedtodothis(althoughitis
possibletodoitwithone,butthatmustwaituntilnextsection).Thefirst
sedcommandwilloutputalinenumberononeline,andthenprinttheline
onthenextline.Thesecondinvocationofsedwillmergethetwolines
together:

#!/bin/sh
sed'='file|\
sed'{
N
s/\n//
}'

Clickheretogetfile: sed_merge_two_lines.sh
Ifyoufinditnecessary,youcanbreakonelineintotwolines,editthem,
andmergethemtogetheragain.Asanexample,ifyouhadafilethathad
ahexadecimalnumberfollowedbyaword,andyouwantedtoconvertthe
firstwordtoalluppercase,youcanusethe"y"command,butyoumust
firstsplitthelineintotwolines,changeoneofthetwo,andmergethem
together.Thatis,alinecontaining
0x1ffftable2
willbechangedintotwolines:
0x1fff
table2
X
andthefirstlinewillbeconvertedintouppercase.Iwillusetrtoconvert
thespaceintoanewline,andthenusesedtodotherest.Thecommand
wouldbe
./sed_split<file
andsed_splitwouldbe:
#!/bin/sh
tr'''\012'|
sed'{
y/abcdef/ABCDEF/
N
s/\n//
}'

Clickheretogetfile: sed_split.sh
Itisn'tobvious,butsedcouldbeusedinsteadoftr.Youcanembedanew
lineinasubstitutecommand,butyoumustescapeitwithabackslash.It
isunfortunatethatyoumustuse"\n"intheleftsideofasubstitute
command,andanembeddednewlineintherighthandside.Heavysigh.
Hereistheexample:
#!/bin/sh
sed'
s//\
/'|\
sed'{
y/abcdef/ABCDEF/
N
s/\n//
}'

Clickheretogetfile: sed_split_merge.sh
SometimesIaddaspecialcharacterasamarker,andlookforthat
characterintheinputstream.Whenfound,itindicatestheplaceablank
usedtobe.Abackslashisagoodcharacter,exceptitmustbeescaped
withabackslash,andmakesthesedscriptobscure.Saveitforthatguy
whokeepsaskingdumbquestions.Thesedscripttochangeablankintoa
"\"followingbyanewlinewouldbe:

#!/bin/sh
sed's//\\\
/'file

Clickheretogetfile: sed_addslash_before_blank.sh
Yeah.That'stheticket.OrusetheCshellandreallyconfusehim!

#!/bin/cshf
sed'\
s//\\\\
/'file

Clickheretogetfile: sed_addslash_before_blank.csh
Afewmoreexamplesofthat,andhe'llneveraskyouaquestionagain!I
thinkI'mgettingcarriedaway.I'llsummarizewithachartthatcoversthe
featureswe'vetalkedabout:
PatternSpace NextInput Command Output NewPatternSpace NewTextInput

AB CD n <default> CD EF
AB CD N AB\nCD EF
AB CD d EF
AB CD D EF
AB CD p AB AB CD
AB CD P AB AB X CD
AB\nCD EF n <default> EF GH
AB\nCD EF N AB\nCD\nEF GH
AB\nCD EF d EF GH
AB\nCD EF D CD EF
AB\nCD EF p AB\nCD AB\nCD EF
AB\nCD EF P AB AB\nCD EF
Usingnewlinesinsedscripts
Occasionallyonewishestouseanewlinecharacterinasedscript.Well,
thishassomesubtleissueshere.Ifonewantstosearchforanewline,
onehastouse"\n."Hereisanexamplewhereyousearchforaphrase,
anddeletethenewlinecharacterafterthatphrasejoiningtwolines
together.
(echoa;echox;echoy)|sed'/x$/{
N
s:x\n:x:
}'
whichgenerates
a
xy
However,ifyouareinsertinganewline,don'tuse"\n"insteadinserta
literalnewlinecharacter:
(echoa;echox;echoy)|sed's:x:X\
:'
generates
a
X

y
TheHoldBuffer
Sofarwehavetalkedaboutthreeconceptsofsed:(1)Theinputstreamor
databeforeitismodified,(2)theoutputstreamordataafterithasbeen
modified,and(3)thepatternspace,orbuffercontainingcharactersthat
canbemodifiedandsendtotheoutputstream.
Thereisonemore"location"tobecovered:theholdbufferorholdspace.
Thinkofitasasparepatternbuffer.Itcanbeusedto"copy"or
"remember"thedatainthepatternspaceforlater.Therearefive
commandsthatusetheholdbuffer.
Exchangewithx
The"x"commandeXchangesthepatternspacewiththeholdbuffer.By
itself,thecommandisn'tuseful.Executingthesedcommand
sed'x'
asafilteraddsablanklineinthefront,anddeletesthelastline.Itlooks
likeitdidn'tchangetheinputstreamsignificantly,butthesedcommandis
modifyingeveryline.
Theholdbufferstartsoutcontainingablankline.Whenthe"x"command
modifiesthefirstline,line1issavedintheholdbuffer,andtheblankline
takestheplaceofthefirstline.Thesecond"x"commandexchangesthe
secondlinewiththeholdbuffer,whichcontainsthefirstline.Each
subsequentlineisexchangedwiththeprecedingline.Thelastlineis
placedintheholdbuffer,andisnotexchangedasecondtime,soit
remainsintheholdbufferwhentheprogramterminates,andnevergets
printed.Thisillustratesthatcaremustbetakenwhenstoringdatainthe
X
holdbuffer,becauseitwon'tbeoutputunlessyouexplicitlyrequestit.
ExampleofContextGrep
Oneuseoftheholdbufferistorememberpreviouslines.Anexampleof
thisisautilitythatactslikegrepasitshowsyouthelinesthatmatcha
pattern.Inaddition,itshowsyouthelinebeforeandafterthepattern.
Thatis,ifline8containsthepattern,thisutilitywouldprintlines7,8and
9.
Onewaytodothisistoseeifthelinehasthepattern.Ifitdoesnothave
thepattern,putthecurrentlineintheholdbuffer.Ifitdoes,printtheline
intheholdbuffer,thenthecurrentline,andthenthenextline.Aftereach
set,threedashesareprinted.Thescriptchecksfortheexistenceofan
argument,andifmissing,printsanerror.Passingtheargumentintothe
sedscriptisdonebyturningoffthesinglequotemechanism,insertingthe
"$1"intothescript,andstartingupthesinglequoteagain:

#!/bin/sh
#grep3printsoutthreelinesaroundpattern
#ifthereisonlyoneargument,exit

case$#in
1);;
*)echo"Usage:$0pattern";exit;;
esac;
#Ihopetheargumentdoesn'tcontaina/
#ifitdoes,sedwillcomplain

#usesedntodisableprinting
#unlessweaskforit
sedn'
'/$1/'!{
#nomatchputthecurrentlineintheholdbuffer
x
#deletetheoldone,whichis
#nowinthepatternbuffer
d
}
'/$1/'{
#amatchgetlastline
x
#printit
p
#gettheoriginallineback
x
#printit
p
#getthenextline
n
#printit
p
#nowaddthreedashesasamarker
a\

#nowputthislineintotheholdbuffer
x
}'

Clickheretogetfile: grep3.sh
Youcouldusethistoshowthethreelinesaroundakeyword,i.e.:
grep3vt100</etc/termcap
HoldwithhorH
The"x"commandexchangestheholdbufferandthepatternbuffer.Both
arechanged.The"h"commandcopiesthepatternbufferintothehold
buffer.Thepatternbufferisunchanged.Anidenticalscripttotheabove
usestheholdcommands: X

#!/bin/sh
#grep3versionbanotherversionusingtheholdcommands

case$#in
1);;
esac;

#againIhopetheargumentdoesn'tcontaina/

#usesedntodisableprinting

sedn'
'/$1/'!{
#putthenonmatchinglineintheholdbuffer
h
}
'/$1/'{
#foundalinethatmatches
#appendittotheholdbuffer
H
#theholdbuffercontains2lines
#getthenextline
n
#andaddittotheholdbuffer
H
#nowprintitbacktothepatternspace
x
#andprintit.
p
#addthethreehyphensasamarker
a\

}'

Clickheretogetfile: grep3a.sh
Keepingmorethanonelineintheholdbuffer
The"H"commandallowsyoutocombineseverallinesintheholdbuffer.It
actslikethe"N"commandaslinesareappendedtothebuffer,witha"\n"
betweenthelines.Youcansaveseverallinesintheholdbuffer,andprint
themonlyifaparticularpatternisfoundlater.
Asanexample,takeafilethatusesspacesasthefirstcharacterofaline
asacontinuationcharacter.Thefiles/etc/termcap,/etc/printcap,makefile
andmailmessagesusespacesortabstoindicateacontinuingofanentry.
Ifyouwantedtoprinttheentrybeforeaword,youcouldusethisscript.I
usea"Î"toindicateanactualtabcharacter:

#!/bin/sh
#printpreviousentry
sedn'
/^[Î]/!{
#linedoesnotstartwithaspaceortab,
#doesithavethepatternweareinterestedin?
'/$1/'{
#yesitdoes.printthreedashes
i\

#getholdbuffer,savecurrentline
x
#nowprintwhatwasintheholdbuffer
p
#gettheoriginallineback
x
}
#storeitintheholdbuffer X
h
}
#whataboutlinesthatstart
#withaspaceortab?
/^[Î]/{
#appendittotheholdbuffer
H
}'

Clickheretogetfile: grep_previous.sh
Youcanalsousethe"H"toextendthecontextgrep.Inthisexample,the
programprintsoutthetwolinesbeforethepattern,insteadofasingle
line.Themethodtolimitthistotwolinesistousethe"s"commandto
keeponenewline,anddeletingextralines.Icallitgrep4:

#!/bin/sh

#grep4:printsout4linesaroundpattern

case$#in
1);;
esac;

sedn'
'/$1/'!{
#doesnotmatchaddthislinetotheholdspace
H
#bringitbackintothepatternspace
x
#Twolineswouldlooklike.*\n.*
#Threelineslooklike.*\n.*\n.*
#Deleteextralineskeeptwo
s/^.*\n$.*\n.*$$/\1/
#nowputthetwolines(atmost)into
#theholdbufferagain
x
}
'/$1/'{
#matchesappendthecurrentline
H
#getthenextline
n
#appendthatonealso
H
#bringitback,butkeepthecurrentlinein
#theholdbuffer.Thisisthelineafterthepattern,
#andwewanttoplaceitinholdincasethenextline
#hasthedesiredpattern
x
#printthe4lines
p
#addthemark
a\

}'

Clickheretogetfile: grep4.sh
Youcanmodifythistoprintanynumberoflinesaroundapattern.Asyou
cansee,youmustrememberwhatisintheholdspace,andwhatisinthe
patternspace.Thereareotherwaystowritethesameroutine.
GetwithgorG
Insteadofexchangingtheholdspacewiththepatternspace,youcancopy
theholdspacetothepatternspacewiththe"g"command.Thisdeletes
thepatternspace.Ifyouwanttoappendtothepatternspace,usethe"G"
command.Thisaddsanewlinetothepatternspace,andcopiesthehold
spaceafterthenewline.
X
Hereisanotherversionofthe"grep3"command.Itworksjustlikethe
previousone,butisimplementeddifferently.Thisillustratesthatsedhas
morethanonewaytosolvemanyproblems.Whatisimportantisyou
understandyourproblem,anddocumentyoursolution:

#!/bin/sh
#grep3versionc:use'G'insteadofH


case$#in
1);;
esac;

#againIhopetheargumentdoesn'tcontaina/

sedn'
'/$1/'!{
#putthenonmatchinglineintheholdbuffer
h
}
'/$1/'{
#foundalinethatmatches
#addthenextlinetothepatternspace
N
#exchangethepreviouslinewiththe
#2inpatternspace
x
#nowaddthetwolinesback
G
#andprintit.
p
#addthethreehyphensasamarker
a\

#removefirst2lines
s/.*\n.*\n$.*$$/\1/
#andplaceintheholdbufferfornexttime
h
}'

Clickheretogetfile: grep3c.sh
The"G"commandmakesiteasytohavetwocopiesofaline.Supposeyou
wantedtotheconvertthefirsthexadecimalnumbertouppercase,and
don'twanttousethescriptIdescribedinanearliercolumn

#!/bin/sh
#changethefirsthexnumbertouppercaseformat
#usessedtwice
#usedasafilter
#convert2uc<in>out
sed'
s//\
/'|\
sed'{
y/abcdef/ABCDEF/
N
s/\n//
}'

Clickheretogetfile: convert2uc.sh
Hereisasolutionthatdoesnotrequiretwoinvocationsofsed:

#!/bin/sh
#convert2ucversionb
#usessedonce
#usedasafilter X
#convert2uc<in>out
sed'
{
#remembertheline
h
#changethecurrentlinetouppercase
y/abcdef/ABCDEF/
#addtheoldlineback
G
#Keepthefirstwordofthefirstline,
#andsecondwordofthesecondline
#withonehumongousregularexpression
s/^$[^]*$.*\n[^]*$.*$/\1\2/
}'

Clickheretogetfile: convert2uc1.sh
CarlHenrikLundesuggestedawaytomakethissimpler.Iwasworking
toohard.

#!/bin/sh
#convert2ucversionb
#usessedonce
#usedasafilter
#convert2uc<in>out
sed'
{
#remembertheline
h
#changethecurrentlinetouppercase
y/abcdef/ABCDEF/
#addtheoldlineback
G
#Keepthefirstwordofthefirstline,
#andsecondwordofthesecondline
#withonehumongousregularexpression
s/.*//#deleteallbutthefirstandlastword
}'

Clickheretogetfile: convert2uc2.sh
Thisexampleonlyconvertstheletters"a"through"f"touppercase.This
waschosentomakethescripteasiertoprintinthesenarrowcolumns.You
caneasilymodifythescripttoconvertallletterstouppercase,orto
changethefirstletter,secondword,etc.
FlowControl
Asyoulearnaboutsedyourealizethatithasitsownprogramming
language.Itistruethatit'saveryspecializedandsimplelanguage.What
languagewouldbecompletewithoutamethodofchangingtheflow
control?Therearethreecommandssedusesforthis.Youcanspecifya
labelwithantextstringprecededbyacolon.The"b"commandbranches
tothelabel.Thelabelfollowsthecommand.Ifnolabelisthere,branchto
theendofthescript.The"t"commandisusedtotestconditions.BeforeI
discussthe"t"command,Iwillshowyouanexampleusingthe"b"
command.
Thisexampleremembersparagraphs,andifitcontainsthepattern
(specifiedbyanargument),thescriptprintsouttheentireparagraph.

#!/bin/sh
sedn'
#ifanemptyline,checktheparagraph
/^$/bpara
#elseaddittotheholdbuffer
H
#atendoffile,checkparagraph
$bpara
#nowbranchtoendofscript
b
#thisiswhereaparagraphischeckedforthepattern
:para X
#returntheentireparagraph
#intothepatternspace
x
#lookforthepattern,ifthereprint
/'$1'/p
'

Clickheretogetfile: grep_paragraph.sh
Testingwitht 36/47
Testingwitht
Youcanexecuteabranchifapatternisfound.Youmaywanttoexecutea
branchonlyifasubstitutionismade.Thecommand"tlabel"willbranchto
thelabelifthelastsubstitutecommandmodifiedthepatternspace.
Oneuseforthisisrecursivepatterns.Supposeyouwantedtoremove
whitespaceinsideparenthesis.Theseparenthesesmightbenested.That
is,youwouldwanttodeleteastringthatlookedlike"(((())))."Thesed
expressions
sed's/([Î]*)/g'
wouldonlyremovetheinnermostset.Youwouldhavetopipethedata
throughthescriptfourtimestoremoveeachsetorparenthesis.Youcould
usetheregularexpression
sed's/([Î()]*)/g'
butthatwoulddeletenonmatchingsetsofparenthesis.The"t"command
wouldsolvethis:
#!/bin/sh
sed'
:again
s/([Î]*)//
tagain
'
Anearlierversionhada'g'afterthe's'expression.Thisisnotneeded.

Clickheretogetfile: delete_nested_parens.sh
Debuggingwithl
The'l'commandwillprintthepatternspaceinanunambiguousform.Non
printingcharactersareprintedinaCstyleescapedformat.
Thiscanbeusefulwhendebuggingacomplexmultilinesedscript.
Analternatewayofaddingcomments
Thereisonewaytoaddcommentsinasedscriptifyoudon'thavea
versionthatsupportsit.Usethe"a"commandwiththelinenumberof
zero:

#!/bin/sh
sed'
/begin/{
0i\
Thisisacomment\
Itcancoverseverallines\
Itwillworkwithanyversionofsed
}'

Clickheretogetfile: sed_add_comments.sh
X
Thepoorlydocumented
Thereisonemoresedcommandthatisn'twelldocumented.Itisthe""
command.Thiscanbeusedtocombinedseveralsedcommandsonone
line.Hereisthegrep4scriptIdescribedearlier,butwithoutthecomments
orerrorcheckingandwithsemicolonsbetweencommands:

#!/bin/sh
sedn'
'/$1/'!{Hxs/^.*\n$.*\n.*$$/\1/x}
'/$1/'{HnHxpa\

}'

Clickheretogetfile: grep4a.sh
Yessireebob!Definitelycharacterbuilding.IthinkIhavemademypoint.
AsfarasIamconcerned,theonlytimethesemicolonisusefuliswhen
youwanttotypethesedscriptonthecommandline.Ifyouaregoingto
placeitinascript,formatitsoitisreadable.Ihavementionedearlierthat
manyversionsofseddonotsupportcommentsexceptonthefirstline.
Youmaywanttowriteyourscriptswithcommentsinthem,andinstall
themin"binary"formwithoutcomments.Thisshouldnotbedifficult.After
all,youhavebecomeasedgurubynow.Iwon'teventellyouhowtowrite
ascripttostripoutcomments.Thatwouldbeinsultingyourintelligence.
AlsosomeoperatingsystemsdoNOTletyouusesemicolons.Soifyou
seeascriptwithsemicolons,anditdoesnotworkonanonLinuxsystem,
replacethesemicolonwithanewlinecharacter.(Aslongasyouarenot
usingcsh/tcsh,butthat'sanothertopic.
Passingregularexpressionsasarguments
Intheearlierscripts,Imentionedthatyouwouldhaveproblemsifyou
passedanargumenttothescriptthathadaslashinit.Infact,regular
expressionmightcauseyouproblems.Ascriptlikethefollowingisasking
tobebrokensomeday:
#!/bin/sh
sed's/'"$1"'//g'
Iftheargumentcontainsanyofthesecharactersinit,youmaygeta
brokenscript:"/\.*[]^$"Forinstance,ifsomeonetypesa"/"thenthe
substitutecommandwillseefourdelimitersinsteadofthree.Youwillalso
getsyntaxerrorsifyouprovidea"]"withouta"]".Onesolutionistohave
theuserputabackslashbeforeanyofthesecharacterswhentheypassit
asanargument.However,theuserhastoknowwhichcharactersare
special.
Anothersolutionistoaddabackslashbeforeeachofthosecharactersin
thescript
#!/bin/sh
arg=ècho"$1"|sed's:[]\[\^\$\.\*\/]:\\\\&:g'`
sed's/'"$arg"'//g'

Clickheretogetfile: sed_with_regular_expressions1.sh
Ifyouweresearchingforthepattern"^../,"thescriptwouldconvertthis
into"\^\.\.\/"beforepassingittosed.
Insertingbinarycharacters
Dealingwithbinarycharacterscanbetrick,expeciallywhenwritingscripts
forpeopletoread.Icaninsertabinarycharacterusinganeditorlike
EMACSbutifIshowthebinarycharacter,theterminalmaychangeitto
showittoyou.
X
TheeasiestwayIhavefoundtodothisinascriptinaportablefashionis
tousethetr(1)command.Itunderstandsoctalnotations,anditcanbe
outputintoavariablewhichcanbeused.
Here'sascriptthatwillreplacethestring"ding"withtheASCIIbell
character:
#!/bin/sh
BELL=èchox|tr'x''\007'`
sed"s/ding/$BELL/"
PleasenotethatIuseddoublequotes.Sincespecialcharactersare
interpreted,youhavetobecarefulwhenyouusethismechanism.
GNUsedCommandLinearguments
OneoftheconventionsUNIXsystemshaveistousesinglelettersare
commandlinearguments.Thismakestypingfaster,andshorted,whichis
anadvantageifyouareinacontest.Normalpeopleoftenfindsed's
tersenesscryptic.Youcanimprovethereadabilityofsedscriptsbyusing
thelongwordequivalentoptions.Thatis,insteadoftyping
sedn20p
Youcantypethelongwordversionofthenargument
sedquiet20p
Or
sedsilent20p
Thelongformofsed'scommandlineargumentsalwayshave2hyphens
beforetheirnames.GNUsedhasthefollowinglongformcommandline
arguments:
GNUCommandLine
Arguments
Short LongForm
Form
n quiet
silent
escript
expression=SCRIPT
f
SCRIPTFILE file=SCRIPTFILE
i[SUFFIX] in
place[=SUFFIX]
lN linelength=N
posix
b binary
X
follow
symlinks
r regular
extended
s separate
u unbuffered
help
version
Let'sdefineeachofthese.
Theposixargument
TheGNUversionofsedhasmanyfeaturesthatarenotavailableinother
versions.Whenportabilityisimportant,testyourscriptwiththeposix
option.IfyouhadanexamplethatusedafeatureofGNUsed,suchasthe
'v'commandtotesttheversionnumber,suchas
#thisisasedcommandfile
v4.0.1
#printthenumberoflines
$=
Andyouexecuteditwiththecommand
sednfsedfileposix<file
thentheGNUversionofsedprogramwouldgiveyouawarningthatyoursedscriptisnotcompatible.Itwou
report:
sed:eexpression#1,char2:unknowncommand:`v'
Theversionargument
YoucandeterminewhichversionofsedyouareusingwiththeGNUsed
versioncommand.Thisiswhatitoutputsonmycomputer
#sedversion
GNUsedversion4.2.1
Copyright(C)2009FreeSoftwareFoundation,Inc.
Thisisfreesoftware;seethesourceforcopyingconditions.ThereisNO
warranty;notevenforMERCHANTABILITYorFITNESSFORAPARTICULARPURPOSE,
totheextentpermittedbylaw.

GNUsedhomepage:<http://www.gnu.org/software/sed/>.
GeneralhelpusingGNUsoftware:<http://www.gnu.org/gethelp/>.
Emailbugreportsto:<buggnuutils@gnu.org>.
Besuretoincludetheword``sed''somewhereinthe``Subject:''field.
ThehHelpargument
Thehoptionwillprintasummaryofthesedcommands.Thelong
sedhelp
Itprovidesanicesummaryofthecommandlinearguments.
ThelLineLengthArgument
X
I'vealreadydescribedthe'l'command.Thedefaultlinewidthforthe'l'
commandis70characters.Thisdefaultvaluecanbechangedbyadding
the'lN'optionandspecifyingthemaximumlinelengthasthenumber
afterthe'l'.
sednl80'l'<file
Thelongformversionofthecommandlineis
sednlinelength=80'l'<file
ThesSeparateargument 40/47
ThesSeparateargument
Normally,whenyouspecifyseveralfilesonthecommandline,sed
concatenatesthefilesintoonestream,andthenoperatesonthatsingle
stream.Ifyouhadthreefiles,eachwith100lines,thenthecommand
sedn'1,10p'file1file2file3
wouldonlyprintthefirst10linesoffilefile1.The's'commandtellsGNUsedtotreatthefilesareindepende
files,andtoprintoutthefirst10linesofeachfile,whichissimilartotheheadcommand.Here'sanother
example:Ifyouwantedtoprintthenumberoflinesofeachfile,youcoulduse'wcl'whichprintsthenumbe
lines,andthefilename,foreachfile,andattheendprintthetotalnumberoflines.Hereisasimpleshellscri
thatdoessomethingsimilar,justusingsed:
#!/bin/sh
FILES=$*
sedsn'$='$FILES#printthenumberoflinesforeachfile
sedn'$='$FILES#printthetotalnumberoflines.
The'wcl'commanddoesprintoutthefilenames,unliketheabovescript.Abetteremulationofthe'wcl'
commandwouldexecutethecommandinaloop,andprintthefilenames.Hereisamoreadvancedscripttha
doesthis,butitdoesn'tusethe's'command:
#!/bin/sh
forFin"$@"
do
NL=`sedn'$='<"$F"`&&printf"%d%s\n"$NL"$F"
done
TOTAL=`sedn'$='"$@"`
printf"%dtotal\n"$TOTAL
Theiinplaceargument
I'vealreadydescribedinEditingmultiplefilesthewayIliketodothis.For
thosewhowantasimplermethod,GNUSedallowsyoutodothiswitha
commandlineoption"i".Let'sassumethatwearegoingtomakethe
samesimplechangeaddingatabbeforeeachline.Thisisawaytodo
thisforallfilesinadirectorywiththe".txt"extensioninthecurrent
directory:
sedi's/^/\t/'*.txt
Thelongargumentnameversionis
sedinplace's/^/\t/'*.txt
Thisverisondeletestheoriginalfile.IfyouareascautiousasIam,youmayprefertospecifyanextension,
whichisusedtokeepacopyoftheoriginal:
sedi.tmp's/^/\t/'*.txt
Andthelongargumentnameversionis
sedinplace=.tmp's/^/\t/'*.txt
Inthelasttwoversions,theoriginalversionofthe"a.txt"filewouldhavethename"a.txt.tmp".Youcanthen
deletetheoriginalfilesafteryoumakesureallworkedasyouexpected.Pleaseconsiderthebackupoption,a
heedmywarning.Youcaneasilydeletethebackeduporiginalfile,aslongastheextensionisunique.
TheGNUversionofsedallowsyoutouse"i"withoutanargument.The
FreeBSD/MacOSXdoesnot.Youmustprovideanextensionforthe
FreeBSD/MacOSXversion.Ifyouwanttodoinplaceeditingwithout
creatingabackup,youcanuse
sedi'''s/^/\t/'*.txt
X
Thefollowsymlinksargument
Theinplaceeditingfeatureishandytohave.Butwhathappensifthefile
youareeditingisasymboliclinktoanotherfile?Let'sassumeyouhavea
filenamed"b"inadirectorycalled"tmp",withasymboliclinktothisfile:
$lslb
lrwxrwxrwx1barnettadm6Mar1616:03b.txt>tmp/b.txt
Ifyouexecutedtheabovecommandtodoinplaceediting,therewillbeanewfilecalled"b.txt"inthecurren
directory,and"tmp/b.txt"willbeunchanged.Nowyouhavetwoversionsofthefile,oneischanged(inthe
currentdirectory),andoneisnot(inthe"tmp"directory).Andwhereyouhadasymboliclink,ithasbeen
replacedwithamodifiedversionoftheoriginalfile.Ifyouwanttoedittherealfile,andkeepthesymboliclin
place,usethe"followsymlinks"commandlineoption:
sedifollowsymlinks's/^/\t/'*.txt
Thisfollowsthesymlinktotheoriginallocation,andmodifiesthefileinthe"tmp"directory,Ifyouspecifyan
extension,theoriginalfilewillbefoundwiththatextensioninthesamedirectoryartherealsource.Without
followsymlinkscommandlineoption,the"backup"file"b.tmp"willbeinthesamedirectorythatheldthe
symboliclink,andwillstillbeasymboliclinkjustrenamedtogiveitanewextension.
ThebBinaryargument
UnixandLinuxsystemsconsiderthenewlinecharacter"\n"tobetheend
oftheline.However,MSDOS,Windows,andCygwinsystemsendeach
linewith"\r\n"Carriagereturnandlinefeed.Ifyouareusinganyof
theseoperatingsystems,the"b"orbinary"commandlineoptionwill
treatthecarriagereturn/newlinecombinationastheendoftheline.
Otherwisethecarriagereturnistreatedasanunprintablecharacter
immediatelybeforetheendofline.Ithink.(Notetoselfverifythis).
TherExtendedRegularExpressionargument
WhenImentionpatterns,suchas"s/pattern/",thepatternisaregular
expression.Therearetwocommonclassesofregularexpressions,the
original"basic"expressions,andthe"extended"regularexpressions.For
moreonthedifferencessee Mytutorialonregularexpressions and
the thesectiononextendedregularexpressions .Becausethe
meaningofcertaincharactersaredifferentbetweentheregularand
extendedexpressions,youneedacommandlineargumenttoenablesed
tousetheextension.Toenablethisextension,usethe"r"command,as
mentionedin theexampleonfindingduplicatedwordsonaline
sedrn'/$[az]+$\1/p'
or
sedregularextendedquiet'/$[az]+$\1/p'
IalreadymentionedthatMacOSXandFreeBSDuses E insteadof r .
TheuUnbufferedargument
NormallyUnixandLinuxsystemsapplysomeintelligencetohandling
standardoutput.It'sassumedthatifyouaresendingresultstoaterminal,
youwanttheoutputassoonasitbecomesavailable.However,ifyouare
sendingtheoutputtoafile,thenit'sassumedyouwantbetter
performance,soitbufferstheoutputuntilthebufferisfull,andthenthe
contentsofthebufferiswrittentothefile.Letmeelaborateonthis.Let's
assumeforthisexampleyouhaveaverylargefile,andyouareusingsed
tosearchforastring,andtoprintitwhenitisfound:
sedn'/MATCH/p'<file
Sincetheoutputistheterminal,assoonasamatchisfound,itisprinted.However,ifsedpipesitsoutputto
anotherprogram,itwillbuffertheresults.Buttherearetimeswhenyouwantimmediateresults.Thisis
especiallytruewhenyouaredealingwithlargefiles,orfilesthatoccasionallygeneratedata.Tosummarize,y
X
havelotsofinputdata,andyouwantsedtoprocessit,andthensendthistoanotherprogramthatprocesses
theresults,butyouwanttheresultswhenithappens,andnotdelayed.Letmemakeupasimpleexample.It
contrived,butitdoesexplainhowthisworks.Here'saprogramcalledSlowTextthatprintsnumbersfrom1to
60,onceasecond:
#!/bin/sh
foriin`seq160`
do
echo$i
sleep1
done
Let'susesedtosearchforlinesthathavethecharacter'1',andhaveitsendresultstoawk,whichwillcalcula
thesquareofthatnumber.Thiswouldbetheadmittedlycontrivedscript:
SlowText|sedn'/1/p'|awk'{print$1*$1}'
Thisworks,butbecausesedisbufferingtheresults,wehavetowaituntilthebufferfillsup,oruntilthe
SlowTextprogramexists,beforewetheresults.Youcaneliminatethebuffering,andseetheresultsassoon
SlowTextoutputsthem,byusingthe"u"option.Withthisoption,youwillseethesquaresprintedassoonas
possible:
SlowText|sedun'/1/p'|awk'{print$1*$1}'
Thelongformoftheargumentis"unbuffered".
MacOSXandFreeBSDusetheargument"l".
GNUSed4.2.2andlaterwillalsobeunbufferedwhilereadingfiles,notjust
writingthem.
ThezNullDataargument
Normally,sedreadsalinebyreadingastringofcharactersuptotheend
oflinecharacter(newlineorcarriagereturn).See thebBinary
commandlineargument TheGNUversionofsedaddedafeaturein
version4.2.2tousethe"NULL"characterinstead.Thiscanbeusefulifyou
havefilesthatusetheNULLasarecordseparator.SomeGNUutilitiescan
genertaeoutputthatusesaNULLinsteadanewline,suchas"find.
print0"or"greplZ".Thisfeatureisusefulifyouareoperatingon
filenamesthatmightcontainspacesorbinarycharacters.
Forinstance,ifyouwantedtouse"find"tosearchforfilesandyouused
the"print0"optiontoprintaNULLattheendofeachfilename,youcould
usesedtodeletethedirectorypathname:
find.typefprint0|sedz's:^.*/::'|xargs0echo
Theaboveexampleisnotterriblyusefulasthe"xargs"useofechodoes
notretaintheabilitytoretainspacesaspartofthefilename.Butisdoes
showhowtousethesed"z"command.
GNUgrepalsohasaZoptiontosearchforstringsinfiles,placinga
"NULL"attheendofeachfilenameinsteadofanewline.Andwiththel
command,grepwillprintthefilenamethatcontainsthestring,retaining
nonprintingandbinarycharacters:
greplZSTRING*/*/*|sedz's:^.*/::'|xargs0echo
Thisfeatureisveryusefulwhenusershavetheabilitytocreatetheirown
filenames.
FreeBSDExtensions
AppleusestheFreeBSDversionofsedforMacOSXinsteadoftheGNU
sed.However,theFreeBSDversionhasacoupleofadditions.
TheaordelayedopenArgument
Normally,assoonassedstartsup,itopensallfilesthatarereferedtoby
the "w" command.TheFreeBSDversionofsedhasanoptiontodelay
X
thisactionuntilthe"w"commandisexecuted.
TheIinplaceargument
FreeBSDaddeda"I"optionthatissimilartothe i option.The"i"
optiontreatstheeditingeachfileasaseparateinstanceofsed.Ifthe"I"
optionisused,thenlinenumbersdonotgetresetatthebeginningofeach
line,andrangesofaddressescontinuefromonefiletothenext.Thatis,if
youusedtherange'/BEGIN/,/END/'andyouusedthe"I"option,youcan
havethe"BEGIN"inthefirstfile,and"END"inthesecondfile,andthe
commandsexecutedwithintherangewouldspanbothfiles.Ifyouused"
i",thenthecommandswouldnot.
Andlikethe i option,theextensionusedtostorethebackupfilemust
bespecified.
EorExtendedRegularExpressions
Imentionedextendedregularexpressions earlier .FreeBSD(andMac
OSX)uses"E"toenablethis.However,FreeBSDlateraddedthe r
commandtobecompatiblewithGNUsed.
Usingwordboundries
Someoneonceaskedmetohelpthemsolveatrickysedprobleminvolving
wordboundaries.Let'ssupposeyouhavethefollowinginput
/usr/bin/usr/local/bin/usr/local/usr/local/project/bin
andyouwantedtodelete'/usr/local'butleavetheother3pathsalone.
Youcouldusethesimple(andincorrect)command:
sed's@/usr/local@@'
whichwouldoutput
/usr/bin/bin/usr/local/usr/local/project/bin
Thatis,itwouldmistakenlychange'/usr/local/bin'to'/bin'andnotdelete
'/usr/local'whichwastheintentionoftheprogrammer.Thebettermethod
istoincludespacesaroundthesearch:
sed's@/usr/local@@'
However,thiswon'tworkif'/usr/local'isatthebeginning,orattheendof
theline.Italsowon'tworkif'/usr/local'istheonlypathontheline.To
handletheseedgecases,youcansimplydescribealloftheseconditionsas
separatecases:
#!/bin/sh
sed'
s@/usr/local@@g
s@^/usr/local@@
s@/usr/local$@@
s@^/usr/local$@@
'
Thisworksfineifthestringyouaresearchingforissurroundedbya
space.Butwhathappensifthestringissurroundedbyothercharacters,
whichmaybeoneofseveralpossiblecharacters?Youcanalwaysmakeup
yourownclassofcharactersthatdefinethe'endofaword'Forinstance,
ifyourstringconsistsofalphanumericcharactersandtheslash,theclass
ofcharacterscanbedefinedby'[azAZ09/]'orthemoreflexible
'[[:alnum:]/]'.Wecandefinetheclassfocharacterstobeallbutthese,by
usingthecaret,i.e.'[^[:alnum:]/]'.Andunlikethespacebefore,ifyouare
goingtousecharacterclasses,youmayhavetorememberwhatthese
charactersareandnotdeletethem.Sowecanreplacethespacewith
'[^[:alnum:]/]'andthenchangethecommandtobe
X
#!/bin/sh
sed'
s@$[^[:alnum:]/]$/usr/local$[^[:alnum:]/]$@\1\2@g
s@^/usr/local$[^[:alnum:]/]$@\1@
s@$[^[:alnum:]/]$/usr/local$@\1@
s@^/usr/local$@@
'
Thefirstversionwouldreplace'/usr/local'withasinglespace.This
methodwouldreplace':/usr/local:'with'::'becausetheredundant
deliniatorsarenotdeleted.Besuretofixthisifyouneedto.
Thismethodalwaysworks,butitisinelegantanderrorprone.Thereare
othermethods,buttheymaynotbeportable.Solaris'sversionofsed
usedthespecialcharacters\<and\>asanchorsthatindicatedaword
boundary.Soyoucoulduse
s@\</usr/local\>@@
However,theGNUversionofsedsaystheusageofthesespecial
charactersareundefined.Accordingtothemanualpage:
Regexsyntaxclashes(problemswithbackslashes)
`sed'usesthePOSIXbasicregularexpressionsyntax.Accordingto
thestandard,themeaningofsomeescapesequencesisundefinedin
thissyntax;notableinthecaseof`sed'are`\|',`\+',`\?',
`\`',`\'',`\<',`\>',`\b',`\B',`\w',and`\W'.

AsinallGNUprogramsthatusePOSIXbasicregularexpressions,
`sed'interpretstheseescapesequencesasspecialcharacters.
So,`x\+'matchesoneormoreoccurrencesof`x'.àbc\|def'
matcheseitheràbc'or`def'.
Whenindoubt,experiment.
CommandSummary
AsIpromisedearlier,hereisatablethatsummarizesthedifferent
commands.Thesecondcolumnspecifiesifthecommandcanhavearange
orpairofaddressesorasingleaddressorpattern.Thenextfourcolumns
specifieswhichofthefourbuffersorstreamsaremodifiedbythe
command.Somecommandsonlyaffecttheoutputstream,othersonly
affecttheholdbuffer.Ifyourememberthatthepatternspaceisoutput
(unlessa"n"wasgiventosed),thistableshouldhelpyoukeeptrackof
thevariouscommands.
Modificationto Modificationto Modificationto Modificationto

Command AddressorRange
InputStream OutputStream PatternSpace HoldBuffer
= Y
a Address Y
b Range
c Range Y
d Range Y Y
D Range Y Y
g Range Y
G Range Y
h Range Y
H Range Y
i Address Y
l Address Y
n Range Y *
N Range Y Y
p Range Y
P Range Y
q Address
r Address Y
s Range Y
t Range
w Range Y
x Range X Y Y
y Range Y
The"n"commandmayormaynotgenerateoutput,dependingonthe"n"
option.The"r"commandcanonlyhaveoneaddress,despitethe
documentation.
Checkoutmynew SedReferenceChart
InConclusion
Thisconcludesmytutorialonsed.Itispossibletofindshorterformsof
someofmyscripts.However,Ichosetheseexamplestoillustratesome
basicconstructs.Iwantedclarity,notobscurity.Ihopeyouenjoyedit.
MoreReferences
Thisconcludesmytutorialonsed.OtherofmyUNIXshelltutorialscanbe
found here. Othershelltutorialsandreferencescanbefoundat
FreeBSDSedManPage
Apple/MacOSXSedManPage
GNUSedManual
GNUSed4.2.2ReleaseNotes
sed(1)SeventhEdditionUnix
sed(1)manualpagefromSun/Oracle
Heiner'sSHELLdorado
ChrisF.A.Johnson'sUNIXShellPage
TheWikipediaEntryonSED
SEDoneliners
Anddon'tforget TheSEDFAQ

ThisdocumentwasoriginallyconvertedfromNROFFtoTEXTtoHTML.
Pleaseforgiveerrorsinthetranslation.
Ifyouareconfused,grabtheactualscriptifpossible.Notranslations
occurredinthescripts.
Thanksforthefeedback,gang

ThankstoKeelanEvans,FredrikNilsson,andKurtMcKeeforspottingsome
typos.
ThankstoWimStolkerandJose'Sebrosaaswell.
ThankstoOlivierMengue.
ThankstoAndrewM.Goth.
ThankstoDavidP.Brown.
ThankstoAxelSchulzeforsomecorrections
ThankstoMartinJanforthecorrectionsinsedformat(grin)
ThankstoDavidWardforsomecorrections
AbigthanksforFazlRahmanforspottingdozensoferrors.
ThankstoCarlHenrikLundewhosuggestedanimprovementto
convert2uc1.sh
AbigthankstoBryanHyunHuhwhospottedanerrorinthetableand
referencechart
Thanksforinputfrom
MartenJan
GordonWilson
TomKonantz
PeterBratton
GrantRoot
KeithBriggs X
ZoltanMiklos
PeggyRussell
LorensKockkum.net
JohnPoulin
Rihards
CoreyRichardson
EricMathison
IldarMulyukov
TomZhu
AbhijeetRastogi @shadyabhi
SteveLeBlanc @sleveo
dontforgetyourtowel @whatissixbynine
Yiming
FeiWang
KennethR.Beesley
DuncanSungW.Kim @DuncanSungWKim
JuanEugenioAbadie
ZanderHill @_ZPH
RobSmith
PeterMoore

Thisdocumentwastranslatedbytroff2htmlv0.21onSeptember22,2001
andthenmanuallyeditedtomakeitcompliantwith:

Sed - An Introduction and Tutorial

Încărcat de

Informații document

Drepturi de autor

Formate disponibile

Partajați acest document

Partajați sau inserați document

Opțiuni de partajare

Vi se pare util acest document?

Este necorespunzător acest conținut?

Drepturi de autor:

Formate disponibile

Sed - An Introduction and Tutorial

Încărcat de

Drepturi de autor:

Formate disponibile

2/27/2017 SedAnIntroductionandTutorial

References n(quietsilent) GNUsed

http://www.grymoire.com/Unix/Sed.html TheHoldBuffer 2/47

We'vecovered quoting and regularexpressions. .That's90%ofthe

MacOSXandFreeBSDuses E insteadof r .Formoreinformationon

PatternSpace NextInput Command Output NewPatternSpace NewTextInput

PatternSpace NextInput Command Output NewPatternSpace NewTextInput

Modificationto Modificationto Modificationto Modificationto

S-ar putea să vă placă și