Cursul 10

Shell scripting

Utilizarea Sistemelor de Operare (USO) 3 decembrie 2012

Departamentul de Calculatoare

CSE Dep

Cursul 10, Shell scripting

1/47

Moto

Back o, or Ill replace you with a small shell script.

CSE Dep

Cursul 10, Shell scripting

2/47

Servicii de ret , ea
port client server TCP conexiune

cerere

servicii

UDP

SSH

E-mail

WWW
MIME

securizat scp sshd

adres

80 HTTP URL

SMTP IMAP 22

22 ssh

remote

mailbox POP3

text

browser page

HTML

CSE Dep

Cursul 10, Shell scripting

3/47

Servicii de ret , ea

http://jokegurus.com/2008/06/06/why-isnt-mom-running-apache/

CSE Dep

Cursul 10, Shell scripting

4/47

Cuprins

Filtre de text Programare n shell Shell scripting avansat Exemple de scripturi shell Concluzie

CSE Dep

Cursul 10, Shell scripting

5/47

Suport curs

Suport (Introducere n sisteme de operare)

Capitolul 12 Shell scripting
Sect , iunile 12.4, 12.5, 12.6, 12.9

CSE Dep

Cursul 10, Shell scripting

6/47

Remember, remember . . . cursul 5

nl ant , uirea comenzilor operatori shell, caractere speciale escaping/citare variabile shell variabile de mediu globbing expresii regulate

CSE Dep

Cursul 10, Shell scripting

7/47

Filtre de text

All is text. (Unix) comenzi care ,,ltreaz a text

Sintax a pentru ltre de text
filtru < input_file > output_file command1 | filtru | command2

operat , ii
selectare linii selectare coloane (dup a separator) prelucrare linii (modicare elemente) contabilizare

n Windows PowerShell se lucreaz a cu obiecte

CSE Dep

Cursul 10, Shell scripting

9/47

cat, tac, rev, nl

cat
as a intrarea standard la ies , eaz , irea standard Ce efect are rularea comenzii cat f ar a nici un argument?
as , area la ies , irea standard a textului de la intrarea standard se ,, nchide intrarea standard folosind CTRL+D

tac as a (prima linie este as a ultima) , are inversat , at rev as a a ec arei linii (primul caracter este , are inversat as , at ultimul) nl number lines

CSE Dep

Cursul 10, Shell scripting

10/47

head, tail

head -10, head, head -n 10

primele 10 linii

head -n -20
toate liniile mai put , in ultimele 20

tail -10, tail, tail -n -10

ultimele 10 linii

tail -n +21
toate liniile ncep and de la linia 21 (adic a mai put , in primele 20)

CSE Dep

Cursul 10, Shell scripting

11/47

cut

selectare coloane de text separatorul implicit este TAB

Sintax a cut
cut -f 1 < file cut -f 1-3 < file cut -f 1,4 -d : < /etc/passwd

CSE Dep

Cursul 10, Shell scripting

12/47

tr

transliterate transformare litere n alte litere

tr a b < file tr a-z A-Z < file

s , tergere
tr -d < file

squeeze
ifconfig | tr -s | cut -d -f 2
Generator de parole
tr -d -c A-Za-z < /dev/urandom | head -c 8

CSE Dep

Cursul 10, Shell scripting

13/47

sort, uniq

sortarea liniilor implicit alfabetic a sort -n file sortare numeric a sort -r file reverse sort sort -u file, sort file | uniq cu unicizare sort -k 3 file sorteaz a n funct , ie de coloana 3

CSE Dep

Cursul 10, Shell scripting

14/47

wc

contorizarea elementelor cu argument s as , ier as , eaz , i numele s , ierului wc -l num arul de linii wc -c num arul de caractere wc -w num arul de cuvinte

CSE Dep

Cursul 10, Shell scripting

15/47

grep

extrage linii ce cont , in un anumit pattern foloses , te expresii regulate grep pattern file grep -r pattern /path/to/folder/ c autare recursiv a grep -v pattern file invert match grep -i pattern file ignore case grep --color=auto
recomand am folosirea unui alias alias grep=grep --color=auto alias grpe=grep alias gpre=grep

CSE Dep

Cursul 10, Shell scripting

16/47

Programare n shell

variabile (vezi cursul 5) instruct , iuni de decizie instruct , iuni de ciclare funct , ii

CSE Dep

Cursul 10, Shell scripting

18/47

if

Exemple de folosire if
if grep dan /etc/passwd; then echo "OK" else echo "NOK" fi if test -f /path/to/file; then echo "File exists" fi if test $# -ne 1; then echo "Usage: $0 file" fi

CSE Dep

Cursul 10, Shell scripting

19/47

if (2)

if primes a , te ca argument o comand

dac a valoarea de retur a comenzii este 0 (succes) condit , ia este ndeplinit a dac a valoarea de retur a comenzii este diferit a de 0 (insucces) condit ndeplinit a , ia nu este

de multe ori if primes , te ca argument comanda test

test test test test test -f file -d file string1 = string2 num1 -eq num2 -z string1

CSE Dep

Cursul 10, Shell scripting

20/47

for, seq

Exemple de sintax a for

for for for for for for for i in 1 2 3 4 5 6 7 8 9 10; do ... done ((i = 1; i <= 10; i++)); do ... done i in $(seq 1 10); do ... done i in $(seq -f "%02g" 1 10); do ... done f in *; do ... done user in $(cut -d : -f 1 < /etc/passwd); do ... done arg in $@; do ... done

CSE Dep

Cursul 10, Shell scripting

21/47

while, read

Sintaxa while + read

while read a b c; do ... done < file

Exemplu while + read

IFS=:; while read uname pass uid extra; do if test $uid -ge 1000; then echo "$uname" fi done < /etc/passwd

CSE Dep

Cursul 10, Shell scripting

22/47

while, read (2)

while primes a (la fel ca if) de , te ca argument o comand obicei read read
ntoarce 0 c and a citit o linie ntoarce diferit de zero la sf ars , itul s , ierului (EOF) folosit
stand-alone (similar fgets, scanf) mpreun a cu while

IFS
input eld separator specic a argumentul care va folosit de read

CSE Dep

Cursul 10, Shell scripting

23/47

Funct n shell , ii

Exemple de funct n shell , ii

hello() { echo "hello" } ip_for_hostname() { host "$1" | head -1 | cut -d -f 4 } DEBUG() { test "$_DEBUG" -eq 1 && $@ }

CSE Dep

Cursul 10, Shell scripting

24/47

Funct n shell (2) , ii

modularizare func_name() { ...} se apeleaz a ca o comand a (func_name) poate folosi return

dac a nu, valoarea de retur a funct , iei este valoarea de retur a ultimei comenzi rulate

argumentele transmise sunt prelucrate cu ajutorul variabilelor specice scripturilor shell

$#, $1, $2, $@

CSE Dep

Cursul 10, Shell scripting

25/47

sed
Echivalent grep
sed -n /abc/p

Echivalent wc -l
sed -n $=

Echivalent head
sed 10q

Substitute
sed s/ana/bogdan sed s/ana/bogdan/g

CSE Dep

Cursul 10, Shell scripting

27/47

sed (2)

stream editor cites ari , te linie cu linie s , i face prelucr sintax a spartan a
range + comenzi $= (la sf ars a num arul liniei) , itul s , ierului, as , eaz 10q (la linia 10, quit) s/ana/bogdan/g (peste tot, substituie ana cu bogdan) 1,10s/ana/bogdan/g ( n primele 10 linii, substituie) /corina/s/ana/bogdan/g (pe liniile ce cont , in corina, substituie)

sintaxa de substitut as n vi/vim , ie este folosit ,i

CSE Dep

Cursul 10, Shell scripting

28/47

awk

Aarea adresei IP
ifconfig | awk -F [ \t]+ {print $5;}

Pachetele de tip Apache instalate n sistem

dpkg -l apache2* | grep ^ii | awk -F [ \t]+ {print $2;}

Utilizatorii cu uid-ul mai mare de 1000

< /etc/passwd awk -F : {if ($3 >= 1000) print $1;}

CSE Dep

Cursul 10, Shell scripting

29/47

awk (2)

cut++
permite specicarea unei expresii regulate ca separator

limbaj de programare sintax a similar aC refer a c ampurile prin variabile interne

$0 ntreaga linie $1, $2, . . . NR number of records (num ar de linii) NF number of elds (num ar de coloane)

CSE Dep

Cursul 10, Shell scripting

30/47

Recomand ari shell scripting

nu folositi shell scripting pentru ceea ce se face mai bine/usor/ecient in Python, C, Java, Ruby, PHP etc. modularitate (do one thing, do one thing well ) nu reinventat , i roata cleanup actions n general, nu folosit n numele variabilelor , i majuscule nu parsat , i ies , irea comenzii ls (folosit , i stat, for i in *, find cu arrays) folosit and referit , i ghilimele c , i variabile http://mywiki.wooledge.org/BashFAQ

CSE Dep

Cursul 10, Shell scripting

31/47

Upload MAC address

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 #!/bin/bash declare IFACE=eth1 declare URL="http://anaconda.cs.pub.ro/razvan/school/uso/lab/macs/putmac.php" declare MY HWADDR function getmac() { MY HWADDR=$(/sbin/ifconfig "$IFACE" | grep HW | awk -F [ \t]+ {print $5}) } function uploadmac() { wget -o /dev/null "$URL?mac=$MY HWADDR" &> /dev/null } function main() { getmac uploadmac } main exit 0

CSE Dep

Cursul 10, Shell scripting

33/47

Run torrent script

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

#!/bin/bash BTDOWNLOADHEADLESS=/usr/bin/btdownloadheadless TORRENT FILE="$1" DOWNLOAD DIR="$2" function check alive() { ps -ef | grep $BTDOWNLOADHEADLESS | grep $TORRENT FILE &> /dev/null } check alive if test $? -ne 0; then pushd . &> /dev/null cd $DOWNLOAD DIR && nohup $BTDOWNLOADHEADLESS $TORRENT FILE &> /dev/null & popd &> /dev/null exit 0

CSE Dep

Cursul 10, Shell scripting

34/47

Mass add users

1 2 3 4 5 6 7 8 9 10

#!/bin/bash IFS="," INPUT FILE=lists/cdl 2010 list.txt while read username email name surname; do ./ldap add $username $email $name $surname done < $INPUT FILE exit 0

CSE Dep

Cursul 10, Shell scripting

35/47

Migrare utilizatori

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

#!/bin/bash # passwd le: sier cu useri gen /etc/passwd passwd file=/etc/passwd while read user skel pass uid gid comm home shell; do save dir="/home/$user/old anaconda" echo "$user $comm $home $shell" useradd -m -d /home/$user -g users -s $shell -c "${comm}" $user mkdir ${save dir} chown $user:users ${save dir} bash ./rsync.sh $home ${save dir} done < passwd file

CSE Dep

Cursul 10, Shell scripting

36/47

Send event
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 #!/bin/bash # get subject if test $# -ge 1; then subject="$1" else echo -en "Subject: " read subject # get le if test $# -ge 2; then le="$2" else echo -en "File: " read le # send mail to all LDAP accounts for i in $(ldapsearch -x mail | grep mail: | cut -d " " -f 2); do mail -s "$subject" $i < $file done exit 0

CSE Dep

Cursul 10, Shell scripting

37/47

CTF CAPTCHA breaker

1 #!/bin/bash 2 3 fname="f.png" 4 captcha gen url="http://dctf.defcamp.ro/l3vels/web200/gen.php" 5 captcha submit url="http://dctf.defcamp.ro/l3vels/web200/%2527.php" 6 cookie file="cookies.sav" 7 8 echo "Download initial file..." 9 wget --cookies=on --keep-session-cookies --save-cookies="$cookie file" -o /dev/null "$captcha submit url" 10 11 echo "Updating cookies..." 12 sed -i s/display\t0/display\t1/g "$cookie file" 13 14 echo "Download CAPTCHA to $fname..." 15 wget --cookies=on --keep-session-cookies --load-cookies="$cookie file" -o /dev/null -O "$fname" "$captcha gen url" 16 17 echo "Converting PNG file to PGM file..." 18 convert $fname -compress none -despeckle -enhance -black-threshold 50% -white-threshold 33% $(basename $fname .png).pgm 19 20 echo "Breaking CAPTCHA..." 21 captchaz=$(python ocr.py f.pgm | tr -d ) 22 23 echo "Submitting CAPTCHA ($captchaz)..." 24 wget --cookies=on --keep-session-cookies --load-cookies="$cookie file" --post-data="user name=Administrator&captchaz=$captchaz&submit=Login" -o /dev/null "$captcha submit url"

CSE Dep

Cursul 10, Shell scripting

38/47

Wanna see something real scary?

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

#!/bin/sed -f { s/\[\(http[^]]\+\)]/[[\1]]/g s/<pre>/<code>/g s/<\/pre>/<\/code>/g s/<source[ \t]\+lang="\([a-z]\+\)">/<code \1>/g s/<\/source>/<\/code>/g s/<tt>/{{{/g s/<\/tt>/}}}/g s/<b>/**/g s/<\/b>/**/g s/<ul>//g s/<\/ul>//g s/[ \t]*<li>\(.*\)<\/li>/* \1/g s//**/g s//\/\//g s/{{{\[\[/[[/g s/\]\]}}}/]]/g s/\(\[\[http:[^ ]\+\) /\1|/g }

CSE Dep

Cursul 10, Shell scripting

39/47

Mastering Unix Shell Scripting

Randal K. Michael edit , ia a doua, 2008 peste 1000 de pagini acoper a mai multe shell-uri util as ncep atori, dar n special pentru avansat , i pentru ,i

CSE Dep

Cursul 10, Shell scripting

41/47

Larry Wall

http://en.wikipedia.org/wiki/File:Larry_Wall_YAPC_2007.jpg

creatorul Perl BDFL pentru proiectul Perl autorul programului patch c as ator al International Obfuscated C Code Contest , tig
CSE Dep Cursul 10, Shell scripting 42/47

Rackspace

companie de hosting IT fondat a n 1996 unul dintre principalii furnizori de servicii cloud (Rackspace Cloud) contribuitor la OpenStack folosit de GitHub

CSE Dep

Cursul 10, Shell scripting

43/47

Lua

limbaj de programare de scripting paradigme multiple (la fel ca Python) proiectat pentru a us ncorporat n alte limbaje , or API C simplu limbaj de sripting pentru dezvoltatorii de jocuri folosit de limbajul de scripting al utilitarului nmap

CSE Dep

Cursul 10, Shell scripting

44/47

Cuvinte cheie

ltre de text cat tac nl head tail sort uniq cut tr

grep if case for while read IFS funct , ii sed awk

CSE Dep

Cursul 10, Shell scripting

45/47

Resurse utile

http://www.shelldorado.com/ http://tldp.org/LDP/abs/html/ http://linuxcommand.org/writing_shell_scripts.php http://sed.sourceforge.net/sed1line.txt http://www.gnu.org/manual/gawk/gawk.html http://mywiki.wooledge.org/BashFAQ

CSE Dep

Cursul 10, Shell scripting

46/47