Sunteți pe pagina 1din 27

Syllabus

May 18: IntroductionSyllabus May 25: Python tutorial Jun 1: NetCDF + Python Jun 8: NetCDF + Ferret Jun

May 25: Python tutorialSyllabus May 18: Introduction Jun 1: NetCDF + Python Jun 8: NetCDF + Ferret Jun 15:

Jun 1: NetCDF + PythonSyllabus May 18: Introduction May 25: Python tutorial Jun 8: NetCDF + Ferret Jun 15: Advanced

Jun 8: NetCDF + FerretIntroduction May 25: Python tutorial Jun 1: NetCDF + Python Jun 15: Advanced processing NetCDF, Python

Jun 15: Advanced processing NetCDF, Python18: Introduction May 25: Python tutorial Jun 1: NetCDF + Python Jun 8: NetCDF + Ferret

After: User requests for projectsMay 25: Python tutorial Jun 1: NetCDF + Python Jun 8: NetCDF + Ferret Jun 15:

Week 1: Intro stuff

Outline

Intro to UNIXOutline 10 most import commands Departmental Computing Layout Computing Tips

10 most import commandsOutline Intro to UNIX Departmental Computing Layout Computing Tips

Departmental Computing LayoutOutline Intro to UNIX 10 most import commands Computing Tips

Computing TipsOutline Intro to UNIX 10 most import commands Departmental Computing Layout

UNIX, the layered OS

Hardware, physical components of the computerUNIX, the layered OS Kernel, software interface layer to hardware Libraries, public methods to interface with

Kernel, software interface layer to hardwarethe layered OS Hardware, physical components of the computer Libraries, public methods to interface with the

Libraries, public methods to interface with the kernelof the computer Kernel, software interface layer to hardware Programs, call libraries to execute code on

Programs, call libraries to execute code on the hardware.of the computer Kernel, software interface layer to hardware Libraries, public methods to interface with the

UNIX Filesystems

UNIX has a 'rooted'filesystem. '/'is referred to as the root directory.UNIX Filesystems Each node in the filesystem is either a file or a directory. Directories are

Each node in the filesystem is either a file or a directory. Directories are special files'/'is referred to as the root directory. Two directory names are reserved “ .” refers to

Two directory names are reservedeither a file or a directory. Directories are special files “ .” refers to the local

“ .” refers to the local directory

.” refers to the local directory

“ ” refers to the parent directory

refers to the parent directory

Referencing files

Local ReferenceReferencing files “ ./myscript.sh” Relative Reference /bin/myscript.sh” Absolute Reference “

Referencing files Local Reference “ ./myscript.sh” Relative Reference /bin/myscript.sh” Absolute Reference “

“ ./myscript.sh”

Relative ReferenceReferencing files Local Reference “ ./myscript.sh” /bin/myscript.sh” Absolute Reference “ /usr/bin/myscript.sh”

/bin/myscript.sh”Referencing files Local Reference “ ./myscript.sh” Relative Reference Absolute Reference “ /usr/bin/myscript.sh”

Absolute ReferenceReferencing files Local Reference “ ./myscript.sh” Relative Reference /bin/myscript.sh” “ /usr/bin/myscript.sh”

Local Reference “ ./myscript.sh” Relative Reference /bin/myscript.sh” Absolute Reference “ /usr/bin/myscript.sh”

“ /usr/bin/myscript.sh”

Ex) “ /home/akrherz/myscript.sh” Which of the following does not reference the above file? /home/./akrherz/ “

Ex) “ /home/akrherz/myscript.sh”

Which of the following does not reference the above file?

/home/./akrherz/“

/akrherz/./myscript.sh”

/“ /home/akrherz/myscript.sh”

/home/akrherz/myscript.sh”

 

/./home/“ /

/

/

/

/

/home/akrherz/myscript.sh”

/home/akrherz/myscript.sh/“

/myscript.sh”

Base UNIX Filesystem

/
/

binBase UNIX Filesystem / etc (System Configs) home (User Data) lib (System Libs) sbin (Admin Binaries)

etc (System Configs)Base UNIX Filesystem / bin home (User Data) lib (System Libs) sbin (Admin Binaries) tmp (Temp

home (User Data)Base UNIX Filesystem / bin etc (System Configs) lib (System Libs) sbin (Admin Binaries) tmp (Temp

lib (System Libs)UNIX Filesystem / bin etc (System Configs) home (User Data) sbin (Admin Binaries) tmp (Temp Space)

sbin (Admin Binaries)bin etc (System Configs) home (User Data) lib (System Libs) tmp (Temp Space) usr (User Binaries)

tmp (Temp Space)Configs) home (User Data) lib (System Libs) sbin (Admin Binaries) usr (User Binaries) var (App Data)

usr (User Binaries)Configs) home (User Data) lib (System Libs) sbin (Admin Binaries) tmp (Temp Space) var (App Data)

var (App Data)Configs) home (User Data) lib (System Libs) sbin (Admin Binaries) tmp (Temp Space) usr (User Binaries)

(System Binaries)

Working within UNIX

ShellsWorking within UNIX bash, csh, ksh, tcsh Scripts shell scripts, perl scripts, python scripts, etc Executed

bash, csh, ksh, tcshWorking within UNIX Shells Scripts shell scripts, perl scripts, python scripts, etc Executed at run­time Programs

ScriptsWorking within UNIX Shells bash, csh, ksh, tcsh shell scripts, perl scripts, python scripts, etc Executed

shell scripts, perl scripts, python scripts, etcWorking within UNIX Shells bash, csh, ksh, tcsh Scripts Executed at run­time Programs / Executables pre­compiled.

Executed at run­timeksh, tcsh Scripts shell scripts, perl scripts, python scripts, etc Programs / Executables pre­compiled. C++, FORTAN,

Programs / Executablesksh, tcsh Scripts shell scripts, perl scripts, python scripts, etc Executed at run­time pre­compiled. C++, FORTAN,

pre­compiled. C++, FORTAN, etcbash, csh, ksh, tcsh Scripts shell scripts, perl scripts, python scripts, etc Executed at run­time Programs

Running Commands

Running Commands Commands need to either be in your $PATH ex) ls referenced directly ex) /bin/ls

Commands need to either be

in your $PATHRunning Commands Commands need to either be ex) ls referenced directly ex) /bin/ls Commands can take

ex) lsRunning Commands Commands need to either be in your $PATH referenced directly ex) /bin/ls Commands can

referenced directlyCommands Commands need to either be in your $PATH ex) ls ex) /bin/ls Commands can take

ex) /bin/lsneed to either be in your $PATH ex) ls referenced directly Commands can take arguments/flags Here

be in your $PATH ex) ls referenced directly ex) /bin/ls Commands can take arguments/flags Here is

Commands can take arguments/flags

Here is a flagdirectly ex) /bin/ls Commands can take arguments/flags ex) ls ­l Here is an argument ex) ls

ex) /bin/ls Commands can take arguments/flags Here is a flag ex) ls ­l Here is an

ex) ls ­l

Here is an argument/bin/ls Commands can take arguments/flags Here is a flag ex) ls ­l ex) ls myscript.sh Here

ex) ls myscript.shCommands can take arguments/flags Here is a flag ex) ls ­l Here is an argument Here

Here is bothCommands can take arguments/flags Here is a flag ex) ls ­l Here is an argument ex)

take arguments/flags Here is a flag ex) ls ­l Here is an argument ex) ls myscript.sh

ex) ls ­l myscript.sh

Getting Help with your Program

Getting Help with your Program Most programs should have one of the following help options available

Most programs should have one of the following help options available

man commandProgram Most programs should have one of the following help options available ex) man ls command

ex) man lsProgram Most programs should have one of the following help options available man command command –help

command –helpyour Program Most programs should have one of the following help options available man command ex)

ex) ls ­­helpyour Program Most programs should have one of the following help options available man command ex)

File Permissions / Ownerships

$ ls ­l qa4.css ­rw­rw­r­­File Permissions / Ownerships Lots going on here! 1 akrherz users 2275 Dec 17 21:01 qa4.css

Lots going on here!Permissions / Ownerships $ ls ­l qa4.css ­rw­rw­r­­ 1 akrherz users 2275 Dec 17 21:01 qa4.css

1 akrherz users

2275 Dec 17 21:01 qa4.css

“ ­rw­rw­r­­” are file permissions

­rw­rw­r­­” are file permissions

“ 1” is the number of directories+files

“ 1” is the number of directories+files

“ akrherz” is the owner of the file

“ akrherz” is the owner of the file

“ users” is the group owner of the file

“ users” is the group owner of the file

“ 2275” is the filesize in bytes

“ 2275” is the filesize in bytes

“ Dec 17 21:01” is the file modification time

“ Dec 17 21:01” is the file modification time

10 Commands you must know

lsw

ls w

w

cdln

cd ln

ln

dfchmod

df chmod

chmod

duchown

du chown

chown

pwdtar

pwd tar

tar

list directory contentsUseful flags ls ­a “ list all” ­d “ don't walk dirs” ­l “ long

Useful flagslist directory contents ls ­a “ list all” ­d “ don't walk dirs” ­l “ long

ls

list directory contents Useful flags ls ­a “ list all” ­d “ don't walk dirs” ­l

­a “ list all”list directory contents Useful flags ls ­d “ don't walk dirs” ­l “ long format” ­r

­d “ don't walk dirs”list directory contents Useful flags ls ­a “ list all” ­l “ long format” ­r “

­l “ long format”flags ls ­a “ list all” ­d “ don't walk dirs” ­r “ reverse output” ­R

­r “ reverse output”all” ­d “ don't walk dirs” ­l “ long format” ­R “ recursive” ­t “ sort

­R “ recursive”dirs” ­l “ long format” ­r “ reverse output” ­t “ sort by mod time” Common

­t “ sort by mod time”dirs” ­l “ long format” ­r “ reverse output” ­R “ recursive” Common uses ls ­l

Common uses

ls ­lformat” ­r “ reverse output” ­R “ recursive” ­t “ sort by mod time” Common uses

ls ­latrlong format” ­r “ reverse output” ­R “ recursive” ­t “ sort by mod time” Common

ls ­dformat” ­r “ reverse output” ­R “ recursive” ­t “ sort by mod time” Common uses

ls ­Rformat” ­r “ reverse output” ­R “ recursive” ­t “ sort by mod time” Common uses

Built­in shell commandchanges your current working directory No meaningful flags cd Examples cd cd /tmp

changes your current working directoryBuilt­in shell command No meaningful flags cd Examples cd cd /tmp

No meaningful flagsBuilt­in shell command changes your current working directory cd Examples cd cd /tmp

cd

Built­in shell command changes your current working directory No meaningful flags cd Examples cd cd /tmp

Examples

cdBuilt­in shell command changes your current working directory No meaningful flags cd Examples cd /tmp

cd /tmpBuilt­in shell command changes your current working directory No meaningful flags cd Examples cd

Reports filesystem disk space usageUseful Flags ­l “ local filesystems” ­h “ human readable” ­k “ 1k blocks” df

Useful FlagsReports filesystem disk space usage ­l “ local filesystems” ­h “ human readable” ­k “ 1k

­l “ local filesystems”Reports filesystem disk space usage Useful Flags ­h “ human readable” ­k “ 1k blocks” df

­h “ human readable”disk space usage Useful Flags ­l “ local filesystems” ­k “ 1k blocks” df Examples df

­k “ 1k blocks”disk space usage Useful Flags ­l “ local filesystems” ­h “ human readable” df Examples df

df

Flags ­l “ local filesystems” ­h “ human readable” ­k “ 1k blocks” df Examples df

Examples

df ­lusage Useful Flags ­l “ local filesystems” ­h “ human readable” ­k “ 1k blocks” df

df ­husage Useful Flags ­l “ local filesystems” ­h “ human readable” ­k “ 1k blocks” df

df /homeusage Useful Flags ­l “ local filesystems” ­h “ human readable” ­k “ 1k blocks” df

Estimate file space usageFlags du ­h “ human readable” ­x “ one filesystem” ­s “ display summary” Examples

FlagsEstimate file space usage du ­h “ human readable” ­x “ one filesystem” ­s “ display

du

Estimate file space usage Flags du ­h “ human readable” ­x “ one filesystem” ­s “

­h “ human readable”Estimate file space usage Flags du ­x “ one filesystem” ­s “ display summary” Examples du

­x “ one filesystem”Estimate file space usage Flags du ­h “ human readable” ­s “ display summary” Examples du

­s “ display summary”Estimate file space usage Flags du ­h “ human readable” ­x “ one filesystem” Examples du

Examples

du ­s ­h *file space usage Flags du ­h “ human readable” ­x “ one filesystem” ­s “ display

du .space usage Flags du ­h “ human readable” ­x “ one filesystem” ­s “ display summary”

Print name of the current working directory pwd One example! pwd

Print name of the current working directory

pwd

Print name of the current working directory pwd One example! pwd

One example!

Print name of the current working directory pwd One example! pwd

pwd

w

show who is logged in and what are they doingw No useful flags Should always be the first command you run after logging in Example:

No useful flagsw show who is logged in and what are they doing Should always be the first

Should always be the first command you run after logging inw show who is logged in and what are they doing No useful flags Example: w

logged in and what are they doing No useful flags Should always be the first command

Example:

logged in and what are they doing No useful flags Should always be the first command

w

make links between filesOne flag: ­s “ source” ln Example: ln ­s realreal linked

One flag:make links between files ­s “ source” ln Example: ln ­s realreal linked

make links between files One flag: ­s “ source” ln Example: ln ­s realreal linked

­s “ source”

ln

make links between files One flag: ­s “ source” ln Example: ln ­s realreal linked

Example:

make links between files One flag: ­s “ source” ln Example: ln ­s realreal linked

ln ­s realreal linked

chmod

Change file permissionschmod Flags: ­R “ recursive” Octet based ex) 755 Named base u+rw Examples chmod +x myscript.sh

Flags:chmod Change file permissions ­R “ recursive” Octet based ex) 755 Named base u+rw Examples chmod

chmod Change file permissions Flags: ­R “ recursive” Octet based ex) 755 Named base u+rw Examples

­R “ recursive”

Octet based ex) 755chmod Change file permissions Flags: ­R “ recursive” Named base u+rw Examples chmod +x myscript.sh chmod

Named base u+rwpermissions Flags: ­R “ recursive” Octet based ex) 755 Examples chmod +x myscript.sh chmod 775 directory

­R “ recursive” Octet based ex) 755 Named base u+rw Examples chmod +x myscript.sh chmod 775

Examples

chmod +x myscript.shFlags: ­R “ recursive” Octet based ex) 755 Named base u+rw Examples chmod 775 directory chmod

chmod 775 directoryFlags: ­R “ recursive” Octet based ex) 755 Named base u+rw Examples chmod +x myscript.sh chmod

chmod u+w,g­r,o­w fileFlags: ­R “ recursive” Octet based ex) 755 Named base u+rw Examples chmod +x myscript.sh chmod

chown

Change file owner and groupchown Flags: ­R “ recursive” Typically, you would only run the command to change group permissions

Flags:chown Change file owner and group ­R “ recursive” Typically, you would only run the command

chown Change file owner and group Flags: ­R “ recursive” Typically, you would only run the

­R “ recursive”

Typically, you would only run the command to change group permissionschown Change file owner and group Flags: ­R “ recursive” Examples chown akrherz.gcp myfile.html akrherz is

you would only run the command to change group permissions Examples chown akrherz.gcp myfile.html akrherz is

Examples

chown akrherz.gcp myfile.htmlTypically, you would only run the command to change group permissions Examples akrherz is the user

akrherz is the useryou would only run the command to change group permissions Examples chown akrherz.gcp myfile.html gcp is

gcp is the groupyou would only run the command to change group permissions Examples chown akrherz.gcp myfile.html akrherz is

Archiving UtilityLots of Flags: tar ­f “ file” ­z “ gzip input/output” ­c “ create” ­x

Lots of Flags:Archiving Utility tar ­f “ file” ­z “ gzip input/output” ­c “ create” ­x “ extract”

tar

Archiving Utility Lots of Flags: tar ­f “ file” ­z “ gzip input/output” ­c “ create”

­f “ file”Archiving Utility Lots of Flags: tar ­z “ gzip input/output” ­c “ create” ­x “ extract”

­z “ gzip input/output”Archiving Utility Lots of Flags: tar ­f “ file” ­c “ create” ­x “ extract” ­t

­c “ create”of Flags: tar ­f “ file” ­z “ gzip input/output” ­x “ extract” ­t “ list”

­x “ extract”“ file” ­z “ gzip input/output” ­c “ create” ­t “ list” ­j “ bzip input/output”

­t “ list”gzip input/output” ­c “ create” ­x “ extract” ­j “ bzip input/output” Examples: tar ­cf myfile.tar

­j “ bzip input/output”­c “ create” ­x “ extract” ­t “ list” Examples: tar ­cf myfile.tar . tar ­xf

Examples:

tar ­cf myfile.tar .­t “ list” ­j “ bzip input/output” Examples: tar ­xf myfile.tar tar ­xzf myfile.tgz tar ­tzf

tar ­xf myfile.tar­j “ bzip input/output” Examples: tar ­cf myfile.tar . tar ­xzf myfile.tgz tar ­tzf myfile.tgz tar

tar ­xzf myfile.tgzbzip input/output” Examples: tar ­cf myfile.tar . tar ­xf myfile.tar tar ­tzf myfile.tgz tar ­xf myfile.tar

tar ­tzf myfile.tgzbzip input/output” Examples: tar ­cf myfile.tar . tar ­xf myfile.tar tar ­xzf myfile.tgz tar ­xf myfile.tar

tar ­xf myfile.tar mydir­j “ bzip input/output” Examples: tar ­cf myfile.tar . tar ­xf myfile.tar tar ­xzf myfile.tgz tar

Our Computing System

/home is NFS mounted and the same on most every machineOur Computing System /usr/local contains common programs and is the same on all systems User passwords

/usr/local contains common programs and is the same on all systems/home is NFS mounted and the same on most every machine User passwords are the same

User passwords are the same on all systemscontains common programs and is the same on all systems Many data drives are NFS mounted

Many data drives are NFS mounted and available on the faster machinesmachine /usr/local contains common programs and is the same on all systems User passwords are the

Tips for efficient computing

Output from the 'ls'command should never cause the terminal window to scrollTips for efficient computing The more directories, the better! Use the /tmp directory for quick processing

The more directories, the better!should never cause the terminal window to scroll Use the /tmp directory for quick processing Place

Use the /tmp directory for quick processingterminal window to scroll The more directories, the better! Place scripts in your $HOME directory, and

Place scripts in your $HOME directory, and data in a /something directorywindow to scroll The more directories, the better! Use the /tmp directory for quick processing Use

Use sym links, if neededUse the /tmp directory for quick processing Place scripts in your $HOME directory, and data in

More tips

Run your processing scripts physically close to your data. Understand what the command 'df'is telling you.More tips Don't just run code to see what happens. This hurts your computing efforts and

Don't just run code to see what happens. This hurts your computing efforts and those of othersUnderstand what the command 'df'is telling you. Keep your $HOME directory clean and organized. Use tar

Keep your $HOME directory clean and organized.This hurts your computing efforts and those of others Use tar to compress your output and

Use tar to compress your output and to move to backups. Be careful when you run tar!to see what happens. This hurts your computing efforts and those of others Keep your $HOME

Assignment for Next Time

I will email you a spreadsheet with all of the data shares listed.Assignment for Next Time Log into each of those machines you have data on and produce

Log into each of those machines you have data on and produce a disk usage report for yourselfemail you a spreadsheet with all of the data shares listed. Save those values in the

Save those values in the spreadsheet. We will use it next time.of the data shares listed. Log into each of those machines you have data on and