Class outline

Introduction, concepts, review & historical perspective Processes

  

Synchronization Scheduling Deadlock

   

Memory management, address translation, and virtual memory Operating system management of I/O File systems Security & protection Distributed systems (as time permits)

Overview: Chapter 1
     

What is an operating system, anyway? Operating systems history The zoo of modern operating systems Review of computer hardware Operating system concepts Operating system structure
 

User interface to the operating system Anatomy of a system call

What is an Operating System?



A program that acts as an intermediary between a user of a computer and the computer hardware. Operating system goals:


Execute user programs and make solving user problems easier. Make the computer system convenient to use.

Use the computer hardware in an efficient manner.

Abstract View of System Components

What is an operating system?



A program that runs on the raw hardware and supports

 

Resource Abstraction Resource Sharing

Abstracts and standardizes the interface to the user across different types of hardware


Virtual machine hides the messy details which must be performed Each program gets time with the resource Each program gets space on the resource Use hardware efficiently Give maximum performance to each user

Manages the hardware resources

 

May have potentially conflicting goals:

 

Brief History


First Generation of computers had no OS: single-user. All coding done directly in machine language, memory resident code (no other resources to manage) Second Generation has basic OS: batch processing. Read input (tape/cards), process, output to tape or print Third Generation improved life: multiprogramming! Careful partitioning of memory space (4-12KB), drums and disks added for reading cards and spooling outputs (Simultaneous Peripherals Operations On-Line) Time-sharing created several virtual machines

History (cont)


Fourth Generation: PCs and workstations. Cheaper, faster, more user-friendly (Thank Macs for interfaces!) UNIX precursor MULTICS (MULTIplexed Information and Computing Services) was the first modern OS. Bell+MIT+GE (MULTICS --> units --> Unix) Berkeley improved on it: paging, virtual memory, file systems, signals (interrupts), networking!

Operating system timeline



First generation: 1945 1955

 

Vacuum tubes Plug boards Transistors Batch systems Integrated circuits Multiprogramming Large scale integration Personal computers Systems connected by high-speed networks? Wide area resource management?

Second generation: 1955 1965

 

Third generation: 1965 1980

 

Fourth generation: 1980 present

 

Next generation: ???

 

First generation: direct input



Run one job at a time

  

Enter it into the computer (might require rewiring!) Run it Record the results Computer was idle during first and last steps Computers were very expensive!

Problem: lots of wasted computer time!

 

Goal: make better use of an expensive commodity: computer time

Second generation: batch systems

    

Bring cards to 1401 Read cards onto input tape Put input tape on 7094 Perform the computation, writing results to output tape Put output tape on 1401, which prints output

Structure of a typical 2nd generation job

Data for program FORTRAN program

$RUN $LOAD

$END

$FORTRAN $JOB, 10,6610802, ETHAN MILLER

Spooling
 

Original batch systems used tape drives Later batch systems used disks for buffering
   

Operator read cards onto disk attached to the computer Computer read jobs from disk Computer wrote job results to disk Operator directed that job results be printed from disk

Disks enabled simultaneous peripheral operation online (spooling)

  

Computer overlapped I/O of one job with execution of another Better utilization of the expensive CPU Still only one job active at any given time

Third generation 1965 1980



 

IBM System/360: combine business and scientific computers into one machine Computer will grow with client(memory, processor speed, number of I/O devices etc.) Forerunner of 370, 4300, 3080 and 3090 Use of integrated circuits provided major price/performance advantage over 2nd generation OS/360 had to meet conflicting needs which resulted in enormous and complex operating system Introduced multiprogramming to make most efficient use of CPU

Third generation: multiprogramming



Multiple jobs in memory



Protected from one another

Job 3 Job 2 Memory partitions Job 1 Operating system

Operating system protected from each job as well Resources (time, hardware) split between jobs Still not interactive
  

User submits job Computer runs it User gets results minutes (hours, days) later

Timesharing


Multiprogramming allowed several jobs to be active at one time

 

Initially used for batch systems Cheaper hardware terminals -> interactive use No more priesthood Quick turnaround meant quick fixes for problems

Computer use got much cheaper and easier

 

Types of modern operating systems

Mainframe operating systems: MVS  Server operating systems: FreeBSD, Solaris  Multiprocessor operating systems: Cellular IRIX  Personal computer operating systems: Windows, Unix  Real-time operating systems: VxWorks  Embedded operating systems  Smart card operating systems Some operating systems can fit into more than one category


Fourth Generation


PCs and workstations. Cheaper, faster, more userfriendly (Thank Macs for interfaces!) UNIX precursor MULTICS (MULTIplexed Information and Computing Services) was the first modern OS. Bell+MIT+GE (MULTICS --> units --> Unix) Berkeley improved on it: paging, virtual memory, file systems, signals (interrupts), networking!

Migration of Operating-System Concepts and Features

Desktop Systems


 

Personal computers computer system dedicated to a single user. I/O devices keyboards, mice, display screens, small printers. User convenience and responsiveness. Can adopt technology developed for larger operating systems. Often individuals have sole use of computer and do not need advanced CPU utilization of protection features. May run several different types of operating systems (Windows, MacOS, UNIX, Linux)

Parallel Systems


Multiprocessor systems with more than on CPU in close communication. Tightly coupled system processors share memory and a clock; communication usually takes place through the shared memory. Advantages of parallel system:
 

Increased throughput with more processors Economical share peripherals, mass storage, power etc. as opposed to individual PCs Increased reliability


graceful degradation / fault tolerant

 

failure of one processor will slow down but not halt the system other processors pick up the slack

Parallel Systems (Cont.)



Symmetric multiprocessing (SMP)



Each processor runs and identical copy of the operating system. Many processes can run at once without performance deterioration. Most modern operating systems support SMP

Asymmetric multiprocessing


Each processor is assigned a specific task; master processor schedules and allocates work to slave processors. More common in extremely large systems

Distributed Systems
 

Distribute the computation among several physical processors. Loosely coupled system each processor has its own local memory; processors communicate with one another through various communications lines, such as high-speed buses or telephone lines. Advantages of distributed systems.
   

Resource sharing Computation speed up load sharing Reliability Communications between processors and processes

Distributed Systems (cont)



Requires networking infrastructure.



TCP/IP is the most common network protocol

Local area networks (LAN) or Wide area networks (WAN) May be either client-server or peer-to-peer systems.

General Structure of Client-Server

Clustered Systems


  

Differs from parallel systems in that they are composed of two or more individual systems coupled together Clustering allows two or more systems to share storage. Provides high reliability. Asymmetric clustering (passive standby): one server runs the application while other servers are in hot standby mode Symmetric clustering(active secondary): all hosts are running the application, monitoring each other

Real-Time Systems


 

Often used as a control device in a dedicated application such as controlling scientific experiments, medical imaging systems, industrial control systems, and some display systems. Well-defined, fixed time constraints. Hard real-time:


Secondary storage limited or absent, data stored in short term memory, or readonly memory (ROM) Not supported by general-purpose operating systems. Limited utility in industrial control of robotics Useful in applications (multimedia, virtual reality) requiring advanced operating-system features.

Soft real-time
 

Handheld Systems
Personal Digital Assistants (PDAs) Cellular telephones Issues:


  

 

Limited memory (512KB to 8MB) requires efficient management Slow processors Small display screens.

Networking


Networked OSs are connected through a network, but user needs to know the name/type/location of everything Distributed OSs (e.g., Amoeba, Mach, Locus) provide transparency to user, yielding one huge virtual machine! Specialized OSs are built for specific purposes: routing engines (Networking), lisp machines (AI), time constrained applications (Real-Time), Internet (WWW servers), massively parallel uses (supercomputers), etc All these are coming together, hard to identify boundaries anymore.

Microsoft World
 

Excellent marketing, some good products OSs started with DOS (Disk OS), no nothing, just very simple commands! Windows 3.1 was a huge jump (based on decadesold technology initially developed at Xerox then Macs) Windows 95 (released in 96) improved tremendously the state-of-the-affairs for MS, but still unreliable Windows NT approaches Unix distributions, with more user-friendly interface.

Unix World
  

 

 

Created at AT&T, re-written/improved by Berkeley ATT had majority control and good support (reliable OS) OSF (Open SW Foundation, now Open Group) is a consortium of several companies to standardize UNIX Different subgroups (syscalls, shells, RT, etc) Standardization is with respect to interfaces and not implementation of primitives. Impln is left to the implr Modern applications are time constrained (tel, video, etc) Real-Time playing an increasingly important role

Components of a simple PC
Outside world

Video controller

Hard drive controller

USB controller

Network controller

CPU Memory

Computer internals (inside the box)

CPU internals
Execute unit

Fetch unit Fetch unit Decode unit Execute unit Fetch unit

Decode unit

Buffer
Decode unit

Execute unit Execute unit

Pipelined CPU

Superscalar CPU

Storage pyramid
Capacity < 1 KB 1 MB 256 MB 40 GB
Better

Access latency Registers Cache (SRAM) Main memory (DRAM) Magnetic disk Magnetic tape 1 ns 25 ns 50 ns 5 ms 50 sec
Better

> 1 TB


Goal: really large memory with very low latency

 

Latencies are smaller at the top of the hierarchy Capacities are larger at the bottom of the hierarchy

Solution: move data between levels to create illusion of large memory with low latency

Disk drive structure



Data stored on surfaces

 

head sector

Up to two surfaces per platter One or more platters per disk Tracks broken into sectors


Data in concentric tracks



platter track cylinder

256B-1KB per sector

Cylinder: corresponding tracks on all surfaces

Data read and written by heads

 

surfaces

Actuator moves heads Heads move in unison

spindle actuator

Memory
Address
0x2ffff 0x2b000 0x27fff

Address User program and data

Limit Base 0x2ffff 0x2d000 0x2bfff 0x29000

User data User data

Limit2 Base2 Limit1 Base1

User program and data 0x23000

0x1dfff

0x24fff User program 0x23000 0x1dfff

Operating system
0 0

Operating system

 

Single base/limit pair: set for each process Two base/limit registers: one for program, one for data

Anatomy of a device request

Instructionn Instructionn+1

3 CPU 1 5 Interrupt controller 6

1: Interrupt

Disk controller 4

Operating system Interrupt handler 2: Process interrupt

3: Return

Left: sequence as seen by hardware

  

Request sent to controller, then to disk Disk responds, signals disk controller which tells interrupt controller Interrupt controller notifies CPU

Right: interrupt handling (software point of view)

OS Structure


  

Interface can be done at USER any level (depends on level of security of OS) OS Interface with the lower Dev Drivers level layer gets translated Machine dependent language Hardware used for accessing hardware Main advantage of direct resource access is efficiency Main advantage of indirect access is portability Completely layered OS? Why or why not?

OS Functions


 

Controls and manages resources (disks, memory, CPU, ); sends/receives control commands and data Allows multiprogramming (several programs at the same time in the same resource) Carries out communication between processes (inter and intra processor) Manages interrupt handlers for HW and SW interrupts Provides protection and security to processes Prioritizes requests and manages multiple resources in a single machine (eg multiprocessors or CPU IO reqs)

OS Functions


OS manages resources, including management of

 

  

processes (creation, deletion, suspension, comm, synch) main memory (usage, alloc/de-alloc, which processes get it) 2ary storage (disk scheduling, alloc/de-alloc, swapping, files) IO interfaces and devices (eg, keyboard, caching, memory) protection (authorization, file and memory protection, etc) InterProcess Communication (intra- and inter-machines) Command interpretation (shells to Xlate user to OS). Typically includes the user interface that the OS uses.

OS Structure
 

USER OS

 

Hardware at the bottom layer Dev Drivers Accessing the lower layer thru Hardware the higher layers Typical DOS DOS programs can access HW Unix has controllers and dev drivers (DD) controlling devices USER system calls are the interface System Programs between user and OS (DDs) Kernel Functions libraries and system programs Dev Drivers invoke sys_calls
Hardware Typical Unix

OS Structure
  

 

Interface can be done at any level (depends on security) Machine dependent language used for accessing HW Main advantage of direct resource access is efficiency (less layers means less overhead, ie, better performance Main advantage of indirect access (syscall) is portability Modular approaches (ind access) have less flexibility, since appls only access HW thru libraries and sys_calls Layering means that one level is defined in terms of the level below (level 0 is the HW, level n is the user appls)

Modular Approach


 

Create well-defined interfaces between any two layers Create well-defined properties of each layer Attempt to decrease the number of layers to improve efficiency and performance The final goal is to make the OS flexible and efficient Create the layers such that each user perceives the machine as belonging solely to himself or herself This is the concept of a virtual machine, which allows each user to avoid thinking about others processes

Operating systems concepts

     

Many of these should be familiar to Unix users Processes (and trees of processes) Deadlock File systems & directory trees Pipes Well cover all of these in more depth later on, but its useful to have some basic definitions now

Processes


Process: program in execution



Address space (memory) the program can use State (registers, including program counter & stack pointer)

OS keeps track of all processes in a process table Processes can create other processes


 

Process tree tracks these relationships A is the root of the tree A created three child processes: B, C, and D C created two child processes: E and F D created one child process: G

Inside a (Unix) process

0x7fffffff Stack


Processes have three segments

 

Text: program code Data: program data

 

Statically declared variables Areas allocated by malloc() or new Automatic variables Procedure call information

Stack
 

Data Data


Address space growth

  

Text 0

Text: doesnt grow Data: grows up Stack: grows down

Deadlock

Potential deadlock

Actual deadlock

Hierarchical file systems

Root directory

bin

cse

faculty ls ps cp csh elm sbrandt kag

grads

amer4

classes stuff

research stuff

Interprocess communication
 

Processes want to exchange information with each other Many ways to do this, including
 

Network Pipe (special file): A writes into pipe, and B reads from it

System calls


Programs want the OS to perform a service

  

Access a file Create a process Others Program passes relevant information to OS OS performs the service if
 

Accomplished by system call

 

The OS is able to do so The service is permitted for this program at this time Dont want programs reading data into other programs memory!

OS checks information passed to make sure its OK



Language
 

  

System calls are the interface between user and OS Access to the resources is done through priviledged instructions (for protection) User applications cannot execute in kernel mode User applications user libraries that invoke sys_calls System procedures are executed to access resources, via priviledged instructions (called from sys_calls) This way, no process can influence other executions, on purpose or by accident: resource protection Example: accounting, priority information

Language (cont)


System calls can be divided into 5 categories:

    

process control file manipulation device manipulation infomation maintenance communication

Special purpose OSs can also have special primitives:

 

specification of deadlines, priorities, periodicity of processes specification of precedence constraints and/or synchronization among processes

Language (cont)


Examples of libraries are language constructs to carry out formatted printing Examples of sys_calls are primitives to create a process For example, the reading of 10 bytes of a file:


The user does fscanf, the kernel requests a block of bytes from the device driver (DD), which talks to the controller of the disk to obtain a block of data. The block is transfered into a buffer, in the kernel address space. The kernel then picks the 10 bytes and copies them into the user-specified location. This way, the kernel accesses kernel and user space, but the user only accesses user space!

Making a system call

0xffffffff Library (read call) Return to caller Trap to kernel 3 Trap code in register User space 4 2 Increment SP Call read 1 Push arguments 9 8
 

System call:
read(fd,buffer,length)

 

Program pushes arguments, calls library Library sets up trap, calls OS OS handles system call Control returns to library Library returns to user program

Kernel space (OS) 0

Dispatch

6 Sys call handler

User code

System Programs


System programs do not interact directly with running user programs, but define a better environmnt for the development of application programs. Sys programs include: compilers, file manipulation and modification, editors, linker/loaders, etc An important one is the command interpreter (or shell), which parses user input, interprets it, and executes it Shells can either execute the command, or invoke other system programs or system calls to do it. Trade-offs: performance, increasing/updating # of commands

System calls for files & directories

Call
fd = open(name,how) s = close(fd) n = read(fd,buffer,size) n = write(fd,buffer,size) s = lseek(fd,offset,whence) s = stat(name,&buffer) s = mkdir(name,mode) s = rmdir(name) s = link(name1,name2) s = unlink(name)

Description
Open a file for reading and/or writing Close an open file Read data from a file into a buffer Write data from a buffer into a file Move the current pointer for a file Get a files status information (in buffer) Create a new directory Remove a directory (must be empty) Create a new entry (name2) that points to the same object as name1 Remove name as a link to an object (deletes the object if name was the only link to it)

More system calls

Call
pid = fork() pid=waitpid(pid,&statloc,options) s = execve(name,argv,environp) exit(status) s = chdir(dirname) s = chmod(name,mode) s = kill(pid,signal) seconds = time(&seconds)

Description
Create a child process identical to the parent Wait for a child to terminate Replace a process core image Terminate process execution and return status Change the working directory Change a files protection bits Send a signal to a process Get the elapsed time since 1 Jan 1970

A simple shell
while (TRUE) { /* repeat forever */ type_prompt( ); /* display prompt */ read_command (command, parameters) /* input from terminal */ if (fork() != 0) { /* fork off child process */ /* Parent code */ waitpid( -1, &status, 0); /* wait for child to exit */ } else { /* Child code */ execve (command, parameters, 0); /* execute command */ } }

Monolithic OS structure
Main procedure

Service routines

Utility routines

Virtual machines
App1 System calls I/O instructions Calls to simulate I/O Real I/O instructions App2 Linux VMware App3 Windows NT VMware Linux Bare hardware FreeBSD VMware

 

First widely used in VM/370 with CMS Available today in VMware



Allows users to run any x86-based OS on top of Linux or NT Only virtual machine failsrest of underlying OS is fine Virtual machine keeps things separated

Guest OS can crash without harming underlying OS



Guest OS can even use raw hardware



Microkernels (client-server)
Client process Client process Process server Terminal server

File server

Memory server

User mode Kernel mode

Microkernel

Processes (clients and OS servers) dont share memory

 

Communication via message-passing Separation reduces risk of byzantine failures

Examples include Mach