Advanced

Operating Systems

Outline
Process Concept
Process Scheduling
Operations on Processes
Cooperating Processes
Threads
Interprocess Communication

Process Concept
An operating system executes a variety of
programs
batch systems - jobs
time-shared systems - user programs or tasks
job and program used interchangeably

Process - a program in execution

process execution proceeds in a sequential fashion

A process contains
program counter, stack and data section

Process =? Program
main ()
{

main ()
{

A() {

A() {

Heap

Stack
A
main

Program
}
}
More to a process than just a program:

Process

Program is just part of the process state

I run emacs on lectures.txt, you run it on homework.java Same program,
different processes

Less to a process than a program:

A program can invoke more than one process
cc starts up cpp, cc1, cc2, as, and ld
4

Process State
A process changes state as it executes.
new
new

admitted

exit

terminated

interrupt
running

ready

I/O or
event
completion

Scheduler
dispatch

I/O or
event wait

waiting

Process States
New - The process is being created.
Running - Instructions are being executed.
Waiting - Waiting for some event to occur.
Ready - Waiting to be assigned to a
processor.
Terminated - Process has finished execution.

Process Control Block

Contains information
associated with each process
Process State - e.g. new,
ready, running etc.
Process Number Process ID
Program Counter - address of
next instruction to be executed
CPU registers - general
purpose registers, stack pointer
etc.
CPU scheduling information process priority, pointer
Memory Management
information - base/limit
information
Accounting information - time
limits, process number
I/O Status information - list
of I/O devices allocated

Process
Control
Block
7

Process Scheduling

Process (PCB) moves from queue to queue

When does it move? Where? A scheduling decision

Process Scheduling Queues

Job Queue - set of all processes in the system
Ready Queue - set of all processes residing in main
memory, ready and waiting to execute.
Device Queues - set of processes waiting for an I/O
device.
Process migration between the various queues.
Queue Structures - typically linked list, circular list
etc.

Process Queues
Device
Queue

Ready
Queue

10

Enabling Concurrency and

Protection: Multiplex
processes
Only one process (PCB) active at a time
Current state of process held in PCB:

snapshot of the execution and protection environment

Process needs CPU, resources

Give out CPU time to different processes

(Scheduling):
Only one process running at a time
Give more time to important processes

Give pieces of resources to different processes

(Protection):
Controlled access to non-CPU resources

Process
Control
Block

E.g. Memory Mapping: Give each process their own

address space

11

Enabling Concurrency:
Context Switch
Task that switches CPU from one process to
another process
the CPU must save the PCB state of the old process and
load the saved PCB state of the new process.

Context-switch time is overhead

System does no useful work while switching
Overhead sets minimum practical switching time; can
become a bottleneck

Time for context switch is dependent on

hardware support ( 1- 1000 microseconds).
12

CPU Switch From Process to

Process

Code executed in kernel above is overhead

Overhead sets minimum practical switching time

13

Schedulers
Long-term scheduler (or job scheduler) selects which processes should be brought into the ready
queue.
invoked very infrequently (seconds, minutes); may be slow.
controls the degree of multiprogramming

Short term scheduler (or CPU scheduler) selects which process should execute next and allocates
CPU.
invoked very frequently (milliseconds) - must be very fast

Medium Term Scheduler

swaps out process temporarily
balances load for better throughput
14

Medium Term (Time-sharing)

Scheduler

15

Process Profiles
I/O bound process spends more time in I/O, short CPU bursts, CPU
underutilized.

CPU bound process spends more time doing computations; few very long CPU
bursts, I/O underutilized.

The right job mix:

Long term scheduler - admits jobs to keep load balanced
between I/O and CPU bound processes
Medium term scheduler ensures the right mix (by
sometimes swapping out jobs and resuming them later)
16

Process Creation
Processes are created and deleted
dynamically
Process which creates another process is
called a parent process; the created process
is called a child process.
Result is a tree of processes
e.g. UNIX - processes have dependencies and form a
hierarchy.

Resources required when creating process

CPU time, files, memory, I/O devices etc.
17

UNIX Process Hierarchy

18

Process Creation
Resource sharing
Parent and children share all resources.
Children share subset of parents resources - prevents
many processes from overloading the system.
Parent and children share no resources.

Execution
Parent and child execute concurrently.
Parent waits until child has terminated.

Address Space
Child process is duplicate of parent process.
Child process has a program loaded into it.
19

UNIX Process Creation

Fork system call creates new processes
execve system call is used after a fork to
replace the processes memory space with a
new program.

20

Process Termination
Process executes last statement and asks
the operating system to delete it (exit).
Output data from child to parent (via wait).
Process resources are deallocated by operating system.

Parent may terminate execution of child

processes.
Child has exceeded allocated resources.
Task assigned to child is no longer required.
Parent is exiting
OS does not allow child to continue if parent terminates
Cascading termination

21

Short Term Scheduling

The kernel runs the scheduler at least when

a process switches from running to waiting
(blocks)
a process is created or terminated.
an interrupt occurs (e.g., timer chip)

Non-preemptive system

Scheduler runs when process blocks or is created, not on

hardware interrupts

Preemptive system

OS makes scheduling decisions during interrupts, mostly

timer, but also system calls and other hardware device
interrupts
22

Criteria for Comparing Scheduling

Algorithms
CPU Utilization The percentage of time that the
CPU is busy.
Throughput The number of processes completing
in a unit of time.
Turnaround time The length of time it takes to
run a process from initialization to termination,
including all the waiting time.
Waiting time The total amount of time that a
process is in the ready queue.
Response time The time between when a process
is ready to run and its next I/O request.

23

Scheduling Policies
Ideal CPU scheduler

Maximizes CPU utilization and throughput

Minimizes turnaround time, waiting time, and response time

Real CPU schedulers implement particular policy

Minimize response time - provide output to the user as quickly

as possible and process their input as soon as it is received.
Minimize variance of average response time - in an interactive
system, predictability may be more important than a low
average with a high variance.
Maximize throughput - two components
1. minimize overhead (OS overhead, context switching)
2. efficient use of system resources (CPU, I/O devices)
Minimize waiting time - be fair by ensuring each process waits
the same amount of time. This goal often increases average
response time.

24

Process activity patterns

CPU bound

mp3 encoding
Scientific applications (matrix multiplication)
Compile a program or document

I/O bound

Index a file system

Browse small web pages

Balanced

Playing video
Moving windows around/fast window updates

Scheduling algorithms reward I/O bound and

penalize CPU bound
Why?

25

Scheduling Policies
Simplifying Assumptions
One process per user
One thread per process (more on this topic next week)
Processes are independent

Researchers developed these algorithms in the 70s when

these assumptions were more realistic, and it is still an open
problem how to relax these assumptions.
Scheduling Algorithms:

FCFS: First Come, First Served

Round Robin: Use a time slice and preemption to alternate jobs.
SJF: Shortest Job First
Multilevel Feedback Queues: Round robin on priority queue.
Lottery Scheduling: Jobs get tickets and scheduler randomly
picks winning ticket.
26

Scheduling Policies
FCFS: First-Come-First-Served (or FIFO: First-In-First-Out)

The scheduler executes jobs to completion in

arrival order.
In early FCFS schedulers, the job did not
relinquish the CPU even when it was doing I/O.
We will assume a FCFS scheduler that runs when
processes are blocked on I/O, but that is nonpreemptive, i.e., the job keeps the CPU until it
blocks (say on an I/O device).

27

FCFS Scheduling Policy

In a non-preemptive
system, the scheduler
must wait for one of
these events, but in a
preemptive system the
scheduler can interrupt a
running process.
If the processes arrive
one time unit apart, what
is the average wait time
in these three cases?
Advantages:
Disadvantages
28

Scheduling Policies
Round Robin: very common base policy.
Run each process for its time slice (scheduling quantum)
After each time slice, move the running thread to the back
of the queue.
Selecting a time slice:
Too large - waiting time su?ers, degenerates to FCFS if
processes are never preempted.
Too small - throughput su?ers because too much time is spent
context switching.
Balance the two by selecting a time slice where context
switching is roughly 1% of the time slice.

A typical time slice today is between 10-100 milliseconds,

with a context switch time of 0.1 to 1 millisecond.
Max Linux time slice is 3,200ms, Why?

Is round robin more fair than FCFS? A)Yes B)No

29

Round Robin Examples

5 jobs, 100 seconds each, time slice 1 second, context switch
time of 0, jobs arrive at time 0,1,2,3,4
Completion Time
Job Length FCFS Round Robin
1

100

100

100

100

100

Wait Time
FCFS

Round Robin

Average
30

Round Robin Examples

5 jobs, of length 50, 40, 30, 20, and 10 seconds each, time
slice 1 second, context switch time of 0 seconds
Completion Time
Job Length FCFS Round Robin
1

50

40

30

20

10

Wait Time
FCFS

Round Robin

Average
31

SJF / SRTF: Shortest Job First

Schedule the job that has the least (expected) amount of work
(CPU time) to do until its next I/O request or termination.
I/O bound jobs get priority over CPU bound jobs.

Example: 5 jobs, of length 50, 40, 30, 20, and 10 seconds each,
time slice 1 second, context switch time of 0 seconds

Completion Time
Job

Length FCFS

50

40

30

20

10

Average

RR

SJF

Wait Time
FCFS

RR

32

SJF

SJF / SRTF: Shortest Job First

Works for preemptive and non-preemptive
schedulers.
Preemptive SJF is called SRTF - shortest
remaining time rst.
Advantages
Disadvantages

33

Multilevel Feedback Queues

Using the Past to Predict the Future: Multilevel
feedback queues attempt to overcome the
prediction problem in SJF by using the past I/O
and CPU behavior to assign process priorities.
If a process is I/O bound in the past, it is also likely to
be I/O bound in the future (programs turn out not to be
random.)
To exploit this behavior, the scheduler can favor jobs
(schedule them sooner) when they use very little CPU
time (absolutely or relatively), thus approximating SJF.
This policy is adaptive because it relies on past behavior
and changes in behavior result in changes to scheduling
decisions. We write a program in e.g., Java.

34

Approximating SJF: Multilevel Feedback

Queues
Multiple queues with different priorities.
OS uses Round Robin scheduling at each priority level,
running the jobs in the highest priority queue first.
Once those finish, OS runs jobs out of the next highest
priority queue, etc. (Can lead to starvation.)
Round robin time slice increases exponentially at lower
priorities.

35

Approximating SJF: Multilevel Feedback

Queues
Adjust priorities as follows (details can vary):
1. Job starts in the highest priority queue
2. If jobs time slices expire, drop its priority one level.
3. If jobs time slices do not expire (the context switch comes
from an I/O request instead), then increase its priority one
level, up to the top priority level.
==> In practice, CPU bounds drop like a rock in priority and I/O
bound jobs stay at high priority

36

Improving Fairness
Since SJF is optimal, but unfair, any increase in
fairness by giving long jobs a fraction of the CPU
when shorter jobs are available will degrade
average waiting time. Possible solutions:

Give each queue a fraction of the CPU time. This solution

is only fair if there is an even distribution of jobs among
queues.
Adjust the priority of jobs as they do not get serviced
(Unix originally did this.) This ad hoc solution avoids
starvation but average waiting time suffers when the
system is overloaded because all the jobs end up with a
high priority.

37

Lottery Scheduling
Give every job some number of lottery tickets.
On each time slice, randomly pick a winning ticket.
On average, CPU time is proportional to the
number of tickets given to each job.
Assign tickets by giving the most to short running
jobs, and fewer to long running jobs
(approximating SJF). To avoid starvation, every
job gets at least one ticket.
Degrades gracefully as load changes. Adding or
deleting a job affects all jobs proportionately,
independent of the number of tickets a job has.

38

Lottery Scheduling
Example: Short jobs get 9 tickets, long jobs get 1 tickets each.
# short jobs /

% of CPU each

% of CPU each

# long jobs

short job gets

long job gets

1/1

90%

10%

0/2
2/0
10/1
1/10
39

Summary of Scheduling Algorithms

FCFS: Not fair, and average waiting time is poor.
Round Robin: Fair, but average waiting time is poor.
SJF: Not fair, but average waiting time is minimized
assuming we can accurately predict the length of the next
CPU burst. Starvation is possible.
Multilevel Queuing: An implementation (approximation) of
SJF.
Lottery Scheduling: Fairer with a low average waiting time,
but less predictable.
Our modeling assumed that context switches took no time,
which is unrealistic.
Next Time: Threads: multiple coordinating processes

40

Threads
Processes do not share resources well
high context switching overhead

Idea: Separate concurrency from protection

Multithreading: a single program made up of a number of
different concurrent activities
A thread (or lightweight process)
basic unit of CPU utilization; it consists of:
program counter, register set and stack space

A thread shares the following with peer threads:

code section, data section and OS resources (open files, signals)

No protection between threads

Collectively called a task.

Heavyweight process is a task with one thread.

41

Single and Multithreaded

Processes

Threads encapsulate concurrency: Active component

Address spaces encapsulate protection: Passive part
Keeps buggy program from trashing the system

42

Single and Multithreaded

Processes

Threads encapsulate concurrency: Active component

Address spaces encapsulate protection: Passive part
Keeps buggy program from trashing the system

43

Benefits
Responsiveness
Resource Sharing
Economy
Utilization of MP Architectures

44

Threads(Cont.)
In a multiple threaded task, while one server
thread is blocked and waiting, a second
thread in the same task can run.
Cooperation of multiple threads in the same job confers
higher throughput and improved performance.
Applications that require sharing a common buffer (i.e.
producer-consumer) benefit from thread utilization.

Threads provide a mechanism that allows

sequential processes to make blocking
system calls while also achieving parallelism.
45

Thread State
State shared by all threads in process/addr
space
Contents of memory (global variables, heap)
I/O state (file system, network connections, etc)

State private to each thread

Kept in TCB Thread Control Block
CPU registers (including, program counter)
Execution stack
Parameters, Temporary variables
return PCs are kept while called procedures are
executing
46

Threads (cont.)
Thread context switch still requires a register
set switch, but no memory management
related work!!
Thread states ready, blocked, running, terminated

Threads share CPU and only one thread can

run at a time.
No protection among threads.
47

Examples: Multithreaded
Embedded systems
programs

Elevators, Planes, Medical systems, Wristwatches

Single Program, concurrent operations

Most modern OS kernels

Internally concurrent because have to deal with
concurrent requests by multiple users
But no protection needed within kernel

Database Servers
Access to shared data by many concurrent users
Also background utility processing must be done

48

More Examples: Multithreaded

programs

Network Servers

Concurrent requests from network

Again, single program, multiple concurrent operations
File server, Web server, and airline reservation systems

Parallel Programming (More than one physical CPU)

Split program into multiple threads for parallelism
This is called Multiprocessing

49

# of addr
spaces:

One

Many

One

MS/DOS, early
Macintosh

Traditional UNIX

Many

Embedded systems
(Geoworks, VxWorks,
JavaOS,etc)
JavaOS, Pilot(PC)

Mach, OS/2, Linux

Windows 9x???
Win NT to XP, Solaris,
HP-UX, OS X

# threads
Per AS:

Real operating systems have either

One or many address spaces
One or many threads per address space
50

Types of Threads
Kernel-supported threads (Mach and OS/2)
User-level threads
Hybrid approach implements both user-level
and kernel-supported threads (Solaris 2).

51

Kernel Threads
Supported by the Kernel
Native threads supported directly by the kernel
Every thread can run or block independently
One process may have several threads waiting on different things

Downside of kernel threads: a bit expensive

Need to make a crossing into kernel mode to schedule

Examples
Windows XP/2000, Solaris, Linux,Tru64 UNIX,
Mac OS X, Mach, OS/2
52

User Threads
Supported above the kernel, via a set of library calls
at the user level.
Thread management done by user-level threads library
User program provides scheduler and thread package

May have several user threads per kernel thread

User threads may be scheduled non-premptively relative to
each other (only switch on yield())

Advantages
Cheap, Fast
Threads do not need to call OS and cause interrupts to kernel

Disadv: If kernel is single threaded, system call from any

thread can block the entire task.

Example thread libraries:

POSIX Pthreads, Win32 threads, Java threads
53

Multithreading Models
Many-to-One
One-to-One
Many-to-Many

54

Many-to-One
Many user-level threads mapped to single
kernel thread
Examples:
Solaris Green Threads
GNU Portable Threads

55

One-to-One
Each user-level thread maps to kernel thread

Examples
Windows NT/XP/2000; Linux; Solaris 9 and later
56

Many-to-Many Model
Allows many user level
threads to be mapped to
many kernel threads
Allows the operating
system to create a
sufficient number of
kernel threads
Solaris prior to version 9
Windows NT/2000 with
the ThreadFiber package

57

Threads in Solaris 2

58

Threads in Solaris 2
Resource requirements of thread types
Kernel Thread: small data structure and stack; thread
switching does not require changing memory access
information - relatively fast.
Lightweight Process: PCB with register data, accounting
and memory information - switching between LWP is
relatively slow.
User-level thread: only needs stack and program
counter; no kernel involvement means fast switching.
Kernel only sees the LWPs that support user-level
threads.

59

Two-level Model
Similar to M:M, except that it allows a user
thread to be bound to kernel thread
Examples
IRIX, HP-UX, Tru64 UNIX, Solaris 8 and earlier

60

Threading Issues
Semantics of fork() and exec() system
calls
Thread cancellation
Signal handling
Thread pools
Thread specific data
Scheduler activations

61

Multi( processing, programming,

Definitions:
threading)
Multiprocessing Multiple CPUs
Multiprogramming Multiple Jobs or Processes
Multithreading Multiple threads per Process

What does it mean to run two threads concurrently?

Scheduler is free to run threads in any order and interleaving: FIFO, Random,

Dispatcher can choose to run each thread to completion or time-slice in big

chunks or small chunks
A
Multiprocessing

B
C

A
Multiprogramming

B
B

C
B

62

Cooperating Processes
Concurrent Processes can be
Independent processes
cannot affect or be affected by the execution of another
process.
Cooperating processes
can affect or be affected by the execution of another process.

Advantages of process cooperation:

Information sharing
Computation speedup
Modularity
Convenience(e.g. editing, printing, compiling)

Concurrent execution requires

process communication and process synchronization

63

Interprocess Communication
(IPC)
Proc 1

Proc 2

Proc 3

Separate address space isolates processes

High Creation/Memory Overhead; (Relatively) High Context-Switch Overhead

Mechanism for processes to communicate and synchronize actions.

Via shared memory - Accomplished by mapping addresses to common DRAM

Read and Write through memory

Via Messaging system - processes communicate without resorting to

shared variables.

send() and receive() messages

Can be used over the network!

Messaging system and shared memory not mutually exclusive

can be used simultaneously within a single OS or a single process.

64

Shared Memory
Data 2
Communication
Code
Data
Heap
Stack
Shared
Prog 1
Virtual
Address
Space 1

Stack 1
Heap 1
Code 1
Stack 2
Data 1
Heap 2
Code 2
Shared

Code
Data
Heap
Stack
Shared
Prog 2
Virtual
Address
Space 2

Communication occurs by simply reading/writing to shared

address page
Really low overhead communication
Introduces complex synchronization problems
65

Cooperating Processes via

Message Passing
IPC facility provides two operations.
send(message) - message size can be fixed or variable
receive(message)

If processes P and Q wish to communicate, they

need to:
establish a communication link between them
exchange messages via send/receive

Fixed vs. Variable size message

Fixed message size - straightforward physical implementation,
programming task is difficult due to fragmentation
Variable message size - simpler programming, more complex
physical implementation.
66

Implementation Questions
How are links established?
Can a link be associated with more than 2
processes?
How many links can there be between every pair
of communicating processes?
What is the capacity of a link?
Fixed or variable size messages?
Unidirectional or bidirectional links?
.
67

Direct Communication
Sender and Receiver processes must name
each other explicitly:
send(P, message) - send a message to process P
receive(Q, message) - receive a message from process Q

Properties of communication link:

Links are established automatically.
A link is associated with exactly one pair of communicating
processes.
Exactly one link between each pair.
Link may be unidirectional, usually bidirectional.

68

Indirect Communication
Messages are directed to and received from
mailboxes (also called ports)

Unique ID for every mailbox.

Processes can communicate only if they share a mailbox.
Send(A, message) /* send message to mailbox A */
Receive(A, message) /* receive message from mailbox A */

Properties of communication link

Link established only if processes share a common mailbox.

Link can be associated with many processes.
Pair of processes may share several communication links
Links may be unidirectional or bidirectional

69

Indirect Communication
using mailboxes

70

Mailboxes (cont.)
Operations

create a new mailbox

send/receive messages through mailbox
destroy a mailbox

Issue: Mailbox sharing

P1, P2 and P3 share mailbox A.

P1 sends message, P2 and P3 receive who gets
message??

Possible Solutions

disallow links between more than 2 processes

allow only one process at a time to execute receive operation
allow system to arbitrarily select receiver and then notify
sender.

71

Message Buffering
Link has some capacity - determine the
number of messages that can reside
temporarily in it.
Queue of messages attached to link
Zero-capacity Queues: 0 messages
sender waits for receiver (synchronization is called
rendezvous)
Bounded capacity Queues: Finite length of n messages
sender waits if link is full
Unbounded capacity Queues: Infinite queue length
sender never waits
72

Message Problems Exception Conditions

Process Termination
Problem: P(sender) terminates, Q(receiver) blocks forever.
Solutions:
System terminates Q.
System notifies Q that P has terminated.
Q has an internal mechanism(timer) that determines how long to wait
for a message from P.

Problem: P(sender) sends message, Q(receiver) terminates.

In automatic buffering, P sends message until buffer is full or
forever. In no-buffering scheme, P blocks forever.
Solutions:
System notifies P
System terminates P
P and Q use acknowledgement with timeout

73

Message Problems Exception Conditions

Lost Messages
OS guarantees retransmission
sender is responsible for detecting it using timeouts
sender gets an exception

Scrambled Messages
Message arrives from sender P to receiver Q, but
information in message is corrupted due to noise in
communication channel.
Solution
need error detection mechanism, e.g. CHECKSUM
need error correction mechanism, e.g. retransmission

74

Producer-Consumer Problem
Paradigm for cooperating processes;
producer process produces information that is
consumed by a consumer process.

We need buffer of items that can be filled by

producer and emptied by consumer.
Unbounded-buffer places no practical limit on the size of
the buffer. Consumer may wait, producer never waits.
Bounded-buffer assumes that there is a fixed buffer size.
Consumer waits for new item, producer waits if buffer is full.

Producer and Consumer must synchronize.

75

Producer-Consumer Problem

76

Bounded Buffer using IPC

(messaging)
Producer
repeat

produce an item in nextp;

send(consumer, nextp);
until false;

Consumer
repeat
receive(producer, nextc);

consume item from nextc;

until false;
77

Bounded-buffer - Shared
Memory Solution
Shared data
var n;
type item = .;
var buffer: array[0..n-1] of item;
in, out: 0..n-1;
in :=0; out:= 0; /* shared buffer = circular array */
/* Buffer empty if in == out */
/* Buffer full if (in+1) mod n == out */
/* noop means do nothing */

78

Bounded Buffer - Shared

Memory Solution
Producer process - creates filled buffers
repeat

produce an item in nextp

while in+1 mod n = out do noop;

buffer[in] := nextp;
in := in+1 mod n;
until false;

79

Bounded Buffer - Shared

Memory Solution
Consumer process - Empties filled buffers
repeat
while in = out do noop;
nextc := buffer[out] ;
out:= out+1 mod n;

consume the next item in nextc

until false

80