NET 4010: Client Server and Distributed Systems

Challenges in Designing and Building

Distributed Systems III:
Time Ordering of Events: Clock Synchronization

Introduction
Several challenges have to be handled in designing and building
distributed systems
Heterogeneity
Transparency
Scalability
Fault Tolerance

Openness
Security
Concurrency
No Global Clock

Introduction
A common
Resource

Q. WHICH REQUEST
WAS MADE FIRST?

Solution
Request
A

Node
A

Request B

Node B

A Distributed
System

Global
Synchronizatio
n?

Introduction (contd)
Q. WHICH REQUEST
WAS MADE FIRST?
A common
Resource

request A
10:00 AM

Node A

A Distributed
System

Individual
Clocks?
request B
10:02 AM

Node B

Are individual
clocks
accurate,
precise?
One clock
might run
faster/slow
er?

Introduction (contd)
The concept of time is fundamental to our way of thinking. It is
derived from the more basic concept of the order in which events
occur.
The concept of the temporal ordering of events pervades our thinking
about systems.
For example, in an airline reservation system we specify that a
request for a reservation should be granted if it is made before the
flight is filled.
In a distributed system, it is
sometimes impossible to say that one
of two events occurred first.

Introduction (contd)

Synchronization in a Distributed System!

Event Ordering : Which event occurred first?
How to sync the clocks across the nodes?
Can we define notion of happened-before without
using physical clocks?

Synchronization of Processes

Clock Synchronization
Algorithms

Introduction

Process 1
Process 2
.
.
.
.

Client/Server
Cooperate
Synchronization
Compete

Process N
Shared
resource

Clock Synchronization
a timer mechanism
(computer clock)

Keep track of current time

Calculate time spent by a process in CPU use
Charging a user

Astronomical Time
Defined in terms of the rotation of the Earth on its axis
And its rotation about the Sun.

Basic time units: Seconds, Minutes, Hours, etc

based on astronomical time

Atomic Time
Based on atom Caesium- 133 (Cs133)
1second

9,192,631,770 periods of transition

two levels of the ground state.

Coordinated Universal Time (UTC)

International Standard for timekeeping;
Based on the atomic time (more accurate than
astronomical time)
UTC signals are synchronized and broadcast regularly
from land-based radio stations (e.g WWV in USA);
Also broadcast from satellites covering many parts of the
world (e.g. Global Position System, GPS)
Computers with receivers attached can synchronize their
clocks with these timing signals

Clock Drift
Let

t = real time (say, UTC time);

Cp(t) = time value of clock p;

= maximum drift rate allowable

Perfectly synchronized: Cp(t) = t for all p for all t.
dC = 1
dt
Nonfaulty
clock

dC
dt

1+

Perfect clock

dC = 1
dt

Fast clock region

dC
dt
Clock
time

Slow clock region

dC
dt

Real time
Fig. 1. Slow, perfect and fast clocks

Clock 1
Local Memory
CPU1

Clock 2
Local Memory
CPU2

Clock 3
Local Memory
CPU3

Communication Network

Distributed System Application

Necessary to synchronize the clocks, in two ways:
Externally;
Internally

External Synchronization
We need an authoritative, external source of time
(UTC). Consider a source, S, of UTC time.
The clocks Ci are accurate to within the bound D if:

S(t) Ci(t)

For i = 1, 2, .. N
and for all real times in I
(I is the interval of real time)

Objective: To know what time of day events occur at

processes in the distributed system.

Internal Synchronization
We need a consistent view of time across all nodes
in the distributed system.
The clocks Ci are accurate to within the bound D if:

Ci(t) Cj(t)

For i, j = 1, 2, .. N
and for all real times in I
(I is the interval of real time)

Objective: To measure the time interval between two

events occurring at different computers

Issues in Clock Synchronization

(a). No perfect synchronization the degree
Clock 1

C1(t)

Clock 2

C2(t)

Clock skew = C2(t) C1(t)

(b). A node must read a neighbour nodes time

Unpredictable communication delays

Prepare message;
Transmit message;

Minimum time
predictable

Receive reply.
Maximum time
unpredictable

Quiz 1
Consider a source, S, of UTC time (authoritative, external source of time) and three
other clocks C1, C2, and C3 that constitute a distributed system. The clock times are
given in Fig. Q1.
Confirm whether the clocks are both internally and externally synchronized, giving
statements of the basic rules being applied. Assume the bound for synchronization
is 0.005 seconds.

S:

16:34:23.480

C1:

16:34:23.430

C2:

16:34:23.725

C3:

16:34:23.728
Fig. Q1

Clock Synchronization Algorithms

Centralized Algorithms
Distributed Algorithms

Centralized Algorithms
Key features:
One node has a real-time receiver, time server node.
Clock time regarded as correct, reference time
Keep the clocks of all other nodes synchronized
with the reference time
Clock 1

Clock 2

Clock 3 (R)

Clock 4

Passive Time Server Centralized Algorithm

T0
Clock N

Clock 3 (R)

time = ?
time = T
T1

Adjust time to:

T + (T1 T0)/2

Time
Server
(T)

Active Time Server Centralized Algorithm

Time server periodically broadcasts its time, T, to the
other nodes.
Each node corrects/adjusts its time as:
T + Ta
Ta = approximate time to propagate information
about T from server

Drawbacks
(a). Single point of failure the time server;
(b). Scalability problem single time server.

Distributed algorithms handle

these drawbacks

Distributed Algorithms
Key features:
Node N broadcasts its local clock time;
Broadcasts dont happen at same time, because clocks
of different nodes run at different rates
The node (N) collects broadcasts from other nodes,
recording the time when received (according to its own
clock time)
Node N then estimates the clock skew of its own with the rest
Compute a fault-tolerant average

Summary
Clock synchronization is important:
Internal synchronization;
External synchronization
Synchronization Algorithms:
Centralized;
Distributed

Network Time Protocol

An architecture for time service;
Protocol to distribute time information over
the Internet

ntp.kenic.or.ke

NTP Architecture
Stratum 1

1
Stratum 2

2
Stratum 3

Fig. 1. Synchronization subnet in an NTP Implementation

Event Ordering
We can not synchronize clocks perfectly in a distributed
system;
Attempting to do so is expensive & non-trivial;
Thus, we can not use physical time to find out the order
of any arbitrary pair of events occurring within a
distributed system

Order is Important, Not Synchronization

If events can be ordered, perfect synchronization
may not be important.
Necessary to order events without
using physical time

Happened-Before Relation
e11 e23

Physical
Time
e13
e12
e11
e10
Process P1

e24
e23
e22
e21
e20
Process P2

e32

e31

Happened
-before
Concurrent

e30
Process P3

Fig. 3. Space-time diagram for three processes

e10 e21

Happened-Before Events
To determine that an event a happened-before event b,
we need:
either a common clock;
or a set of perfectly synchronized physical clocks.
None of which is available
in a distributed system

Logical Clock Concept [Lamport]

Associate a timestamp with each system event.

Process Pi
Clock Ci

event
a

Timestamp
Ci(a)

No relationship with physical time

Clock Condition
The timestamps assigned to the events by the system
of logical clocks must satisfy the following clock
condition:

For any events a and b, if a b, then C(a)<C(b).

Implementation
To satisfy the clock condition, the following conditions
should hold:
C1:

If a and b are two events within the same process Pi

and a occurs before b, then Ci(a)<Ci(b).

C2:

If a is the sending of a message by process Pi

and b is the receipt of that message by process Pj,
then Ci(a)<Cj(b).

Implementation (Contd)

C3:

A clock Ci associated with a process Pi must

always go forward, never backward. That is,
adding a positive value to the clock, never by
subtracting, must always be the way to make
corrections to time of a logical clock.

Rules of the Lamport Algorithm

To meet conditions C1, C2 and C3, Lamports algorithm
uses the following implementation rules:
IR1: Each process Pi increments Ci between any two
consecutive events
IR2: If event a is the sending of a message m by process Pi,
the message m contains a timestamp Tm = Ci(a),
and upon receiving the message m a process Pj sets
Cj greater than or equal to its present value but
greater than Tm

Rules of the Lamport Algorithm

Rule IR1:

ensures that condition C1 is met

Rule IR2:

ensures that condition C2 is met

Both rules IR1 & IR2:

ensures that conditions

condition C3 is met

The simple implementation rules IR1 & IR2

guarantee a correct system of logical clocks.

Time

Implementation Using Counters

C1 = 8

e08

C1 = 7

e07

e14 C2 = 6

C1 = 6

e06

e13

C1 = 5

e05
e04

C1 = 4
C1 = 3
C1 = 2
C1 = 1
C1 = 0

Tm = 6

Tm = 4

e03
e02
e01
Process P1

C2 = 3 = 5
(since 3 < Tm =
4)

e12

C2 = 2

e11

C2 = 1

Process P2

C2 = 0

Fig. 4. Implementation of Logical Clocks Using Counters

Summary: Problem
Assume two nonfaulty clocks (a slow clock and a fast clock).
Further, assume that t is the time elapsed after the two clocks were
synchronized.
(i). Determine the maximum deviation between the time value of the
two clocks at time t after they were synchronized.
(ii). To guarantee that no two clocks in a set of clocks ever differ by
more than , the clocks in the set must be resynchronized
periodically. Determine the interval between two synchronizations.