Shortest paths

What is the shortest path from a to d?

Shortest paths

BFS

Shortest paths

What is the shortest path from a to d?

3
3

A
1

E
4

Shortest paths

We can still use BFS

3
3

A
1

E
4

Shortest paths

We can still use BFS

3
3

A
1

E
4

Shortest paths

We can still use BFS

Shortest paths

What is the problem?

Shortest paths

Running time is dependent on the weights

100

A
4

50

A
200

Shortest paths

B
100
50

A
200

C
C

Shortest paths

Shortest paths

Shortest paths

Nothing will change as we expand the frontier

until weve gone out 100 levels
B

Dijkstras algorithm

Dijkstras algorithm

Dijkstras algorithm
prev keeps track of
the shortest path

Dijkstras algorithm

Dijkstras algorithm

Dijkstras algorithm

Single source shortest paths

All of the shortest path algorithms well look

at today are call single source shortest
paths algorithms
Why?

3
1

A
1

E
4

3
1
1

A
1

C
4

Heap

A
B
C
D
E

1
1

A
1

C
4

Heap

B
C
D
E

1
1

A
1

C
4

Heap

B
C
D
E

1
1

A
1

C
4

Heap

C
B
D
E

1
1

A
1

C
4

Heap

C
B
D
E

1
1

A
1

C
4

Heap

C
B
D
E

3
3

1
1

A
1

C
4

1
3

Heap

C
B
D
E

3
3

1
1

A
1

C
4

1
3

Heap

B 3
D
E

3
3

1
1

A
1

C
4

Heap

B 3
D
E

3
3

1
1

A
1

C
4

Heap

B 3
D
E

3
3

1
1

A
1

C
4

Heap

B 2
D
E

2
3

1
1

A
1

C
4

Heap

B 2
D
E

2
3

1
1

A
1

C
4

Heap

B 2
E 5
D

2
3

1
1

A
1

E 5

C
4

Heap

B 2
E 5
D

2
3

1
1

A
1

E 5

C
4

Frontier?

Heap

B 2
E 5
D

2
3

1
1

A
1

E 5

C
4

All nodes reachable

from starting node
within a given distance

Heap

E 3
D 5
2

5
3

1
1

A
1

E 3

C
4

Heap

D 5
2

5
3

1
1

A
1

E 3

C
4

Heap

5
3

1
1

A
1

E 3

C
4

Heap

1
1

A
1

E 3

Is Dijkstras algorithm correct?

Invariant:

Is Dijkstras algorithm correct?

Invariant: For every vertex removed from the heap,

dist[v] is the actual shortest distance from s to v

Is Dijkstras algorithm correct?

Invariant: For every vertex removed from the

heap, dist[v] is the actual shortest distance
from s to v

The only time a vertex gets visited is when the

distance from s to that vertex is smaller than the
distance to any remaining vertex
Therefore, there cannot be any other path that
hasnt been visited already that would result in a
shorter path

Running time?

Running time?

1 call to MakeHeap

Running time?

|V| iterations

Running time?

|V| calls

Running time?

O(|E|) calls

Running time?

Depends on the heap implementation

1 MakeHeap

|V| ExtractMin

|E| DecreaseKey

Array

O(|V|)

O(|V|2)

O(|E|)

Bin heap

O(|V|)

O(|V| log |V|)

O(|E| log |V|)

Total
O(|V|2)

O((|V|+|E|) log |V|)

O(|E| log |V|)

Running time?

Depends on the heap implementation

1 MakeHeap

|V| ExtractMin

|E| DecreaseKey

Array

O(|V|)

O(|V|2)

O(|E|)

Bin heap

O(|V|)

O(|V| log |V|)

O(|E| log |V|)

Total
O(|V|2)

O((|V|+|E|) log |V|)

O(|E| log |V|)

Is this an improvement?

If |E| < |V|2 / log |V|

Running time?

Depends on the heap implementation

1 MakeHeap

|V| ExtractMin

|E| DecreaseKey

Array

O(|V|)

O(|V|2)

O(|E|)

Bin heap

O(|V|)

O(|V| log |V|)

O(|E| log |V|)

Total
O(|V|2)

O((|V|+|E|) log |V|)

O(|E| log |V|)

Fib heap

O(|V|)

O(|V| log |V|)

O(|E|)

O(|V| log |V| + |E|)

What about Dijkstras on?

D
5

10

-10

What about Dijkstras on?

D
5

10

-10

What about Dijkstras on?

Dijkstras algorithm only works
for positive edge weights
1

D
5

10

Bounding the distance

Another invariant: For each vertex v, dist[v]

is an upper bound on the actual shortest
distance

start of at
only update the value if we find a shorter distance

An update procedure

dist[v] min{dist[v], dist[u] w(u, v)}

dist[v] min{dist[v], dist[u] w(u, v)}

Can we ever go wrong applying this update

rule?

We can apply this rule as many times as we want

and will never underestimate dist[v]

When will dist[v] be right?

If u is along the shortest path to v and dist[u] is

correct

dist[v] min{dist[v], dist[u] w(u, v)}

dist[v] will be right if u is along the shortest

path to v and dist[u] is correct
Consider the shortest path from s to v

p1

p2

p3

pk

dist[v] min{dist[v], dist[u] w(u, v)}

dist[v] will be right if u is along the shortest

path to v and dist[u] is correct
What happens if we update all of the vertices
with the above update?

p1

p2

p3

pk

dist[v] min{dist[v], dist[u] w(u, v)}

dist[v] will be right if u is along the shortest

path to v and dist[u] is correct
What happens if we update all of the vertices
with the above update?

s
correct

p1

p2

p3

pk

dist[v] min{dist[v], dist[u] w(u, v)}

dist[v] will be right if u is along the shortest

path to v and dist[u] is correct
What happens if we update all of the vertices
with the above update?

p1

correct

correct

p2

p3

pk

dist[v] min{dist[v], dist[u] w(u, v)}

dist[v] will be right if u is along the shortest

path to v and dist[u] is correct
Does the order that we update the vertices
matter?

p1

correct

correct

p2

p3

pk

dist[v] min{dist[v], dist[u] w(u, v)}

dist[v] will be right if u is along the shortest path to v

and dist[u] is correct
How many times do we have to do this for vertex pi
to have the correct shortest path from s?

i times

p1

p2

p3

pk

dist[v] min{dist[v], dist[u] w(u, v)}

dist[v] will be right if u is along the shortest path to v

and dist[u] is correct
How many times do we have to do this for vertex pi
to have the correct shortest path from s?

i times

p1

correct

correct

p2

p3

pk

dist[v] min{dist[v], dist[u] w(u, v)}

dist[v] will be right if u is along the shortest path to v

and dist[u] is correct
How many times do we have to do this for vertex pi
to have the correct shortest path from s?

i times

p1

p2

correct

correct

correct

p3

pk

dist[v] min{dist[v], dist[u] w(u, v)}

dist[v] will be right if u is along the shortest path to v

and dist[u] is correct
How many times do we have to do this for vertex pi
to have the correct shortest path from s?

i times

p1

correct

correct

p2

p3

correct correct

pk

dist[v] min{dist[v], dist[u] w(u, v)}

dist[v] will be right if u is along the shortest path to v

and dist[u] is correct
How many times do we have to do this for vertex pi
to have the correct shortest path from s?

i times

p1

correct

correct

p2

p3

correct correct

pk

dist[v] min{dist[v], dist[u] w(u, v)}

dist[v] will be right if u is along the shortest

path to v and dist[u] is correct
What is the longest (vetex-wise) the path from
s to any node v can be?

|V| - 1 edges/vertices

p1

correct

correct

p2

p3

correct correct

pk

Bellman-Ford algorithm

Bellman-Ford algorithm
Initialize all the
distances

Bellman-Ford algorithm

iterate over all

edges/vertices
and apply update
rule

Bellman-Ford algorithm

Bellman-Ford algorithm

check for negative

cycles

Negative cycles
What is the shortest path
from a to e?
1

D
5

10

-10

C
3

Bellman-Ford algorithm

Bellman-Ford algorithm
S

10

How many edges is

the shortest path
from s to:

A
1

-4

A:

2
1

1
-2

C
3

-1

D
-1

Bellman-Ford algorithm
S

10

How many edges is

the shortest path
from s to:

A
1

-4

A:

2
1

1
-2

C
3

-1

D
-1

Bellman-Ford algorithm
S

10

How many edges is

the shortest path
from s to:

A
1

-4

A:

2
1

1
-2

C
3

-1

D
-1

B:

Bellman-Ford algorithm
S

10

How many edges is

the shortest path
from s to:

A
1

-4

B
2

A:

B:

1
-2

C
3

-1

D
-1

Bellman-Ford algorithm
S

10

How many edges is

the shortest path
from s to:

A
1

-4

B
2

A:

B:

1
-2

C
3

-1

D
-1

D:

Bellman-Ford algorithm
S

10

How many edges is

the shortest path
from s to:

A
1

-4

B
2

A:

B:

D:

1
-2

C
3

-1

D
-1

Bellman-Ford algorithm

10

Iteration: 0
1

-4

B
2
1

-2

C
3

-1

D
-1

Bellman-Ford algorithm
0

10

10

Iteration: 1
1

-4

B
2
1

-2

C
3

-1

D
-1

Bellman-Ford algorithm
0

10

10

Iteration: 2
1

-4

B
2
1

1
9

-2

C
3

-1

D
-1

12

Bellman-Ford algorithm
0

10

Iteration: 3
1

10
8

-4

2
1

1
9

A has the correct

distance and path

-2

C
3

-1

D
-1

Bellman-Ford algorithm
0

10

Iteration: 4
1

6
8

-4

B
2
1

1
9

-2

C
3

-1

D
-1

11

Bellman-Ford algorithm
0

10

Iteration: 5
1

5
8

-4

2
1

1
9

B has the correct

distance and path

-2

C
3

-1

D
-1

14

Bellman-Ford algorithm
0

10

Iteration: 6
1

5
8

-4

B
2
1

1
9

-2

C
3

-1

D
-1

10

Bellman-Ford algorithm
0

10

Iteration: 7
1

5
8

-4

2
1

1
9

D (and all other

nodes) have the
correct distance
and path

-2

C
3

-1

D
-1

Correctness of Bellman-Ford

Loop invariant:

Correctness of Bellman-Ford

Loop invariant: After iteration i, all vertices with

shortest paths from s of length i edges or less have
correct distances

Runtime of Bellman-Ford

O(|V| |E|)

Runtime of Bellman-Ford

Can you modify the algorithm to run

faster (in some circumstances)?

All pairs shortest paths

Simple approach

Call Bellman-Ford |V| times

O(|V|2 |E|)

Floyd-Warshall (|V|3)
Johnsons algorithm O(|V|2 log |V| + |V| |E|)

Minimum spanning trees

What is the lowest weight set of edges that

connects all vertices of an undirected graph
with positive weights

Input: An undirected, positive weight graph,

G=(V,E)
Output: A tree T=(V,E) where E E that
minimizes

weight (T ) we
eE '

MST example
1

A
3

C
4

E
2

D
4

F
6

C
2

MSTs

Can an MST have a cycle?

A

C
2

MSTs

Can an MST have a cycle?

A

C
2

Applications?

Connectivity

Networks (e.g. communications)

Circuit desing/wiring

hub/spoke models (e.g. flights,

transportation)
Traveling salesman problem?

Cuts

A cut is a partitioning of the vertices into two sets S

and V-S
An edges crosses the cut if it connects a vertex
uV and vV-S
1

A
3

C
4

E
2

D
4

F
6

Minimum cut property

Given a partion S, let edge e be the minimum

cost edge that crosses the partition. Every
minimum spanning tree contains edge e.
S

V-S
e

Consider an MST with edge e that is not the minimum edge

Minimum cut property

Given a partion S, let edge e be the minimum

cost edge that crosses the partition. Every
minimum spanning tree contains edge e.
S

V-S
e

Using e instead of e, still connects the graph,

but produces a tree with smaller weights

Algorithm ideas?

Given a partion S, let edge e be the minimum cost

edge that crosses the partition. Every minimum
spanning tree contains edge e.

Kruskals algorithm
Add smallest edge that connects
two sets not already connected
1

A
3

C
4

E
2

MST
B

D
4

F
6

Kruskals algorithm
Add smallest edge that connects
two sets not already connected
1

A
3

C
4

E
2

MST
B

D
4

F
6

G
B

Kruskals algorithm
Add smallest edge that connects
two sets not already connected
1

A
3

C
4

E
2

MST
B

D
4

F
6

E
2

Kruskals algorithm
Add smallest edge that connects
two sets not already connected
1

A
3

C
4

E
2

MST
B

D
4

F
6

Kruskals algorithm
Add smallest edge that connects
two sets not already connected
1

A
3

C
4

E
2

MST
B

D
4

F
6

Kruskals algorithm
Add smallest edge that connects
two sets not already connected
1

A
3

C
4

E
2

MST
B

D
4

F
6

Correctness of Kruskals

Never adds an edge that connects already

connected vertices
Always adds lowest cost edge to connect two sets.
By min cut property, that edge must be part of the
MST

Running time of Kruskals

|V| calls to MakeSet

Running time of Kruskals

O(|E| log |E|)

Running time of Kruskals

2 |E| calls to FindSet

Running time of Kruskals

|V| calls to Union

Running time of Kruskals

Disjoint set data structure

O(|E| log |E|) +

MakeSet

Linked lists

|V|

FindSet
|E| calls
O(|V| |E|)

Union
|V| calls
|V|

Total

O(|V||E| + |E| log |E|)

O(|V| |E|)

Linked lists +
heuristics

|V|

O(|E| log |V|)

|V|

O(|E| log |V|+ |E| log |E|)

O(|E| log |E| )

Prims algorithm

Prims algorithm

Prims algorithm

Prims algorithm

Start at some root node and build out the MST by

adding the lowest weighted edge at the frontier

Prims
1

A
3

C
4

E
2

D
4

F
6

MST
A

Prims
1

A
3

C
4

E
2

D
4

F
6

MST

Prims
1

A
3

C
4

E
2

D
4

F
6

MST

Prims
1

A
3

C
4

E
2

D
4

F
6

MST

Prims
1

A
3

C
4

E
2

D
4

F
6

MST
1

Prims
1

A
3

C
4

E
2

D
4

F
6

MST
1

Prims
1

A
3

C
4

E
2

D
4

F
6

MST
1

Prims
1

A
3

C
4

E
2

D
4

F
6

MST
1

Prims
1

A
3

C
4

E
2

D
4

F
6

MST
1

Prims
1

A
3

C
4

E
2

D
4

F
6

MST
1

Prims
1

A
3

C
4

E
2

D
4

F
6

MST
1

Prims
1

A
3

C
4

E
2

D
4

F
6

MST
1

Correctness of Prims?

Can we use the min-cut property?

Given a partion S, let edge e be the minimum cost

edge that crosses the partition. Every minimum
spanning tree contains edge e.

Let S be the set of vertices visited so far

The only time we add a new edge is if its the
lowest weight edge from S to V-S

Running time of Prims

(|V|)

Running time of Prims

(|V|)

Running time of Prims

|V| calls to Extract-Min

Running time of Prims

|E| calls to Decrease-Key

Running time of Prims

Same as Dijkstas algorithm

1 MakeHeap

|V| ExtractMin

|E| DecreaseKey

Array

O(|V|)

O(|V|2)

O(|E|)

Bin heap

O(|V|)

O(|V| log |V|)

O(|E| log |V|)

Total
O(|V|2)

O((|V|+|E|) log |V|)

O(|E| log |V|)

Fib heap

O(|V|)

O(|V| log |V|)

O(|E|)

O(|V| log |V| + |E|)

Kruskals: O(|E| log |E| )