Dynamic Programming

1
Content
What is Dynamic Programming?
Knapsack Problem
All-Pairs Shortest Path Problem

2
Dynamic Programming
What is Dynamic Programming?
3
What is Dynamic Programming?
Dynamic Programming (DP) tends to break the
original problem to sub-problems, i.e., in a smaller
size
The optimal solution in the bigger sub-problems is
found through a retroactive formula which
connects the optimal solutions of sub-problems.

Used when the solution to a problem may be
viewed as the result of a sequence of decisions.
4
Properties for Problems Solved by DP
Simple Subproblems

The original problem can be broken into smaller
subproblems with the same structure

Optimal Substructure of the problems

The solution to the problem must be a
composition of subproblem solutions (the
principle of optimality)

Subproblem Overlap

Optimal subproblems to unrelated problems can
contain subproblems in common
5
The Principle of Optimality
The basic principle of dynamic programming
Developed by Richard Bellman
An optimal path has the property that whatever
the initial conditions and control variables (choices)
over some initial period, the control (or decision
variables) chosen over the remaining period must
be optimal for the remaining problem, with the
state resulting from the early decisions taken to
be the initial condition.
6
Example: Shortest Path Problem
10
5
3
Goal Start
7
25
28
40
Example: Shortest Path Problem
10
5
3
Start Goal
8
25
28
40
Example: Shortest Path Problem
10
5
3
Start Goal
9
Recall Greedy Method for
Shortest Paths on a Multi-stage Graph
Problem
Find a shortest path from v
0
to v
3

Is the greedy solution optimal?
10
Recall Greedy Method for
Shortest Paths on a Multi-stage Graph
Problem
Find a shortest path from v
0
to v
3

Is the greedy solution optimal?
The optimal path

11
Example Dynamic Programming
min 1,1 3
min 1,2 3
min 0 3
min 1,3 3
min 1,4 3
( , )
( , )
( , )
( , )
( )
3
5
,
1
7
d v v
d v v
d v v
d v v
d v v
+

=

+

12
Dynamic Programming
Knapsack Problem
13
Knapsack Problems
Given some items, pack the knapsack to get the maximum
total value. Each item has some weight and some benefit.
Total weight that we can carry is no more than some fixed
capacity.
Fractional knapsack problem
Items are divisible: you can take any fraction of an item.
Solved with a greedy algorithm.
0-1 knapsack problem
Items are indivisible; you either take an item or not.
Solved with dynamic programming.
14
0-1 Knapsack Problem
Given a knapsack with maximum capacity
W, and a set S consisting of n items
Each item i has some weight w
i
and
benefit value b
i
(all w
i


and W are integer
values)
Problem: How to pack the knapsack to
achieve maximum total value of packed
items?
15
Example: 0-1 Knapsack Problem
Which boxes should be chosen
to maximize the amount of
money while still keeping the
overall weight under 15 kg ?
16
Example: 0-1 Knapsack Problem
1
{0,1} x e
2
{0,1} x e
3
{0,1} x e
5
{0,1} x e
4
{0,1} x e
Objective Function

Unknowns or Variables

Constraints
's, 1, , 5
i
x i =
{ }
, , 0,1 1, 5
i
x i e =
1 2 3 4 5
4 2 10 2 1
Maximize
x x x x x + + + +
1 2 3 4 5
12 1
Subject t
15 4 2
o
1 x x x x x + + s + +
17
Formulation: 0-1 Knapsack Problem
1
1
Maximize
Subject t
{0,1}
o
n
i i
i
n
i i
i
i
b x
w x W
x
=
=
s
e

18
0-1 Knapsack Problem:
Brute-Force Approach
Since there are n items, there are 2
n

possible combinations of items.
We go through all combinations and
find the one with maximum value and
with total weight less or equal to W
Running time will be O(2
n
)
19
DP Define Subproblem
Suppose that items are labeled 1,..., n.

Define a subproblem, say, P
k
as to finding
an optimal solution for items in S
k
= {1, 2,...,
k}.
original problem is P
n
.


Is such a scheme workable?
Is the principle of optimality held?
20
A Counterexample
1. 2kgs, 3$
2. 3kgs, 4$
3. 4kgs, 5$
4. 5kgs, 8$
5. 9kgs, 10$
20 kgs
P
1
P
2
P
3
P
4
P
5
21
A Counterexample
Sub-
problem

Optimum Value
P
1
1 3$
P
2
1, 2 7$
P
3
1, 2, 3 12$
P
4
1, 2, 3, 4 20$
P
5
1, 3, 4, 5 26$
1. 2kgs, 3$
2. 3kgs, 4$
3. 4kgs, 5$
4. 5kgs, 8$
5. 9kgs, 10$
20 kgs
22
A Counterexample
Sub-
problem

Optimum Value
P
1
1 3$
P
2
1, 2 7$
P
3
1, 2, 3 12$
P
4
1, 2, 3, 4 20$
P
5
1, 3, 4, 5 26$
1. 2kgs, 3$
2. 3kgs, 4$
3. 4kgs, 5$
4. 5kgs, 8$
5. 9kgs, 10$
20 kgs
Solution for P
4
is not part of
the solution for P
5
!!!
23
DP Define Subproblem
Suppose that items are labeled 1,..., n.

Define a subproblem, say, P
k
as to finding
an optimal solution for items in S
k
= {1, 2,...,
k}.
original problem is P
n
.


Is such a scheme workable?
Is the principle of optimality held?


24
DP Define Subproblem
Suppose that items are labeled 1,..., n.
Define a subproblem, say, P
k,w
as to
finding an optimal solution for items in S
k

= {1, 2,..., k} and with total weight no more
than w.
original problem is P
n,W
.


Is such a scheme workable?
Is the principle of optimality held?
New version
25
DP Principle of Optimality
( )
1,
,
1, 1,
if
max , if
k
k w k
k w
k w k w w k k
B w w
B
B B b w w

>

=

+ s

Denote the benefit for the optimal solution

of P
k,w
as B
k,w
.
26
DP Principle of Optimality
( )
1,
,
1, 1,
if
max , if
k
k w k
k w
k w k w w k k
B w w
B
B B b w w

>

=

+ s

Denote the benefit for the optimal solution

of P
k,w
as B
k,w
.
In this case, it is
impossible to include the k
th

object.



Not include
the k
th
object



include
the k
th
object
There are two
possible
choices.
27
Example
w

k
0 1 2 3 4
0
1
2
3
4
5
1. 2kgs, 3$
2. 3kgs, 4$
3. 4kgs, 5$
4. 5kgs, 6$
5 kgs
( )
1,
,
1, 1,
if
max , if
k
k w k
k w
k w k w w k k
B w w
B
B B b w w

>

=

+ s

28
( )
1,
,
1, 1,
if
max , if
k
k w k
k w
k w k w w k k
B w w
B
B B b w w

>

=

+ s

Example
w

k
0 1 2 3 4
0
1
2
3
4
5
1. 2kgs, 3$
2. 3kgs, 4$
3. 4kgs, 5$
4. 5kgs, 6$
5 kgs
0 0 0 0 0
0
0
0
0
0
Step 1. Setup table and initialize
base conditions.
29
( )
1,
,
1, 1,
if
max , if
k
k w k
k w
k w k w w k k
B w w
B
B B b w w

>

=

+ s

Example
w

k
0 1 2 3 4
0
1
2
3
4
5
1. 2kgs, 3$
2. 3kgs, 4$
3. 4kgs, 5$
4. 5kgs, 6$
5 kgs
0 0 0 0 0
0
0
0
0
0
0
3
3
3
3
Step 2. Fill all table entries
progressively.
30
( )
1,
,
1, 1,
if
max , if
k
k w k
k w
k w k w w k k
B w w
B
B B b w w

>

=

+ s

Example
w

k
0 1 2 3 4
0
1
2
3
4
5
1. 2kgs, 3$
2. 3kgs, 4$
3. 4kgs, 5$
4. 5kgs, 6$
5 kgs
0 0 0 0 0
0
0
0
0
0
0
3
3
3
3
0
3
4
4
7
Step 2. Fill all table entries
progressively.
31
( )
1,
,
1, 1,
if
max , if
k
k w k
k w
k w k w w k k
B w w
B
B B b w w

>

=

+ s

Example
w

k
0 1 2 3 4
0
1
2
3
4
5
1. 2kgs, 3$
2. 3kgs, 4$
3. 4kgs, 5$
4. 5kgs, 6$
5 kgs
0 0 0 0 0
0
0
0
0
0
0
3
3
3
3
0
3
4
4
7
0
3
4
5
7
Step 2. Fill all table entries
progressively.
32
( )
1,
,
1, 1,
if
max , if
k
k w k
k w
k w k w w k k
B w w
B
B B b w w

>

=

+ s

Example
w

k
0 1 2 3 4
0
1
2
3
4
5
1. 2kgs, 3$
2. 3kgs, 4$
3. 4kgs, 5$
4. 5kgs, 6$
5 kgs
0 0 0 0 0
0
0
0
0
0
0
3
3
3
3
0
3
4
4
7
0
3
4
5
7
0
3
4
5
7
Step 2. Fill all table entries
progressively.
33
( )
1,
,
1, 1,
if
max , if
k
k w k
k w
k w k w w k k
B w w
B
B B b w w

>

=

+ s

Example
w

k
0 1 2 3 4
0
1
2
3
4
5
1. 2kgs, 3$
2. 3kgs, 4$
3. 4kgs, 5$
4. 5kgs, 6$
5 kgs
0 0 0 0 0
0
0
0
0
0
0
3
3
3
3
0
3
4
4
7
0
3
4
5
7
0
3
4
5
7
Step 3. Trace back
34
Pseudo-Polynomial Time Algorithm
The time complexity for 0-1 knapsack
using DP is O(Wn).

Not a polynomial-time algorithm if W is
large.

This is a pseudo-polynomial time
algorithm.
35
Dynamic Programming
All-Pairs
Shortest Path Problem
36
All-Pairs Shortest Path Problem
Given weighted graph G(V,E), we want
to determine the cost d
ij
of the
shortest path between each pair of
nodes in V.
37
V
k
Floyd's Algorithm
Let be the minimum cost of a path from node i
to node j, using only nodes in V
k
={v
1
,,v
k
}.
k
ij
d
k

V
k1

i

j

1 k
ij
d

1 k
ik
d

1 k
kj
d

( )
1 1 1
min , 0
0
k k k
ij ik kj
k
ij
ij
d d d k
d
c k

+ >

The all-pairs shortest path problem

is to find all paths with costs

n
ij ij
d d i j = =
38
Floyd's Algorithm
( )
1 1 1
min , 0
0
k k k
ij ik kj
k
ij
ij
d d d k
d
c k

+ >

Input Parameter: D
Output Parameter: D, next
all_paths(D, next)
n = D.NumberOfRows;
// initialize next: if no intermediate
// vertices are allowed next[i][j] = j
for i = 1 to n
for j = 1 to n
next[i][j] = j;
for k = 1 to n // compute D(k)
for i = 1 to n
for j = 1 to n
if (D[i][k] + D[k][j] < D[i][j])
D[i][j] = D[i][k] + D[k][j];
next[i][j] = next[i][k];
O(n
3
)
39
Floyd's Algorithm
Input Parameters: next, i, j
Output Parameters: None
print_path(next, i, j) {
// if no intermediate vertices, just
// print i and j and return
if (j == next[i][j]) {
print(i + + j);
return;
}
// output i and then the path from the vertex
// after i (next[i][j]) to j
print(i + );
print_path(next,next[i][j], j);
}
40