Dynamic Programming

Dynamic programming is used in solving combinatorial optimization (i.e. extremum of a function f among all input
combinations, while excluding combinations according to certain constraints), which involves a sequence of decisions,
each decision is a choice among multiple possibilities. The combinatorial problem can be represented as a tree : each
vertex represents the state after a unique sequence of decision made, while height of the vertex denotes the number of
decisions it has already gone through, children of a vertex form a set of all possible options for the next decision. The
set of options is path dependent, hence in general, each vertex has different number of children, which is somehow
related to the problem size. In short, the complexity of the problem can be described as a tree with tree height T and
problem size N. Please note : (1) root vertex v0 is regarded as the starting point with height zero, (2) for a vertex in the
middle of the tree, v0,a,b,c, denotes the state after selecting choice a for decision 1, followed for selecting choice b for
decision 2 and finally selecing choice c for decision 3 etc, (3) leaf vertex v0,1,2, T has a height of T, is one of the all
possible input combinations, thus all leaf vertices form a finite (but very large) feasible solution set, (4) since decision
option set is path dependent and the existence of constraints restricting the validity of some subtrees, the tree may
not be full nor complete. By evaluating function f at each leaf vertex, we can obtain the solution to the combinatorial
optimization. This is called Brute force method, which can be implemented by transversing the tree using either depth
first search or breadth first search. Here is an example with T equals to 4, where vn,m denotes a child of vn, whereas
vn,m,k denotes a child of vn,m etc.
height
0
1
2
3
4

vertices
v0
v0,0
v0,0,0
v0,0,1,0
v0,0,1,1,0

v0,1
v0,0,1
v0,0,2
v0,1,0
v0,1,1
v0,0,1,1 v0,0,1,2 v0,0,2,0 v0,0,2,1 v0,1,1,0
v0,0,1,1,1 v0,0,1,1,2 v0,0,2,0,0 v0,0,2,0,1

v0,1,1,1

v0,1,1,2

Fortunately, the combinatorial problem can be solved efficiently by dynamic programming if it can be reformulated in
a way such that (1) there exists optimal substructure, i.e. optimium of the original problem is constructed from the
optimium of one or multiple subproblem(s) and (2) the subproblems overlap with each other. Condition 1 enables us to
represent the original problem in a recursive way, while condition 2 implies that brute force method does have some
redundancy and hence there are rooms to speed up. Furthermore, reformulating the problem in recursive way means
transforming the original tree into a new tree with recursive shape, governed by the recursive formula. Please note
that vertex represent two different concepts in the trees : each leaf vertex in the brute force tree denote a possible input
combination in the combinatorial optimization, while each vertex in the dynamic programming tree denote the
optimium of a subproblem. Besides, height of Brute force tree T denotes the number of decision rounds, while height of
the dynamic programming tree denotes the number of iterations needed to reduce the original problem into simplest
form subproblems, please note that T cannot be visualised in the dynamic programming tree.
Problem dimension and problem size
Vertices in both the brute force tree and the dynamic programming tree are identified by index vector, each index is in
fact, a branch ID (or a child vertex ID). Index vector of brute force tree denotes the sequence of decision making, and
thus has a dimension of T+1 (where T is tree height). Index vector of dynamic programming tree denotes different ways
to reduce a problem into subproblems (in some example, this is also called the state vector), and thus has a smaller
dimension, usually 1, 2 or 3 (in some exmample, this is also called the state dimension). The number of branches in
brute force tree varies from vertex to, while the number of branches in dynamic programming tree is fixed (which is
governed by recursive formula). We define problem size as the maximum possible number of branches a vertex has.
Thus, problem size is equivalent to the number of states in dynamic programming tree.
Typical problem
maximum value contiguous subsequence
longest increasing subsequence
longest common subsequence
building bridge
edit distance
car assembly
balanced partition
two persons traversal of cities
bitonic tour
optimal game strategy
making change
(M types, N for each)
knapsack problem
(M types, N for each)
job scheduling
function approximation
bin packing
(n1+n2+n3 = N)
box stacking
boolean parenthesizations (N operators)
Bell number, Dijkstras shortest path
TraderRuns recursive least square
TraderRuns min max error problem

Brute force tree

dim-1 size
#leaves
2
N
N2
2
N
N2
N
2
22N
N
2
22N
N
2
*
N
2
2N
N-1
2
2N-1
N-1
2
2N-1
N-2
2
2N-2
N-2
2
2N-2
M
N
MN
M
N
MN
N
N-NPN = N!
M
N-NCM
N
N-?
N
N-N!3N
N
?
?

Dynamic programming tree

dim
size
1
N
1
N
2
NN
2
NN
2
NN
1
N
(2 trees)
2
NN
2
NN
(2 trees)
2
NN
(2 trees)
2
NN
1
C
1
W
1
N
2
NM
3
n1n2n3
2
N3
2
NN
(2 trees)

Note : N! > 2N > N2 > NlnN > N > lnN

where * = (NC0)2+(NC1)2+(NC2)2++(NCN)2 2N

There are multiple ways to do brute force search, which results in vertex with different dimensions and sizes, there are
also multiple ways to do dynamic programming, which also results in vertex with different dimensions and sizes, thus
the figures in the above table is just a typical value only. Suppose a T-round combinatorial optimization with problem
size N being reformulated as a two dimensional (for example) dynamic programming with problem size MN, then :
opt

it [1.N ]t[1,T ]

f (vi0 ,i1 ,i2 ..., iT )

opt

nN ,mM

f (un,m )

g0
opt
f (un,m )
nN 1,mM 1

or

g1
opt
f (un,m )
nN ,mM 1

or

g2
opt
f (un,m )
nN 1,mM

or other recursive forms

Maximum value contiguous subsequence

Given a sequence of N real numbers x1, x2, xN, find the maximum sum of a contiguous subsequence. If all numbers
are positive, then sum of the whole sequence is the result.
Brute force solution
Let n1 and n2 be the starting position and ending position of the optimal contiguous subsequence, such that n1,n2
[1,N]. The complexity of brute force search is thus O(N2). This problem can also be visualized in the form of a tree with
height 2. There is one root vertex at height 0, it has N children (denoting N choices for n1), while each vertex at height
1 has N children (denoting N choices for n2), forming a feasible solution with N2 leaves.
Dynamic programming (failed)
Let f(n) be the optimum of the problem with {x1, x2, xn}, in order to reduce it into subproblems, we also need to know
whether f(n-1) includes xn-1, thus we define g(n) as :
f (n)

g (n)

max

n0[1,n],n1[1,n]

1 if

0 if

in1 n0 xi

st

n 0 n1

n1 n
n1 n

where {n0 , n1} = arg

max

n0[1,n],n1[1,n]

in1 n0 xi

st

n 0 n1

The recursion is :
f (n)

max( f (n 1) x n , f (n 1))
if

max(
x

...

x
,
f
(
n

1
))

n
if
m
n

1
n

g (n)

f (n 1) x n f (n 1)
if

...

f
(
n

1
)

n
if
n 1
n
m

g (n 1) 1
g (n 1) 0

g (n 1) 1
g (n 1) 0

With the above definition of f(n) and g(n), m is unknown, we fail to establish dynamic programming, as it does not
show any optimal substructure property. Thus we need to reformulate the problem.
Dynamic programming
Let f(n) be the optimum of the problem with {x1, x2, xn}, we define g(n) as the optimum of a redefined problem.

g (n)

f (n)

=
=
=
=

max

n0[1,n]

inn0 xi

max

n0[1,n],n1[1,n]

in1 n0 xi

s.t. n 0 n1

max

max

in1 n0 xi

(1)

Condition n0 n1 is included in the 2 max operators.

max

max

in1 n0 xi

(2)

Equation (1)&(2) are identical. Both are brute force.

n0[1,n] n1[ n0 ,n]

n1[1,n] n0[1,n1 ]

(3)

max g (i)
i[1,n]

Although f(n) does not exhibit optimal substructure, g(n) does !! The recursion is :
g (n)

max

n0[1,n]

inn0 xi

max max inn1 xi x n , x n

n [1,n 1]

0
0

max(g (n 1) x n , x n ) if

x1
if

n 1
n 1

(4)

Hence, we firstly perform dynamic programming using equation (4), and finally solve f(N) using equation 3.

Longest increasing subsequence

Given a sequence of N real numbers x1, x2, xN, find the longest contiguous subsequence with increasing value. The
brute force solution is omitted, it is the same as that of maximum value contiguous subsequence.
Dynamic programming
Let f(n) be the optimum of the problem with {x1, x2, xn}, we define g(n) as the optimum of a redefined problem.
g (n)

f (n)

=
=
=

max (n n0 1)

s.t. xi xi1 i [n0 , n 1]

n0[1,n]

max

max (n1 n0 1)

s.t. xi xi1 i [n0 , n1 1]

(5)

max

max (n1 n0 1)

s.t. xi xi 1 i [n0 , n1 1]

(6)

n0[1,n] n1[n0 ,n]

n1[1,n] n0[1,n1 ]

Equation (5)&(6) are identical.

max g (i)
i[1,n]

Although f(n) does not exhibit optimal substructure, g(n) does !! The recursion is :
g (n)

=
=

max (n n0 1)

n0[1,n]

g (n 1) 1 if

1
if

x n 1 x n
x n1 x n

or n 1

Longest common subsequence

Given a sequence of N symbols x1, x2, xN and another sequence of M symbols y1, y2, yM, find the longest common
subsequence, i.e. the maximum set of matched pairs {(xi1, yj1), (xi2, yj2), }, such that i1, i2, [1,N] and j1, j2, [1,M],
where xi1 = yj1, xi2 = yj2, etc. The longest common subsequence is not necessarily contiguous. Windows software
windiff is an example of longest common subsequence.
Brute force solution
Consider the case when there are exactly n matched pairs, we have to select n symbols from the xn sequence and n
symbols from the yn sequence, once they are selected, they are automatically paired in ascending order, i.e. i1 < i2 <
< in and j1 < j2 < < jn, the number of combinations is thus NCn MCn. Since the possible number of matched pairs
ranges from 0 to N, hence brute force search involves :
# leaves =

N ,M ) N M
Cn Cn
min(
n0

nN0 (C Nn ) 2

nN0 CnN 2

2 2N

if N = M
special case of Cauchy Schwarz inequality

Dynamic programming 1
Let f(n,m) be the length of the longest common subsequence between x1, x2, xn and y1, y2, ym. Consider the case
in which xik = yjk and xik+1 = yjk+1, if f(ik, jk) = k, then f(ik+1,jk+1) = k+1. Lets look at how f(n,m) changes, horizontal axis
denotes n, while vertical axis denotes m.
f(n,m)
jk-1
jk
jk+1
jk+2
jk+3
jk+4

jk+1-1
jk+1
jk+1+1
jk+1+2

ik-1
k-1
k-1
k-1
k-1
k-1
k-1

k-1
k-1
k-1
k-1

ik
k-1
k
k
k
k
k

k
k
k
k

ik+1
k-1
k
k
k
k
k

k
k
k
k

ik+2
k-1
k
k
k
k
k

k
k
k
k

ik+3
k-1
k
k
k
k
k

k
k
k
k

ik+4
k-1
k
k
k
k
k

k
k
k
k

ik+1-1
k-1
k
k
k
k
k

k
k
k
k

ik+1
k-1
k
k
k
k
k

k
k+1
k+1
k+1

ik+1+1
k-1
k
k
k
k
k

k
k+1
k+1
k+1

ik+1+2
k-1
k
k
k
k
k
k
k+1
k+1
k+1

The recursive formula is different for different values of n and m. For convenience, lets define g(n,m) as a state variable,
which has 4 possible states : red state, green state, blue state and black state. The recursion of g(n,m) is :

g (n, m)

red

black

blue
green
if x n y m
if x n y m

if
if
if

g (n 1, m) red , blue
g (n, m 1) red , green
otherwise

red

blue
green

black

if
if
if
if

xn
xn
xn
xn

ym
ym
ym
ym

and g (n 1, m) red , blue

and g (n, m 1) red , green
and
otherwise

RHS of the above recursion formula is incorrect, which may result in one to many matching or many to one matching.
For example, consider the case when xik = yjk = xik+1 or xik = yjk = yjk+1. Now, with the help of g(n,m), the recursion of
f(n,m) is obvious :

f (n, m)

f (n 1, m 1) 1

f (n 1, m 1) 1
f (n 1, m 1) 1

f (n 1, m 1)

f (n 1, m) 1
f (n 1, m)
f (n 1, m) 1
f (n 1, m)

f (n, m 1) 1
f (n, m 1) 1
f (n, m 1)
f (n, m 1)

g (n, m) red
g (n, m) blue
g (n, m) green
g (n, m) black

if
if
if
if

(7)

Dynamic programming 2
However, this is not the simplest recursion. Now, we will try to rewrite the recursion for f(n,m) and eliminate g(n,m) by
grouping the cases in equation 7.

f (n, m)

f (n 1, m 1) 1

f (n 1, m 1) 1
f (n 1, m 1) 1

f (n 1, m 1) 1
f (n 1, m 1) 1

f (n 1, m 1)

f (n 1, m) 1
f (n 1, m)
f (n 1, m)
f (n 1, m) 1
f (n 1, m) 1
f (n 1, m)

f (n, m 1) 1
f (n, m 1) 1
f (n, m 1) 1
f (n, m 1)
f (n, m 1)
f (n, m 1)

if
if
if
if
if
if

g (m, n) red which _ im plies

g (n, m) blue
and
g (n, m) blue
and
g (n, m) green
and
g (n, m) green
and
g (n, m) black which _ im plies

xn
xn
xn
xn
xn
xn

ym
ym
ym
ym
ym
ym

f (n 1, m 1) 1

f (n 1, m 1) 1
f (n 1, m 1) 1

f (n 1, m 1) 1
f (n 1, m 1) 1

f (n 1, m 1)

f (n 1, m) 1
f (n 1, m)
f (n 1, m) 1
f (n 1, m)
f (n 1, m) 1
f (n 1, m)

f (n, m 1) 1
f (n, m 1) 1
f (n, m 1)
f (n, m 1) 1
f (n, m 1)
f (n, m 1)

if
if
if
if
if
if

g (m, n) red which _ im plies

g (n, m) blue
and
g (n, m) green
and
g (n, m) blue
and
g (n, m) green
and
g (n, m) black which _ im plies

xn
xn
xn
xn
xn
xn

ym
ym
ym
ym
ym
ym

min( f (m 1.m 1), f (n 1.m), f (n, m 1)) 1 if

max( f (m 1.m 1), f (n 1, m), f (n, m 1)) if

Since

Hence

f (n, m)

xn y m
xn y m

min( f (m 1.m 1), f (n 1.m), f (n, m 1)) =

f (m 1.m 1)

max(f (m 1.m 1), f (n 1.m), f (n, m 1)) =

max(f (n 1.m), f (n, m 1))

f (n 1, m 1) 1
if

max( f (n 1, m), f (n, m 1)) if

(swapping rows)

xn y m
xn y m

LIS vs LCS
The longest increasing subsequence of xn is equivalent to the longest common subsequence between xn and yn, where
yn is sorted xn. Can you prove it?
Building bridge
Given N cities along the north coast of a horizonal river, having x-coordinates : x1, x2, xN, and M cities along the
south of the river, having x-coordinates : y1, y2, yN. Suppose there are K = max(N,M) unique labels L1, L2, L3, LK,
which are randomly assigned to the cities on the north coast and the cities in the south coast, so that cities on the
same coast have different labels. We can now build a bridge joining a city on the north coast and a city on the south
coast with the same label. What is the maximum number of bridge that we can build without crossing each other?
Dynamic programming
If we sort the cities by x-coordinates, and draw a line joining cities with the same label, we have for example :
L2
x1

L6
x2

L5
x3

L3
x4

L7
x5

L1
x6

L4
x7

y1
L5

y2
L3

y3
L1

y4
L7

y5
L2

y6
L4

y7
L6

Maximizing the number of city pair having identical label without crossing is thus equivalent to a special case of the
longest common subsequence, in which all elements in the sequence are unique (city label does not duplicate). There
exist some solutions in the web claiming that the building bridge problem can be solved by applying the longest
increasing subsequence, however this is incorrect. Applying the longest increasing subsequence for solving building
bridge problem provides a feasible but non optimium result only !!

Edit distance
Given two strings with length N and M, A={a1,a2,a3,,aN} and B = {b1,b2,b3,,bM}, edit distance is defined as the
minimum number of operations needed to convert string A into string B, providing 3 different valid operations : (1)
delete a character from A, (2) insert a character into A, (3) modify a character in A.
Brute force solution
Suppose there K modifications, where min(N,M) K max(N,M), then there will be N-K deletions and M-K insertions.
The number of combinations will then be NCKMCK, as we need to pick K characters from string A and from string B.
Besides, K ranges from zero to min(N,M), hence the total number of combinations are :
N
C 0N C 0M C1N C1M C 2N C 2M ... C min(
CM
N , M ) min( N , M )

Dijkstras algorithm
This is the same as finding Dijkstras shortest path in a graph. Each vertex denotes a unique string, there are infinity
many vertices. There exists an edge between vertex i and j if string i can be converted into string j by one operation,
hence a vertex denoting a n-characters string has 28n+1 neighbours (which includes n possible deletions, n+1 possible
insertions and 26n possible modifications). The problem becomes finding the shortest path from vertex representing
string A to vertex representing string B with minimum cost, providing that cost of each edge is 1. This is a huge graph
with numerous edges. Running Dijkstras algorithm is equivalent to region growing from vertex A, it stops when vertex
B is poped from the priority queue (please refer to ASM notes for details of Dijkstras algorithm), this algorithm will be
slow.
Dynamic programming
If this problem is handled using dynamic programming, it is similar to the longest common subsequence. Let f(n,m) be
the edit distance between string {a1,a2,a3,,an} and string {b1,b2,b3,,bm}, then recursion is :
subproblem with string {a1,a2,a3,,an-1} and string {b1,b2,b3,,bm} plus one deletion
subproblem with string {a1,a2,a3,,an} and string {b1,b2,b3,,bm-1} plus one insertion
subproblem with string {a1,a2,a3,,an-1} and string {b1,b2,b3,,bm-1} plus one modification if an bm

f (n, m)

min( f (n, m 1) 1, f (n 1, m) 1, f (n 1, m 1) 1) if

min( f (n, m 1) 1, f (n 1, m) 1, f (n 1, m 1)) if

a n bm
a n bm

If we visualise the above dynamic programming as a graph, i.e. f(n,m) is a value assigned to vertex vn,m, and each edge
denotes an operation, then each vertex has 3 neighbours only, which is simpler than the Dijkstras algorithm.
Car assembly
Suppose there are 2 production lines producing the same product, each line has N steps. Let x1, x2, x3, , xN and y1,
y2, y3, , yN be the processing time for each step in production line 1 and production line 2 respectively. After
completing step n, we can transfer the intermediate product between the two lines, so that it can continue with step
n+1. Let t1,n and t2,n be the transfer time from line 1 to line 2 and from line 2 to line 1 respectively. What is the
minimum time for completing all steps in sequence?
Brute force solution
Before starting step 1, we need to select one production line out of two. After completing step n n[1,N-1], we need to
choose between keeping the same production line or switching to another production line. Thus, the total number of
possible routes is 2N.
Dynamic programming
Let T1(n) and T2(n) be the minimum time needed to complete step n in production line 1 and step n in production line 2
respectively. By visualising two production lines, we can derive recursion formula as :
x1

x2
t1,1 t2,1

y1

where

x3
t1,2 t2,2

y2

x4
t1,3 t2,3

y3

T1(4)

xN-2

t1,4 t2,4
y4

T2(4)

T1 ( n)

min(T1 (n 1) x n , T2 (n 1) y n t 2,n )

T2 (n)

min(T2 (n 1) y n , T1 (n 1) x n t1,n )

T1 (1)

=
=

x1

T2 (1)

xN-1
t1,N-2 t2,N-1

yN-2

yN-1

xN

T1(N)

t1,N-1 t2,N-1
yN

T2(N)

y1

Balanced partitions
Given a set of N floating points {x1, x2, , xN}, design an algorithm to separate the set into two, such that the absolute
difference between the sum of each set is minimised.
Brute force solution
Brute force search can be implemented by putting the first integer x1 into set 1, all the other N-1 integers should then
be put into either set 1 or set 2. Thus there are 2N-1 combinations in total.
Dynamic programming
If we define f(N) as the minimum absolute difference between the sum of each set, we cannot derive recursion, because
the minimum absolute difference f(N) is nothing related to f(N-1) or f(N-2) etc. We need to reformulate the problem.
Instead of directly make a recursion on the absolute difference, we may think about the question How many different
sums can we make with the subsets of X ? The total number of subset of X is NC0 + NC1 + NC2 + = 2N. The possible
sums of Xn = {x1, x2, , xn} can be constructed from the possible sums of Xn-1 = {x1, x2, , xn-1} by :
{ y, Y X n }
yY

{ y, Y X n 1} { y x n , Y X n 1}
yY

yY

Thus here is our dynamic programming : let f(n,:) be the set of sums constructed from {x1, x2, , xn}.
=
=

f (1,1)
f (n,2m)

f (n 1, m)

m [0,2 n1 1]

f (n,2m 1)

f (n 1, m) xn

m [0,2 n1 1]

f (1,0)

x0

Hence the sum of all numbers in set from {x1, x2, , xn} is :

xX n x

f (n,2(2 n1 1) 1)

f (n,2 n 1)

Scan through the set f(N,:), find the sum that is nearest to f(N,2N-1)/2, then the minimum absolute difference is :

y y

min

Y X N ,Y ' X N ,Y Y '0 yY

| ( f ( N ,2 N 1) v) v |

| f ( N ,2 N 1) 2v |

where v arg min | u f ( N ,2 N 1) / 2 |

u f ( N ,:)

yY '

Modified problem : integer

If the given set is bounded integers instead of floating points, i.e. xn[0,M] n[1,N], the dynamic programming can be
further speeded up by redefining f(n,m) as a boolean, which is true if there exists a subset of {x1, x2, , xn} with sum
equals to m and false otherwise.

f (1, m)

f (n, m)

true if

true if
false if
true if

true if
false if

m0
m x1

m[0, M ]

otherwise
f (n 1, m) true
f (n 1, m x n ) true

m [0, M n]

otherwise

The main difference in implementation between the floating point scenario and the integer scenario is that f(n,:) is a
set<float> with variable size in the former case, while it is a vector<bool> with fixed size in the latter case.
Two persons traversal of cities
Given an ordered sequence of N cities (in any dimensional space) and the matrix xn,m n,m[1,N] denoting the distance
between every pair of cities, find algorithm to divide the set into two, so that when person A visits all the set 1 cities in
sequence, while person B travels all the set 2 cities in sequence, the total travelling distance is minimised. This can be
regarded as a special case of travelling salesman problem with : (1) two salesmen and (2) predefined ordered cities.
Brute force solution
Brute force search can be implemented by putting the first city into set 1, all the other N-1 cities should then be put
into either set 1 or set 2. Thus there are 2N-1 combinations in total.
Dynamic programming (fail)
This is completely different from the car assembly problem, we cannot apply the car assembly solution here. However,
car assembly problem can be made closer to this problem by introducing many more machines in the production lines,

which can perform multiple contiguous steps, we are then given a processing time matrix xn,m n<m, which denotes the
processing time needed for completing steps n+1, n+2, , m continuously. In this case, the solution of traversal of
cities problem may be applicable to the car assembly problem after some modifications : (1) there are two processing
time matrices, one for production line 1 and one for production line 2, (2) switching between the two production lines
involves transfer time.
Dynamic programming
We need to define a state variable on which recursion formula is constructed (in this example, we can treat the index
vector as a state vector). Recursion formula is the relationship between the optimium of problem with the optimium of
subproblem plus delta change. In this problem, the delta change would probably be the distance matrix. In order to
apply distance matrix, we need to know where A and B is standing right now. For car assembly problem, the optimum
depends only on the last completed task. For traversal of cities, the optimum does not only depend on the last visited
city, but depends on the last visited city by A and the last visited city by B. In other words, this is a two dimensional
state variable (n,m). Lets define f(n,m) be the shortest total distance travelled by A and B, when A ends at city n and B
ends at city m, this is equivalent to saying that :
Among the N ordered cities, the first max(n,m) of them form a subproblem, which has an optimum of f(n,m), in which the last city visited by A
is n, the last city visited by B is m, the range of cities starting from min(n,m)+1, min(n,m)+2, min(n,m)+3, and up to max(n,m) are visited by
the same traveller, we dont know whether he is A or B, we know that he is the one who stopped at city max(n,m). Besides, we do not know the
starting point of A and B, we only know that if the starting point of A and B are n 0 and m0 respectively, then min(n0, m0)=1, and the traveller
who starts with city 1 should also visit the following cities in order : 1, 2, 3, max(n 0, m0)-1. Furthermore, f(1,m) means A visited city 1 and
stops here, all the other cities are visited by B in order : 2, 3, 4, , m, while f(n,1) means B visied city 1 and stops here, all the other cities are
visited by A in order : 2, 3, 4, , n.
In general, we have f(n,m) = f(m,n), because the path of A and B can be swapped without affecting the result, please note that f(n,n) is invalid, as
A, B cannot visit the same city. Thus, there is no need to calculate all elements in matrix f(n,m), we can consider cases with n<m only. With
n<m, we are not assuming that B visits the last city, because f(m,n) does exist and valid, it equals to f(n,m), we skip it in dynamic programming
simply for the sake of efficiency. In other words, f(n,m) is not a triangular matrix, but a symmetric matrix, excluding diagonal. Remind that :
(n,m) is a state vector, while f(n,m) is the optimum at this state.
How many paths which can reach the state (n,m) (suppose n<m)? The answer is 2n-1, because city n is visited by A, city
n+1, n+2, , m are visited by B, thus the combinations are generated by the first n-1 cities. Since city n+1, n+2, , m
must be visited by B, the subproblem of state (n,m) is straight forward : f(n,m) = f(n,m-1)+xm-1,m. However this equation
doesnt work when m = n+1, as f(n,n) is invalid, in this case, f(n,m) should be reduced into f(1,m), f(2,m), f(n-1,m).
Lets think about how we proceed from state (nm) to state (n,m) where n < n < m = n+1.
state
(nm)
(n,m)

city
traveller
traveller

1
?
?

2
?
?

?
?

distance travelled by A to reach (n,m)

distance travelled by B to reach (n,m)
distance travelled by A to reach (n,m)
distance travelled by B to reach (n,m)
total distance to reach (n,m)

n
A
A
=
=
=
=
=

n+1
B
B

B
B

n-1
B
B

n
B
A

m=n+1
B
B

a
b
a + xn,n
b - xn-1,n - xn,n+1 + xn-1,n+1
total distance to reach (n,m) + xn,n - xn-1,n - xn,n+1 + xn-1,n+1 (8)

The only exception to equation (8) is when n=n-1.

state
(nm)
(n,m)

city
traveller
traveller

1
?
?

2
?
?

?
?

distance travelled by A to reach (n,m)

distance travelled by B to reach (n,m)
distance travelled by A to reach (n,m)
distance travelled by B to reach (n,m)
total distance to reach (n,m)

n=n-1 n
A
B
A
A
=
=
=
=
=

m=n+1
B
B

a
b
a + xn,n
b - xk,n - xn,n+1 + xk,n+1
total distance to reach (n,m) + xn,n - xk,n - xn,n+1 + xk,n+1

Index k is the third last visited city by traveller B. If k doesnt exist (i.e. when B visits one or two cities only), then :
distance travelled by A to reach (n,m)
distance travelled by B to reach (n,m)
total distance to reach (n,m)

=
=
=

a + xn,n
b - xn,n+1
total distance to reach (n,m) + xn,n - xn,n+1

Recursion cannot be completed without solving index k. Lets define g(n,m) as As path (a set of city indices) so as to
reach the optimum f(n,m), while function h(path) returns the third last visited city by traveller B given an input path, it
returns zero if the third last visited city doesnt exist. We also define xn,0=0 and x0,m=0 for consistence of formula.

f (1, m)

g (1, m)

=
=

distance travelled by A + distance travelled by B from city 2, 3, to m

im21 xi,i1

(1) This is invalid for m=1.

{1}

(2) This is zero for m=2.

(3) Traveller A visits city 1 only.

( x1,1 ) ( x 2,3 x3,4 x 4,5 .. x m1,m )

The recursion of f(n,m) and g(n,m) are :

f (n, m)

f (n, m 1) x m,m1
if

min
(
f
(
n
'
,
m
)

x
)
if
n
'
,
n
h
(
g
(
n
'
,
m
)),
n
n
,
n

1
h
(
g
(
n
'
,
m
)),
n

1
n'[1,n 1]

g (n, m)

g (n, m 1) if

g (k , m) {n} if

Finally : f opt

min

n[1, N 1]

m n 1
m n 1

m n 1
m n 1

where k arg min ( f (n' , m) xn',n xh( g (n',m)),n xn,n1 xh( g (n',m)),n1 )
n'[1,n1]

f (n, N )

In order to implement the recursion, we have to scan f(n,m) by iterating through n externally and iterating through m
internally. We can see that the path g(n,m) doesnt grow as m increases, until ++n, such that m = n+1.
n
m
1
2
3
4
5
6
7

x
#
*
*
*
*
*

x
x
#
*
*
*
*

x
x
x
#
*
*
*

x
x
x
x
#
*
*

x
x
x
x
x
#
*

x
x
x
x
x
x
#

(1) Diagonal elements are invalid.

(2) * denotes elements updated by : f(n,m) = f(n,m-1)+xm-1,m
(3) # denotes elements updated by : f(n,m) = min()
for(n=1:N-1)
{
for(m=n+1:N)

if(m==n+1) f(n,m)=min(...);
else
f(n,m)=f(n,m-1)+x(m-1,m); }
fopt=min(f(1:N-1,N));

This is what happens inside function h.

unsigned long void h(std::set<unsigned long>& g, unsigned long m)
{
std::set<unsigned long>::reverse_iterator i=g.rbegin(); unsigned long count=0, mm=m;
while(i!=g.rend() && mm>0)
{
if (mm>*i) { ++count; if (count==3) return mm; }
else ++i;
--mm;
}
return 0;
}

The official solution provided by MIT is incorrect as (1) it ignores the red part of the above equations, and therefore (2)
it does not record traveller As optimal path. Can you check it again?
Bitonic tour
This is another specialized travelling salesman problem : travelling salesman problem with bitonic tour. Bitonic tour is
defined as a path joining multiple points in 2D space, such that the x coordinate of the path keeps increasing until it
reaches a maximum, and then decreases until it reaches the starting point. Another equivalent definition of bitonic
tour is a path that always cuts vertical lines at exactly two points.
Brute force solution
Brute force search can be implemented by dividing the N-2 cities (excluding the LHS-most and the RHS-most cities)
into two sets, the first point set constitutes the path for out-bound trip, while second point set constitutes the path for
in-bound trip. Thus the total number of combination is 2N-2.
Dynamic programming
For convenience, we assume that there are N cities (excluding the LHS-most and the RHS-most cities), with coordinate
(xn, yn) n[1,N], while the coordinate of the LHS-most and the RHS-most cities are (x0, y0) and (xN+1, yN+1) respectively,
so that x0 < x1 x2 x3 xN < xN+1. Since (1) cities in the bitonic tour problem have predefined order, based on x
coordinates, (2) distance matrix can be calculated using the 2D coordinates of cities, (3) the out-bound trip and inbound trip can be regarded as two travellers starting with cities xn0 and xm0 respectively, where min(n0, m0) = 1., and
ending with cities nlast and mlast, where nlast < mlast = N. Now, let dn,m be the distance matrix. The only minor difference
between these two combinatoric problems is the overhead time cost in the bitonic tour :
traversal of cities :
bitonic tour :
where : overhead

min (total distance of As path+ total distance of Bs path)

min (total distance of outbound + total distance of inbound) + overhead
=

d 0,1 d 0,max( n0 ,m0 ) d min( nlast.mlast), N 1 d N , N 1

Therefore there is only a slight change in the initialization and final step. The recursion is exactly the same. Let dn,m be
the distance matrix.
f (1, m)

g (1, m)

Finally : f opt

=
=

distance of outbound + distance of inbound

d 0,1 d 0,2 im21 d i,i 1

{1}

(d 0,1 ) (d 0,2 d 2,3 d 3,4 .. d m1,m )

min

n[1, N 1]

f (n, N ) d n, N 1 d N , N 1

Optimal game strategy

Consider a row of N coins with values {x1, x2, x3, , xN}, where N is even. We play a game against an opponent by
alternating turns. In each turn, a player selects either the first or last coin from the row, removes it from the row
permanently, and receives the value of the coin. Determine the maximum possible amount of money we can definitely
win if we move first. The opponent is as clever as the user.
Brute force solution
Since no decision is needed when there are two or less coins left, the total number of combination is 2N-2.
Dynamic programming
Let f(n,m) be the maximum values we can get by finishing the whole subgame, which containing coins {xn, xn+1, xn+2, ,
xm}, where 0n<mN and m-n+1 is even. If we pick coin n, our opponent will then either pick coin n+1, leaving us with
subgame optimum f(n+2,m), or pick poin m, leaving us with subgame optimum f(n+1,m-1), our opponent will pick in a
way so as to minimise our subgame optimum, thus he will pick coin n+1 if f(n+2,m) < f(n+1,m-1), and vice versa. The
same logic applies when we pick coin m, hence the recursion is :
f (n, m)

max(xn min( f (n 2, m), f (n 1, m 1)), xm min( f (n 1, m 1), f (n, m 2)))

Lets initialise matrix f(n,m), fill the diagonal with zero, i.e. f(n,n) = 0, and fill the lower subdiagonal with a max operator,
i.e. f(n,n+1) = max(xn, xn+1), then perform region growing with subdiagonal as multiple seeds. For example, the value at
red * depends on the values of all the blue elements.
n
m
1
2
3
4
5
6
7
8

0
#
*
*
*
*
*
*

x
0
#
*
*
*
*
*

x
x
0
#
*
*
*
*

x
x
x
0
#
*
*
*

x
x
x
x
0
#
*
*

x
x
x
x
x
0
#
*

x
x
x
x
x
x
0
#

x
x
x
x
x
x
x
0

# = subdiagonal

Please note that we should scan through f(n,m) diagonally as shown above, implementation like looping n externally,
looping m internally or looping m externally, looping n internally do not work.
Making change
Given N types of coins with integer values {x1, x2, x3, , xN}, such that x1 = 1 < x2 < x3 < xN, there are infinite many
number for each of them. Design an algorithm for finding the smallest number of coins, which sum to a given integer
value C exactly.
Brute force solution
Let Mn be the minimum number of coins needed to give a sum greater than or equal to C, when only the same kind of
coin with value xn is used, then the total number of combination is : M1 M2 M3 MN-1 MN.
Dijkstras algorithm
Lets review the Dijkstra algorithm. Given a graph G = {V,E}, V = {v1, v2, v3, , vN}, where en,m = {vn, vm} and cn,m denotes
the cost for travelling from vertex n to vertex m. Our aim is to find the route with minimum cost from v1 to reach vN.
We define f(n) be the minimum cost needed v1 from to reach vn. Recursion is simple :
f (n)

min ( f (m) c m,n )

m,em, n

Please note that the main difference between the dynamic programming in Dijkstras algorithm with all the previous
dynamic programming is the fact that, f(n) is updated multiple times in Dijkstra, and converges to optimal, yet f(n) is
usually calculated once in the previous dynamic programming. This is because the state n keeps increasing until it
reaches N or keeps decreasing until it reaches 1 as decisions are made in the previous problems, yet in Dijkstras
algorithm, we can keep travelling to a new vertex within the graph unstopped, i.e. we can route an infinite long path.

Dijkstra algorithm can be implemented in two ways. (1) Direct implementation of the recursive formula using recursive
function call : this is in fact a region growing starting from vN, there is no need to implement a stack nor a queue for
the region growing, since the region growing is done using the function call stack. (2) Iterative implementation, this is
in fact a region growing starting from v1 (please refer to ASM notes), you need to implement a stack, a queue or a
priority queue explicitly. However, the first method is infeasible as infinite long paths can blow up function call stack.
Dynamic programming
The making change problem is a special case of Dijkstra algorithm, with each vertex representing a unique state c,
which is defined as the sum of selected coins. There exists an edge between state a and state b (where b > a) if there
exists a coin with value xn such that b a = xn. The cost of each edge is 1. Now, we are looking for the minimum cost
f(c) to reach state c starting from state 0. Note : here I use state c instead of vertex v, recall that vertex is just a point on
a graph, state is a variable associated to the vertex, while f(c) is the optimum cost to reach state c.

where

f (c)

f (0)

min

m[1, N ],c xm

( f (c x m ) 1)

Unlike Dijkstras algorithm : (1) all edges are unidirectional, (2) there exists no edge from state b to state a where b > a,
and therefore (3) there exists finite number of routes. Thus we can implement the dynamic programming either by (1)
recursive function call or (2) iterative method. The recursive implementation is straight forward, while the iterative
implementation is a little bit tricky : if you do the region growing using a stack or a queue, you should wait until all f(c)
to converge before quiting, this is slow of course, or you can do the region growing with a priority queue, like Dijkstras
algorithm. Thus for Dijkstras algorithm, we prefer iterative method, for making change, we prefer recursive method.
Knapsack problem
Given N types of objects with weights {x1, x2, x3, , xN} and values {v1, v2, v3, , vN}, there are infinite many number for
each of them. Design an algorithm for finding the maximum value of objects packed inside a knapsack, which has a
maximum load of W. All xn, vn and W are real values.
Dynamic programming
Knapsack problem is very similar to the making change problem, with the following correspondence.
making change problem
coins num
=
{m1 , m2 , m3 ,...,m N }

knapsack problem
objects num
=

{m1 , m2 , m3 ,...,m N }

coins value

{x1 , x2 , x3 ,..., x N }

objects weight =

{x1 , x2 , x3 ,..., x N }

coins count

objects value

{v1 , v2 , v3 ,...,v N }

objective

max nN1 mn v n

constrain

min nN1 mn
nN1 mn x n C

constrain

nN1 mn xn W

vertex state

vertex state

objective

Each vertex denotes a unique state c or w, which denotes the sum of selected coins value (in making change problem)
or the sum of selected objects weight (in knapsack problem). Each edge denotes a state transition, there exists an
edge between two states (or two vertices) when there exists a coin or an object that can fulfill the state change. We
define f(c) and f(w) as the optimum of objective functions to reach state c (in making change problem) and to reach
state w (in knapsack problem). The recursion for knapsack is then straight forward :

where

f (w)

f (w)

max

m[1, N ],w xm

( f (w x m ) v m )

when w min(x1 , x2 , x3 ,..., x N )

Job scheduling
Given one machine and N jobs, each job requires processing time xn and generates profit pn if it can be completed
within deadline tn. The machine can process one job at a time, running job should not be interrupted until completion.
Find an algorithm to maximise profit, assuming xn tn n[1,N] and t1 t2 t3 tn (processing time and deadlines
can be real values).
Dynamic programming
Job scheduling problem can be regarded as generalised version knapsack problem. The main differences are (1) there
is only one weight limit in knapsack, while there are different deadlines for different jobs, (2) multiple objects of the
same type can be packed in knapsack, while the same job can either be done once or skipped in job scheduling.
knapsack
objects value
objects weight
knapack max load

job scheduling
jobs profit
jobs processing time
jobs deadline

10

objective

knapsack
max nN1 mn v n where mn

job scheduling
max nN1 mn p n where mn {0,1}

constrain

nN1 mn xn W

1n1 mn xn t1

2n1 mn xn t 2

nN1 mn xn t N
Let f(t) be the maximum profit we can make within time t. Can we simply copy the recursion from knapsack problem?

where

f (t )

f (t )

max

m[1, N ], xm t tm

( f (t x m ) p m )

when t min(t1 , t 2 , t 3 ,...,t N )

The answer is no, because it ignores the fact that each job can either be done once or skipped. Thus we should record
a set of job done D, so that we dont consider D in subproblem, i.e.
f (t )

max

m[1, N ],mD, xm t tm

( f (t x m ) p m )

(9)

D is different for different decision paths, it is terrible to record all possible D. Since we make one decision for a job (we
dont need to reconsider a job again), instead of considering which job we should eliminate to reduce to subproblem
(i.e. integer m in equation 9), we can scan through the job with nearest deadline, and consider whether to do or to
skip the job so as to reduce to subproblem (i.e. boolean m in equation 10). We can define f(n,t) as the maximum profit
that we can generate from n jobs within final deadline t.
f (n, t )

where

f (n, t )

max

m[0,1], xn t tn

( f (n 1, t mxn ) mpn )

max(f (n 1, t ), f (n 1, t x n ) p n ) if

f (n 1, t )
if

(10)
xn t t n
otherwise

when t min(t1 , t 2 , t 3 ,...,t N )

Function approximation
Given a sequence of N data points {(x1,y1), (x2,y2), (x3,y3), , (xN,yN)}, where x1 < x2 < x3 < xN, all generated by function
y = f(x), we need an algorithm to fit the data points using a piecewise linear function gM(x), having M segments (defined
by M+1 control points) to minimise the fitting error. The M+1 control points must be a subset of the N data points,
with (x1,y1) and (xN,yN) as the first and the last control point. Fitting error of N data points using M segments is :
e( N , M ) =

nN1 ( g M ( xn ) y n ) 2

Dynamic programming
The problem can be reduced in two dimensions : number of data points and number of control points. Lets define (n,m)
as the state, e(n,m) be the minimum error of fitting n data points using m segments. The recurion cannot simply be :
e(n, m)

nor

e(n, m)

=
=

e(n 1, m) ...
e(n, m 1) ...

i.e. e(n-1,m) and e(n,m-1) are not the subproblem of e(n,m)

because all the existing best-fit segments (or control points) in the problem will changed completely (as if redoing the
regression again) when we add a new data point or add a new segment, we can understand why by considering these
two cases :
add a new point

add a new segment

11

However, the optimal substructure happens when we consider the following problem and subproblem :
reduce into subproblem

This is Bellman's principle of optimality : if {a1, a2, a3, , aK, , aN} is the optimal path to reach state aN, then {a1, a2,
a3, , aK} is the optimal path to reach state aK. Hence as shown in the above example, if control points {(x1,y1), (x3,y3),
(x6,y6), (x11,y11)} is optimal path of the problem with data size 11, then control points {(x1,y1), (x3,y3), (x6,y6)} is also the
optimal path for subproblem with data size 6. Thus the recursion for function approximation is :

where

e(n, m)

e(n,1)

min (e(k , m 1) in k 1 (h( x i ) y i ) 2 )

k[1,n 1]

min (e(k , m 1) in k 1 (h( x i ) y i ) 2 )

k[1,n 1]

where h(x) is the straight line joining the points (xk,yk) and (xn,yn) :
h(x)

yn yk
(x xk ) yk
xn xk

The implementation is : looping m externally and looping n internally.

for(m=1;m!=M;++m)
{
for(n=1;n!=N;++n) e(n,m) = min(...);
}

Bin packing
Given N1 objects with size s1, N2 objects with size s2 and N3 objects with size s3, find the minimum number of boxes
needed to pack all objects, assuming the capacity of box is C, such that s1 s2 s3 C.
Dynamic programming
This is a little bit similar to the function approximation problem that an increase by 1 in the problem size (or state
variable) n1, n2 or n3 will result in a complete different distribution of objects in the boxes), hence state (n11,n2,n3) is
not the subproblem of state (n1,n2,n3) and so on. Let m = f(n1,n2,n3) be the minimum number of boxes needed to pack
all the n = n1+n2+n3 objects, and suppose the distribution of objects is :
box 1

object 1

object 2

object 3

remaining capacity

a1

b1

c1

C a1s1 b1s 2 c1s3

C a2 s1 b2 s 2 c2 s3

box 2

a2

b2

c2

box m

am

bm

cm

subtotal

n1 m
k 1 a k

n2 m
k 1 bk

n3 m
k 1 c k

C am s1 bm s 2 cm s3

table#

With Bellman's principle of optimality, we can claim that :

f (a1 , b1 , c1 )
f ( a 2 , b2 , c 2 )

=
=

f (a1 a 2 , b1 b2 , c1 c 2 )

f (a1 a3 , b1 b3 , c1 c3 )

f (am , bm , cm )

=
=

|K|

f (a1 a3 , b1 b3 , c1 c3 )

and in general we have :

f ( a k , bk , c k )
kK , K [1,m]

(10)

where | K | is cardinality (size) of set K

otherwise, if any one of the above statements is not fulfilled, we can then reduce the number of boxes used to pack all
the (n1,n2,n3) objects, i.e. f(n1,n2,n3) is not the optimum, contradicting our assumptions. Now we have :

f (n1 , n2 , n3 )

f (n1 , n2 , n3 )

f(

a k , bk , c k ) f ( a k , bk , c k )

kK

(11)

where K K ' {1,2,...,m}, K K ' 0

(12)

where a a ' n1 b b' n 2 c c' n3

kK '

f (a, b, c) f (a' , b' , c' )

12

Equation (11) is valid because all the object combinations (ak,bk,ck) are one of those combinations listed in table#,
Bellmans principle of optimality implies that each of the row or sum of the rows in the table# are optimal, as stated in
equation (10). Although a+a=n1, b+b=n2 and c+c=n3 in equation (12), (a,b,c) and (a,b,c) may not be one of the rows
in table#, Bellmans principle of optimality doesnt imply equation (12). Thus we need to add minimise operator in the
recursion : as we loop through (a,b,c), there must exists some (a,b,c) combinations that can be found in table#.

where

f (n1 , n2 , n3 )

f (n1 , n2 , n3 )

( f (n1 a, n2 b, n3 c) f (a, b, c))

min

a[0,n1 ],b[0,n2 ],c[0,n3 ]

where n1s1 n2 s 2 n3 s3 C

Box stacking
Given N boxes, having sides sn,1, sn,2 and sn,3 n[1,N]. What is the highest stack of boxes, so that base of each box in
the stack is strictly smaller than the base of the box lying below in both dimensions. The orientation of boxes can be
changed. Each box can be used once only.
Brute force solution
To stack all boxes, with 3 orientations for each box (select one out of the three faces as the base), while ignoring the
base constraint, there are N!3N permutations in total. For each permutation, we can check the stack against the base
constraint, as we scan through the boxes in the stack starting from bottom, the scanning stop once we come across a
violation of base constraint, we can then calculate the sum of height for the valid section of the stack. We repeat this
calculation for all permutations and return the highest one.
Dynamic programming
Lets define orientation i, where i = {1,2,3}, then height, width and depth of box n with orientation i are hn,i, wn,i and dn,i.
hn,i

w n,i

d n,i

i.e. different orientation will result in different height

s n,i

max

(s n, j )

min

(s n, j )

j{1,2,3}, j i
j{1,2,3}, j i

i.e. d n wn

Lets define the state variable (or state vector) to describe a building stack, we should include orientation of the highest
box in the state vector to handle the base constraint, yet this is implying that we know which one is the highest box in
a stack. Let f(n,i) be the maximum height achieved by a subset of first n boxes, with box n having orientation i at the
bottom of stack. Recursive approach starts stacking from the bottom, while iterative approach starts stacking from the
top. By including box n in the subproblem optimum, that means we need to scan through f(n,i) as a final step to get
the global optimum (this is similar to the longest increasing subsequence problem, LIS). The recursion is :
f (n, i)

f opt

max

f (m, j ) hn,i

max

f (n, i)

1mn, j{1,2,3}
1n N ,i{1,2,3}

Enhancement 1
However, there are problems with the above recursion : (1) it ignores base constraint, (2) it ignores possible optimum
having box m at the bottom and box n lying above it (where m<n). To solve the problem, we need to modify the problem
a little bit : change in orientation is limited, we can swap between width and depth only (sn,2 and sn,3), while height is
fixed (sn,1), besides we need to modify the solution a little bit : boxes should be sorted in ascending order of base area
before recursion starts, so that if box m is selected as the bottom of the stack, then box n will never be included in the
stack (where m<n). The recursion becomes :
f (n, i)

f opt

max
max
f (m, j ) hn,1 , hn,1
1mn, j{2,3}, wm, j wn,i ,d m, j d n,i

max
f (n, i)

(13)

1n N ,i{2,3}

Conditions wm,j<wn,i and dm,j<dn,i are added to equation (13), since area comparison is not sufficient to execute base
constraint. Besides j=2,3 only as the change in orientation is limited. Finally, if no m fulfills base constraint, then f(n,i)
equals to height of box n, i.e. it is the only box in the stack.
Enhancement 2
Now, lets relax the restriction, if orientation is free to change (i.e. width, depth and height can be interchanged), then
how can we enhance the algorithm? The answer is to generate 3 copies for a box, with height hn,1, hn,2 and hn,3, then
sort all 3N boxes by base area, perform recursion in equation (14). In this case, each box may be used more than once.
f (n)

f opt

max
max
f ( m) hn , hn
1m3 N , w w ,d d

m
n m
n

max f ( n)

(14)

1 n3 N

13

In conclusion, this problem can be solved using dynamic programming if (1) orientation of boxes cannot be changed, each
box can be used once or (2) orientation of boxes can be changed, each box can be used more than once. In both cases, we
should sort the box by base area. The implementation for enhancement 2 is :
for(n=1;n<=3*N;++n)
{
double max_value = 0;
for(m=1;m!=n;++m)
{
if (w(m)<w(n) && d(m)<d(n) &&
max_value<f(m)+h(n))
max_value=f(m)+h(n);
}
f(n)=h(n);
if (max_value>0) f(n)+=max_value;
}

// Can we check against the optimum path of f(m),

// to see if box n is already used in f(m)?

// Can we associate an optimum path with f(n) here?

Enhancement 3
Now, lets consider the original version, in which orientation is free to change and each box can be used once only, can
we solve it by dynamic programming? Can we solve by recording the optimum path and associate with f(n,i) (see the
comments in the above code), so that we can check if a box is used already in the stack? The answer is no, because
f(n,1) is a the subproblem of f(n,2), f(n,2) is a the subproblem of f(n,1) neither (beware : if A is a subproblem of B and if
B is a subproblem of C, then A is a subproblem of C). Without optimal substructure, we cannot formulate the recursion.
Boolean parenthesizations
Given an expression with N binary operators and N+1 operands, find the number of different parenthesizations that
gives TRUE as the final result, providing that (1) there are 3 possible operators : AND, OR, XOR (which returns TRUE
when inputs are different), (2) there are 2 possible operands : TRUE and FALSE. In this example, we assume no
default operator precedence, the priority of operator execution depends only on parenthesization, hence bracket is
needed to resolve the priority between operators whenever there exist more than one operators in an expression or a
sub-expression.
# operand
1
2
3
4

# parenthesization
1
1
11+11
=2
12+11+21
=5

15+12+21+51
= 14

parenthesization
x1
x1 op1 x2
x1 op1 (x2 op2 x3)
(x1 op1 x2) op2 x3
x1 op1 (x2 op2 (x3 op3 x4))
x1 op1 ((x2 op2 x3) op3 x4)
(x1 op1 x2) op2 (x3 op3 x4)
(x1 op1 (x2 op2 x3)) op3 x4
((x1 op1 x2) op2 x3) op3 x4
x1 op1 (x2 op2 (x3 op3 (x4 op4 x5)))
x1 op1 (x2 op2 ((x3 op3 x4) op4 x5))
x1 op1 ((x2 op2 x3) op3 (x4 op4 x5))
x1 op1 ((x2 op2 (x3 op3 x4)) op4 x5)
x1 op1 (((x2 op2 x3) op3 x4) op4 x5)
... etc

# bracket
0
0
1
1
2
2
2
2
2
3
3
3
3
3

Thus the following parenthesizations are not allowed, as there exists operators having precedence unresolved.
x1 op1 x2 op2 x3 op3 x4
(x1 op1 x2) op2 x3 op3 x4
(x1 op1 x2 op2 x3) op3 x4
(x1 op1 x2 op2 x3 op3 x4)

Need 2 brackets for 3 operator-expression.

# bracket of an expression with N operators = N-2

# parenthesization of an expression
# parenthesization of an expression that returns TRUE +
# parenthesization of an expression that returns FALSE

These values can be found by dyn prog.

Dynamic programming
We perform counting for expressions returning TRUE and expressions returning FALSE together. Like the bin packing
problem, we can divide the N+1 operands into 2 groups, separated by a binary operator. Suppose we pick operator k,
then operand subset (1,k) and operand subset (k+1,N+1) are the subproblems of the original problem. Why not divide
the operators into 3 or even more groups? No, we dont do that because we have binary operator only, we dont have
trinary operator. Hence, if we are given unary operator, binary operator and trinary operator, the way we divide the n
operators will become very complicated. Given a sub-expression starting with operand n and ending with operand m,
where 1n<mN+1, let f(n,m) and g(n,m) respectively be the number of parenthesizations that give TRUE output and
FALSE, then recursion is :

14

f (n, m)

k ,and f (n, k ) f (k 1, m) k , xor f (n, k ) g (k 1, m) k , xor g (n, k ) f (k 1, m)

k[ n,m1] k ,or f (n, k ) f (k 1, m) k ,or f (n, k ) g (k 1, m) k ,or g (n, k ) f (k 1, m)

g (n, m)

k ,or g (n, k ) g (k 1, m) k , xor f (n, k ) f (k 1, m) k , xor g (n, k ) g (k 1, m)

k[n,m1] k ,and g (n, k ) g (k 1, m) k ,and f (n, k ) g (k 1, m) k ,and g (n, k ) f (k 1, m)

where k,or equals to 1 if operator k is OR and 0 otherwise, while k,and and k,xor are defined similarly. Besides, property
3 and 4 in the problem statement are implemented as the boundary cases :

f (n, m)

g (n, m)

1
0

if
if
if
if

nm
nm
n 1 m
n 1 m

and
x n false
and
x n true
and x n opn x m false
and x n opn x m true

1
0

if
if
if
if

nm
nm
n 1 m
n 1 m

and
x n true
and
x n false
and x n opn x m true
and x n opn x m false

To implement the recursion, need to perform scanning starting from diagonal of the lower triangular matrix f(n,m).
n
m
1
2
3
4
5
6
7

#
@
%
$
*
?
.

x
#
@
%
$
*
?

x
x
#
@
%
$
*

x
x
x
#
@
%
$

x
x
x
x
#
@
%

x
x
x
x
x
#
@

scanning sequence = #@%$*?.

Hint : In this problem, recursion is derived from divide and conquer, while scanning the division point within a range,
and finally combine the result of subproblems through appropriate logic. Recall that the box packing problem is solved
in a similar approach.
Bell number
Bell number is the maximum number of ways to partition N elements.
Dynamic programming
Lets try induction approach. Suppose f(n) is the Bell number :
f(0) = 1
f(1) = 1
f(2) = 2

()
(x1)
(x2)(x1)
(x2x1)
(x3)(x2)(x1)
(x3x2)(x1)
(x3x2x1)
(x4)(x3)(x2)(x1)
(x4x3)(x2)(x1)
(x4x3x2)(x1)
(x4 x3 x2 x1)

f(3) = 5
f(4) = 5

(x3)(x2x1)
(x3x1)(x2)
(x4)(x3)(x2x1)
(x4x3)(x2x1)
(x4x3x1)(x2)

(x4)(x3x2)(x1)
(x4x2)(x3)(x1)
(x4x2x1)(x3)

(x4)(x3x1)(x2)
(x4x2)(x3x1)

(x4)(x3x2x1)
(x4x1)(x3)(x2)

(x4x1)(x3x2)

The algorithm is : when we add the n+1th element to a problem with size n, the first thing we should determine is the
size of the partition that element n+1 belongs to, suppose it is k, then we need to pick k elements out of n elements
(recall that problem size is n just before element n+1 is added), these k elements together with element n+1 forms one
partition, while there are n-k elements left, the number of possible partitions formed by these n-k elements are f(n-k).
Since there are nCk ways to pick k elements, and k ranges from 0 to n, hence :
f (n 1)

f (n 1, k 0) f (n 1, k 1) f (n 1, k 2) ... f (n 1, k n 1) f (n 1, k n)

C0n f (0) C1n f (1) C2n f (2) ... C nn f (n)

15