Documente Academic
Documente Profesional
Documente Cultură
Lecture 7
COSC 31011
Techniques
Optimization Problems
A Sequence of Decisions
The Little Bird & Friend
Optimal Substructure
Memoization
Set of Sub-Instances
Tracing Dyn. Prog. Alg
Reversing
Code
Speeding Up Running Time
Multiple Opt Solutions
Review
Question for Little Bird
Review & Don'ts
Problems
Bellman Ford
Best Path
Printing Neatly
Longest Common Subsequence
Knapsack Problem
The Event Scheduling Problem
Parsing
Satisfiability
b 10
25
30
40
3
15
181
l=3
2
3
1
2
v
8
l=4
4
40
15
181
c
2
3
1
2
v
8
l=4
6
for l = 2,4,8,16,,2n
% Find Dist[uv,l] from Dist[u,v,l/2]
for all u,v Verticies
Dist[u,v,l] = Dist[u,v,l/2]
for all k Verticies
Dist[u,v,l] = min( Dist[u,v,l], Dist[u,k,l/2]+Dist[k,v,l/2] )
b 10
1
c
u
25
30
40
3
15
181
3
1
2
v
8
l=4
7
10
0 (sometimes useful)
b 10
25
30
40
3
15
181
c
2
3
1
2
v
8
l=4
8
Exit
When to exit?
A simple path never uses a node more than once
and so does not have more than l=n-1.
for all u,v Verticies
Dist[u,v] = Dist[u,v,n]
b 10
25
30
40
3
15
181
c
2
3
1
2
v
8
l=4
9
25
-5
3
10
12
Dynamic Programming
A hard topic.
I try to provide a unified way to think of it
and a fixed set of steps to follow.
Even if you dont get the details of the
algorithm correct, at least get the right structure.
I provide analogies (little bird) to make it
hopefully more fun & easier to follow.
13
Optimization Problems
An important and practical class of computational
problems.
For most of these, the best known algorithm runs in
exponential time.
Industry would pay dearly to have faster algorithms.
Heuristics
Some have quick Greedy or Dynamic Programming
algorithms
For the rest, Recursive Back Tracking is the best option.
14
Optimization Problems
Ingredients:
Instances: The possible inputs to the problem.
Solutions for Instance: Each instance has an
exponentially large set of solutions.
Cost of Solution: Each solution has an easy to
compute cost or value.
15
Optimization Problems
Specification of an Optimization Problem
<preCond>: The input is one instance.
<postCond>:
The output is one of the valid solutions for this
instance with optimal cost.
(minimum or maximum)
The solution might not be unique.
Be clear about these ingredients!
16
17
3
5
2
3
3
2
1
18
2+6+3+7=18
4+2+1+5=12
19
20
An Algorithm As
A Sequence of Decisions
I ask a question about the solution.
Which edge should we take first?
Some how I decide <s,v3>.
My friend asks the next question.
Which edge do we take second?
Some how he decides <v3,v5>.
His friend asks the next question.
Which edge do we take third?
Some how he decided <v5,v8>.
21
An Algorithm As
A Sequence of Decisions
I ask a question about the solution.
Which edge should we take first?
How do I decide?
23
An Algorithm As
A Sequence of Decisions
I ask a question about the solution.
Which edge should we take first?
How do I decide?
In reality we will try
all possible first edges.
But let's skip this part
by pretending that we have
a little bird to answer this
little question.
24
25
27
Faulty Bird
But what if we do not have a bird that we trust?
This work is not wasted, because we have found
the best solution to our instance
from amongst those
consistent with this bird' s answer.
i.e. the best path from s to t
from amongst those
starting with <s,v1>.
Define optS<I,k> to be:
the optimum solution
for instance I
consistent with the kth bird' s answer.
29
Faulty Bird
But what if we do not have a bird that we trust?
This work is not wasted, because we have found
the best solution to our instance
from amongst those
consistent with this bird' s answer.
i.e. the best path from s to t
from amongst those
starting with <s,v1>.
In reality we will try
all possible first edges,
giving ..
30
Faulty Bird
31
Faulty Bird
32
Faulty Bird
33
Faulty Bird
34
Faulty Bird
A sequence of question to
a little bird about a solution
forms a tree of possible
answers.
36
Solutions consistent
with the kth bird' s answer.
37
optS<I,k>
Solutions consistent
with the kth bird' s answer.
38
optS<I,k>
optS[I]
max
max
39
Recursive
backtracking
code always has this
same basic structure.
41
42
43
44
45
47
optSolk
is a best solution
for our instance
from amongst
those consistent
with the bird's
kth answer.
Take the best
of the best
49
Base Cases:
Instances that are too
small to have smaller
instances to give
to friends.
What are these?
What are their
solutions
and costs?
51
Optimal Substructure
In order to be able to design a recursive backtracking
algorithm for a computational problem,
the problem needs to have a recursive structure,
i.e. for a path from s to t to be optimal,
the sub-path from vi to t must optimal.
If shorter from vi to t.
shorter to s to t.
52
Optimal Substructure
In order to be able to design a recursive backtracking
algorithm for a computational problem,
the problem needs to have an optimal substructure,
i.e. for a path from s to t to be optimal,
the sub-path from vi to t must optimal.
And finding such a sub-path is a
sub-instance of the same
computational problem.
53
Optimal Substructure
Optimal substructure means that
54
Optimal Substructure
55
Optimal Substructure
Longest simple path
B
3
1
4
Consider the
C
D
A
following graph:
The longest simple path (path not containing a
cycle) from A to D is A B C D
However, the subpath A B is not the longest
simple path from A to B (A C B is longer)
The principle of optimality is not satisfied for
this problem
Hence, the longest simple path problem cannot
be solved by a dynamic programming
approach
NP-Complete
56
57
58
60
61
Exponential Time
Redoing Work
Which is the best path from v7 to t?
How many friends solve this sub-instance?
62
Exponential Time
Redoing Work
Which is the best path from s to t?
63
Exponential Time
Redoing Work
Which is the best path from v1 to t?
64
Exponential Time
Redoing Work
Which is the best path from v4 to t?
65
Exponential Time
Redoing Work
Which is the best path from v7 to t?
Theres one.
66
Exponential Time
Redoing Work
Which is the best path from s to t?
67
Exponential Time
Redoing Work
Which is the best path from v3 to t?
68
Exponential Time
Redoing Work
Which is the best path from v5 to t?
69
Exponential Time
Redoing Work
Which is the best path from v7 to t?
Theres another.
70
Exponential Time
Redoing Work
Which is the best path from v7 to t?
How many friends solve this sub-instance?
Once for each
path to v7
Waste time
redoing work
Save time by only
doing once.
71
Depth First
Search
Drop
bread crumbs
and dont revisit.
But we need
shortest path
72
Dynamic Programming
73
Dynamic Programming
It is my job to learn
and remember
the optSol to my sub-Instance
i.e. the best path from v7 to t
74
Dynamic Programming
Dynamic Programming
When I need to find
the best path from v2 to t
I will ask you for
the best path from v7 to t
Dynamic Programming
Dynamic Programming
Avoid waiting.
Dynamic Programming
79
Set of Sub-Instances
But what sub-instance need to be solved
and in which order?
Given an instance I,
Imagine running the
recursive algorithm on it.
Determine the complete
set of sub-Instances ever
given to you, your friends,
their friends,
80
Set of Sub-Instances
Guess the complete set S of sub-Instances.
Best path from v7 to t?
Best path from v21 to t?
v21
Yes
No
81
Set of Sub-Instances
Guess the complete set S of sub-Instances.
Best path from v7 to t?
Best path from v21 to t?
Best path from v3 to v7?
Yes
No
No
82
Set of Sub-Instances
Guess the complete set S of sub-Instances.
Best path from v7 to t?
Best path from v21 to t?
Best path from v3 to v7?
Yes
No
No
83
Set of Sub-Instances
Guess the complete set S of sub-Instances.
Best path from v7 to t?
Yes
Best path from v21 to t?
No
Best path from v3 to v7? No
i Best path from vi to t? Yes
84
Set of Sub-Instances
Guess the complete set S of sub-Instances is
i Best path from vi to t?
Assign one friend
to each sub-Instance.
85
Set of Sub-Instances
Guess the complete set S of sub-Instances is
i Best path from vi to t?
86
Set of Sub-Instances
Guess the complete set S of sub-Instances is
i Best path from vi to t?
87
Set of Sub-Instances
Guess the complete set S of sub-Instances is
i Best path from vi to t?
88
Set of Sub-Instances
Guess the complete set S of sub-Instances is
i Best path from vi to t?
The set S of sub-Instances needs to:
include our given I
closed under friend operation
each sub-Instance needs to be
asked of some friend, friend,
A fine set of sub-instances!
89
Order to complete
The complete set S of sub-Instances is
i Best path from vi to t?
In what order should they go?
in an order such that
no friend must wait.
from smallest to largest
For this problem,
the order relies on
the graph being leveled.
90
Order to complete
The complete set S of sub-Instances is
i Best path from vi to t?
In what order should they go?
in an order such that
no friend must wait.
from smallest to largest
First
Last
Instance to be solved.
91
Dynamic Programming
"Which is the best path from t to t?"
Base Case
easy
92
Dynamic Programming
"Which is the best path from v8 to t?"
easy
93
Dynamic Programming
"Which is the best path from v7 to t?"
easy
94
Dynamic Programming
"Which is the best path from v6 to t?"
easy
95
Dynamic Programming
"Which is the best path from v5 to t?"
Harder
96
Dynamic Programming
"Which is the best path from v5 to t?"
Little bird
suggests first
edge <v5,v7>
Friend gives best
path <v7,t>.
97
Dynamic Programming
"Which is the best path from v5 to t?"
Little bird
suggests first
edge <v5,v8>
Friend gives best
path <v8,t>.
98
Dynamic Programming
"Which is the best path from v5 to t?"
Take best of best
99
Dynamic Programming
"Which is the best path from v4 to t?"
100
Dynamic Programming
"Which is the best path from v4 to t?"
Little bird
suggests first
edge <v4,v6>
Friend gives best
path <v7,t>.
101
Dynamic Programming
"Which is the best path from v4 to t?"
Little bird
suggests first
edge <v4,t>
Friend gives best
path <t,t>.
102
Dynamic Programming
"Which is the best path from v4 to t?"
Little bird
suggests first
edge <v4,v7>
Friend gives best
path <v7,t>.
103
Dynamic Programming
"Which is the best path from v4 to t?"
Take best of best
104
Dynamic Programming
"Which is the best path from v3 to t?"
105
Dynamic Programming
"Which is the best path from v3 to t?"
Little bird
suggests first
edge <v3,v5>
Friend gives best
path <v5,t>.
106
Dynamic Programming
"Which is the best path from v3 to t?"
Little bird
suggests first
edge <v3,v8>
Friend gives best
path <v8,t>.
107
Dynamic Programming
"Which is the best path from v3 to t?"
Take best of best
108
Dynamic Programming
"Which is the best path from v2 to t?"
109
Dynamic Programming
"Which is the best path from v2 to t?"
Little bird
suggests first
edge <v2,v4>
Friend gives best
path <v4,t>.
110
Dynamic Programming
"Which is the best path from v2 to t?"
Little bird
suggests first
edge <v2,v7>
Friend gives best
path <v7,t>.
111
Dynamic Programming
"Which is the best path from v2 to t?"
Take best of best
112
Dynamic Programming
"Which is the best path from v1 to t?"
113
Dynamic Programming
"Which is the best path from v1 to t?"
Little bird
suggests first
edge <v1,v3>
Friend gives best
path <v3,t>.
114
Dynamic Programming
"Which is the best path from v1 to t?"
Little bird
suggests first
edge <v1,v4>
Friend gives best
path <v4,t>.
115
Dynamic Programming
"Which is the best path from v1 to t?"
Little bird
suggests first
edge <v1,v5>
Friend gives best
path <v5,t>.
116
Dynamic Programming
"Which is the best path from v1 to t?"
Take best of best
117
Dynamic Programming
"Which is the best path from s to t?"
Original Problem
118
Dynamic Programming
"Which is the best path from s to t?"
Little bird
suggests first
edge <s,v1>
Friend gives best
path <v1,t>.
119
Dynamic Programming
"Which is the best path from s to t?"
Little bird
suggests first
edge <s,v2>
Friend gives best
path <v2,t>.
120
Dynamic Programming
"Which is the best path from s to t?"
Little bird
suggests first
edge <s,v3>
Friend gives best
path <v3,t>.
121
Dynamic Programming
"Which is the best path from s to t?"
Little bird
suggests first
edge <s,v4>
Friend gives best
path <v4,t>.
122
Dynamic Programming
"Which is the best path from s to t?"
Take best of best
DONE
123
Dynamic Programming
Construct a table
for storing an optimal solution & cost
for each sub-instance.
Map
i Best path from vi to t?
Sub-Instances
i [n], i.e. for each node vi
Indexes
Cell of table
v,
v , vi , v , .,
124
Dynamic Programming
Fill out a table containing
an optimal solution for each sub-instance.
Which is the best path from vi to t?
t, v8,
Base case
s
Original
125
Communication
Friend k gives friend i a best path from vk to t.
Recursive BackTracking
i <optSubSol,optSubCost>
= LeveledGraph(<G,vk,t>)
k return(optSolmin,optCostmin) ,
i
k
Dynamic Programming
k optSol[k] = optSolmin
i
optSubSol = optSol[k]
optSolk = <vi,vk> + optSol[k]
126
Dynamic
Programming
code always has this
same basic structure.
127
128
Dynamic Programs
do not recurse making
the instance smaller
and smaller.
Instead, it up front
determines the set S
of all sub-instances
that ever need
to be solved.
Be clear what
sub-instances are.
129
Be clear what
sub-instances are.
130
The set S
of sub-instances
are solved from
smallest to largest
so that no body waits.
Base Cases:
Instances that are too
small to have smaller
instances to give
to friends.
They get solved first
and their solutions
stored.
131
Then we iterate
through the remaining
sub-instances.
From smallest
to largest.
Each gets solved
and their solutions
stored.
132
Consider yourself
to be a friend
working on one
of these.
Be clear which
sub-instance is yours.
Solve this as you
did before.
133
135
136
Be clear what
sub-instance
you give your friend.
137
138
optSol<i,k>
is a best solution for
our instance subI[i]
from amongst
those consistent
with the bird's
kth answer.
Take the best
of the best
141
142
Base Cases:
Instances that are too
small to have smaller
instances to give
to friends.
Is this code correct?
143
Dynamic Programs
do not recurse making
the instance smaller
and smaller.
Hence, lets not worry
about our instance I
being a base case.
144
145
t=
n
146
147
s=
0
Path from s to t.
Path from t to t.
149
An Esthetically Better
Path from s to s.
Path from s to t.
150
Reversing
151
Reversing
Determine the complete set of sub-instances
Which is the best path from s to vi? i
152
Reversing
Fill out a table containing
an optimal solution for each sub-instance.
Which is the best path from s to vi?
s, v1,
Base case
t
Original
153
154
Running Time
Time =
# of Sub-Instances
# of Bird Answers
?
= n d
155
Communication Time
optSolk = <optSol[k],vi>
Friend k gives best path from s to vk
to friend i, who adds the edge <vk,vi>.
k
Time =
Size of path = n
Time = n
156
Running Time
Time =
# of Sub-Instances
# of Bird Answers
size of solution
= (n d n)
Store path costs,
not paths
Space =
# of Sub-Instances
= n)
157
158
159
160
161
We also learn
the wise little birds advice.
We will store this in the table too.
162
Running Time
birdsAdvice[i] = kmin
163
164
165
166
167
168
169
170
6
172
6
173
Review
Designing Recursive Back Tracking Algorithm
What are instances, solutions, and costs?
Given an instance I,
What question do you ask the little bird?
Given a bird answer k [K],
What instance sub-Instance do your give your friend?
Assume he gives you optSubSol for subI.
How do you produce an optSol for I from
the birds k and
the friends optSubSol?
How do you determine the cost of optSol from
the birds k and
the cost of the friends optSubSol?
Try all birds answers and take best of best.
174
176
179
180
t
s
181
# of solutions.
186
51
31
4 21
28 35
I ask the bird:
What object is at the root?
I ask my friend:
42
40
63
49
55
71
51
31
42
63
4 21
28 35 40 49
55 71
I ask the bird:
What object is at a leaf?
Bad Question:
How does this help us solve the problem?
What is a good follow up question for the friend
to ask?
188
Printing Neatly
189
Printing Neatly
An instance: text to print neatly & # chars per line
Love life man while there as we be, 11
A solution: # of words to put on each line.
The cost: a measure of how neat,
few blanks on the end of each line.
small punishment
8
8
216 big punishment
27
259
11
The goal is to to print it as neatly as possible. 190
Love.life..
man.while..
there......
as.we.be...
23
23
63
33
=
=
=
=
I combine
birds and
friends answers.
192
193
194
Memoization
Assign one friend to each sub-instance.
Which is the best path from vi to t? i
195
Set of Sub-Instances
Determine the complete set of sub-Instances.
Given an instance I,
Love life man while there as we be, 11
Imagine running the recursive algorithm on
it.
Determine the complete set of subInstances ever given to you, your friends,
their friends,
196
Set of Sub-Instances
Guess the complete set of sub-Instances.
Love life man while there as we be, 11
Love life man while there, 11
Hi there, 81
man while, 11
Yes
No
No
Set of Sub-Instances
The set of sub-Instances is the set of prefixes.
Love life man while there as we be, 11
Love life man while there as we, 11
Love life man while there as, 11
Love life man while there, 11
Love life man while, 11
Love life man, 11
Love life, 11
Love, 11 The set S of sub-Instances needs to:
, 11
include our given I
closed under friend operation
sub-Instance S
198
Set of Sub-Instances
The set of sub-Instances is the set of prefixes.
Love life man while there as we be, 11
Love life man while there as we, 11
Love life man while there as, 11
Love life man while there, 11
Love life man while, 11
Love life man, 11
Love life, 11
Love, 11 The set S of sub-Instances needs to:
, 11
include our given I
closed under friend operation
The bird
sub-Instance S
answers 12words,
subsub-Instance i
34
5
199
Set of Sub-Instances
The set of sub-Instances is the set of prefixes.
Love life man while there as we be, 11
Love life man while there as we, 11
Love life man while there as, 11
Love life man while there, 11
Love life man while, 11
Love life man, 11
Love life, 11
Love, 11 The set S of sub-Instances needs to:
, 11
include our given I
closed under friend operation
each sub-Instance needs to be
asked of some friend, friend,
200
Set of Sub-Instances
The set of sub-Instances is the set of prefixes.
Love life man while there as we be, 11
Love life man while there as we, 11
Love life man while there as, 11
Love life man while there, 11
Love life man while, 11
The bird
answers 1.
Set of Sub-Instances
The set of sub-Instances is the set of prefixes.
The Table
Construct a table
for storing the cost of opt sol and birds advice.
for each sub-instance.
Map
The set of prefixes of words.
Sub-Instances
Indexes
Cell of table
203
Dynamic Programming
Fill out a table containing
an optimal solution for each sub-instance.
Which is the best printing of first i words?
Base case
Original
204
207
Love.life..
man.while..
there.as.we
be.........
208
Love.life..
man.while..
there.as...
we.be......
209
Love.life..
man.while..
there......
as.we.be...
210
there.as.we.be
211
Dynamic
Programming
code always has this
same basic structure.
Amusingly,
when formatting
this code, I had to fight
with line breaks to get
the height/width ratio
Printing Neatly.
214
215
Dynamic Programs
do not recurse making
the instance smaller
and smaller.
Instead, it up front
determines the set S
of all sub-instances
that ever need
to be solved.
Be clear what
sub-instances are.
216
Be clear what
sub-instances are.
217
The set S
of sub-instances
are solved from
smallest to largest
so that no body waits.
Base Cases:
Instances that are too
small to have smaller
instances to give
to friends.
They get solved first
and their solutions
stored.
218
Then we iterate
through the remaining
sub-instances.
From smallest
to largest.
Each gets solved
and their solutions
stored.
Actually, we store the
birds advice instead
of the solution.
219
Consider yourself
to be a friend
working on one
of these.
Be clear which
sub-instance is yours.
Solve this as you
did before.
220
222
223
i-k
i-k
i-k
Be clear what
sub-instance
you give your friend.
224
i-k
i-k
i-k
Instead of recursing,
we simply look
in the table
for the solution.
Because his instance
is smaller, he has
already solved it and
stored sol in the table.
225
optSol<i,k>
is a best solution for
our instance subI[i]
from amongst
those consistent
with the bird's
kth answer.
Take the best
of the best
228
229
Base Cases:
Instances that are too
small to have smaller
instances to give
to friends.
Is this code correct?
230
Dynamic Programs
do not recurse making
the instance smaller
and smaller.
Hence, lets not worry
about our instance I
being a base case.
231
232
233
234
Time =
# of Sub-Instances
# of Bird Answers
= (n n)
Space =
# of Sub-Instances
= n)
236
237
Love.life..
man.while..
there
as we be
<2 ,2 ,1,3>
238
X=asbefchda
Y=rtwabgjcktfd
239
240
242
birds and
friends answers
and give the same Z.
243
244
246
248
249
I combine
birds and
friends answers
and give
250
251
birds and
friends answers
and give
Za = ta.
252
255
Memorization
Assign one friend to each sub-instance.
Which is the best path from vi to t? i
256
Set of Sub-Instances
Determine the complete set of sub-Instances.
Given an instance I,
X=asbetchda
Y=rtwabgjcktfd
Imagine running the recursive alg on it.
Determine the complete set of subInstances ever given to you, your friends,
their friends
Is this a sub-Instance?
X = a s b e t c
Yes
Y = r t w a b g j c k
257
Set of Sub-Instances
Determine the complete set of sub-Instances.
Given an instance I,
X=asbetchda
Y=rtwabgjcktfd
Imagine running the recursive alg on it.
Determine the complete set of subInstances ever given to you, your friends,
their
Is this
a sub-Instance?
friends,
No
X = b e t
Y = a b g j c k
258
Set of Sub-Instances
Determine the complete set of sub-Instances.
Given an instance I,
X=asbetchda
Y=rtwabgjcktfd
Imagine running the recursive alg on it.
Determine the complete set of subInstances ever given to you, your friends,
their
Is this
a sub-Instance?
friends,
Yes
X = x1,xi
i [0..|X|] j [0..|Y|]
Y = y1,,yj |X| |Y| of these.
259
Set of Sub-Instances
Guess the complete set S of sub-Instances.
Xi = x1,xi
Yj = y1,,yj
i [0..|X|] j [0..|Y|]
The set S of sub-Instances needs to:
include our given I
Yes: i = |X| & j = |Y|
260
Set of Sub-Instances
Guess the complete set S of sub-Instances.
Xi = x1,xi
Yj = y1,,yj
i [0..|X|] j [0..|Y|]
The set S of sub-Instances needs to:
include our given I
closed under friend operation
sub-Instance S subsub-Instance S
Xi-1 = x1,xi-1 Yj = y1,,yj
Xi = x1,xi
S Xi = x1,xi Yj-1 = y1,,yj-1
Yj = y1,,yj
Xi-1 = x1,xi-1 Yj-1 = y1,,yj-1
S
261
Set of Sub-Instances
Guess the complete set S of sub-Instances.
Xi = x1,xi
Yj = y1,,yj
i [0..|X|] j [0..|Y|]
The set S of sub-Instances needs to:
include our given I
closed under friend operation
sub-Instance S subsub-Instance S
each sub-Instance needs to be
asked of some friend, friend,
We showed this.
This is a fine set of sub-Instances!262
The Table
Construct a table
for storing the cost of opt sol and birds advice.
for each sub-instance.
Map
Xi = x1,xi
Sub-Instances
Yj = y1,,yj
i [0..|X|] j [0..|Y|]
Indexes
Cell of table
263
The Table
264
Table
Y
Original instance
I = <X,Y>
265
Table
Yj
Xi
j=
sub-Instancei,j =
Xi = x1,xi
Yj = y1,,yj
Optimal Solution
= Longest Common
i=
Subsequence
266
Table
Yj
Xi
sub-Instancei,j =
Xi = x1,xi
Yj = y1,,yj
Optimal Solution
= Longest Common
Subsequence
Birds Advice
delete xi
267
Table
Yj
Xi
sub-Instancei,j =
Xi = x1,xi
Yj = y1,,yj
Optimal Solution
= Longest Common
Subsequence
Birds Advice
delete xi
take both xi and yj
268
Table
Yj
Xi
sub-Instancei,j =
Xi = x1,xi
Yj = y1,,yj
Optimal Solution
= Longest Common
Subsequence
Birds Advice
delete xi
delete yj
take both xi and yj
269
Fill in Box
Yj
sub-Instancei,j =
Xi = x1,xi
Xi
Yj = y1,,yj
Fill in box
Try all birds ans.
delete xi
Friends sub-Instance
Our cost
= friends cost
270
Fill in Box
Yj
sub-Instancei,j =
Xi = x1,xi
Xi
Yj = y1,,yj
Fill in box
Try all birds ans.
delete yj
Friends sub-Instance
Our cost
= friends cost
271
Fill in Box
Yj
sub-Instancei,j =
Xi = x1,xi
Xi
Yj = y1,,yj
Fill in box
Try all birds ans.
take both xi and yj
Friends sub-Instance
Our cost
= friends cost
+1
272
Fill in Box
Yj
sub-Instancei,j =
Xi = x1,xi
Xi
Yj = y1,,yj
Fill in box
Try all birds ans.
Take best of best
273
Fill in Box
Yj
sub-Instancei,j =
Xi = x1,xi
Xi
Yj = y1,,yj
Fill in box
Try all birds ans.
delete xi
Friends sub-Instance
Our cost
= friends cost
274
Fill in Box
Yj
sub-Instancei,j =
Xi = x1,xi
Xi
Yj = y1,,yj
Fill in box
Try all birds ans.
delete yj
Friends sub-Instance
Our cost
= friends cost
275
Fill in Box
Yj
sub-Instancei,j =
Xi = x1,xi
Xi
Yj = y1,,yj
Fill in box
Try all birds ans.
take both xi and yj
Sorry bird is
wrong.
Our cost
= -
276
Fill in Box
Yj
sub-Instancei,j =
Xi = x1,xi
Xi
Yj = y1,,yj
Fill in box
Try all birds ans.
Take best of best
277
Fill in Box
278
Fill in Box
279
280
(later)
281
Base Cases
Base Cases:
general algorithm
does not work
This guys
friends are
282
Base Cases
Base Cases:
general algorithm
does not work
283
Base Cases
284
With Advice
285
With Advice
Done286
Knapsack Problem
287
Knapsack Problem
Ingredients:
Instances: The volume V of the knapsack.
The volume and price of n objects
<<v1,p1>,<v2,p2>, ,<vn,pn>>.
Solutions: A set of objects that fit in the knapsack.
i.e. i S vi V
Cost of Solution: The total value of objects in set.
i.e. i S pi
Goal: Get as much value as you can
288
Greedy Algorithm
Greedy Criteria: Most valuable pi
v=4,p=4
v=7,p=5 v=4,p=4
v=4,p=4
V=8
Greedy give 5
v=4,p=4
V=8
Optimal gives 8
289
Greedy Algorithm
Greedy Criteria: Most dense in value
pi
vi
v=7,p=5
V=8
V=7
Greedy give 4
v=4,p=4
v=4,p=4
v=7,p=5
V=7
Optimal gives 5
290
Greedy Algorithm
of
v=4,p=4
v=7,p=5
V=7
V=7
291
V=12
v=7,p=5
v=4,p=4
v=4,p=4
A solution:
<<v5,p5>,<v9,p9>,...........,<v82,p82>>.
I ask the bird:
What is the last object to take?
# of answers K = n
292
V=12
v=7,p=5
v=4,p=4
v=4,p=4
A solution:
<<v5,p5>,<v9,p9>,...........,<v82,p82>>.
I ask the bird:
Do we keep the last object?
# of answers K = 2 Yes & No
293
V=12
v=7,p=5
v=4,p=4
v=4,p=4
294
V=12-4 v=7,p=5
v=4,p=4
v=4,p=4
His solution:
<<v5,p5>,<v9,p9>,...........,<v82,p82>>.
My solution:
<<v5,p5>,<v9,p9>,...........,<v82,p82>,<vn,pn>>
My cost: same + pn
295
V=12
v=7,p=5
v=4,p=4
v=4,p=4
296
V=12
v=7,p=5
v=4,p=4
v=4,p=4
297
V=12
v=7,p=5
v=4,p=4
v=4,p=4
His solution:
<<v5,p5>,<v9,p9>,...........,<v82,p82>>.
My solution: same
My cost: same
299
Memoization
Assign one friend to each sub-instance.
Which is the best path from vi to t? i
300
Set of Sub-Instances
Determine the complete set of sub-Instances.
Given an instance I,
<V:<v1,p1>,<v2,p2>,<v3,p3>,<v4,p4>,<v5,p5>,<v6,p6>>.
Imagine running the recursive algorithm on it.
Determine the complete set of sub-Instances
ever given to you, your friends, their friends,
Is this a sub-Instance?
<V:<v1,p1>,<v2,p2>,<v3,p3>>.
Yes, if the bird keeps saying No.
301
Set of Sub-Instances
Determine the complete set of sub-Instances.
Given an instance I,
<V:<v1,p1>,<v2,p2>,<v3,p3>,<v4,p4>,<v5,p5>,<v6,p6>>.
Imagine running the recursive algorithm on it.
Determine the complete set of sub-Instances
ever given to you, your friends, their friends,
Is this a sub-Instance?
<V:<v1,p1>,<v2,p2>,<v3,p3>,<v4,p4>,<v5,p5>,<v6,p6>>.
No, the set of objects is always a prefix
of the original set.
302
Set of Sub-Instances
Determine the complete set of sub-Instances.
Given an instance I,
<V:<v1,p1>,<v2,p2>,<v3,p3>,<v4,p4>,<v5,p5>,<v6,p6>>.
Imagine running the recursive algorithm on it.
Determine the complete set of sub-Instances
ever given to you, your friends, their friends,
Quite possibly, if V V.
It is easier to solve than to
Is this a sub-Instance? determine if it is a sub-instance.
<V:<v1,p1>,<v2,p2>,<v3,p3>>.
303
Set of Sub-Instances
My instance:
<V:<v1,p1>,<v2,p2>,..........................,<vn,pn>>.
Guess the complete set S of sub-Instances.
V [0..V]
<V:<v1,p1>,<v2,p2>,......,<vi,pi>>.
i [0..n]
The set S of sub-Instances needs to:
include our given I
Yes: V=V & i = n
closed under friend operation
sub-Instance S subsub-Instance S
<V:<v1,p1>,<v2,p2>,......,<vi,pi>> S
No <V :<v1,p1>,<v2,p2>,...,<vi-1,pi-1>>
S
Yes <V-vi:<v1,p1>,<v2,p2>,...,<vi-1,pi-1>>
304
The Table
Construct a table
for storing the cost of opt sol and birds advice.
for each sub-instance.
Map
<V:<v1,p1>,<v2,p2>,......,<vi,pi>>.
Sub-Instances
Indexes
Cell of table
V [0..V]
i [0..n]
305
The Table
The complete set S of sub-Instances.
<V:<v1,p1>,<v2,p2>,......,<vi,pi>>.
OptSol Cost
0
&
1
Birds Advice
2
for this
sub-Instance
No i-1
Yes
i
Our cost? n
0 1 2
V-vi
V [0..V]
i [0..n]
V
V
same + pi
same
Take best
of best.
306
The Table
The complete set S of sub-Instances.
<V:<v1,p1>,<v2,p2>,......,<vi,pi>>.
OptSol Cost
0
&
1
Birds Advice
2
for this
sub-Instance
i-1
Order to fill
i
so nobody
waits?
n
0 1 2
V-vi
V [0..V]
i [0..n]
V
V
307
The Code
308
309
310
Running Time
My instance:
<V:<v1,p1>,<v2,p2>,..........................,<vn,pn>>.
The complete set S of sub-Instances is V [0..V]
<V:<v1,p1>,<v2,p2>,......,<vi,pi>>.
i [0..n]
No
Yes
Running time
= ( # of sub-instances # bird answers )
= (
Vn
2
)
= ( 2#bits in V n )
Polynomial?
Exponential in size in instance!
311
312
done
315
Schedule as
many events
in your room
as possible
316
317
Greedy Algorithm
318
319
Greedy Algorithm
1
100
320
321
322
323
Carefull
I ask my friend:
<<s1,f1,w1>,<s2,f2,w2>, ,<sn,fn,wn>>.
His solution:
<<s5,f5,w5>,<s9,f9,w9>, ,<s82,f82,w82>>.
My solution: same + <sn,fn,wn>. My cost: same +w
324n
Bird answers:
Yes keep the last event.
Give the rest to my friend.
No this solution
is not valid!
Bird answers:
No we trust the
Yes keep the last event.
bird!
Then I should politely tell the bird
she is wrong
326
Bird answers:
No we trust the
Yes keep the last event.
bird!
You only tell her she is wrong
if you really know.
Eg k words dont fit on the last line
The bear does not fit into the knapsack
327
Bird answers:
No we trust the
Yes keep the last event.
bird!
Your friend could have just as easily
given you this subsolution that does not
conflict with the birds answer.
328
Bird answers:
No we trust the
Yes keep the last event.
bird!
Or maybe he needs to make a sacrifice
when finding his answer in order that the
overall solution is the best.
329
Bird answers:
No we trust the
Yes keep the last event.
bird!
Or goal now is to find the best solution to
our instance that is consistent with the
birds answer.
Then we will take the best of best.
330
Bird answers:
No we trust the
Yes keep the last event.
bird!
Dude! It is your job to give me the right
subinstance so that I give you a
subsolution that does not conflict with the
bird
331
Bird answers:
Yes keep the last event.
Cant keep any events that overlap with it.
332
Bird answers:
Yes keep the last event.
I ask my friend:
333
Bird answers:
Yes keep the last event.
I ask my friend:
334
Bird answers:
Yes keep the last event.
I ask my friend:
My solution: same + <sn,fn,wn>.
Valid?
Yes
335
Bird answers:
Yes keep the last event.
I ask my friend:
My solution: same + <sn,fn,wn>. My cost: same +wn
336
337
Memorization
Assign one friend to each sub-instance.
Which is the best path from vi to t? i
338
Set of Sub-Instances
Determine the complete set of sub-Instances.
Given an instance I,
Imagine running the recursive algorithm on it.
Determine the complete set of sub-Instances
ever given to you, your friends, their friends,
339
341
Set of Sub-Instances
My instance:
<<s1,f1,w1>,<s2,f2,w2>,................ ,<sn,fn,wn>>.
Guess the complete set S of sub-Instances.
<<s1,f1,w1>,<s2,f2,w2>, ,<si,fi,wi>> i [0..n]
The set S of sub-Instances needs to:
include our given I
Yes: i = n
closed under friend operation
sub-Instance S subsub-Instance S ?
each sub-Instance needs to be
asked of some friend, friend Only n sub-Instances
Good enough. 342
Set of Sub-Instances
<<s1,f1,w1>,<s2,f2,w2>,................................,<si,fi,wi>>
Events sorted by finishing time.
343
Set of Sub-Instances
<<s1,f1,w1>,<s2,f2,w2>,................................,<si,fi,wi>>
Bird answers:
Set of Sub-Instances
<<s1,f1,w1>,<s2,f2,w2>,.....,<sj,fj,wj>>
Bird answers:
Set of Sub-Instances
<<s1,f1,w1>,<s2,f2,w2>,.....,<sj,fj,wj>>
Event j is kept
fj si
set of kept jobs is a prefix of events.
subsub-Instance S
346
Set of Sub-Instances
My instance:
<<s1,f1,w1>,<s2,f2,w2>,................ ,<sn,fn,wn>>.
The complete set S of sub-Instances is
<<s1,f1,w1>,<s2,f2,w2>, ,<si,fi,wi>> i [0..n]
Table:
0, 1,
Base case
2,
3, 4, .
n
Original
347
Set of Sub-Instances
My instance:
<<s1,f1,w1>,<s2,f2,w2>,................ ,<sn,fn,wn>>.
The complete set S of sub-Instances is
<<s1,f1,w1>,<s2,f2,w2>, ,<si,fi,wi>> i [0..n]
Running time
= # of sub-instances # bird answers
=
n
2
But to find your friends yes sub-instance
you must know how many events overlap
with your last event. This takes time:
O(logn) using binary search
Done348
for a total of O(nlogn) time.
Parsing
Input: s=6*8+((2+42)*(5+12)+987*7*123+15*54)
Output:
349
Parsing
Recursive Alg:
GetExp calls GetTerm
GetTerm calls GetFact
GetFact calls GetExp
350
Parsing
T AB
CA
TT
A AA
BT
a
B TA
BC
b
e
C CB
AC
c
d
351
Parsing
T AB
CA
TT
A AA
BT
a
B TA
BC
b
e
C CB
AC
c
d
T
A
C
C
A
A
C B
A C
T
C A
T
A B
b a e a a d b b d a352
Parsing
T AB
CA
TT
A AA
BT
a
B TA
BC
b
e
C CB
AC
c
d
T
C
Recursive Algorithm:
C
GetT does not know
B T
A
whether to call
A
B
C
A
A
C
GetA, GetC, or GetT.
A C
B T
A B
b a e a a d b b d a353
Parsing
T AB
CA
TT
A AA
BT
a
B TA
BC
b
e
C CB
AC
c
d
T
A
C
C
A
A
C B
A C
T
C A
T
A B
b a e a a d b b d a354
Parsing
T AB
CA
TT
A AA
BT
a
B TA
BC
b
e
C CB
AC
c
d
T
C
?
b a e a a d b b d a355
Parsing
T AB
CA
TT
A AA
BT
a
B TA
BC
b
e
C CB
AC
c
d
Parsing
T AB
CA
TT
A AA
BT
a
B TA
BC
b
e
C CB
AC
c
d
Parsing
T AB
CA
TT
A AA
BT
a
B TA
BC
b
e
C CB
AC
c
d
b a e a a d b b d a358
Parsing
T AB
CA
TT
A AA
BT
a
B TA
BC
b
e
C CB
AC
c
d
A
Ask left friend:
C
Instance: C baeaadb
C
A
Solution: Left parsing
A
A C B
A C
B T
A B
b a e a a d b b d a359
Parsing
T AB
CA
TT
A AA
BT
a
B TA
BC
b
e
C CB
AC
c
d
T
C
A
B
T
C A
b a e a a d b b d a360
Parsing
T AB
CA
TT
A AA
BT
a
B TA
BC
b
e
C CB
AC
c
d
A
Combine:
C
Instance:
C
B T
A
Birds Answer
Left Friends Answer A
C A
A C B
Right Friends Answer
A C
B T
A B
b a e a a d b b d a361
362
363
Memoization
Assign one friend to each sub-instance.
Which is the best path from vi to t? i
364
Set of Sub-Instances
Determine the complete set of sub-Instances.
Given an instance I,
Imagine running the recursive algorithm on it.
Determine the complete set of sub-Instances
ever given to you, your friends, their friends,
365
Set of Sub-Instances
Determine the complete set of sub-Instances.
My instance I:
T a1a2a3 ..... an
gives:
My left sub-Instance.
gives:
His right sub-Instance.
b a e a a d b b d a366
Set of Sub-Instances
Determine the complete set of sub-Instances.
My instance I:
T a1a2a3 ..... an
sub-Instances:
T aiai+1 ..... aj
T= C
non-terminals T
i,j [1,n]
# of sub-Instances = # of non-terminals n2
a1...ai-1 aiai+1...aj aj+1...an
a d b
367
The Table
Construct a table
for storing the cost of opt sol and birds advice.
for each sub-instance.
Map
Sub-Instances
T aiai+1 ..... aj
T i,j [1,n]
Indexes
T
Cell of table
j
368
369
Running Time
Running time
= ( # of sub-instances
# bird answers )
= ( # of non-terminals n2 # of rules n )
sub-Instances:
T aiai+1 ..... aj non-terminals T
gives: First rule and split
& i,j [1,n]
Done
370
true
true
true
true
false true
true
371
false
true
373
x1
x1 x2
x2
374
Memoization
Assign one friend to each sub-instance.
Which is the best path from vi to t? i
375
Set of Sub-Instances
Determine the complete set of sub-Instances.
Given an instance I,
Imagine running the recursive algorithm on it.
Determine the complete set of sub-Instances
ever given to you, your friends, their friends,
376
Set of Sub-Instances
Determine the complete set of sub-Instances.
Given an instance I,
c = (x1 or y1) and (x2 or y2) and (x3 or y3) and (x4 or y4)
Is this a sub-Instance?
c = (x1)
and (x3)
Commit to
y1=0,
y2=1,
y3=0,
y4=1,
and simplify
Yes
True for any subset of the xi.
could be an exponential # of different sub-Instances.
running time is exponential.
377
x1
x1 x2
x2
378
x1
x1 x2
x2
379
x1
x1 x2
x2
380
x1
x1 x2
x2
381
382
383
Review
Designing Recursive Back Tracking Algorithm
What are instances, solutions, and costs?
Given an instance I,
What question do you ask the little bird?
Given a bird answer k [K],
What sub-Instance do your give your friend?
Assume he gives you optSubSol for sub-Instance.
How do you produce an optSol for I from
the birds k and
the friends optSubSol?
How do you determine the cost of optSol from
the birds k and
the cost of the friends optSubSol?
Try all birds answers and take best of best.
384
Optimization Problems
What is an instance
What are the objects in an instance
What is a solution
What are the objects in a solution
What is the cost of a solution
Greedy algorithm
386
Dynamic Programming
Donts
Yes, the code has a basic structure that you should learn.
But dont copy other code verbatim
Dont say if(ai = cj)
(i.e. Longest Common Subsequence)
when our problem does not have cj
387
Dynamic Programming
Donts
When looping over the sub-instances
be clear what the set of sub-instances are
which is currently being solved,
i.e. which instance is cost(i,j)?
If you know that the set of sub-instances are the
prefixes of the input, i.e. <a1,a2, , ai>,
then dont have a two dimensional table.
Table[1..n,1..n].
Dont loop over i and loop over j if j never gets
mentioned again.
388
Dynamic Programming
Donts
When trying all bird answers
be clear what the set of bird answers are,
which is currently being tried,
& what it says about the solution being looked for.
When getting help from your friend,
be clear what the sub-instance is that you are giving him
How do you use the current instance and the bird' s
answer to form his sub-instance?
Dont simply say cost(i-1,j-1)
389
Dynamic Programming
Donts
Think about what the base cases should be.
Dont make an instance a base cases
if they can be solved using the general method.
% is used to start a comment.
Dont put it in front of code.
390
25
51
17
31
21
28
42
35
40
63
49
55
71
Matrix Multiplication
392
393
394
395
396
397
398
399
400
401
402
end
403
Dynamic Programming
Construct a table
for storing an optimal solution & cost
for each sub-instance.
Map
i Best path from vi to t?
Sub-Instances
i [n], i.e. for each node vi
Index
Cell of table
v,
v , vi , v , .,
404