Documente Academic
Documente Profesional
Documente Cultură
I.
Amortized Analysis
II.
Data structures
Static problems. Given an input, produce an output.
Ex. Sorting, FFT, edit distance, shortest paths, MST, max-flow, ...
33
22
55
23
16
63
86
33
10
44
46
86
83
90
33
93
47
99
60
2
Appetizer
Goal. Design a data structure to support all operations in O(1) time.
Assumptions.
Appetizer
Data structure. Three arrays A[1.. n], B[1.. n], and C[1.. n], and an integer k.
A[ ]
22
55
99
33
B[ ]
C[ ]
k=4
A[4]=99, A[6]=33, A[2]=22, and A[3]=55 initialized in that order
4
Appetizer
INIT (A, n)
READ (A, i)
___________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________
___________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________
IF (INITIALIZED (A[i]))
IF (INITIALIZED (A[i]))
___________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________
k 0.
A MALLOC(n).
B MALLOC(n).
C MALLOC(n).
RETURN A[i].
ELSE
RETURN 0.
A[i] value.
ELSE
k k + 1.
___________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________
___________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________
A[i] value.
B[i] k.
C[k] i.
INITIALIZED (A, i)
___________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________
_______________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________
Appetizer
Theorem. A[i] is initialized iff both 1 B[i] k and C[B[i]] = i.
Pf.
A[ ]
22
55
99
33
B[ ]
C[ ]
k=4
A[4]=99, A[6]=33, A[2]=22, and A[3]=55 initialized in that order
6
Appetizer
Theorem. A[i] is initialized iff both 1 B[i] k and C[B[i]] = i.
Pf.
A[ ]
22
55
99
33
B[ ]
C[ ]
k=4
A[4]=99, A[6]=33, A[2]=22, and A[3]=55 initialized in that order
7
A MORTIZED A NALYSIS
binary counter
multipop stack
dynamic table
Amortized analysis
Worst-case analysis. Determine worst-case running time of a data structure
operation as function of the input size.
Ex. Starting from an empty stack implemented with a dynamic table, any
sequence of n push and pop operations takes O(n) time in the worst case.
Splay trees.
Dynamic table.
Fibonacci heaps.
Garbage collection.
Move-to-front list updating.
Push-relabel algorithm for max flow.
Path compression for disjoint-set union.
Structural modifications to red-black trees.
Security, databases, distributed computing, ...
SIAM J. ALG. DISC. METH.
Vol. 6, No. 2, April 1985
10
A MORTIZED A NALYSIS
binary counter
multipop stack
dynamic table
CHAPTER 17
Binary counter
Goal. Increment a k-bit binary counter (mod 2k).
Counter
value
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
A[
7
A[ ]
6
A[ ]
5
A[ ]
4
A[ ]
3
A[ ]
2
A[ ]
1
A[ ]
0]
17.1 Aggregate
analysis bit of counter.
Representation. aj = jth least
significant
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
1
0
0
0
0
0
0
0
0
1
1
1
1
1
1
1
1
0
0
0
0
0
1
1
1
1
0
0
0
0
1
1
1
1
0
0
0
1
1
0
0
1
1
0
0
1
1
0
0
1
1
0
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
455
Total
cost
0
1
3
4
7
8
10
11
15
16
18
19
22
23
25
26
31
Figure 17.2 An 8-bit binary counter as its value goes from 0 to 16 by a sequence of 16 I NCREMENT
Bits that flip to achieve the next value are shaded. The running cost for flipping bits is
ofoperations.
bits flipped.
shown at the right. Notice that the total cost is always less than twice the total number of I NCREMENT
operations.
12
operations on an initially zero counter causes A1! to flip bn=2c times. Similarly,
Binary counter
Goal. Increment a k-bit binary counter (mod 2k).
Counter
value
A[
7
A[ ]
6
A[ ]
5
A[ ]
4
A[ ]
3
A[ ]
2
A[ ]
1
A[ ]
0]
17.1 Aggregate
analysis bit of counter.
Representation. aj = jth least
significant
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
1
0
0
0
0
0
0
0
0
1
1
1
1
1
1
1
1
0
0
0
0
0
1
1
1
1
0
0
0
0
1
1
1
1
0
0
0
1
1
0
0
1
1
0
0
1
1
0
0
1
1
0
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
455
Total
cost
0
1
3
4
7
8
10
11
15
16
18
19
22
23
25
26
31
Bits that flip to achieve the next value are shaded. The running cost for flipping bits is
shown at the right. Notice that the total cost is always less than twice the total number of I NCREMENT
operations.
flipped
per increment.
13
operations on an initially zero counter causes A1! to flip bn=2c times. Similarly,
Counter
value
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
455
A[
7
A[ ]
6
A[ ]
5
A[ ]
4
A[ ]
3
A[ ]
2
A[ ]
1
A[ ]
0]
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
1
0
0
0
0
0
0
0
0
1
1
1
1
1
1
1
1
0
0
0
0
0
1
1
1
1
0
0
0
0
1
1
1
1
0
0
0
1
1
0
0
1
1
0
0
1
1
0
0
1
1
0
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
Total
cost
0
1
3
4
7
8
10
11
15
16
18
19
22
23
25
26
31
Figure 17.2 An 8-bit binary counter as its value goes from 0 to 16 by a sequence of 16 I NCREMENT
operations. Bits that flip to achieve the next value are shaded. The running cost for flipping bits is
shown at the right. Notice that the total cost is always less than twice the total number of I NCREMENT
operations.
14
operations on an initially zero counter causes A1! to flip bn=2c times. Similarly,
k 1
k 1
j=0
j=0
n
n
2jj
2
<
<
=
=
1
1
n
j
n
2
j
j=0 2
j=0
2n
2n
ci
i=1
ci
i=1
16
i=1
ci
ci
i=1
Theorem. Starting from the initial data structure D0, the total actual cost of
any sequence of n operations is at most the sum of the amortized costs.
n
ci
i=1
ci .
i=1
17
Accounting.
Flip bit j from 0 to 1: charge two credits (use one and save one in bit j).
increment
7
0
1
18
Accounting.
Flip bit j from 0 to 1: charge two credits (use one and save one in bit j).
Flip bit j from 1 to 0: pay for it with the one credit saved in bit j.
increment
7
1
0
0
1
0
1
1
0
1
0
19
Accounting.
Flip bit j from 0 to 1: charge two credits (use one and save one in bit j).
Flip bit j from 1 to 0: pay for it with the one credit saved in bit j.
7
20
Accounting.
Flip bit j from 0 to 1: charge two credits (use one and save one in bit j).
Flip bit j from 1 to 0: pay for it with the one credit saved in bit j.
Pf.
21
(D0) = 0.
(Di) 0 for each data structure Di.
ci
i
22
(D0) = 0.
(Di) 0 for each data structure Di.
ci
i
Theorem. Starting from the initial data structure D0, the total actual cost of
any sequence of n operations is at most the sum of the amortized costs.
Pf. The amortized cost of the sequence of operations is:
n
n
n
n
ccii =
=
i=1
i=1
(c
(cii +
+ (D
(Dii ))
(D
(Dii
ccii +
+ (D
(Dnn ))
(D
(D00 ))
)
1
1)
i=1
i=1
n
n
=
=
i=1
i=1
n
n
ccii
i=1
i=1
23
(D0) = 0.
(Di) 0 for each Di.
increment
0
1
24
(D0) = 0.
(Di) 0 for each Di.
increment
0
1
1
0
1
0
1
0
1
0
25
(D0) = 0.
(Di) 0 for each Di.
26
(D0) = 0.
(Di) 0 for each Di.
27
Splay trees.
log2 size(x)
(T ) =
x T
Preflow-push.
(f ) =
height(v)
v : excess(v) > 0
Red-black trees.
w(x)
(T ) =
x T
w(x) =
0
1
0
2
B7
B7
B7
B7
x
x
x
x
Bb
Bb
Bb
Bb
`2/
#H+F M/ ?b MQ `2/ +?BH/`2M
#H+F M/ ?b QM2 `2/ +?BH/
#H+F M/ ?b irQ `2/ +?BH/`2M
28
A MORTIZED A NALYSIS
binary counter
multipop stack
dynamic table
SECTION 17.4
Multipop stack
Goal. Support operations on a set of elements:
MULTIPOP (S, k)
___________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________
FOR i = 1 TO k
POP (S).
___________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________
Multipop stack
Goal. Support operations on a set of elements:
overly pessimistic
upper bound
31
An object is popped at most once for each time it is pushed onto stack.
There are n PUSH operations.
Thus, there are n POP operations
(including those made within MULTIPOP).
32
PUSH(S, x):
Pf. The algorithm maintains the invariant that every object remaining on
the stack has 1 credit number of credits in data structure 0.
33
(D0) = 0.
(Di) 0 for each Di.
= 1 + 1 = 2.
34
(D0) = 0.
(Di) 0 for each Di.
= 1 1 = 0.
35
(D0) = 0.
(Di) 0 for each Di.
36
A MORTIZED A NALYSIS
binary counter
multipop stack
dynamic table
SECTION 17.4
Dynamic table
Goal. Store items in a table (e.g., for hash table, binary heap).
Two operations:
Requirement:
overly pessimistic
upper bound
38
new size
cost
16
insert
ci =
i
1
B7 i 1 Bb M 2t+i TQr2` Q7 k
Qi?2`rBb2
Starting from empty table, the cost of a sequence of n INSERT operations is:
n
n
i=1
i=1
ci
ci
lg n
lg n
n +
n +
<
<
=
=
j=0
j=0
2jj
2
n + 2n
n + 2n
3n
3n
40
10
11
12
13
14
15
16
41
INSERT:
Pf. The algorithm maintains the invariant that there are 2 credits with each
item in right half of table.
When table doubles, one-half of the items in the table have 2 credits.
This pays for the work needed to double the table.
42
capacity of
array
43
number of
elements
capacity of
array
Actual cost ci = 1.
(Di) (Di1) = 2.
Amortized costs i = ci + (Di)
(Di1) = 1 + 2 = 3.
Efficient solution.
45
insert
10
11
12
10
11
12
delete
46
INSERT: charge 3 credits (1 credit for insert; save 2 with new item).
DELETE: charge 2 credits (1 credit to delete, save 1 in emptied slot).
Pf. The algorithm maintains the invariant that there are 2 credits with each
item in the right half of table; 1 credit with each empty slot in the left half.
When table doubles, each item in right half of table has 2 credits.
When table halves, each empty slot in left half of table has 1 credit.
47
Pf sketch.
(Di ) =
2 size(Di )
capacity(Di )
1
size(Di )
2 capacity(Di )
B7 (Di ) 1/2
B7 (Di ) < 1/2
...
48