CSCE 310: Review of Algorithm Analysis

Ryan Patrick

Outline
Algorithm Analysis
Asymptotic Notation
Non-Recursive Algorithms
Recursive Algorithms

Review
Patrick
Analysis
Asymptotic
Non-Recursive
Recursive Algorithms
Recurrence Relations

Recurrence Relations

Recursive Algorithm Analysis

The Master Theorem
Counting
Special Cases
The Binomial Coefficient
Applications

Recursive
The Master Theorem

Counting
Special Cases

Binomial
Coefficient
Applications

31

Outline
Algorithm Analysis
Asymptotic Notation
Non-Recursive Algorithms
Recursive Algorithms

Review
Patrick
3

Analysis
Asymptotic
Non-Recursive
Recursive Algorithms
Recurrence Relations

Recurrence Relations

Recursive Algorithm Analysis

The Master Theorem
Counting
Special Cases
The Binomial Coefficient
Applications

Recursive
The Master Theorem

Counting
Special Cases

Binomial
Coefficient
Applications

31

Asymptotic Notation
Review
Patrick
Analysis
4

Asymptotic
Non-Recursive
Recursive Algorithms
Recurrence Relations

Recursive
The Master Theorem

Measure of General Performance

Counting

Relative or Absolute

Special Cases

Binomial
Coefficient
Applications

31

Asymptotic Notation
Notations
Review
Patrick

Notation

Mathematical Definition

Analysis
5

O (g (n))

f (n) O (g (n)) if there is a positive constant c and natural

number n0 such that f (n) c g (n) for all n n0

Asymptotic
Non-Recursive
Recursive Algorithms
Recurrence Relations

Recursive
The Master Theorem

Counting

(g (n))

f (n) (g (n)) if there is a positive constant c and natural

number n0 such that f (n) c g (n) for all n n0

Special Cases

Binomial
Coefficient
Applications

(g (n))

f (n) (g (n)) if there are positive constants c1 and c2 and

natural number n0 such that f (n) c1 g (n) and f (n)
c2 g (n) for all n n0

31

Non-Recursive Algorithms
MaxElement
Review
Patrick

maxElement.js
function maxElement ( array )
{
maxVal = array [ 0 ];

Analysis
6

Asymptotic
Non-Recursive
Recursive Algorithms
Recurrence Relations

Recursive
The Master Theorem

for ( var i = 1; i < array . length ; i ++ )

{
if ( array [ i ] > maxVal )
{
maxVal = array [ i ];
}
}
return maxVal ;

Counting
Special Cases

Binomial
Coefficient
Applications

}
31

Non-Recursive Algorithms
MatrixMultiplication

matMult.js
function m a t r i x M u l t i p l i c a t i o n ( a , b )
{
// a and b are square , nxn matrices
for ( var i =
{
for ( var
{
c[ i
for (
{

0; i < n ; i ++ )
j = 0; j < n ; j ++ )
][ j ] = 0.0;
var k = 0; k < n ; k ++ )
c [ i ][ j ] = c [ i ][ j ] + a [ i ][ k ] * b [ k ][ j ];

}
}
}
return c ;
}

Non-Recursive Algorithms
Binary
Review
Patrick
Analysis

binary.js
function binary ( n )
{
var count = 1;

Asymptotic
Non-Recursive
Recursive Algorithms
Recurrence Relations

Recursive
The Master Theorem

Counting

while ( n > 1 )
{
count = count + 1;
n = Math . floor ( n / 2 ) ;
}

Special Cases

Binomial
Coefficient
Applications

return count ;
}

31

Recursive Algorithms
Review
Patrick
Analysis

1. Break problem into smaller (but similar) subproblems

Asymptotic
Non-Recursive
Recursive Algorithms
Recurrence Relations

Recursive

2. Solve the smallest possible subproblem

The Master Theorem

Counting

3. (Optionally) recombine the subproblem solutions to solve the original

problem

Special Cases

Binomial
Coefficient

Idea Will be Used in Dynamic Programming

Applications

31

Recurrence Relations
Review
Patrick
Analysis

10

Asymptotic
Non-Recursive
Recursive Algorithms
Recurrence Relations

Recursive
The Master Theorem

Mathematical formula describing execution of a recursive algorithm

Counting

Defined for problem of size n

Special Cases

Binomial
Coefficient
Applications

31

Recurrence Relations
Example: T (n) = T (n 1) + 2

T (n)

= T (n 1)
= T (n 2)
= T (n 3)

Review

+2
+2
+2

+2
+2

Patrick

..
.
= T (n k)
= T (1)
= O (n)

+2k
+2 (n 1)

Analysis

+2
11

Asymptotic
Non-Recursive
Recursive Algorithms
Recurrence Relations

Recursive
The Master Theorem

For k = n 1

Counting
Special Cases

Binomial
Coefficient
Applications

31

Recurrence Relations c

Example: T (n) = T (n 1) + n for c 1

T (n)

= T (n k)
= T (1)

+nc
+ (n 1)c
+ (n 2)c
..
.
Pk1
+ i=0 (n i)c
P
c
+ n2
i=0 (n i)

= T (1)

+nc

= T (n 1)
= T (n 2)
= T (n 3)

= T (1)

= O nc+1

(n 1)

Review

+nc
+ (n 1)c

Patrick
Analysis

+nc
12

Asymptotic
Non-Recursive
Recursive Algorithms
Recurrence Relations

Recursive
The Master Theorem

Counting

For k = n 1

Special Cases

Binomial
Coefficient

nc is largest term
in
polynomial,
summed n 1
times

Applications

+nc+1
31

Recurrence Relations


Example: T (n) = T

T (n)

n +1

 1
= T n2
 1
= T n4
 1
= T n8
..
.
 1
= T n 2k

+k

= T (c)

+k

Review

+1
+1
+1

Patrick

+1
+1

Analysis

+1
13

Asymptotic
Non-Recursive
Recursive Algorithms
Recurrence Relations

Recursive
The Master Theorem

Counting
Special Cases

Binomial
Coefficient

if n 2k = c (a small constant)
..
.

Applications

31

Recurrence Relations


Example: T (n) = T

n + 1 (continued)

= n 2k

log2 c

1
2k

Review
Patrick
Analysis

log2 n

Rules of logarithms
14

2k

Asymptotic
Non-Recursive
Recursive Algorithms
Recurrence Relations

Recursive

log2 n
log2 c

The Master Theorem

Counting
Special Cases

= logc n

Binomial
Coefficient
Applications

= log2 logc n

31

Recurrence Relations


Example: T (n) = T

T (n)

n + 1 (continued)

 1
= T n2
 1
= T n4
 1
= T n8


=T n

1
2k

Review

+1
+1
+1

Patrick

+1
+1
..
.

Analysis

+1
15

Asymptotic
Non-Recursive
Recursive Algorithms
Recurrence Relations

Recursive
The Master Theorem

+k

Counting
Special Cases

= T (c)

+k

Binomial
Coefficient

if n 2k = c (a small constant)
= T (c)
= O (log log n)

Applications

+ log2 logc n

31

Outline
Algorithm Analysis
Asymptotic Notation
Non-Recursive Algorithms
Recursive Algorithms

Review
Patrick
Analysis
Asymptotic
Non-Recursive
Recursive Algorithms
Recurrence Relations

Recurrence Relations

Recursive Algorithm Analysis

The Master Theorem
Counting
Special Cases
The Binomial Coefficient
Applications

16

Recursive
The Master Theorem

Counting
Special Cases

Binomial
Coefficient
Applications

31

Recursive Algorithm Analysis

Review
Patrick
Analysis
Asymptotic
Non-Recursive
Recursive Algorithms
Recurrence Relations

Seen Non-Recursive Algorithm Analysis

17

Recursive
The Master Theorem

Seen Some Recursive Algorithm Analysis

Counting

Analysis with Subproblems of Equal Size?

Binomial
Coefficient

Special Cases

Applications

31

The Master Theorem

Review
Patrick

T (n) = aT

n
b

+ f (n) T (n) nd if a
< b d
T (n) nd log
n if a = b d
T (n) nlogb a if a > b d

a is the number of subproblems

1
b

Analysis
Asymptotic
Non-Recursive
Recursive Algorithms
Recurrence Relations

Recursive
18

The Master Theorem

Counting

is the size of each subproblem (relative to the larger problem)

Special Cases

Binomial
Coefficient

f (n) is the function that combines solutions to subproblems

Applications

a is any number > 1

b is any number > 1


f (n) nd
31

Outline
Algorithm Analysis
Asymptotic Notation
Non-Recursive Algorithms
Recursive Algorithms

Review
Patrick
Analysis
Asymptotic
Non-Recursive
Recursive Algorithms
Recurrence Relations

Recurrence Relations

Recursive Algorithm Analysis

The Master Theorem
Counting
Special Cases
The Binomial Coefficient
Applications

Recursive
The Master Theorem
19

Counting
Special Cases

Binomial
Coefficient
Applications

31

Some Purposes of Counting in Computing

Review
Patrick
Analysis
Asymptotic
Non-Recursive
Recursive Algorithms
Recurrence Relations

Recursive

Feasibility

The Master Theorem

20

Confirmation

Counting
Special Cases

Binomial
Coefficient
Applications

31

Feasibility
Review
Patrick
Analysis
Asymptotic
Non-Recursive
Recursive Algorithms
Recurrence Relations

5220 > 2 1034 possible 20-character long passwords that only contain
uppercase and lowercase letters
Is a brute force attack feasible if you can check 1, 000, 000, 000 possible
passwords per second?

Recursive
The Master Theorem
21

Counting
Special Cases

Binomial
Coefficient

> 6 1017 years required to check all possible passwords

Applications

31

Confirmation
Review
Patrick
Analysis
Asymptotic
Non-Recursive
Recursive Algorithms
Recurrence Relations

dermatoglyphics (the skin patterns on hands and feet) contains 15 unique

letters

Recursive
The Master Theorem
22

15! = 2, 004, 310, 016 possible, unique permutations of the letters

Counting
Special Cases

Binomial
Coefficient

Confirmation simplified

Applications

31

Duplicates
How many possible unique permutations of baseball are there?
Review
Patrick
Analysis
Asymptotic
Non-Recursive
Recursive Algorithms
Recurrence Relations

A. 8! (40, 320)
B.
C.
D.
E.

8!
3
8!
6

Recursive

(13, 440)
(6, 720)

The Master Theorem

Counting
23

8!
2!2!2! (5, 040)
8!
2! (20, 160)

Special Cases

Binomial
Coefficient
Applications

31

Ordering
Review
Patrick
Analysis
Asymptotic
Non-Recursive
Recursive Algorithms
Recurrence Relations

There are 66 students in this class

How many unique ways are there to have
of the students remain seated?

2
3

of the students stand up and

1
3

Recursive
The Master Theorem

Counting

Does it matter if Person A stands up before Person B?

24

Special Cases

Binomial
Coefficient

Is it easier to choose the fraction of students who will stand up (and

assume that the remainder will stay seated) or the fraction of students who
will stay seated (and assume that the remainder of students will stand up)?

Applications

31

Outline
Algorithm Analysis
Asymptotic Notation
Non-Recursive Algorithms
Recursive Algorithms

Review
Patrick
Analysis
Asymptotic
Non-Recursive
Recursive Algorithms
Recurrence Relations

Recurrence Relations

Recursive Algorithm Analysis

The Master Theorem
Counting
Special Cases
The Binomial Coefficient
Applications

Recursive
The Master Theorem

Counting
Special Cases
25

Binomial
Coefficient
Applications

31

The Binomial Coefficient

Review
Patrick
Analysis

Represented as:

Asymptotic
Non-Recursive
Recursive Algorithms
Recurrence Relations

n

k

Recursive
The Master Theorem

Counting

Read as: n choose k

Represents the number of ways k items can be simultaneously selected from
a larger collection of n items

Special Cases
26

Binomial
Coefficient
Applications

31

The Formula
Review
Patrick
Analysis

n

n!
= k!(nk)!

Asymptotic
Non-Recursive
Recursive Algorithms
Recurrence Relations

Number of ways to order all

items

Recursive
The Master Theorem

Counting
Special Cases

Number of ways to order chosen

items

27

Binomial
Coefficient
Applications

Number of ways to order remaining items

31

Example:

3
2

3!
2!(32)!

6
2

=3
Review
Patrick
Analysis

,,,
,,/
,/,
/,,

Asymptotic
Non-Recursive
Recursive Algorithms
Recurrence Relations

Recursive
The Master Theorem

Counting
Special Cases
28

Binomial
Coefficient
Applications

31

Example: Basketball Team

Review
Patrick
Analysis
Asymptotic
Non-Recursive
Recursive Algorithms
Recurrence Relations

2 centers, 3 forwards, 7 guards on roster

Recursive

1 center, 2 forwards, 2 guards play at a time

The Master Theorem

Counting

How many different squads can the coach assemble?

2 3 7
1

Special Cases

Binomial
Coefficient

2 3 21 = 126

29

31

Applications

Example: Versatile Basketball Team

Review
Patrick
Analysis
Asymptotic
Non-Recursive
Recursive Algorithms
Recurrence Relations

2 centers, 3 forwards, 5 guards . . . and 2 players who can play either guard
or forward on roster

Recursive

1 center, 2 forwards, 2 guards play at a time

The Master Theorem

Counting

How many different squads can the coach assemble?

   
2
3
5
1
|

2
{z

2
}

No Flex Players

   
2
3
5
1
|

2+

{z
One Flex Guard

   
2
3
5
1
|

2+

{z
One Flex Forward

   
2
3
5
1
|

1
{z

2
}

One Flex Guard, One Flex Forward

Special Cases

  
2
3
1
|

2
{z

Two Flex Guards

  
2
5
1
|

Binomial
Coefficient

2
{z

Two Flex Forwards

30

2 3 10 + 2 3 5 2 + 2 3 10 2 + 2 3 5 2 + 2 3 + 2 10
= 326

31

Applications

Any Questions?
Review
Patrick
Analysis
Asymptotic
Non-Recursive
Recursive Algorithms
Recurrence Relations

Recursive
The Master Theorem

Counting
Special Cases

Binomial
Coefficient
31

31

Applications