CSE 326

Asymptotic Analysis
David Kaplan
Dept of Computer Science & Engineering
Autumn 2001

Housekeeping
Homework 1 status
Join cse326@cs
Poll:
Slide format(s)?
Office hours today?

Asymptotic

CSE 326 Autumn 2001

Analyzing Algorithms
Analyze algorithms to gauge:
Time complexity (running time)
Space complexity (memory use)

Input size is indicated by a number n

sometimes have multiple inputs, e.g. m and
n

Running time is a function of n

n,

Asymptotic

n 2,

n log n,

18 + 3n(log n2) + 5n3

CSE 326 Autumn 2001

RAM Model
RAM (random access machine)
Ideal single-processor machine (serialized
operations)
Standard instruction set (load, add, store,
etc.)
All operations take 1 time unit (including, for
our purposes, each C++ statement)

Asymptotic

CSE 326 Autumn 2001

Order Notation (aka Big-O)

T(n) = O( f(n) )

Big-O

Exist positive constants c, n0 such

that

Upper
bound

T(n) cf(n) for all n n0

T(n) = ( f(n) )

Omega

Exist positive constants c, n0 such

that

Lower
bound

T(n) cf(n) for all n n0

Theta

T(n) = ( f(n) )
T(n) = O(f(n)) AND T(n) =
(f(n))

Tight bound

T(n) = o( f(n) )
Strict upper
Asymptotic
CSE
326
Autumn
2001
little-o T(n) = O(f(n)) AND T(n) !=

Simplifying with Big-O

By definition, Big-O allows us to:
Eliminate low order terms
4n + 5 4n
0.5 n log n - 2n + 7 0.5 n log n
Eliminate constant coefficients
4n n
0.5 n log n n log n
log n2 = 2 log n log n
log3 n = (log3 2) log n log n

But when might constants or low-order terms

matter?

Asymptotic

CSE 326 Autumn 2001

Big-O Examples
n2 + 100 n = O(n2)
follows from ( n2 + 100 n ) 2 n2

for n 10

n2 + 100 n = (n2)
follows from ( n2 + 100 n ) 1 n2

for n 0

n2 + 100 n = (n2)
by definition

n log n = O(n2)
n log n = (n log n)
n log n = (n)
Asymptotic

CSE 326 Autumn 2001

Big-O Usage
Order notation is not symmetric:
we can say 2n2 + 4n = O(n2)
but never O(n2) = 2n2 + 4n
Right-hand side is a crudification of the left

Order expressions on left can produce

unusual-looking, but true, statements:
O(n2)
(n3)
Asymptotic

= O(n3)
= (n2)
CSE 326 Autumn 2001

Big-O Comparisons
Function A

Function #2

n3 + 2n2
n0.1
n + 100n0.1
5n5

100n2 + 1000

vs.

log n
2n + 10 log n
n!

n-152n/100

1000n15

82log n

3n7 + 7n

Asymptotic

CSE 326 Autumn 2001

Race 1
n3 + 2n2

Asymptotic

vs. 100n2 + 1000

CSE 326 Autumn 2001

Race 2
n0.1

vs.

log n

In this one, crossover point is very late! So, which algorithm is really better???

Asymptotic

CSE 326 Autumn 2001

Race C
n + 100n0.1

vs. 2n + 10 log n

Is the better algorithm asymptotically better???

Asymptotic

CSE 326 Autumn 2001

Race 4
5n5

Asymptotic

vs.

n!

CSE 326 Autumn 2001

Race 5
n-152n/100

Asymptotic

vs.

1000n15

CSE 326 Autumn 2001

Race VI
82log(n)

Asymptotic

vs.

3n7 + 7n

CSE 326 Autumn 2001

Big-O Winners (i.e. losers)

Function #2

Winner

100n2 + 1000

O(n2)

log n

O(log n)

2n + 10 log n

O(n) TIE

n!

O(n5)

n-152n/100

1000n15

O(n15)

82log n

3n7 + 7n

O(n6) why???

Function A
n3 + 2n2
n0.1
n + 100n0.1
5n5

Asymptotic

vs.

CSE 326 Autumn 2001

Big-O Common Names

constant:
logarithmic:
linear:
log-linear:
superlinear:
quadratic:
polynomial:
exponential:

Asymptotic

O(1)
O(log n)
O(n)
O(n log n)
O(n1+c)
(c is a constant > 0)
O(n2)
O(nk) (k is a constant)
O(cn) (c is a constant > 1)

CSE 326 Autumn 2001

Kinds of Analysis
Running time may depend on actual input,
not just length of input
Distinguish
Worst case
Your worst enemy is choosing input

Average case
Assume probability distribution of inputs

Amortized
Average time over many runs

Best case (not too useful)

Asymptotic

CSE 326 Autumn 2001

Analyzing Code
C++ operations
Consecutive stmts
Conditionals
Loops
Function calls
Recursive
functions

Asymptotic

constant time
sum of times
larger branch plus
test
sum of iterations
cost of function body
solve recursive
equation
CSE 326 Autumn 2001

Nested Loops
for i = 1 to n do
for j = 1 to n do
sum = sum + 1

i 1

Asymptotic

n
j 1

1 n n

i 1

CSE 326 Autumn 2001

Dependent Nested Loops

for i = 1 to n do
for j = i to n do
sum = sum + 1
n

i 1

j i

i 1

i 1

i 1

1 (n i 1) (n 1) i

n(n 1) n(n 1)
2
n(n 1)

n
2
2
Asymptotic

CSE 326 Autumn 2001

Recursion
A recursive procedure can often be

analyzed by solving a recursive equation

Basic form:
T(n) =
base case: some constant
recursive case: T(subproblems) + T(combine)

Result depends upon

how many subproblems
how much smaller are subproblems
how costly to combine solutions
(coefficients)

Asymptotic

CSE 326 Autumn 2001

Sum of Queue
SumQueue(Q)
if (Q.length == 0 )
return 0
else
return Q.dequeue() + SumQueue(Q)

One subproblem
Linear reduction in size (decrease by 1)
Combining: constant (cost of 1 add)
T(0) b
T(n) c + T(n 1)
Asymptotic

for n>0
CSE 326 Autumn 2001

Sum of Queue Solution

Equation:
T(0) b
T(n) c + T(n 1)

for n>0

Solution:
T(n) c + c + T(n-2)
c + c + c + T(n-3)
kc + T(n-k) for all k
nc + T(0) for k=n
cn + b = O(n)

Asymptotic

CSE 326 Autumn 2001

Binary Search
BinarySearch(A, x)
Search A, a sorted array, for item x

12 30 35 75 83 87 90 97 99

One subproblem, half as large

Equation:
T(1) b
T(n) T(n/2) + c

Asymptotic

for n>1

CSE 326 Autumn 2001

Binary Search: Solution

Equation:
T(1) b
T(n) T(n/2) + c

for n>1

Solution:
T(n) T(n/2) + c
T(n/4) + c + c
T(n/8) + c + c + c
T(n/2k) + kc
T(1) + c log n where k = log n
b + c log n = O(log n)
Asymptotic

CSE 326 Autumn 2001