Dr. Md.

Rafiqul Islam
INTRODUCTION TO ALGORITHMS
Slide # 1

CHAPTER 2

ASYMPTOTIC NOTATION
INTRODUCTION TO ALGORITHMS MD. RAFIQUL ISLAM
2
ASYMPTOTIC NOTATION
To estimate complexity (time and space
complexities) of algorithms several notations
are used in literatures.
They are O (Big oh), (theta), (Omega)
notations.
These notations are known as asymptotic
notations.

INTRODUCTION TO ALGORITHMS MD. RAFIQUL ISLAM
3
O-notation
The big-Oh O-Notation
asymptotic upper bound
f(n) = O(g(n)), if there exists constants c>0 and n
0
>0, s.t.
f(n) s c g(n) for n > n
0
f(n) and g(n) are functions over non-
Used for worst-case analysis
) (n f
( ) c g n
0
n
Input Size
R
u
n
n
i
n
g

T
i
m
e

INTRODUCTION TO ALGORITHMS MD. RAFIQUL ISLAM
4
O-notation
Example 2.1:

The function ) ( 2 3 n O n = + as n n 4 2 3 s + for all . 2 > n Here g(n) = n, c= 4.

) ( 2 4 10
2 2
n O n n = + + as
2 2
11 2 4 10 n n n s + + for all . 5 > n
Here g(n) = n
2
, c=11.

) 2 ( 2 * 6
2 n n
O n = + as
n n
n 2 * 7 2 * 6
2
s + for . 4 > n


) ( 3 3
2
n O n = + as
2
3 3 3 n n s + for all . 2 > n ) ( 2 4 10
4 2
n O n n = + + as
4 2
10 2 4 10 n n n s + + for . 2 > n You can go to up. Because, O is for upper bound.

) 1 ( 2 3 O n = + as 2 3 + n is not less than or equal to c for any constant c and all
.
0
n n > ). ( 2 4 10
2
n O n n = + +
You cannot go down or lower.

INTRODUCTION TO ALGORITHMS MD. RAFIQUL ISLAM
5
Mashiour Rahman AIUB::CSC2105::Algorithm Introduction5
O-notation
Simple Rule:
Drop lower order terms and constant factors.
50 n log n is O(n log n)
7n - 3 is O(n)
8n
2
log n + 5n
2
+ n is O(n
2
log n)
Note: Although for 50 n log n we can write
O(n
3
), it is expected that an approximation is of
the smallest possible order.
INTRODUCTION TO ALGORITHMS MD. RAFIQUL ISLAM
6
The big-Omega ONotation
asymptotic lower bound
f(n) = O(g(n)) if there exists constants c>0 and n
0
>0, such that
f(n) >= c g(n) for n > n
0
Used to describe best-case
running times or lower bounds
of algorithmic problems.
E.g., lower-bound of searching
in an unsorted array is O(n). Input Size
R
u
n
n
i
n
g

T
i
m
e

) (n f
( ) c g n
0
n
-notation
INTRODUCTION TO ALGORITHMS MD. RAFIQUL ISLAM
7
-notation
Example 2.2:

The function ) ( 2 3 n n O = + as n n 3 2 3 > + for all 1 > n (the inequality holds for
, 0 > n but the definition of requires an 0
0
> n ).
) ( 3 3 n n O = + as n n 3 3 3 > + for . 1 > n ) ( 2 4 10
2 2
n n n O = + + as
2 2
2 4 10 n n n > + + for . 1 > n ) 2 ( 2 * 6
2 n n
n O = + as
n n
n 2 2 * 6
2
> + for . 1 > n
Observe also that ), 1 ( 3 3 O = + n ), ( 2 4 10
2
n n n O = + +
), 1 ( 2 4 10
2
O = + + n n ), ( 2 * 6
2 2
n n
n
O = + ), ( 2 * 6
2
n n
n
O = + and
). 1 ( 2 * 6
2
O = + n
n
You can go down, but you cannot go up as you wish.
INTRODUCTION TO ALGORITHMS MD. RAFIQUL ISLAM
8
The big-Theta ONotation
asymptoticly tight bound
f(n) = O(g(n)) if there exists constants c
1
>0, c
2
>0, and n
0
>0, s.t. for
n > n
0
c
1
g(n) s f(n) s c
2
g(n)

f(n) = O(g(n)) if and only if
f(n) = O(g(n)), f(n) = O(g(n))
O(f(n)) is often abused
instead of O(f(n))
-notation


Input Size
R
u
n
n
i
n
g

T
i
m
e

) (n f
0
n
) (n g c
2
) (n g c
1
INTRODUCTION TO ALGORITHMS MD. RAFIQUL ISLAM
9
-notation


Example 2.3:

The function ) ( 2 3 n n O = + as n n 3 2 3 > + for all 2 > n and n n 4 2 3 s + for all . 2 > n So
, 3
1
= c , 4
2
= c and . 2
0
= n ), ( 3 3 n n O = + ). ( 2 4 10
2 2
n n n O = + + ), 2 ( 2 * 6
2 n n
n O = +
and ). (log 4 log * 10 n n O = +
Observe that ), 1 ( 2 3 O = + n ), ( 3 3
2
n n O = + ), ( 2 4 10
2
n n n O = + +
), 1 ( 2 4 10
2
O = + + n n ), ( 2 * 6
2 2
n n
n
O = + ), ( 2 * 6
100 2
n n
n
O = + and ). 1 ( 2 * 6
2
O = + n
n

You cannot go up or down as you wish. It is tight bound.
INTRODUCTION TO ALGORITHMS MD. RAFIQUL ISLAM
10
Theorem

Theorem 2.1
For any two function ) (n f and ), (n g )) ( ( ) ( n g n f O = if and only if )) ( ( ) ( n g O n f = and
(g(n)). ) ( O = n f
INTRODUCTION TO ALGORITHMS MD. RAFIQUL ISLAM
11
Typical growth rate


Table 2.1: Typical growth rate
Function Name
c
n log
n
n
2

n
3

2
n

Constant
logarithmic
linear
quadratic
cubic
exponential

n
n n n n n n 2 log log
3 2
< < < < < for all . 4 > n
INTRODUCTION TO ALGORITHMS MD. RAFIQUL ISLAM
12
Recurrences

A recurrence is an equation or inequality that describes a function in terms of its
value on smaller inputs.
The running time of merge sort algorithm could be describe by the recurrence

> +
=
=
. 1 if ) 2 / ( 2
1 if
) (
n n n T
n c
n T


The solution of the above recurrence claim to be
) log ( ) ( n n O n T =
or
) log ( ) ( n n n T u =
.
There are three methods for solving recurrences that is, for obtaining asymptotic or
O bounds on the solution.
INTRODUCTION TO ALGORITHMS MD. RAFIQUL ISLAM
13
Recursion Tree
A recursion tree is a convenient way to visualize
what happens when a recurrence is iterated.
INTRODUCTION TO ALGORITHMS MD. RAFIQUL ISLAM
14
14
Recursion tree of
T(n)=2T(n/2)+cn
INTRODUCTION TO ALGORITHMS MD. RAFIQUL ISLAM
15
Iteration method

The iteration method converts the recurrence into a summation and then
relives on techniques for bounding summations to solve the recurrence.

> +
=
=
1 2 2
constant a , 1
) (
n n ) / T(n
is a n a
n T




INTRODUCTION TO ALGORITHMS MD. RAFIQUL ISLAM
16
Analysis
( )
kn
k
n
T
k
. . . . . .
n
n
T
n
n
T
n
n
T
n
n n
T
n
n
T n T
+
|
|
.
|

\
|
=
+
|
|
.
|

\
|
=
+
|
.
|

\
|
=
+
|
.
|

\
|
=
+
)
`

|
.
|

\
|
=
+
|
.
|

\
|
=
2
2
3
2
3
2
3
3
8
8
2
4
4
2 4
2 2
2
2
INTRODUCTION TO ALGORITHMS MD. RAFIQUL ISLAM
17
Analysis


( )
( )
( ) n n O
n k n n an
a T kn an
n kn T
kn
n
T
k k
k
k
log
] log [ log
] 1 [
] [ 1
2 2
2
2
=
= + =
= + =
= + =
+
|
.
|

\
|
=
INTRODUCTION TO ALGORITHMS MD. RAFIQUL ISLAM
18
The master method

The master method provides a cookbook method for solving recurrences of the form
(2.3)
Where and are constant and is an asymptotically positive function. The
master method requires memorization of three cases, but then the solution of many recurrences
can be determined quite easily, often without pencil and paper.
The recurrence (2.3) describes the running time of an algorithm that divides a problem of size
n into a subproblems, each of size , where a and b are positive constants. The a
subproblems are solved recursively, each in time . The cost of dividing the problem and
combining the result of the subproblems is described by the function .

INTRODUCTION TO ALGORITHMS MD. RAFIQUL ISLAM
19
The master method

The master method depends on the following theorem.
Theorem 2.2 (master theorem)
Let and be constants, let be a function, and let be defined
on the nonnegative integers by the recurrence
,
Where we interpret to mean either or . The can be
bounded asymptotically as follows.
1. If for some constant , then
2. If then
3. If for some constant , and if for
some constant and all sufficiently large n, then .
INTRODUCTION TO ALGORITHMS MD. RAFIQUL ISLAM
20
Master method
T(n) = a T(n/b) + f(n)
At first we will find n
log
b
a
with respect to the above
recurrence and see the followings:
1) If n
log
b
a
>f(n) polynomially (in power of n)
T(n) = (n
log
b
a
)
2) If f(n) = n
log
b
a
,T(n) = (n
log
b
a
lgn)
3) If f(n) > n
log
b
a
, polynomially, T(n) = (f(n))
[lg means log base 2].

INTRODUCTION TO ALGORITHMS MD. RAFIQUL ISLAM
21
Use of Master method
T(n) = 9T(n/3) +n
Here a = 9, b = 3 and f(n) = n.
n
log
b
a
= n
log
3
9
= n
2
> n. We got case 1,
that is

n
log
b
a
> f(n), So we can write
T(n) = (n
log
3
9
) = (n
2
)
INTRODUCTION TO ALGORITHMS MD. RAFIQUL ISLAM
22
Use of Master method
T(n) = T (2n/3) + 1
T(n) = divide deno. and num. by 2
Here a =1, b = 3/2 and f(n) =1.
n
log
b
a
= n
log
3/2
1
= n
0
= 1. That is, n
logba
= f(n)
According to case 2 we can write
T(n) = (n
log
b
a
lgn) = (1.lgn) = (lgn)
1 )
2
3
( +
n
T
INTRODUCTION TO ALGORITHMS MD. RAFIQUL ISLAM
23
Use of Master method
T(n) = 3 T(n/4) + nlgn)
Here a =3, b = 4 and f(n) = nlgn.
n
log
b
a
= n
log
4
3
= n
0.793
. That is, nlgn > n
0.793

or
f(n) > n
log
b
a
. So, according to case 3 we
can write
T(n) =(f(n)= (nlgn).
INTRODUCTION TO ALGORITHMS MD. RAFIQUL ISLAM
24
Use of Master method
T(n) = 3 T(n/4) + nlgn)
Here a =3, b = 4 and f(n) = nlgn.
n
log
b
a
= n
log
4
3
= n
0.793
. That is, nlgn > n
0.793

or
f(n) > n
log
b
a
. So, according to case 3 we
can write
T(n) =(f(n)= (nlgn).
INTRODUCTION TO ALGORITHMS MD. RAFIQUL ISLAM
25
Failure of Master method
T(n) = 2 T(n/2) + nlgn
Here a =2, b = 2 and f(n) = nlgn.
n
log
b
a
= n
log
2
2
= n. That is, nlgn > n, but
not greater in polynomial (in power of n)
Consequebtly this recurrence can not be
solved using master theorem.
INTRODUCTION TO ALGORITHMS MD. RAFIQUL ISLAM
26
Use of Master method
T(n) = 7 T(n/2) + n
2
Here a =7, b = 2 and f(n) = n2.
n
log
b
a
= n
log
2
7
= n
2.8
. That is, n
2.8
> n
2
or
n
log
b
a
> f(n). So, according to case 1 we
can write
T(n) =(f(n))= (n
2.8
)= (n
lg7
)
INTRODUCTION TO ALGORITHMS MD. RAFIQUL ISLAM
27
References
1. Introduction to algorithms, Third Edition
Thomas H. Corman, Charles E. Leiserson, Ronald L.
Rivest.

2. Fundamentals of computer algorithm
Ellis Horowitz, Sartaj Sahni and Rajasekaran.


INTRODUCTION TO ALGORITHMS MD. RAFIQUL ISLAM
28
This is the end of chapter 2
THANK YOU