AlgorithmAnalysis-BookChapter_Draft Page 1 of 33 Pathak V.

COMPREHENSIVE Conventional Algorithms Analysis

1. Elements of Algorithm Design and Analysis

2. Analyzing Time complexity of Iterative Algorithms
General Rules
Arithmetic Series observation
Relevant exercises on solving arithmetic series

3. Analyzing Time Complexity of Recursive Algorithms
Recursive Algorithms and Recurrences
Solving Recurrences-I
3.2.1 Induction Method
3.2.2 Substitution Method
3.2.3 Change of Variable Method

4. Asymptotic Analysis
Asymptotic Notations: Basic definitions and relevance
Estimating order of growth of functions
4.2.1 Tabular Method
4.2.2 Ratio Theorem
Solving Recurrences-II
4.3.1 Recursion Tree Method
4.3.2 The Master Method
4.3.3 Justification of the Rules of Master Method
Some Properties of Asymptotic Notations

5. Best-case, Worst-case, Average Case Analysis
6. Improving Time complexity: Some Examples
7. Probabilistic Analysis and Randomized Algorithms


[ Chapter 7 intends to provide a link between the conventional analysis and the
probabilistic analysis of algorithms besides providing basic exposure to use of
randomization as a tool to design efficient algorithms, and use of probabilistic analysis in
analysis and design of algorithms ]




AlgorithmAnalysis-BookChapter_Draft Page 2 of 33 Pathak V.



1. Elements of Algorithm Design and Analysis

Understanding of algorithms essentially involves focusing on two concerns:
1. Design of algorithms
2. Analysis of Algorithms
1.1. Design of Algorithms
Design of an algorithm to solve a particular problem is guided by understanding of basic
algorithm design techniques (approaches) with regard to
I) Characteristics of the problem to which the technique is applicable
II) Formulation of the problem in terms of the technique
III) Basic steps of solving the problem according to the technique
IV) Typical (example) problems to which that technique is applicable
V) Limitations of the technique
Some Algorithm design techniques are
I) Divide and Conquer: e.g. Binary Search, QuickSort, MergeSort etc.
II) Greedy Technique: e.g. Knapsack problem, Min. Spanning Tree etc.
III) Back Tracking: e.g. N-Queen Problem etc.
IV) Branch & Bound: e.g. 15-puzzle problem etc.
V) Dynamic Programming: e.g. 0/1 knapsack (technique: generating profit values for
all (element, remaining capacity) pair), TSP problem etc.
VI) Evolutionary Algorithms
1.2. Analysis of Algorithms
We need algebraic and logical ability to estimate the run-time performance of algorithms, so
that we can compare two or more algorithms available for performing the same task. We
need to analyze an algorithm with regard to:
I) Time Complexity
II) Space Complexity
III) Correctness
IV) Effect of data characteristics on the performance of algorithm: Best-case, Worst-
case, Average-case performance.
AlgorithmAnalysis-BookChapter_Draft Page 3 of 33 Pathak V.


2. Analyzing Time complexity of Iterative Algorithms
2.1 General Rules
In analyzing the time complexity of an algorithm, we normally ignore actual CPU-cycles of
each instruction execution. Rather we aim to approximately estimate the growth of the run-
time of the algorithm with respect to input data size, n. For this, we just calculate the step
count.
Some thumb rules we may use for estimating time complexity of an iterative algorithm
involving the major language constructs can be observed as suggested
I) for a statement block t (n) = count of the number of statements.
II) for each single level loop involved: t (n) = no. of iterations of the loop.
III) for each nested loop: t (n) = (no. of iterations of the outer loop)
* (no. of iterations of the inner loop).
IV) for control structure: t (n) = larger of the counts in the if and the
else part statement blocks.
We can refer to the following examples to make the idea clear.
i) Algo1 (n) Step Count
1. P:= 1 1
2. for I = 1 to 2n do
3. P:= P * I 2n
=========
Estimating time complexity: f(n) = 2n + 1.

ii) Algo2 (m,n) Step Count
1. P:= 1 1
2. for I = 1 to m do
3. for J = 1 to n do
4. P:= P * I m*n
=========
Estimating time complexity: f(n) = m*n + 1.

iii) Algo3 (n) Step Count
1. P:= 1 1
2. if (<cond1>) then
3. for J = 1 to n do
4. P:= P * I [ n ]
5. else
6. for J = 1 to 2n do
7. P:= P * I [ 2n ]
8. end if 2n [since 2n > n]
=========
Estimating time complexity: f(n) = 2n + 1.
AlgorithmAnalysis-BookChapter_Draft Page 4 of 33 Pathak V.


But the basic rules are not always helpful. Sometimes we need apply reasoning to estimate
the step counts.
For example, we consider the following algorithm.
iv) Algo4 (n) Step Count
1. P:= 1 1
2. for I = 1 to n do
3. for J = 1 to I do
4. P:= P * I ???
========
Here, we need to apply reasoning. We observe that step counts of (P:=P*I): it executes 1 time
for I = 1; 2 times for I = 2; and so on
So, estimating time complexity, we get
f(n) = 1 + 2 + 3+ . + n +1=

n
i
1
+ 1=
2
) 1 ( + n n
= n
2
+ n + 1.
Further, we observe that such algorithms may take other forms also.
v) Algo5 (n) Step Count
1. P:= 1 1
2. for I = 1 to n do
3. for J = 1 to I
2
do
4. P:= P * I 1
2
+ 2
2
+ 3
2
+ ... + n
2
.
Estimating time complexity: f(n) =

n
i
1
2
+ 1

vi) Algo6 (n) Step Count
1. P:= 1 1
2. for I = 1 to n do
3. for J = 1 to log(I) do
4. P:= P * I log(1)+log(2)+...+log(n).
Estimating time complexity: f(n) =

n
i
1
) log( + 1
In the due course, we see that we need to know the algebraic series and summations. A quick
go through section 2.2 and solving the exercises in section 2.3 may refresh the related
concepts.




AlgorithmAnalysis-BookChapter_Draft Page 5 of 33 Pathak V.


2.2 Algebraic Series and Other Relevant Mathematical Results Review

Arithmetic Series
) 1 (
2
1
... 2 1
1
+ = + + + =

=
n n n k
n
k
--- [I]
) 1 2 )( 1 (
6
1
... 2 1
2 2 2
1
2
+ + = + + + =

=
n n n n k
n
k
--- [II]

2 2 3 3 3
1
3
) 1 (
4
1
... 2 1 + = + + + =

=
n n n k
n
k
--- [III]

Geometric Series
For real x 1, x < 1,

1
1
... 1
1
2
0

= + + + + =
+
=

x
x
x x x x
n
n
n
k
k
--- [IV]

=

=
0
1
1
k
k
x
x --- [V]

Harmonic Series
For positive integers n,
c n
k n
H
n
k
n
+ = = + + + + =

=
) ln(
1 1
...
3
1
2
1
1
1
, --- [VI]
where c is positive integer constant.

Additional formulae can be gained by integrating or differentiating the formulae
above.
e.g. by differentiating both sides of infinite geometric series [v], we get:

=

=
0
2
) 1 (
k
k
x
x
kx , for x < 1 --- [VII]

Telescopic Series
For any sequence
n
a a a ,..., ,
1 0
,

=
n
k
n k k
a a a a
0
0 1
) ( --- [VIII]
AlgorithmAnalysis-BookChapter_Draft Page 6 of 33 Pathak V.


since, each of the terms
1 1 0
,..., ,
n
a a a is added in exactly once and
subtracted once. We say that the sum telescopes.
Similarly,

=
+
=
n
k
n k k
a a a a
0
0 1
) ( --- [IX]

n k k k k
n
k
n
k
1
1 )
1
1 1
(
) 1 (
1
1
1
1
1
=
+
=
+


=

=
--- [X]

Some More Results
Exponentials

n m n m m n n m mn n m
a a a a a a a a a a a a
+
= = = = = = , ) ( ) ( , ) ( , / 1 , , 1
1 1 0

For all real a,b, s.t. a>1, 0 lim =

n
b
n
a
n

--- [XI]
Logarithms
lg n = log
2
n, ln n = log
e
n, lg
k
n = (lg n)
k
, lg lg n = lg (lg n),
For all real a, b, c >0

a c
a
b b b
c
c
b
b
n
b c c c
a
b b
b
c a
b
a a a
b
a
a
a n a b a ab b a
log log
log
log
1
log , log ) / 1 ( log ,
log
log
log
, log log , log log ) ( log ,
=
= = =
= + = =

--- [XII]
Factorials

>
=
=
0 ), 1 .(
0 , 0
!
n n n
n
n
Stirlings Approximation
|

\
|
|

\
|
+
|

\
|
=
n e
n
n n
n
1
1 2 ! --- [XIII]

n n
e
e
n
n n
n for
n
n
n
12
1
1 12
1
, 2 !
, 1
< <
+
|

\
|
=

--- [IX]
AlgorithmAnalysis-BookChapter_Draft Page 7 of 33 Pathak V.


Use: Applying Stirlings approximation, we get lg(n!) = (n lg n) [ the -notation to
be introduced later ]
L-Hospital Rule for Solving Limits
When,
0 ) (
) (
lim

=

or
n g
n f
n
form,
Then,
dn n g d
dn n f d
n g
n f
n n
) ( (
) ( (
lim
) (
) (
lim

= --- [X]

2.3 Relevant exercises on solving arithmetic series
1. Evaluate the following
a)

n
k
k
1
) 1 2 ( b)

=
+
n
k
k
x k
1
2
) 1 2 ( c)

=
n
k
k
1
4 . 2 d)

n
k
k
2
2
)
1
1 (
e)

n
i
1
) log(
2. Verify the following:
a) c n
k
n
k
+ =

=
) ln(
1 2
1
1
b) 1 0 ,
) 1 (
) 1 (
0
3
2
< <

+
=

=
x
x
x x
x k
k
k

c) 0
2
) 1 (
0
=

= k
k
k

















AlgorithmAnalysis-BookChapter_Draft Page 8 of 33 Pathak V.





3. Analyzing Time complexity of Recursive Algorithms
3.1. Recursive Algorithms and Recurrences
A recursive algorithm and its corresponding step count can be considered to have typically
the following scheme.
Single Recursive Call
RecAlgo1 (n) Step Count
1. program termination condition {program terminates for n=n
0
}
2. corresponding statements f1(n) {gives basic case count}
3. [ statements ] f2(n)
4. RecAlgo1(n) t(n){general recurrence for
recursive step count}
5. [ statements ] c3

i.e. an algorithm having a mix of statement blocks having constant counts and those
having count as linear function of the input size n.
Representing time complexity: T(n) = f1(n) , for n= n
0
;
= T(n) + f(n), for n> n0;
Here, f(n) = f2(n) +c3, f1(n) may give a constant
n = f(n) = modified parameter as function of n.
Alternative representation as follows.
T(n0) = f1(n), T(n) = T(n) + f(n).

Example 3.1. Binary Search
Bin_Srch( Ar [], l, u, v )
1. if ( l == u )
2. if ( Ar[l] == v ) return l;
4. else return false;
5. m = (l+u) /2;
6. if (v == Ar[m]) return m;
7. if (v < Ar[m]) Bin_Srch(Ar, l, m-1, v);
8. if (v > Ar[m]) Bin_Srch(Ar, m+1, u, v);

Estimating time complexity for Bin_Srch(n):
AlgorithmAnalysis-BookChapter_Draft Page 9 of 33 Pathak V.


Here we see that Bin_Srch() recursive calls are made over half (n/2) of the portion
(n) of input list. Moreover, only one of the recursive calls is executed at any call.
Therefore, the recurrence for the time complexity representation becomes,
T(n) = T(n/2) + C; where C = no. of supplementary steps other than
the recursive calls = max( 4 (step 7) or 5(step8) ) = 5.
So, the complete recurrence representing time complexity of Bin_Srch algorithm is:
T(n) = 2, for n = 1
= T(n/2) + 5 for n > 1
Alternative, T(n) = T(n/2) + 5, with T(1) = 2.

Multiple Recursive Calls
RecAlgo1 (n) Step Count
1. program termination condition {program terminates for n=n
0
}
2. corresponding statements f1(n) {gives base case count}
3. [ statements ] c3
4. RecAlgo1(n
1
) t(n
1
){general recurrence for
recursive step count}
5. [ statements ] c4
6. RecAlgo1(n
2
) t(n
2
)
.
.
. RecAlgo1(n
n
) t(n
n
)
k. [ statements ] cn

Representing time complexity:
T(n) = f1(n) , for n= n
0
;
=

=
n
i 1
i
) T(n + f(n), for n> n
0
;
Here, f1(n) may be constant;
f(n) = function representing contribution of non-
recursive calls
ni = fi(n), 1<=i<=n. i.e. all recurrences with
different modified parameters are summed up.

Multiple Recursive Calls with Same Parameter Modification Function
AlgorithmAnalysis-BookChapter_Draft Page 10 of 33 Pathak V.


If all the recursive calls use the same function, say n =f(n), for modifying the parameter for
the recursive call, and let m such calls are there, then the recurrence derived above can be
written as:
T(n) = f1(n) , for n= n
0
;
= m T(n) + f(n), for n> n0;
Moreover, if each n = n/b, for some positive constant b, then
T(n) = m T(n/b) + f(n)

Example 3.2. Merge Sort
M_Sort( Ar[], l, u ) Step Count
1. if ( l == u ) 1 {base case: n
0
= 1; T(1) = 1}
2. return;
3. m = (l+u)/2; 1
4. M_Sort (Ar, l, m-1); T(n/2) { size(l to m-1) = n/2 }
5. M_Sort (Ar, m+1, u); T(n/2) { similarly }
6. Merge ( Ar, l, m, m+1, u); f(n) { Merge is an iterative
algorithm and its step count is
given by a linear function of n
(... ... verify!!)}

Estimating time complexity for M_Sort(n):
Here we see that both the steps 4 and 5 are executed on each recursive call of the function.
So, we observe the recurrence to be
T(n) = 1, for n = 1
= 2 T(n/2) + f(n) + 2, for n > 1, where f(n) is a linear function of n.

Review Exercise:
1. Write the iterative algorithm for Merge() function in the M_Sort algorithm. Verify
its time complexity.
2. Write the recursive Fibonacci () function. Determine the recurrence representing its
time complexity.
3. Write the Recursive Inorder() function for traversing a binary tree. Determine its
recurrence for a complete binary tree.
4. Write the Quicksort() algorithm. Determine its recurrence assuming that in each
recursive call, the list is divided into two equal parts.
AlgorithmAnalysis-BookChapter_Draft Page 11 of 33 Pathak V.


5. Devise a binary search algorithm that splits the set not into two sets of (almost) equal
sizes but into two sets, one of which is twice the size of the other. Determine its
recurrence.

3.2. Solving Recurrences-I
We need to solve the recurrence obtained for a recursive algorithm. In the following, we see
solutions to some typical recurrences.
3.2.1 Induction Method
I. The following recurrence arises for a recursive program that loops through the input to
estimate one item:
T(N) = T(N-1) + N for N > 1 with T(1) = 1.
Solution:
T(N) = T(N-1) + N
= T(N-2) + (N-1) + N
= T(N-3) + (N-2) + (N-1) + N

= T(1) + 2 + + (N-2) + (N-1) + N
= 1 + 2 + + (N-2) + (N-1) + N
=
2
) 1 ( + N N

=
2 2
2
N N
+

II. The following recurrence arises for a recursive program that halves the input in one step.
T(N) = T(N/2) + C for N > 1 with T(1) = C1 ;
C,C1 being positive constants.
Solution:
Let N = 2
k
i.e. k = log
2
N.
T(N) = T(N/2) + C
= T( N/2
2
) + C+ C [ (N / 2) /2 = N/(2*2) = N/2
2
]
= T( N/2
3
) + C + 2C

= T( N/2
k
) + C + (k-1)C
= T(1) + k C [ since N = 2
k
, hence, N/2
k
= 1]
= k C+ C1 [ since T(1) = 1]
= C.log
2
N + C1

AlgorithmAnalysis-BookChapter_Draft Page 12 of 33 Pathak V.


[Note: The recurrence of Bin_Srch() conforms to this recurrence model, with C = 5, C1 = 2 ]

III. The following recurrence arises for a recursive program that halves the input for the next
call, but alongside must examine every item in the input.
T(N) = T(N/2) + N for N > 1 with T(1) = 0.
Solution:
Let N = 2
k
i.e. k = log
2
N.
T(N) = T(N/2) + N
= T( N/2
2
) + N/2 + N [ (N / 2) /2 = N/(2*2) = N/2
2
]
= T( N/2
3
) + N/4 + N/2 + N

= T( N/2
k
) + N/2
k-1
+ + N/4 + N/2 + N
= T(1) + 2 + + N/4 + N/2 + N [since N = 2
k
, hence, N/2
k
=1
and N/2
k-1
= 2 * N/2
k
= 2 ]
= 0 + 2 + + N/4 + N/2 + N
= N + N/2 + N/4 + + N/2
k

= N ( 1 + + ()
2
+ +()
k
)
=
( )
( ) 1
2
1
1
2
1
1

+ k
N
The result above is a bit clumsy to interpret. So, it can be approximated to
estimate the order of growth of the resultant function. For this, we look for the result of
summation for the case, k , i.e. above series becomes a infinite geometric series.
Hence, T(N) N ( 1 + + ()
2
+ terms)
= N.
2
1
1
1

= 2N

IV. The following recurrence arises for a recursive program that has to make a linear pass
through the input, before, during, or after splitting that input into two equal halves.
T(N) = T(N/2) + N for N > 1 with T(1) = C ;
C being positive constant.
Solution:
Let N = 2
k
i.e. k = log
2
N.
T(N) = 2 T(N/2) + N
= 2 [2 T( N/2
2
) + N/2 ]+ N = 2
2
T(N/2
2
) + 2 * (N/2) + N
= 2
2
[2 T(N/2
3
) + ( N/2
2
)] + 2N = 2
3
T(N/2
3
) + 2
2
*(N/2
2
)+2N
= 2
3
T(N/2
3
) + 3N

= 2
k
T(N/2
k
) + kN
AlgorithmAnalysis-BookChapter_Draft Page 13 of 33 Pathak V.


=
N
2
log
2 T(1) + N. log
2
N
= N T(1) + N. log
2
N [ since
N
2
log
2 =
2 log
2
N = N , Ref. XII ]
= N * C + N. log
2
N
= N. log
2
N + C.N

[Note: The recurrence of M_Sort() conforms to this recurrence model, as f(n)+2 being a
linear function of N can be approximated as N; C = 1. ]
Review Exercise:
1. Solve following recurrences using the induction method:
i) T(N) = T(N/2) + 1 for N > 1 with T(1) = C ; C being positive constant.
ii) T(N) = T(N-1) + N, T(1) = 1
iii) T(N) = T(N/2) + N, T(1) = 0
iv) T(N) = 2 T(N/2) + C, T(1) = 0, C being positive constant.
v) T(N) = 2 T(N/2) + N, T(1) = 1
2.


3.2.2 Substitution Method

3.2.3 Change of Variable Method




















AlgorithmAnalysis-BookChapter_Draft Page 14 of 33 Pathak V.









4. Asymptotic Analysis
4.1 Asymptotic Notations: Basic definitions and Relevance
Our purpose is to estimate the order of growth of run time of an algorithm. The concept of
asymptotic notations helps in estimating and representing the order of growth of functions.
We usually try to compare the representative function - of the algorithms input size N - with
respect to some common asymptotic functions. For example,
1 (or any integer) Constant; log (N) Logarithmic; C
1
.N + C
2
Linear; N log (N) - N
log(N); C
1
.N
2
+ C
2
Quadratic; C
1
.N
3
+ C
2
Cubic; in general, C
1
.N
p
+ C
2

Polynomial; C
1
. C
2
N
Exponential; N! Factorial.
Definitions
Big- Oh Notation
The Big-Oh notation expresses an upper bound for the function f(N).
f(N) = O(g(N)) iff f(N) <= c. g(N) ;
0 0
, 0 ; n c n N
Omega Notation
The Big-Omega-notation expresses an upper bound for the function f(N).
f(N) = (g(N)) iff f(N) >= c. g(N);
0 0
, 0 ; n c n N
Theta Notation
The Theta-notation expresses an exact bound (or equivalence) for the function f(N).
f(N) = (g(N)) iff c
1
.g(N)<=f(N) <= c
2
. g(N);

0 2 1 0
, , 0 ; n c c n N
We may also say that f(N) = (g(N)) iff f(N) = O(g(N)) and f(N) = (g(N)).
AlgorithmAnalysis-BookChapter_Draft Page 15 of 33 Pathak V.


Interpreting meanings of upper lower bounds with graphical representation

0
10
20
30
40
0 2 4 6
f(n)
g(n)

The relevance of the value n
0
can be understood by the above plot (rightmost) for f(n) =
n+6, g(n) = n
2
. We observe, n
0
= 3, since f(n)<g(n) before N=3; f(n)=g(n), at N=3, but
f(n)>g(n) after N=3.
4.2 Estimating order of growth of functions
Comparison of Order of Growth of Some Standard Reference Functions
Constant <. log (N) <. N <. N log (N) <. N
2
<. N
3
<. N
p
<. C
2
N

Here, the relation <. shows ascending order of growth of the corresponding functions.
To estimate the order of growth of a given function f(n), we first need to guess a function
g(n) to compare with. As our choice, we guess a standard function nearest in comparison
(upper or lower bound) to a major element in f(n).
e.g. Consider the function f(n) = 2n
3
+5n+4, we may choose to compare it with g(n) = n
3
.
In another function f(n) = 2n
2
+5nlogn+4n, we may choose to compare it with g(n) = n
2
, or
with g(n) = nlogn.
Sometimes, we are provided the functions f(n) and g(n), are required to compare the two.
e.g. We may be categorically asked to compare the function f(n) = 2n
2
+5n+4 with g(n) =
nlog
2
n, or with g(n) = n.
Particularly, when it concerns to the main purpose of all these endeavors, that is comparison
of performance of two given algorithms (for the same task), the functions representing time
complexities of the two algorithms simply work as f(n) and g(n).
e.g. Suppose two algorithms A and B have time-complexity functions as I) 5nlogn+2n, and
II) n
2
+10n; then we can have either f(n) =5nlogn+2n, g(n) = n
2
+10n, or f(n) = n
2
+10n, g(n)
= 5nlogn+2n.

g(n)
f(n)
c.g(n)
n
0
g(n)
f(n)
c.g(n)
n
0
c.g(n)
c.g(n)
g(n)
f(n)
n
0
f(n) = O(g(n)) f(n) = (g(n))
f(n) = (g(n))
AlgorithmAnalysis-BookChapter_Draft Page 16 of 33 Pathak V.


4.2.1 Tabular Method
As a crude method to determine (if exist) the values for c, and n
0
so that the O, or
relations between two given functions f(n) and g(n) could be verified, we can use a trial and
error approach. The trial values can be tabulated; exemplified as below.

Example 4.1.
Let f(n) = 5n
2
+2, g(n) = n
2
.
The table given below suggests that
f(n) <= c. g(n), for n
0
= 1, and c = 7
So, f(n) = 5n
2
+2 = O(n
2
) for n
0
= 1, and c = 7. [4.1.a]

n
f(n) g(n) c c.g(n) f(n)<= c.g(n)?
0 2 0 1 0 no
1 7 1 1 1 no ( set c =7 to get f(n)= c.g(n) )
1 7 1 7 7 yes (check further for same c
value)
2 22 4 7 28 yes (verify more)
3 45 9 7 63 yes (seems verified,.Stop)

Similarly, the table given below suggests that
f(n) >= c. g(n), for n
0
= 0, and c = 1;
So, f(n) = 5n
2
+2 = (n
2
) for n
0
= 1, and c = 7. [4.1.b]


n
f(n) g(n) c c.g(n) f(n)>= c.g(n)?
0 2 0 1 0 yes
1 7 1 1 1 yes (verify more)
2 22 4 1 4 yes (verify more)
3 45 9 1 9 yes (seems verified,.Stop)

From observations 4.1.a, and 4.1.b above, we may conclude that
c
1
. g(n) <= f(n) <= c
2
. g(n), for n
0
= 1, c
1
=1, and c
2
= 7.
AlgorithmAnalysis-BookChapter_Draft Page 17 of 33 Pathak V.


Hence, by definition of the Theta-notation
5n
2
+2 = (n
2
) for n
0
= 1, c
1
=1, and c
2
= 7. [4.1.c]
In short, we may also say that since 5n
2
+2 = O(n
2
), and 5n
2
+2 = (n
2
) hence ,
5n
2
+2 = (n
2
).


Review Exercise:
1. Find some f(n) and g(n), such that f(n) (g(n)). Use the tabular method to verify.
2. Find some f(n) and g(n), such that f(n)=O(g(n), but f(n) (g(n)). Use the tabular
method to verify.
3. Find some f(n) and g(n), such that f(n)= (g(n)), but f(n) O(g(n). Use the tabular
method to verify.
4. Find some f(n) and g(n), such that f(n)=O(g(n), but g(n) O(f(n). Use the tabular
method to verify.
4.2.2 Ratio Theorem
Let f(n) and g(n) be some asymptotically non negative functions of n. Then the following
properties are defined.
Big- Oh Notation
f(N) = O(g(N)) iff c
n g
n f
n

) (
) (
lim , for some finite constant c.
Big- Omega Notation
f(N) = (g(N)) iff c
n f
n g
n

) (
) (
lim , for some finite constant c.
Theta Notation
f(N) = (g(N)) iff c
n g
n f
n

) (
) (
lim and c
n f
n g
n

) (
) (
lim ,
for some finite constant c.
Example 4.2.
Let f(n) = 3n + 2, g(n) = n.
Then, 3
) 2 3 (
lim =
+

n
n
n
, 3n + 2 = O (n)
and 3
3
1
) 2 3 (
lim < =
+

n
n
n
3n + 2 = (n)
Thus, with c = 3, we determine that 3n + 2 = (n)
Example 4.3. [ use of L-Hospital Rule Ref. X ]
Consider f(n) = 5n
2
+3n+2, g(n) = n
2
.
AlgorithmAnalysis-BookChapter_Draft Page 18 of 33 Pathak V.


Here, form
n
n n
n

+ +

2
2
) 2 3 5 (
lim , so we apply L-Hospitals rule.
Then, form
n
n
dn n d
dn n n d
n g
n f
n n n

+ +
=
+ +
=

2
0 3 10
lim
/ ) (
/ ) 2 3 5 (
lim
) (
) (
lim
2
2

5
2
0 10
/ ) 2 (
/ ) 3 10 (
lim =
+
=
+
=

dn n d
dn n d
n

Similarly, form
n n
n
n

+ +

) 2 3 5 (
lim
2
2
, We get, 5
5
1
) (
) (
lim < =

n f
n g
n

Thus for c =5, we see, f(n) = O (n
2
), f(n) = (n
2
), and f(n) = (n
2
).

Review Exercise:
1. Determine the asymptotic relations between following pairs of functions:
i) (3n
2
+ 100 n + 6; n
3
) ii) (2n
3
+ 4n
2
log(n); n
3
) iii) (2n
3
+ 4n
2
log(n); n
3
log(n))
2. Determine the order of growth the function f(n) = 3 log n + log log n.
3. Estimate the lower bound on growth of the function f(n) = n3 log n. Verify.
4.3 Solving Recurrences-II
4.3.1 Recursion Tree Method
The recursion tree method is best to generate a good guess. It is useful when the recurrence
describes the running time of a divide-and-conquer algorithm.
Example 4.4.
Let us consider the recurrence
T(N) = 3T(
4
N
)+ ) (
2
n , T(1)=1.
T(N)


[ since, T(N) expands as T(N) T(N/4)+T(N/4)+T(N/4), as well as, contributes step count
cN
2
].






cN
2

T(N/4) T(N/4) T(N/4)
C(N/4)
2
T(N/16) T(N/16) T(N/16)
C(N/4)
2

T(N/16) T(N/16) T(N/16)
C(N/4)
2

T(N/16) T(N/16) T(N/16)

cN
2

AlgorithmAnalysis-BookChapter_Draft Page 19 of 33 Pathak V.


[ since, each T(N/4) expands as T(N/4) 3T(N/(4*4)) 3T(N/16)
T(N/16)+T(N/16)+T(N/16) as well as, contributes step count c(N/4)
2
].








We observe,
Depth of Tree:-
The subproblem size for a node at depth i is = (N/4)
i
.
Thus the subproblem size hits N=1 when (N/4
i
) = 1 i = log
4
N
Therefore, No. of levels (Depth) of Tree = log
4
N + 1 [i.e. o, 1, 2, , log
4
N]
Step Count at last level:-
At the last level, each node contributes T(1) i.e. 1 step,
Since No. of nodes at level i is 3
i
.
Hence, the last level at depth log
4
N has a cost of 3
log4N
= N
log
4
3
[ref. XII ]
Thence, summing up the cost of all the levels:
T(N) = cN
2
+(3/16) cN
2
+(3/16)
2
cN
2
+ + (3/16)
log4N-1
cN
2
+ ) (
3 log
4
N .
= ) (
16
3
3 log 2
1 log
0
4
4
N cN
i
N
i
+
|

\
|

=

= ) ( .
1
16
3
1
16
3
3 log 2
log
4
4
N cN
N
+

\
|

\
|
.
This result is a bit messy. So, we can make an approximate about the order of
growth of the function using an infinite decreasing geometric series as an upper bound.
C(N/4)
2

C (N/16)
2
C (N/16)
2
C (N/16)
2

C(N/4)
2


C (N/16)
2
C (N/16)
2
C (N/16)
2


cN
2
C(N/4)
2

C (N/16)
2
C (N/16)
2
C (N/16)
2

CN
2
(3/16)CN
2
(3/16)
2
CN
2


(3/16)
2
CN
2
(3/16)
i
CN
2


T(1) T(1) T(1) T(1) T(1) T(1) T(1)

) (
3 log
4
N
AlgorithmAnalysis-BookChapter_Draft Page 20 of 33 Pathak V.


i.e. T(N) = ) (
16
3
3 log 2
1 log
0
4
4
N cN
i
N
i
+
|

\
|

=

< ) (
16
3
3 log 2
0
4
N cN
i
i
+
|

\
|

=

= ) ( .
16
3
1
1
3 log 2
4
N cN +
|

\
|

[ ref. V ]
= ) ( .
13
16
3 log 2
4
N cN + = O(N
2
) [ Justify!! ]
Thus the solution for the given recurrence using the recursion tree method is obtained to
be
T(N) = O(N
2
)
Solving Recurrences when the recursion tree is unbalanced
Such derivation is helpful when more than one recursive calls are involved, each of them
working on unequal portions of input data (list) set.
e.g. Consider the recurrence
T(N) = T(N/3) + T(2N/3) + O(N)
The Recursion Tree








Let the constant factor in O(N) be c.
When we add the values across the levels of the recursion tree, we get the value of cost for
each level = cN.
Depth of Tree:- i.e. the number of levels in the tree.
The longest path from the root to any leaf, is across N -> (2/3)N -> (2/3)
2
N -> ->1.
cN
c(N/3) c(2N/3)
c(N/9) c(2N/9) c(2N/9) c(4N/9)

T(1)
T(1)
T(1)
T(1)
cN

cN
N N
c =
|

\
|
+
3
2
3
cN
AlgorithmAnalysis-BookChapter_Draft Page 21 of 33 Pathak V.


Let the no. of steps in path ( i.e. no. of levels in the tree ) = k.
Then 1
3
2
=
|

\
|
N
k
k = log
3/2
N.
The height of the tree = log
3/2
N.
Thus for the recurrence yielding recursion tree of max level log
3/2
N, we have to deal by
doing an approximate estimation.
The Guess
We expect the substitution to the recurrence to be at most the number of levels times the
cost of each level.
Thus the solution for the given recurrence using the recursion tree method is obtained to
be
T(N) = O(c N log
3/2
N) = O(N logN)
4.3.2 The Master Method
The Master method, in fact derived from the results of recursion tree approach to guess
approximate solution for recurrences of a particular form, can be stated as below
When egers positive being b a n f
b
n
aT n T int , ), ( ) ( +
|

\
|
= ,
Then,
I) ) ( ) ( ) ( ) (
log log a a
b b
n n T then n O n f If = =


II) ) log . ( ) ( ) ( ) (
2
log log
n n n T then n n f If
a a
b b
= =
III ( ) 0 ), ( ) ( ) (
log
> =
+
n n cf
b
n
af and n n f If
a
b

, c being a
positive integer, ) ( ( ) ( n f n T then = .

Example 4.4.
Consider the recurrence
2
3
9 ) ( n
n
T n T +
|

\
|
=
Here, we see that
a
b
n n n and n n f
log 9 log 2 2
3
, ) ( = = = ) ( ) (
log a
b
n n f =
Hence rule II is applicable, So, ) log . ( ) (
2
log
n n n T
a
b
= ) log . ( ) (
2
2
n n n T =
Example 4.5.
Consider the recurrence n
n
T n T +
|

\
|
=
3
9 ) (
Here, we see that
1 9 log
3
, ) (

= = n n and n n f 1 ), ( ) (
log
=

a
b
n n f
Hence rule I is applicable, So, ) ( ) (
log a
b
n n T = ) ( ) (
2
n n T =
Example 4.6.
AlgorithmAnalysis-BookChapter_Draft Page 22 of 33 Pathak V.


Consider the recurrence
3
3
9 ) ( n
n
T n T +
|

\
|
=
Here, we see that
1 9 log 3 3
3
, ) (
+
= = n n and n n f 1 ), ( ) (
log
=
+

a
b
n n f
Moreover, ( ) , * . . ) ( *
3
* 9 . .
3
3
n c e i n f c
n
e i
b
n
af <=
|

\
|
for c = 1, for all n >0.
Hence rule III is applicable, So, )) ( ( ) ( n f n T = ) ( ) (
3
n n T =

Review Exercise:
1.Devise a binary search algorithm that splits the set not into two sets of (almost) equal
sizes but into two sets, one of which is twice the size of the other. How does this
algorithm compare with binary search?
2. Solve following recurrences using appropriate method:
i) T(n) = 2T(n/2) + n
3
, T(1) =1 ii) T(n) = 8T(n/3)+n, T(1) = 0
iii) T(n) = 2T(n/4)+n, T(1) = 1 iv) T(n) = 4T(n/2) + nlogn
3. Solve following recurrences using any method, verify using recursion tree method.
i) T(n) = 2T(n/2)+n, T(1) =1 ii) T(n) = 2T(n/2)+n
2
, T(1) =1

4.3.3 Justification of the Rules of Master Method
Reconsider the theorem. The three rules can be justified by intuition as below
RuleI.

RuleII.

RuleIII.


f(n) = 2f(n/2) + 1 = 4f(n/4) + 2 + 1 = . . .
= n f(n/n) + n/2 + . . . + 4 + 2 + 1
The first term
dominates
f(n) = 2f(n/2) + n = 4f(n/4) + n + n = . . .
= n f(n/n) + n + n + n + . . . + n
All terms are
comparable
The last term
dominates
f(n) = f(n/2) + n = f(n/4) + n/2 + n = . . .
= f(n/n) + 2 + 4 + . . . + n/4 + n/2 + n
AlgorithmAnalysis-BookChapter_Draft Page 23 of 33 Pathak V.


4.4 Some Properties of Asymptotic Notations
Little Oh, Big Oh, and Their Buddies
Notation Growth rate Example of use
f(n) = o(g(n)) strictly less than T(n) = cn
2
+ o(n
2
)
f(n) = O(g(n)) no greater than T(n, m) =O(nlogn+m)
f(n) = (g(n)) the same as T(n) = (n log n)
f(n) = (g(n)) no less than T(n) = (n)
f(n) = (g(n)) strictly greater than T(n) = (log n)
Notation Growth rate Example of use
f(n) = o(g(n)) strictly less than T(n) = cn
2
+ o(n
2
)
f(n) = O(g(n)) no greater than T(n, m) =O(nlogn+m)
f(n) = (g(n)) the same as T(n) = (n log n)
f(n) = (g(n)) no less than T(n) = (n)
f(n) = (g(n)) strictly greater than T(n) = (log n)
< << <

= == =

> >> >







5. Best-case, Worst-case, Average Case Analysis

The actual performance of an algorithm largely depends on the data set instance. Depending
on the data characteristics, the best, the worst, and an average performance of an algorithm
can be determined. We can understand the notion going through following examples.
Example 5.1. Quick Sort
The running time of the QuickSort algorithm is equal to the running time of the two recursive
calls plus the linear time spent in the Partition step; the pivot selection takes only constant
time.
Typical Quick_Sort Algorithm
Q_Sort(A[],l, u)
1. if (l==u) return;
2. p = select_pivot_element();
3. m = Partition (A, l, m, u) ;//determines correct position
//for p in the list.
4. if (l<m-1) Q_Sort(A, l, m-1);
5. if (m+1<u) Q_Sort(A, m+1, u);

AlgorithmAnalysis-BookChapter_Draft Page 24 of 33 Pathak V.


This gives the basic QuickSort recurrence as
T(N) = T(i) + T(N i 1) + cN ; Take T(0) = T(1) = 1.
Where, i =
1
S is the number of elements in S
1
, the first partition. -- [5.1.a]

Worst Case Performance
When the pivot is the smallest element while sorting in ascending order (typically when the
input list is already sorted in desired order), all the time, then, i = 0.
Moreover, if we ignore T(0) = 1, which is insignificant with respect to large N, then, the
resulting recurrence becomes
T(N) = T(N-1) + cN, N>1 -- [5.1.b]
Solving this recurrence, we get
T(N) = T(1) + c. ) (
2
2
N i
i
=

=
-- [5.1.c]
Hence, for QuickSort the Worst Case Time Complexity = O(N
2
).

Best Case Performance
Best case of QuickSort occurs in ideal sequence of input data elements, when pivot is always
(in each partition) found in the middle.
Then, T(N) = T(N/2) + cN, N>1 -- [5.1.d]
Solving this recurrence, we get the solution as :
T(N) = O( NlgN ) -- [5.1.e]
Hence, for QuickSort the Best Case Time Complexity = O(NlgN).

Average Case Performance
For, average case, we assume that each of the sizes for S
1
is equally likely, and hence the
probability for each size is
N
1
. Then we can analyze as following for all partitioning
strategies which preserve randomness.
As per the assumptions above,
The average value of T(i), and hence of T(N i 1), is ( )

=
N
j
j T
N
0
) (
1
. -- [5.1.f]
Then equation 5.1.a becomes
AlgorithmAnalysis-BookChapter_Draft Page 25 of 33 Pathak V.


T(N) = ( ) cN j T
N
N
j
+

=
1
0
) (
2
-- [5.1.g]
Solve eqn. 5.1.g :
N.T(N) =
2
1
0
) ( . 2 cN j T
N
j
+

=
[Multiply both sides by N] -- [5.1.h]
(N-1).T(N-1) =
2
2
0
) 1 ( ) ( . 2 +

=
N c j T
N
j
-- [5.1.i]
If we subtract eqn. 5.1.i from eqn 5.1.h, we get,
N.T(N) ( N-1).T(N-1) = 2T(N-1) + 2cN c [Applying telescopic sums]
We rearrange the terms and drop the insignificant c on the right.
NT(N) = (N+1)T(N-1)+2cN -- [5.1.j]
Solve eqn. 5.1.j :

1
2 ) 1 (
1
) (
+
+

=
+ N
c
N
N T
N
N T
-- [5.1.k]
Similarly,

N
c
N
N T
N
N T 2
1
) ( ) 1 (
+

-- [5.1.l]

1
2
2
) 2 (
1
) 2 (

N
c
N
N T
N
N T
-- [5.1.m]




3
2
2
) 1 ( ) 2 ( c T
N
T
+ = -- [5.1.n]
Adding eqns. 5.1.k to 5.1.n, we get

+
=
+ =
+
1
3
1
2
2
) 1 (
1
) (
N
i
i
T
N
N T
-- [5.1.o]
) (log
1
) (
N O
N
N T
=
+
[since the sum on the right is approximately
log
e
(N+1)+ - 3/2 ; 0.577 is Euler
Constant.]
Thus, for QuickSort the Average Case Time Complexity = O(NlgN).
AlgorithmAnalysis-BookChapter_Draft Page 26 of 33 Pathak V.



Example 5.2. Merge_Sort
In a typical Merge_Sort algorithm, [ref. example 3.2] in each pass (or in each recursive call
in a recursive version) regardless of what the data is, the list is divided in two equal parts, and
each portion is processed in similar fashion. Moreover, in each call of the Merge_Sort, after
completion of two calls over two (N/2)-length portions, a Merge routine processes the whole
length (i.e. N) to combine the results. So, in all cases, whatever the data characteristic is, the
recurrence (we consider for the recursive version), is the same, i.e.,
T(N) = 2T(N/2)+N -- [5.2.a]
We, know the above recurrence has the solution
T(N) = O(NlogN)
Thus, for MergeSort the Best, Worst, Average Case Time Complexity = O(NlgN).






6. Improving Time complexity: Some Algorithm Design Examples

Sometimes applying some reasoning helps:
Example 6.1. Computing X
n

Comp_X_n_ Algo_I (X,n)
1. power := x
2. for i := 1 to n-1 do
3. power := power * X;
time complexity - (n).
Comp_X_n_ Algo_II (X,n) // applicable particularly if n = 2
k
, k being
// some positive integer.
1. power := x
2. for i := 1 to log
2
n do
3. power := power * power;
time complexity - (log
2
n).
Comp_X_n_ Algo_III (X,n) // applicable even if n != 2
k
, k being some
// positive integer.
1. power := 1, z:=X
2. for m := n to 1 do
3. while ((m mod 2) == 0) do
4. m:= floor(m/2), z:= z*z;
5. m:= m-1
AlgorithmAnalysis-BookChapter_Draft Page 27 of 33 Pathak V.


6. power := power * z;
7. return power
time complexity - (log
2
n).

Sometimes divide and conquer technique helps:
Example 6.2. Processing a bit stream
Given a bit stream such that all 1s precede any 0, we are required to find the position
of the last 1.
Bit_Proc1_Algo_I (A[], n) //n is the length of the string.
//The last one can be found simply by performing a sequential
//search. [try!! write algo]
time complexity - (n).

Bit_Proc1_Algo_II (A[], l, u) // u-l+1 = n
1. if (l == u)
2. if (A[l] == 1) return l
3. else return -1 // -1 indicates 1 not found
4. m:= (l+u)/2
5. if (A[m]==1)
6. if (A[m+1]==0)return m else Bit_Proc1_Algo_I(A, m+1, u)
7. else // i.e. if (A[m]==0)
8. if (A[m-1]==1)return m else Bit_Proc1_Algo_I(A, l, m-1)
time complexity - (log
2
n). [try!! verify]

Review Exercise:
1. Suppose that each row of an n x n array A consists of 1s and 0s such that, in any
row I of A, all the 1s come before any 0s in that row. Suppose further that the number
of 1s in row I is at least the number in row i+1, for i= 0,1, ,n-2. Assuming A is
already in memory, describe a method running in O(nlogn) time ( not O(n2) time) for
finding the row of A that contains the most 1s.
2. Write a recursive algorithm for finding both the maximum and the minimum
elements in an array A of n elements. Determine its time complexity. Try to revise the
algorithm to improve time complexity.
3. Suppose that each node in a BST also has the field leftsize. Design an algorithm to
insert an element x into such a BST. The complexity of your algorithm should be O(h),
where h is the height of the search tree. Verify it.
4. An array A contains n-1 unique integers in the range [0, n-1], that is there is one
number from this range that is not in A. Design an O(n)-time algorithm for finding that
number. You are allowed to use only O(1) additional space besides the array itself.
5. Let x[1:n] and y[1:n] contain two sets of integers, each sorted in nondecreasing order.
Write an algorithm that finds the median of te 2n combined elements. What is the time
complexity of your algorithm?
6. The k-th quantiles of an n-element set are the k-1 elements from the set that divide
the sorted set into k equal-sized sets. Give an O(n logk) time algorithm to list the kth
quantiles of a set.
AlgorithmAnalysis-BookChapter_Draft Page 28 of 33 Pathak V.


7. Given two vectors X=(x1,, xn) and Y=(y1,, yn), X<Y if there exists an i, 1<=i<=n,
such that xj = yj for 1<=j<I and xi<yi. Given m vectors each of size n, write an
algorithm that determines the minimum vector. Analyze time complexity





















8. Probabilistic Analysis and Randomized Algorithms

Probabilistic analysis is the use of probability in the analysis of problems. As such, this
approach helps in assessing average performance of an algorithm. To do so, we need to
know, or make assumptions about, the characteristic distribution of inputs. Then we analyze
our algorithm, computing an expected running time. The expectation is taken over the
distribution of the possible inputs. Thus, in effect, we estimate the average-case behavior of
the algorithm, i.e. average running time over all possible inputs. Thus we may use
probabilistic analysis as a technique in designing an efficient algorithm and as a means to
gain an insight into a problem.
However, for some problems, we can not describe a reasonable input distribution, and in
these cases we cannot use probabilistic analysis. In fact, here, we know very little about the
distribution or we may not be able to model the distribution information computationally.
Even in such situations, we can use probability and randomness as a tool for algorithm design
AlgorithmAnalysis-BookChapter_Draft Page 29 of 33 Pathak V.


and analysis, by implementing part of the algorithm using randomization. In these situations
probabilistic analysis guides in the development of such randomized algorithms.
More generally, for a randomized algorithm, the performance behavior is determined not
only by its input but also by values produced by a random-number generator.

Randomized Algorithms and their Performance Estimation with Probabilistic
Analysis
Example 8.1. Randomized QuickSort
RANDOM_QUICKSORT (A, lb, ub)
1. if lb < ub
2. then q<- RANDOM_PARTITION(A,lb,ub)
3. RANDOM_QUICKSORT (A, lb, q-1)
4. RANDOM_QUICKSORT (A, q+1, ub)
RANDOM_PARTITION (A, lb , ub)
1. i <- Random (lb, ub)
2. exchange A[ub], A[i]
3. return PARTITION(A, lb, ub)
PARTITION (A, lb, ub)
1. x <- A[r]
2. I <- p-1
3. for j <- p to r-1
4. do if A[j] <= x
5. then I <- i+1
6. exchange A[i], A[j]
7. exchange A[i+1], A[r]
8. return i+1
Expected Running Time
The running time of Quicksort is dominated by the time spent in the PARTITION. Each time
PARTITION is called, a pivot element is selected and this element is never included in any
future recursive calls to RANDOM_QUICKSORT and PARTITION. Thus there can be at
most n calls to PARTITION over the entire execution of the quicksort algorithm. One call to
PARTITION takes O(1) time plus some time to execute the for loop ( line 3-6 ). Each
iteration of this for loop performs a comparison in line 4.
Let X be the number of comparisons performed in line 4 of PARTITION over the entire
execution of the algorithm on an n-element array. Then the running time of the sorting
algorithm as whole would be O(n + X).
So we need to compute X, the total number of comparisons performed in all calls to
PARTITION. In fact, we derive an overall bound on the total number of comparisons.
Let us rename the elements of A as z
1
, z
2
, ., z
n
, z
i
being the i
th
smallest element.
AlgorithmAnalysis-BookChapter_Draft Page 30 of 33 Pathak V.


Let Z
ij
= { z
i
, z
i+1
, , z
j
} to be the set of elements between z
i
and z
j
inclusive.
Let X
ij
= I { z
i
is compared to z
j
}, where we consider any occurrence of comparison during
the execution of the algorithm, not just during one iteration or one call of PARTITION.
Since each element is compared at most once, we have:

= + =
=
1
1 1
n
i
n
i j
ij
X X .
Taking expectations on both sides, and then using linearity of expectations and using the
lemma X
A
= I{A} E[X
A
] = Pr[A].
We get,i
E[X] =

= + =
1
1 1
n
i
n
i j
ij
X E =

= + =
1
1 1
] [
n
i
n
i j
ij
X E
=

= + =
1
1 1
} Pr{
n
i
j
n
i j
i
z to compared is z . -- eq. [A]
Now,
} Pr{
j i
z to compared is z =
} Pr{
ij j i
Z from chosen pivot first is z or z
= } Pr{
ij i
Z from chosen pivot first is z
+ } Pr{
ij j
Z from chosen pivot first is z
=
1
2
1
1
1
1
+
=
+
+
+ i j i j i j
[since the two events are mutually
exclusive.] --eq. [B]
Combining equations [A] and [B], we get,
E[X] =

= + =
+
1
1 1
1
2
n
i
n
i j
i j
.
=

=
+
1
1 1
1
2
n
i
i n
k
k

<

= =
1
1 1
2
n
i
n
k
k

=

1
1
) (lg
n
i
n
AlgorithmAnalysis-BookChapter_Draft Page 31 of 33 Pathak V.


= O(n lg n).
Thus we conclude that using RANDOM_PARTITION, the expected running time of
quicksort is O(n lg n).
Average case performance versus estimated time of randomized algorithm
The comparison between O(n lg n) as average case run-time of quicksort and O(n lg n) as
expected run time of randomized quicksort is made clear by considering the fact that the
average case performance estimation hinges on the assumption that all the input permutations
are equally likely. However, we may ever encounter the worst case estimated O(n
2
)
performance across several runs of ordinary quicksort.

Example 8.2. Randomized version of Statistically O(1) New_Sort() Algorithm
[ the New_Sort() algorithm proposed by Dr. S. Chakrabarty (Dept. of Appl. Maths,
BIT,Mesra), published in ACM, is shown to bear statistically O(1) time complexity.]

//RANDOMIZED NSORT: source code in C.
#include<iostream.h>
#include<conio.h>
#include<stdlib.h>

int ctr=0, ctc=0;

void RNsort(int * a, int f, int l)
{ int t[10];
ctr++;
if (f==l) return;
randomize();
int r = rand()%(l-f) +f;
int p = a[r];
int i=f, b=f, e=l;

while (i<=l)
{ if(i==r) i++;
else { if (a[i] <p)
{ t[b] = a[i]; i++; b++; ctc++;}
else
{ t[e] = a[i]; i++; e--; ctc++;}
}
}
t[b] = p;
for ( i=f; i<=l; i++) a[i] = t[i];
if (f<b-1) RNsort( a, f, b-1);
if (b+1<l) RNsort( a, b+1, l);
}

void main()
AlgorithmAnalysis-BookChapter_Draft Page 32 of 33 Pathak V.


{
int k[8] = {5,4,8,9,2,3,7,1};
clrscr();
cout<<"Original List\n";
for (int i =0; i<8; i++)
cout<< k[i]<<'\t';

RNsort(k, 0, 7);

cout<<"\nList after sorting\n";
for (i =0; i<8; i++)
cout<< k[i]<<'\t';

cout<<"\nTotal no. of Calls to RNsort: "<<ctr<<endl;
cout<<"\n Total no. of comparisions: "<<ctc<<endl;
getch();
}
___________________________________________________________________________


================================================
To DO Further: Test the algo for different characteristic sets of data i.e. those generated by
Gaussian distribution function, Bell function etc..





Problem Solving with Randomized algorithms and probabilistic analysis
{ref: Ex.5-1 and 5-2 of Cormen --determining expected time.}
Example 8.3. Search in an unordered list.
//RANDOMIZED SEARCH ALGORITHM: pseudocode
R_SEARCH (A, l, u, val, k)
1. count = 0
2. while (count<u-l+1) do step 3-7
3. i = random() % (u-l)+u
4. if (A[i] == val) then do step 5,6
5. print i
6. exit
7. count = count + 1
8. print value not found
9. end
Expected number of comparisons to find given value
I. If there is only single occurrence of val, i.e. k=1.
AlgorithmAnalysis-BookChapter_Draft Page 33 of 33 Pathak V.


Here,
)] ] [ Pr[( ] [ val i Ar X E
i
=
)] ] [ Pr[( 1 val i Ar = =
Since single value of val is present in A, so all the indices are equally likely to have it.
So,
n
val i Ar
1
)] ] [ Pr[( = = , and,

n
n
n
X E
i
1 1
1 ] [

= = , where, n = u-l+1 = size of A.
Hence,


= =
= =
n
i
i
n
i
i
X E X E X E
1 1
] [ ] [ ] [
1
) 1 ( 1
1
=

=

=
n
n
n n
n
n
n
i

II. If there are more than single occurrences of val, i.e. 1 k .
Generalizing for 1 k ,

n
k n
val i Ar X E
i

= = )] ] [ Pr[( ] [
Therefore,


= =
= =
n
i
i
n
i
i
X E X E X E
1 1
] [ ] [ ] [ k n
n
k n n
n
k n
n
i
=

=

=
) (
1

III. If there is no occurrence of val, i.e. k =0.
1 )] ] [ Pr[( ] [ = = =
n
n
val i Ar X E
i

So,

= = =
= = = =
n
i
n
i
i
n
i
i
n X E X E X E
1 1 1
1 ] [ ] [ ] [







Conventional Randomized Algorithm references: Monte Carlo, Las Vegas techniques