(Jignesh Joshi) 1 - Introduction PDF

CO-303
DESIGN AND ANALYSIS OF

ALGORITHMS
Introduction
• What is Algorithm?
Mr. Jignesh Joshi, Design and Analysis of Algorithms, B.Tech-III (CoED), SVNIT, 2019-20
Introduction
• What is Algorithm?
– An algorithm is a finite set of instructions that, if followed,
accomplishes a particular task.
– Word defined by a Persian Mathematician , Abu Ja’far
Mohammed ibn Musa al Khowarizmi (825 A.D.)
Introduction…
• Characteristics of an Algorithm
Introduction…
– Input: ?
– Output: ?
– Definiteness: ?
– Finiteness: ?
– Effectiveness: ?
Introduction…
– Input: zero or more inputs, taken from a specified set of objects
– Output: At least one quantity is produced relation to the inputs
– Definiteness: Each instruction must be precisely defined
– Finiteness: It terminates after a finite number of steps
– Effectiveness: All operations to be performed must be sufficiently
basic that they can be done exactly and in finite length.
Problems vs Algorithms vs Programs
– For each problem or class of problems, there may
be many different algorithms.
– For each algorithm, there may be many different
implementations (programs).
– For each problem or class of problems, there may
be many different algorithms.
– For each algorithm, there may be many different
implementations (programs).
Analysis of algorithms
Analysis of algorithms
• Measuring efficiency of an algorithm
– Time : How long the algorithm takes (running time)
– Space : Memory requirement
Time and space
Time and space
• Time depends on processing speed
– Not possible to change for given hardware
• Space is a function of available memory
– Easier to reconfigure
• Typically, we will focus on time, not space
Measuring running time
Measuring running time
• Analysis independent of underlying hardware
– Don’t use actual time
– Measure in term of “Basic operations”
Input size
Input size
• Running time depends on input size
• Measure time efficiency as function of input size
– Input size 𝑛
– Running time 𝑡(𝑛)
– Typically 𝑡(𝑛) is worst case estimate
Worst-case analysis
Worst-case analysis
• Why do we usually focus on the worst case analysis?
– Being upper bound, the worst case guarantees that the algorithm will
not take any longer.
– Fairly occurs in many applications.
– Average case is often roughly as bad as the worst case.
Example 1: Sorting
Example 1: Sorting
• Sorting as array with 𝑛 elements
Example 1: Sorting
– Basic algorithms : time proportional to 𝑛2
Example 1: Sorting
– Best algorithms : time proportional to 𝑛 log 𝑛
Example 1: Sorting
– Typical CPUs process up to 108 operation per second ( for approximate
calculation)
Example 1: Sorting
calculation)
– Telephone directory for mobile phone users in India
Example 1: Sorting
calculation)
– India has about 1 billion = 109 phones
Example 1: Sorting
calculation)
– Basic 𝑛2 algorithm requires 1018 operations
Example 1: Sorting
calculation)
– 108 operation per second => 1010 seconds
Example 1: Sorting
calculation)
– 2778000 hours
Example 1: Sorting
calculation)
– 2778000 hours
– 115700 days
Example 1: Sorting
calculation)
– 2778000 hours
– 115700 days
– 300 years!!!
Example 1: Sorting
calculation)
– India has about 1 billion = 109 phones • Best 𝑛 log 𝑛 algorithm
– Basic 𝑛2 algorithm requires 1018 operations takes only about
– 108 operation per second => 1010 seconds 3 × 1010 operations
– 2778000 hours
– 115700 days
– 300 years!!!
Example 1: Sorting
calculation)
– 2778000 hours • About 300 seconds
– 115700 days
– 300 years!!!
Example 1: Sorting
calculation)
– 2778000 hours • About 300 seconds
– 115700 days • About 5 minutes
– 300 years!!!
Typical functions
Problem #1
• Problem: print “𝐻𝑒𝑙𝑙𝑜 𝑁𝐼𝑇 𝑆𝑢𝑟𝑎𝑡” for 𝑛 times
Problem #1
• Algorithm: Print
------
------
------
Problem #1
• Algorithm: Print
------
------
------
• What could be the running time of the
solution?
Analyzing Problem #1
• Prepare a table as shown below
Statement cost times_executed
1
2
3
4
5
• Algorithm/ pseudo code: Print (𝑛)
1. 𝑖 = 1
2. While 𝑖 ≤ 𝑛
3. Print “𝐻𝑒𝑙𝑙𝑜 𝑁𝐼𝑇 𝑆𝑢𝑟𝑎𝑡”
4. 𝑖 = 𝑖 + 1
5. Exit
1. 𝑖 = 1………………………………………….𝐶1
2. While 𝑖 ≤ 𝑛…………………………….….𝐶2
3. Print “𝐻𝑒𝑙𝑙𝑜 𝑁𝐼𝑇 𝑆𝑢𝑟𝑎𝑡”.………𝐶3
4. 𝑖 = 𝑖 + 1…………………………….…𝐶4
5. Exit………………………………………….….𝐶5
1. 𝑖 = 1………………………………………….𝐶1
2. While 𝑖 ≤ 𝑛………………………………..𝐶2
3. Print “𝐻𝑒𝑙𝑙𝑜 𝑁𝐼𝑇 𝑆𝑢𝑟𝑎𝑡”.………𝐶3
4. 𝑖 = 𝑖 + 1……………………………….𝐶4
5. Exit……………………………………………..𝐶5
Total steps = Total time = 𝑪𝟏 + 𝒏 + 𝟏 𝑪𝟐 + 𝒏𝑪𝟑 + 𝒏𝑪𝟒 + 𝑪𝟓
Problem #2
• Problem: To determine the largest of 𝑛 nos.
• Problem: To determine the largest of 𝒏 nos.
• Running time if the input is strictly ascending/strictly descending ?
Algorithm Largest1 𝑥𝑖 , 𝑛 Algorithm Largest2 𝑥𝑖 , 𝑛
1 Let 𝑚𝑎𝑥 = 𝑥1 1 For 𝑖 = 1 to (𝑛 − 1)
2 For 𝑖 = 2 to 𝑛 2 Do if 𝑥𝑖 > 𝑥𝑖+1
3 Do if 𝑥𝑖 > 𝑚𝑎𝑥 3 Then swap < 𝑥𝑖 , 𝑥𝑖+1 >
4 Then 𝑚𝑎𝑥 = 𝑥𝑖 4 Print 𝑥𝑛
5 Print 𝑚𝑎𝑥
1 Let 𝑚𝑎𝑥 = 𝑥1 …………….…𝐶1 1 For 𝑖 = 1 to (𝑛 − 1) ………….……𝐶1
2 For 𝑖 = 2 to 𝑛……………..…𝐶2 2 Do if 𝑥𝑖 > 𝑥𝑖+1 ……………..…….…𝐶2
3 Do if 𝑥𝑖 > 𝑚𝑎𝑥…………..…𝐶3 3 Then swap < 𝑥𝑖 , 𝑥𝑖+1 >…𝐶3
4 Then 𝑚𝑎𝑥 = 𝑥𝑖 …...𝐶4 4 Print 𝑥𝑛 ………………………………..…𝐶4
5 Print 𝑚𝑎𝑥………………….....𝐶5
1 Let 𝑚𝑎𝑥 = 𝑥1 …………….…𝐶1 1 For 𝑖 = 1 to (𝑛 − 1) ………….……𝐶1
2 For 𝑖 = 2 to 𝑛……………..…𝐶2 2 Do if 𝑥𝑖 > 𝑥𝑖+1 ……………..…….…𝐶2
3 Do if 𝑥𝑖 > 𝑚𝑎𝑥…………..…𝐶3 3 Then swap < 𝑥𝑖 , 𝑥𝑖+1 >…𝐶3
4 Then 𝑚𝑎𝑥 = 𝑥𝑖 …...𝐶4 4 Print 𝑥𝑛 ………………………………..…𝐶4
5 Print 𝑚𝑎𝑥………………….....𝐶5
Total steps = Total time = 𝐶1 + 𝑛𝐶2 + Total steps = Total time = 𝑛𝐶1 +
𝑛 − 1 𝐶3 + 𝑛 − 1 𝐶4 + 𝐶5 (𝑛 − 1)𝐶2 + 𝑛 − 1 𝐶3 + 𝐶4
Problem #3
• Consider the code snippet
1 For 𝑖 = 1 to 𝑛
2 For 𝑗 = 1 to 𝑛
3 Print "𝐷𝐴𝐴 2019"
• What is the cost of execution?
1 For 𝑖 = 1 to 𝑛………………………….…………..𝐶1
2 For 𝑗 = 1 to 𝑛………………….………….𝐶2
3 Print "𝐷𝐴𝐴 2019"….……….….𝐶3
1 For 𝑖 = 1 to 𝑛………………………….…………..𝐶1
2 For 𝑗 = 1 to 𝑛………………….………….𝐶2
3 Print "𝐷𝐴𝐴 2019"….……….….𝐶3
Total time = 𝑪𝟏 𝒏 + 𝟏 + 𝑪𝟐 𝒏 𝒏 + 𝟏 + 𝑪𝟑 𝒏𝟐
1 For 𝑖 = 1 to 𝑛………………………….…………..𝐶1
2 For 𝑗 = 1 to 𝑛………………….………….𝐶2
3 Print "𝐷𝐴𝐴 2019"….……….….𝐶3
Total time = 𝑪𝟏 𝒏 + 𝟏 + 𝑪𝟐 𝒏 𝒏 + 𝟏 + 𝑪𝟑 𝒏𝟐
= 𝑪𝟏 𝒏 + 𝟏 + 𝑪𝟐 (𝒏𝟐 +𝒏) + 𝑪𝟑 𝒏𝟐
Problem #4
2 For 𝑗 = 1 to 𝑖
1 For 𝑖 = 1 to 𝑛………………………….…………..𝐶1
2 For 𝑗 = 1 to 𝑖………………….………….𝐶2
3 Print "𝐷𝐴𝐴 2019"….……….….𝐶3
1 For 𝑖 = 1 to 𝑛………………………….…………..𝐶1
2 For 𝑗 = 1 to 𝑖………………….………….𝐶2
3 Print "𝐷𝐴𝐴 2019"….……….….𝐶3
(𝒏+𝟏) 𝒏+𝟐 𝒏 𝒏+𝟏

Total time = 𝑪𝟏 𝒏 + 𝟏 + 𝑪𝟐 − 𝟏 + 𝑪𝟑
𝟐 𝟐
1 For 𝑖 = 1 to 𝑛………………………….…………..𝐶1
2 For 𝑗 = 1 to 𝑖………………….………….𝐶2
3 Print "𝐷𝐴𝐴 2019"….……….….𝐶3
(𝒏+𝟏) 𝒏+𝟐 𝒏 𝒏+𝟏

Total time = 𝑪𝟏 𝒏 + 𝟏 + 𝑪𝟐 − 𝟏 + 𝑪𝟑
𝟐 𝟐
𝒏𝟐 +𝟑𝒏 𝒏𝟐 +𝒏
=𝑪𝟏 𝒏 + 𝟏 + 𝑪𝟐 + 𝑪𝟑
𝟐 𝟐
Problem #5
2 For 𝑗 = 𝑖 to 𝑛
1 For 𝑖 = 1 to 𝑛………………………….…………..𝐶1
2 For 𝑗 = 𝑖 to 𝑛………………….………….𝐶2
3 Print "𝐷𝐴𝐴 2019"….……….….𝐶3
1 For 𝑖 = 1 to 𝑛………………………….…………..𝐶1
2 For 𝑗 = 𝑖 to 𝑛………………….………….𝐶2
3 Print "𝐷𝐴𝐴 2019"….……….….𝐶3
(𝒏+𝟏) 𝒏+𝟐 𝒏 𝒏+𝟏

Total time = 𝑪𝟏 𝒏 + 𝟏 + 𝑪𝟐 − 𝟏 + 𝑪𝟑
𝟐 𝟐
1 For 𝑖 = 1 to 𝑛………………………….…………..𝐶1
2 For 𝑗 = 𝑖 to 𝑛………………….………….𝐶2
3 Print "𝐷𝐴𝐴 2019"….……….….𝐶3
(𝒏+𝟏) 𝒏+𝟐 𝒏 𝒏+𝟏

Total time = 𝑪𝟏 𝒏 + 𝟏 + 𝑪𝟐 − 𝟏 + 𝑪𝟑
𝟐 𝟐
𝒏𝟐 +𝟑𝒏 𝒏𝟐 +𝒏
=𝑪𝟏 𝒏 + 𝟏 + 𝑪𝟐 + 𝑪𝟑
𝟐 𝟐
Problem #6
• Problem: Insertion sort
• Algorithm Insertion-Sort 𝐴[], 𝑛
2 Do 𝑘𝑒𝑦 = 𝐴[𝑗]
3 𝑖 =𝑗−1
4 While (𝑖 > 0) and (𝐴 𝑖 > 𝑘𝑒𝑦)
5 Do 𝐴 𝑖 + 1 = 𝐴[𝑖]
6 𝑖 =𝑖−1
7 𝐴 𝑖 + 1 = 𝑘𝑒𝑦
3 𝑖 =𝑗−1
5 Do 𝐴 𝑖 + 1 = 𝐴[𝑖]
6 𝑖 =𝑖−1
7 𝐴 𝑖 + 1 = 𝑘𝑒𝑦
• How do we analyze the time complexity?
3 𝑖 =𝑗−1
5 Do 𝐴 𝑖 + 1 = 𝐴[𝑖]
6 𝑖 =𝑖−1
7 𝐴 𝑖 + 1 = 𝑘𝑒𝑦

• We need to analyze how many times the while loop is executed?
3 𝑖 =𝑗−1
5 Do 𝐴 𝑖 + 1 = 𝐴[𝑖]
6 𝑖 =𝑖−1
7 𝐴 𝑖 + 1 = 𝑘𝑒𝑦

• We need to analyze how many times the while loop is executed?
– Assume while loop is executed 𝒕𝒋 times…
• Then the running time is given by the expression
3 𝑖 = 𝑗−1
5 Do 𝐴 𝑖 + 1 = 𝐴[𝑖]
6 𝑖 = 𝑖−1
7 𝐴 𝑖 + 1 = 𝑘𝑒𝑦
• Then the running time is given by the expression
𝑛 𝑛 𝑛
𝐶1 𝑛 + 𝐶2 𝑛 − 1 + 𝐶3 𝑛 − 1 + 𝐶4 𝑡𝑗 + 𝐶5 (𝑡𝑗 −1) + 𝐶6 (𝑡𝑗 −1) + 𝐶7 (𝑛 − 1)

𝑗=2 𝑗=2 𝑗=2
3 𝑖 = 𝑗−1
5 Do 𝐴 𝑖 + 1 = 𝐴[𝑖]
6 𝑖 = 𝑖−1
7 𝐴 𝑖 + 1 = 𝑘𝑒𝑦
• When does the best case occur?
3 𝑖 = 𝑗−1
5 Do 𝐴 𝑖 + 1 = 𝐴[𝑖]
6 𝑖 = 𝑖−1
7 𝐴 𝑖 + 1 = 𝑘𝑒𝑦
• 𝑡𝑗 = 1 in every case
• i.e. when the array is sorted
3 𝑖 = 𝑗−1
5 Do 𝐴 𝑖 + 1 = 𝐴[𝑖]
6 𝑖 = 𝑖−1
7 𝐴 𝑖 + 1 = 𝑘𝑒𝑦
• Then the best case running time is
𝑛 𝑛 𝑛
• 𝐶1 𝑛 + 𝐶2 𝑛 − 1 + 𝐶3 𝑛 − 1 + 𝐶4 𝑗=2 1 + 𝐶5 𝑗=2 0 + 𝐶6 𝑗=2 0 + 𝐶7 (𝑛 − 1)
3 𝑖 = 𝑗−1
5 Do 𝐴 𝑖 + 1 = 𝐴[𝑖]
6 𝑖 = 𝑖−1
7 𝐴 𝑖 + 1 = 𝑘𝑒𝑦
• Then the best case running time is
𝑛 𝑛 𝑛
𝐶1 𝑛 + 𝐶2 𝑛 − 1 + 𝐶3 𝑛 − 1 + 𝐶4 1 + 𝐶5 0 + 𝐶6 0 + 𝐶7 𝑛 − 1
𝑗=2 𝑗=2 𝑗=2
= 𝐶1 𝑛 + 𝐶2 𝑛 − 1 + 𝐶3 𝑛 − 1 + 𝐶4 𝑛 − 1 + 𝐶7 𝑛 − 1
• When does the worst case occur?
3 𝑖 = 𝑗−1
4 While (𝑖 > 0) and (𝐴 𝑗 > 𝑘𝑒𝑦)
5 Do 𝐴 𝑖 + 1 = 𝐴[𝑖]
6 𝑖 = 𝑖−1
7 𝐴 𝑖 + 1 = 𝑘𝑒𝑦
• At least when the while loop is executed for all the values of 𝑖……
• i.e. when the array is reverse sorted
3 𝑖 = 𝑗−1
5 Do 𝐴 𝑖 + 1 = 𝐴[𝑖]
6 𝑖 = 𝑖−1
7 𝐴 𝑖 + 1 = 𝑘𝑒𝑦
• Thus, 𝑡𝑗 = 𝑗.
3 𝑖 = 𝑗−1
5 Do 𝐴 𝑖 + 1 = 𝐴[𝑖]
6 𝑖 = 𝑖−1
7 𝐴 𝑖 + 1 = 𝑘𝑒𝑦
• Thus, 𝑡𝑗 = 𝑗. Therefore the expression is
𝑛 𝑛 𝑛
• 𝐶1 𝑛 + 𝐶2 𝑛 − 1 + 𝐶3 𝑛 − 1 + 𝐶4 𝑗=2 𝑗 + 𝐶5 𝑗=2(𝑗 − 1) + 𝐶6 𝑗=2(𝑗 − 1) +
𝐶7 (𝑛 − 1)
3 𝑖 = 𝑗−1
5 Do 𝐴 𝑖 + 1 = 𝐴[𝑖]
6 𝑖 = 𝑖−1
7 𝐴 𝑖 + 1 = 𝑘𝑒𝑦
𝑛 𝑛 𝑛
𝐶1 𝑛 + 𝐶2 𝑛 − 1 + 𝐶3 𝑛 − 1 + 𝐶4 𝑗 + 𝐶5 (𝑗 − 1) + 𝐶6 (𝑗 − 1) + 𝐶7 (𝑛 − 1)
𝑗=2 𝑗=2 𝑗=2
𝑛 𝑛+1 𝑛 𝑛−1 𝑛 𝑛−1

= 𝐶1 𝑛 + 𝐶2 𝑛 − 1 + 𝐶3 𝑛 − 1 + 𝐶4 − 1 + 𝐶5 + 𝐶6
2 2 2
+ 𝐶7 (𝑛 − 1)
𝑛 𝑛 𝑛
𝐶1 𝑛 + 𝐶2 𝑛 − 1 + 𝐶3 𝑛 − 1 + 𝐶4 𝑗 + 𝐶5 (𝑗 − 1) + 𝐶6 (𝑗 − 1) + 𝐶7 (𝑛 − 1)
𝑗=2 𝑗=2 𝑗=2
𝑛 𝑛+1 𝑛 𝑛−1 𝑛 𝑛−1

= 𝐶1 𝑛 + 𝐶2 𝑛 − 1 + 𝐶3 𝑛 − 1 + 𝐶4 − 1 + 𝐶5 + 𝐶6
2 2 2
+ 𝐶7 (𝑛 − 1)
𝑛2 + 𝑛 − 2 𝑛2 − 𝑛 𝑛2 − 𝑛
= 𝐶1 𝑛 + 𝐶2 𝑛 − 1 + 𝐶3 𝑛 − 1 + 𝐶4 + 𝐶5 + 𝐶6
2 2 2
+ 𝐶7 (𝑛 − 1)
Growth of functions
• Consider
• An algorithm A which for a problem, does 𝟐𝒏 basic operation & 𝟐𝑪𝟏 𝒏
total operations, while some other algorithm B does 𝟒. 𝟓𝒏 basic
operations & 𝟒. 𝟓𝑪𝟐 𝒏 total operations.
Growth of functions
• Consider
• Consider constant of proportionality representing overhead operations.
Growth of functions
• Consider
• Which algorithm of the two do you think is better?
Growth of functions …
𝒏 𝟐𝒏 𝟒. 𝟓𝒏
5 10 22
10 20 45
100 200 450
1000 2000 4500
10000 20000 45000
100000 2.0 ∗ 105 4.5 ∗ 105
1000000 = 106 2.0 ∗ 106 4.5 ∗ 106
Growth of functions
• Consider
𝒏𝟑
• Another such example with algo1 taking multiplicative steps while
𝟐
algo2 taking 𝟓𝒏𝟐 steps.
Growth of functions
• Consider
𝒏𝟑
• Another such example with algo1 taking multiplicative steps while
𝟐
algo2 taking 𝟓𝒏𝟐 steps.
• Which algorithm of the two do you think is better?
𝒏 𝟐𝒏 𝟒. 𝟓𝒏 𝒏𝟑 /𝟐 𝟓𝒏𝟐
5 10 22 62 125
10 20 45 500 500
100 200 450 5 ∗ 105 5 ∗ 104
1000 2000 4500 5 ∗ 108 5 ∗ 106
10000 20000 45000 5 ∗ 1011 5 ∗ 108
100000 2.0 ∗ 105 4.5 ∗ 105 5 ∗ 1014 5 ∗ 1010
1000000 2.0 ∗ 106 4.5 ∗ 106 5 ∗ 1017 5 ∗ 1012
= 106
• A relook at costs of insertion sort with 𝑪′𝒊 𝒔 = 𝟏
• Best case
𝑇 𝑛 = 𝐶1 𝑛 + 𝐶2 𝑛 − 1 + 𝐶3 𝑛 − 1 + 𝐶4 𝑛 − 1 + 𝐶7 𝑛 − 1
• Best case
𝑇 𝑛 = 𝐶1 𝑛 + 𝐶2 𝑛 − 1 + 𝐶3 𝑛 − 1 + 𝐶4 𝑛 − 1 + 𝐶7 𝑛 − 1
= 5𝑛 − 4
• Best case
𝑇 𝑛 = 𝐶1 𝑛 + 𝐶2 𝑛 − 1 + 𝐶3 𝑛 − 1 + 𝐶4 𝑛 − 1 + 𝐶7 𝑛 − 1
= 5𝑛 − 4
• Worst Case
𝑛2 +𝑛−2 𝑛2 −𝑛
𝑇 𝑛 = 𝐶1 𝑛 + 𝐶2 𝑛 − 1 + 𝐶3 𝑛 − 1 + 𝐶4 + 𝐶5
2 2
𝑛2 − 𝑛
+𝐶6 + 𝐶7 (𝑛 − 1)
2
• Best case
𝑇 𝑛 = 𝐶1 𝑛 + 𝐶2 𝑛 − 1 + 𝐶3 𝑛 − 1 + 𝐶4 𝑛 − 1 + 𝐶7 𝑛 − 1
= 5𝑛 − 4
• Worst Case
𝑛2 +𝑛−2 𝑛2 −𝑛
𝑇 𝑛 = 𝐶1 𝑛 + 𝐶2 𝑛 − 1 + 𝐶3 𝑛 − 1 + 𝐶4 + 𝐶5
2 2
𝑛2 − 𝑛
+𝐶6 + 𝐶7 (𝑛 − 1)
2
1
= (3𝑛2 + 7𝑛 − 8)
2
• Best case
𝑇 𝑛 = 𝐶1 𝑛 + 𝐶2 𝑛 − 1 + 𝐶3 𝑛 − 1 + 𝐶4 𝑛 − 1 + 𝐶7 𝑛 − 1
= 5𝑛 − 4
• Worst Case
𝑛2 +𝑛−2 𝑛2 −𝑛
𝑇 𝑛 = 𝐶1 𝑛 + 𝐶2 𝑛 − 1 + 𝐶3 𝑛 − 1 + 𝐶4 + 𝐶5
2 2
𝑛2 − 𝑛
+𝐶6 + 𝐶7 (𝑛 − 1)
2
1
= (3𝑛2 + 7𝑛 − 8)
2
• Which term dominates the overall result in the above expression,
especially at large values of 𝑛 ?
• Which term dominates the overall result in the above expression,
especially at large values of 𝑛 ?
𝒏 𝑻 𝒏 = 𝟑𝒏𝟐 + 𝟕𝒏 − 𝟖 𝑻 𝒏 = 𝟑𝒏𝟐
10 362 300
100 30692 30000
1000 3.006992 ∗ 106 3.00 ∗ 106
10000 3.0000699992 ∗ 1010 3.00 ∗ 1010
• Does constant of proportionality matter when 𝑛 gets very
large?
large?
• Then, what is asymptotic growth rate, asymptotic order or
order of functions?
large?
• Then, what is asymptotic growth rate, asymptotic order or
order of functions?
• Is it reasonable to ignore smaller values and constants?
• Hence, we shall now also drop the all the terms
• Except the highest degree of the polynomial for the running time of
the algorithm
the algorithm
• Example
the algorithm
• Example
• Insertion sort Best case complexity …
𝑻 𝒏 = 𝟓𝒏 − 𝟒
• So, we will say that complexity is of the order of 𝒏
the algorithm
• Example
𝑻 𝒏 = 𝟓𝒏 − 𝟒
• So, we will say that complexity is of the order of 𝒏
𝟏
𝑻 𝒏 = (𝟑𝒏𝟐 + 𝟕𝒏 − 𝟖)
𝟐
• So, we will say that complexity is of the order of 𝒏𝟐
Order of growth and abstractions
• So, now we have assumed/abstracted at three different levels
viz.
viz.
• Level 1 – ignored the actual cost of execution of each statement.
viz.
• Level 2 – ignored even the abstract cost (𝐶𝑖 ) of each statement.
viz.
• Level 3 – ignore all the terms except for the one with the highest
degree in the expression of time complexity
viz.
• Such analysis is based on the asymptotic growth rate,
viz.
• Such analysis is based on the asymptotic growth rate,

• Asymptotic order or order or functions and called asymptotic analysis
Primary Observations …
• There can be at least three different ways of analyzing the
algorithms
• Empirical analysis
• Mathematical analysis
• Asymptotic analysis
Typical functions
• We are interested in order of magnitude
• 𝑡(𝑛) may proportional to log 𝑛, … , 𝑛2, 𝑛3, … , 2𝑛
• Logarithmic, polynomial, exponential …
Basic Asymptotic Efficiency classes
1 Constant
log 𝑛 Logarithmic
𝑛 Linear
𝑛 log 𝑛 𝑛 log 𝑛
𝑛2 Quadratic
𝑛3 Cubic
2𝑛 Exponential
𝑛! Factorial
Typical functions 𝑡(𝑛)
Input 𝒍𝒐𝒈 𝒏 𝒏 𝒏 𝒍𝒐𝒈 𝒏 𝒏𝟐 𝒏𝟑 𝟐𝒏 𝒏!
𝟏𝟎 3.3 10 33 100 1000 1000 106
𝟏𝟎𝟎 6.6 100 66 104 106 1030 10157
𝟏𝟎𝟎𝟎 10 1000 104 106 109
𝟏𝟎𝟒 13 104 105 108 1012
𝟏𝟎𝟓 17 105 106 1010
𝟏𝟎𝟔 20 106 107
𝟏𝟎𝟕 23 107 108
𝟏𝟎𝟖 27 108 109
𝟏𝟎𝟗 30 109 1010
𝟏𝟎𝟏𝟎 33 1010
Typical functions 𝑡(𝑛)
An interesting “seconds” conversion
102 1.7 min
104 2.8 hours
105 1.1 days
106 1.6 weeks
107 3.8 months
108 3.1 years
109 3.1 decades
1010 3.1 centuries
Asymptotic notations: The Big-O notation
• Definition
• For a given function 𝒈 𝒏 , we say that
𝑶 𝒈 𝒏 = 𝒇 𝒏 if there exists positive constants
𝒄 and 𝒏𝟎 such that 𝟎 ≤ 𝒇 𝒏 ≤ 𝒄𝒈(𝒏)
for all 𝒏 ≥ 𝒏𝟎 }
• Definition
• Defines an upper bound for a function within a constant factor.
• Definition
• 𝒇 𝒏 =𝑶 𝒈 𝒏 ⇒
• 𝒇(𝒏) is dominated in the growth by 𝒈 𝒏
• 𝒇(𝒏) is of the order at most 𝒈 𝒏
• 𝒈(𝒏) grows at least as fast as 𝒇 𝒏
• Definition
• 𝒇 𝒏 =𝑶 𝒈 𝒏 ⇒
• 𝒇(𝒏) is dominated in the growth by 𝒈 𝒏
• 𝒇(𝒏) is of the order at most 𝒈 𝒏
• 𝒈(𝒏) grows at least as fast as 𝒇 𝒏
• Can 𝒇(𝒏) grow faster than 𝒈(𝒏)?
• Can 𝒈(𝒏) grow faster than 𝒇(𝒏)?
The Big-oh notation
Figure: Big-oh notation: 𝑡 𝑛 𝜖 𝑂(𝑔 𝑛 )

The Big-O notation: Illustrations
Function Notation in O
𝑓 𝑛 = 5𝑛 + 8 𝑓 𝑛 = 𝑂(? )
𝑓 𝑛 = 𝑛2 + 3𝑛 − 8 𝑓 𝑛 = 𝑂(? )
𝑓 𝑛 = 12𝑛2 − 11 𝑓 𝑛 = 𝑂(? )
𝑓 𝑛 = 5 ∗ 2𝑛 + 𝑛2 𝑓 𝑛 = 𝑂(? )
𝑓 𝑛 = 3𝑛 + 8 𝑓 𝑛 = 𝑂 𝑛2 ?
𝑓 𝑛 = 5𝑛 + 8 𝑓 𝑛 =𝑂 1 ?
The Big-O notation: Illustrations
• The big-O notation

• Allows us to keep track of the leading team while ignoring
smaller terms…
• Allows us to make concise statements that give
approximations to the quantities to analyze.
• If 𝑓 𝑛 = 𝑂 𝑔 𝑛
• 𝑔(𝑛) is the upper bound, but do we specify how tight this upper
bound is?
• Consider that 𝑓 𝑛 = 𝑂 𝑛 & 𝑔 𝑛 = 𝑂 𝑛2
• Is 𝑓 𝑛 = 𝑂(𝑔(𝑛)) saying the same as reverse i.e. 𝑔 𝑛 = 𝑂 𝑓 𝑛 ?
• The symbol “=” is not proper
• Truly it is 𝜖 which should be used i.e. 𝑓 𝑛 𝜖 𝑂(𝑔 𝑛 )
Asymptotic notations: The Big- Ω notation
Asymptotic notations: The Big- Ω notation
• Definition
𝛀 𝒈 𝒏 = 𝒇 𝒏 if there exists positive constants
𝒄 and 𝒏𝟎 such that 𝟎 ≤ 𝒄𝒈 𝒏 ≤ 𝒇(𝒏)
Asymptotic notations: The Big- Ω notation
• Definition
𝛀 𝒈 𝒏
= 𝒇 𝒏 if there exists positive constants
𝒄 and 𝒏𝟎 such that 𝟎 ≤ 𝒄𝒈 𝒏 ≤ 𝒇(𝒏)
• Defines an lower bound for a function within a constant factor.
The Big-omega notation
Figure: Big-omega notation: 𝑡 𝑛 𝜖 Ω(𝑔 𝑛 )

The Big-omega notation: Illustrations
Function Notation in Ω
𝑓 𝑛 = 3𝑛 + 8 𝑓 𝑛 = Ω(? )
𝑓 𝑛 = 𝑛2 + 3𝑛 − 8 𝑓 𝑛 = Ω(? )
𝑓 𝑛 = 12𝑛2 − 11 𝑓 𝑛 = Ω(? )
𝑓 𝑛 = 6 ∗ 2𝑛 + 𝑛2 𝑓 𝑛 = Ω(? )
𝑓 𝑛 = 3𝑛 + 8 𝑓 𝑛 = Ω 𝑛2 ?
𝑓 𝑛 = 5𝑛 + 8 𝑓 𝑛 =Ω 1 ?
Asymptotic notations: The Big- 𝜃 notation
• Definition
𝜽 𝒈 𝒏 = 𝒇 𝒏 if there exists positive constants
𝒄 and 𝒏𝟎 such that 𝟎 ≤ 𝒄𝟏 𝒈 𝒏 ≤ 𝒇 𝒏 ≤ 𝒄𝟐 𝒈(𝒏)
• Definition
𝜽 𝒈 𝒏
• 𝒇 𝒏 = 𝜽(𝒈 𝒏 ) iff 𝒇 𝒏 = 𝑶(𝒈 𝒏 ) and 𝒇 𝒏 = 𝛀(𝒈 𝒏 )
• Definition
𝜽 𝒈 𝒏
• Neither the big-O notation nor the big-Ω notation describe
the asymptotically tight bounds.
• Definition
𝜽 𝒈 𝒏
• Neither the big-O notation nor the big-Ω notation describe
the asymptotically tight bounds.
• 𝜃-notation to express tighter bounds
The Big-theta notation
Figure: Big-theta notation: 𝑡 𝑛 𝜖 𝜃(𝑔 𝑛 )

The Big-theta notation: Illustrations
Function Notation in Ω
𝑓 𝑛 = 3𝑛 + 8 𝑓 𝑛 = 𝜃(? )
𝑓 𝑛 = 10𝑛2 + 3𝑛 − 8 𝑓 𝑛 = 𝜃(? )
𝑓 𝑛 = 12𝑛2 − 11 𝑓 𝑛 = 𝜃(? )
𝑓 𝑛 = 6 ∗ 2𝑛 + 𝑛2 𝑓 𝑛 = 𝜃 2𝑛 ?
𝑓 𝑛 = 6 ∗ 2𝑛 + 𝑛2 𝑓 𝑛 = 𝜃 𝑛2 ?
𝑓 𝑛 = 3𝑛 + 8 𝑓 𝑛 = 𝜃 𝑛2 ?
𝑓 𝑛 = 5𝑛 + 8 𝑓 𝑛 =𝜃 1 ?
Calculating complexity
• Iterative programs
• Recursive programs
Example 1
• Problem: Maximum value in an array
Example 1
• Problem: Maximum value in an array

• Solution:
Function 𝑀𝑎𝑥𝐸𝑙𝑒𝑚𝑒𝑛𝑡(𝐴, 𝑛)
1 𝑚𝑎𝑥𝑣𝑎𝑙 = 𝐴[1]
3 If 𝐴 𝑖 > 𝑚𝑎𝑥𝑣𝑎𝑙
4 𝑚𝑎𝑥𝑣𝑎𝑙 = 𝐴[𝑖]
5 Return 𝑚𝑎𝑥𝑣𝑎𝑙
Example 2
• Problem: Check if all element in an array are

distinct
Example 2
• Problem: Check if all element in an array are

distinct
• Solution:
Function 𝑁𝑜𝐷𝑢𝑝𝑙𝑖𝑐𝑎𝑡𝑒𝑠(𝐴, 𝑛)
2 For 𝑗 = 𝑖 + 1 to 𝑛
3 If 𝐴 𝑖 == 𝐴[𝑗]
4 Return 𝐹𝑎𝑙𝑠𝑒
5 Return 𝑇𝑟𝑢𝑒
Example 3
• Problem: Matrix multiplication
Example 3
• Problem: Matrix multiplication

• Solution:
Function 𝑀𝑎𝑡𝑟𝑖𝑥𝑀𝑢𝑙𝑡𝑖𝑝𝑙𝑦(𝐴, 𝐵)
1.For 𝑖 = 1 to 𝑛
2. For 𝑗 = 1 to 𝑛
3. 𝐶[𝑖][𝑗] = 0
4. For 𝑘 = 1 to 𝑛
5. 𝐶 𝑖 𝑗 = 𝑐 𝑖 𝑗 + 𝐴 𝑖 [𝑘] × 𝐵[𝑘][𝑗]
6.Return 𝐶
Example 4
• Problem: Towers of Hanoi
Example 4
• Problem: Towers of Hanoi

• Three pegs, 𝐴, 𝐵 and 𝐶
• Move 𝑛 disks from 𝐴 to 𝐵
• Never put a larger disk above a smaller one
• 𝐶 is transit peg
Example 4 …
• Recursive solution
Example 4 …
• Move 𝑛 − 1 disks from 𝐴 to 𝐶, using 𝐵 as transit peg
• Move largest disk from 𝐴 to 𝐵
• Move 𝑛 − 1 disks from 𝐶 to 𝐵, using 𝐴 as transit peg
Example 4 …
• Solve recurrence by repeated substitution
• 𝑀 𝑛 : Number of moves to transfer 𝑛 disks
Example 4 …
• 𝑀 𝑛 = 𝑀 𝑛 − 1 + 1 + 𝑀(𝑛 − 1)
Example 4 …
• 𝑀 𝑛 = 𝑀 𝑛 − 1 + 1 + 𝑀(𝑛 − 1)
• 𝑀 1 =1
Example 4 …
• 𝑀 𝑛 = 𝑀 𝑛 − 1 + 1 + 𝑀(𝑛 − 1)
• 𝑀 1 =1
• Answer ?
Example 4 …
• 𝑀 𝑛 = 𝑀 𝑛 − 1 + 1 + 𝑀(𝑛 − 1)
• 𝑀 1 =1
• Answer ⇒ 𝑴 𝒏 = 𝟐𝒏 − 𝟏

(Jignesh Joshi) 1 - Introduction PDF

Încărcat de

Informații document

Titlu original

Drepturi de autor

Formate disponibile

Partajați acest document

Partajați sau inserați document

Opțiuni de partajare

Vi se pare util acest document?

Este necorespunzător acest conținut?

Drepturi de autor:

Formate disponibile

(Jignesh Joshi) 1 - Introduction PDF

Încărcat de

Drepturi de autor:

Formate disponibile

CO-303

DESIGN AND ANALYSIS OF

• What is the cost of execution?

• What is the cost of execution?

(𝒏+𝟏) 𝒏+𝟐 𝒏 𝒏+𝟏

(𝒏+𝟏) 𝒏+𝟐 𝒏 𝒏+𝟏

• What is the cost of execution?

(𝒏+𝟏) 𝒏+𝟐 𝒏 𝒏+𝟏

(𝒏+𝟏) 𝒏+𝟐 𝒏 𝒏+𝟏

• How do we analyze the time complexity?

• How do we analyze the time complexity?

• How do we analyze the time complexity?

𝐶1 𝑛 + 𝐶2 𝑛 − 1 + 𝐶3 𝑛 − 1 + 𝐶4 𝑡𝑗 + 𝐶5 (𝑡𝑗 −1) + 𝐶6 (𝑡𝑗 −1) + 𝐶7 (𝑛 − 1)

𝑛 𝑛+1 𝑛 𝑛−1 𝑛 𝑛−1

𝑛 𝑛+1 𝑛 𝑛−1 𝑛 𝑛−1

• Which algorithm of the two do you think is better?

• Which algorithm of the two do you think is better?

• Such analysis is based on the asymptotic growth rate,

• Such analysis is based on the asymptotic growth rate,

Figure: Big-oh notation: 𝑡 𝑛 𝜖 𝑂(𝑔 𝑛 )

• The big-O notation

Figure: Big-omega notation: 𝑡 𝑛 𝜖 Ω(𝑔 𝑛 )

Figure: Big-theta notation: 𝑡 𝑛 𝜖 𝜃(𝑔 𝑛 )

• Problem: Maximum value in an array

• Problem: Maximum value in an array

• Problem: Check if all element in an array are

• Problem: Check if all element in an array are

• Problem: Matrix multiplication

• Problem: Matrix multiplication

• Problem: Towers of Hanoi

• Problem: Towers of Hanoi

S-ar putea să vă placă și