Divide and conquer algorithms

Analysis and Design of Algorithms
Chapter 4 Divide And Conquer
Divide-and-Conquer
Chapter 4 Objective
Divide-and-Conquer is probably well known algorithm design technique. It is well suited for parallel computations, in which one problem can be solved simultaneously by its own processor.
Divide and Conquer

The most well known algorithm design strategy: 1. Divide instance of problem into two or more smaller instances 2. Solve smaller instances recursively 3. Obtain solution to original (larger) instance by combining these solutions
Prepared By Manjunath Kammar
Page 1
As shown in the figure the case of dividing a problem into two sub problems, by far the most far widely occurring case These are designed in such away that these can run in single processor. Appling this method for sum of n natural numbers we can solve this by Dividing the array of n numbers into two n/2 sub array. Recursively divide the array two equal parts We know that when there is only one number the sum is the number only. Then add their value to get the sum in equations.
n/2 -1
a0+..+an-1=( a0++a
) +( a
n/2
+..+an-1)
In this problem the input of size n can be divided into several instances of size n/b, with a of them needing to be solved. (Here, a and b are constants; a1 and b>1). Assuming that size n is a power of b, to simplify our analysis, we get the following recurrence for the running time t(n): T(n)= aT(n/b)+f(n). Where f(n) is the time spent in dividing the problem into smaller ones and then combining their solutions. T(n) depends on the values of the constants a and b and the order of growth of the function f(n). Divide and Conquer Examples Sorting: mergesort and quicksort Tree traversals Binary search Matrix multiplication-Strassens algorithm
Page 2
Master Theorem Master Theorem Master Theorem
Master Theorem
IF f (n) (nd) where d0 in the general divide and conquer recurrence T(n) = aT(n/b) + f (n) with a1 and b>1 then,
Ex: Consider the problem of computing the sum of n numbers recursively.

+
a0+a1+a2+. +an-1 = ( a0++.an/2 ) A(n) = 2A(n/2)+1 A=2, b=2 and d=0 a > bd
Therefore
(an/2+. +an-1 )
A(n) (nlogb a) = (n)
Mergesort
Merge sort is a perfect example of a successful application of the divide-and-conquer technique. It sorts a given array A [0.n-1] by dividing it into two halves A[0... n/2 -1] and A[ n/2.n-1].
Sort each of them recursively. Merging the smaller sorted arrays into a single sorted one.
Page 3
Merge
This can be done as follows:
Two pointers (array indices) are initialized to point to the first elements of the arrays being merged. Then the value of the index compared and smaller of them is added to new array being constructed. Now the index of the smaller element array is incremented to point to its immediate successor in the array it was copied from. This operation is continued until one of the two given arrays is exhausted. The elements of the remaining array will be copied to the end of the new array.
Page 4
Merge sort Example
Efficiency of merge sort

Prepared By Manjunath Kammar Page 5
Assuming n is a power of 2
For the worst Case
This is because after every key comparission the total element to be processed will be reduced by 1. As this is the worst case the smallest number may come from the alternating arrays. So total comparission is n-1.
Cworst (n) = ( n log n)
Quick sort (Partition exchange Sort)

It was invented by a British scientist C.A.R Hoare. Here the divide and conquer technique is used in the manner:
Divide:
Divide the array A[0] A[1]A[n-1] into two Sub arrays. A[k] A[k+1] A[k+2].. A[n1]
Right Part
A[0] A[1]. A[k-1]

Left part Pivot
We continue the same procedure for the sub arrays also.
Conquer: Sort the left part of the Array.

Sort the right part of the array. Then the resulting array will be sorted. Merge sort : Input array is divided based on position.
Page 6
Quick Sort : Input array is divided based on their value. Method :
Partition the given array of size l to r using the pivot element say a[s] such that all the elements towards the left of pivot are <= pivot and all the elements towards the right of pivot are >= pivot. Now we have 2 sub arrays of size l-(s-1) and (s+1)-r Repeat the process for sub arrays.
Left to right scan
: if A[i]<=p i=i+1
It starts with the second element. it skips all those elements which are smaller than the pivot and stops when found the bigger or equal element. Right to left scan : if A[j]>=p j=j-1
It starts with the last element of the sub array. It skips all those elements which are bigger smaller than the pivot and stops when found the smaller or equal element. Three cases may arise depending on whether or not the indices are crossed over or not.
if(i<j) i.e. i and j have not crossed.

We simply exchange a[i] and a[j] and continue scanning by incrementing i and decrementing j.
i
P all are p all are p p
j
.. p
if(i>j) ) i.e. i and j have crossed over.

Then we have partitioned the array after exchanging the pivot with the A[j]. j
P all are p all are p p
i
p
if(i==j)
element.
i.e. when the scanning indices stops while pointing to the same
Here the value they are pointing to must be equal to p. i=j

P all are p are p =p all
The partition algorithm

ALGORITHM Partition(A[l.r])
// Partitions a sub array by using its first element as a pivot. // Input : A Sub array A[l.r] of A[0....n-1], defined by its left and right indices l and r (l<r). // Output : A partition of A[lr], with the split position returned as this functions value. p A[l] i l ; j r+1 while (true) repeat i i+1 until A[i] p repeat j i-1 until A[i] p if(i<j) then Swap (A[i],A[j]) else
Swap (A[i],A[j]) return j
Quicksort Example
Efficiency of quicksort
Best case: The number of key comparisons in the best case satisfy the recurrence.
cbest (n) will
cbest (n)=2 cbest (n/2)+n for n>1 cbest (1)=0 cbest (n)=2 cbest (n/2)+n
2[2C(n/22)+ (n/2)]+n =>22 C(n/22 )+2n[by replacing n by n/2]
Page 9
Analysis and Design of Algorithms C(n/23)+ 22 (n/22) 2k C(n/2k)+ k.n
=>23 C(n/23 )+3n [by replacing n by n/2]
2k C(n/n)+k.n [putting n=2k then k= log2n]
Therefore
cbest (n) = n log2 n
Worst case: sorted array ( n2)

This worst case is because of the array in sorted order. In this we will get the one of the partitioned array as empty, and the size of another is one less than the array which is partitioned from. In this case of the sorted array in ascending order where A [0] is the pivot then the left to right to scan will stop ay A [1]. Right to scan will stop at A [0]. So we make n+1 comparisons to get to this partition and exchanging with the pivot element A[0] with itself.[this is because the array alady sorted]. This sorting arrays of diminishing sizes will continue until the last one A[n-2..n-1]has been processed.
cworst (n)=(n+1)+n+.3
This is because for n elements n+1 comparisons. So for 2 elements 3 comparisons. When one element remaining then no need to be sorted. So
cworst (n)=((n+1)+n+.3+(2+1)) 3
(n+1)(n+2)/2 -3

(n2)
Average case: random arrays ( n log2 n) But in the average case sorting needs 38% more comparisons.
Page 10
The recurrence relation is given by

n-2
cavg (n)=1/n
(n+1)+ cavg (s) + cavg (n-1-s)] for n>1

i=0
cavg (0)=0, cavg (1)=0. [where s is a partition 0 s n-1]
cavg (n)=1.38 (nlog2n)
Binary Search Defn:

A binary search is a simple technique which can be applied if the items to be compared are either in ascending or descending order. Very efficient algorithm for searching in sorted array K
A[0] . . . A[0..m-1] A[m] A[m+1] A[n-1]

K<A[m ] K>A[m ]
If K = A[m], stop (successful search); otherwise, continue searching by the same method in A[0..m-1] if K < A[m] and in A[m+1..n-1] if K > A[m] To search an element 83
1 8
Page 11
23
Iteratio n1 Iteratio n2 Iteratio n3
35
45
54
58 m
65
75
83
95 r
l r
m l,m
Analysis of binary search:

To analyze the efficiency of binary search we count the number of key comparisons made. In the best case: Cbest(n) = (1). In the worst case: Cw (n) = 1 + Cw( n/2 ), Cw (1) = 1 solution: Cw(n) = log 2n + 1
Binary Tree Traversals And Related Properties. Defn: A binary tree is defined as a set of nodes that is either empty or consists of a root and two disjoint binary trees TL and TR called, respectively the left and right sub tree of the root.
As the binary tree is divided into two parts many problems can be solved using divide-conquer technique. Height of a tree: It is defined as the length of the longest path from the root to a leaf. Height of an empty tree is -1. Height of a tree with one node is 0. Height of a tree with two nodes is 1. The recursive algorithm is Algorithm Height(T) //computes recursively the height of a binary tree. //input: A binary tree T //output: The height of T. if T return -1. else return max{height(TL), height (TR)} + 1
TL
TR
A Standard representation of a binary tree. The number of comparisons made to compute the maximum of two numbers and the number of additions A(n(T)) made by the algorithm are the me. The recurrence relation is given for A(n(T))
A(n(T))= A(n/2(TL))+ A(n/2(TR))+1 for n(T)>0 A(n(T))=0 n(T)=0 A(n(T))= 2 A(n/2(T))+1 Applying master theorem, A=2, b=2, d=0 a>bd

i.e.
n n (n)
log a b log 2 2
Tracing:
8 Height =2
0
-1
-1 0
-1
-1
-1
-1
Number of comparisons is more than number of additions. When T= we are not performing any addition, we do one comparison Therefore it is necessary to calculate number of comparisons along with additions. n n=0 n=1 Additions 0 1 Comparisons 1 3
Page 14
n=2
Thus we replace the empty sub trees by special nodes. These nodes are called as external. The original nodes are called as internal. So by definition we can say that a empty binary tree is a single external node.
Internal
External
Numbers of external nodes are always one greater than the number of internal nodes. Number of internal nodes gives number of additions. Sum of internal and external nodes gives the number of comparisons to be made. Therefore A(n(T))= n A(c(T))= n+n+1= 2n+1. Prove that number of external nodes is always one more than the number of internal nodes. Basis of induction: When n=1 we know that the number of internal nodes is 1 and number of external nodes are 2.
Therefore the given statement is true for n=1. Assume the given statement is true for n=k such that n=k. X
Extern al
= k
interna l
Let T be the extended binary tree with n internal nodes and x external nodes. Let nL and xL be the internal and external nodes of the left sub tree of T. Similarly nR and xR be the internal and external nodes of the right sub tree of T. Since n>0 the binary tree has a root , which is an internal node. Number of internal nodes n= nL+ nR +1 Number of external nodes = xL+ xR

(nL+1)+( nR +1) ( nL+ nR +1)+1 n+1
therefore number of additions required is n. number of comparison required is 2n+1. Algorithm for preorder , inorder and postorder traversals. Algorithm preorder(T) //purpose: this algorithm will traverse the binary tree in node[N],left[L], and right[R] order. // input: A binary tree. // output: An preorder traversed list. if(T=) return print (info(T)) //visit the node
Page 16
preorder(TL) preorder(TR)
//visit the left sub tree TL in preorder // visit the right sub tree TR in preorder
Algorithm inorder(T) //purpose: this algorithm will traverse the binary tree in left[L],node[N]and right[R] order. // input: A binary tree. // output: An inorder traversed list. if(T=) return inorder(TL) print (info(T)) inorder(TR) //visit the left sub tree TL in inorder //visit the node // visit the right sub tree TR in inorder
Algorithm postorder(T) //purpose: this algorithm will traverse the binary tree in left[L], right[R] and node[N] order. // input: A binary tree. // output: An postrder traversed list. if(T=) return postorder(TL) postorder(TR) print (info(T)) //visit the left sub tree TL in potsorder // visit the right sub tree TR in postorder //visit the node
Multiplication of Large Integers
Ex: 29 * 15 = 435 29 = 2*101 + 9*100 and 15 = 1*101 + 5*100 29 * 15 = (2*101 + 9*100 ) * (1*101 + 5*100)
Page 17
= (2*1)102 + (9*1 + 2*5 )101 + (9*5)100 = 200 + = 435 We can reduce 2 multiplication in the middle term with multiplication. (9*1 + 2*5 ) = (2+9)*(1+5) (2*1) (9*5) 190 + 45
We can obtain the following formula for any pair of two digit numbers a=a1a0 & b=b1b0 their product c can be computed by the following formula. C = a*b = c2102 + c1101 +c0 C2 = a1*b1 product of their first digits C0 = a0*b0 product of their second digits C1 = (a1+a0) * (b1+b0) - (c2+c1)

For 2 n-digit integers where n is positive even number. a=a1a0 b=b1b0
First half of a g1 First half of b gb1 a Second half of a ga0 Second half of b gb0 b=b1b0 g b=b110n/2 + b0
Page 18
a=a1a0 g a=a110n/2 + a0
C=a*b
= (a110n/2 + a0)*( b=b110n/2 + b0) = (a1*b1)10n +(a1*b0+a0*b1)10n/2+(a0*b0) = c210n + c110n/2 +c0
Where C2 = a1*b1 product of their first digits C0 = a0*b0 product of their second digits C1 = (a1+a0) * (b1+b0) - (c2+c1)
Analysis:
Multiplication of n-digit numbers requires three multiplications of n/2-digit numbers, the recurrence for the number of multiplications M (n) will be M (n)=3 M (n/2) for n>1 M (1)=1. Consider M (n)=3 M (n/2) M (n)=32 M (n/22) putting n=n/2 M (n)=3k M (n/2k) substituting 2k=n M (n)=3k M (n)=3log2n M (n)=n log23 n1.585 <n2
Strassens matrix multiplication

m1 = (a00 + a11) * (b00 + b11) m2 = (a10 + a11) * b00 m3 = a00 * (b01 - b11)
Page 19
m4 = a11 * (b10 - b00) m5 = (a00 + a01) * b11 m6 = (a10 - a00) * (b00 + b01) m7 = (a01 - a11) * (b10 + b11)
Strassens matrix multiplication

c00 c01 = c10 c11 a10 a11 a00 a01 * b10 b11 b00 b01
m1 + m4 - m5 + m7 = m2 + m4
m3 + m5
m1 + m3 - m2 + m6
Page 20
Efficiency of Strassens algorithm

If M(n) is the number of multiplications made by in multiplying 2 n-by-n matrices,
It is smaller than n3 required by the brute force algorithm. Since this saving in the number of multiplications was achieved at the expense of making extra additions., we must check the number of additions A(n) made by strassens algorithm.
Questions:
1. Write an algorithm to mergesort using divide and conquer strategy. Trace the algorithm for input set {4,7,1,3,8,5}. (July-2006)(10 marks) 2. Discuss strassens matrix multiplication algorithm. Find its time complexity. (July-2006)(10 marks). 3. Give the algorithm for merge sort and trace its operation on the following sequence of numbers. (Dec 2005) (10 Marks)
Page 21
92376481 4. Describe strassens matrix multiplication algorithm and evaluate the asymptotic efficiency. (Jan-Feb 2005) (10 marks). 5. Write a pseudo code for a merge sort algorithm. Set up and solve a recurrence relation for the number of key comparisons made by your algorithm. (Jan-Feb 2005) (10 marks) 6. Write quick sort algorithm and apply it to sort list E, X, A, M, P, 7. L, E (July Aug 2004) (10 Marks). 8. Briefly explain a method to multiply 2 large numbers based on the divide and conquer method. Hence compute the 1234 * 2101 using the same. (July Aug 2004) (10 Marks).
9. Explain Merge sort with its complexities.
10. 11. 12. 13. 14.
Explain Quick sort with its complexities. Explain Binary Search with its complexities. Explain Binary Tree Traversals and Related Properties. Explain Multiplication of Large Integers with its complexities. Explain Matrix Multiplication with its complexities
Page 22

Divide and conquer algorithms

Încărcat de

Informații document

Descriere originală:

Titlu original

Drepturi de autor

Formate disponibile

Partajați acest document

Partajați sau inserați document

Opțiuni de partajare

Vi se pare util acest document?

Este necorespunzător acest conținut?

Drepturi de autor:

Formate disponibile

Divide and conquer algorithms

Încărcat de

Drepturi de autor:

Formate disponibile

Analysis and Design of Algorithms

Chapter 4 Divide And Conquer