Lecture 04 (Math Preliminaries)

Design & Analysis of Algorithms
Anwar Ghani
Semester: Spring 2013
Department of Computer Science & Software Engineering, International Islamic University, Islamabad.
Mathematical Preliminaries
Mathematical Preliminaries
Topics Set Theory Combinatorics Mathematical Functions Summations Probability Theory
Mathematical Preliminaries/IIU 2008/Dr.A.Sattar/2
Set Theory
Set Theory
Definitions
A Set is a collection of objects called members. If a, b, c, d are members of a set S we write : S ={a, b, c, d} where a, b, c, d
S ( read a, b, c, d belong to S )
The set builder notation can be used to define a set. For example, the set S={1, 3, 5, 7, 11, 13} can be defined as S = {x: where x is prime number and x<15} In set notation the order of members does not matter S = {a, b, c, d} is identical to set {d, b, c, a} In a set all members are distinct objects , i. e, duplicate members are not allowed A set that contains no elements is called empty set. It is symbolically denoted by
Set Theory
Subsets
A set A is called a subset of set B if every element of A is also a member B. Symbolically: A
B,
if x
then x
The set A is proper subset of B, if A contains only some of the elements of B. Symbolically the relationship is represented as: A
The relationship among sets is usually shown by a picture, which is called Venn diagram. Figures (a), (b) show Venn diagrams for a subset and proper subset B A=B A A
(a) A is proper subset of B
(b) A is subset of B
Set Theory
Universal Set
The largest set for a given collection of objects is called Universal Set. All other sets are subsets of the Universal set U. Symbolically:
U,
U, B U, C
Example: The Universal set, U, of all small English alphabets is U = {a, b, c,..,x, y, z} The subsets of the Universal set are, for example, A = {a, f, g}, B = {p, q, r, s , t}, C = {x ,y, w}, as shown in the diagram U A B C
Subsets of Universal set of alphabets (a, b, c,x, y ,z}
Given a Universal set, we define complement Ac of set A as Ac = U A The set Ac contains all of elements of Universal set, excluding those in the set A. Example: If A={a, b, c}, then Ac = { d, e, f,, x, y, z}
Set Operations
Union
The union of set A and set B is set of all elements x such that x is in A or x is in B. Symbolically: A U B, if x A or x B
The Union operation is pictorially represented by Venn diagram, as shown below:
B Union set
A U B
Example (1): A={ 1, 3, 4, 5}, B={ 3, 4, 7, 8, 9}. The Union of sets A and B is the set { 1, 3, 4, 5, 7,8, 9}. Example(2): Union operation can be used to add elements to a set. Consider sets {a, b}, {c, d} (i) Setting S to empty set S= (ii) To insert elements of first set into S, we perform union of S with {a, b} S = S U {a, b} = {a, b} (iii) To insert elements of second set {b, c} into S, we perform union of S with {c, d} S = S U {c, d} = {a, b, c, d}
Set Operations
Intersection
The intersection of set A and set B is set of all elements x, such that x is in A and x is in B. Symbolically: A
B,
if x
and
The intersection operation is pictorially represented by Venn Diagram, as shown below.
A I B
Set intersection
Example: Let A={1,2, 3, 4, 5} and B={2, 4, 7, 8} then A I B = {2, 4} Two sets are called disjoint, if their intersection is empty set. Thus, if A and B are disjoint, A I B=
Set Operations
Difference Set
then x The difference of set A and set B is set of all elements x, such that x is not in B if x is in A. Symbolically: B A , if x
A
B
The difference operation is illustrated by Venn Diagram, as shown below.
Difference set
B-A
Example(1): If A={1,2,3,4,5} and B={4, 6, 8, 10} then A B = {1, 2, 3, 5} Example(2): We can use difference operation to delete elements from a set. Consider S = {a, b, c, d, e, f } (i) To delete element a we perform difference operation on S and set {a}. S = S {a} = { b, c, d, e, f } (ii) To delete element d we perform difference operation on set S and set {d}. S = S-{d} = { b, c, e, f }
Set Theory
Cardinality
The cardinality of a set is the number of elements in the set. It is also called size. The cardinality of set S is denoted by |S| . Example: If S = {a, b, c, d, e}, then | S | = 5 | A UB | = | A | + | B | - | A B | Consider, for example, the sets A, B Let A = {a, b, c, d, e, f}, |A|=6 For any sets A and B, Let B ={d, e, f, g, h, i, j ,k}, |B|=8
A U B = {a, b, c, d, e, f, g, h, i, j, k}, | A U B | = 11 A B = {d, e, f}, |AB|=3
It follows that | A | + | B | - | A B | = 6 + 8 3 = 11
Set Theory
Cartesian Product
The Cartesian product of two sets A and B, denoted by A X B, is a set of all ordered pairs such that the first element of the pair is in set A and the second element is in set B. It is also called cross product. Mathematically, A X B = { (a , b) : a
and
b B }
Example: Let A = {a, b, c} and B = {g, h}, then A X B = {(a, g), (a, h), (b, g), (b, h), (c, g), (c, h) } The cardinality of Cartesian product is the product of cardinalities of sets | A X B | = | A | .| B |
Set Theory
Binary Relation
A binary relation R on sets A and B is a subset of the Cartesian product A X B of the sets . In set notation, the relation is represented as: R
AXB
Since a cross product set can have one or more subsets there can be many binary relations Example: Consider the sets A = {a, b, c}, B = {x, y} Their cross product is set A X B = {(a, x), (a, y), (b, x), (b, y), (c, x), (c, y) } (1) A binary relation R1 is R1 = { (a, y), (b, y), (c, y) } (2) Another binary relation R2 is R2 = {(a, x), (a, y) }
Binary Relation
Example
A graph G is a pair (V, E ), where V is a finite set, called vertex set, and E is a binary relation on V, called edge set Example: Consider the graph G = (V, E) , and V = { a, b, c }, vertex set for the graph A binary relation is E = {(a, a), (a, b), (a, c), (b, c), (c, b) } The set E represents edges of graph G. The graph is shown pictorially in the diagram below.
Combinatorics
Permutations
Definition
A permutation is an arrangement of n objects in some order, such that each object appears exactly once. Elements of a set of n objects can be arranged in the following ways n.(n-1).(n-2)..3.2.1 because, the first place can be filled in n ways, and the second place in n-1 ways. Thus, first and second places together can be filled in n(n-1) ways. The first, second and third places can be filled in n.(n-1).(n-2) ways, and so on The product n(n-1)(n-2).3.2.1 is called n-factorial, which is denoted as n! Example: The elements of Set S={ a, b, c } can be permuted in six ways because 3! = 3.2.1 = 6 The six arrangements are: abc, acb, bac, bca, cab, cba It is assumed that 0! = 1 (factorial of zero is one ). The factorial of a negative integer is not defined
Factorial Function
Stirlings Approximation
The n! factorial increases rapidly with the increase of n. The table below shows the growth. n
2 4 8 16 32 2 24 40,320 20,922,789,888,000 2.6 x 10 35
n!
It can be shown that 1000! is an integer consisting of 2,500 digits. The analysis of algorithms sometimes involves large inputs. For large values of n, the n! is approximated by the following formula,:
n! ( 2 n ) ( n / e) n , where e = 2.718 The above formula is referred to as Stirlings approximation

Combinations
Definition
The arrangement of n objects taken k ( k n ) at a time, without regard to the order, is called k-combination. Example: The objects a, b, c, d have following 2-combinations: ab, ac, ad, bc, bd, cd Observe that the combination ab ba i.e, order does not matter The number of k-combinations of n objects is denoted by the symbol Another notation is C(n ,k). It can be shown that :
C(n,k) =
The k-combinations are called binomial coefficients because they occur in binomial theorem for the expansion of the expression:
Mathematical Functions
Use in Analysis
Several mathematical functions are used to analyze algorithms and study their behavior. We briefly examine the properties of some these functions From analysis perspective, the following functions are particularly useful:
i. ii. y=x y=x
(Floor function) function )
(Ceil
iii. y = log a x (Logarithm function) iv. y = x (Linear function ) v. y = x2 (Quadratic function)
vi. y = x3 (Cubic function) vii. y = 2x (Exponential function with base 2) viii. y =nn (Exponential)
When a mathematical function is used to represent the running time of an algorithm, the function is referred to as growth function. The behavior (growth rate) of a function can be represented pictorially by plotting a graph of (x,y) points given by the functional relation
Floor Function
Definition
The floor function y(x) is defined as: y= x where x is a real number and y is the largest integer which is smaller than or equal to number x. Example: 4.9= 4,
4.0 = 4
It follows from the definition :
x -1 < x x
The floor function is used for mapping of a real valued function into integer function
Ceil Function
Definition
The ceil function y(x) is defined as: y= x where x is real number and y is the smallest integer larger than or equal to x. Example: 4.9 = 5
4.0 = 4
It follows from the definition :
x x < x +1
The ceil function is used for mapping a real valued functions into integer function
Logarithm Function
The logarithm function y= log a x is the inverse of exponential function ay, where a is some positive constant. Thus, if y = log a x , then x=ay Example: Consider, y= log 5 25 . Since 25= 52, it follows y=2 . The figure shows a plot of logarithm function to the base 2
Definition
The logarithm function will frequently arise in the analysis of algorithms

Logarithm Function
Bases
The choice of base depends on the nature of application and data type. The base 10 is normally used in computations involving decimal numbers. Here are some examples log 10 1 =0 because 1= 10 0 by definition log 10 10 =1 10= 10 1 log 10 100 = 2 100=10 2 log 10 1000000 = 6 1000000=10 6 The base 2 is often used in the analysis of algorithms. Some examples of base 2 are because 1= 20 by definition log 2 1=0 log 2 2= 1 2=21 log 2 16= 4 16= 24 log 2 64= 6 64= 26 The logarithm to base 2 are referred to as binary logarithm. A special notation lg is used to represent a binary algorithm, as follows lg (n) log 2 (n) The base e 2.718 is used in calculus for purposes of differentiation and integration It is called natural logarithm. A special notation ln is used to represent natural logarithms. ln (n) log e (n)
Logarithm Function
Properties
The following properties of logarithms are often used in several applications log b x . y = logb x + logb y logb x / y = logb x logb y logb x n = n logb x x logb y = y logb x (Sum rule) (Difference rule) (Exponentiation rule) ( Symmetry rule )
The last property is particularly useful for interchanging logarithmic exponential and base of exponentiation. Here are some examples.
2log 2 n = n log 2 2 = n 8log 2 n = n log 2 8 = n3 3log 2 n = n log 2 3 = n1.585
(Interchanging 2 and n) (Interchanging 8 and n)
( Interchanging 3 and n and using lg 3 = 1.585)

Logarithm Function
Change of Base
Sometimes it is necessary to change the base of a logarithm. The formula for the transformation from base a to b is as follows:
loga x = logb x / logb a

Example : lg (binary logarithm) can be converted to ln (natural logarithm): lg x = log2 x = log e x / loge2 = ln x / ln 2 ( ln 2 = 0.6931)
Growth Rates
The table shows the growth rate of common mathematical functions, which are useful in the analysis of algorithms
n
2 4 8 16 32 64 128
lg n
1 2 3 4 5 6 7
n
1.4 2 2.8 4 5.7 8 11
n lg n
2 8 24 64 160 384 896 4 16 64
n2
8 64 512
n3
4 16 256
2n
2 24 40,320
n!
2 256
nn
16,777,216 1.845 x 1019 1.461 x 10 48 3.940 x 10 115 5.283 x 10 269
256 1,024 4,096 16,384
4,096 32,768 262,144 2,097,152
65,536 4,294,967,296 1.8 x 10 19 3.4 x 10 38
20,922,789,888,000 2.631 x 10 35 1.269 x 10 89 3.856 x 10

215
The logarithm lg n has the lowest growth rate , and the exponential function n n the highest The relationship of functions in terms of their growth rates is symbolically represented as lg n < n < n < n lgn < n2 < n3 < 2n < n! < n n
Growth Functions
Graph
The plot of mathematical functions lg n, n, n lg n n2, n3, 2n is shown below.
f(n)
Because of extremely rapid growth of n! and nn , these functions are not shown in the ranges shown in the graph scale
Summations
Summation
Thenarithmetic summation consist of sum natural numbers (1, 2, 3.n)
Arithmetic
k
k =1
= 1 + 2 + 3+..+ n = n ( n+1 ) / 2
The sum can be simply evaluated, as under.

Let S be sum of first n terms of the series. Therefore, S = 1+ 2 + 3 + + (n-2)+(n-1)+ n Writing the summation in reverse order S = n + (n-1)+(n-2)++3 + 2 + 1
[ n terms] (1) [n terms] . .(2)
Adding (1) and (2) 2S = (n+1)+(n+1)+(n+1)+.(n+1) + (n+1) + (n+1) [ n terms] ...(3) = n(n+1) Thus, S = n ( n + 1) / 2
Another summation sometimes used in analysis is sum of squares of natural numbers. It has the sum: S= 12 + 22 + 32+ +n2 = n(n+1)(2n+1)/6
Summation
Geometric
The geometric summation consists of sum of powers of some fixed number. The fixed number is referred to as geometric ratio. Hers is a common form of exponential summation:
r
k =0
= r 0 + r 1+ r 2 + ..+ r where r is the geometric ratio.
n-1
+r
= ( rn+1 - 1) / (r - 1)
The summation is evaluated as follows. Let S be the sum of first n terms of the series. Then, S = r 0 + r 1 + r 2+r3 ..+ r n-1 + r n .(1) Multiplying both sides of (1) with r r .S = r 1 + r 2+r 3+ +rn-1 + rn +r n+1 ..(2) Subtracting (1) from (2) (r-1).S = rn+1-r0 = rn+1 -1 Or S = (rn+1-1) / ( r -1) (3)
Example(1) : Let r = 2, the sum of geometric series is as under 20 + 21+ 22+..+2n =2n+1 - 1 Example (2): Let r = 2/3, the sum is as under (2/3)0 + (2/3)1 + +(2/3)n = ( (2/3)n+1 -1)/( 2/3 - 1 ) = 3 3(2/3)n+1
Summation
Arithogeometric
Another useful series, referred to as arithogeometric . It consists of sum of the product of natural numbers with exponentials of a constant. :
km
k =0
k =n
= 1.m1 + 2.m2 + 3.m3+ .+ n.mn
where m is some constant The following summation of arithogeomatric series, with base 2, arises in the analysis of some algorithms.
k.2 = 1.21 + 2.22 + 3.23+ .+ n.2n = (n-1) 2n+1 + 2

k =0
Summation
Harmonic
The harmonic summation consists of reciprocals of natural numbers (1,2,..n).
1 / k
k =1
= 1/1 + 1/2 + 1/3 +.+1/n ln n + + 1 / 2n - 1 / 12n
=0.57721 is called Eulers constant The Harmonic series does not have an exact formula for the sum . The approximate sum is obtained by converting the summation into an integral By using mathematical analysis it can be shown that upper and lower bounds of harmonic series are given by the relation ln(n+1)
1 / k 1 + ln(n)
k =1
Summation
Logarithmic
The logarithmic summation includes logarithms of natural numbers (1,2,,n ). Assuming base 2 for the logarithm, the series is expressed as follows:
lg(k )
k =1
= lg(1) + lg(2) + lg(3)+..+lg(n) . = lg(1.2.3..n) =lg( n! )
An estimate for the above summation is obtained, by using the Stirling s approximation for large factorial. Since n! ( 2 n ) ( n / e) n where e = 2.718, it follows that lg n! n lg( n) ( ignoring lower order terms and constants) Therefore, the approximate summation for logarithm series for large n is given by : lg(1) + lg(2) + lg(3)+..+lg(n) n lg( n )
Probability Theory
Sample Space
Definition
The Probability Theory is concerned with the study of experiments whose outcome is not known in advance ( a priori). However, all likely outcomes are assumed to be finite and known before the conduct of the experiment For example, tossing of a coin is an experiment. The outcome of tossing cannot be predicted in advance, but outcome is likely be Head or Tail The set of all possible outcomes of an experiment are called sample space or population.. Let e1, e2, e3, en are the probable n outcomes of an experiment. Then the sample space is the set = { e1, e2, e3, en }
Sample Space
Examples
Example (1): A fair coin is tossed. The result can be Head( H ) or Tail( T ). The sample space for the tossing experiment is: = {H, T}. Example (2) : A fair coin is tossed twice ( or two coins are tossed simultaneously) the sample space for the toss is: = { HH, HT, TH, TT} Example (3): A six-faced die is thrown. The sample space for the event is = {1, 2, 3, 4, 5, 6 } . where digits represent number of dots on each face of the die. Example(4): Two six-faced dice are tossed. The sample space consists of 36 pairs of integers: ={(1,1),(1,2)(1,3),(1,4),(1,5), (1,6), (2,1),(2,2)(2,3,),(2,4),(2,5), (2,6), (3,1),(3,2)(3,3),(3,4),(3,5), (3,6), (4,1),(4,2)(4,3,),(4,4),(4,5), (4,6), (5,1),(5,2)(5,3),(5,4),(5,5), (5,6), (6,1),(6,2)(6,3),(6,4),(6,5), (6,6) } Example(5): The sample space for weather forecast can be = (fair, rainy, cloudy, partly cloudy, hailstorm, snow}
Events Space
Definition
A subset E of the sample space is called event space . Symbolically , E
A six-faced die is thrown. The sample space consists of all numbers that are likely to be shown i. e, ={1, 2, 3, 4, 5, 6 } Example(1): The event space that an even number is shown: E1={2, 4, 6} Example(2): The event space that an of odd number is shown: E2=(1, 3, 5} Example(3) : The event space that a number greater than 4 is shown E3={ 5, 6} An event set consisting of a single element is called elementary event. The empty set is referred to as impossible event
Probabilities
Let E be an event space consisting of events e1, e2, e3..en. Each of the events in the is assigned a real value (number) denoted by P(e1), P(e2) , P(e3)..P(en), called probabilities, such that P(e1) + P(e2) + P(e3)+.+ P(en) = 1 The function P(e) is called probability function. The probabilities are generally assigned on the basis of experience or judgment. Some times the probabilities are expressed as percentages, or in terms frequencies of happening of different events.
Example(1): Chance that a flight will be on time is 90% and would be late is 10% . Thus, P(on time)= 90/100=0.9 and P(late)=10/100=0.1 Example(2): Based on experience, we say that chance of finding a document on Yahoo is twice that of finding on MSN, and four times on Google. If p is probability of finding the document on MSN then, we have :P (msn ) = p ; P(yahoo ) = 2p ; P(google) ) = 4p By probability axiom , p + 2p + 4p =1 or p=1/7 Thus, we have P(msn)= 1 / 7, P(yahoo)= 2 / 7, and P(google)=4 / 7
Definition
Probability Theory
Equiprobable Events
In several cases the probabilities are not known, but it is expected that all events are equally likely to occur . Therefore, all the elementary events are assigned equal probabilities. Let E be event space for equiprobable events E ={e1, e2, .en} Then, P( e1 ) = P( e2 ) = P(e3 ) =.= P( en ) = p , say, where p is some constant Since by definition P(e1)+P(e2)+P(e3)++P(en)=1 it follows that p +p + p+..+ p ( n terms ) = 1. i.e n.p= 1 or p = 1/n Thus, for equiprobable n possible events, P(ej) = 1 / n ( 1 j n)
The function P(n) is called uniform probability function.

Equiprobable Events
Examples
Example (1) In tossing a fair coin heads and tails have equal chance. The event set is {H, T}., where H refers to occurrence of head and T to occurrence of tail. Therefore, P(H)=1/2; P(T)=1/2 Example (2) When six-faced die is thrown, all numbers, 1 to 6, have equal chance of being shown. Therefore, each event has probability 1/6. Thus, probabilities of showing different numbers are . P(1)=1/6, P(2)=1/6, P(3)=1/6, P(4)=1/6, P(5)=1/6, P(6)=1/6 Example(3): An array stores n random keys. Assuming that a search key has equal chance of being in any of the array cells, the probability of finding the key in a given cell is 1/n Example(4): A six-faced die is tossed. The event set that a odd number is shown consists of the elements 1, 3, 5. The probability that an odd number would be displayed is 3/6=1/2
Probability Theory
Law of Addition
Two events A, B are said to be mutually exclusive provided that if A occurs then B cannot occur; conversely, if B occurs then A cannot occur. For example, if a six-faced die is tossed then showing of 1 and 6 are mutually exclusive events because if 1 is shown then 6 will not be shown and vice versa. According to Law of Addition of Probabilities, If A and B are two mutually exclusive events with probabilities P(A) and P(B) ,then probability that A occurs OR B occurs is given by the sum of probabilities P(A) + P(B). Simply stated , P(A OR B)= P(A)+ P(B) In general, if A1, A2, A3Ak are mutually exclusive events then P(A1 OR A2 OR A3..OR Ak)=P(A1)+ P(A2)+P(A3)+.+P(Ak)
Law of Addition
Examples
Example (1) : A six-faced die is tossed. What is the probability that an even number would be show.? The likely even numbers are 2,4,6. Each has the probability of 1/6. Thus, according law of addition P( Even Number)= P(2 OR 4 OR 6)= 1/6 + 1/6+ 1/6 =1/2 Example(2): A die is thrown. What is probability of showing a prime number or a number greater than 5 ? (1) Events set that a prime number would be shown is A= {2,3,5}. Each event has the probability of 1/6. Since these are mutually exclusive events, probability that a prime number would be shown is given by P( Prime Number) =P( 2 OR 3 OR 5) = 1/6+ 1/6+ 1/6 =1/2 (2)Events set that a number than greater 5 would be shown is B={6} . The Probability that a number greater than 5 would be shown is P( Number greater than 5) =1/6 (3) Combining the above results, the probability that outcome would be a prime number or a number greater than 5 is P(Prime Number OR Number greater than 5)=P(Prime Number) + P(Number greater than 5) =1/2 + 1/6 = 2/3
Probability Theory
Law of Multiplication
Two events A , B are said to be stochastically independent if occurrence of A is not influenced by the occurrence of B. Conversely, occurrence of B is not influenced by occurrence of A. For example, if a coin is tossed several times the result of previous toss does affect the result of current toss, and result of current toss would not influence the result of next toss. We can say that the events of successive tossing are stochastically independent events. According to Law of Multiplication, if A and B are stochastically independent events then the probability of occurrence A AND occurrence of B is given by the product of the probabilities P(A).P(B). Mathematically, P( A AND B) = P(A).P(B) In general, if A1, A2, A3, .Ak are stochastically independent events then P( A1 AND A2 AND A3 AND Ak ) = P(A1) .P(A2 ).P(A3 ). P(Ak )
Law of Multiplication
Examples
Example (1): A coin is tossed three times. What is the probability of occurrence of three heads? The three tosses are independent events. In each of three tosses the probability of showing a head is 1/2. Thus, After the first toss, P( Head )= 1/2 After the second toss, P( Head AND Head)= P( Head )..P( Head ) =1/2 . 1/2= 1/4 After the third toss , P( Head AND Head AND Head) = 1/2 . 1/2 . 1/2 = 1/8 Example(2): A six faced die is tossed three times. What is probability that three sixes would be shown or three fives would be shown consecutively. In each toss the probability of showing 5 or 6 is 1/6. (1)According multiplication law, the probability of showing three consecutive sixes is P(6 AND 6 AND 6) =(1/6).(1/6).(1/6)= 1/216 (2) Again according to law of multiplication, the probability of showing three fives P(5 AND 5 AND 5) =(1/6) .(1/6). (1/6)=1/216 (3) According to law of addition , the probability that 6 is shown three times OR 5 is shown three times, is obtained by adding the above probabilities P( (6 AND 6 AND 6) OR ( 5 AND 5 AND 5 ) )= P ( 6 AND 6 AND 6) + P(5 AND 5 AND 5) =1/216 + 1/216 = 1/108
Probability Theory
Expected Cost
Let c1, c2, ..cn be the costs associated with events e1, e2, en . Let the probabilities for the occurrence of events be P(e1), P(e2)..P(en) , then the expected cost (EC) is defined as EC = c1.P( e1) + c2.P( e2 )+cn.P (en) Example: Suppose a coin is tossed 2 times. Assume that the costs associated with events are : if two heads appear cost is 2 if one head appears cost is 1 if no heads appear (equivalently only tails show) cost is 0 The event space S for the experiment is = { HH, HT, TH, TT} Probability that event HH occurs = 1/2.1/2= 1/4 Probability that event HT occurs =1/2.1/2=1/4 By law of multiplication Probability that event TH occurs = 1/2.1/2=1/4 Probability that event TT occurs =1/2.1/2=1/4 The four events have associated costs of 2, 1,1, 0 with probabilities 1/4, 1/4. 1/4, 1/4. Thus, the expected cost of the tossing experiment = 2.(1/4) + 1.(1/4) + 1.(1/4) + 0.(1/4) = 1/ 2 + 1/ 4 + 1/ 4 = 1
Searching an Array
Consider the problem of performing linear search on an array of size n . It is assumed that the search key has equal probability of being found in any of the n array cells. Thus, the probability that key is in any of the n cells is 1/n
Cell Probability
Expected Cost
A[1] 1/n
A[2] 1/n
A[3] 1/n
A[n-1] 1/n
A[n] 1/n
Assuming that unit cost of searching one cell of the array is c, the probabilities and the costs associated with search operation up to each cell are summarized in the table
Search Key location First cell Second cell Third cell nth cell Probability 1/n 1/n 1/n 1/n nc Cost of Searching c 2c 3c
Thus, expected cost of search is Texpected(n) =1c.1/n + 2.c.1/n+.+(n-1).c.1/n + n.c.1/n = [1+2+3+.+n ].c./ n = c. (n+1)/2 (Using arithmetic summation)

Lecture 04 (Math Preliminaries)

Încărcat de

Informații document

Titlu original

Drepturi de autor

Formate disponibile

Partajați acest document

Partajați sau inserați document

Opțiuni de partajare

Vi se pare util acest document?

Este necorespunzător acest conținut?

Drepturi de autor:

Formate disponibile

Lecture 04 (Math Preliminaries)

Încărcat de

Drepturi de autor:

Formate disponibile

Design & Analysis of Algorithms

Semester: Spring 2013

Mathematical Preliminaries/IIU 2008/Dr.A.Sattar/2

Mathematical Preliminaries/IIU 2008/Dr.A.Sattar/3

Mathematical Preliminaries/IIU 2008/Dr.A.Sattar/4

(a) A is proper subset of B

Subsets of Universal set of alphabets (a, b, c,x, y ,z}

The Union operation is pictorially represented by Venn diagram, as shown below:

The intersection operation is pictorially represented by Venn Diagram, as shown below.

The difference operation is illustrated by Venn Diagram, as shown below.

A U B = {a, b, c, d, e, f, g, h, i, j, k}, | A U B | = 11 A B = {d, e, f}, |AB|=3

Mathematical Preliminaries/IIU 2008/Dr.A.Sattar/10

Mathematical Preliminaries/IIU 2008/Dr.A.Sattar/11

Mathematical Preliminaries/IIU 2008/Dr.A.Sattar/13

Mathematical Preliminaries/IIU 2008/Dr.A.Sattar/14

Mathematical Preliminaries/IIU 2008/Dr.A.Sattar/15

n! ( 2 n ) ( n / e) n , where e = 2.718 The above formula is referred to as Stirlings approximation

Mathematical Preliminaries/IIU 2008/Dr.A.Sattar/17

Mathematical Preliminaries/IIU 2008/Dr.A.Sattar/18

(Floor function) function )

iii. y = log a x (Logarithm function) iv. y = x (Linear function ) v. y = x2 (Quadratic function)

Mathematical Preliminaries/IIU 2008/Dr.A.Sattar/20

It follows from the definition :

Mathematical Preliminaries/IIU 2008/Dr.A.Sattar/21

The logarithm function will frequently arise in the analysis of algorithms

2log 2 n = n log 2 2 = n 8log 2 n = n log 2 8 = n3 3log 2 n = n log 2 3 = n1.585

(Interchanging 2 and n) (Interchanging 8 and n)

( Interchanging 3 and n and using lg 3 = 1.585)

loga x = logb x / logb a

Mathematical Preliminaries/IIU 2008/Dr.A.Sattar/25

16,777,216 1.845 x 1019 1.461 x 10 48 3.940 x 10 115 5.283 x 10 269

256 1,024 4,096 16,384

4,096 32,768 262,144 2,097,152

65,536 4,294,967,296 1.8 x 10 19 3.4 x 10 38

20,922,789,888,000 2.631 x 10 35 1.269 x 10 89 3.856 x 10

Mathematical Preliminaries/IIU 2008/Dr.A.Sattar/28

The sum can be simply evaluated, as under.

[ n terms] (1) [n terms] . .(2)

= r 0 + r 1+ r 2 + ..+ r where r is the geometric ratio.

= 1.m1 + 2.m2 + 3.m3+ .+ n.mn

k.2 = 1.21 + 2.22 + 3.23+ .+ n.2n = (n-1) 2n+1 + 2

Mathematical Preliminaries/IIU 2008/Dr.A.Sattar/31

= 1/1 + 1/2 + 1/3 +.+1/n ln n + + 1 / 2n - 1 / 12n

Mathematical Preliminaries/IIU 2008/Dr.A.Sattar/32

= lg(1) + lg(2) + lg(3)+..+lg(n) . = lg(1.2.3..n) =lg( n! )

Mathematical Preliminaries/IIU 2008/Dr.A.Sattar/33

Mathematical Preliminaries/IIU 2008/Dr.A.Sattar/34

Mathematical Preliminaries/IIU 2008/Dr.A.Sattar/35

The function P(n) is called uniform probability function.

Mathematical Preliminaries/IIU 2008/Dr.A.Sattar/40

Mathematical Preliminaries/IIU 2008/Dr.A.Sattar/41

Mathematical Preliminaries/IIU 2008/Dr.A.Sattar/43

S-ar putea să vă placă și