Probability Theory and

Random Processes
Title : Probability Theory and Random Processes
Course Code : 07B41MA106 (3-1-0), 4 Credits
Pre-requisite : Nil
Objectives :
To study
● Probability: its applications in studying the outcomes of random experiments
● Random variables: types, characteristics, modeling random data
● Stochastic systems: their reliability
● Random Processes: types, properties and characteristics with special
reference to signal processing and trunking theory.

Learning Outcomes :students will be able to

(i) model real life random processes using appropriate statistical distributions;
(ii) compute the reliability of different stochastic systems;
(iii) apply the knowledge of random processes in signal processing and trunking
theory.
Evaluation Scheme

Evaluation Components Weightage ( in percent)

Teacher Assessment 25
(based on assignments, quizzes,attendence etc.)
T1 (1 hour) 20
T2 (1hour 15 min) 25
T3 (1 hour 30 min) 30
----------------------------------------------------------------------
Total 100
Reference Material :
1. T. Veerarajan. Probability, Statistics and Random
processes. Tata McGraw-Hill.
2. J. I. Aunon & V. Chandrasekhar. Introduction to Probability
and Random Processes. McGraw-Hill International Ed.
3. A. Papoulis & S. U. Pillai. Probability, Random Variables
and Stochastic Processes. Tata WcGraw-Hill.
4. Stark, H. and Woods, J.M. Probability and Random
Processes with Applications to Signal Processing.
.
 Origins of Probability

The study of probabilities

originally
came from gambling!
 Why are Probabilities
Important?
• They help you to make good decisions, e.g.,
– Decision theory
• They help you to minimize risk, e.g.,
– Insurance
• They are used in average-case
time complexity analyses of
- Computer algorithms.
• They are used to model processes
in
- Engineering.
 Random Experiments
• An experiment whose outcome or result can
be predicted with certainty is called a
deterministic experiment.
•Although all possible outcomes of an experiment
may be known in advance, the outcome of a
particular performance of the experiment cannot be
predicted owing to a number of unknown causes.
Such an experiment is called a random experiment.
•A random experiment is an experiment that can be
repeated over and over, giving different results.
•e.g A fair 6-faced cubic die, the no. of telephone
calls received in a board in a 5-min. interval.
Probability theory is a study of random or
unpredictable experiments and is helpful
in investigating the important features of
these random experiments.
 Probability Definitions
• For discrete math, we focus on the
discrete version of probabilities.
• For each random experiment, there is
assumed to be a finite set of discrete
possible results, called outcomes. Each
time the experiment is run, one outcome
occurs. The set of all possible outcomes
is called the sample space.
Example.

If the experiment consists of flipping two

coins, then the sample space is:

S = {(H, H), (H, T), (T, H), (T, T)}

 Example.

If the experiment consists of tossing two

dice, then the sample space is:

S = {(i, j) | i, j = 1, 2, 3, 4, 5, 6}
 More Probability Definitions
• A subset (say E) of the sample space is
called an event. In other words, events
are sets of outcomes.

( If the outcome of the experiment is

contained in E, then we say E has
occurred.)
For each event, we assign a number between 0
and 1, which is the probability that the event
occurs.
Example.

If the experiment consists of flipping two

coins, and E is the event that a head
appears on the first coin, then E is:

E = {(H, H), (H, T)}

Example.

If the experiment consists of tossing two

dice, and E is the event that the sum of the
two dice equals 7, then E is:

E = {(1, 6), (2, 5), (3, 4), (4, 3), (5, 2), (6, 1)}
Union: EF

Intersection:E  F also denoted as EF

(If E  F = , then E and F are said to be

mutual exclusive.)
 NOTE
It may or may not be true that all
outcomes are equally likely. If they
are, then we assume their
probabilities are the same.
 Definition of Probability in
Text
The probability of an event E is the sum of the
probabilities of the outcomes in E

p(E) = n(E)/n(S)

number of cases favourable to E


exhasustiv e number of cases in S
 Example

• Rolling a die is a random experiment.

• The outcomes are: 1, 2, 3, 4, 5, and 6,
presumably each having an equal
probability of occurrence (1/6).
• One event is “odd numbers”, which
consists of outcomes 1, 3, and 5. The
probability of this event is 1/6 + 1/6 +
1/6 = 3/6 = 0.5.
 Example
• An urn contains 4 green balls and 6 red balls.
What is the probability that a ball chosen from
the urn will be green?

• There are 10 possible outcomes, and all are

assumed to be equally likely. Of these, 4 of them
yield a green ball. So probability is 4/10 = 0.4.
 Example
• What is the probability that a person wins the
lottery by picking the correct 6 lucky numbers
out of 40? It is assumed that every number has
the same probability of being picked (equally
likely).

Using combinatorics, recall that the total

number of ways we can choose 6 numbers out
of 40 is:
C(40,6) = 40! / (34! 6!) = 3,838,380.
Therefore, the probability is 1/3,838,380.
 Examples
• Consider an experiment in which a coin is tossed
twice.
• Sample space: { HH, HT, TH, TT }
• Let E be the event that at least one head shows up
on the two tosses. Then E = { HH, HT, TH }
• Let F be the event that heads occurs on the first
toss. Then F = { HH, HT }
• A natural assumption is that all four possible events
in the sample space are equally likely, i.e., each
has probability ¼.
Then the P(E) = ¾ and P(F) = ½.
 Probability as a Frequency
Let a random experiment be repeated n times and
let an event A occur n Atimes out of the n trials.
nA
The ratio is called the relative frequency of
n
the event A
nA
As n increases, shows a tendency to stabilise and to
n
approach a constant v alue. This value, denoted by P(A),
nA
is called the probabilit y of the event A, i.e., P(A)  lim n  .
n
 Frequency Definition of
Probability
• Consider probability as a measure of the
frequency of occurrence.
– For example, the probability of “heads” in a
coin flip is essentially equal to the number of
heads observed in T trials, divided by T, as T
approaches infinity.

number of heads
Pr( heads )  lim T 
T
 Probability as a Frequency
• Consider a random experiment with possible
outcomes w1, w2, …,wn. For example, we roll a
die and the possible outcomes are 1,2,3,4,5,6
corresponding to the side that turns up. Or we
toss a coin with possible outcomes H (heads) or
T (tails).
• We assign a probability p(wj) to each possible
outcome wj in such a way that:
p(w1) + p(w2) +… + p(wn) = 1
• For the dice, each outcome has probability 1/6.
For the coin, each outcome has probability ½.
Example
To find the probability that a spare part produced
by a machine is defective.

If , out of 10,000 items produced , 500 are defective,

it is assumed that the probability of a defective item
is 0.05
 Axioms of Probability

A B
Axioms (where A and B are events):
• 0 <= P(A) <= 1 
AB
• P(S) = 1; P({}) = 0 B
• P(A  B) = P(A) + P(B) – P(A  B)
A

• If A and B are disjoint then P(A  B) = A B

P(A) + P(B) (mutually exclusive events)
A
•
A
 Example
• Rolling a die is a random experiment.
• The outcomes are: 1, 2, 3, 4, 5, and 6. Suppose
the die is “loaded” so that 3 appears twice as often
as every other number. All other numbers are
equally likely. Then to figure out the probabilities,
we need to solve:
p(1) + p(2) + p(3) + p(4) + p(5) + p(6) = 1 and p(3) = 2*p(1)
and p(1) = p(2) = p(4) = p(5) = p(6).
Solving, we get p(1) = p(2) = p(4) = p(5) = p(6) = 1/7 and p(3)
= 2/7.
• One event is “odd numbers”, which consists of
outcomes 1, 3, and 5. The probability of this event
is:
p(odd) = p(1) + p(3) + p(5) = 1/7 + 2/7 + 1/7 = 4/7.
Theorem 1
The probabilit y of the impossible event is zero,
i.e. , if  is the subset (event) containing no sample
point, P(  )  0.
Pr oof
The certain event S and the impossible event
 are mutually exclusive.
Hence P(S   )  P(S)  P(  )
 But S    S.
 P(S)  P(S)  P(  )
 P(  )  0
 Theorem
If A and B are any 2 events,
P(A  B)  P(A)  P(B) - P(A  B)
 P(A)  P(B)
Pr oof :
A is the union of the mutually exclusive events
AB and AB and B is the union of the mutually
exclusive events AB and AB.
 P( A )  P( AB )  P( AB )
& P(B )  P( AB )  P( AB )

S
AB AB AB

A B
 P( A )  P( B )

 P( AB)  P( AB )  P( AB)  P( AB ) 
 P( A  B )  P( A  B )

Hence proved.
S
AB AB AB

A B
 Conditional Probability

If event F occurs, what is the probability

that event E also occurs?
This probability is called conditional
probability and denoted as p(E|F).

Definition of Conditional Probability

If p(F) > 0, then
p E  F 
p E | F  
p F 
Example.

An urn contains 8 red balls and 4 white

balls. We draw 2 balls from the urn
without replacement. What is the
probability that both balls are red?
Solution:
Let E be the event that both balls drawn are
red. Then
p(E) = C(8, 2)/C(12, 2)

or, we can solve the problem using conditional

probability approach,

Let E1 and E2 denote, respectively, the events

that the first and second balls drawn are red.
Then
p(E1E2) = p(E1) p(E2 | E1 ) = (8/12) (7/11)
 Multiplication Rule

pE1E2 E3 ...En 

 pE1  pE2 | E1  pE3 | E1E2 ... pEn | E1E2 ...En1 

Example.

A deck of 52 playing cards is randomly

divided into 4 piles of 13 cards each.
Compute the probability that each pile
has exactly one ace.
Solution:

Define events

E1 = the first pile has exactly one ace

E2 = the second pile has exactly one ace

E3 = the third pile has exactly one ace

E4 = the fourth pile has exactly one ace

p(E1E2E3E4) = p(E1)p(E2| E1)p(E3|E1E2)p(E4|E1E2E3)

p(E1) = C(4,1)C(48,12)/C(52,13)

p(E2|E1) = C(3,1)C(36,12)/C(39,13)

p(E3|E1E2) = C(2,1)C(24,12)/C(26,13)

p(E4|E1E2 E3) = C(1,1)C(12,12)/C(13,13)

p(E1E2E3E4)  0.1055
Let E and F be events. We can express E as

E = EF  EFc

Where Fc is the complementary event of F.

Therefore, we have

p(E) = p(EF) +p(EFc) = p(E|F)p(F) +p(E|Fc)p(Fc)

 Independent Events

Two events E and F are independent if

p(EF)=p(E)p(F)

Two events are not independent are said to be

dependent.
p(EF) = p(E)p(F) if and only if p(E|F) = p(E).

If E and F are independent, then so are E and

Fc.
Theorem
If the events A and B are independen t , the events
A & B are also independen ts.
Pr oof
The events A  B & A  B are mutually exclusive
such that P(A  B)  P(A  B)  P(B)( by addition t heorem)

 P(A  B)  P(B)  P(A  B)

 P(B)  P(A)P(B)( by product th eorem)
 P(B)1  P(A)  P(A)P(B)
Three events E, F, and G are independent if

p(EFG) = p(E)p(F)p(G)
p(EF) = p(E)p(F)
p(EG) = p(E)p(G)
p(FG) = p(F)p(G)
Example.

Two fair dice are thrown. Let E denote

the event that the sum of the dice is 7.
Let F denote the event that the first
die is 4 and let G be the event that the
second die is 3. Is E independent of
F? Is E independent of G? Is E
independent of FG?
p(E) = 6/36 = 1/6

p(F) = 1/6

p(G) = 1/6

p(EF) = 1/36

 E and F are independent.

p(EG) = 1/36

 E and G are independent.

p(FG) = 1/36

 F and G are independent.

p(EFG) = 1/36

 but, E and FG are NOT independent.

Baye' s Theorem
Let B1 , B2 ,.......Bk be a partition of the sample space S.
Let P(A/B i ) & P(Bi / A) be the conditiona l probabilit ies
for i  1 to k, then
P ( A / Bi ) P ( Bi )
P(Bi / A)  k i  1,2,....., k
 P( A / B j ) P( B j )
j1

Pr oof
Given B1 , B2 ,....Bk be a partition of the sample space S.
i.e., (i) Bi  B j  i  j.
S
B1
B2
B4
B3 A
 (ii )  Bi  S(iii )P(Bi )  0i.
k
i 1
Let A be the event associated with S. Then
(A  B1 )  (A  B2 )  .......  (A  Bk )
 A  (B1  B2 )  (A  B3 )  .......  (A  Bk )
(by distributi ve law)
 A  (B1  B2  ....  Bk )  A  S  A
Also all the events (A  B1 ), (A  B2 ),......., (A  Bk )
are pairwise mutually exclusive. For,
(A  B1 )  (A  B2 )  .......  (A  Bk )  A  (Bi  B j ), i  j
 A  
Then P(A)  P(A  B1 )  P(A  B2 )  ...
 P( A  B k )
However each term P(A  B j ) may be expressed
as P(A/B j )P(B j ) & hence we obtain the theorem
on total probabilit y.
P(A)  P(A / B1 )P(B1 )  P(A / B2 )P(B2 )  ...
 P( A / B k ) P( Bk )
k
  P( A / B j ) P( B j )
j1
P(Bi  A)  P(Bi )  P(A / Bi )  P(A)  P(Bi / A)
P(Bi )  P(A / Bi )
 P(Bi / A) 
P( A)

P(Bi )  P(A / Bi )
 k
 P( B j )  P( A / B j )
j1
Example.

In answering a question on a multiple-choice

test, a student either knows the answer or
guesses. Let p be the probability that the
student knows the answer and 1p the
probability that the student guesses.
Assume that a student who guesses at the
answer will be correct with probability 1/m,
where m is the number of multiple-choice
alternatives. What is the (conditional)
probability that a student knew the answer to a
question, given that his answer is correct?
Solution:

C = the student answers the question correctly,

K = the student actually knows the answer

pC | K  pK 
p K | C  
pC | K  pK   pC |~ K  p~ K 
p

p  1 p 1 / m 
mp

1  m  1 p
Example.

When coin A is flipped it comes up heads

with probability ¼, whereas when coin B
is flipped it comes up heads with probability ¾.
Suppose that one coin is randomly chosen and
is flipped twice. If both flips land heads, what is
the probability that coin B was the one chosen?
Solution:

C = coin B is chosen
H = both flips show head
pH | C  pC 
pC | H  
pH | C  pC   pH |~ C  p~ C 
 3 3 1
   
  4 4  2
 3 3 1 1 1 1
        
 4 4  2  4 4  2
9
  0.9
10
Example.
A laboratory test is 95 percent correct in detecting a certain
disease when the disease is actually present. However, the test
also yields a “false” result for 1 percent of the healthy people
tested. If 0.5 percent of the population has the disease, what is
the probability a person has the disease given that his test
result is positive?

Example.
A suspect is believed 60 percent guilty. Suppose now a new
piece of evidence shows the criminal is left-handed. If 20
percent of the population is left-handed,and it turns out that the
suspect is also left-handed, then does this change the guilty
probability of the suspect? By how much?
Solution:

D = the person has the disease

E = the test result is positive

p E | D  p D 
p D | E  
pE | D  pD   pE |~ D  p~ D 


0.950.005
0.950.005  0.010.995
95

294
 0.323
Example.
A suspect is believed 60 percent guilty. Suppose now a new
piece of evidence shows the criminal is left-handed. If 20
percent of the population is left-handed,and it turns out that
the suspect is also left-handed, then does this change the
guilty probability of the suspect? By how much?
Solution:

G = the suspect is guilty

LH = the suspect is left-handed

pLH | G  pG 
pG | LH  
pLH | G  pG   pLH |~ G  p~ G 


1.00.6
1.00.6  0.20.4
60

68

 0.88
Random Variable
Definition:
A random variable (RV) is a function X : S  R
that assigns a real number X(s) to every element
s  S,where S is the sample space corresponding to
a random experiment E.

Discrete Random Variable

If X is a random variable (RV) which can take a finite
number or countably infinite number of values, X is
called a discrete RV.
Eg. 1. The number shown when a die is thrown
2. The number of alpha particles emitted by a
radioactive source are discrete RVs.
Example
Suppose that we toss two coins and consider the sample
Space associated with this experiment. Then
S={HH,HT,TH,TT}.
Define the random variable X as follows:
X is the number of heads obtained in the two tosses.
Hence X(HH) = 2, X(HT) = 1 = X(TH) & X(TT) = 0.

Note that to every s in S there corresponds exactly

one value X(s). Different values of x may lead to
the same value of S.

Eg. X(HT) = X(TH)

Probability Function
If X is a discrete RV which can take the values
x1 , x 2 , x 3 ,.... such that

P(X  x i )  pi , then pi is called the probabilit y

function or probabilit y mass function or point
probabilit y function, provided pi (i  1,2,3,...)
satisfy the following conditions:

(i)p i  0, i,&
(ii ) p i  1
i
 Example of a Discrete PDF
• Suppose that 10% of all households have
no children, 30% have one child, 40%
have two children, and 20% have three
children.
• Select a household at random and let X =
number of children.
• What is the pmf of X?
 Example of a Discrete PDF
• We may list each value.
– P(X = 0) = 0.10
– P(X = 1) = 0.30
– P(X = 2) = 0.40
– P(X = 3) = 0.20
 Example of a Discrete PDF
• Or we may present it as a chart.

x P(X = x)
0 0.10
1 0.30
2 0.40
3 0.20
 Example of a Discrete PDF
• Or we may present it as a stick graph.

P(X = x)

0.40

0.30

0.20

0.10

x
0 1 2 3
 Example of a Discrete PDF
• Or we may present it as a histogram.

P(X = x)

0.40

0.30

0.20

0.10

x
0 1 2 3
Example
The pmf. of a random variable X is given by
ci
p(i)  i  0,1,2,....., where  is some positive
i!
value. Find (i) P(X  0)(ii)P(X  2)
Solution .
  
i
Since  p(i)  1, we have c   1
i 0 i 0 i!
 
i
As e  
 
, we have ce  1.
i 0 i!
- 0
e  
Hence P(X  0)  e
0!

P(X  2)  1  P(X  2)
   e 
2 
 1  e  e  
 2 
 -
e  x
P(X  x) 
x!
If X represents the total number of heads obtained,
when a fair coin is tossed 5 times, find the
probability distribution of X.

X :0 1 2 3 4 5
1 5 10 10 5 1
P:
32 32 32 32 32 32
Continuous Random Variable
If X is an RV which can take all values (i.e., infinite
number of values ) in an interval, then X is called
a continuous RV.

0, x  0,

X( x )  x,0  x  1,
1, x  1

 Probability Density Function
If X is a continuous RV,then f is said to be the
probability density function (pdf) of X , if it satisfies the
following conditions:

(i)f ( x )  0, x  R x , and (ii )  f ( x )dx  1


(iii) For any a, b with -   a  b  ,

b
P(a  X  b)   f ( x )dx
a
 When X is a continuous RV

a
P(X  a )  P(a  X  a )   f ( x )dx  0
a

This means that it is almost impossible that a continuous

RV assumes a specific value.

Hence P(a  X  b)  P(a  X  b)  P(a  X  b)

 Probability Density Function
0.6 0 if x  0
0.5
f X ( x)   1  x / 
0.4  e if 0  x
f X (x ) 0.3 
0.2
0.1
0.0
-3 0 3 6 9 12 15
x
Example
Check whether t he function f(x)  4x ,0  x  1
3

is a probabilit y density function or not.

Solution . Clearly f(x)  0x  [0,1].

1 1
Also  f ( x )dx   4x dx  1.
3

0 0
A random variable X has the density function
1 / 4 - 2  x  2
f(x)  
0 elsewhere
Obtain (i) P(X  1)(ii)P( X  1)(iii )P(2X  3  5)

(¾,1/2,1/4)
Find the formula for the probability distribution
of the number of heads when a fair coin is
tossed 4 times.
 Cumulative Distribution Function (cdf)
If X is an RV, discrete or continuous , then P(X<=x)
is called the cumulative distribution function of X
or distribution function of X and denoted as F(x).

If X is discrete , F(x)   p j
j
X j x

X
If X is continuous , F(x)  P(-  X  x)   f(x)dx
-
 Probability Density Function
0.6
0.5 0 if x  0
f X ( x)   1  x / 
 e if 0  x
0.4
f X (x ) 0.3 
0.2
0.1
0.0
-3 0 3 6 9 12 15
x
 Cumulative Distribution Function
1.2
1.0
0.8
F X (x ) 0.6
0.4
0.2
0.0
-3 0 3 6 9 12 15
x
0 if x  0
FX ( x)    x /
1  e if 0  x
 Probbility Density Function
0.0016
0.0012
f X (x ) 0.0008
0.0004
0.0000
-100 0 100 200 300 400 500 600 700 800
0 if x  0 x
1
f X ( x)   if 0  x  u
u
0 if u  x

 Cumulative Distribution Function
1.0
0.8
0.6
F X (x )
0.4
0.2
0.0
-100 0 100 200 300 400 500 600 700 800
0 if x  0 x
x
FX ( x)   if 0  x  u
u
1 if u  x

If the probabilit y density of a random variable is
K(1 - x 2 ),0  x  1
given by f(x)  
0 otherwise
Find (i) K, (ii) the cumulative distributi on function of
the random variable.  3 / 2[x  x / 3] 0  x  1
3

0 otherwise

A random variable X has the density fuction

 1
K. if    x  
f(x)   1  x 2 K  1/ 

0 otherwise 1 / (tan x ),  x  

1
F( x )  
0, otherwise

Deter min e K and the distributi on function.

 Properties of the cdf F(x)
1.F(x) is a non - decreasing function of x, i.e. , if x1  x 2 ,
then F(x1 )  F(x 2 ) .

2.F()  0 & F()  1.

3. If X is a discrete RV taking values x1 , x 2 ,...., where

x1  x 2  x 3  ....  x i 1  x i  ....., then P(X  x i )  F( x i )  F( x i 1 ).

d
4.If X is a continuous RV, then F( x )  f ( x ), at all
dx
points where F(x) is differenti able.
Definition s
If X is a discrete RV, then the expected value or
the mean value of g(X) is defined as
E{g(X)}   g(x i )p i , where p i  P(X  x i )
i

the probabilit y mass function of X.

If X is a continuous RV with pdf f(x), then
E{g(X)}   g(x)f(x)dx
Rx
Two expected values which are most commonly
used for characteri sing a RV X are its mean  x & var iance  2x

 x  E(X)   x i p i , if X is discrete
i

  xf(x)dx, if X is continuous
Rx
2
x 
Var ( x )    E (X   x ) 2

  ( x i   x ) pi , if X is discrete
2
i

  (x -  x ) f ( x )dx, if X is continuous
2

Rx
The square root of variance is called the standard
deviation.
 Example
Find the expected value of the number on a die when
thrown.(7/2)

Var (X)  E(X )  {E(X)}

2 2

Var (X)  E{( X   x ) }  E{X  2 x X   x }

2 2 2

 E(X )  2 x E(X)   x (sin ce  x is a constant)

2 2

 E(X )   x (sin ce x  E(X))

2 2

Var (X)  E(X )  {E(X)}

2 2
Example
A random variable X has the following probabilit y distributi on
x : - 2 -1 0 1 2 3
p(x) : 0.1 K 0.2 2K 0.3 3K
(a) Find K, (b) Evaluate P(X  2) and P(-2  X  2), (c) find the cdf of
X and (d) evaluate the mean of X.

The probabilit y function of an infinite discrete

distributi on is given by P(X  j)  1/2 ( j  1,2,3,..).
j

Verify that the total probabilit y is 1and

find the mean and variance of the distributi on.
Find also P(X is even), P(X  5) & P(X is divisible by 3).
 xe x0
-x 2 / 2
If p(x)  
0 x0
(a )show that p(x) is a pdf(of a continuous RV X.)
(b) find its distributi on function P(x).
F(x)  1  e x / 2 , x  0
2
Example
A continuous random variable has the probabilit y
kxe -x x  0,   0
density function defined by f(x)  
0otherwise
Deter min e the cons tan t k and find mean and variance.
(k   ,2 / ,6 /  )
2 2
Moments
r
If X is the discrete or continuous RV, E ( X )is called rth
order raw moment of S about the origin and denoted
by 'r .
 x f ( x ) if X is discrete
r

x
 r '  E(X )   
r

  x f ( x )dx if X is continuous
r

-
Since the first and second moments about the origin are
given by 11  E(X) & 12  E(X 2 ),
mean  first moment about the origin
Var(X)    E(X)     
1
2
2 1
2  .
1 2
1
 E{( X   x ) n is called the nth order central moment
of X and denoted by  n .

E{ X } & E{ X   x } are called absolute moments of X.

n n

E{( X  a ) } & E{ X  a } are called generalise d moments of X.

n n

1  E(X  E(X))  E(X)  E(X)  0

 2  E(X  E(X))  Var (X) 2
 Two-Dimensional Random Variables
Definition s : Let S be the sample space associated with a random
experiment E. Let X  X(s) and Y  Y(s) be two functions each
assigning a real number to each outcomes s  S. Then (X, Y) is
called a two - dimensiona l random variable.

Two-dimensional continuous RV.

Two-dimensional discrete RV.

(X, Y)  (xi , yi ), i  1,2,3,...., m,......; j  1,2,3,..., n,.....

 Probability Function of (X,Y)

If (X, Y) is a two - dimensiona l discrete RV such that P(x  x i , y  y i )  p ij

then p ij is called the probabilit y mass function of (X, Y) provided
(i) p ij  0, i & j
(ii )  p ij  1
j i

The set of triplets{ x i , y i , p ij}, i  1,2,..., j  1,2,3...

is called the joint probabilit y distributi on of (X, Y)
 • If (X,Y) is a two-dimensional continuous RV.
The joint probability density function (pdf) f is
a function satisfying the following conditions
f ( x, y)  0,   x     y  
 
  f ( x, y)dxdy  1

Pr[ x1  X  x 2 , y1  Y  y 2 ]  x 2 y2
x1 y1 f ( x, y)dydx
 Cumulative Distribution Function

If (X, Y) is a two - dimensiona l RV(discret e or continuous ),

then F(x, y)  P{X  x & Y  y} is called the cdf of (X, Y)

F( x, y)    p ij
j i

 x y
  f (u, v)dvdu
 •Properties of joint PDF
0  F ( x, y )  1,    x     y  
F (, y )  F ( x,)  F (,)  0
F ( ,  )  1
F ( x, y ) is a nondecreas ing function as either x or y, or both, increase.
F (, y )  FY ( y ) F ( , x )  F X ( x )

F ( x, y )  Pr( X  x, Y  y )
 2 F ( x, y )
f ( x, y ) 
xy
 Examples
tossing two coins
X ... random variable associated with the first coin
Y ... random variable associated with the second coin
sample space {( 0,0), (0,1), (1,0), (1,1)} 0 for tail, 1 for head
F(x,y)

1
F(0,0) 
4
2
F(0,1) 
4
3
1/2 F(1,0) 
1/4 4
x
F(1,1)  1
1

y
Example
Three balls are drawn at random without replacement
from a box containing 2 white,3red and 4 black balls.
If X denotes the number of white balls drawn and Y
denotes the no of red balls drawn, find the joint
probability distribution of (X,Y).

Solutions
As there are only 2 white balls in the box, X can take
the values 0,1,2and Y can take the values 0,1,2,3.
P(X=0,Y=0)=P(drawing 3 balls none of which is
white or red)=P(all the 3 balls drawn are black)

 C3 / C3  1 / 21
4 9
P(X=0,Y=1)=3/14,P(X=0,Y=2)=1/7………

X Y

0 1 2 3

2
X Y

0 1 2 3

0 1/21 3/14 1/7 1/84

1 1/7 2/7 1/14 0

2 1/21 1/28 0 0
For the bivariate probabilit y distributi on of (X, Y)
given below, find P(X  1), P(X  1, Y  3), P(X  1/Y  3)
P(Y  3/X  1) & P(X  Y  4).

Y 1 2 3 4 5 6
X
0 0 0 1/32 2/32 2/32 3/32

1 1/16 1/16 1/8 1/8 1/8 1/8

2 1/32 1/32 1/64 1/64 0 2/64

(ans.7/8,9/32,18/32,9/28,13/32)
Example
The joint density function of (X, Y) is given by
2e e 0  x  ,0  y  
-x 2 y
f(x, y)  
0otherwise
Compute (i) P(X  1, Y  1), (ii)P(X  Y), (iii)P(X  a)
ans. e -1 (1  e  2 ),1 / 3,1  e a
 Marginal probability density function

For every fixed j

p(xj, y1) + p(xj, y2) + p(xj, y3) + … = p{X= xj} = f(xj)
and for every fixed k
p(x1, yk) + p(x2, yk) + p(x3, yk) + … = p{Y= yk} = f(yk)

The probability functions f(xj) and g(yk) are also

called marginal probability density functions.
 
f X (x)   f (x, y)dy, f Y ( y)   f (x, y)dx
As an illustrative example, consider a joint pdf of the
form
6
f ( x, y)  (1  x y) for 0  x  1,
2
0  y 1
5
0 elsewhere

-Integrating this wrt y alone and wrt x alone gives the

two marginal pdf
6 x2
f X ( x )  (1  ) 0  x 1
5 2
6 y
f Y ( y)  (1  ) 0  y 1
- 5 3
 Independent Random Variables

Two random variables X and Y with joint

probability density function f(x,y) and
marginal probability functions f X (x) and f Y ( y)
If
F(x,y) = F(x) G(y)
Or p(x,y) = f X ( x) f ( y)
Y

for all x, y, then X and Y are independent.

Example
A machine is used for a particular job in the forenoon
and for a different job in the afternoon. The joint
probability of (X,Y), where X and Y represent the
number of times the machine breaks down in the
forenoon and in the afternoon respectively, is given
in the following table. Examine if X and Y are
independent RVs.
X Y 0 1 2
0 0.1 0.04 0.06
1 0.2 0.08 0.12
2 0.2 0.08 0.12
 X Y 0 1 2
0 0.1 0.04 0.06
1 0.2 0.08 0.12
2 0.2 0.08 0.12
X & Y are independen t , if Pi*  P* j  Piji, j
P0*  f (0)  0.1  0.04  0.06  0.2; P1*  0.4; P2*  0.4

P*0  0.5; P*1  0.2; P*2  0.3

P*0  P0*  0.2  0.5  0.1  P00
P0*  P*1  0.2  0.2  0.04  P01
Hence the RVs X and Y are independent
The cumulative distributi on function of the
continuous random variable (X, Y) is given by
1 - e  e  e
-x -y -(x  y)
, x, y  0
F(x, y)  
0otherwise
Pr ove that X and Y are independen t.

2
 F( x, y) e ( x  y)
x, y  0
f ( x , y)  
xy 0otherwise
 e  x x  0 e  y y  0
f1 ( x )   f 2 ( y)  
0otherwise 0otherwise

0x  0 0 y  0
F1 ( x )   x
F2 ( y)   x
1  e x  0 1  e y  0

x y
F1 (x)F2 ( y)  (1  e )(1  e )  F(x, y)
Example
Let X and Y be the lifetimes of two electronic devices.
Suppose that thei r joint pdf is given by f(x, y)  e -(x  y) , x  0, y  0
then X and Y are independen t
Example
Suppose that f(x, y)  8xy,0  x  y  1.
Check the independen ce of X and Y.
Expectation of Product of random variables

If X and Y are mutually independent random

variables, then the expectation of their
product exists and is
E(XY) = E(X) E(Y)
 Example

Assuming that the lifetime X and the brightness Y of a

lightbulb are being modeled as continuous random variables
Let the pdf be given by f(x, y)  1 2e ( 1x  2 y ) ,0  x  ,0  y  

Find the joint distribution function

A line of length a units is divided into two parts. If the

first part is of length X, find E(X), Var(X) and E{X(a-X)}.
 Expectation of Sum of random variables

If X1, X2, …, Xn are random variables, then the

expectation of their sum exists and is

E(X1+ X2+…+ Xn) = E(X1) + E(X2) +… + E(Xn)

E  X   E Y    x j p x j , yk    yk p x j , yk 
j ,k j ,k

  x j  yk p x j , yk 
j ,k

 E X  Y 
Example
What is the mathematical expectation of the sum of
points on n dice?
Ans. (7/2)n

n n
A box contains 2 tickets among which Cr
tickets bear the number r (r = 0,1,2,…,n). A group of m
tickets is drawn . Let S denote the sum of their
numbers. Find E(S) and Var S.

Ans. (n/2)m
E  XY    x j yk px j , yk 
j ,k

  x j yk f x j g  yk 
j ,k

   x jf x j   y k gy k 

j  k 

 E X EY 
Example
If the joint pdf of (X, Y) is given by f(x, y)  24y(1 - x),0  y  x  1
Find E(XY)

y
11
E(XY )    xyf ( x, y)dxdy
0y

Ans. 4/15
0 x
 Binomial Distribution (re-visit)

Suppose that n Bernoulli trials, each of which results

in a success with probability p and results in a failure
with 1–p, are performed. If Sn represents the number
of successes that occur in the n Bernoulli trials, then
Sn is said to be a binomial random variable with
parameter n and p.
Let Xk be the number successes scored at the kth
trial. Since Xk assumes only the values 0 and 1 with
corresponding probabilities q and p, we have

E(Xk) = 0  q + 1  p = p
Since

Sn = X1 + X2+…+ Xn

We have

E(Sn) = E(X1+ X2+…+ Xn)

= E(X1) + E(X2) +… + E(Xn)
= np
 Conditional Probability

F x M   PrX  x M 
Prx  x , M 
 Pr( M )  0
Pr( M )

Now, consider the case where the event M depends

on some other random variable Y.
 Conditional Probability Density Function

f ( x, y) or f ( x , y)
f (y | x)  f ( x | y) 
fX ( x ) f Y ( y)
- the continuous version of Bayes’ theorem

f ( x | y ) fY ( y )
f ( y | x) 
f X ( x)
- another expression of the marginal pdf
 
f X ( x)   f ( x, y )dy   f ( x | y ) fY ( y )dy
 
 
fY ( y )   f ( x, y )dx   f ( y | x) f X ( x)dx
 
For the bivariate probabilit y distributi on of (X, Y)
given below, find P(X  1), P(X  1, Y  3), P(X  1/Y  3)
P(Y  3/X  1) & P(X  Y  4).
1 2 3 4 5 6
Y
X
0 0 0 1/32 2/32 2/32 3/32

1 1/16 1/16 1/8 1/8 1/8 1/8

2 1/32 1/32 1/64 1/64 0 2/64

(ans.7/8,9/32,18/32,9/28,13/32)
Suppose that p(x,y) the joint probability mass
function of X and Y , is given by p(0,0) =.4
p(0,1)=.2,p(1,0)=.1,p(1,1)=.3 Calculate the
conditional probability mass function of X given
that Y = 1
Ans. 2/5,3/5
Example
Suppose that 15 percent of the families in a certain
community have no children, 20% have 1, 35% have
2, & 30% have 3 children; suppose further that each
child is equally likely (and independently) to be a
boy or a girl. If a family is chosen at random from this
community, then B, the number of boys, and G , the
number of girls, in this family will have the joint
probability mass function .
 j 0 1 2 3
i
0 .15 .10 .0875 .0375 P(B  1, G  1)  P(BGorGB )
 P(BG )  P(GB)
1 .10 .175 .1125 0 1 1
 .35    2  .175
2 2
2 .0875 .1125 0 0

3 .0375 0 0 0

P(B  2, G  1)  P(BBGorBGBor GBB )

 1 1 1
 P(BBG )  P(BGB )  P(GBB )   .30      3
 2 2 2
.9
  .1125
8
If the family chosen has one girl, compute the
conditional probability mass function of the
number of boys in the family

Ans.8/31,14/31,9/31,0
Example
The joint density of X and Y is given by
12
 x 2  x  y ,0  x  1,0  y  1
f(x, y)   5
0otherwise
compute the conditiona l density of X , given that
Y  y , where 0  y  1

f ( x , y) 6x 2  x  y 
f ( x | y)  ans 
f Y ( y) 4  3y
Example
The joint pdf of a two - dimensiona l RV is given by
2
x
f(x, y)  xy  ,0  x  2,0  y  1.
2

8
1 1 1
Compute P(X  1), P(Y  ), P(X  1/Y  ), P(Y  / X  1)
2 2 2
P(X  Y) & P(X  Y  1)

(ans. 19/24,1/4,5/6,5/19,53/480,13/480)
Variance of a Sum of random variables

If X and Y are random variables, then the

variance of their sum is

Var(X + Y) = E({(X+Y) – (X + Y)}2)

 2
Var(X + Y) = E({(X+Y) – ( x+  y)} )

   
 E X   X   E Y   Y   2EX   X Y   Y 
2 2

 Var( X )  Var(Y )  2E(( X   X )Y  Y )

 Var X   VarY   2E XY    X Y 

The covariance of X and Y is defined by

Cov X , Y   E X   X Y  Y   E XY    X Y

• If X and Y are mutually independent, then
Cov(X,Y) = 0.

Q: Is the reverse of the above true?

• If X and Y are mutually independent, then

Var(X + Y) = Var(X) + Var(Y)

• If X1, …, Xn are mutually independent, and
Sn = X1 + …+ Xn, then

Var(Sn) = Var(X1) + … + Var(Xn)

Q: Let Sn be a binomial random

variable with parameter n and p.
Show that

Var(Sn) = np(1-p)
Example
Compute Var(X) when X represents the outcome
when we roll a fair die.

Solution
Since P(X=i)=1/6, i = 1,2,3,4,5,6, we obtain

6
E(X )   i P[X  i]
2 2
i 1
1 2 1 2 1 2 1 2 1 2 1
1 ( )  2 ( ) 3 ( )  4 ( ) 5 ( )  6 ( )
2

6 6 6 6 6 6
=91/6
Var (X)  E(X )  E(X)
2 2

2
91  7 
     35 / 12
6  2

Compute the variance of the sum obtained when 10

independent rolls of a fair die are made.

Ans 175/6

Compute the variance of the number of heads resulting

from 10 independent tosses of a fair coin.
The coefficien t of correlatio n between X and Y
denoted by  xy , is defined as
C xy
 xy 
xy
The correlation co-efficient is a measure of dependence
between RV’s X and Y.

If  xy = 0 , we say that X and Y are uncorrelated

If E(XY) = 0 , X and Y are said to be orthogonal RV’s.

 xy  1 or C xy   x  y
Example
Calculate the correlation coefficient for the following
heights (in inches) of fathers (X) & their sons (Y):

X Y
65 67 (Ans .603)
66 68
67 65
67 68
68 72
69 72
70 69
72 71
 C xy
 xy 
xy

 xy x y

 n n n
2 2
x y
2 2
x y
   
n  n  n  n 
 Example
Let the random variables X and Y have the
x  y 0  x  1,0  y  1
joint pdf f(x, y)  
0 elsewhere
Find the correlatio n coefficien t between X and Y.
(ans. - 1/11)

Example
If X,Y and Z are uncorrelated RVs with zero means
and standard deviations 5, 12 and 9 respectively
and if U=X+Y and V=Y+Z find the correlation
coefficient between U and V.
(ans 48/65)
 Conditiona l Expected Values
(X, Y) p ij
g ( X, Y )
E{g(X, Y) / Y  y j}   g(x i , y j )P(X  x i / Y  y j )
i

P{X  x i Y  y j} p ij
  g( x i , y j )   g( x i , y j )
i P{Y  y j} i p* j

E{g(X, Y) / Y}   g( x, y)f ( x / y)dx


E{g(X, Y) / X}   g( x, y)f ( y / x )dy

 Conditiona l means

 y/x  E(Y / X)   yf ( y / x )dy


Conditiona l variance are


 2
y/x  E(Y   y / x )   ( y   y / x ) f ( y / x )dy
2 2


 Example
The joint pdf of (X,Y) is given by f(x,y)=24xy, x>0,y>0,
x+y<=1,and 0, elsewhere, find the conditional
mean and variance of Y, given X.

f ( y / x )  2 y /(1  x ) , E(Y / X)  2 / 3(1  x),

2

var  1 / 18(1  x ) 2

If (X, Y) is uniformly distribute d over the semicircle

bounded by y  1 - x & y  0 , find E(X/Y) and E(Y/X)
2

Also verify th at E{E(X/Y)}  E(X) and E{E(Y/X)}  E(Y)

Properties
(1)If X and Y are independent RV’s, then E(Y/X)=E(Y)
and E(X/Y)=E(X).
(2)E[E{g(X,Y)/X)=E{g(X,Y)} in particular E{E(X/Y)}=E(X)
(3)E(XY)=E[X.E(Y/X)]

E(X 2 Y 2 )  E(X 2 E(Y 2 / X)]

Example
Three coins are tossed. Let X denote the number of
heads on the first two coins,Y denote the no of tails
on the last two, and z denote the number of heads
on the last two. Find
(a)The joint distribution of (i) X and Y (ii) X and Z
(b) Conditional distribution of Y given X = 1
(c) Find covariance of x,y and x,z
(d) Find E(Z/X=1)
(e)Give a joint distribution , that is not the joint
distribution of X and Z in (a), but has the same
marginals as of (b)
RVs (X1 , X 2 ,...X n )  independen t
f ( x1 , x 2 ,..., x n )  f ( x1 )  f ( x 2 )  ...  f ( x n )

conditiona l density
f(x 1 , x 2 , x 3 )
f(x 1 , x 2 / x 3 ) 
f( x 3 )
f(x 1 , x 2 , x 3 )
f(x 1 / x 2 , x 3 ) 
f(x 2 , x 3 )
Definition
Let X denote a random variable with probability
density function f(x) if continuous (probability
mass function p(x) if discrete)
Then
M(t) = the moment generating function of X

 
 E etX
  tx
  e f  x  dx if X is continuous
  
  etx p  x  if X is discrete
 x
Example
xe  x ( y 1)
x  0, y  0
If f ( x, y)  
0otherwise
Find the moment - generating function of XY.

 x ( y 1)
Solution : M XY    e xe xyt
dydx
00
 
x xyt  x ( y )
  xe {  e e dy}dx
0 0
 
x  xy (1 t )
  xe {  e dy}dx =1/1-t
0 0
 M X (t)   e f (x) tx
Properties
  (1  tx 
tx 
2
 .....)f ( x )
2!
MX(0) = 1
2
2tx
M 'X ( t )   ( x   ...)f ( x )
2!
M'X ( t ) t 0   xf ( x )  E(X)  1
3
6tx
M X ( t )   (x 
2 2
...)f ( x )
3!
M X ( t ) t 0   x f ( x )   2
2 2
 mX  0   k derivative of mX  t  at t  0.
k  th

2 3 k
mX  t   1  1t  t 
2
t 
3
 t 
k
.
2! 3! k!

 f  x  dx
 k
x X continuous
k  E  X k
 
  x p  x
k
 X discrete
Let X be a random variable with moment
generating function MX(t). Let Y = bX + a
Then MY(t) = MbX + a(t) = E(e [bX + a]t) = eatMX (bt)

Let X andY be two independent random

variables with moment generating
functionMX(t)and MY(t) .
Then MX+Y(t) = MX (t) MY (t)
6. Let X and Y be two random variables
with moment generating function MX(t)
and MY(t) and two distribution functions
FX(x) and FY(y) respectively.
Let MX (t) = MY (t) then FX(x) = FY(x).

This ensures that the distribution of a

random variable can be identified by its
moment generating function
Example
If X represents the outcome, when a fair die is tossed,
find the MGF of X and hence find E(X) and Var(X).
(Ans. 7/2,35/12)
If a RV X has the MGF M(t) = 3/(3-t), obtain the
standard deviation of X. (ans, 1/3)

1 t 2 2 t 3 3t 4 4 t
M( t )  e  e  e  e
10 10 10 10
Find p.d.f.
 1 t 2 2 t 3 3t 4 4 t
M( t )  e  e  e  e
10 10 10 10

M( t )   e f ( x )
tx
x

1 t 2 2 t 3 3t 4 4 t
e  e  e  e  f (a )e  f (b)e  ...
at bt

10 10 10 10

x
 , x  1,2,3,4
f ( x )  10
0, otherwise
 1
 2 ,1  x  
If X has the p.d.f f(x)   x
0, otherwise
find the mean of X.
The characteri stic function of a random variable X is defined by
X ( w )  E(eiwx )

 eiwxf ( x ),if X is discrete

x

  e f ( x )dx, if X is continuous
iwx

 
e iwx
 cos wx  i sin wx
1/ 2

 cos wx  sin wx
2 2

1/ 2

 1 1
 
x ( w )  E(e iwx
)   e f ( x)dx
iwx



 
  e iwx
f (x)dx   f (x)dx  1
 

Hence the characteristic function always exist even

when moment-generating function may not exist.
 Properties of Characteristic Function
i  n n
1.  E(X )  the co - efficient of
'
n
n
in the
n!
expansion of () in series of asending powers of i.

 X ( w )  E (e iwx
)   e f (x) iwx
x

  1  iwx 
iwx  iwx 
2

3

 ...f ( x )
x
 2! 3! 

  f ( x )  iw  xf ( x ) 
iw 
2

 x f ( x )  .....
2
x x 2! x
 1 d n

2.'n  n  n ()
i  d  0
3.If the characteri stic function of a RV X
is  x () and if Y  aX  b, then  y ()  eib x (a)
4. If X and Y are independen t RVs, then
 x  y ()   x ()   y ().
5. If the characteri stic function of a continuous RV X with
1  ix
density function f(x) is (), then f(x)   ( ) e d.
2  
6.If the density function of x is known , the
density function of Y  g(X) can be found from the
CF of Y, provided Y  g(X) is one - one.
The characteri stic function of a random variable X is
given by
1  w , w  1
x (w )  
0, w  1
Find the pdf of X.

The pdf of X is
1  iwx
f(x)    x ( w )e dw
2  
 1 1 iwx 
  (1  w )e dw
2 1 

1 0 iwx
1
iwx 
  (1  w )e dw   (1  w )e dw
2 1 0 

(2  e  e )  2 1  cos x 
1 ix 1
 ix

2x 2
x

1  sin x / 2
2

   ,   x  
2  x / 2 
Show that the distributi on for which the
-
characteri stic function is e has the density
1 1
function f(x)   ,  x  
 1 x 2

1  ix
f (x)   ()e d
2  
If X1 & X 2 are two independen t RVs that
follow Poisson distributi on with parameters
1 &  2 , prove that (X1  X 2 ) also follows a
Poisson distributi on with parameter (1   2 ).
Reproductive property of Poisson distribution

1 ( e i 1)
 x1 ( t )  e
 2 ( e i 1)
x 2 ( t )  e
since X1 & X 2 are independen t RVs,

1   2 ( e i 1)
 x1  x 2 ( t )  e
 Joint Characteristic Function
If (X, Y) is a two - dimensiona l RV, then
i1X  i2 Y
E(e ) is called the joint characteri stic
function of (X, Y) and denoted by xy (1 , 2 ).
 
i1x  i2 y
 xy (1 , 2 )    e f ( x, y)dxdy
 
i1x  i2 y
 e p( x i , y j )
i j

(i) xy (0,0)  1
1   mn

(ii )E{X Y }  m  n  m
m n
 xy (1 , 2 )
 1 2
n
i  1 0,2 0
(iii ) x ()   xy ( ,0) &  y ()   xy (0,  )
(iv )If X and Y are independen t
 xy (1 , 2 )   x (1 )   y (2 )
and conversely .
 Compute the characteri stic function of the
discrete r.v.' s X and Y if the joint PMF is
1 / 3, x  y  0
1 / 6, x  1, y  0

PXY 
1 / 6, x  y  1
0, else .
1 1 i ( w 1k  w 2 l )
 XY ( w1 , w 2 )    e PXY
k  1 l  1

1 iw1 0iw 2 0 1 iw1 1 iw1 iw 2 1 iw1 1 iw1 iw 2 

 e  e  e  e  e
3 6 6 6 6
  cos w1  i sin w1   cosw1  w 2   i sin w 1  w 2  
1 1 1
3 6 6
1
cos w1  i sin w1   cosw1  w 2   i sin w1  w 2 
1
6 6

  cos w1   cosw1  w 2 
1 1 1
3 3 3
 Example
Two RVs X and Y have the joint characteri stic
function  xy (1 , 2 )  e  2 8  2 2 . Show that
2
1

X and Y are both zero mean RVs and also that

they are uncorrelat ed
1   m n

(ii )E{X Y }  m  n 
m n
 xy (1 , 2 )
 1 2
m n
i  1 0,2 0
By the property of joint CF
1    212 82 2  
E(X)   e 
i  1    0 ,
1 2 0

e 
2 1
2
8 2
2
 4i 
1 1 0,2 0 0
E(Y)  e 2
 1
2
8 2
2
16i 
2 1 0,2 0 0
1    2 12 8 2 2  
2
E(XY)  2  e 
i  12  1 0,2 0
   2 12 82 2  
 e 162 
 1  1 0,2 0
 {64  e 2 1
2
8  2 2 }
1 2 1  0 ,2  0

0
C xy  E(XY )  E(X)  E(Y)  0
 Compute the joint characteristic function of X and Y if

1  1 2 2 
f xy  exp  ( x  y )
2  2 
Ans .
1 2 2
1    (x y )
i1x  i2 y
xy ( w1 , w 2 )   e 2
e dxdy
2    
Random Variable
 Binomial Distribution
The Bernoulli probability mass function is the density
function of a discrete variable X having 0 and 1 as
the only possible values

The pmf of X is given by P(0) = P{X = 0} = 1-p

P(1) = P{P = 1} = p
where p, 0<=p<=1 is the probability that the trial is a
success.

A random variable X is said to be Bernoulli random variable

if pmf satisfies above equation for some p  (0,1).
An experiment consists of performing a sequence of
subexperiments. If each subexperiment is
identical, then the subexperiments are called
trials.
 Bernoulli Trials
• Each trial of an experiment that has only two possible
outcomes (success or failure) is called a “Bernoulli trial.”
• If p is the probability of success, then (1-p) is the
probability of failure.
• The probability of exactly k successes in n independent
Bernoulli trials, with probability of success p and
probability of failure q = 1-p, is given by a formula called
the Binomial Distribution:
C(n, k) pk q(n-k)
 Example of Bernoulli Trials
• Suppose I roll a die and I consider a 3 to be
success and any other number to be a failure.
• What is the probability of getting exactly 5
successes if I roll the die 20 times?
• Solution: C(20, 5) (1/6)5 (5/6)15

• What is the probability of getting 5 or more

successes?
Theorem
If the probabilit y of occurrance of an event (probabili ty of success) in a
single trial of a Bernoulli' s experiment is p, then the probabilit y that the event
occurs exactly r times out of n independen t trials is equal to nC r q n  r p r , where
q  1 - p, the probabilit y of failure of the event.

Pr oof
Getting exactly r successes means getting r
successes and (n - r) failures simultaneo usly.
 P(getting r successes and n - r failures)
n r
p q r
 The trials, from which the successes are obtained
are not specified. There are nC r ways of choosing
r trials for successes. Once the r trials are chosen
for successes, the remaining (n - r) trials should
result in failures.
These nC r ways are mutually exclusive. In
each of these nC r ways, P(getting exactly
r successes )  p r q n  r .

, by the addition t heorem, the required probabilit y

n r
 nC r q r
p.
 Example
If war breaks out on the average once in 25 years,
find the probability that in 50 years at a strech,
there will be no war.
1 24
p  ,q  , n  50;
25 25
0 50 50
 1   24   24 
P  C(50,0)      
 25   25   25 
Example
Each of two persons A and B tosses 3 fair coins.
What is the probability that they obtain the same
number of heads?

P(A&B get the same no. of heads)

=P(they get no head each or 1 head each or 2 heads
each or 3 heads each)
= P(A gets 0 head) P(B gets o head)+------
3 2 3 2
 1   1 
 C(3,0)    C(3,1)  
  2     2  
3 2 3 2 =5\16
 1   1 
 C(3,2)    C(3,3)  
  2     2  
Example.

A game consists of 5 matches and two

players, A and B. Any player who firstly
wins 3 matches will be the winner of the
game.

If player A wins a match with probability

2/3. Suppose matches are independent.
What will be the probability for player A to
win the game?
Solution:

Player A wins 3 matches or more out of the 5

matches.

This event has probability equal to:

 5  2   1   5  2   1   5  2 
3 2 4 1 5


 3 3   3   
 4 3   3   
 5 3 
            

64

81
 Example.

A sequence of independent trials is to be

performed. Each trial results in a success with
probability p and a failure with probability 1–p.
What is the probability that

(a) at least 1 success occurs in the first n trials;

(b) exactly k success occurs in the first n trials;

Solution:
(a)The probability of no success in the
first n trials is (1-p)n. Thus, the answer
is 1–(1–p)n.

n k
  p 1  p 
(b) nk

k
 Assuming that p remains the same for all repetitions,
if we consider n independent repetitions (or trials) of
E and if the random variable X denotes the
number of times the event A has occurred, then
X is called a binomial random variable with
parameters n and p

The pmf of a binomial random variable having

parameters (n,p) is given by

P(i)  nC i pi q n i , i  0,1,......, n, where p  q  1

Example
It is known that car produced by an automobile company
will be defective with probability 0.01 independently of
each other. The company sells the cars in packages of
10 and offers a money-back guarantee that atmost 1 of
the10 cars is defective. What proportion of packages
sold must the company replace?

P(X  1)  1  {P(X  1)}  1  {P(X  0)  P(X  1)

 1  {10C0 (.01) (.99)  (10C1 (.01)(.99) }  .004
0 10 9
 n n
mean  E(X)   xnC x p (1  p) x
  xnC x p (1  p)
n x
x nx
x 0
x 1
n
n (n  1)! x
 x p (1  p) n x

x 1 x!(n  x )!
n
n (n  1)!
 x x 1
pp (1  p) n x

x 1 x ( x  1)!(n  1)  ( x  1)!
n
(n  1)!
 np  x 1
p (1  p) n x

x 1 ( x  1)!( n  1)  ( x  1) !
n
 np  n  1C x 1p x 1 (1  p) ( n 1) ( x 1)
x 1

 np (p  (1  p)) n 1
, sin cep  (1  p)   n C x p x (1  p) n  x
n n

x 0
 np
 n
Second moment 12  E(X 2 )   x 2 nC x p x (1  p) n  x
x 0

Show that variance is np(1-p)

Example
For a binomial distributi on with mean 6, standard deviation
2 , find the first two terms of the distributi on.
9
((1 / 3) ,2 / 2187)
The mean and variance of binomial distributi on
are 4 and 3 respective ly. Find P(X  1).

(1  (3 / 4) )  .9899
16
Poisson Distributi on
The probabilit y density function of the poisson va riate can
be obtained as a limiting case of the Binomial probabilit y
density function under the following assumption .
(i) The number of trials is increased indefinite ly (n  )
(ii) The probabilit y of success in a single trial is very small(p  0)
(iii) np is a finite constant say np  .
Cosider the probabilit y density function of a Binomial random
variable X as
n x n!
P(X  x)  nC x p (1  p) 
x
p x (1  p)n  x
x!(n  x )!
n x
n (n  1)...( n  x  1)      
x

   1  
x! n  n
  
n

1  
n (n  1)....( n  x  1) x  n 
 
 
x x
x!n
1  
 n
 
n
n (n  1) (n  x  1)
.... 1  
 n n n x  n
 
x
x!
1  
 n
For given x, as n  , the terms 1 - 1/n,1 - 2/n,...1 - x - 1/n,
(1 - /n) x tends to 1.
 
Also , Lt n  (1   / n )  e .
n

 
x
Hence , Lt n  P(X  x )  e .
x!
Poisson random variable
A random variable X taking on one of the values 0,1,2,...
is said to be a Poisson random variable with parameter 
e - x
if for some   0, P(x)  P(X  x)  x  0,1,2,....
x!
  
e  x 
x


x 0
P ( x )  
x 0 x!
 e 

x 0 x!
 e  
e 1

 e  x  e  x

Mean  E(X)   xP (X  x)   x 
x 0 x 0 x! x 1( x  1)!

x 1

     
 e 
 e 1    ...
x 1 ( x  1)!  1! 2! 
 
 e e  

Var (X)  
 Example
The average number of radioactive particles passing
through a counter during 1 millisecond is in a
laboratory experiment is 4. What is the probability that
6 particles enter the counter in a given millisecond?

If the probability of a defective fuse from a manufacturing

unit is 2%, in a box of 200 fuses, find the probability
that exactly 4 fuses are defective.

4 6 4 4
e 4 4 e
,
6! 4!
Example
At a busy traffic intersection the probability p of an
Individual car having an accident is very small say
p=0.0001. However, during a certain peak hours of
the day say between 4 p.m. and 6 p.m., a large
number of cars (say 1000) pass through the
intersection. Under these conditions what is the
probability of two or more accidents occurring during
that period.
In a component manufacturing industry, there is a small
probability of 1/500 for any component to be defective.
The components are supplied in packets of 10. Use
Poisson distribution to calculate the app no of packets
containing (i) no defective (ii)one defective components
in a consignment of 10,000 packets..9802 10000,.0196 10000
Binomial Distribution

n r
P( X  r )  C r p q
n r
; r  0,1,2,..., n
If we assume that n trials constitute a set and if we
consider N sets, the frequency function of the
binomial distribution is given by f(r)=N p(r)

n r
 N Cr p q
n r
 Example
Fit a binomial distribution for the following data and
hence find the theoretical frequencies:
x:0 1 2 3 4
f: 5 29 36 25 5

The following data are the number of seeds

germinating out of 10 on damp filter paper for 80
set of seeds. Fit a binomial distribution to these
data: x 0 1 2 3 4 5 6 7 8 9 10

y 6 20 28 12 8 6 0 0 0 0 0

Ans. 7,26,37,34,6
6.89,19.14,23.94,17.74,8.63,2.88,0.67,0.1,0.01,0,0
Fit a Poisson distribution for the following distribution:
x: 0 1 2 3 4 5
f: 142 156 69 27 5 1

Fit a Poisson distribution to the following data which

gives the number of yeast cells per square for
400 squares
No. of cells per 0 1 2 3 4 5 6 7 8 9 10
square (x)

No. of squares (f) 103 143 98 42 8 4 2 0 0 0 0

Ans. 147 147 74 25 6 1

107,141,93,41,4,0,0,0,0,0
 GEOMETRIC DISTRIBUTION

Suppose that independen t trials, each having a

probabilit y p of success, are performed until a success
occurs. If we let X  n, then P(X  n)  (1 - p) n -1 p, n  1,2,...
Since in order that X  n, it is necessary and sufficient
that the first (n - 1) trials are failures and the nth trial is
a success.
  n 1 p
 P(X  n )   (1  p) p  1.
n 1 n 1 1  (1  p)
Geometric Random Variable
 In a chemical engineering process industry it is
known that , on the average, 1 in every 100 items
is defective. What is the probability that the fifth
item inspected is the first defective item found.
ans (.01)(.99) 4  .0096
At busy time, a telephone exchange will be working
busy with full capacity . So people cannot get a line to
use immediately. It may be of interest to know the
number of attempts necessary in order to get a
connection. Suppose that p = 0.05, then find the
probability that 5 attempts are necessary for a
successful call connection.
Ans .041
Mean
 n 1  n 1
E(X)   n (1  p) p  p  nt , t  1 p
n 1 n 1

 p[1  2t  3t  4t  ....]
2 3
(sin ce t  1)
2 p 1
2
 p[1  t ]  p[1  (1  p)]  2 
p p
  n 1  2 n 1
E(X )   n (1  p)
2 2
p  n t p
n 1 n 1
 n 1
  [n (n  1)  n ]t p
n 1
 n 1  n 1
  [n (n  1)]t p   nt p
n 1 n 1

 n 2
 pt  [n (n  1)]t 1/ p
n 2
  pt[2  3.2t  4.3t  5.4t  ....]  1 / p
2 3

3
 2pt[1  3t  6t  ...]  1 / p  2pt[1  t ]  1 / p
2

2p(1  p) 2  p
 3
 2
p p
1 p
Var (X)  2
p
 Negative Binomial Distribution
Trials repeated until a fixed number of success occur.

Instead of finding the probability of r success in n trials

when n is fixed
Probability that rth success occurs on the xth trial.
Negative binomial experiment

The number of x trials to produce r success in a negative

binomial experiment is called a negative binomial
random variable and its probability distribution is
called the negative binomial distribution
The negative binomial distribution is used when the
number of successes is fixed and we're interested in
the number of failures before reaching the fixed number
of successes. An experiment which follows a negative
binomial distribution will satisfy the following requirements
1.The experiment consists of a sequence of independent
trials.
2.Each trial has two possible outcomes, S or F.
3.The probability of success,is constant from
one trial to another.
4.The experiment continues until a total of r successes
are observed, where r is fixed in advance
Suppose we repeatedly throw a die, and consider
a "1" to be a "success". The probability of success
on each trial is 1/6. The number of trials needed to get
three successes belongs to the infinite set
{ 3, 4, 5, 6, ... }. That number of trials is a (displaced)
negative-binomially distributed random variable.
The number of failures before the third success
belongs to the infinite set { 0, 1, 2, 3, ... }. That number
of failures is also a negative-binomially distributed
random variable.
A Bernoulli process is a discrete time process, and
so the number of trials, failures, and successes are
integers. For the special case where r is an integer,
the negative binomial distribution is known as the
Pascal distribution.

A further specialization occurs when r = 1: in this

case we get the probability distribution of failures
before the first success (i.e. the probability of success
on the (k+1)th trial), which is a geometric distribution.
Let the random variable y denotes the no of failures
before the occurrence of the rth success. Then y+r
denotes the number of trials necessary to produce
exactly r success and y failures with the rth success
occurring at the (y+r)th trial.
pmf of y  g(y)   ( n 1) r 1
C( r 1) p q py

 ( y  r 1) r 1

C( r 1) p q p, q  1  p
y

( y  r 1)
 C( r 1) p q r y
 Example

Pat is required to sell candy bars to raise money for

the 6th grade field trip. There are thirty houses in the
neighborhood, and Pat is not supposed to return home
until five candy bars have been sold. So the child goes
door to door, selling candy bars. At each house, there
is a 0.4 probability of selling one candy bar and a 0.6
probability of selling nothing.

What’s the probability that Pat finishes on the tenth

house?
A fair die is cast on successive independent trials until
the second six is observed. The probability of observing
exactly ten non-sixes before the second six is cast is
……..

Find the probability that a person tossing three coins will

get either all heads or all tails for the second time
on the fifth toss.

2 10 2 3
1 5 4 1  3
11
C1     , C1    
6 6  4  4
The probability that an experiment will succeed is 0.8.
If the experiment is repeated until four successful
outcomes have occurred, what is the expected number
of repetitions required?

Ans. 1
 
 g(y)  
y 0


y 0

( y  r 1) r
C( r 1) p q y


 y  r  1! r y 

   pq 
y 0
 r  1! y! 




y 0
 ( y  r 1)
Cy p q  p 
r y
 r 
y 0

( y  r 1)
Cyq y

 y  r  1! y
r 
p    q
y 0
 r  1! y! 
  r  1! r! r  1! 2 
p 
r
 q q  ...
 r  1! r  1! r  1!2! 

 r r  1 2 
 p 1  rq 
r
q  ...
 2! 
 p 1  q   p p  1.
r r r r

rq rq
Mean  E(X)  Variance  2
p p
 PROBABILITY
DISTRIBUTIONS
Discrete Distributions

Binomial Distribution

n r
P( X  r )  C r p q
n r
; r  0,1,2,..., n
If we assume that n trials constitute a set and if we
consider N sets, the frequency function of the
binomial distribution is given by f(r)=N p(r)

n r
 N Cr p q
n r
 Example
Fit a binomial distribution for the following data and
hence find the theoretical frequencies:
x:0 1 2 3 4
f: 5 29 36 25 5

The following data are the number of seeds

germinating out of 10 on damp filter paper for 80
set of seeds. Fit a binomial distribution to these
data: x 0 1 2 3 4 5 6 7 8 9 10

y 6 20 28 12 8 6 0 0 0 0 0

Ans. 7,26,37,34,6
6.89,19.14,23.94,17.74,8.63,2.88,0.67,0.1,0.01,0,0
Fit a Poisson distribution for the following distribution:
x: 0 1 2 3 4 5
f: 142 156 69 27 5 1

Fit a Poisson distribution to the following data which

gives the number of yeast cells per square for
400 squares
No. of cells per 0 1 2 3 4 5 6 7 8 9 10
square (x)

No. of squares (f) 103 143 98 42 8 4 2 0 0 0 0

Ans. 147 147 74 25 6 1

107,141,93,41,4,0,0,0,0,0
 Probbility Density Function
0.0016
0.0012
f X (x ) 0.0008
0.0004
0.0000
-100 0 100 200 300 400 500 600 700 800
0 if x  0 x
1
f X ( x)   if 0  x  u
u
0 if u  x

 Cumulative Distribution Function
1.0
0.8
0.6
F X (x )
0.4
0.2
0.0
-100 0 100 200 300 400 500 600 700 800
0 if x  0 x
x
FX ( x)   if 0  x  u
u
1 if u  x

If X is uniformly distributed over the interval [0,10]
compute the probability (a) 2<X<9(b) 1<X<4
(c)X<5 (d) X>6

Ans. 7/10,3/10,5/10,4/10
 b b 1
mean  E(X)   xf ( x )dx   x dx
a a ba
1
mean  (b  a )
2
If X has uniform distributi on in (-3,3), find P( X - 2  2)
Ans. 1/2
If X has uniform distributi on in (-a, a), a  0, find
a such that P( X  1)  P( X  1)
a=2
Buses arrive at a specific stops at 15min. Intervals
starting at 7 A.M., that is , they arrive at 7,7:15,7:30
and so on. If a passenger arrives at the stop at a
random time that is uniformly distributed between
7 and 7:30A.M., find the probability that he waits
(a) less than 5 min (b) at least 12 min. for a bus.

Ans. 1/3,1/5
Example:

The total time it takes First Bank to process a loan application is

uniformly distributed between 3 and 7 days. What is the
probability that the application will be processed in less than 4
days?1 Total Area
ba 1
 .25 (b  a )( )
ba

0 3 5 7
a 4 6.5 b

1
P (3  x  4)  ( 4  3)( )  .25
73

What is the probability that it will take more than 6.5 days?
1
P (6.5  x  7)  (7  6.5)( )  .125
73
 var iance  E(X 2 )  E(X) 2
ab
E(X) 
2
2
b
E(X )   x f ( x )dx 
a
2 1
3(b  a )
 
x3
b
a


b  a  b  ab  a
3

3 2 2

3(b  a ) 3

a  ab  b a  2ab  b
2 2 2 2
var  
3 4
a  b  2ab b  a 
2 2 2
 
12 12
 n 1 n 1
b a
moments  E(X )  n

n  1(b  a )
central moments  r  E{(X - E(X)) } r

a  b
r
1 b
  x   dx
ba a 2 
r 1 b r 1 r 1
 a b  ba  a b
x      
1  2   2   2 
 
ba r 1 b  a r  1
a
 0 if r is odd

  1  b  a  if r is even
r

  
r 1  2 
1  ba 
2n

 2 n 1  0,  2 n    for n  1,2,3,..
2n  1  2 

mean deviation  E( X - E(X)

b ab 1
 b  a 
  x * dx 1
a 2 ba
4
 The gamma function, denoted by  is defined as

(x)   e t - t x 1
dt , x  0
0
Pr operties

1.( x )   e t    x  1t
 t x 1 
0
x 2 t
e dt
0

 ( x  1)  e  t t x 11dt  ( x  1)( x  1)
0


2.(1)   e t dt  1
0
3.put x  n
(n)  n - 1n - 1  n  1n  2(n  2)  ......
 (n  1)( n  2)....2.1
 (n  1)!
(1)  0! 1
 Exponential Distribution
If the occurances of events over nonoverlapping
intervals are independent, such as arrival times of
telephone calls or bus arrival times at a bus stop,
then the waiting time distribution of these events
can be shown to be exponential

• Time between arrivals to a queue (e.g. time

between people arriving at a line to check out in a
department store. (People, machines, or telephone calls may
wait in a queue)

• Lifetime of components in a machine

Exponentia l distributi on
e -x x  0
f(x)  
0 otherwise
parameter   0
Exponentia l distributi on
e -x x  0
f(x)  
0 otherwise
parameter   0

x
moments  'r  E(X )   x e
r r
dx
0

1  r y (r  1) r!
 r  y e dy  
 0  r
r
 Example
Let X have an exponential distribution with mean of
100 . Find the probability that X<90
Ans. 0.593
Customers arrive in a certain shop according to an
approximate Poisson process at a mean rate of 20
per hour. What is the probability that the shopkeeper
will have to wait for more than 5 minutes for his first
customer to arrive?
-15
ans . e
 Memoryless Property of the Exponential Distribution

If X is exponentia lly distribute d, then

P(X  s  t/X  s)  P(X  t), for any s, t  0

 

x x  k
P(X  k)   e dx   e k e
PX  s  t & X  s
k
Now P(X  s  t/X  s) 
PX  s
PX  s  t e   (s t )
 t
 e  P(X  t ).
   s
PX  s e
If x represents the lifetime of an equipment then
above property states that if the equipment has
been working for time s, then the probability that it
will survive an additional time t depends only on t
(not on s) and is identical to the probability of survival
for time t of a new piece of equipment.

Equipment does not remember that it has been in

use for time s.
A crew of workers has 3 interchangeable machines,
of which 2 must be working for the crew to its job.
When in use, each machine will function for an
exponentially distributed time having parameters 
before breaking down. The workers decide to initially
use machines A and B and keep machine C in
reserve to replace whichever of A or B breaks down
first. They will then be able to continue working until
one of the remaining machines breaks down. When
the crew is forced to stop working because only
one of the machines has not yet broken down, what
is the probability that the still operable machine is
machine C?
 Suppose the life length of an appliance has an
exponential distribution with   10 years.
A used appliance is bought by someone. What is
the probability that it will not fail in the next 5 years?

0.368
Suppose that the amount of waiting time a customer
spends at a restaurant has an exponential distribution
with a mean value of 5 minutes Then find the
probability that a customer will spend more than
10 minutes in the restaurant
0.1353
Example
Suppose that the length of a phone call in minutes is
an exponenetial random variable with parameter
1

10.
If A arrives immediately ahead of B at a public
telephone booth, find the probability that B will have
to wait (i) more than 10 minutes, and (ii) between
10 and 20 minutes.
(ans. 0.368,0.233)
 Erlang distribution or General Gamma distribution

A continuous RV X is said to follow an Erlang

distribuio n or General Gamma distribuio n with parameter
  0, k  0, if its pdf is given by
 k x k 1e x
 , for x  0
f(x)   (k )
0, otherwise



 f ( x )dx  1
0
  1,
k 1  x
x e
pdf  , x  0; k  0.
(k )
Gamma distributi on or simple gamma distributi on
with parameter k.

When k  1, the Erlang distributi on reduces

to the exponentia l distributi on with parameter
  0.
Example
The random variable X has the gamma distributi on
with density function given by
0 for x  0
f(x)   -2x
2e for x  0
Find the probabilit y that X is not smaller th an 3.


2 x  e  6
2 x
P(X  3)  2  e dx  2  e
3  2 
Suppose that an average of 30 customers per hour
arrive at a shop in accordance with a Poisson Process
That is, if a minute is our unit , then   1 / 2. What
is the probability that the shopkeeper wait more than
5 minutes before both of the first two customer arrive?

Solution.
If X denotes the waiting time in minutes until the
second customer arrives, then X has Erlang(Gamma)
distribution with k = 2,   1 / 2
x  x
k 1

P(X  5)   e dx
5 ( k )
=0.287
Mean and Variance of Erlang Distribution

moments  'r  E(X ) r

  k  r 1 x
k
 x e dx
0 ( k )
k
1  k  r 1  t 1 ( k  r )
  t e dt  r
k r
( k )  0  ( k )

k k
mean  E(X)  var (X)  2
 
m.g.f .  M X (t )  E(e ) tx

  k
k 1 x tx k 
 x e e dx  k 1 (   t ) x
x e dx
0 ( k ) (k) 0
k
1  k 1  y  k
 k 
y e dy 
(k )   t  0   t k

k
 t
 1  
 
 Reproductive Property
M ( X1  X 2 ...  X n ) ( t )  M X1 ( t )M X 2 ( t ).....M X n ( t )
M X1 ( t )M X 2 ( t ).....M X n ( t )
 k1 k 2 k n
 k  k  k
 1   1   .....1  
     
  k1k 2 ...k n 
 t
 1  
 
 M ( X1  X 2 ...  X n ) ( t )
The sum of a finite number of Erlang variables is also
an Erlang variable.

if X1 , X 2 ,...., X n are independen t Erlang variables

with parameters (, k1 ), (, k 2 ),......(, k n ), then
X1  X 2  ....  X n is also an Erlang variable with
parameter (, k1  k 2  .....  k n )
If a company employes n sales persons, its gross
sales in thousan ds of rupees may be regarded as
RV having an Erlang distribuit ion with   1/2,
and k  80 n . If the sales cost is Rs. 8000 per
person, how many salesperso ns should the company
employ to maximise the expected profit?

Let X represent the gross sales (in Rupees) by n

Sales persons

X will have Erlang distribution

Weibull Distribution

1  x 
f (x)  x e ,x  0
parameter ,  ,   0
when   1, Weibull distributi on reduces to
the exponentia l distributi on with parameter .

r  1 x 
 r '  E(X )    x
r
e dx
0
r 1 1
1 1
 y
    y
y

   e   dy
0   
 1 
1


Mean  E(X)  '1     1
 
  2    1  2

Var (X)     1    1 

2 / 

        
Each of the 6 tubes of a radio set has a life
length(in years) which may be considered as
a RV that follows a Weibull distributi on with
parameters   25 and   2. If these tubes
function independen tly of one another, what is
the probabilit y that no tube will have to be replaced
during the first 2 months of service?

If X represents the life length of each tube, then

-1 x 
its density function f(x) is given by f(x)  x e ,x  0
 25x 2
f (x)  50xe ,x  0
P( a tube is not to be replaced during the first
2 months)

 p(X  1 / 6)   50xe


1/ 6
 25 x 2

dx   e  25 x 2


1
6
 25 / 36
e

P(all the 6 tubes are not to be replaced during

the first 2 months)
 e  
- 25/36 6
 0.0155
 Properties of a
Normal Distribution
• Continuous Random Variable
• Symmetrical in shape (Bell shaped)
• The probability of any given range of
numbers is represented by the area under
the curve for that range.
• Probabilities for all normal distributions are
determined using the Standard Normal
Distribution.
 Probability for a
Continuous Random Variable
Probability Density Function for
Normal Distribution
x 
2

1 1
2( )
f (x)  e 
 2
   x  ,    ,   0

N(, )
 
2
1 x 7
f (x)   (
1
e 2 )
32 4

   x  ,    ,   0

N(7,4)
 
 f ( x )dx  1

Standard Normal Distribution
N(0,1)
z2
1 
(z)  e 2
,  z  .
2
  0,   1 & by changing x and f respective ly
into z and .
 X -
If X has distributi on N(, ) and if Z  ,

then Z has distributi on N(0,1)

z
values of (z),  (z)dz are tabulated .
0
 X  N (, )

E(X)   xf (x)dx

1   x  2 / 22
  xe dx
 2  

1 
 
t 2
   2t e dt
 

t 
x 
 2


   t 2 2  t 2
  e dt    te dt
   


   .


similarly , var iance   2

Figure 6.3
Figure 6.5
 Determining the Probability for a
Standard Normal Random Variable
• P(- Z  1.62) = .5 + .4474 = .9474
• P(Z > 1.62) = 1 - P(- Z  1.62) =
1 - .9474 = .0526
z 0.00 0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08 0.09

0.0 0.0000 0.0040 0.0080 0.0120 0.0160 0.0190 0.0239 0.0279 0.0319 0.0359

0.1 0.0398 0.0438 0.0478 0.0517 0.0557 0.0596 0.0636 0.0675 0.0714 0.0753

0.2 0.0793 0.0832 0.0871 0.0910 0.0948 0.0987 0.1026 0.1064 0.1103 0.1141

0.3 0.1179 0.1217 0.1255 0.1293 0.1331 0.1368 0.1406 0.1443 0.1480 0.1517

0.4 0.1554 0.1591 0.1628 0.1664 0.1700 0.1736 0.1772 0.1808 0.1844 0.1879

0.5 0.1915 0.1950 0.1985 0.2019 0.2054 0.2088 0.2123 0.2157 0.2190 0.2224

0.6 0.2257 0.2291 0.2324 0.2357 0.2389 0.2422 0.2454 0.2486 0.2517 0.2549

0.7 0.2580 0.2611 0.2642 0.2673 0.2704 0.2734 0.2764 0.2794 0.2823 0.2852
0.8 0.2881 0.2910 0.2939 0.2969 0.2995 0.3023 0.3051 0.3078 0.3106 0.3133

0.9 0.3159 0.3186 0.3212 0.3238 0.3264 0.3289 0.3315 0.3340 0.3365 0.3389

1.0 0.3413 0.3438 0.3461 0.3485 0.3508 0.3513 0.3554 0.3577 0.3529 0.3621

1.1 0.3643 0.3665 0.3686 0.3708 0.3729 0.3749 0.3770 0.3790 0.3810 0.3830

1.2 0.3849 0.3869 0.3888 0.3907 0.3925 0.3944 0.3962 0.3980 0.3997 0.4015

1.3 0.4032 0.4049 0.4066 0.4082 0.4099 0.4115 0.4131 0.4147 0.4162 0.4177

1.4 0.4192 0.4207 0.4222 0.4236 0.4251 0.4265 0.4279 0.4292 0.4306 0.4319

1.5 0.4332 0.4345 0.4357 0.4370 0.4382 0.4394 0.4406 0.4418 0.4429 0.4441

1.6 0.4452 0.4463 0.4474 0.4484 0.4495 0.4505 0.4515 0.4525 0.4535 0.4545
1.7 0.4554 0.4564 0.4573 0.4582 0.4591 0.4599 0.4608 0.4616 0.4625 0.4633

1.8 0.4641 0.4649 0.4656 0.4664 0.4671 0.4678 0.4686 0.4693 0.4699 0.4706

1.9 0.4713 0.4719 0.4726 0.4732 0.4738 0.4744 0.4750 0.4756 0.4761 0.4767

2.0 0.4772 0.4778 0.4783 0.4788 0.4793 0.4798 0.4803 0.4808 0.4812 0.4817

2.1 0.4821 0.4826 0.4830 0.4834 0.4838 0.4842 0.4846 0.4850 0.4854 0.4857

2.2 0.4861 0.4864 0.4868 0.4871 0.4875 0.4878 0.4881 0.4884 0.4887 0.4890

2.3 0.4893 0.4896 0.4898 0.4901 0.4904 0.4906 0.4909 0.4911 0.4913 0.4916

2.4 0.4918 0.4920 0.4922 0.4925 0.4927 0.4929 0.4931 0.4932 0.4934 0.4936

2.5 0.4938 0.4940 0.4941 0.4943 0.4945 0.4946 0.4948 0.4949 0.4951 0.4952

2.6 0.4953 0.4955 0.4956 0.4957 0.4959 0.4960 0.4961 0.4962 0.4963 0.4964
2.7 0.4965 0.4966 0.4967 0.4968 0.4969 0.4970 0.4971 0.4972 0.4973 0.4974

2.8 0.4974 0.4975 0.4976 0.4977 0.4977 0.4978 0.4979 0.4979 0.4980 0.4981

2.9 0.4981 0.4982 0.4982 0.4983 0.4984 0.4984 0.4985 0.4985 0.4986 0.4986

3.0 0.4987 0.4987 0.4987 0.4988 0.4988 0.4989 0.4989 0.4989 0.4990 0.4990

3.1 0.4990 0.4991 0.4991 0.4991 0.4992 0.4992 0.4992 0.4992 0.4993 0.4993

3.2 0.4993 0.4993 0.4994 0.4994 0.4994 0.4994 0.4994 0.4995 0.4995 0.4995

3.3 0.4995 0.4995 0.4995 0.4996 0.4996 0.4996 0.4996 0.4996 0.4996 0.4997

3.4 0.4997 0.4997 0.4997 0.4997 0.4997 0.4997 0.4997 0.4997 0.4997 0.4998
Determining the probability of any Normal Random
Variable
 Interpreting Z
• In figure Z = -0.8 means that the value 360 is
.8 standard deviations below the mean.
• A positive value of Z designates how many
standard deviations () X is to the right of the
mean ().
• A negative value of Z designates how may
standard deviations () X is to the left of the
mean ().
Example: A group of achievement scores are
normally distributed with a mean of 76 and a
standard deviation of 4. If one score is randomly
selected what is the probability that it is at least 80.

 4

76 80

x   80  76
Z  1
 4
P( x  80)  P(z  1)  .5  P(0  z  1) 
.5  .3413  .1587
  4

76 80

xu 80  76
Z  1
 4
P ( x  80)  P ( z  1)  .5  P (0  z  1) 
.5  .3413  .1587
.3413

.1587

0 1
Continuing, what is the probability that it is less than 70.

70 76

xu 70  76
Z   1.5
 4
P ( x  70)  P ( z  1.5)  .5  P (1.5  z  0.0) 
.5  .4332  .0668 .4332

.0668

-1.5 0
What proportion of the scores occur within 70 and 85.

.4332

.4878
-1.5 0 2.25

xu 70  76
Z    1.5
 4
xu 85  76
Z    2.25
 4
P (70  x  85)  P ( 1.5  z  2.25) 
P ( 1.5  z  0.0)  P (0.0  z  2.25) 
.4332  .4878  .9210
Time required to finish an exam is known to be normally
distributed with a mean of 60 Min. and a Std Dev. of 12
minutes. How much time should be allowed in order for
90% of the students to finish?

.9   12

60 x
x 
z

z  x  
z    x
1.28(12)  60  x
x  75.36
An automated machine that files sugar sacks has an adjusting device to
change the mean fill per sack. It is now being operated at a setting that
results in a mean fill of 81.5 oz. If only 1% of the Sacks filled at this
setting contain less than 80.0 oz, what is the value of the variance for this
population of fill weights. (Assume Normality).

.01

-2.33 0
  81.5 x  80 PROB  .01
xu
Z 

80.0  81.5
 2.33 

(80.0  81.5)
   .6437
 2.33
  .4144
2
 Moment generating function of N(0,1)

M Z ( t )  E(e )   e (z)dz
tZ tz


1  tz z 2 / 2 1 
  2 tz / 2
 dz   2
e e  e
z
dz
2   2  
 z  t 2
1    
22
 z t t  / 2 1  
dz  e
t2 / 2
  e
 
 e 2
dz
2   2  
1
1  u du  e t2 / 2
(1 / 2)  e t2 / 2
e t2 / 2
 e 
2   2u
 The moment generating function of N(, )
 M X ( t )  M Z ( t )
t ( z  ) t tz
 E(e )  e E(e )
t
 e M Z (t )

e et  2 t 2 / 2
e    
t  2 t / 2

t t t 2
t 2 2 2
 1  (  )  (  )  .....  
1! 2 2! 4
If X has the distributi on N(, ) then Y  aX  b
has the distributi on N(a  b, a)

MX ( t )  e    
t   2 t / 2

M Y ( t )  M aX  b ( t )
  e M X (at )
bt

   
bt t a  a 2  2 t / 2
e e

e    
t a  b  a 2 2 t / 2

MGF N(a  b, a)

X -
If X has distributi on N(, ), then Z 

1  1
has the distributi on N(   , .)  N(0,1)
  
Additive property of normal distributi on
If X i (i  1,2,...., n ) be n independen t normal
RVs with mean  i and variance  , then 2
i
n n
 a i X i is also a normal RV with mean  a i  i
i 1 i 1
n
& var iance  a i  i .
2 2
i 1

Mn ( t )  M a1X1 ( t )M a 2 X 2 ......M a n X n , (by independen ce)

 a i Xi
i 1
 a11t  a12 12 t 2 / 2 a 2 2 t  a 22  22 t 2 / 2
e .e ........
 a ii t   a i2i2 t 2 / 2
e
When n is very large and neither p nor q is very small

X—B(n,p)
s tan dard binomial variable Z is given by
X - np
Z
npq
as X varies from 0 to n with step size 1, Z varies
- np np 1
from to with step size .
npq npq npq
  1 t 2 / 2
P(z)  F(z)   e dt,  z  
 2
Let X be the number of times that a fair coin, flipped 40
times, land heads. Find P(X=20). Use normal
approximation and compare it to the exact solution.

P(X=20)=P(19.5<X<20.5)

 19.5  20 X  20 20.5  20 
 P   
 10 10 10 
 (.16)  (.16)  .1272
0 1 2 3 4 5 6 7 8 9 10 11 12
01 2 3 4 5 6 7 8 9 10 11 12
If 20% of the momory chips made in a certain plant
are defective, what are the probabilities that in a
lot of 100 randomly chosen for inspection
(a) at most 15 will be defective?
(b) exactly 15 will be defective ?

  100(.20)  20,   4

15.5  20
F( )  0.1292
4
 15.5  20   14.5  20 
F   F   0.0454
 4   4 
 Fit a normal distribution to the following distribution
and hence find the theoretical frequencies:
Class Freq
60-65 3
65-70 21
70-75 150
75-80 335
80-85 336
85-90 135
90-95 26
95-100 4
-------------
1000
 1  x 
f (x)  x e ,x  0
The special case of Weibull with   1/ &   2 2

is known as the Rayleigh Distributi on.

Thus Rayleigh has linear rate .
x  x 2 / 2a 2
If the pdf of a continuous RV X is f(x)  2 e ,0  x  ,

then X follows a Rayleigh distributi on with parameter .

In communication systems, the signal amplitude values

of a randomly received signal usually can be modeled
as a Rayleigh distribution.
Let X have a gamma distributi on with   1/2 and k  r/2
where r is a positive integer.
 x r/21 x / 2
 e ,0  x  
The pdf of X is f(x)   (r / 2)2 r/2

0 x0


X has a chi - square distributi on  2 (r ) with r degrees

of freedom.
 k r / 2
E( X)   r
 1 / 2
k r / 2
Var (X)  2   2r
 1 / 4
Mean equals the number of degrees of freedom and
the variance equals twice the number of degrees of
freedom.
r / 2
M X (t )  (1  2t ) , t  1/ 2
df
.005 .01 .025 .05 .10 .90 .95 .975 .99 .995
\p
.0000 .0001 .0009 .003
1 .0158 2.71 3.84 5.02 6.63 7.88
4 6 8 9
.102
2 .0100 .0201 .0506 .2107 4.61 5.99 7.38 9.21 10.60
6
3 .0717 .115 .216 .352 .584 6.25 7.81 9.35 11.34 12.84
4 .207 .297 .484 .711 1.064 7.78 9.49 11.14 13.28 14.86
5 .412 .554 .831 1.15 1.61 9.24 11.07 12.83 15.09 16.75
.6 .676 .872 1.24 1.64 2.20 10.64 12.59 14.45 16.81 18.55
7 .989 1.24 1.69 2.17 2.83 12.02 14.07 16.01 18.48 20.28
8 1.34 1.65 2.18 2.73 3.49 13.36 15.51 17.53 20.09 21.96
9 1.73 2.09 2.70 3.33 4.17 14.68 16.92 19.02 21.67 23.59
10 2.16 2.56 3.25 3.94 4.87 15.99 18.31 20.48 23.21 25.19
11 2.60 3.05 3.82 4.57 5.58 17.28 19.68 21.92 24.73 26.76
12 3.07 3.57 4.40 5.23 6.30 18.55 21.03 23.34 26.22 28.30
13 3.57 4.11 5.01 5.89 7.04 19.81 22.36 24.74 27.69 29.82
df
.005 .01 .025 .05 .10 .90 .95 .975 .99 .995
\p
14 4.07 4.66 5.63 6.57 7.79 21.06 23.68 26.12 29.14 31.32
15 4.6 5.23 6.26 7.26 8.55 22.31 25 27.49 30.58 32.80
16 5.14 5.81 6.91 7.96 9.31 23.54 26.30 28.85 32.00 34.27
18 6.26 7.01 8.23 9.39 10.86 25.99 28.87 31.53 34.81 37.16
10.8
20 7.43 8.26 9.59 12.44 28.41 31.41 34.17 37.57 40.00
5
13.8
24 9.89 10.86 12.40 15.66 33.20 36.42 39.36 42.98 45.56
5
18.4
30 13.79 14.95 16.79 20.60 40.26 43.77 46.98 50.89 53.67
9
26.5
40 20.71 22.16 24.43 29.05 51.81 55.76 59.34 63.69 66.77
1
43.1
60 35.53 37.48 40.48 46.46 74.40 79.08 83.30 88.38 91.95
9
95.7 100.6 140.2 146.5 152.2 158.9 163.6
120 83.85 86.92 91.58
0 2 3 7 1 5 4
 Let X be  (10). Find P(3.25  X  20.5).
2

Ans.0.95

If 1 - 2t  , t  1 / 2, is the m.g.f. of the

6

random variable , find P(X  5.23).

0.05
If X is  (5) , determine the constant c and d so that
2

P(c  X  d)  0.95 & P(X  c)  0.025

0.831, 12.8
Beta Distribution
The random variable x is said to have beta
distributi on with nonnegativ e parameters  & 
 1  1  1
if f(x)   x (1  x ) ,0  x  b
 B( , )
0 otherwise
1
 -1  1
where B( , )   x (1  x ) dx
0
2  
 2  sin  cos 
2  1 2 1
d 
0   
when     1, beta distributi on is uniform distributi on
on (0,1)

beta function provides greater flexibilit y than

uniform distributi on on (0,1)

Depending on the values of  &  , the beta

distributi on takes a variety of shapes.
 
 , 
2

         1
2
In a certain country, the proportion of highway
requiring repairs in any given year is a random
variable having the beta distributi on with   3 and
  2. Find
(a) on the average what percentage of the highway
sections require repairs in any given year :
(b) the probabilit y that at most half of the highway
sections will require repairs in any given year.
 3
(a )   0.60, i.e.60% of the high way
3 2
sections require repairs in any given year.
1/2
(b)P(X  1/2)   12x (1  x )dx  5 / 16
2

0
 The lognormal distribution

If X  log T follows a normal distributi on N(, ),

then T follows a lognormal distributi on whose pdf is given

1  1  t  
2

by f(t)  exp  2 log  , t  0

st 2  2s  tm  

where s   is a shape parameter and t m , the
median time(or mean time) to failure is the location parameter,
given by log t m  .
 2
s
MTTF  E( t )  t m exp( )
2
2 2 2

2

var( T)   T  t m exp( s ) exp( s )  1
F(t)  P(T  t)  Plog T  log t
 log T   log t   
 P  
   
 log T  log t m 1 t 
 P  log 
  s tm 
 1 t 
 P Z  log 
 s tm 
 We can compute R(T) and (t).
Fatigue wearout of a component has a log normal distributi on
with t M  5000 hours and s  0.20.
(a) Compute the MTTF and SD.
(b) Find the reliabilit y of the component for 3000 hours.
(c) Find the design life of the component for a reliabilit y of 0.95.

5101hours,1030hours
 
R (3000)   f ( t )dt  1
 (z)dz
3000
3000 log( )
0.2 5000
 
  (z)dz  0.9946
 2.55

3598.2
z 0.00 0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08 0.09
0.000 0.004 0.008 0.012 0.016 0.019 0.023 0.027 0.031 0.035
0.0
0 0 0 0 0 0 9 9 9 9
0.039 0.043 0.047 0.051 0.055 0.059 0.063 0.067 0.071 0.075
0.1
8 8 8 7 7 6 6 5 4 3
0.079 0.083 0.087 0.091 0.094 0.098 0.102 0.106 0.110 0.114
0.2
3 2 1 0 8 7 6 4 3 1
0.117 0.121 0.125 0.129 0.133 0.136 0.140 0.144 0.148 0.151
0.3
9 7 5 3 1 8 6 3 0 7
0.155 0.159 0.162 0.166 0.170 0.173 0.177 0.180 0.184 0.187
0.4
4 1 8 4 0 6 2 8 4 9
0.191 0.195 0.198 0.201 0.205 0.208 0.212 0.215 0.219 0.222
0.5
5 0 5 9 4 8 3 7 0 4
0.225 0.229 0.232 0.235 0.238 0.242 0.245 0.248 0.251 0.254
0.6
7 1 4 7 9 2 4 6 7 9
0.258 0.261 0.264 0.267 0.270 0.273 0.276 0.279 0.282 0.285
0.7
0 1 2 3 4 4 4 4 3 2
 0.288 0.291 0.293 0.296 0.299 0.302 0.305 0.307 0.310 0.313
0.8
1 0 9 9 5 3 1 8 6 3
0.315 0.318 0.321 0.323 0.326 0.328 0.331 0.334 0.336 0.338
0.9
9 6 2 8 4 9 5 0 5 9
0.341 0.343 0.346 0.348 0.350 0.351 0.355 0.357 0.352 0.362
1.0
3 8 1 5 8 3 4 7 9 1
0.364 0.366 0.368 0.370 0.372 0.374 0.377 0.379 0.381 0.383
1.1
3 5 6 8 9 9 0 0 0 0
0.384 0.386 0.388 0.390 0.392 0.394 0.396 0.398 0.399 0.401
1.2
9 9 8 7 5 4 2 0 7 5
0.403 0.404 0.406 0.408 0.409 0.411 0.413 0.414 0.416 0.417
1.3
2 9 6 2 9 5 1 7 2 7
0.419 0.420 0.422 0.423 0.425 0.426 0.427 0.429 0.430 0.431
1.4
2 7 2 6 1 5 9 2 6 9
0.433 0.434 0.435 0.437 0.438 0.439 0.440 0.441 0.442 0.444
1.5
2 5 7 0 2 4 6 8 9 1
0.445 0.446 0.447 0.448 0.449 0.450 0.451 0.452 0.453 0.454
1.6
2 3 4 4 5 5 5 5 5 5
 0.455 0.456 0.457 0.458 0.459 0.459 0.460 0.461 0.462 0.463
1.7
4 4 3 2 1 9 8 6 5 3
0.464 0.464 0.465 0.466 0.467 0.467 0.468 0.469 0.469 0.470
1.8
1 9 6 4 1 8 6 3 9 6
0.471 0.471 0.472 0.473 0.473 0.474 0.475 0.475 0.476 0.476
1.9
3 9 6 2 8 4 0 6 1 7
0.477 0.477 0.478 0.478 0.479 0.479 0.480 0.480 0.481 0.481
2.0
2 8 3 8 3 8 3 8 2 7
0.482 0.482 0.483 0.483 0.483 0.484 0.484 0.485 0.485 0.485
2.1
1 6 0 4 8 2 6 0 4 7
0.486 0.486 0.486 0.487 0.487 0.487 0.488 0.488 0.488 0.489
2.2
1 4 8 1 5 8 1 4 7 0
0.489 0.489 0.489 0.490 0.490 0.490 0.490 0.491 0.491 0.491
2.3
3 6 8 1 4 6 9 1 3 6
0.491 0.492 0.492 0.492 0.492 0.492 0.493 0.493 0.493 0.493
2.4
8 0 2 5 7 9 1 2 4 6
0.493 0.494 0.494 0.494 0.494 0.494 0.494 0.494 0.495 0.495
2.5
8 0 1 3 5 6 8 9 1 2
0.495 0.495 0.495 0.495 0.495 0.496 0.496 0.496 0.496 0.496
2.6
 0.496 0.496 0.496 0.496 0.496 0.497 0.497 0.497 0.497 0.497
2.7
5 6 7 8 9 0 1 2 3 4
0.497 0.497 0.497 0.497 0.497 0.497 0.497 0.497 0.498 0.498
2.8
4 5 6 7 7 8 9 9 0 1
0.498 0.498 0.498 0.498 0.498 0.498 0.498 0.498 0.498 0.498
2.9
1 2 2 3 4 4 5 5 6 6
0.498 0.498 0.498 0.498 0.498 0.498 0.498 0.498 0.499 0.499
3.0
7 7 7 8 8 9 9 9 0 0
0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499
3.1
0 1 1 1 2 2 2 2 3 3
0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499
3.2
3 3 4 4 4 4 4 5 5 5
0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499
3.3
5 5 5 6 6 6 6 6 6 7
0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499
3.4
7 7 7 7 7 7 7 7 7 8