PROBABILITY AND RANDOM

PROCESS

MODULE-3
 2

DEPT EC

Introduction
VCET PUTTUR

• A signal is random if it is not possible predict its

precise value in advance
• The received signal consists of information
bearing signal component, random interference
component & receiver noise
• Receiver noise is caused by the random motion
of electron in conductors & devices at the front
end of receiver
• Received signal is random in nature
 3

DEPT EC
VCET PUTTUR

Contd….

• It is not possible to predict the random signal in

advance, it may be described in terms of its
statistical properties such as average power in
the random signal
 4

DEPT EC
VCET PUTTUR

Idea of Probability
• Chance behavior is unpredictable in the short
run, but has a regular and predictable pattern in
the long run.
• The probability of any outcome of a random
phenomenon is the proportion of times the
outcome would occur in a very long series of
repetitions.
 5

probability DEPT EC
VCET PUTTUR

• There are 2 approaches to design

▫ Based on relative frequency of occurrence:
 In n trials of a random experiment, if we expect an
event A to occur m times, then we assign the
probability m/n to event A
 ie P(A)=m/n
▫ Based on the set theory & set of related
mathematical axioms
 When we perform a experiment it is natural for us to
aware of various outcomes we have a point called
sample point, which we denoted by Sk
 7

DEPT EC

Terminology
VCET PUTTUR

• Sample Space - the set of all possible outcomes

of a experiment.
• Event - any set of outcomes of interest
• Single sample point is called an elementary
event
• Entire sample space S is called sure event & null
set ϕ is called null or impossible event
• Two events are mutually exclusive if the
occurrence of one event precludes the
occurrence of other event
 8

DEPT EC
VCET PUTTUR

Probability measure P

• It is a function that assign a non negative

number to a event in a sample space S & satisfies
the following 3 properties
▫ 0≤P[A]≤1
▫ P[S]=1
▫ If A &B are mutually exclusive events then,
 P[AUB]= P[A] +P[B]
 9

DEPT EC
VCET PUTTUR
• Sample space S is mapped to events via random experiments

• 3 axioms & their relationship to the relative frequency approach is illustrated

in venn diagram
 10

DEPT EC
VCET PUTTUR

Following properties of probability measure P may

be derived from following axioms

• When an event A & B are not mutually exclusive,

then the probability of union “A or B “satisfies

• If A1,A2,……..Am are mutually exclusive events that

include all possible outcomes of random experiment,
then
 11

DEPT EC
VCET PUTTUR

Conditional Probability
• The conditional probability of B given A

P[B|A]= P[ A ∩ B ] /P[A ]

• A and B are independent events if and only if

• P[B|A]= P[ B]
 12

DEPT EC
VCET PUTTUR

Dependent Events
• When two events occur, if the outcome of one
event affects the outcome of the other, they are
called dependent events.
• Consider the aforementioned example of
drawing a pen from a pack, with a slight
difference.
 13

DEPT EC
VCET PUTTUR

Random Processes
• Analysis of communication systems is the
characterization of random signals such as voice
signals, television signals, noise signals etc…
• 2 properties
1. The signals are functions of time.
2. The signals are random in nature, it is not
possible to describe exactly the waveforms.
 14

• Each sample point in the sample space is function of

DEPT EC
VCET PUTTUR

time. The sample space comprised of functions time

is called random process or stochastic process.
X(t, S), -T≤ t ≤ T
• where 2T is the total observational interval.
• The graph of the function Vs time t is called
realization or sample function of the random process .
x(t) = X(t, S)
 15

DEPT EC
VCET PUTTUR
 16

DEPT EC
VCET PUTTUR

What is the difference between random

variable and processes?
• For a random variable, the outcome
of a random experiment is mapped
onto a variable, e.g., a number.
• For a random processes, the
outcome of a random experiment is
mapped onto a waveform that is a
function of time.
 17

DEPT EC
VCET PUTTUR

Mean
• Mean of the process x(t) as the expectation of the
random variable obtained by observing the
process at some time t.
 18

DEPT EC
VCET PUTTUR

Auto-correlation
• The expectation of the product of 2 random
variables x(t1) and x(t2) obtained by observing
x(t) at times t1 and t2 respectively.
 19

DEPT EC
VCET PUTTUR
 20

DEPT EC
VCET PUTTUR
 21

DEPT EC
VCET PUTTUR
 22

DEPT EC
VCET PUTTUR

Auto-covariance
• Function of a stationary random process x(t) is
written as

= Rx(t2-t1) –
 23

DEPT EC
VCET PUTTUR
 24

DEPT EC
VCET PUTTUR

Proof:
 25

DEPT EC
VCET PUTTUR

Cross correlation

 Rxy(t,u) = E[x(t) y(u)]

• Not an even function.
• Doesn’t have maximum value at origin.
• Obeys symmetric relationship.
 26

1. Define probability. Explain conditional and joint probabilities.

DEPT EC
VCET PUTTUR

2. State and prove various properties of probability.

3. What is conditional probability? Prove that
P(AB)= P(B/A).P(A)
4. For a binary symmetric channel derive the equations for P(Bo),
P(B1), P(Ao/Bo), P(A1/B1) if Ao , A1 are the inputs. Bo, B1 are the
outputs.
5. Define random variables and differentiate between discrete and
continuous random variables
6. Describe mean, correlation and covariance functions with respect
to stationary random process
7. Give the properties of the auto-correlation function.
8. Explain the terms joint probability density function of random
variables x and y; conditional probability density function of y;
statistically independent random variables.
9. Show that the area under PDF curve is always equal to unity
10. Explain cross correlation functions. Give the properties of cross
correlation function.