Random Signal

Probability Theory
The probability theory is used in the analysis of non-deterministic or random signals and systems. An experiment is called as random experiment if the outcome of the experiment cannot be predicted precisely.

Examples of random experiment are

Tossing a coin Rolling a die Drawing a card from a deck.

A random experiment can have many different "outcomes".

For example, a tossed coin has two possible outcomes (H or T) or A rolling die has six possible outcomes (1, 2, 3, .... 6).

sample space
The sample space S is defined as a" collection of all the possible, separately identifiable outcomes of a random experiment. For example, the sample space for tossing a coin will be, S = (H,T) Similarly sample space for an experiment of rolling of a die will be, S = { 1, 2, 3, 4, 5, 6 }

Probability
Let us assume that a specific desired event is A. Now repeat the experiment N times and record the number of times the event A has occurred i.e. nA. Relative frequency of occurrence = nA/N As N then the ratio (nA / N) can be defined as the probability of occurrence of event A.

Random variable
A random variable X is a process by which a (real) number x(s) is assigned to each possible outcome of a statistical experiment A random variable is neither random nor a variable.

Random Variable

RV are of two types

Discrete

RVs and Continuous RVs

Discrete Random Variables

If s represents the outcome of the experiment then the RV is represented by X(s) or simply X RV X(s) is a function that maps the sample points into real numbers x1, x2, x3. If S contains a countable number of sample points, then X will be a discrete RV having a countable number of distinct values.

Continuous Random Variables

A continuous RV may take on any value within a certain range of the real line. Continuous RV has an uncountable number of possible values

Cumulative distribution function

The CDF of a RV is defined as the probability that the RV X takes values less than or equal to x.

Where {X x} denotes an event and x is a real number.

Properties of cdf

Since CDF represents probability so it must be bounded between 0 and 1

With extreme values

It is an null event & probability is 0 It includes all possible outcomes or event so probability is 100%

The CDF Fx(x) is a non-decreasing function of x, i.e., if x1 < x2 Fx(x1) Fx(x2) The complementary events X x and X > x encompass the entire real line, so P(X> x) = 1 - Fx(x)

Probability density function

PDF is more convenient way of describing a continuous RV. Probability density function PDF is defined by

Properties of PDF CDF can be derived from the PDF. It is non negative function for all values of x

The area under the PDF curve is always unity

Problem

A three digit message is transmitted over noisy channel having a probability of error P(e) = 2/5 per digit. Find out corresponding CDF and plot it.

Write sample space Define RV Calculate CDF

Obtain probabilities

Plot CDF

Sample space = {ccc, cce, cec, ecc, cee, ece, eec, eee} {ccc, cce, cec, ecc, cee, ece, eec, eee}

RV X = Number of errors

x0

x1

x2

x3

RV X = { no error, one error, two error, three error}

Fx(x) = P (x = x0) + P (x =x1) + P(x = x2) + P (x =x3) Fx(x0) = P (x x0) = P (x < x0) + P (x = x0) = 0+ 27/125 = 27/125 Fx(x1) = P (x x1) = P (x < x0) + P (x = x0) + P (x =x1) = 0+ 27/125 + 54/125 = 81/125

for x0 x x3

Fx(x2) = P (x x2) = P (x < x0) + P (x = x0) + P (x =x1) + P(x = x2) = 0+ 27/125 + 54/125 + 36/125 = 117/125 Fx(x3) = P (x x3) = P (x < x0)+ P (x = x0)+ P(x =x1)+ P(x = x2)+ P(x =x3) = 0+ 27/125 + 54/125 + 36/125 + 8/125 = 125/125 =1

Probability and CDF

P(X= xi )
54/125 36/125

27/125

8/125

FX(x) )
117/125 81/125 27/125

Statistical Averages of Random Variables

The Probability Density Function (PDF) provides more information about the random variable. But the interpretation of this information is little complex. There are other numbers which provide more convenient and useful information about the random variable quickly. These characteristic numbers are combinely called statistical averages.

Mean/Average or Expected Value

The mean of the random variable is given by summation of the values of X weighted by their probabilities. Mean value is denoted by mx and is also called expected value of X. mx = E [X]

Mean value of a discrete random variable

Consider a discrete random variable X which has a possible values of x1, x2, with the probabilities P(x1), P (x2) ,

As the number of trials N approaches to , the above equation can be written as

Mean value of continuous random variable

Moments and Variance

The nth moment of a random variable X is defined as the mean value of X n. g (x) = X n, then

Thus first moment of random variable X is same as its mean value.

When n = 2

is called mean square value of random variable X. The central moments are the moments of the difference between random variables X and its mean mx . Thus the nth central moment is defined as

Variance of random variable

The second central moment ; i.e. n = 2 is called variance of random variable X.

Thus variance gives an indication about randomness of the random variable. Variance is also denoted by 2

Standard deviation
The square root of variance is called standard deviation of random variable X. Standard deviation provides the measure of spread observed over the values of X relative to mean value Standard deviation = (variance)

Probability Models
We know that probability density function provides very useful information about the occurrence of random variable X. It is just impossible to study all the type of probability distribution functions

Common PDFs.
Following are the commonly used PDFs.

Binomial Distribution Poisson Distribution. Uniform Distribution Gaussian Distribution Rayleighs Distribution

Uniform Distribution

If the continuous random variable X is equally likely to be observed in a finite range and is likely to have a zero value outside this finite range then the random variable is said to have a uniform distribution.

The PDF for a uniform distribution is given as

The value of PDF is same for all possible value of a random variable. Therefore this distribution is called Uniform Distribution. The uniform distribution is useful in describing the quantization noise.

Gaussian Distribution
Gaussian Distribution is also called Normal Distribution. It is defined for continuous random variables. The PDF for a Gaussian random variable is given as,

Gaussian PDF.

This function defines the bell-shaped curve

Properties of Gaussian PDF The peak value occurs at x = m (i.e. mean value).

The plot of Gaussian PDF has even symmetry around mean value

Error function

The error function are some integral which can not be solved directly, that can be solved by numerical methods. The error function is defined as

0 erf(x) 1 As x approaches to then erf (x) tends to unity i.e.

Complementary error function

The complementary error function is defined as

The value of erf (x) at some fixed values of x are available in the form of table. This table is called as error function table.

The function Q (x)

The function Q (x) is closely related to error and complementary error function