Introduction to the theory and algorithms used for the analysis and
representation of random signals (stochastic signals).
Understanding the random signals via:
Statistical description
theory of probability,
random variables and
stochastic processes
Modeling and the dependence between the samples of one or
more discretetime stochastic signals.
Spectrum analysis.
Parameter estimation of stochastic processes.
6
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
Random Signals vs. Deterministic Signals (1)
Example:
Discretetime random signals (a) and the dependence
between the samples (b), [4].
7
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
Random Signals vs. Deterministic Signals (2)
A random signal is not (precisely) predictable, it is not able to find a
mathematical formula that provides its values as a function of time.
A white noise is random signal when every sample is independent of all
other samples. Such a signal is completely unpredictable.
When signal samples are dependent and can be predicted precisely, it is
deterministic signal.
8
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
Methods for Analysis and Processing Random Signals
9
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
Random Signal Analysis
Random signal analysis (signal modeling, spectral estimation):
Primary goal is to extract useful information for understanding and
classifying the signals.
Typical applications: Detection of useful information from receiving
signals, system modeling/identification, detection and classification of
radar and sonar targets, speech recognition, signal representation for
data compression etc.
10
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
Random Signal Analysis (1)
The objective of signal analysis is the development of quantitative
techniques to study the properties of a signal and the differences and
similarities between two or more signals from the same or different
sources.
The prominent tool in signal analysis is spectral estimation, which is a
generic term for a multitude of techniques used to estimate the
distribution of energy or power of a signal from a set of observations.
The major areas of random signal analysis are:
Statistical analysis of signal amplitude (i.e., the sample values);
Analysis and modeling of the correlation among the samples of an
individual signal;
Joint signal analysis (i.e., simultaneous analysis of two signals in
order to investigate their interaction or interrelationships).
11
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
Random Signal Analysis (2)
12
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
Random Signal Analysis (3)
Amplitude distribution: The range of values taken by the samples of
a signal and how often the signal assumes these values together
determine the signal variability.
The signal variability can be quantified by the histogram (probability
density) of the signal samples, which shows the percentage of the
signal amplitude values within a certain range.
The numerical description of signal variability, which depends only on
the value of the signal samples and not on their ordering, involves
quantities such as mean value, median, variance, and dynamic
range.
13
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
Random Signal Analysis (4)
Example:
Two random signals (infrared 1 and infrared 2)
14
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
Random Signal Analysis (5)
Histograms for infrared 1 and infrared 2:
15
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
Random Signal Analysis (6)
Histogram of low contrast image Low contrast image
16
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
Random Signal Analysis (7)
Histogram of high contrast image
High contrast image
17
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
Random Signal Analysis (8)
Correlation and spectral analysis: The scatter plot (see Slide 6) can
be used to illustrate the existence of correlation, however, to obtain
quantitative information about the correlation structure of a time series
signal x(n)
with zero mean value, we use the empirical normalized
autocorrelation sequence:
The spectral density function shows the distribution of signal power
or energy as a function of frequency. The autocorrelation and the
spectral density of a signal form a Fourier transform pair
and hence
contain the same information.
18
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
Random Signal Analysis (9)
Joint signal analysis. In many applications, we are interested in the
relationship between two different random signals. There are two
cases of interest:
In the first case, the two signals are of the same or similar nature,
and we want to ascertain and describe the similarity or interaction
between them.
In the second case, we may have reason to believe that there is a
causal relationship
between the two signals. For example, one
signal may be the input to a system and the other signal the output.
The task in this case is to find an accurate description of the
system, that is, a description that allows accurate estimation of
future values of the output from the input. This process is known
as system modeling or system identification and has many
practical applications (including understanding the operation of
a
system in order to improve the design of new systems or to achieve
better control of existing systems).
19
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
Random Signal Modeling (1)
In many theoretical and practical applications, we are interested in
generating random signals with certain properties or obtaining an
efficient representation of realworld random signals that captures a
desired set of their characteristics (e.g., correlation or spectral features) in
the best possible way. We use the term model to refer to a mathematical
description that provides an efficient representation of the essential
properties of a signal.
Example: A finite segment of any signal can be approximated
by a linear combination of constant (
k
= 1)
or exponentially fading
(0 <
k
< 1)
sinusoids:
where are the model parameters.
20
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
Random Signal Modeling (2)
From a practical viewpoint, we are most interested in parametric
models, which assume a given functional form completely specified
by a finite number of parameters. In contrast, nonparametric models
do not put any restriction on the functional form or the number of
model parameters.
In practice, signal modeling involves the following steps:
Selection of an appropriate model.
Selection of the right
number of parameters.
Fitting of the model to the actual data.
Model testing to see if the model satisfies the user requirements for
the particular application.
21
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
Random Signal Modeling (3)
If we can develop a successful parametric model for the behavior
of a
signal, then we can use the model for various applications:
To achieve a better understanding of the physical mechanism
generating the signal.
To track changes in the source of the signal and help identify their
cause.
To synthesize artificial signals similar to the natural ones (e.g.,
speech, infrared backgrounds, natural scenes, data network traffic).
To extract parameters for pattern recognition applications (e.g.,
speech and character recognition).
To get an efficient representation of signals for data compression
(e.g., speech, audio, and video coding).
To forecast future signal behavior.
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
Chapter 0:
DiscreteTime Signals
zTransform.
Linear TimeInvariant Filters.
Discrete Fourier Transform (DFT).
23
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
Ztransform (1)
Discretetime signals
signals described as a time series, consisting
of sequence of uniformly spaced samples
whose varying amplitudes
carry the useful information content
of the signal.
Consider time series (sequence) {u(n)} or u(n) denoted by samples:
u(n), u(n1), u(n2),, n: discrete time.
Ztransform
of u(n):
z: complex variable.
ztransform pair: u(n) U(z).
Region of convergence (ROC): set of values of z
for which U(z) is
uniformly convergent.
= =
n
n
z n u n u z z U ) ( )] ( [ ) (
(1.1)
24
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
Ztransform (2)
Properties:
Linear transform (superposition):
ROC of (1.2): intersection of ROC of U
1
(z) and ROC of U
2
(z).
Timeshifting:
n
0
: integer.
ROC of (1.3): same as ROC of U(z).
Special case:
z
1
: unitdelay element.
Convolution theorem:
ROC of (1.4): intersection of ROC of U
1
(z) and ROC of U
2
(z).
) ( ) ( ) ( ) (
2 1 2 1
z bU z aU n bu n au + +
(1.2)
) ( ) ( ) ( ) (
0
0
z U z n n u z U n u
n
(1.3)
) ( ) 1 ( 1
1
0
z U z n u n
=
) ( ) ( ) ( ) (
2 1 2 1
z U z U i n u n u
i
=
(1.4)
25
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
Linear TimeInvariant (LTI) Filters (1)
Definition:
Impulse response h(n):
For arbitrary input v(n): convolution sum
) ( ) (
2 1
n bv n av +
LTI
Filter:
Linearity,
Timeinvariant
v(n) u(n)
) ( ) (
2 1
n bu n au +
) ( k n v ) ( k n u
LTI
Filter
h(n)
u(n)=h(n)
t=0
=
=
i
i n v i h n u ) ( ) ( ) (
(1.5)
26
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
LTI Filters (2)
Transfer function:
Applying ztransform to both side of (1.5)
H(z): transfer function
of the filter
When input sequence v(n) and output sequence u(n) are related
by difference equation of order N:
a
j
, b
j
: constant coefficients. Applying ztransform, obtaining:
h(n) v(n), H(z) u(n), V(z) U(z)
z V z H z U
= ) ( ) ( ) (
(1.6)
) (
) (
) (
z V
z U
z H =
(1.7)
= =
=
N
j
j
N
j
j
j n v b j n u a
0 0
) ( ) (
(1.8)
[
[
= = =
N
k
k
N
k
k
N
j
j
j
N
j
j
j
z d
z c
b
a
z b
z a
z V
z U
z H
1
1
1
1
0
0
0
0
) 1 (
) 1 (
) (
) (
) (
(1.9)
27
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
LTI Filters (3)
From (1.9), two distinct types of LTI filters:
Finiteduration impulse response (FIR) filters: d
k
=0 for all k.
allzero filter, h(n)
has finite duration.
Infiniteduration impulse response (IIR) filters: H(z)
has at least
one nonzero pole, h(n)
has infinite duration. When c
k
=0 for all
k allpole filter.
See examples of FIR and IIR filters in next two slides.
28
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
LTI Filters (4)
z
1
z
1
z
1
a
1
a
2
a
M1
a
M
+ + + +
v(n)
v(n1) v(n2)
v(nM+1)
v(nM)
u(n)
FIR filter
29
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
LTI Filters (5)
z
1
z
1
z
1
a
1
a
2
a
M1
a
M
+
+
+
+
.
.
.
v(n) u(n)
u(n1)
u(n2)
u(nM+1)
u(nM)
IIR filter
30
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
LTI Filters (6)
Causality and stability:
LTI filter is causal
if:
LTI filter is stable
if output sequence is bounded for all bounded
input sequences. From (1.5), necessary and sufficient condition:
0 for 0 ) ( < = n n h
(1.10)
<
= k
k h ) ( (1.11)
Im
Re
Unit circle
Region of
stability
zplane
A causal LTI filter is stable
if and only if all of the poles
of the filters transfer
function lie inside the unit
circle in the zplane
(See more in [3])
31
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
Discrete Fourier Transform (DFT) (1)
Fourier transform
of a sequence u(n) is obtained from ztransform
by setting z=exp(j2tf), f: real frequency variable.
When u(n) has a finite duration, its Fourier representation
discrete Fourier transform (DFT).
For numerical computation of DFT efficient fast Fourier
transform (FFT).
u(n): finiteduration sequence of length N, DFT of
u(n):
Inverse DFT (IDFT)
of U(k):
u(n), U(k): same length N Npoint DFT
1 ,..., 0 ,
2
exp ) ( ) (
1
0
=

.

\

=
=
N k
N
kn j
n u k U
N
n
t
(1.12)
1 ,..., 0 ,
2
exp ) (
1
) (
1
0
=

.

\

=
=
N n
N
kn j
k U
N
n u
N
k
t
(1.13)
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
Chapter 1:
Probability and Random Variables
33
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Probability and Random Variables
Axioms of Probability.
Repeated Trials.
Concepts of Random Variables.
Functions of Random Variables.
Moments and Conditional Statistics.
Sequences of Random Variables.
34
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Probability (1)
Probability theory deals with the study of random phenomena, which
under repeated experiments yield different outcomes that have
certain underlying patterns about them. The notion of an experiment
assumes a set of repeatable conditions that allow any number of
identical repetitions. When an experiment is performed under these
conditions, certain elementary events occur in different but
completely uncertain ways. We can assign nonnegative number
as the probability of the event in various ways:
Laplaces Classical Definition: The Probability of an event A
is
defined apriori without actual experimentation as
provided all these outcomes are equally likely.
i
), (
i
P
i
,
outcomes possible of number Total
to favorable outcomes of Number
) (
A
A P =
35
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Probability (2)
Relative Frequency Definition: The probability of an event A
is defined
as
where n
A
is the number of occurrences of A
and n
is the total number of
trials.
The axiomatic approach to probability, due to Kolmogorov, developed
through a set of axioms (below) is generally recognized as superior to the
above definitions, as it provides a solid foundation for complicated
applications.
n
n
A P
A
n
lim ) (
=
36
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Probability (3)
The totality of all known a priori, constitutes a set O, the set of all
experimental outcomes.
O
has subsets Recall that if A
is a subset of O, then
implies From A
and B, we can generate other related subsets
,
i
{ } , , , ,
2 1 k
= O
A e
. O e
. , , , C B A
, , , , B A B A B A
and
{ }
{ } B A B A
B A B A
e e =
e e =
and 
or 
{ } A A e = 
37
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Probability (4)
A B
B A
A B A
B A
A
If the empty set, then A
and B
are said to be mutually
exclusive (M.E).
A partition of O
is a collection of mutually exclusive subsets of O
such
that their union is O.
,  = B A
. and ,
1
O = =
=
i
i j i
A A A 
B
A
 = B A
1
A
2
A
n
A
i
A
A
j
A
38
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Probability (5)
DeMorgans Laws:
B A B A B A B A = = ;
A B
B A
A B
B A
A B
B A
A
B
Often it is meaningful to talk about at least some of the subsets of O
as
events, for which we must have mechanism to compute their probabilities.
Example: Consider the experiment where two coins are simultaneously
tossed. The various elementary events are
39
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Probability (6)
{ }. , , ,
4 3 2 1
= O
) , ( ), , ( ), , ( ), , (
4 3 2 1
T T H T T H H H = = = =
and
The subset is the same as Head has occurred at least
once
and qualifies as an event.
Suppose two subsets A and B
are both events, then consider
Does an outcome belong to
A or
B
Does an outcome belong to
A and
B
Does an outcome fall outside
A?
{ } , ,
3 2 1
= A
B A =
B A =
40
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Probability (7)
Thus the sets etc., also qualify as events. We shall
formalize this using the notion of a Field.
Field: A collection of subsets of a nonempty set O
forms a field F
if
Using (i) 
(iii), it is easy to show that etc., also belong to F.
For example, from (ii) we have and using (iii) this gives
applying (ii) again we get where we
have used De Morgans theorem in slide 22.
, , , , B A B A B A
. then , and If (iii)
then , If (ii)
(i)
F B A F B F A
F A F A
F
e e e
e e
e O
, , B A B A
, , F B F A e e
; F B A e , F B A B A e =
41
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Probability (8)
Axioms of Probability
For any event A, we assign a number P(A), called the probability of
the event A. This number satisfies the following three conditions that
act the axioms of probability.
(Note that (iii) states that if A
and B
are mutually exclusive (M.E.)
events)
). ( ) ( ) ( then , If (iii)
unity) is set whole the of ty (Probabili 1 ) ( (ii)
number) e nonnegativ a is ty (Probabili 0 ) ( (i)
B P A P B A P B A
P
A P
+ = =
= O
>

42
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Probability (9)
The following conclusions follow from these axioms:
a.
b.
c. Suppose A
and B are not mutually exclusive (M.E.)?
Conditional Probability and Independence
In N
independent trials, suppose N
A
, N
B
,
N
AB
denote the number of times
events A, B
and AB occur respectively, for large N
Among the N
A
occurrences of A, only N
AB
of them are also found among
the N
B
occurrences of B. Thus the ratio
). ( 1 ) or P( 1 ) P( ) ( ) P( A P A A A P A A = = + =
{ } . 0 =  P
). ( ) ( ) ( ) ( AB P B P A P B A P + =
. ) ( , ) ( , ) (
N
N
AB P
N
N
B P
N
N
A P
AB B A
~ ~ ~
) (
) (
/
/
B P
AB P
N N
N N
N
N
B
AB
B
AB
= =
43
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Probability (10)
is a measure of the event A given that B has already occurred. We
denote this conditional probability by
P(AB) = Probability of the event A given that B has occurred.
We define
provided As we show below, the above definition satisfies all
probability axioms discussed earlier.
Independence: A
and B
are said to be independent events, if
Then
Thus if A
and B
are independent, the event that B
has occurred does not
shed any more light into the event A. It makes no difference to A
whether B
has occurred or not.
,
) (
) (
)  (
B P
AB P
B A P =
. 0 ) ( = B P
). ( ) ( ) ( B P A P AB P =
). (
) (
) ( ) (
) (
) (
)  ( A P
B P
B P A P
B P
AB P
B A P = = =
44
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Probability (11)
Let
a union of n
independent events. Then by DeMorgans law
and using their independence
Thus for any A
as in (*)
a useful result.
,
3 2 1 n
A A A A A =
n
A A A A
2 1
=
. )) ( 1 ( ) ( ) ( ) (
1 1
2 1
[ [
= =
= = =
n
i
i
n
i
i n
A P A P A A A P A P
, )) ( 1 ( 1 ) ( 1 ) (
1
[
=
= =
n
i
i
A P A P A P
PILLAI
(*)
45
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Probability (12)
Bayes
theorem:
Although simple enough, Bayes
theorem has an interesting
interpretation: P(A) represents the apriori probability of the event A.
Suppose B
has occurred, and assume that A
and B
are not independent.
How can this new information be used to update our knowledge about
A? Bayes
rule above take into account the new information (B
has
occurred) and gives out the aposteriori probability of A
given B.
We can also view the event B as new knowledge obtained from a fresh
experiment. We know something about A as P(A). The new information
is available in terms of B. The
new information should be used to
improve our knowledge/understanding of A. Bayes
theorem gives the
exact mechanism for incorporating such new information.
) (
) (
)  (
)  ( A P
B P
A B P
B A P =
46
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Probability (13)
Let are pair wise disjoint
we have
A more general version of Bayes
theorem
n
A A A ,..., ,
2 1
,  =
j i
A A
= =
= =
n
i
i i
n
i
i
A P A B P BA P B P
1 1
). ( )  ( ) ( ) (
,
) ( )  (
) ( )  (
) (
) ( )  (
)  (
1
=
= =
n
i
i i
i i i i
i
A P A B P
A P A B P
B P
A P A B P
B A P
47
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Probability (14)
Input Output
, ) ( p A P
i
=
). ( ) ( ) ( ) ( ); ( ) ( ) (
3 2 1 3 2 1
A P A P A P A A A P A P A P A A P
j i j i
= =
. 3 1 = i
Example: Three switches connected in parallel operate independently.
Each switch remains closed with probability p. (a) Find the probability of
receiving an input signal at the output. (b) Find the probability that switch S
1
is open given that an input signal is received at the output.
Solution:
a. Let A
i
= Switch S
i
is closed. Then
Since switches operate independently, we have
1
s
2
s
3
s
48
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Probability (15)
).  (
1
R A P
.
3 3
2 2
3 3
) 1 )( 2 (
) (
) ( )  (
)  (
3 2
2
3 2
2
1 1
1
p p p
p p
p p p
p p p
R P
A P A R P
R A P
+
+
=
+
= =
Let R
= input signal is received at the output. For the event R
to
occur either switch 1 or switch 2 or switch 3 must remain closed, i.e.,
.
3 2 1
A A A R =
. 3 3 ) 1 ( 1 ) ( ) (
3 2 3
3 2 1
p p p p A A A P R P + = = =
Using results in slide 38,
b. We need From Bayes
theorem
Because of the symmetry of the switches, we also have
).  ( )  ( )  (
3 2 1
R A P R A P R A P = =
49
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Repeated Trials (1)
Consider two independent experiments with associated probability
models
(O
1
, F
1
, P
1
) and (O
2
, F
2
, P
2
). Let eO
1
, qeO
2
represent elementary events.
A joint performance of the two experiments produces an elementary events
e = (, q).
How to characterize an appropriate probability to this
combined event
?
Consider the Cartesian product space O
= O
1
O
2
generated from O
1
and O
2
such that if e O
1
and q e O
2
, then every e in O
is an ordered pair of the
form e = (, q). To arrive at a probability model we need to define the
combined trio (O, F, P).
Suppose AeF
1
and B e
F
2
. Then A
B
is the set of all pairs (, q), where
e
A and q e
B. Any such subset of O
appears to be a legitimate event for the
combined experiment. Let F denote the field composed of all such subsets A
B
together with their unions and compliments. In this combined
experiment, the probabilities of the events A O
2
and O
1
B are such that
). ( ) ( ), ( ) (
2 1 1 2
B P B P A P A P = O = O
50
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Repeated Trials (2)
, ) ( ) (
1 2
B A B A = O O
) ( ) ( ) ( ) ( ) (
2 1 1 2
B P A P B P A P B A P = O O =
) (
2 1
P P P
Moreover, the events A O
2
and O
1
B are independent for any A e
F
1
and
B e
F
2
. Since
then
for all A e
F
1
and B e
F
2
. This equation extends to a unique probability
measure on the sets in F and defines the combined trio (O, F, P).
Generalization: Given n
experiments and their associated
let
represent their Cartesian product whose elementary events are the ordered
ntuples
where Events in this combined space are of
the form
where
, , , ,
2 1 n
O O O
, 1 , and n i P F
i i
=
n
O O O = O
2 1
, , , ,
2 1 n
.
i i
O e
n
A A A
2 1
,
i i
F A e
51
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Repeated Trials (3)
1 2 1 1 2 2
( ) ( ) ( ) ( ).
n n n
P A A A P A P A P A =
{ } , , , ,
0 2 1
O e =
n
e
O e
i
If all these n
experiments are independent, and P(A
i
) is the probability of
the event A
i
in F
i
then as before
Example: An event A
has probability p
of occurring in a single trial. Find
the probability that A
occurs exactly k
times, k s
n
in n
trials.
Solution: Let (O, F, P)
be the probability model for a single trial. The
outcome of n
experiments is an ntuple
where every and . The event A
occurs
at trial # i , if Suppose A
occurs exactly k
times in e. Then k of
the belong to A, say and the remaining nk
are
contained in its compliment in
O O O = O
0
. A
i
e
i
, , , ,
2 1 k
i i i
. A
52
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Repeated Trials (4)
. ) ( ) ( ) ( ) ( ) ( ) (
}) ({ }) ({ }) ({ }) ({ }) , , , , , ({ ) (
2 1 2 1
0
k n k
k n k
i i i i i i i i
q p A P A P A P A P A P A P
P P P P P P
n k n k
= =
= =
_
_
e
N
e e e , , ,
2 1
. trials" in times exactly occurs "
2 1 N
n k A e e e =
Using independence, the probability of occurrence of such an e is given by
However the k
occurrences of A can occur in any particular location inside
e. Let represent all such events in which A occurs exactly
k
times. Then
But, all these e
i
s are mutually exclusive, and equiprobable. Thus
, ) ( ) (
) trials" in times exactly occurs ("
0
1
0
k n k
N
i
i
q Np NP P
n k A P
=
= = =
e e
53
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Repeated Trials (5)


.

\

=
=
+
=
k
n
k k n
n
k
k n n n
N
! )! (
!
!
) 1 ( ) 1 (
, , , 2 , 1 , 0 ,
) trials" in times exactly occurs (" ) (
n k q p
k
n
n k A P k P
k n k
n
=


.

\

=
=
= =
= =
 
= =

\ .
 
= =

\ .
But X
i
, X
j
are mutually exclusive. Thus
For a given
n
and
p
what is the most likely value of
k
?
From Figure, the most probable value of k
is that number which maximizes
P
n
(k) in Bernoulli formula. To obtain this value, consider the ratio
. 2 / 1 , 12 = = p n
) (k P
n
k
.
1 !
! )! (
)! 1 ( )! 1 (
!
) (
) 1 (
1 1
p
q
k n
k
q p n
k k n
k k n
q p n
k P
k P
k n k
k n k
n
n
+
=
+
=
+
56
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Repeated Trials (8)
.
1 !
! )! (
)! 1 ( )! 1 (
!
) (
) 1 (
1 1
p
q
k n
k
q p n
k k n
k k n
q p n
k P
k P
k n k
k n k
n
n
+
=
+
=
+
Thus, P
n
(k)
P
n
(k1), if k(1p)
(nk+1)p or k
(n+1)p . Thus
P
n
(k) as a
function of k
increases until
p n k ) 1 ( + =
if it is an integer, or the largest integer k
max
less than (n+1)p, it represents
the most likely number of successes (or heads) in n
trials.
Example: Parity check coding.
57
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Repeated Trials (9)
Approximate evaluation of
, [1],
with :
where G(x) is expressed in terms of the error function:
(see the table next slide)
If not only but also
then
Example: see Example 316, p. 52, [1].
( )
i j
P X X
( )
j
k n k
i j
k i
n
j np i np
P X X p q G G
k
npq npq
=
   
 
= =
 

 
\ .
\ . \ .
2
/ 2
0
1 1
( ) ( )
2
2
x
y
erf x e dy G x
t
= =
}
1 n > 1 np >
0
j
k n k
k
n
j np
p q G
k
npq
=
 
 
=



\ .
\ .
1 npq >
58
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Repeated Trials (10)
59
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Repeated Trials (11)
Poisson theorem: if np
is of order of one, np
a, p
0, n
,
then
Therefore,
Example: see Example 321, p. 56, [1].
!
!( )! !
k
k n k k n k a
n
n
n a
p q p q e
k k n k k
 
=

\ .
( )
( )
!
k j j
k n k np
i j
k i k i
n
np
P X X p q e
k k
= =
 
= ~

\ .
60
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Random Variables (1)
Let (O, F, P) be a probability model for an experiment, and X
a function
that maps every , e O to a unique point x
e
R,
the set of real numbers.
Since the outcome , is not certain, so is the value X(, ) = x.
Thus if B is some
subset of R, we may want to determine the probability of X(, ) e
B
. To
determine this probability, we can look at the set A = X
1
(B) e O that contains
all , e O that maps into B
under the function X.
O
R
) ( X
x
A
B
Obviously, if the set A = X
1
(B)
also belongs to the associated field F, then it
is an event and the probability of A
is well defined; in that case we can say
)). ( ( " ) ( " event the of y Probabilit
1
B X P B X
= e
61
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Random Variables (2)
However, X
1
(B)
may not always belong to F
for all B, thus creating
difficulties. The notion of random variable
(r.v) makes sure that the inverse
mapping always results in an event so that we are able to determine the
probability for any B
e
R.
Random Variable (r.v):
A finite single valued function X(.) that maps
the set of all experimental outcomes O
into the set of real numbers R
is said
to be a r.v, if the set { , ,
X(,)
x
} is an event e
F
for every x
in R.
Alternatively, X
is said to be a r.v, if X
1
(B)
e
F
where B
represents semi
definite intervals of the form {
< x
a} and all other sets that can be
constructed from these sets by performing the set operations of union,
intersection and negation any number of times. Thus if X
is a r.v, then
is an event for every x . (see Example 42, p. 65, [1])
{ } { } ) (  x X x X s = s
62
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Random Variables (3)
=
= =
)
`
s <
1
} {
1
n
a X a X
n
a
What about { a
< X
b
}, { X
= a
} ? Are they also events ?
In fact with b
> a ,
since { X
a
} and { X
b
} are events,
{ X
a
}
c
= { X
>
a
} is an event and hence
{ X
>
a
}
{ X
b
} = { a
< X
b
},
is also an event. Thus, { a
1/n
< X
a
}, is an event for every n.
Consequently,
is also an event. All events have well defined probability. Thus
the
probability of the event { , ,
X(,)
x
}
must depend on x. Denote
{ } 0 ) ( ) (  > = s x F x X P
X
which is referred to as the Probability Distribution Function
(PDF)
associated with the r.v
X. (see Example 44, p. 67, [1])
63
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Random Variables (4)
{ } ). ( 1 ) ( x F x X P
X
= >
{ } . ), ( ) ( ) (
1 2 1 2 2 1
x x x F x F x X x P
X X
> = s <
Distribution Function: If g(x) is a distribution function, then
(i) g(+) = 1; g() = 0
(ii) if x
1
< x
2
, then g(x
1
)
g(x
2
)
(iii) g(x
+
) = g(x) for all x.
It is shown that the PDF F
X
(x) satisfies these properties for any r.v
X.
Additional Properties of a PDF
(iv) if F
X
(x
0
) = 0 for some x
0
, then F
X
(x) = 0 for x
x
0
(v)
(vi)
(vii) ( ) ). ( ) ( ) (
= = x F x F x X P
X X
64
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Random Variables (5)
X
is said to be a continuoustype r.v
if its distribution function F
X
(x) is
continuous. In that case F
X
(x

) = F
X
(x) for all x, and from property (vii) we
get P{X = x} = 0.
If F
X
(x) is constant except for a finite number of jump discontinuities
(piecewise constant; steptype), then X
is said to be a discretetype r.v.
If x
i
is such a discontinuity point, then from (vii)
{ } ). ( ) (
= = =
i X i X i i
x F x F x X P p
65
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Random Variables (6)
Example: X is a r.v
such that X(,) = c, , e O. Find F
X
(x).
Solution: For x < c, {X(,)
x } = {}, so that F
X
(x) = 0, and for x > c,
{X(,)
x } = O, so that F
X
(x) = 1.
) (x F
X
x
c
1
Example: Toss a coin. O
= {H, T}.
Suppose the r.v
X
is such that X(T) = 0,
X(H) = 1. Find F
X
(x).
Solution: For x < 0, {X(,)
x } = {}, so that F
X
(x) = 0.
{ } { } { }
{ } { } . 1 ) ( that so , , ) ( , 1
, 1 ) ( that so , ) ( , 1 0
= O = = s >
= = = s < s
x F T H x X x
p T P x F T x X x
X
X
) (x F
X
x
1
q
1
66
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Random Variables (7)
{ }
{ } { } { }
{ } { } { }
{ } . 1 ) ( ) ( , 2
,
4
3
, , ) ( , , ) ( , 2 1
,
4
1
) ( ) ( ) ( ) ( , 1 0
, 0 ) ( ) ( , 0
= O = s >
= = = s < s
= = = = s < s
= = s <
x F x X x
TH HT TT P x F TH HT TT x X x
T P T P TT P x F TT x X x
x F x X x
X
X
X
X

Example: A fair coin is tossed twice, and let the r.v
X
represent the number
of heads. Find F
X
(x).
Solution: In this case O
= {HH, HT, TH, TT},
and X(HH) = 2, X(HT) =
X(TH) = 1, X(TT) = 0.
{ } . 2 / 1 4 / 1 4 / 3 ) 1 ( ) 1 ( 1 = = = =
X X
F F X P
) (x F
X
x
1
4 / 1
1
4 / 3
2
67
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Random Variables (8)
.
) (
) (
dx
x dF
x f
X
X
=
, 0
) ( ) (
lim
) (
0
>
A
A +
=
A
x
x F x x F
dx
x dF
X X
x
X
, ) ( ) (
=
i
i i X
x x p x f o
Probability Density Function (p.d.f):
The derivative of the distribution
function F
X
(x) is called the probability density function
f
X
(x) of the r.v
X.
Thus
Since
from the monotonenondecreasing
nature of F
X
(x), it follows that f
X
(x)
0
for all x. f
X
(x) will be a continuous function, if X
is a continuous type r.v.
However, if X
is a discrete type r.v, then its p.d.f
has the general form
) ( x f
X
x
i
x
i
p
68
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Random Variables (9)
. ) ( ) ( du u f x F
x
x X
}
=
, 1 ) ( =
}
+
dx x f
x
where x
i
represent the jumpdiscontinuity points in F
X
(x). In the figure,
f
X
(x) represents a collection of positive discrete masses, and it is known as
the probability mass function (p.m.f
) in the discrete case.
We also obtain
Since F
X
(+ ) = 1, it yields
which justifies its name as the density function. Further, we also get
) (x f
X
x
i
x
i
p
{ } . ) ( ) ( ) ( ) (
2
1
1 2 2 1
dx x f x F x F x X x P
x
x
X X X
}
= = s <
69
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Random Variables (10)
Thus the area under f
X
(x) in the interval (x
1
, x
2
) represents the probability in
the equation
{ } . ) ( ) ( ) ( ) (
2
1
1 2 2 1
dx x f x F x F x X x P
x
x
X X X
}
= = s <
) (x F
X
x
1
(a)
1
x
2
x
) (x f
X
(b)
x
1
x
2
x
Often, r.vs
are referred by their specific density functions 
both in the
continuous and discrete cases 
and in what follows we shall list a number
of them in each category.
Example: Transmitting a 3digit message over a noisy channel
70
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Random Variables (11)
.
2
1
) (
2 2
2 / ) (
2
o
to
=
x
X
e x f
,
2
1
) (
2 2
2 / ) (
2
}

.

\

= =
x
y
X
x
G dy e x F
o
to
o
Continuoustype Random Variables
1.
Normal (Gaussian):
X is said to be normal or Gaussian r.v, if
This is a bell shaped curve, symmetric around the parameter and its
distribution function is given by
where is often tabulated.
Since f
X
(x) depends on
two parameters and o
2
,
the notation X
~
N(, o
2
) will be used to
represent f
X
(x).
dy e x G
y
x
2 /
2
2
1
) (
}
=
t
) (x f
X
x
71
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Random Variables (12)
s s
=
otherwise. 0,
, ,
1
) (
b x a
a b
x f
X
2. Uniform: X
~
U(a,b), a < b, if
) (x f
X
x
a
b
a b
1
3. Exponential: X
~
c (), if
) (x f
X
x
4. Gamma: X
~
G(o, ), if o > 0,  > 0
>
I
=
otherwise. 0,
, 0 ,
) (
) (
/
1
x e
x
x f
x
X

o
o
 o
) ( x f
X
If o = n an integer, I(n) = (n1)!
>
=
otherwise. 0,
, 0 ,
1
) (
/
x e
x f
x
X
72
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Random Variables (13)
< <
=
otherwise. 0,
, 1 0 , ) 1 (
) , (
1
) (
1 1
x x x
b a
x f
b a
X

}
=
1
0
1 1
. ) 1 ( ) , ( du u u b a
b a

5. Beta: X
~
(a,b) if (a > 0, b > 0) ) ( x f
X
x
1
0
where the Beta function (a,b)
is defined as
6. ChiSquare: X
~
_
2
(n) if
>
I
=
otherwise. 0,
, 0 ,
) 2 / ( 2
1
) (
2 / 1 2 /
2 /
x e x
n
x f
x n
n
X
Note that _
2
(n) is the same as Gamma (n/2, 2)
x
) ( x f
X
73
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Random Variables (14)
>
=
otherwise. 0,
, 0 ,
) (
2 2
2 /
2
x e
x
x f
x
X
o
o
2
2 1 /
2
, 0
( )
( )
0 otherwise
X
m
m mx
m
x e x
f x
m
O
 
>

=
I O
\ .
7. Rayleigh: X
~
R(o
2
), if
) ( x f
X
x
8. Nakagami
m distribution:
9. Cauchy: X
~
C(o, ), if
. ,
) (
/
) (
2 2
+ < <
+
= x
x
x f
X
o
t o
) ( x f
X
x
74
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Random Variables (15)
. ,
2
1
) (
/  
+ < < =
x e x f
x
X
10. Laplace:
x
) ( x f
X
11. Students
tdistribution with n degrees of freedom:
( )
. , 1
) 2 / (
2 / ) 1 (
) (
2 / ) 1 (
2
+ < <


.

\

+
I
+ I
=
+
t
n
t
n n
n
t f
n
T
t
t
( )
T
f t
12. Fishers Fdistribution:
/ 2 / 2 / 2 1
( ) / 2
{( ) / 2}
, 0
( ) ( / 2) ( / 2) ( )
0 otherwise
m n m
m n
z
m n m n z
z
f z m n n mz
+
I +
>
= I I +
75
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Random Variables (16)
. ) 1 ( , ) 0 ( p X P q X P = = = =
. , , 2 , 1 , 0 , ) ( n k q p
k
n
k X P
k n k
=


.

\

= =
. , , 2 , 1 , 0 ,
!
) ( = = =
k
k
e k X P
k
Discretetype Random Variables
1.
Bernoulli: X
takes the values (0,1), and
2. Binomial: X
~
B(n,p) if
3. Poisson: X
~
P() if
) ( k X P =
1
2
n
k
) ( k X P =
4. DiscreteUniform
. , , 2 , 1 ,
1
) ( N k
N
k X P = = =
76
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Function of Random Variables (1)
Let X
be a r.v
defined on the model (O, F, P)
and suppose g(x) is a
function of the variable x. Define
Y = g(X)
Is Y
necessarily a r.v
? If so what is its PDF F
Y
(y) and pdf
f
Y
(y) ?
Consider some of the following functions to illustrate the technical details.
( ) ( ) . ) ( ) ( ) ( ) (

.

\

=

.

\

s = s + = s =
a
b y
F
a
b y
X P y b aX P y Y P y F
X Y
Example 1: Y
= aX
+ b
Suppose
a > 0,
and
.
1
) (

.

\

=
a
b y
f
a
y f
X Y
77
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Function of Random Variables (2)
( ) ( )
, 1
) ( ) ( ) ( ) (

.

\

=

.

\

> = s + = s =
a
b y
F
a
b y
X P y b aX P y Y P y F
X
Y
.
1
) (

.

\

=
a
b y
f
a
y f
X Y
If a < 0, then
and hence
For all a
.
 
1
) (

.

\

=
a
b y
f
a
y f
X Y
78
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Function of Random Variables (3)
( ) ( ). ) ( ) ( ) (
2
y X P y Y P y F
Y
s = s =
. 0 , 0 ) ( < = y y F
Y
( )
. 0 ), ( ) (
) ( ) ( ) ( ) (
1 2 2 1
> =
= s < =
y y F y F
x F x F x X x P y F
X X
X X Y
Example 2: Y
= X
2
If y
< 0 then the event {X
2
(,)
y} = , and hence
For y
> 0, from figure, the event {Y(,)
y} = {X
2
(,)
y} is equivalent to
{x
1
X(,)
x
2
}. Hence
By direct differentiation, we get
2
X Y =
X
y
2
x
1
x
( )
. otherwise , 0
, 0 , ) ( ) (
2
1
) (
> +
=
y y f y f
y
y f
X X
Y
79
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Function of Random Variables (4)
( ) ). (
1
) ( y U y f
y
y f
X Y
=
,
2
1
) (
2 /
2
x
X
e x f
=
t
If f
X
(x) represents an even function, then f
Y
(y) reduces to
In particular, if X
~
N(0,1), so that
then, we obtain the p.d.f
of Y
= X
2
to be
we notice that this equation represents a Chisquare r.v
with n = 1, since
I(1/2) = \t .
Thus,
if X is a Gaussian r.v
with
= 0, then Y
= X
2
represents a
Chisquare r.v
with one degree of freedom (n = 1).
). (
2
1
) (
2 /
y U e
y
y f
y
Y
=
t
80
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Function of Random Variables (5)
s +
s <
>
= =
. ,
, , 0
, ,
) (
c X c X
c X c
c X c X
X g Y
). ( ) ( ) ) ( ( ) 0 ( c F c F c X c P Y P
X X
= s < = =
( )
( ) . 0 ), ( ) (
) ) ( ( ) ( ) (
> + = + s =
s = s =
y c y F c y X P
y c X P y Y P y F
X
Y
Example 3: Let
In this case
For y
> 0, we have x
> c,
and Y(,) = X(,)
c, so that
Similarly y
< 0, if x
< 
c, and Y(,) = X(,) + c, so that
Thus
( )
( ) . 0 ), ( ) (
) ) ( ( ) ( ) (
< = s =
s + = s =
y c y F c y X P
y c X P y Y P y F
X
Y
( ), 0,
( ) [ ( ) ( )] ( ),
( ), 0.
X
Y X X
X
f y c y
f y F c F c y
f y c y
o
+ >
<
) ( X g
X
c
c
x
) (x F
X
( )
Y
F y
y
81
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Function of Random Variables (6)
s
>
= =
. 0 , 0
, 0 ,
) ( ); (
x
x x
x g X g Y
). 0 ( ) 0 ) ( ( ) 0 (
X
F X P Y P = s = =
( ) ( ) ). ( ) ( ) ( ) ( y F y X P y Y P y F
X Y
= s = s =
Example 4: Halfwave rectifier
In this case
and for y
> 0, since Y
= X
Thus
( ), 0,
( ) (0) ( ) 0, ( ) ( ) (0) ( ).
0, 0,
X
Y X X X
f y y
f y F y y f y U y F y
y
o o
>
= = = +
<
Y
X
82
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Function of Random Variables (7)
( ) y X g P y F
Y
s = )) ( ( ) (
Note: As a general approach, given Y
= g(X), first sketch the graph
y
= g(x), and determine the range space of y. Suppose a
< y
< b
is the range
space of y
= g(x). Then clearly for y
< a, F
Y
(y) = 0, and for y
> b, F
Y
(y) = 1,
so that F
Y
(y) can be nonzero only in a
< y
< b.
Next, determine whether there
are discontinuities in the range space of y. If so, evaluate P(Y(,) = y
i
) at these
discontinuities. In the continuous region of y, use the basic approach
and determine appropriate events in terms of the r.v
X
for every y. Finally,
we must have F
Y
(y) for 
< y < +
and obtain
( )
( ) in .
Y
Y
dF y
f y a y b
dy
= < <
83
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Function of Random Variables (8)
'
= =
i
i X
i
i X
i
x
Y
x f
x g
x f
dx dy
y f
i
). (
) (
1
) (
/
1
) (
However, if Y
= g(X) is a continuous function, it is easy to obtain f
Y
(y)
directly as
The summation index i
in this equation depends on y, and for every y
the
equation y = g(x
i
) must be solved to obtain the total number of solutions at
every y, and the actual solutions x
1
, x
2
,
all in terms of y.
For example, if Y
= X
2
, then for all y
> 0, x
1
= \y
and x
2
= +\y
represent
the two solutions for each y. Notice that the solutions x
i
are all in terms of
y
so that the right side of (+) is only a function of y. Moreover
(+)
y
dx
dy
x
dx
dy
i
x x
2 that so 2 = =
=
2
X Y =
X
y
2
x
1
x
84
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Function of Random Variables (9)
( )
> +
=
, otherwise , 0
, 0 , ) ( ) (
2
1
) (
y y f y f
y
y f
X X
Y
,
/ 1
1
that so
1
2
2 2
1
y
y dx
dy
x dx
dy
x x
= = =
=
Using (+), we obtain
which agrees with the result in Example 2.
Example 5: Y = 1/X. Find
f
Y
(y)
Here for every y, x
1
= 1/y
is the only solution, and
Then, from (+), we obtain
.
1 1
) (
2 

.

\

=
y
f
y
y f
X Y
85
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Function of Random Variables (10)
. ,
/
) (
2 2
+ < <
+
= x
x
x f
X
o
t o
In particular, suppose X
is a Cauchy r.v
with parameter o so that
In this case, Y = 1/X has the p.d.f
But this represents the p.d.f
of a Cauchy r.v
with parameter (1/o).
Thus if
X
~
C(o), then 1/X
~
C(1/o).
Example 6: Suppose f
X
(x) = 2x / t
2
, 0 < x
< t and Y
= sinX
. Determine f
Y
(y).
Since X
has zero probability of falling outside the interval (0, t), y
= sinx
has
zero probability of falling outside the interval (0, 1). Clearly
f
Y
(y) = 0 outside
this interval.
. ,
) / 1 (
/ ) / 1 (
) / 1 (
/ 1
) (
2 2 2 2 2
+ < <
+
=
+
= y
y y y
y f
Y
o
t o
o
t o
86
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Function of Random Variables (11)
2 2
1 sin 1 cos y x x
dx
dy
= = =
For any 0 < y
< 1, the equation y
= sinx
has an infinite number of solutions
x
1
, x
2
, x
3
,
(see the Figure below), where x
1
= sin
1
y
is the principal
solution. Moreover, using the symmetry we also get x
2
= t  x
1
etc. Further,
so that
. 1
2
y
dx
dy
i
x x
=
=
) ( x f
X
x
x
x y sin =
1
x
1
x
2
x
3
x
y
(a)
(b)
t
t
3
x
87
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Function of Random Variables (12)
2
0
1
( ) ( ).
1
Y X i
i
i
f y f x
y
+
=
=
=
< <
+
=

.

\

+
= +
=
otherwise. , 0
, 1 0 ,
1
2
1
) ( 2
2 2
1
1
) ( ) (
1
1
) (
2
2 2
1 1
2
2
2
1
2
2 1
2
y
y
y
x x
x x
y
x f x f
y
y f
X X Y
t
t
t
t t
) ( y f
Y
y
t
2
1
88
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Function of Random Variables (13)
, 2 , 1 , 0 ,
!
) ( = = =
k
k
e k X P
k
= = = = = +
89
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Moments (1)
}
+
= = =
. ) ( ) ( dx x f x X E X
X X
q
. ) (
) ( ) ( ) (
1
} }
= = =
= = = =
i
i i
i
i i
i
i i i
i
i i X
x X P x p x
dx x x p x dx x x p x X E X
_
o o q
}
+
=
=
b
a
b
a
b a
a b
a b x
a b
dx
a b
x
X E
2 ) ( 2 2
1
) (
2 2 2
Mean
or the Expected Value
of a r.v
X
is defined as
If X
is a discretetype r.v, then we get
Mean represents the average (mean) value of the r.v
in a very large number
of trials. For example if X
~
U(a,b), then
is the midpoint of the interval (a,b).
90
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Moments (2)
} }
= = =
0
/
0
, ) (
dy ye dx e
x
X E
y x
On the other hand if X
is exponential with parameter , then
implying that the parameter represents the mean value of the exponential
r.v.
Similarly if X
is Poisson with parameter , we get
Thus the parameter also represents the mean of the Poisson r.v.
.
! )! 1 (
! !
) ( ) (
0 1
1 0 0
= = =
=
= = = =
=
e e
i
e
k
e
k
k e
k
ke k X kP X E
i
i
k
k
k
k
k
k
k
91
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Moments (3)
. ) (
! )! 1 (
)! 1 (
)! 1 ( )! (
!
! )! (
!
) ( ) (
1 1
1
0 1
1 0 0
np q p np q p
i i n
n
np q p
k k n
n
q p
k k n
n
k q p
k
n
k k X kP X E
n i n i
n
i
k n k
n
k
k n k
n
k
k n k
n
k
n
k
= + =
=
=
=


.

\

= = =
= =
In a similar manner, if X
is binomial, then its mean is given by
Thus np
represents the mean of the binomial r.v.
For the normal r.v,
.
2
1
2
1
) (
2
1
2
1
) (
1
2 /
2
0
2 /
2
2 /
2
2 / ) (
2
2 2 2 2
2 2 2 2
to
to
to to
o o
o o
= + =
+ = =
} }
} }
+
+
+
+
_
_
dy e dy ye
dy e y dx xe X E
y y
y x
92
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Moments (4)
( )
} }
+
+
= = = =
. ) ( ) ( ) ( ) ( ) ( dx x f x g dy y f y X g E Y E
X Y Y
Given X
~
f
X
(x), suppose Y
= g(X) defines a new r.v
with p.d.f
f
Y
(y), the new
r.v
Y
has a mean
Y
given by
In the discrete case,
From the equations, f
Y
(y)
is not required to evaluate E(Y) for Y=g(X).
). ( ) ( ) (
i
i
i
x X P x g Y E = =
93
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Moments (5)
Example: Determine the mean of Y = X
2
where X
is a Poisson r.v.
( )
( ) .
! )! 1 (
! ! !
!
) 1 (
)! 1 (
! !
) (
2
0
1
1
1 0 0
0
1
1
1
2
0
2
0
2 2
+ = + =


.

\

+ =


.

\

+
=


.

\

+ =


.

\

+ =
+ =
=
= = = =
=
+
=
+
=
e e e
e
m
e e
i
e
e
i
i e
i i
i e
i
i e
k
k e
k
k e
k
e k k X P k X E
m
m
i
i
i
i
i i
i i
i
i
k
k
k
k
k
k
k
In general, E(X
k
) is known as the kth moment of r.v X. Thus if X
~
P(), its
second moment is given by the above equation.
PILLAI
94
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Moments (6)
( ) . 0 ] [
2
2
> = o X E
X
. 0 ) ( ) (
2 2
> =
}
+
dx x f x
X
X
o
For a r.v
X
with mean
, X 
represents the deviation of the r.v
from its
mean. Since this deviation can be either positive or negative, consider the
quantity (X 
)
2
and its average value E[(X 
)
2
] represents the average
mean square deviation of X
around its mean. Define
With g(X) = (X 
)
2
, we get
where o
X
2
is known as the variance
of the r.v
X, and its square root
is known as the standard deviation
of X. Note that the
standard deviation represents the root mean square spread of the
r.v
X
around its mean
.
2
) ( o = X E
X
95
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Moments (7)
( )
( ) ( )   . ) (
) ( 2 ) (
) ( 2 ) (
2
2
___
2 2 2 2
2
2
2 2 2
X X X E X E X E
dx x f x dx x f x
dx x f x x X Var
X X
X X
= = =
+ =
+ = =
} }
}
+
+
+
o
Alternatively, the variance can be calculated by
Example: Determine the variance of Poisson r.v
Thus for a Poisson r.v, mean and variance are both equal to its parameter
( ) .
2 2
___
2 2
2
o = + = = X X
X
96
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Moments (8)
( ) .
2
1
] ) [( ) (
2 / ) (
2
2
2
2 2
}
+
= = dx e x X E X Var
x o
to
} }
+
+
= =
2 / ) (
2
1
2
1
) (
2 2
dx e dx x f
x
X
o
to
}
+
=
2 / ) (
. 2
2 2
o t
o
dx e
x
Example: Determine the variance of the normal r.v
N(
, o
2
)
We have
Use of the identity
for a normal p.d.f. This gives
Differentiating both sides with respect to o ,
we get
or
}
+
=
2 / ) (
3
2
2
) (
2 2
t
o
o
dx e
x
x
( ) ,
2
1
2
2 / ) (
2
2
2 2
o
to
o
=
}
+
dx e x
x
97
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Moments (9)
1 ), (
___
> = = n X E X m
n n
n
] ) [(
n
n
X E =
( ) . ) ( ) (
) ( ] ) [(
0 0
0
k n
k
n
k
k n k
n
k
k n k
n
k
n
n
m
k
n
X E
k
n
X
k
n
E X E


.

\

=


.

\

=


.

\



.

\

= =
Moments: In general
are known as the moments
of the r.v
X, and
are known as the central moments
of X. Clearly, the mean
= m
1
,
and the variance o2 =
2
. It is easy to relate m
n
and
n
.
Infact
In general, the quantities E[(X 
a)
n
] are known as the generalized
moments of X about a, and E[X
n
] are known as the absolute moments
of X.
98
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Moments (10)
( )
}
+
= = u . ) ( ) ( dx x f e e E
X
jx jX
X
e e
e
= = u
k
jk
X
k X P e ). ( ) (
e
e
.
!
) (
!
) (
) 1 (
0 0
=
= = = = u
e e
e
e
e
j j
e e
k k
k j k
jk
X
e e e
k
e
e
k
e e
Characteristic Function of a r.v
X
is defined as
Thus u
X
(0) = 1,
and u
X
(e) s 1
for all e.
For discrete r.vs
the characteristic function reduces to
Example: If X
~
P(), then its characteristic function is given by
Example: If X is a binomial r.v, its characteristic function is given by
. ) ( ) ( ) (
0 0
n j
n
k
k n k j
n
k
k n k jk
X
q pe q pe
k
n
q p
k
n
e + =


.

\

=


.

\

= u
=
=
e e e
e
99
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Moments (11)
( )
.
!
) (
! 2
) (
) ( 1
!
) (
!
) (
) (
2
2
2
0 0
+ + + + + =
=
(
= = u
=
=
k
k
k
k
k
k
k
k
k
jX
X
k
X E
j
X E
j X jE
k
X E
j
k
X j
E e E
e e e
e
e
e
e
.
) ( 1
) ( or ) (
) (
0 0 = =
c
u c
= =
c
u c
e e
e
e
e
e
X X
j
X E X jE
,
) ( 1
) (
0
2
2
2
2
=
c
u c
=
e
e
e
X
j
X E
The characteristic function can be used to compute the mean, variance and
other higher order moments of any r.v
X.
Taking the first derivative with respect to e,
and letting it to be equal to zero,
we get
Similarly, the second derivative gives
and repeating this procedure k
times, we obtain the kth
moment of X
to be
. 1 ,
) ( 1
) (
0
>
c
u c
=
=
k
j
X E
k
X
k
k
k
e
e
e
100
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Moments (12)
,
) (
e
e
e
e
j e
X
je e e
j
=
c
u c
( ), ) (
) (
2 2
2
2
e e
e
e
e e
j e j e
X
e j e je e e
j j
+ =
c
u c
Example: If X
~
P(),
then
so that E(X) = .
The second derivative gives
so that
, ) (
2 2
+ = X E
101
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Two Random Variables (1)
( )
}
= = s <
2
1
, ) ( ) ( ) ( ) (
1 2 2 1
x
x
X X X
dx x f x F x F x X x P
( ) . ) ( ) ( ) ( ) (
2
1
1 2 2 1
}
= = s <
y
y
Y Y Y
dy y f y F y F y Y y P
  ? ) ) ( ( ) ) ( (
2 1 2 1
= s < s < y Y y x X x P
Let X
and Y
denote two random variables (r.v) based on a probability
model (O, F, P). Then
and
What about the probability that the pair of r.vs
(X,Y) belongs to an arbitrary
region D? In other words, how does one estimate, for example,
Towards this, we define the joint probability distribution function
of X
and Y
to be
where x
and y
are arbitrary real numbers.
 
, 0 ) , (
) ) ( ( ) ) ( ( ) , (
> s s =
s s =
y Y x X P
y Y x X P y x F
XY
102
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Two Random Variables (2)
. 1 ) , ( , 0 ) , ( ) , ( = + + = =
XY XY XY
F x F y F
( ) ). , ( ) , ( ) ( , ) (
1 2 2 1
y x F y x F y Y x X x P
XY XY
= s s <
( )
). , ( ) , (
) , ( ) , ( ) ( , ) (
1 1 2 1
1 2 2 2 2 1 2 1
y x F y x F
y x F y x F y Y y x X x P
XY XY
XY XY
+
= s < s <
Properties
(i)
(ii)
(iii)
This is the probability that (X,Y) belongs to the rectangle R
0
.
1
y
2
y
1
x
2
x
X
Y
0
R
( ) ). , ( ) , ( ) ( , ) (
1 2 2 1
y x F y x F y Y y x X P
XY XY
= s < s
103
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Two Random Variables (3)
.
) , (
) , (
2
y x
y x F
y x f
XY
XY
c c
c
=
. ) , ( ) , (
dudv v u f y x F
x y
XY XY
} }
=
. 1 ) , (
=
} }
+
+
dxdy y x f
XY
Joint probability density function (Joint p.d.f): By definition, the
joint p.d.f
of X
and Y
is given by
and hence we obtain the useful formula
Using property (i), we also get
The probability that (X,Y) belongs to an arbitrary region D is given by
( )
} }
e
= e
D y x
XY
dxdy y x f D Y X P
) , (
. ) , ( ) , (
104
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Two Random Variables (4)
. ) , ( ) ( ), , ( ) ( y F y F x F x F
XY Y XY X
+ = + =
. ) , ( ) ( , ) , ( ) (
} }
+
+
= = dx y x f y f dy y x f x f
XY Y XY X
= = = = =
j j
ij j i i
p y Y x X P x X P ) , ( ) (
Marginal Statistics: In the context of several r.vs, the statistics of each
individual ones are called marginal statistics. Thus F
X
(x) is the marginal
probability distribution function
of X, and f
X
(x) is the marginal p.d.f
of X.
It is interesting to note that all marginals
can be obtained from the joint p.d.f.
In fact
Also
If X
and Y
are discrete r.vs, then p
ij
= P(X
= x
i
, Y
= y
i
) represents their
joint p.d.f, and their respective marginal p.d.fs
are given by
The joint P.D.F and/or the joint p.d.f
represent complete information about the r.vs,
and their marginal p.d.fs
can be evaluated from the joint p.d.f. However, given
marginals, (most often) it will not be possible to compute the joint p.d.f.
= = = = =
i i
ij j i j
p y Y x X P y Y P ) , ( ) (
105
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Two Random Variables (5)
( ) ) ) ( ( ) ) ( ( ) ) ( ( ) ) ( ( y Y P x X P y Y x X P s s = s s
) ( ) ( ) , ( y F x F y x F
Y X XY
=
). ( ) ( ) , ( y f x f y x f
Y X XY
=
Independence of r.vs
Definition: The random variables X
and Y
are said to be statistically
independent if the events {X(,) e
A} and {Y(,) e
B}
are independent
events for any two sets A
and B
in x
and y
axes respectively. Applying the
above definition to the events {X(,)
x} and {Y(,)
y}, we conclude that,
if the r.vs
X
and Y
are independent, then
i.e.,
or equivalently, if X
and Y
are independent, then we must have
If X
and Y
are discretetype r.vs
then their independence implies
. , all for ) ( ) ( ) , ( j i y Y P x X P y Y x X P
j i j i
= = = = =
106
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Two Random Variables (6)
otherwise. , 0
, 1 0 , 0 ,
) , (
2
\

+ =
= =
}
} }
+
x x dy ye ye x
dy e y x dy y x f x f
y y
y
XY X
. 0 ,
2
) , ( ) (
2
1
0
< < = =
}
y e
y
dx y x f y f
y
XY Y
The equations in previous slide give us the procedure to test for
independence. Given f
XY
(x,y), obtain the marginal p.d.fs
f
X
(x) and f
Y
(x)
and examine whether these equations (last two equations) are valid. If so,
the r.vs
are independent, otherwise they are dependent.
Example: Given
Determine whether X
and Y
are independent.
We have
Similarly,
In this case, and hence X
and Y
are independent r.vs.
), ( ) ( ) , ( y f x f y x f
Y X XY
=
107
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Function of Two Random Variables (1)
Given two random variables X
and Y
and a function g(x,y), we form a new
random variable Z
= g(X, Y).
Given the joint p.d.f
f
XY
(x, y), how does one obtain f
Z
(z) the p.d.f
of Z ?
Problems of this type are of interest from a practical standpoint. For example,
a receiver output signal usually consists of the desired signal buried in noise,
and the above formulation in that case reduces to Z = X + Y.
It is important to know the statistics of the incoming signal for proper
receiver design. In this context, we shall analyze problems of the following
type:
) , ( Y X g Z =
Y X +
) / ( tan
1
Y X
Y X
XY
Y X /
) , max( Y X
) , min( Y X
2 2
Y X +
108
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Function of Two Random Variables (2)
( ) ( )  
} }
e
=
e = s = s =
z
D y x
XY
z Z
dxdy y x f
D Y X P z Y X g P z Z P z F
,
, ) , (
) , ( ) , ( ) ( ) ( Start with:
where D
z
in the XY
plane represents the region such that g(x, y)
z
is
satisfied. To determine F
Z
(z), it is enough to find the region D
z
for every z,
and then evaluate the integral there.
Example 1: Z = X + Y. Find f
Z
(z).
since the region D
z
of the xy
plane where x+y
z
is the shaded area in the figure
to the left of the
line x+y
= z. Integrating over the horizontal strip
along the xaxis first (inner integral) followed by
sliding that strip along the yaxis from 
to +
(outer integral) we cover the entire shaded area.
( )
} }
+
=
=
= s + =
, ) , ( ) (
y
y z
x
XY Z
dxdy y x f z Y X P z F
y z x =
x
y
109
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Function of Two Random Variables (3)
}
=
) (
) (
. ) , ( ) (
z b
z a
dx z x h z H
( ) ( )
}
c
c
+ =
) (
) (
.
) , (
), (
) (
), (
) ( ) (
z b
z a
dx
z
z x h
z z a h
dz
z da
z z b h
dz
z db
dz
z dH
( , )
( ) ( , ) ( , ) 0
( , ) .
z y z y
XY
Z XY XY
XY
f x y
f z f x y dx dy f z y y dy
z z
f z y y dy
+ +
+
c c
   
= = +
 
c c
\ . \ .
=
} } } }
}
We can find f
Z
(z) by differentiating F
Z
(z) directly. In this context, it is useful
to recall the differentiation rule due to Leibnitz. Suppose
Then
Using above equations, we get
If X
and Y
are independent, f
XY
(x,y) = f
X
(x)f
Y
(y), then we get
. ) ( ) ( ) ( ) ( ) (
} }
+
=
+
=
= =
x
Y X
y
Y X Z
dx x z f x f dy y f y z f z f
(Convolution!)
110
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Function of Two Random Variables (4)
( ) . ) , ( ) (
2 2
2 2
} }
s +
= s + =
z Y X
XY Z
dxdy y x f z Y X P z F
. ) , ( ) (
2
2
} }
=
=
=
z
z y
y z
y z x
XY Z
dxdy y x f z F
Example 2: X
and Y
are independent normal r.vs
with zero mean and
common variance o
2
. Determine f
Z
(z) for Z
= X
2
+ Y
2
.
We have
But, X
2
+ Y
2
z
represents the area of a circle with radius \z
and hence
This gives after repeated differentiation
( ) . ) , ( ) , (
2
1
) (
2 2
2
}
=
+
=
z
z y
XY XY
Z
dy y y z f y y z f
y z
z f
x
y
z
z Y X = +
2 2
z
z
(*)
111
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Function of Two Random Variables (5)
2 2
2 2 2
( ) 2 ( )( ) ( ) 1
2(1 )
2
1
( , ) ,
2 1
, ,   1.
X X Y Y
X Y
X Y
x r x y y
r
XY
X Y
f x y e
r
x y r
o o
o o
to o
 
+ 

\ .
=
\

+
=
o
o o
o
o to
y rxy x
r
XY
e
r
y x f
2
2 2 2
2
2
/ 2
( ) / 2
2 2
2 2 0
/ 2
/2
/ 2
2 2
0
1 1 1
( ) 2
2
2
cos 1
( ),
2
cos
z
z z
z y y
Z
y z
z
z
e
f z e dy dy
z y z y
e z
d e U z
z
o
o
o
t
o
to to
u
u
to o
u
+
=
 
= =

\ .
= =
} }
}
Moreover, X
and Y
are said to be jointly normal (Gaussian) distributed, if
their joint p.d.f
has the following form:
with zero mean
with r
= 0 and o
1
= o
2
= o, then direct substitution into (*), we get
. sinu z y = with
112
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Function of Two Random Variables (6)
.
2 2
Y X Z + =
. ) , ( ) (
2 2
2 2
} }
=
=
=
z
z y
y z
y z x
XY Z
dxdy y x f z F
( ) . ) , ( ) , ( ) (
2 2 2 2
2 2
}
=
z
z
XY XY Z
dy y y z f y y z f
y z
z
z f
Thus, if X and Y are independent zero mean Gaussian r.vs
with common
variance
o
2
then
X
2
+ Y
2
is an exponential r.vs
with parameter
2o
2
.
Example 3: Let Find f
Z
(z)
From the figure, the present case corresponds to a
circle with radius
z
2
. Thus
Now suppose X
and Y
are independent Gaussian as in Example 2, we obtain
x
y
z
z Y X = +
2 2
z
z
), (
cos
cos 2
1 2
2
1
2 ) (
2 2 2 2
2 2 2 2 2 2
2 /
2
/2
0
2 /
2
0
2 2
2 /
2
0
2 / ) (
2
2 2
z U e
z
d
z
z
e
z
dy
y z
e
z
dy e
y z
z
z f
z z
z
z
z
y y z
Z
o
t
o
o o
o
u
u
u
to
to to
+
= =
=
}
} }
Rayleigh distribution!
113
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Function of Two Random Variables (7)
2 2
Y X W + =
? tan
1

.

\

=
Y
X
u
. ,
1
/ 1
) (
2
< <
+
= u
u
u f
U
t
Thus, if
W= X
+ iY, where X and Y are real, independent normal r.vs
with
zero mean and equal variance, then the r.v
has a Rayleigh
density. Wis said to be a complex Gaussian r.v
with zero mean, whose real
and imaginary parts are independent r.vs
and its magnitude has Rayleigh
distribution.
What about its phase
Clearly, the principal value of u lies in the interval (t/2, +t/2). If we let
U
= tan u = X/Y
, then it is shown that U
has a Cauchy distribution with
As a result
< <
=
+
= =
. otherwise , 0
, 2 / 2 / , / 1
1 tan
/ 1
) sec / 1 (
1
) (tan
 / 
1
) (
2 2
t u t t
u
t
u
u
u
u
u U
f
du d
f
114
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Function of Two Random Variables (8)
,
2
1
) , (
2 2 2
2 / ] ) ( ) [(
2
o
to
Y X
y x
XY
e y x f
+
=
( )
,
2
2
2
) (
2
0
2
2 / ) (
/2 3
/2
/ ) cos(
/2
/2
/ ) cos(
2
2 / ) (
/2
/2
/ ) cos( / ) cos(
2
2 / ) (
2 2 2
2 2
2 2 2
2 2
2 2 2

.

\

=

.

\

+ =
+ =
+
+
+
} }
}
o
to
u u
to
u
to
o
t
t
o  u
t
t
o  u
o
t
t
o  u o  u
o
z
I
ze
d e d e
ze
d e e
ze
z f
z
z z
z
z z
z
Z
To summarize, the magnitude and phase of a zero mean complex Gaussian r.v
has Rayleigh and uniform distributions respectively. Interestingly, as we will
see later, these two derived r.vs
are also independent of each other!
Example 4:
Redo example 3, where X
and Y
have nonzero means
X
and
Y
respectively.
Since
Similar to example 3, we obtain the Rician
probability density function
to be
115
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Function of Two Random Variables (9)
} }
= =
t
u q
t
 u q
u
t
u
t
q
0
cos
2
0
) cos(
0
1
2
1
) ( d e d e I
2 2
cos , sin , , cos , sin ,
X Y X Y
x z y z u u   = = = + = =
where
and
is the modified Bessel function of the first kind and zeroth
order.
Thus, if X and Y have nonzero means
X
and
Y
, respectively. Then
is said to be a Rician
r.v. Such a scene arises in fading multipath
situation where there is a dominant constant component (mean) in
addition to
a zero mean Gaussian r.v. The constant
component may be the line of sight
signal and the zero mean Gaussian r.v
part could be due to random multipath
components adding up incoherently
(see diagram). The envelope of such
a signal is said to have a Rician
p.d.f.
2 2
Y X Z + =
Rician
Output
Line of sight
signal (constant)
a
Multipath/Gaussian
noise
116
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Joint Moments (1)
. ) ( ) (
}
+
= = dz z f z Z E
Z Z
( ) ( ) ( , ) ( , ) .
Z XY
E Z z f z dz g x y f x y dxdy
+ + +
= =
} } }
[ ( , )] ( , ) ( , ).
i j i j
i j
E g X Y g x y P X x Y y = = =
o o
o o
+ + +
=
124
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Joint Moments (9)
.
2 1
n
X X X
Y
n
+ + +
=
To summarize, any two linear combinations of jointly Gaussian random
variables (independent or dependent) are also jointly Gaussian r.vs.
Gaussian r. vs
are also interesting because of the following result:
Central Limit Theorem: Suppose X
1
, X
2
,, X
n
are a set of zero mean
independent, identically distributed (i.i.d) random variables
with some
common distribution. Consider their scaled sum
Then asymptotically (as n
)
Linear
operator
Gaussian
input
Gaussian
output
). , 0 (
2
o N Y
125
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Joint Moments (10)
The central limit theorem states that a large sum of independent random
variables each with finite variance tends to behave like a normal random
variable. Thus the individual p.d.fs
become unimportant to analyze the
collective sum behavior. If we model the noise phenomenon as the
sum of
a large number of independent random variables (eg: electron motion in
resistor components), then this theorem allows us to conclude that noise
behaves like a Gaussian r.v.
General:
Given n
independent r.vs
X
i
, we form their sum: X = X
1
++X
n
X
is a r.v
with mean
q = q
1
++q
n
and the variance
o
2
= o
1
2
++ o
n
2
The
central limit theorem states that under certain general conditions, the
distribution of X
approaches a normal distribution
with mean q and
variance
o
2
as
n increases
( )
x
F x G
q
o
 
~

\ .
126
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Joint Moments (11)
Example: See example 815, p.215, [4]
and
http://en.wikipedia.org/wiki/Illustration_of_the_central_limit_theorem
Furthermore, if the r.vs
X
i
are continuous type, the p.d.f
f(x) of X
approaches a nomal
density:
2 2
( ) / 2
2
1
( )
2
x
f x e
q o
to
~
127
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Joint Moments (12)
Normal distribution =0, o=1
128
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Joint Moments (13)
Central limit theorem with sum of 2 identically distributed r.vs
(normal dis. =0,o=1)
129
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Joint Moments (14)
Central limit theorem with sum of 1000 identically distributed r.vs
(normal dis. =0,o=1)
130
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Joint Moments (15)
Uniform distribution [0, 2]
131
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Joint Moments (16)
Central limit theorem with sum of 2 identically distributed r.vs
(uniform dis. [0,2])
132
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Joint Moments (17)
Central limit theorem with sum of 1000 identically distributed r.vs
(uniform dis. [0,2])
133
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Joint Moments (18)
Rayleigh distribution, o=1
134
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Joint Moments (19)
Central limit theorem with sum of 2 identically distributed r.vs
(Rayleigh dis. ,o=1)
135
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
1. Joint Moments (20)
Central limit theorem with sum of 1000 identically distributed r.vs
(Rayleigh dis. ,o=1)
136
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
2. Review of Stochastic Processes and Models
Mean, Autocorrelation.
Stationarity.
Deterministic Systems.
Discrete Time Stochastic Process.
Stochastic Models.
137
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
2. Review of Stochastic Processes: Introduction (1)
Let , denote the random outcome of an experiment. To every such
outcome suppose a waveform X(t, ,)
is assigned. The collection of such
waveforms form a stochastic process. The set of {,
k
} and the time index t
can be continuous or discrete (countably
infinite or finite) as well. For fixed
,
i
e
S (the set of all experimental outcomes), X(t, ,) is a specific time
function. For fixed t, X
1
= X(t
1
, ,
i
)
is a random variable. The ensemble of
all such realizations X(t, ,)
over
time represents the stochastic
process
(or random process)
X(t) (see the figure).
For example:
X(t) = acos(e
0
t +)
where  is a uniformly distributed
random variable in (0, 2t),
represents a stochastic process.
t
1
t
2
t
) , (
n
t X
) , (
k
t X
) , (
2
t X
) , (
1
t X
.
.
.
) , ( t X
0
138
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
2. Review of Stochastic Processes: Introduction (2)
} ) ( { ) , ( x t X P t x F
X
s =
If X(t) is a stochastic process, then for fixed t, X(t) represents a random
variable. Its distribution function is given by
Notice that F
X
(x, t) depends on t, since for a different t, we obtain a
different random variable. Further
represents the firstorder probability density function of the process X(t).
For t
= t
1
and t
= t
2
, X(t) represents two different random variables
X
1
= X(t
1
) and X
2
= X(t
2
), respectively. Their joint distribution is given by
and
represents the secondorder density function of the process X(t).
dx
t x dF
t x f
X
X
) , (
) , ( =
A
} ) ( , ) ( { ) , , , (
2 2 1 1 2 1 2 1
x t X x t X P t t x x F
X
s s =
2
1 2 1 2
1 2 1 2
1 2
( , , , )
( , , , )
X
X
F x x t t
f x x t t
x x
c
=
c c
A
139
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
2. Review of Stochastic Processes: Introduction (3)
Similarly, f
X
(x
1
, x
2
,
x
n
, t
1
, t
2
,
t
n
) represents the nth order density function
of the process X(t). Complete specification of the stochastic process X(t)
requires the knowledge of f
X
(x
1
, x
2
,
x
n
, t
1
, t
2
,
t
n
) for all t
i
, i
= 1, 2, n
and
for all n. (an almost impossible task in reality!).
Mean of a stochastic process:
represents the mean value of a process X(t). In general, the mean of
a process can depend on the time index t.
Autocorrelation
function of a process X(t) is defined as
and it represents the interrelationship between the random variables
X
1
= X(t
1
) and X
2
= X(t
2
) generated from the process X(t).
( ) { ( )} ( , )
X
t E X t x f x t dx
+
= =
}
A
* *
1 2 1 2 1 2 1 2 1 2 1 2
( , ) { ( ) ( )} ( , , , )
XX X
R t t E X t X t x x f x x t t dx dx = =
} }
A
140
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
2. Review of Stochastic Processes: Introduction (4)
*
1
*
2 1 2
*
2 1
)}] ( ) ( { [ ) , ( ) , ( t X t X E t t R t t R
XX XX
= =
. 0 }  ) ( { ) , (
2
> = t X E t t R
XX
= =
>
n
i
n
j
j i j i
t t R a a
XX
1 1
*
. 0 ) , (
Properties:
(i)
(ii)
(Average instantaneous power)
(iii) R
XX
(t
1
, t
2
) represents a nonnegative definite function, i.e., for any
set
of constants {a
i
}
n
i=1
this follows by noticing that
The function
represents the autocovariance
function of the process X(t).
. ) ( for 0 }  {
1
2
=
= >
n
i
i i
t X a Y Y E
) ( ) ( ) , ( ) , (
2
*
1 2 1 2 1
t t t t R t t C
X X XX XX
=
141
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
2. Review of Stochastic Processes: Introduction (5)
). 2 , 0 ( ~ ), cos( ) (
0
t e U t a t X + =
, 0 } {sin sin } {cos cos
)} {cos( )} ( { ) (
0 0
0
= =
+ = =
e e
e
E t a E t a
t aE t X E t
X
}
= = =
t
t
2
0
}. {sin 0 cos } {cos since
2
1
E d E
Example:
This gives
Similarly,
). ( cos
2
)} 2 ) ( cos( ) ( {cos
2
)} cos( ) {cos( ) , (
2 1 0
2
2 1 0 2 1 0
2
2 0 1 0
2
2 1
t t
a
t t t t E
a
t t E a t t R
XX
=
+ + + =
+ + =
e
e e
e e
142
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
2. Review of Stochastic Processes: Stationary (1)
Stationary processes exhibit statistical properties that are invariant to
shift in the time index. Thus, for example, secondorder stationarity
implies
that the statistical properties of the pairs {X(t
1
) , X(t
2
)} and {X(t
1
+c),X(t
2
+c)}
are the same for any
c. Similarly, firstorder stationarity
implies that the
statistical properties of X(t
i
) and X(t
i
+c) are the same for any c.
In strict terms, the statistical properties are governed by the joint probability
density function. Hence a process is nthorder StrictSense Stationary
(S.S.S)
if
for any
c, where the left side represents the joint density function of the
random variables X
1
= X(t
1
), X
2
= X(t
2
), , X
n
= X(t
n
), and the right side
corresponds to the joint density function of the random variables X
1
=X(t
1
+c),
X
2
= X(t
2
+c), , X
n
= X(t
n
+c). A process X(t) is said to be strictsense
stationary
if (*) is true for all t
i
, i
= 1, 2, , n; n
= 1, 2,
and any c.
) , , , , , ( ) , , , , , (
2 1 2 1 2 1 2 1
c t c t c t x x x f t t t x x x f
n n n n X X
+ + +
(*)
143
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
2. Review of Stochastic Processes: Stationary (2)
) , ( ) , ( c t x f t x f
X X
+
) ( ) , ( x f t x f
X X
=
[ ( )] ( ) , E X t x f x dx a constant.
+
= =
}
For a firstorder strict sense stationary process, from (*) we have
for any c. In particular c
=
t
gives
i.e., the firstorder density of X(t) is independent of t. In that case
Similarly, for a secondorder strictsense stationary process
we have
from (*)
for any c. For c
=
t
2
we get
i.e., the second order density function of a strict sense stationary process
depends only on the difference of the time indices t
1
t
2
= t .
) , , , ( ) , , , (
2 1 2 1 2 1 2 1
c t c t x x f t t x x f
X X
+ +
) , , ( ) , , , (
2 1 2 1 2 1 2 1
t t x x f t t x x f
X X
144
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
2. Review of Stochastic Processes: Stationary (3)
= )} ( { t X E
In that case the autocorrelation function is given by
i.e., the autocorrelation function of a second order strictsense stationary
process depends only on the difference of the time indices t .
However, the basic conditions for the first and second order stationarity
are
usually difficult to verify. In that case, we often resort to a looser definition
of stationarity, known as WideSense Stationarity
(W.S.S). Thus, a process
X(t) is said to be WideSense Stationary
if
(i)
and
(ii)
*
1 2 1 2
*
1 2 1 2 1 2 1 2
*
1 2
( , ) { ( ) ( )}
( , , )
( ) ( ) ( ),
XX
X
XX XX XX
R t t E X t X t
x x f x x t t dx dx
R t t R R
t
t t
=
= =
= = =
} }
A
A
), ( )} ( ) ( {
2 1 2
*
1
t t R t X t X E
XX
=
145
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
2. Review of Stochastic Processes: Stationary (4)
i.e., for widesense stationary processes, the mean is a constant and the
autocorrelation function depends only on the difference between the time
indices. Strictsense stationarity
always implies widesense stationarity.
However, the converse is not true
in general, the only exception being the
Gaussian process. If X(t) is a Gaussian process, then
widesense stationarity
(w.s.s)
strictsense stationarity
(s.s.s).
146
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
2. Review of Stochastic Processes: Systems (1)
A deterministic system
transforms each input waveform X(t, ,
i
) into an
output waveform Y(t, ,
i
) = T[X(t, ,
i
)] by operating only on the time variable t.
A stochastic system
operates on both the variables
t and , .
Thus, in deterministic system, a set of realizations at the input corresponding
to a process X(t) generates a new set of realizations Y(t, ,) at the output
associated with a new process Y(t).
Our goal is to study the output process statistics in terms of the input
process statistics and the system function.
] [ T
) (t X
) (t Y
t
t
) , (
i
t X
) , (
i
t Y
147
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
2. Review of Stochastic Processes: Systems (2)
Deterministic Systems
Systems with Memory
Timeinvariant
systems
Linear systems
LinearTime Invariant
(LTI) systems
Memoryless Systems
)] ( [ ) ( t X g t Y =
)] ( [ ) ( t X L t Y =
Timevarying
systems
. ) ( ) (
) ( ) ( ) (
}
}
+
+
=
=
t t t
t t t
d t X h
d X t h t Y ( ) h t ( ) X t
LTI system
148
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
2. Review of Stochastic Processes: Systems (3)
Memoryless Systems:
The output Y(t) in this case depends only on the present value of the input X(t).
i.e.,
)} ( { ) ( t X g t Y =
Memoryless
system
Memoryless
system
Memoryless
system
Strictsense
stationary input
Widesense
stationary input
X(t) stationary
Gaussian with
) (t
XX
R
Strictsense
stationary output.
Need not
be
stationary in
any sense.
Y(t) stationary,but
not
Gaussian with
). ( ) ( t q t
XX XY
R R =
149
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
2. Review of Stochastic Processes: Systems (4)
)}. ( { ), ( ) ( X g E R R
XX XY
'
= = q t q t
)}. ( { )} ( { )} ( ) ( {
2 2 1 1 2 2 1 1
t X L a t X L a t X a t X a L + = +
Theorem:
If X(t) is a zero mean stationary Gaussian process, and
Y(t) = g[X(t)], where g(.) represents a nonlinear memoryless
device, then
where g(x) is the derivative with respect to x.
Linear Systems:
L[.]
represents a linear system if
Let Y(t) = L{X(t)} represent the output of a linear system.
TimeInvariant System: L[.] represents a timeinvariant system if
i.e., shift in the input results in the same shift in the output
also.
If L[.] satisfies both conditions for linear and timeinvariant, then it
corresponds to a linear timeinvariant (LTI) system.
) ( )} ( { )} ( { ) (
0 0
t t Y t t X L t X L t Y = =
150
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
2. Review of Stochastic Processes: Systems (5)
LTI systems can be uniquely represented in terms of their output
to a
delta function
then
where
LTI
) (t o ) (t h
Impulse
Impulse
response of
the system
t
) ( t h
Impulse
response
LTI
}
}
+
+
=
=
) ( ) (
) ( ) ( ) (
t t t
t t t
d t X h
d X t h t Y
arbitrary
input
t
) (t X
t
) (t Y
) (t X ) (t Y
}
+
=
) ( ) ( ) ( t t o t d t X t X
151
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
2. Review of Stochastic Processes: Systems (6)
Thus
Then, the mean of the output process is given by
. ) ( ) ( ) ( ) (
)} ( { ) (
} ) ( ) ( {
} ) ( ) ( { )} ( { ) (
} }
}
}
}
+
+
+
+
+
= =
=
=
= =
t t t t t t
t t o t
t t o t
t t o t
d t X h d t h X
d t L X
d t X L
d t X L t X L t Y
By Linearity
By Timeinvariance
). ( ) ( ) ( ) (
} ) ( ) ( { )} ( { ) (
t h t d t h
d t h X E t Y E t
X X
Y
 = =
= =
}
}
+
+
t t t
t t t
152
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
2. Review of Stochastic Processes: Systems (7)
Similarly, the crosscorrelation function between the input and output
processes is given by
). ( ) , (
) ( ) , (
) ( )} ( ) ( {
} ) ( ) ( ) ( {
)} ( ) ( { ) , (
2
*
2 1
*
2 1
*
2 1
*
2 1
2
*
1 2 1
t h t t R
d h t t R
d h t X t X E
d h t X t X E
t Y t X E t t R
XX
XX
XY
 =
=
=
=
=
}
}
}
+
+
+
o o o
o o o
o o o
*
*
153
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
2. Review of Stochastic Processes: Systems (8)
), ( ) , (
) ( ) , (
) ( )} ( ) ( {
} ) ( ) ( ) ( {
)} ( ) ( { ) , (
1 2 1
2 1
2 1
2
*
1
2
*
1 2 1
t h t t R
d h t t R
d h t Y t X E
t Y d h t X E
t Y t Y E t t R
XY
XY
YY
 =
=
=
=
=
}
}
}
+
+
+
  
  
  
*
Finally the output autocorrelation function is given by
or
In particular, if X(t) is widesense stationary, then we have
X
(t)
=
X
Also
). ( ) ( ) , ( ) , (
1 2
*
2 1 2 1
t h t h t t R t t R
XX YY
  =
. ), ( ) ( ) (
) ( ) ( ) , (
2 1
*
*
2 1 2 1
t t R h R
d h t t R t t R
XY XX
XX XY
= =  =
+ =
}
+
t t t t
o o o
A
154
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
2. Review of Stochastic Processes: Systems (9)
). ( ) ( ) (
, ) ( ) ( ) , (
2 1
2 1 2 1
t t t
t   
YY XY
XY YY
R h R
t t d h t t R t t R
=  =
= =
}
+
Thus X(t) and Y(t) are jointly w.s.s. Further, the output autocorrelation
simplifies to
or
). ( ) ( ) ( ) (
*
t t t t h h R R
XX YY
  =
155
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
2. Review of Stochastic Processes: Systems (10)
LTI system
h(t)
Linear system
widesense
stationary process
strictsense
stationary process
Gaussian process
(also
stationary)
widesense
stationary process.
strictsense
stationary process
Gaussian process
(also stationary)
) (t X
) (t Y
LTI system
h(t)
) (t X
) (t X
) (t Y
) (t Y
(a)
(b)
(c)
156
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
2. Review of Stochastic Processes: Systems (11)
), ( ) ( ) , (
2 1 1 2 1
t t t q t t R
WW
= o
). ( ) ( ) , (
2 1 2 1
t o o q t t q t t R
WW
= =
[ ( )] ( ) ,
W
E N t h d t t
+
=
}
a constant
White Noise Process: W(t) is said to be a white noise process if
i.e., E[W(t
1
) W*(t
2
)] = 0 unless t
1
= t
2
.
W(t) is said to be widesense stationary (w.s.s) white noise if
E[W(t)] = constant, and
If W(t) is also a Gaussian process (white Gaussian process), then all of
its samples are independent random variables (why?).
For w.s.s. white noise input W(t), we have
) ( ) ( ) (
) ( ) ( ) ( ) (
*
*
t t t
t t t o t
q h qh
h h q R
nn
=  =
  = and
where . ) ( ) ( ) ( ) ( ) (
* *
}
+
+ =  = o t o o t t t d h h h h
157
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
2. Review of Stochastic Processes: Systems (12)
Thus the output of a white noise process through an LTI system represents
a (colored) noise process.
Note: White noise need not be Gaussian.
White
and Gaussian
are two different concepts!
White noise
W(t)
LTI
h(t)
Colored noise
( ) ( ) ( ) N t h t W t = 
158
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
2. Review of Stochastic Processes: Discrete Time (1)
)} ( ) ( { ) , (
)} ( {
2
*
1 2 1
T n X T n X E n n R
nT X E
n
=
=
*
2 1 2 1
2 1
) , ( ) , (
n n
n n R n n C =
A discrete time stochastic process (DTStP)
X
n
= X(nT) is a sequence of
random variables. The mean, autocorrelation and autocovariance functions
of a discretetime process are gives by
respectively. As before strict sense stationarity
and widesense stationarity
definitions apply here also. For example, X(nT) is wide sense stationary if
and
i.e., R(n
1
, n
2
) = R(n
1
n
2
) = R*(n
2
n
1
).
constant a nT X E , )} ( { =
* *
[ {( ) } {( ) }] ( )
n n
EX k nT X k T Rn r r
+ = = =
A
159
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
2. Review of Stochastic Processes: Discrete Time (2)
) ( ) ( ) (
*
n h n R n R
XX XY
 =
) ( ) ( ) ( n h n R n R
XY YY
 =
If X(nT) represents a widesense stationary input to a discretetime
system {h(nT)}, and Y(nT) the system output, then as before the cross
correlation function satisfies
and the output autocorrelation function is given by
or
Thus widesense stationarity
from input to output is preserved for discrete
time systems also.
). ( ) ( ) ( ) (
*
n h n h n R n R
XX YY
  =
160
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
2. Review of Stochastic Processes: Discrete Time (3)
=
=
1
0
1
M
n
n
X
M
Mean
(or
ensemble average
) of a stochastic process is obtained by
averaging across
the process, while time average is obtained by averaging
along
the process as
where
M is total number of time samples used in estimation.
Consider widesense DTStP
X
n
, time average converge
to ensemble average
if:
the process X
n
is said mean ergodic
(in the meansquare error sense).
( )   0
lim
2
=
M
161
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
2. Review of Stochastic Processes: Discrete Time (4)
Let define an (M1)observation vector
x
n
represents elements of time
series X
n
, X
n1
, .X
nM+1
x
n
= [X
n
, X
n1
, .X
nM+1
]
T
An (MM)correlation matrix
R (using condition of widesense stationary)
can be defined as
Superscript H denotes Hermitian
transposition.
 
(
(
(
(
+ +
= =
) 0 ( ) 2 ( ) 1 (
) 2 ( ) 0 ( ) 1 (
) 1 ( ) 1 ( ) 0 (
R M R M R
M R R R
M R R R
E
H
n n
. . .
x x R
162
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
2. Review of Stochastic Processes: Discrete Time (5)
(
(
(
(
=
 

) 0 ( ) 2 ( ) 1 (
) 2 ( ) 0 ( ) 1 (
) 1 ( ) 1 ( ) 0 (
R M R M R
M R R R
M R R R
. . .
R
Properties:
(i) Correlation matrix R
of stationary DTStP
is Hermitian: R
H
= R or
R(k)=R*(k).Therefore:
(ii)
Matrix
R of stationary DTStP
is Toeplitz: all elements on main
diagonal are equal, elements on any subdiagonal
are also equal.
(iii) Let x
be arbitrary (nonzero) (M1)complexvalued vector, then
x
H
Rx
>
0 (nonnegative definition).
(iv) If x
B
n
is backward arrangement of x
n
:
Then, E[x
B
n
x
BH
n
] = R
T
T
n M n M n
B
n
x x x ] ,..., , [
2 1 + +
= x
163
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
2. Review of Stochastic Processes: Discrete Time (6)
(
=
+
M
H
M
R
R r
r
R
) 0 (
1
(v) Consider correlation matrices R
M
and R
M+1
, corresponding to M
and M+1
observations of process, these matrices are related by
or
where r
H
= [R(1), R(2),, R(M)] and r
BT
= [r(M), r(M+1),, r(1)]
(
=

+
) 0 (
1
R
BT
B
M
M
r
r R
R
164
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
2. Review of Stochastic Processes: Discrete Time (7)
1 ,..., 0 ), ( ) exp( ) ( = + = = N n n v n j n u u
n
e o
 
=
=
=

0 0
0
) ( ) (
2
k
k
k n v n v E
v
o
Consider a time series consisting of complex sine wave plus noise:
Sources of sine wave and noise are independent. Assumed that v(n) has
zero mean and autocorrelation function given by
For a lag k, autocorrelation function of process u(n):
 
=
= +
= =

0 ,
0 ,
) ( ) ( ) (
2
2
2
k e
k
k n u n u E k r
k j
v
e
o
o o
165
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
2. Review of Stochastic Processes: Discrete Time (8)
(
(
(
(
(
(
(
+
+
+
=
e e
e
e
e e
o
1
1 )) 2 ( exp( )) 1 ( exp(
)) 2 ( exp(
1
1 ) exp(
)) 1 ( exp( ) exp(
1
1
2
. . .
M j M j
M j j
M j j
R
Therefore, correlation matrix of u(n):
where : signal to noise ratio (SNR).
2
2
v
o
o
=
166
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
2. Review of Stochastic Models (1)
, ) ( ) ( ) (
0 1
= =
+ =
q
k
k
p
k
k
k n W b k n X a n X
0
0 0
( ) ( ) , 1
p q
k k
k k
k k
X z a z W z b z a
= =
=
Consider an input
output representation
where X(n) may be considered as the output of a system {h(n)} driven by
the input W(n). Using z
transform, it gives
or
represents the transfer function
of the associated system response {h(n)} so
that
A
1 2
0 1 2
1 2
0
1 2
( ) ( )
( ) ( )
( ) ( ) 1
q
q
k
p
k
p
b b z b z b z
X z B z
H z h k z
W z A z a z a z a z
=
+ + + +
= = = =
+ + + +
h(n)
W(n)
X(n)
. ) ( ) ( ) (
0
=
=
k
k W k n h n X
167
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
2. Review of Stochastic Models (2)
Notice that the transfer function H(z) is rational with p
poles and q
zeros that
determine the model order
of the underlying system. The output X(n)
undergoes regression
over p
of its previous values
and at the same time a
moving average
based on W(n), W(n1), , W(nq) of the input over (q + 1)
values is added to it, thus generating an Auto Regressive
Moving Average
(ARMA
(p, q)) process X(n).
Generally the input {W(n)} represents a sequence of uncorrelated random
variables of zero mean and constant variance so that
If in addition, {W(n)} is normally distributed
then the output {X(n)} also
represents a strictsense stationary normal process.
If q = 0, then X(n) represents an Auto Regressive
AR(p) process (allpole
process), and if p = 0, then X(n) represents an Moving Average
MA(q)
process (allzero process).
). ( ) (
2
n n R
W WW
o o =
168
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
2. Review of Stochastic Models (3)
) ( ) 1 ( ) ( n W n aX n X + =
=
0
1
1
1
) (
n
n n
z a
az
z H
1   , ) ( < = a a n h
n
AR(1) process:
An AR(1) process has the form
and the corresponding system transfer function
provided  a  < 1. Thus
represents the impulse response of an AR(1) stable system. We get the
output autocorrelation sequence of an AR(1) process to be
The normalized (in terms of R
XX
(0)) output autocorrelation sequence is
given by
2
 
2
0
  2 2
1
} { } { ) ( ) (
a
a
a a a a n n R
n
k
k k n n n
W W W XX
= =   =
=
+
o o o o
. 0   ,
) 0 (
) (
) (
 
> = = n a
R
n R
n
n
XX
XX
X
169
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
2. Review of Stochastic Models (4)
) ( ) ( ) ( n V n X n Y + =
) (
1
) ( ) ( ) ( ) ( ) (
2
2
 
2
2
n
a
a
n n R n R n R n R
V W
V XX VV XX YY
n
o o o
o o
+
=
+ = + =
It is instructive to compare an AR(1) model discussed above by
superimposing a random component to it, which may be an error term
associated with observing a first order AR process X(n). Thus
where X(n) ~ AR(1), and V(n) is an uncorrelated random sequence with
zero mean and variance o
2
V
that is also uncorrelated with {W(n)}. Then,
we obtain the output autocorrelation of the observed process Y(n) to be
so that its normalized version is given by
where
 
1 0
( )
( )
(0) 1, 2,
YY
Y
YY
n
n
R n
n
R c a n
= =
A
. 1
) 1 (
2 2 2
2
<
+
=
a
c
V W
W
o o
o
170
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
2. Review of Stochastic Models (5)
The results demonstrate the effect of superimposing an error sequence on
an AR(1) model. For nonzero lags, the autocorrelation of the observed
sequence {Y(n)} is reduced by a constant factor compared to the original
process {X(n)}. The superimposed error sequence V(n) only affects the
corresponding term in Y(n) (term by term). However, a particular term in
the input sequence
W(n) affects X(n) and Y(n) as well as all subsequent
observations.
n
k
) ( ) ( k k
Y X
>
1 ) 0 ( ) 0 ( = =
Y X
0
171
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
2. Review of Stochastic Models (6)
) ( ) 2 ( ) 1 ( ) (
2 1
n W n X a n X a n X + + =
1
2
2
1
1
1
0
2
2
1
1
1 1 1
1
) ( ) (
=
= =
z
b
z
b
z a z a
z n h z H
n
n
2 ), 2 ( ) 1 ( ) ( , ) 1 ( , 1 ) 0 (
2 1 1
> + = = = n n h a n h a n h a h h
AR(2) Process:
An AR(2) process has the form
and the corresponding transfer function is given by
so that
and in term of the poles of the transfer function, we have
that represents the impulse response of the system. We also have
and H(z) stable implies
0 , ) (
2 2 1 1
> + = n b b n h
n n
. , 1
1 2 2 1 1 2 1
a b b b b = + = +
, ,
2 2 1 1 2 1
a a = = +
. 1   , 1  
2 1
< <
172
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
2. Review of Stochastic Models (7)
Further, the output autocorrelations satisfy the recursion
and hence their normalized version is given by
By direct calculation using, the output autocorrelations are given by
) 2 ( ) 1 (
)} ( ) ( {
)} ( )] 2 ( ) 1 ( {[
)} ( ) ( { ) (
2 1
*
*
2 1
*
+ =
+ +
+ + + =
+ =
n R a n R a
m X m n W E
m X m n X a m n X a E
m X m n X E n R
XX XX
XX
0
1 2
( )
( ) ( 1) ( 2).
(0)
XX
X X X
XX
R n
n a n a n
R
= = +
A

.

\

=
 + =
 =   =
=
2
2
*
2
2
2
*
2 1
*
2
*
2 1
2
*
1
*
1 2
*
1
2
1
*
1
2
1
2
0
* 2
* 2 *
  1
) (  
1
) (
1
) (
  1
) (  
) ( ) (
) ( ) ( ) ( ) ( ) ( ) (
o
o
o
n n n n
k
b b b b b b
k h k n h
n h n h n h n h n R n R
W
W
W WW XX
173
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
2. Review of Stochastic Models (8)
n n
XX
XX
X
c c
R
n R
n
*
2 2
*
1 1
) 0 (
) (
) ( + = =
Then, the normalized output autocorrelations may be expressed as
174
Dept. of Telecomm. Eng.
Faculty of EEE
SSPre2012
DHT, HCMUT
2. Review of Stochastic Models (9)
0
( )
k
k
k
H z h z
=
=
Mult mai mult decât documente.
Descoperiți tot ce are Scribd de oferit, inclusiv cărți și cărți audio de la editori majori.
Anulați oricând.