Topic 6-Linear Systems and Power Spectra

(Chapters 9 and 10 in Papoulis)

1.5 weeks

Linear Filtering of a Random Signals

Power Spectrum Analysis

(Deterministic) Systems with Stochastic Inputs

A deterministic system1 transforms each input waveform X (t, i ) into
an output waveform Y (t, i ) = L[X(t, i )] by operating only on the
time variable t. Thus a set of realizations at the input corresponding
to a process X(t) generates a new set of realizations {Y (t, )} at the
output associated with a new process Y(t).
Y (t, i )

X (t, i )
X (t )
!!
!

L[]

(t )
!Y!
!
t

Our goal is to study the output process statistics in terms of the input
process statistics and the system function.
1A stochastic

system on the other hand operates on both the variables t and

PILLAI

Deterministic System (not necessarily linear)

Deterministic Systems
Memoryless Systems
Y (t ) = g[ X (t )]
Time-varying
systems

Systems with Memory

Time-Invariant
systems

Linear systems
Y (t ) = L[ X (t )]

Linear-Time Invariant
(LTI) systems

X (t )

h (t )
LTI system

Y (t ) = h(t ) X ( )d

(6-1)

= h( ) X (t )d . = X (t ) h(t )

Memoryless Systems (not necessarily linear)

The output Y(t) in this case depends only on the
present value of the input X(t). i.e., Y (t ) = g{ X .(t )}
Strict-sense
stationary input

Memoryless
system

Strict-sense
stationary output.

Wide-sense
stationary input

Memoryless
system

Need not be
stationary in
any sense.

X(t) stationary
Gaussian

Memoryless
system
Note: the filters are not random

Y(t) stationary, but

not necessarily
Gaussian
4

LTI Systems: WSS input good enough

X (t )
wide-sense
stationary process

LTI system
h(t)

Y (t )
wide-sense
stationary process.

(a)

X (t )
strict-sense
stationary process

LTI system
h(t)
(b)

X (t )
Gaussian
process (also
stationary)

Linear system

Y (t )
strict-sense
stationary process
Y (t )
Gaussian process
(also stationary)

(c)
5

Linear Time-Invariant (LTI) Continuous Systems

Time-Invariant System
Shift in the input results in the same shift in the output.

Y (t ) = L{X (t )} L{X (t t0 )} = Y (t t0 )

(6-2)

Linear Time-Invariant System: linear system with time-invariant property.

Linear system has two important properties: superposition and
homogeneity

L{a1 X (t1 ) + a2 X (t2 )} = a1 L{ X (t1 )} + a2 L{ X (t2 )}.

(t )dt = 1

(t )

h(t ) = L{ (t )}
LTI

h(t )

h (t )

(6-3)

Impulse
response of
the system
t

Impulse

Impulse
response

Linear (LTI) Filtering of a Random Signal

Y (t )
X (t )
t

X (t )

Y (t )

LTI

Y (t ) = h(t ) X ( )d

arbitrary
input

X (t ) = X ( ) (t )d

(6-4a)

(6-1)

= h( ) X (t )d

Y (t ) = L{ X (t )} = L{ X ( ) (t )d }

Note: the filter

is not random.

= L{ X ( ) (t )d }

By Linearity

= X ( ) L{ (t )}d
+

By Time-invariance
+

= X ( )h (t )d = h( ) X (t )d .

(6-4b)
7

Mean of the Output

Expectation and linear filtering are both linear operations

that are interchangeable.

Not all linear operations are interchangeable (e.g., matrix
multiplication).
The mean of the output of a LTI filter is the convolution of
the mean of the input with the the filter impulse response:

$
E[Y (t)] = E & h( )X(t )d ') =
%
(
= E[X(t)] h(t)

h( )E[X(t )]d
(6-5)

Question: suppose the input is a deterministic function s(t)

plus a zero-mean random input X(t), what is the mean of

the filter output.
8

Filter Output With a Wide Sense Stationary (WSS) Input

If the input to an LTI filter with impulse response h(t) is a WSS

process X(t), the output Y(t) has the following properties:

Y(t) is a WSS process with expected value and autocorrelation

function
(6-6)
Y = E[Y (t )] = X h( )d

h(u) h(v)R

RY ( ) =

( + v u)dudv

(6-7)

X(t) and Y(t) are jointly WSS and have I/O cross-correlation

RXY ( ) =

h(u)RX ( u)du = RX ( ) h(t)

(6-8)

The output autocorrelation is related to the I/O cross-correlation

by

RY ( ) =

h(w)RXY ( w)dw = RXY ( ) h( ) (6-9)

9

Proof:

To show that
we recall that

RY ( ) =

Y (t) =

h(u) h(v)R

( + v u)dudv

(6-10)

X(t u)h(u)du = h(t u)X(u)du

So,

RY ( ) = E[Y (t + )Y (t)] = E #$

X(t + u)X(t v)(u)h(u)h)v)du dv%&

E [ X(t + u)X(t v)]h(u)h)v)dudv

= R ( + v u)h(u)h)v)dudv
=

(6-11)

See if you can prove the other relationships on the previous chart
on your own !. If not, see Papoulis.
10

Linear Filtering of a Discrete Time Random Signal

If the input to a discrete-time LTI filter with impulse response hn is
a WSS random sequence, Xn, the output Yn has the following
properties.
Yn is a WSS random sequence with expected value and autocorrelation
function

Y = E[Yn ] = X
RY [n] =

hn .

(6-12)

n =

h h R
i

[n + i j ].

(6-13)

i = j =

Yn and Xn are jointly WSS with I/O cross-correlation

RXY [n] =

h R
i

[n i ].

(6-14)

i =

The output autocorrelation is related to the I/O cross-correlation by

RY [n] =

i =

RXY [n i ].

(6-15)
11

Power Spectrum Analysis

Definition: Fourier Transform

Functions g (t ) and G ( f ) are a Fourier transform pair if

G ( f ) = g (t )e

j 2ft

dt ,

g (t ) = G ( f )e j 2ft df

(6-16)

Definition: Power Spectral Density

The power spectral density function of the WSS stochastic

process X(t) is defined as
1 $
2&
SX ( f ) lim
E % XT ( f ) '
T 2T
(6-17)
2
&
1 $ T
j 2 ft
= lim
E * X(t)e
dt +.
T
T 2T
%
'
12

Power Spectrum Density (PSD) and the Autocorrelation

Function are a Fourier Transform Pair

If X(t) is a WSS stochastic process, RX ( ) and SX ( f ) are the

Fourier transform pair
SX ( f ) =

RX ( )e

j 2 f

d ,

RX ( ) =

SX ( f )e j 2 f df

(6-18)

We will use the symbol to denote the Fourier Transform

This is a major result, called the Wiener-Khinchin Theorem.
On the next charts we will prove this important result.

13

Proof that the PSD and Autocorrelation are Fourier Pairs

For a deterministic signal x(t), the spectrum is well defined: If X ( )
represents its Fourier transform, i.e., if
+

(6-19)

X ( ) = x(t )e jt dt ,

then | X ( ) |2 represents its energy spectrum. This follows from

Parseval s theorem since the signal energy is given by
+

x (t )dt = 21
2

2
|
X
(

)
|
d = E.

(6-20)

Thus | X ( ) |2 represents the signal energy in the band ( , + )

(see Figure below).
2
| X ( )|

X (t )

Energy in ( , + )

PILLAI

14

However for stochastic processes, the Fourier relationship

generates a sequence of random variables for every . Moreover,
for a stochastic process, E{| X(t) |2} represents the ensemble average
power (instantaneous energy) at the instant t.
To obtain the spectral distribution of power versus frequency for
stochastic processes, it is best to avoid infinite intervals to begin with,
and start with a finite interval ( T, T ) so that the Fourier Transform
of a process X(t) based over the interval ( T, T ) is given by
T

(6-21)
X T ( ) = T X (t )e jt dt
so that the energy per unit time, or the power spectral density (PSD)
2
(6-22)
| X T ( ) |2 1 T
j t
=
X (t )e dt

T
2T
2T
represents the power distribution associated with that realization based
on ( T, T ). Notice that (6-22) represents a random variable for every
and its ensemble average gives, the average power distribution
based on ( T, T ). Thus the power spectral density is given by
15

# | X T ( ) |2 $ 1 T T
j ( t1 t2 )
*
PT ( ) = E %
=
E
{
X
(
t
)
X
(
t
)}
e
dt1dt2
&
1
2

T T
' 2T
( 2T
1 T T
j ( t1 t2 )
=
R
(
t
,
t
)
e
dt1dt2
(6-23)

T T XX 1 2
2T
represents the power distribution of X(t) based on ( T, T ). For wide
sense stationary (WSS) processes, it is possible to further simplify
(6-23). Thus if X(t) is assumed to be WSS, then RXX (t1 , t2 ) = RXX (t1 t2 )
and (6-23) simplifies to

1
PT ( ) =
2T

j ( t1 t2 )
R
(
t

t
)
e
dt1dt2 .
T T XX 1 2

(6-24)

Let = t1 t2 and using this change of variables we have

1 2T
j
PT ( ) =
R
(

)
e
(2T | |)d

2 T XX
2T
2T

= 2T RXX ( )e j (1 2|T| )d 0
to be the power distribution of the WSS. process X(t) over
( T, T ). Finally letting T in (6-25), we obtain

(6-25)
PILLAI

16

S XX ( ) = lim PT ( ) = RXX ( )e j d 0
T

(6-26)

to be the power spectral density of the WSS process X(t). Notice that

"$$(')!!!!!!!!!!!!*.+!!!!!!!!!!-$$(.)0 (6-27)
i.e., the autocorrelation function and the power spectrum of a WSS
Process form a Fourier transform pair, a relation known as the
Wiener-Khinchin Theorem. From (6-27), the inverse formula gives

RXX ( ) = 21

j
S
(

)
e
d
XX

(6-28)

and in particular for = 0, we an expression for the total power

1
2

2
(

)
d

=
R
(0)
=
E{|
X
(t)
|
}= P
XX
XX

(6-29)

From (6-29), the area under S XX ( ) represents the total power of the
process X(t), and hence S XX ( ) truly represents the power
spectrum. (See the figure on the next chart).
PILLAI

17

S XX ( )

S XX ( ) represents the power

in the band ( , + )

The nonnegative-definiteness property of the autocorrelation function

translates into the nonnegative property for its Fourier
transform (power spectrum), since from (6-28)
n

ai a RXX (ti t j ) = ai a 21
i =1 j =1

*
j

And it follows that

i =1 j =1

= 21

*
j

S XX ( )e
n

XX

( ) i =1 ai e

j ti

j ( ti t j )

d 0.

RXX ( ) nonnegative - definite S XX ( ) 0.

(6-30)
(6-31)
PILLAI

18

Since X(t) is a real WSS process, then RXX ( ) = RXX ( ) so that

+
S XX ( ) = RXX ( )e j d
+

= RXX ( ) cos d

= 2 0 RXX ( ) cos d = S XX ( ) 0

(6-32)

so that the power spectrum is an even function, (in addition to being

real and nonnegative).

PILLAI

19

Power Spectral Density Summary

For a WSS stochastic process, X(t), the Power Spectral Density

(PSD), Sx(f), and the Autocorrelation function, RX() are a Fourier

Transform pair.
This is celebrated formula and is known as the Wiener-Khinchin
Theorem
The PSD, Sx(f), is a real-valued function with the following
properties:

(a) SX ( f ) 0 for all f

2
"
S
(
f
)df
=
E
X
X
# (t)$% = RX (0)
(c) SX ( f ) = SX ( f )

(b)

(6-33)

20

Noise in Communication Systems

Thermal noise is described by a zero-mean Gaussian random

process, n(t).
If the PSD is constant, then it is called white noise..

[w/Hz]

Power spectral
density

Autocorrelation
function
Probability density function
21

White Noise Process and LTI Systems

W(t) is said to be wide-sense stationary (w.s.s) white noise if
E[W(t)] = constant, and

RWW (t1 , t2 ) = q (t1 t2 ) = q ( ).

The Power Spectral Density of white noise is flat as a function of
frequency (where W")
[w/Hz]

If W(t) is also a Gaussian process, then all of its samples are

independent RVs
Colored noise
LTI
White noise
N (t ) = h (t ) W (t )
h(t)
W(t)
22

Power Spectral Density of a Discrete Time Random Sequence

The discrete-time Fourier Transform (DTFT) is defined as

X( ) =

xn e

j 2 n

xn =

n=

1/2
1/2

X( )e j 2 n d (6-34)

The power spectral density function of the WSS random sequence Xn is

2(
% L
1
SX ( ) = lim
E ' X n e j 2 n * .
L 2L +1 '
*)
& n=L

(6-35)

The discrete-time Wiener Khinchin formula is

If X n is a WSS stochastic process, RX [k ] and S X ( ) are

a discrete - time Fourier transform pair :
S X ( ) =

RX [k ]e

k =

j 2k

1/ 2

RX [k ] =

1 / 2

S X ( )e j 2k d

(6-36)
23

Properties of the PSD for a Discrete Time WSS RP

For a WSS random sequence X n , the power spectral density S X ( ) has
the following properties :
(a) S X ( ) 0 for all ,
1/ 2

(b)

-1 / 2

[ ]
2

S X ( )d = E X n = RX [0] .

(6-37)

(c) S X ( ) = S X ( ),
(d) for any integer n, S X ( + n) = S X ( ).

24

Power Spectra and Linear Systems

If a WSS process X(t) with autocorrelation
h(t)
function RXX ( ) S XX ( f ) 0 is
X(t)
Y(t)
applied to a linear system with impulse
response h(t), then the cross correlation
function RXY ( ) and the output autocorrelation function RYY ( ) are
given below:

RXY ( ) = RXX ( ) h* ( ), RYY ( ) = RXX ( ) h* ( ) h( ). (6-38)

But if

f (t ) F ( ),

g (t ) G( )

(6-39)

Then
since

f (t ) g (t ) F ( )G( )
{ f (t) g(t)} =

f (t) g(t)e jt dt

(6-40)
(6-41)
PILLAI

25

{ f (t) g(t)}=

f ( )g(t )d e jt dt

(6-42)
f ( )e j d g(t )e j (t ) d (t )=F ( )G( ).
+

Using the above in (6-38), we get

S XY ( ) = {RXX ( ) h* ( )} = S XX ( )H * ( )

(6-43)

and since
+

h ( )e
where

d =
+

h(t )e

j t

dt

= H * ( ),

H ( ) = h(t )e jt dt

(6-44)
(6-45)

represents the transfer function of the system, and we find

SYY ( ) = {RYY ( )} = S XY ( )H ( ) = S XX ( ) | H ( ) |2 .

(6-46)

The above (6-46) is very useful: the PSD of the output signal
is the product of the PSD of the input and the magnitude
squared of the transfer function.
PILLAI

26

The output power spectrum is real and nonnegative and is

related to the input spectrum and the system transfer function as in
(6-46), which can be used for system identification as well.
WSS White Noise Process: If W(t) is a WSS white noise process,
then
RWW ( ) = q ( ) SWW ( ) = q.
(6-47)
Thus the spectrum of a white noise process is flat, thus justifying its
name. Notice that a white noise process is unrealizable since its total
power is arbitrarily large (" ).
From (6-46), if the input to an unknown system is
a white noise process, then the output spectrum is given by

SYY ( ) = q | H ( ) |2

(6-48)

Notice that the output spectrum captures the system transfer function
characteristics entirely, and for rational systems (6-48) may be
used to determine the pole/zero locations of the underlying system.

PILLAI

27

The Power Spectral Density of the Output Signal

SX(f)

H(f)

SY(f) =|H(f)|2Sx(f)

Frequency Domain
The above relationship is very useful in system design and analysis.

28

Example: A WSS white noise process W(t) is passed

through a low pass filter (LPF) with bandwidth B/2. Find the
autocorrelation function of the output process.
Solution: Let X(t) represent the output of the LPF. Then from (6-48)
#q, | | B / 2
2
S XX ( ) = q | H ( ) | = $
.
(6-49)
%0, | |> B / 2
The inverse transform gives the output autocorrelation function
B/2

B/2

RXX ( ) = B / 2 S XX ( )e j d = q B / 2 e j d
sin( B / 2)
= qB
= qB sinc( B / 2)
( B / 2)
R ( )

(6-50)

XX

| H ( )|2

qB

B / 2

(a) LPF

B/2

(b) Output autocorrelation

PILLAI

29

Example: Let

Y (t ) =

t +T

2T t T

X ( )d

(6-51)

represent a smoothing operation using a moving window on the input

process X(t). Find the spectrum of the output Y(t) in term of that of X(t).
Solution: Consider a LTI system with impulse
response h(t) as in the figure to the right ,
then using h(t), (6-51) reduces to
+

Y (t ) = h(t ) X ( )d = h(t ) X (t )
so that
where

SYY ( ) = S XX ( ) | H ( ) |2 .
+T

H ( ) = T

1
2T

e jt dt = sinc(T )

h (t )
1 / 2T
T

(6-52)
(6-53)
(6-54)
PILLAI

30

so that

SYY ( ) = S XX ( ) sinc 2 ( T ).

(6-55)

sinc2 ( T )

S XX ( )

SYY ( )

Notice that the effect of the smoothing operation in (6-51) is to

suppress the high frequency components in the input (beyond / T ),
and the equivalent linear system acts as a low-pass filter (continuoustime moving average) with bandwidth 2 /T in this case.
By the way, you could also determine the power spectral density by
finding the correlation function of Y(t) directly.
PILLAI

31

Discrete Time Processes

For discrete-time w.s.s stochastic processes X(nT) with
autocorrelation sequence {rk }+
(proceeding as above) or formally
,
defining a continuous time process X (t ) = n X (nT ) (t nT ), we get
the corresponding autocorrelation function to be

RXX ( ) =

rk ( kT ).

k =

Its Fourier transform is given by

S XX ( ) =

k =

rk e jT 0,

(6-56)

and it defines the power spectrum of the discrete-time process X(nT).

From (6-56),

S XX ( ) = S XX ( + 2 / T )
so that S XX ( ) is a periodic function with period
2
2B =
.
T

(6-57)

(6-58)

PILLAI

32

This gives the inverse relation

1 B
jkT
rk =
S
(

)
e
d
(6-59)

B XX
2B
and
1 B
2
(6-60)
r0 = E{| X (nT ) | } =
S ( )d

B XX
2B
represents the total power of the discrete-time process X(nT). The
input-output relations for discrete-time system h(nT) translates into

and
where

S XY ( ) = S XX ( ) H * (e j )

(6-61)

SYY ( ) = S XX ( ) | H (e j ) |2

(6-62)

H (e ) =

n =

h( nT ) e j nT

represents the discrete-time system transfer function.

(6-63)
PILLAI

33

Example: Amplitude Modulated (AM) Signal Noise Analysis

Consider the noisy AM signal

X (t ) = m(t ) cos(0t + ) + n(t ),

(6-64)

Question: What is the Power Spectral Density of the AM signal?

Solution: In this case from (6-64), if we assume ~ U (0, 2 ),
then it can be shown that
1
RXX ( ) = Rmm ( ) cos 0 + Rnn ( )
2

so,

S XX ( ) =

S XX ( 0 ) + S XX ( + 0 )
+ S nn ( ).
2

(a)

(6-66)

S XX ( )

Smm ( )

(6-65)

(b)

S mm ( 0 )

34

Summary: Signal Transmission with Linear Systems

Input

Output
Linear system

Deterministic signals:
Random signals:

(6-66)

35