Pulse-Shaping and Inter-symbol Interference

1. Introduction

Optimum design of the receive filter is important concept which gives us an idea as to how the matched filter
(or correlator) should be designed to maximize the received SNR and, in turn, the performance of the system in
AWGN noise. Further the mathematical analysis of the interaction of the signal with the filter involves the
convolution process, which is the result of band-limited nature if the system.

The objective of this chapter is to analyze baseband pulse transmission and to design the receiver filter such

that overall performance of the system is improved. Thus, we learn (i) to design an matched filter such that signal-

to-noise ration becomes maximum (ii) obtaining impulse and frequency response of receive filter for different

form of transmitted pulse (iii) to analyze Inter-symbol interference (ISI) and optimum design of overall transmit-

receive filter such that ISI effect can be mitigated (iv) To analyze and investigate error probability of receiver for

different format of pulse waveform. Next, doubinary scheme, the approach to achieve the BW efficiency of ideal

Nyquist filter with practical design of filter is discussed followed by various equalizers to mitigate ISI have been

discussed.

2. Design of Matched filter

In this section, our goal is to obtain the impulse response of matched filter such that the effect of receiver

AWGN noise is minimized (and hence to maximize the filter output SNR). Figure 1 shows the composite

transmitter-receiver block diagram. Let us understand the effect of noise by one example. Let us assume that a

positive pulse of 1 V is transmitted at point ‘2’ in this figure. At point ‘3’, the received signal is corrupted by

adding AWGN noise. Let the signal at ‘4’ be denoted by g 0(t) and the corresponding noise at this point is n(t).

After sampling at time interval t=T, the signal value at ‘5’ be, say, 2 V and noise be -1 V. Hard decision device

(DD) takes decision based on the input. Hence if input is positive, DD output will be 1 or else 0. Thus, in this

example the total signal at ‘5’ is 2-1=1 V and hence output is bit 1. Thus, positive pulse, that is 1, is transmitted

and bit 1 is received, thus, no error has occurred. But if the noise value at point ‘5’ is, say, -2.5 V, then, the output

of DD will be 0, and hence we say that error has occurred during the transmission of pulse. Thus, we note that it

is the noise value which affects the performance of the system. Therefore, in order to improve the performance of

the system, the noise value is required to be as low as possible. In other word, we are interested in increasing the

SNR (signal-to-noise ratio) at matched filter output. Thus, the problem formulation is that “what is the optimum
impulse response h(t) (or frequency response H(f)) of the receive filter such that overall SNR at its output becomes

maximum”

Figure 1: Transmit-Receive system

Matched filter is the first stage at the receiver side as shown in Figure 1. Since impulse response is matched to
pulse shape (and hence frequency response of filter is matched to that of pulse being transmitted), hence, it is
called matched filter. Consider the model shown in Figure 1, the total signal at the input of matched filter is given
as

x t   g t    t  , 0t T (1)

where g  t  is the transmitted pulse and  t  is the noise random with zero mean and PSD N 0 2 . Here, for a

time being, ISI is ignored and it is assumed that source of uncertainty in detection is noise only. Here our goal is

to detect g  t  in an optimum manner. That means, at the time of sampling, the ratio of g  t  and   t  at t  T

should be maximum to minimize bit error rate (BER). Hence, our concentration will be on the optimum design of

matched filter such that our objective of maximizing the SNR at matched filter output is met. Since filter is

considered to be linear, time-invariant, hence, individual signal and noise will have individual response. Hence,

let us assume that the signal output and noise output of the matched filter are g 0  t  and n  t  respectively in

response to the input signal g  t  and noise g  t  . As a result, matched filter output is given by

y  t   g0  t   n  t 
(2)

So, we want to maximize

  g0 T  / E n2  t 
2
(3)

where g 0 T  is the sampled value of matched filter output signal at time instant t=T. Next important question is

why we are considering ensemble of noise and why not absolute noise? The reason is that the random variable is
meaningful if and only if we take its average. Here, a question may arise as to why are we maximizing power ratio
in (3) and not maximizing g0 T  / E[n(t )] , a ratio of signal voltage and noise voltage? It is because n  t  is

assumed to have zero mean  E  n  t    0

and hence g0 / E n  t   Undefined
 

Thus it will not help us to achieve the goal (to obtain optimum impulse response of filter). Therefore we have to

maximize the power ratio of signal and noise only. Note that in (3), numerator part is power of signal as it is the
square of voltage. Similarly, power of noise is nothing but its variance which is given by E  n2  t  . Hence our

requirement is to obtain the impulse response of matched filter such that SNR in (3) is maximized. The output of
matched filter g 0  t  is obtained by convolving input with impulse response of filter. That is

g 0 (t )  g (t )  h(t ) (4)

where (*) is the convolutional operator. In frequency domain

G0 ( f )  G ( f ) H ( f ) (5)

Thus, to find g 0 (t ) , we can take inverse Fourier transform of G0 ( f ) which is given as


g0  t    H  f G  f  exp  j 2 ft  df (6)

Sampling at time instant t=T yields
2

g 0 T    H  f  G  f  exp  j 2 fT  df
2
(7)


Hence, numerator in (3) has been found. Let us find denominator. If S N ( f ) is the power spectral density (PSD)
of nt  and N0 / 2 is the PSD of AWGN noise applied to input of filter with frequency response H  f  , then
( PSD)output | H ( f ) |2 ( PSD)input
knowing the relation of input and output PSD of a filter as , these two are related
as

N0
SN  f  
Hf
2
(8)
2
Thus, power of noise is obtained by integrating PSD of noise in the given BW range which is assumed to be in

the range   to  . Hence power of noise is

 
N0
E  n 2  t     S N  f df   Hf 
2
df (9)

2 
Hence, substituting (7) and (9) in (3), we obtain

2


 H  f  G  f  exp  j 2 fT  df

 
(10)
N0
 Hf
2
df
2 
Here, our goal is to find particular form of H  f  , given G  f  , to maximize  . Here, we invoke Schwarz’s

inequality. It says that if we have two complex functions 1 x  and  2  x  in the real variable x satisfying the
conditions

 

 1  x  dx    2  x  dx  
2 2
and
 

Then, we may write

2
  

 1  x  2  x  dx   1  x  dx    2  x  dx
2 2

  

Equality holds if and only if 1  x   k 2  x 

where  2  x  is the complex conjugate of  2  x  . Hence, assuming 1  x   H ( f ) and  2  x   G ( f ) exp( j 2 fT )

, numerator part of (10) can be split as

2 2 2
  

 H ( f )G  f  exp  j 2 fT df   H  f  df  G  f  df (11)

  

Equality holds if and only if

H opt  f   k  G  f  exp  j 2 fT 

*
(12)

 H opt ( f )  kG  f  exp   j 2 fT  (13)

Note that in the above result, the property  X 1 X 2   X 1 X 2 is used where X 1 and X 2 are the complex variables.


NOTE: X 1  A1  jB1 , X 2  A2  jB2

 X1 X 2   ( A1  jB1 )( A2  jB2 )  ( A1 A2  B1B2 )  j ( A1B2  A2 B1 )

  

L.H.S term
 ( A1 A2  B1 B2 )  j ( A1 B2  A2 B1 )

X 1 X 2  ( A1  jB1 ) ( A2  jB2 )  ( A1  jB1 )( A2  jB2 )

R.H.S term
 ( A1 A2  B1 B2 )  j ( A1 B2  A2 B1 )

Hence the property  X 1 X 2   X 1 X 2 is proved as L. H. S is equal to R. H. S.



Hence, from (10) and (11)

 2
2
  G  f  df
N0 

Using (12), maximum SNR is obtained as

  2
2
max 
N0 
 G f  df (13)

Under the condition that

H opt  f   kG  f  exp   j 2 fT  (14)

Result in (14) states that except for scaling factor k and the term exp   j 2 fT  , the frequency response of
optimum filter is same as the complex conjugate of Fourier transform of pulse and hence the name “matched
Filter”.


hopt  t   k  G   f  exp   j 2 fT exp  j 2 ft  df

Now, 
 k  G  f  exp -j2 f  T-t  df 


Now, it can be shown that for real signal g  t  , G  f   G   f 

.

NOTE:


G( f )   g (t ) exp( j 2 ft )dt

 
 G ( f )  

g  (t ) exp( j 2 ft )dt   g (t ) exp( j 2 ft )dt


  g (t ) exp( j 2 ( f )t )dt  G( f )


Hence, hopt  t   k  G   f  exp   j 2 f T  t  df




f   f  hopt  t   k  G   f  exp   j 2 f T  t   df


 k  G  f  exp  j 2 f T  t   df


 k  g T  t 
(15)

This result shows that the impulse response of the optimum filter, except for scaling factor k , is a time-reversed
and delayed version of the input signal g t  , that is, it is matched to input signal. A linear time-invariant filter
defined this way is called a matched filter.

Steps to get impulse response of matched filter

Important results
 (1) hopt t   kg T  t  H opt  f   kG  f  exp  j 2fT 
or

 

 G f   g t 
2E
 max  E df 
2 2
(2) where
N0  

(16)

Hence, it says that peak pulse signal-to-noise of a matched filter depends only on the ratio of the signal energy to
the PSD of while noise at the filter input.

Note: E  In joules, N 0 / 2  Watts/hertz  max is dimension on less

Matched filter signal output

G0  f   H opt  f  G ( f )  kG   f  exp   j 2 fT  G ( f )

 k G  f  exp   j 2 fT 
2

 
g 0 T    G0  f  exp  j 2 fT  df   k G f  exp   j 2 fT  exp  j 2 fT  df
2

 


 k G f 
2
 df  kE


(17)

If pulse is having magnitude A and duration Tb, then, E=A2Tb and hence g0 T   kE  kA2Tb .

Example 1: Find the impulse response of matched filter when the following pulse is being transmitted.

Figure 3

Solution:
 Figure 4

It may be noted that since h(t )  g (T  t ) and hence k  1 . If it is to be made equal to correlator (note that, in
correlator, the multiplying basis function to be used is given as  (t )  g (t ) / E ), then, in matched filter k should

be taken as k  1 / E . Then, matched filter and correlator will be perfectly same. The same equivalence can
also be achieved by taking k  1 in matched filter and using multiplying basis function in correlator as E   (t )
instead of only  (t ) .

Inter-symbol Interference (ISI)

4.5.1 Band-limited Channel

When the pulse is passed through a band-limited channel of bandwidth W, then, the frequency content of the

signal lying outside the bandwidth W is cut-off. This is shown in Figure 30. As a result, in time domain, the effect

is observed by expansion of pulse (called pulse broadening or pulse dispersion).

 Figure 30

The effect of pulse broadening can be observed in Figure 31 where a series of three pulses, say, k-1, k and k+1th

pulses are transmitted through a channel. Assuming the channel as having infinite BW, the shape of the pulse will

not distort (we have ignored the effect of AWGN noise). This is shown in Figure 31 (a). Thus, at time instant t=T,

there is no tail of either preceding or following pulse and hence value of signal at t=T is not distorted. In Figure

31 (b), a band-limited channel is used. As a result, the received pulse is dispersed. The broadening of pulse is

further widened once this pulse is passed through a matched filter. We can see that at t=T time instant, the value

of the pulse is corrupted by tail of the preceding and the following pulses. Thus, pulse energy is said to have

spilled over in the region of other pulse. This is called Inter-symbol Interference (ISI). Consider, for example, the

value of kth pulse is 2V and value of tail of k-1 and k+1 th pulse at time t=T are 1.5 V and 1.5 V. Hence total

signal value at t=T is 2+1.5+1.5=5V. Thus, hard decision device output becomes ‘1’. Therefore, we see that bit 1

is transmitted and bit 1 is received and hence though ISI has occurred but it has not created any problem. Consider

another case shown in Figure 32 where a series of pulses (k-1 and k+1th pulses are negative and kth pulse is

positve). As a result at point ‘4’, the preceding and the following pulses are negative and kth pulse is positive.

Considering, for example, the value of kth , (k-1)th and (k+1)th pulses at time t=T are 2V, -1.5 V and -1.5 V

respectively, the total signal value at t=T is 2-1.5-1.5=-1V. Thus, hard decision device output becomes ‘0’.

Therefore, we note that, though bit ‘1’ was sent, the received bit is ‘0’ and, thus, error is said to occur due to ISI

effect.
And thus

Figure 31

Figure 32

Example 4: If three impulses occurring at t=-T,0,T are passed through a filter with impulse response h(t) as shown
in Figure 41, find the output.

Figure 33
The output can be written as

1
y (t )  a1h(t  T )  a0 h(t )  a1h(t  T )   ai h(t  iT )
i 1

Here we use an important property of convolution of an impulse which is

 (t  iT )  h(t )  h(t  iT )

This states that pulse is shifted at the position where an impulse function exists.

The analysis to follow is based on assumption of PAM modulation scheme. Further it is assumed that PAM is
very narrow pulses approximating impulse. It helps us directly write output of filter (with input as PAM pulse) as
impulse response of the filter. Further, PAM is easy to generate and PAM modulator output is a square of short-
pulse (approximating impulse) whose amplitude is ( bk is the kth input bit at the input of PAM generator)

ak  1 If Symbol bk is 1
 1 If Symbol bk is 0 (44)

Figure 34

s  t   ...  a1 g (t  Tb )  a0 g (t )  a1 g (t  Tb )  ...   ak g t  kTb  (45)

k

Let us treat g  t  , c  t  , h  t  as a single block defining its impulse response to be   p  t  . Since input is a short-
pulse approximating impulse, so, we can write

y  t     ak p  t  kTb   w  t 
(46)

Here,  is a scaling factor and

  p t   g t   h t   c t    P  f   G  f  H  f  C  f  (47)
The filter output y t  is sampled at t i  iTb

 y  ti    a k p  i  k  Tb   w  ti 
k 


y (ti )    ai  a k p  i  k  Tb   w(ti )
k 
k i (48)
Here, the first term is the value of original pulse sampled at ti  iTb and second term is the tail value of
neighboring pulses (See Figure 31(b) and 32). The third term corresponds to noise value at ti  iTb . Let us five
pulses {a2 , a1, a0 , a1, a2} . So k is varying from -2 to 2. Let the ith pulse be the 0th pulse (the desired pulse whose
estimate is to be found), then,

2
y (t0 )    a0  a k p  k  Tb   w(t0 )    a0  a2 p (2Tb )  a1 p(Tb )  a1 p(Tb )  a2 p(2Tb )
k 2
k 0 ISI term

.Similarly, if the ith pulse be the first pulse, then,

2
y (t1 )    a1  a k p 1  k  Tb   w(t0 )    a1  a2 p (3Tb )  a1 p (2Tb )  a1 p (Tb )  a2 p (Tb )
k 2
k 1 ISI term

Now, our goal is to design the pulse shaping filter and matched filter so that the effect of second term in (48) is
minimized. Thus for the sake of convenience, we ignore the noise term w(ti ) .

4.5.2 Nyquist Criterion for ISI-free transmission

Here our objective is to achieve a result for distortion less baseband binary transmission. Consider a pulse p  t  ,
we want the following condition to be satisfied for distortion less criterion

p  i  k  Tb   1 if i  k
(49)
0 if ik

This implies that if i pulse is decoded and k  i (it only implies that the sampling time-instant matches with
th

that of pulse being decoded), then, it is set to be 1 because p  0  has been normalized to be 1 and at the time of
th
decoding i pulse, the contribution due to other pulse (that is, ISI) would be zero. It is because if it is not the
case, then, the sampled value at sampler output will be the scaled version of the input PAM as is obvious from
(47). If, somehow, the condition in (49) is satisfied, then

y  ti     ai for all i

Suppose if p  t  is sampled with sampling interval Tb  1 / Rb , then, the change that takes place in frequency
spectrum (as shown in Fig. 35) is


P  f   Rb  P  f  nR 
b (50)
n 

Figure 35

The P  f  is a collection of number of copies of P  f  to be delayed by nRb , where n  0,1,2,....... .

Therefore, P  f  can also be written by
  
P  f      p  mT   t  mT  exp   j 2 ft dt
b b (51)
 m 

Let us assume m  i  k (Comment on this way of assuming) then, m  0 if i  k and m  0 if i  k . Hence,

substituting (49) in (51) results in

 
P  f    p  0 t  exp   j 2 ft  dt  p  0    t  exp   j 2 ft  dt  p  0  1 (52)
 
The conclusion is that when we apply distortion less condition of (49) to the spectrum of sampled signal, we get
P  f   1 indicating that to avoid ISI, the frequency response of the system has to be constant satisfying
P  f   1 . Hence, this is requisite condition to avoid inter-symbol interference.

 
P  f   1  Rb  P  f  nR    P  f  nR   T
b b b (53)
m  n 

Therefore, if the condition in (53) is satisfied, then, we will ensure ISI-free detection. Now, the next question is
“in what way, we can implement the above result and how, after satisfying it, detection of signal will be free from
ISI. The next section is going to answer these two questions.

4.5.3 Implementation of ISI-free condition

A) Ideal Nyquist channel

The simplest way to satisfy equation (53) is to have the shape of frequency response P  f  in the following way

1
P f   W  f  W
2W
0 f W
(54)

The above can also be written as P  f   (1/ 2W )rect  f / 2W  . This is shown in Figure 9.36 (a).

Figure 36

With overall system BW defined as

Rb 1
W  (55)
2 2Tb
Note that Rb is sample rate. This is the rate at which the signal y(t) is sampled at point 4 in Figure 9.28. Thus this
is same as that of bit rate because it is sampled at ti  iTb . This is the minimum BW requirement of system for
ISI free system. Here, Nyquist rate  Rb  2W and W is called Nyquist BW.

Figure 37

Example 5: If we have 10 kbps to be transmitted, then find the minimum BW of system required to transmit this
data rate to ensure ISI free condition.

Rb
Solution: W 
2

Ideal Nyquist channel BW needed is just 5 KHz and not more than that to transmit this data rate without ISI.

The ideal baseband pulse transmission system is called ideal Nyquist Channel. Impulse response p t  is obtained
by taking the inverse FT of P  f  in (54) and is given by

Sin  2Wt 
p t    Sinc  2Wt  (56)
2 Wt
Its plot is as shown in Fig. 38.
 Fig. 38: Sinc pulse

Hence, if we sample the p t  at instants t  0,Tb ,2Tb ,......., then we will get ISI free output due to the fact
that at these sampling time instant, the neighboring pulses will cross the time axis, thus, their contribution will be
zero. This situation can better be explained in Figure 38. But we encounter two practical difficulties:

(i) To achieve this, we require frequency response to be flat from  W to W and zero outside. This is
practically unrealizable.
(ii) Due to sharp cut-off at W in p  f  spectrum, there is slow decay in p  t  with respect to t . But, it
is
desired that it should decay fast and, at large t , its amplitude should disappear because even if there exists
some time error in sampling instant, then, effect of this pulse at large t will no longer exist. Due to
abovementioned reason, we go for next possible solution to realize the condition of (53) ensure ISI free
condition.

B) Raised Cosine Spectrum: Let us assume another shape of P  f  such that

1
P  f   P  f  2W   P  f  2W   W  f  W (57)
2W

We may design several band-limited functions that can satisfy (57). Raised-cosine (RC) spectrum is one such
shape which satisfies the condition in (53). It consists of flat portion and roll-off portion that has sinusoidal form
as follows.

1
P f   0  f  f1
2W
1 
    f  W   
 1  Sin   f1  f  2W  f1
4W   2W  2 f1  

0 f  2W  f1
(58)

A frequency parameter f1 and BW are related as   1  ( f1 / W ) .Here, the parameter  is roll-off factor. This
is a measure of the excess BW over ideal BW W . Thus, higher is the value of the  , the more the filter will be
practically realizable.   0 implies ideal BW i.e. f 1  W . Increase in the value of  implies going away
from ideal situation. Transmission BW BT is defined as
 f 
BT  2W  f1  W  2  1   W 1    (59)
 W 
 Figure 39: Raised-cosine filter for roll-off factor 0.6

C) Time Response

Figure 40: Time-domain response of pulse for different roll-off factors.

 Cos  2Wt  
p  t    Sinc  2Wt   2 2 
(60)
 1  16 W t 
2

Conclusion:

(i) We see the frequency response for   0,0.6,1. We note that for   0, P f  cuts-off gradually as
compared ideal Nyquist channel (   0 ) and hence, it is easier to implement the filter. At the same
time, P f  exhibits odd symmetry with respect to W and hence it possible to satisfy the condition
of (53).
(ii) For   0 , p t  falls off sharply with time, thus, facilitating that even if there is some timing error
in sampling time, it would add a small value of ISI on its own. This is obvious from Fig. 40 where
it is seen that for   0.5 , the peak of the tail decays much faster than that of ideal sinc pulse. For
  1, tail almost disappears.
 (iii)   1(i.e. f1  0 ) is called full-cosine role-off factor. In practice, this can be implemented in an
easiest way but price to be paid for this is the double BW required to transmit the same data rate.

Example: Let us consider a binary data stream of 80 kbps. This has to be transmitted through a PAM
system. What is the minimum BW required to transmit this data rate successfully? Let us assume that
the system is having raised cosine spectrum. Draw this spectrum with suitable computed parameters.
Compute the transmission BW required for roll-off factor   0,0.3,0.6,0.9,1 . Write down the
expression of impulse response. Comment on how the tail of impulse response is affected by the roll-
off factor.

Solution:

  1  f1 / W , W  Rb / 2  28 kHz
BT  W (1   )  28  (1   )
 28,36.4, 44.8,53.2 and 56kHz for   0, 0.3, 0.6, 0.9,1

For   0.3 , the impulse response is given as


 Cos 52.75  103 t 
  
p  t   Sinc 56  10 t 
 3 

  1  1128.96 106 t 2 
 