Digital Signal Processing, Discrete-time Fourier Analysis

Dr. Alan Tan 28 August 2011

(Edition #3.8)

c 2011 Alan Tan

Lecture outline
Discrete Fourier transform Denition, Properties of the DFT, Circular convolution, Linear convolution using DFT, Zero padding Fast Fourier transform Denition, Decimation-in-time FFT, Decimation-in-frequency FFT Exercises

c 2011 Alan Tan

Page 2/40

Discrete Fourier transform

Suppose x[n] is a discrete-time signal of length N . The Fourier transform of x[n] is given by N 1 jw X(e ) = x[n]ejwn
n=0

and X(ejw ) is periodic with period 2. Sampling X(ejw ) over the interval [0, 2) at the points wk = 2k/N, k = 0, 1, . . . , N 1 we get X[k] = X(ejwk ) =
N 1 n=0

x[n]ej2kn/N , k = 0, 1, . . . , N 1

(1)

The sampled Fourier transform X[k] in (1) is referred to as the discrete Fourier transform (DFT) of the discrete-time signal x[n].

c 2011 Alan Tan

Page 3/40

Discrete Fourier transform (cont.)

Figure 1 illustrates this spectral sampling process.
{X(ejw )}
4 0 4 0 1 2

{X(ejw )}

4 0 4 0 1 2

w/
(a)

w/
(b)

Figure 1: The (a) real, and (b) imaginary parts of the Fourier transform of a discrete-time signal of length N = 16.

c 2011 Alan Tan

Page 4/40

Discrete Fourier transform (cont.)

In working with the DFT, it is common to use the notation WN = ej2/N such that (1) becomes N 1 kn X[k] = x[n]WN , k = 0, 1, . . . , N 1
n=0 j

(2)

Eulers formula states that e = cos + j sin , i.e., the powers of WN lie on the unit circle in the complex plane spaced = 2/N apart.
6 W8 5 W8 1 7 W8 = W8 8 0 W8 = W8 = 1 + j0 1 W8 2 W8

4 W8 3 W8

Figure 2: The powers of W8 around the unit circle.

c 2011 Alan Tan Page 5/40

Discrete Fourier transform (cont.)

kN k The powers of WN is periodic with period N , i.e., WN = WN . Due to this, the DFT is also periodic with the same period N or

X[k N ] =

N 1 n=0

x[n]WN

(kN )n

N 1 n=0

kn x[n]WN = X[k]

The discrete-time signal x[n] can be recovered from X[k] by means of the inverse discrete Fourier transform (IDFT), i.e.,
N 1 1 kn X[k]WN , n = 0, 1, . . . , N 1 x[n] = N k=0

c 2011 Alan Tan

Page 6/40

Discrete Fourier transform (cont.)

The DFT formula may also be conveniently expressed in vector/matrix notation as
X[0] X[1] X[2] X[N 1]

0 WN = W 0 N

0 WN

0 WN

0 WN

0 WN

1 WN 2 WN

2 WN 4 WN

N WN 1 2(N 1) WN

2(N 1)

WN
(N 1)(N 1)

0 N WN WN 1 WN

x[0] x[1] x[2]

(3)

x[N 1]

or more compactly as X = Wx. It follows straightforwardly that the IDFT is x = W1X.

c 2011 Alan Tan

Page 7/40

Discrete Fourier transform (cont.)

Problem 1: Determine the 4-point DFT of x[n] = {1, 0, 0, 1}1. Solution: Using (3), we have
0 W4 0 W4 0 W4 0 W4 0 W4 1 W4 2 W4 3 W4

0 W4 2 W4 4 W4 6 W4

0 W4 3 W4 6 W4 9 W4

0 1 j 1 j 0 1 j = = 0 1 1 1 1 0 2
1 1 j 1 j 1 1+j

and so X[k] = {0, 1 j, 2, 1 + j}.

The notation {x[0], x[1], . . . , x[N 1]} is used to denote a discrete-time signal x[n] of N elements.
Page 8/40

c 2011 Alan Tan

Discrete Fourier transform (cont.)

There are various ways the horizontal axis of the spectrum can be represented. One is to use the radian frequency w [0, 2) as in Figure 1. Another is the use of the integer k = 0, 1, . . . , N 1. Yet another is to label according to the analog frequency f [0, fs) where fs is the sampling frequency of the signal. For real-valued signals, the real part of the DFT is symmetrical and the imaginary part is anti-symmetrical.
4 0 4 0 8 16 4 0 4 0 8 16

{X[k]}

k
(a) (b)

{X[k]}

Figure 3: The (a) real, and (b) imaginary spectra of a real-valued signal.

c 2011 Alan Tan

Page 9/40

Discrete Fourier transform (cont.)

Any real-valued signal x[n] can be separated into an even part xe[n] and an odd part xo[n], i.e., xe[n] = (x[n] + x[n])/2 and xo[n] = x[n] xe[n] The even signal xe[n] = xe[n] is symmetrical about n = 0. The odd signal xo[n] = xo[n], on the other hand, is anti-symmetrical about n = 0.
1 0 1 0 8 16 1

xo [n]

xe [n]

0 1 0 8 16

n
(a) (b)

Figure 4: The (a) even, and (b) odd parts of a real-valued signal.

c 2011 Alan Tan

Page 10/40

Discrete Fourier transform (cont.)

The DFT is real-valued for even signals and imaginary-valued for odd signals.
{Xe [k]}
4 0 4 0 8 16

{Xe [k]}

4 0 4 0 8 16

k
(a) (b)

{Xo [k]}

4 0 4 0 8 16

{Xo [k]}

4 0 4 0 8 16

k
(c) (d)

Figure 5: The real and imaginary spectra of an even signal in (a, b), and the real and imaginary spectra of an odd signal in (c, d).

c 2011 Alan Tan

Page 11/40

Discrete Fourier transform (cont.)

The DFT has many properties similar to those of the Fourier transform.
Property Linearity Time shifting Frequency shifting Time reversal Conjugation Convolution Modulation

x[n] ax[n] + by[n] x[n n0N ] x[n]WN

k0 n

X[k] aX[k] + bY [k] X[k]WN

kn0

X[k k0N ] X[kN ] X [kN ] X[k]Y [k] X[k] Y [k]

x[nN ] x[n] x[n] y[n] N x[n]y[n]

Table 1: Properties of the DFT.

c 2011 Alan Tan

Page 12/40

Discrete Fourier transform (cont.)

Table 2 presents the DFT of several elementary signals.
x[n] [n n0] ej2k0n/N e
an/N

X[k] ej2kn0/N N [k k0] 1 eaej2k 1 ea/N ej2k/N N [k k0] + N [k + k0] e

jk(m1)/N sin(km/N )

x[n] =

2 cos(2k0n/N ) { 1, n = 0, . . . , m 1 0, n = m, . . . , N 1

sin(k/N )

Table 2: Some common DFT pairs.

c 2011 Alan Tan

Page 13/40

Discrete Fourier transform (cont.)

It is important to note that the shift and reversal operations on x[n] and X[k] are circular or modulo N operations, e.g., x[167] = x[2].
1

x[n]

0 1 0 8 16

n
(a)
16 ] 16 ]

1 0 1 0 8 16

1 0 1 0 8 16

x[ n + 1

n
(b) (c)

x[ n

Figure 6: The (a) signal x[n], (b) circularly shifted x[n + 116], and (c) time-reversed x[n16].
c 2011 Alan Tan Page 14/40

Discrete Fourier transform (cont.)

Unlike the Fourier transform, the convolution property of the DFT holds only for circular convolution. The circular convolution of x[n] and y[n] of length N is given by z[n] = x[n] y[n] =
N 1 m=0

x[m]y[n mN ], n = 0, 1, . . . , N 1

The resultant z[n] is also of length N .

c 2011 Alan Tan

Page 15/40

Discrete Fourier transform (cont.)

Table 3 illustrates the linear and circular convolution of x[n] = {1, 2, 3, 4} and y[n] = {5, 6, 7, 8}.
Linear convolution, z[n] = x[n] y[n] Circular convolution, z[n] = x[n] y[n]

1 2 3 4 8 7 6 5 1 2 3 4 8 7 6 5 1 2 3 4 8 7 6 5 1 2 3 4 8 7 6 5

z[0] = 5 z[1] = 16 z[2] = 34 z[3] = 60

1 2 3 4 5 8 7 6 1 2 3 4 6 5 8 7 1 2 3 4 7 6 5 8 1 2 3 4 8 7 6 5

z[0] = 66 z[1] = 68 z[2] = 66 z[3] = 60

z[n] = {5, 16, 34, 60, 61, 52, 32}

z[n] = {66, 68, 66, 60}

Table 3: The linear and circular convolution of x[n] = {1, 2, 3, 4} and y[n] = {5, 6, 7, 8}.
c 2011 Alan Tan

Page 16/40

Discrete Fourier transform (cont.)

Expressing the circular convolution in vector/matrix notation, we have
z[0] z[1] z[N 1]

y[0] y[1]

y[N 1] y[1] y[0]

y[2]

x[0] x[1]

(4)

y[N 1] y[N 2] y[0]

x[N 1]

The matrix of ys is Toeplitz, i.e., constant along the diagonals. Problem 2: Determine the circular convolution of x[n] = {2, 3, 1, 5} and y[n] = {1, 4, 2, 3}. Solution: Using (4) we have

4 1 3 2 3 4 = 2 4 1 3 1 8
3 2 4 1 5 11

3 2

15

and hence x[n]

c 2011 Alan Tan

y[n] = {15, 4, 8, 11}.

Page 17/40

Discrete Fourier transform (cont.)

Linear convolution can be made similar to a circular convolution. Specically, the linear convolution of x[n] and y[n] (of length N1 and N2, respectively) is similar to a circular convolution of these signals modulo N where N N1 + N2 1. To that end, N N1 zeros are foremost appended to the tail of x[n]2 so that a new xa[n] of length N is formed; by the same token, N N2 zeros are appended to y[n] to form ya[n] also of length N , i.e., { x[n], n = 0, 1, . . . , N1 1 xa[n] = 0, n = N1, N1 + 1, . . . , N 1 { y[n], n = 0, 1, . . . , N2 1 ya[n] = 0, n = N2, N2 + 1, . . . , N 1 Formally, we have xa[n] ya[n] = x[n] y[n]

The operation of adding zeros to the tail of a discrete-time signal is called zero padding.
Page 18/40

c 2011 Alan Tan

Discrete Fourier transform (cont.)

Problem 3: Determine the linear convolution of x[n] = {2, 3, 1, 5} and y[n] = {1, 4, 2, 3}. Solution: Both x[n] and y[n] are of length N1 = N2 = 4. Pick N = N1 + N2 1 = 7 so that N N1 = N N2 = 3 zeros are appended to both x[n] and y[n], producing xa[n] = {2, 3, 1, 5, 0, 0, 0} and ya[n] = {1, 4, 2, 3, 0, 0, 0}, respectively. Applying (4) on xa[n] and ya[n], we get 4 2 3 0 0
0 1 0 1 4 2 0 0 0 0 3 2 0 0 0 1 4 0 0 0 1 4 0 1 4 0 0 1 4 3 2 3

3 2 0

3 2

11 7 3 1 5 = 11 0 0 13 0 0 0 7
1 0 15

3 2

and so xa[n]
c 2011 Alan Tan

ya[n] = x[n] y[n] = {2, 11, 7, 11, 13, 7, 15}.

Page 19/40

Discrete Fourier transform (cont.)

The use of a linear convolution arises in many practical applications, e.g., the ltering of a signal or the calculation of the autocorrelation of a signal. Since linear convolutions can be made similar to circular convolutions and circular convolutions (in time) are multiplications of the DFTs, linear convolutions may, hence, be implemented by means of the latter. Ecient algorithms for computing the DFT (and therefore linear convolutions) have long been reported in the literature and is the subject of our study later in the slides.

Figure 7: Linear convolution using DFT.

c 2011 Alan Tan

Page 20/40

Discrete Fourier transform (cont.)

For any discrete-time signal x[n] of length N , the DFT is also length N . The DFT of the zero-padded xa[n] of length M , where M > N , is length M . Essentially, the zero padding of x[n] results in an interpolation of the spectrum3 producing a high-density spectrum4. To illustrate, the spectra of the signal x[n] and the zero-padded xa[n] of Figure 8 appear in Figure 9.
1 0 1 0 8 16 1

xa [n]

x[n]

0 1 0 8 16

n
(a) (b)

Figure 8: The (a) signal x[n], and (b) zero-padded xa[n].

For M = LN , where L is an integer, an interpolation by a factor of L is accomplished, i.e., the points Xa [kL] of the zero-padded DFT are identical to the points X[k] of the conventional DFT. 4 A high-density spectrum is not the same as a high-resolution spectrum; to generate the latter, one has to obtain more data from the measurements.
c 2011 Alan Tan Page 21/40 3

Discrete Fourier transform (cont.)

Figure 9 shows the interpolated spectrum obtained through zero padding.
{X[k]} {X[k]}
4 8 4 0 4 0 4 0 4 0 4 8

k
(a) (b)

{Xa [k]}

4 0 4 0 8 16

{Xa [k]}

4 0 4 0 8 16

k
(c) (d)

Figure 9: The DFT X[k] of x[n] in (a, b), and the interpolated DFT Xa[k] of xa[n] in (c, d). Note that the interpolation factor is L = 2, i.e., Xa[2k] = X[k].
c 2011 Alan Tan Page 22/40

Fast Fourier transform

Recall that the DFT formula in (2) is X[k] =
N 1 n=0 kn x[n]WN , k = 0, 1, . . . , N 1

In particular, N complex multiplications and N 1 complex additions are involved in the calculation of each point of the DFT. To compute all the N points, therefore, involves N 2 complex multiplications and N (N 1) complex additions; the computation time is, thus, approximately proportional to N 2. Similar complexity also applies for computing the IDFT. Evidently, the number of arithmetic operations grows rapidly as N increases. For this reason, we are interested in computational procedures that reduce the number of multiplications and additions. In the literature, these procedures are collectively referred to as the fast Fourier transforms (FFT).

c 2011 Alan Tan

Page 23/40

Fast Fourier transform (cont.)

Let x[n] be of length N where N = 2v . Since N is an even integer, we can separate x[n] into two shorter segments of length N/2 consisting of the evennumbered points and the odd-numbered points of x[n], respectively. This means N 1 kn X[k] = x[n]WN
n=0

kn x[n]WN

n odd

kn x[n]WN

n even

Using the substitution n = 2r for even n, and n = 2r + 1 for odd n, we get

N/21

X[k] =

r=0

N/21 k(2r) x[2r]WN

r=0

x[2r + 1]WN
N/21

k(2r+1)

N/21

r=0

2 x[2r](WN )kr

k WN

r=0

2 x[2r + 1](WN )kr

c 2011 Alan Tan

Page 24/40

Fast Fourier transform (cont.)

Noting that thus
N/21 2 WN = (ej2/N )2 = ej2/(N/2) = WN/2

X[k] =

r=0

N/21 kr x[2r]WN/2

k WN

r=0

kr x[2r + 1]WN/2

k = G[k] + WN H[k], k = 0, 1, . . . , N 1

(5)

In the above, G[k] is the N/2 point DFT of the even-numbered points of x[n], and H[k] is the other N/2 point DFT of the odd-numbered points of x[n]. Although k ranges over N points, each of the sums are computed only for k = 0, 1, . . . , N/2 1 since G[k] and H[k] are both periodic in k with period N/2. Essentially, (5) corresponds to breaking up the original N point DFT into two shorter N/2 point DFTs. This is the basis of the decimation-in-time FFT.

c 2011 Alan Tan

Page 25/40

Fast Fourier transform (cont.)

The ow graph realization of (5) for the 8-point DFT appears below.

Figure 10: The 8-point DFT as two 4-point DFTs.

c 2011 Alan Tan

Page 26/40

Fast Fourier transform (cont.)

In the ow graph diagram, all branches entering a node are weighted by the branch transmittance and summed; the branch transmittance is assumed unity when omitted. In the slides, dashed lines are used for all branches with accompanying branch transmittances; solid lines, hence, denote branches with unity branch transmittance.

Figure 11: Flow graph representation.

c 2011 Alan Tan

Page 27/40

Fast Fourier transform (cont.)

As N/2 is also even, we can then proceed to separating the sums in (5) into even shorter N/4 point DFTs.

Figure 12: The 4-point DFT as two 2-point DFTs.

c 2011 Alan Tan

Page 28/40

Fast Fourier transform (cont.)

The radix of the FFT is determined by the size of the smallest transform. For instance, the decimation-in-time algorithm is a radix-2 FFT since N = 2v . The structure of the elementary 2-point DFT (also called buttery) appears in Figure 13.

Figure 13: The 2-point DFT.

c 2011 Alan Tan

Page 29/40

Fast Fourier transform (cont.)

Figure 14 shows the complete 8-point decimation-in-time FFT.

Figure 14: The complete 8-point decimation-in-time FFT.

c 2011 Alan Tan

Page 30/40

Fast Fourier transform (cont.)

It is straightforward to observe that the structure in the left of Figure 15 r+N/2 r appears in various forms in Figure 14. Noting that WN = WN , this structure, hence, is functionally similar to the one in the right of Figure 15.

Figure 15: Functionally similar ow graphs.

c 2011 Alan Tan

Page 31/40

Fast Fourier transform (cont.)

Yet another realization of the 8-point decimation-in-time FFT appears below.

Figure 16: Yet another 8-point decimation-in-time FFT.

c 2011 Alan Tan

Page 32/40

Fast Fourier transform (cont.)

Time decimation leads to a scrambled order of the input data index n. This scrambled order, nevertheless, can be readily obtained by reversing the bits of the binary representation of the normal input data index order.
Normal order of index n 0 1 2 3 4 5 6 7 Binary bits of index n 000 001 010 011 100 101 110 111 Reversed bits of index n 000 100 010 110 001 101 011 111 Bit-reversed order of index n 0 4 2 6 1 5 3 7

Table 4: Bit reversal for an 8-point FFT.

c 2011 Alan Tan

Page 33/40

Fast Fourier transform (cont.)

Previously, we noted that the direct computation of DFT involves N 2 complex multiplications and additions5. By comparison, the computation of (5) involves 2(N/2)2 complex multiplications and additions for the N/2 point DFTs, and an additional N complex multiplications and additions when combining these DFTs, i.e., 2(N/2)2 + N = N 2/2 + N complex multiplications and additions in total. It is straightforward to verify that N 2/2 + N < N 2 for N > 2. For the most part, the procedure in (5) is repeated until the basic 2-point DFTs remain. If N = 2v , then there are v stages of computation and the number of arithmetic operations involved is6 N v = N log2 N . This oers huge computational savings, e.g., if N = 210 = 1024, then N 2 = 1048576 but N log2 N = 10240, a reduction of over two orders of magnitude!

For simplicity, we shall assume that N is large so that N (N 1) can be approximated by N 2 . 6 To be precise, there are (N/2) log2 N complex multiplications (ignoring multiplications by 1) and N log2 N complex additions.
5 c 2011 Alan Tan Page 34/40

Fast Fourier transform (cont.)

The alternative to the decimation-in-time FFT is the decimation-in-frequency FFT. Unlike the former, this other method separates the output X[k] (instead of x[n]) into successively smaller subdivisions. For starters, suppose x[n] is of length N = 2v . Recall that the DFT is N 1 kn X[k] = x[n]WN , k = 0, 1, . . . , N 1
n=0

The even-numbered points of X[k] are N 1 (2r)n x[n]WN X[2r] =

n=0 N 1 n=0 N/21 2 x[n](WN )rn N 1 n=N/2

2 x[n](WN )rn +

2 x[n](WN )rn

n=0

c 2011 Alan Tan

Page 35/40

Fast Fourier transform (cont.)

The substitution n = n + N/2 in the second sum gives
N/21

X[2r] =

N/21 2 x[n](WN )rn

2 x[n + N/2](WN )r(n +N/2)

n=0 N/21

n =0 N/21 2 x[n](WN )rn

x[n +

2 rn N/2](WN )

n=0 N/21

n =0 2 (x[n] + x[n + N/2])(WN )rn

n=0 N/21

rn g[n]WN/2, r = 0, 1, . . . , N/2 1

(6)

n=0

where g[n] is formed by adding both halves of x[n].

c 2011 Alan Tan

Page 36/40

Fast Fourier transform (cont.)

In a similar fashion, the odd-numbered points of X[k] are X[2r + 1] =
N 1 n=0

x[n]WN

(2r+1)n

N 1 n=0

n 2 x[n]WN (WN )rn N 1 n=N/2

N/21

n 2 x[n]WN (WN )rn +

n 2 x[n]WN (WN )rn

n=0

c 2011 Alan Tan

Page 37/40

Fast Fourier transform (cont.)

The substitution n = n + N/2 in the second sum gives
N/21

X[2r + 1] =

n 2 x[n]WN (WN )rn

n=0 N/21

x[n +

n +N/2 2 r(n +N/2) N/2]WN (WN ) N/21

n =0 N/21 n 2 x[n]WN (WN )rn

= Note that WN
N/2

n 2 x[n + N/2]WN (WN )rn

n=0

n =0

= 1 is used in the above derivation.

c 2011 Alan Tan

Page 38/40

Fast Fourier transform (cont.)

Subsequently, we have
N/21

X[2r + 1] =

n 2 (x[n] x[n + N/2])WN (WN )rn

n=0 N/21

n rn h[n]WN WN/2, r = 0, 1, . . . , N/2 1

(7)

n=0

where h[n] derives from subtracting one half of x[n] from the other. The overall N point DFT is, thus, given by the N/2 point DFTs of g[n] n and h[n]WN corresponding to the even- and odd-numbered points of X[k], respectively. As before, the procedures in (6) and (7) are repeated until only the 2-point DFTs remain.

c 2011 Alan Tan

Page 39/40

Fast Fourier transform (cont.)

Figure 17 presents the complete 8-point decimation-in-frequency FFT.

Figure 17: The complete 8-point decimation-in-frequency FFT.

c 2011 Alan Tan

Page 40/40

Exercises
Question 6, May 2010

Let x[n] = {1, 2, 0, 1} and y[n] = {1, 1, 1}. Using circular convolution, compute the linear convolution of x[n] and y[n]. The system below uses the DFT to perform the convolution of x[n] and y[n].

Determine ya[n], X[k], Ya[k], Z[k], and z[n]. Is the convolution process similar to the one previously?
X[k] = {2, 1 j3, 0, 1 + j3}, Ya[k] = {1, j, 3, j}, z[n] = {2, 0, 1, 1} No, one is a 6-point circular convolution whereas the other is a 4-point circular convolution.

Exercises (cont.)
Question 6, September 2010

Prove that, for real-valued signals, the real part of the DFT is symmetrical and the imaginary part is anti-symmetrical. Using zero padding where necessary, determine the DFT of x[n] = {1, 1, 1, 0, 0, 1} based on the 8-point decimationin-time FFT.
X[k] = {4, 1 j, j2, 1 + j, 0, 1 j, j2, 1 + j}

Exercises (cont.)
Question 6, February 2011

The circular convolution of two discrete-time signals x1[n] and x2[n] produces yet another discrete-time signal y[n] of the same length. How does one recover x2[n] from knowing x1[n] and y[n]? Demonstrate the procedure you described to compute x2[n], assuming x1[n] = {1, 2, 2, 0} and y[n] = {3, 3, 0, 1}. Now, supposing z[n] is the linear convolution of x1[n] and x2[n], how should this procedure be changed so that x2[n] can also be recovered from knowing x1[n] and z[n]?
x2[n] = {1, 1, 0, 1}

Exercises (cont.)
Question 6, September 2011

Using appropriate zero padding where necessary, determine the DFT of x[n] = {1, 0, 0, 1, 1} based on the 8-point decimation-in-time FFT. Without modifying the decimation-in-time FFT structure, explain how it be used to realize the inverse discrete Fourier transform (IDFT). As a hint, express the DFT in vector notation as y = Wx, and proceed from there.
X[k] = {1, 0.7071(1+j), 2j, 0.7071(1+j), 3, 0.7071(1j), 2+j, 0.7071(1j)
y y y/N x x
complex conjugate divide N FFT complex conjugate