Lecture 9

The Discrete Fourier Transform (cont’d)

An orthonormal periodic basis in CN

The goal is to provide a representation of a set of data in terms of periodic basis vectors in CN . First
of all, the following inner product will be used in CN :
N
X −1
hf, gi = f [n]g[n], (1)
n=0

where the bar once again denotes complex conjugation.

Of course, any orthogonal set of complex N -vectors will serve as a basis for CN , but we wish to
use a set of periodic vectors. The family uk ∈ CN examined in the previous lecture will do the trick:
For k = 0, 1, · · · , N − 1, define the vector

uk = (uk [1], uk [2], · · · , uk [N − 1]), (2)

with components  
i2πkn
uk [n] = exp , n = 0, 1, · · · , N − 1. (3)
N
Once again, the index n plays the role of the time or spatial variable and k is the index of the frequency.
Note that in the special case k = 0, all elements uk [n] = 1. In other words, for all N ≥ 2, the
N -vector u0 ∈ CN is a row of 1’s:

u0 = (1, 1, 1, · · · , 1). (4)

This will have important implications in for the DFT.

Let us show once again that the vectors uk are N -periodic. First consider a given k ∈ {0, 1, · · · , N −
1}. Then consider a given component uk [n], n ∈ {0, 1, · · · , N − 1}, in the vector uk . From Eq. (3),
 
i2πk(n + N )
uk [n + N ] = exp
N
   
i2πkn i2πkN
= exp exp
N N
 
i2πkn
= exp exp(i2πk)
N

108
  
i2πkn
= exp
N
= uk [n]. (5)

We claim that the set of N -vectors {uk } forms an orthogonal set in CN . To prove this, consider
the inner product between two elements, uk and ul :
N −1    
X i2πkn i2πln
huk , ul i = exp exp −
N N
n=0
N −1  
X i2π(k − l)n
= exp . (6)
N
n=0

Case 1: k = l. In this case, the above inner product reduces to

N
X −1
huk , ul i = 1 = N. (7)
n=0

Case 2: k 6= l. First let p = k − l, an integer. Then the inner product in (6) becomes
N −1  
X i2πpn
huk , ul i = exp
N
n=0
N
i2πp n
X −1   
= exp
n=0
N
= 1 + r + · · · + r N −1 , (8)
 
i2πp
where r = exp . The sum of this finite geometric series is
N

1 − rN 1 − ei2πp 1−1
S= = = = 0. (9)
1−r 1−r 1−r

Therefore,
huk , ul i = N δkl , (10)

i.e., the set {uk } is an orthogonal set. Therefore it is a basis in CN . In particular, it is the desired
basis because of its internal periodicity. Once again, we may view the n index as a spatial index – in
n
fact, the term plays the role of t or x.
N

109
From this orthogonal basis set {uk }, we construct the orthonormal basis vectors,

1
ek = √ uk , k = 0, 1, · · · , N − 1, (11)
N

with components  
1 i2πkn
ek [n] = √ exp , n = 0, 1, · · · , N − 1. (12)
N N

Once again, the case k = 0 is special. For N ≥ 2,

1
e0 = √ (1, 1, · · · , 1). (13)
N

Before examining a few particular examples, let us make some more general mathematical observations
1
about these basis vectors. We’ll ignore the prefactor √ and examine only the components of the
N
unnormalized vectors introduced earlier, i.e.,
 
i2πkn
uk [n] = exp , n = 0, 1, · · · , N − 1 , (14)
N

for k = 0, 1, · · · , N − 1. In what follows, it is understood that we shall keep N fixed. Perhaps the most
obvious feature is that for all k and n, the magnitudes of all complex coefficients uk [n] are unity. This
follows from the fact that
   
2 i2πkn i2πkn
|uk [n]| = uk [n] uk [n] = exp exp − = 1. (15)
N N

Now rewrite the expression for the coefficient uk [n] as follows,

 
i2πkn
uk [n] = exp
N
n
= αkN , n = 0, 1, · · · , N − 1 , (16)

where  
i2πk
αkN = exp . (17)
N
Note that for a fixed N > 0, αkN is an N th root of unity, i.e.,

[αkN ]N = exp (i2πk) = 1 . (18)

For a fixed N and k, the elements of the vector uk [n] = αnkN in Eq. (16) must therefore lie on the unit
circle C in the complex plane, in agreement with our earlier observation.

110
Let us now recall that the N N th roots of unity are given by
 
i2πp
rp = exp , p = 0, 1, · · · N − 1 . (19)
N

Clearly, the first root is

r0 = 1 . (20)

The N roots rp lie on the unit circle C of the complex plane C as shown in the figure below. Consecutive
2π
roots, i.e., rp and rp+1 , are separated from each other by the angle φ = .
N
Im(z)

Complex plane z = x + iy

Unit circle C
r3
|z| = 1

r2
2π/N

2π/N r1

2π/N
Re(z)
O 1 r0

rN −1

The N roots of unity rp , p = 0, 1, · · · , N − 1, which lie on the unit circle C in the complex plane C.

From Eq. (16), the location of the elements of the vector uk [n] can be determined geometrically.
Starting with n = 0,
uk [0] = α0kN = 1 = r0 , (21)

which is the first root of unity. Each successive element uk [n] for 1 ≤ n ≤ N − 1 is obtained from the
previous one by multiplying by αkN . This represents a rotation in the complex plane by angle

2πk
θ= . (22)
N

Now let us consider some particular cases of k in order to see the pattern:

111
 • When k = 0, there is no rotation: We remain at the point 1. This is in agreement with our
earlier result that the u0 [n] vector is

u0 = (1, 1, 1, · · · , 1) . (23)

2π
• When k = 1, there is a rotation of . The consecutive elements of u1 [n] are obtained by
N
starting at u1 [0] = 1 and taking each successive N th root of unity as an element until we have
taken all N roots.

4π
• When k = 2, there is a rotation of . The consecutive elements of u2 [n] are obtained by
N
starting at u2 [0] = 1 and taking every second N th root of unity as an element, until we have
an N -vector.

3π
• When k = 3, there is a rotation of . The consecutive elements of u3 [n] are obtained by
N
starting at u3 [0] = 1 and taking every third N th root of unity as an element, until we have an
N -vector.

The reader should see the pattern now.

Examples:

1. N = 2:

In this very simple case, one can probably guess the vectors that are generated. First of all,
from Eq. (13), for k = 0,
1
e0 = √ (1, 1). (24)
2
For k = 1,

(a) n = 0:
1 1
e1 [0] = √ exp(0) = √ . (25)
2 2
(b) n = 1:  
1 i2π · 1 · 1 1 1
e1 [1] = √ exp = √ exp (iπ) = − √ . (26)
2 2 2 2
Therefore,
1
e1 = √ (1, −1). (27)
2

112
2. N = 3:

Once again, the case k = 0 is simple. From Eq. (13),

1
e0 = √ (1, 1, 1). (28)
3

For k = 1, using Eq. (12),

(a) n = 0:
1 1
e1 [0] = √ exp(0) = √ . (29)
3 3
(b) n = 1:
  " √ #
1 i2π 1 1 3
e1 [1] = √ exp =√ − + i . (30)
3 3 3 2 2

(c) n = 2:
  " √ #
1 i4π 1 1 3
e1 [2] = √ exp =√ − − i . (31)
3 3 3 2 2
In summary,
√ √ !
1 1 3 1 3
e1 = √ 1, − + i, − − i (32)
3 2 2 2 2

For k = 2, using Eq. (12),

(a) n = 0:
1 1
e2 [0] = √ exp(0) = √ . (33)
3 3
(b) n = 1:
  " √ #
1 i4π 1 1 3
e2 [1] = √ exp =√ − − i . (34)
3 3 3 2 2

(c) n = 2:
  " √ #
1 i8π 1 1 3
e2 [2] = √ exp =√ − + i . (35)
3 3 3 2 2
In summary,
√ √ !
1 1 3 1 3
e2 = √ 1, − − i, − + i (36)
3 2 2 2 2

113
Discrete Fourier Transform, Version 1

We now employ the orthonormal basis developed above to construct our first version of the DFT. Any
element f ∈ CN will have an expansion of the form
N
X −1
f= hf, ek iek . (37)
k=0

Recall that the elements of the basis vector ek are given by

 
1 i2πkn
ek [n] = √ exp , n = 0, 1, · · · , N − 1 . (38)
N N

In component form,
N
X −1 N
X −1
f [n] = hf, ek iek [n] = c[k]ek [n], (39)
k=0 k=0

where the c[k] = hf, ek i denote the Fourier coefficients of f in the ek basis. Let us now examine these
coefficients:

c[k] = hf, ek i
N
X −1
= f [n]ek [n], (40)
n=0

or
N −1  
1 X i2πkn
c[k] = √ f [n] exp − , k = 0, 1, · · · , N − 1. (41)
N n=0 N

This relation defines a discrete Fourier transform (DFT) of f . The components of the vector
c = (c[1], c[2], · · · , c[N − 1]) comprise the DFT of the vector f = (f [1], f [2], · · · , f [N − 1]). Mathemat-
ically, we can write
c = F(f ), (42)

where F : CN → CN denotes the discrete Fourier transform operator on complex N -vectors.

Important comment: Note the choice of “a” instead of “the” before “discrete Fourier transform.”
Unfortunately, there are several closely-related definitions, and it is important to recognize this fact.
For this reason, we refer to the above DFT as DFT, Version 1.

114
 Let us return to Eq. (41) to show that, indeed, the DFT vector c with components c[k] is N -
periodic:
N −1  
1 X i2π(k + N )n
c[k + N ] = √ f [n] exp −
N n=0 N
N −1    
1 X i2π(k)n i2πN n
= √ f [n] exp − exp −
N n=0 N N
N −1  
1 X i2πkn
= √ f [n] exp − exp (−i2πn)
N n=0 N
N −1  
1 X i2πkn
= √ f [n] exp −
N n=0 N
= c[k]. (43)

Eq. (41) is the definition of the discrete Fourier transform implemented in the MAPLE programming
language. In MAPLE, the relation would be written as

c = FourierTransform(f)
f = InverseFourierTransform(c),

where we still have to define the inverse DFT.

Mathematically, the above formula is elegant because of the following result,

kf k2 = kck2 , (44)

where k · k2 denotes the L2 norm defined by the complex inner product in CN . To see this:

kf k22 = hf, f i
N
X −1
= f [n]f [n]
n=0
N −1
"N −1 # "N −1 #
X X X
= c[k]ek [n] c[l]el [n]
n=0 k=0 l=0
N −1 N −1
"N −1 #
X X X
= c[k]c[l] ek [n]el [n]
k=0 l=0 n=0
N
X −1 N
X −1
= c[k]c[l]hek , el i
k=0 l=0

115
 N
X −1
= c[k]c[k]
k=0
= kck22 , (45)

from which (44) follows. This means that the DFT operator F is norm-preserving, i.e., the norm
of c is the norm of f .

Inverse DFT, Version 1

Let us now see if we can find a result for the inverse discrete Fourier transform, i.e., given the
DFT c, how can we find f , written mathematically as

f = F −1 (c). (46)

In order to invert relation (41), we shall utilize the orthonormality of the ek vectorsin (12).For a
1 i2πkm
particular value of m ∈ {0, 1, · · · , N − 1}, multiply both sides of Eq. (41) by √ exp and
N N
then sum over k:
N −1   N −1 N −1  
1 X i2πkm 1 X X i2πk(m − n)
√ c[k] exp = f [n] exp . (47)
N N N N
k=0 n=0 k=0

We have already seen earlier that the final summation is N δmn . Thus, for each m, only the term
n = m from the sum over n contributes. As a result, we have
N −1  
1 X i2πkm
f [m] = √ c[k] exp . (48)
N k=0 N

This relation is true for each m = 0, 1, · · · , N − 1. It is customary to let n denote the spatial or time
variable, so we rewrite the above result as
N −1  
1 X i2πkn
f [n] = √ c[k] exp , n = 0, 1, · · · , N − 1. (49)
N k=0 N

These relations comprise the inverse discrete Fourier transform (IDFT) associated with the DFT
in Eq. (41).
A closer look at Eq. (49) shows that, in fact, the inverse DFT is nothing more than the expansion
of the discrete vector f in terms of the orthonormal basis {ek }. the DFT coefficients c[k] are used to
construct the “signal” elements f [n].

We now summarize the results obtained above:

116
DFT and IDFT, Version 1

N −1  
1 X i2πkn
c[k] = √ f [n] exp − , k = 0, 1, · · · , N − 1,
N n=0 N
N −1  
1 X i2πkn
f [n] = √ c[k] exp , n = 0, 1, · · · , N − 1. (50)
N N
k=0

DFT and IDFT, Version 2

A second version of the DFT and its inverse is employed in many mathematics books (e.g., the book
by Kammler). Unlike the first version, it is not symmetric. But there is a legitimate reason for its
definition, since it arises naturally from a discretization of the integrals used to compute Fourier series
coefficients. We shall postpone the discussion of this result to another lecture. For the moment, we
simply state the second version of the DFT.
The DFT, Version 2 is defined as follows,
N −1  
1 X i2πkn
F [k] = f [n] exp − , k = 0, 1, · · · , N − 1. (51)
N n=0 N

Note the use of F to denote the DFT: It is customary to let capital letters denote the FT/DFTs of
functions. The only difference between this version and Version 1 in Eq. (41) is that the factor in front
√
is 1/N instead of 1/ N . In the same manner as was done for Version 1, the inverse DFT associated
with the above DFT is given by
N −1  
X i2πkn
f [n] = F [k] exp , n = 0, 1, · · · , N − 1. (52)
N
k=0

DFT and IDFT, Version 3

This is the version that appears in most of the signal processing literature (e.g. Mallat) as well as
mathematics books that deal with signal processing applications (e.g., Boggess and Narcowich). It
appears to be the version that is most widely used by research workers in signal and image processing,
as witnessed by the fact that it is the version implemented in MATLAB. As such, unless specified
otherwise, this will be the version used in this course.
The DFT, Version 3 is defined as follows,
N −1  
X i2πkn
F [k] = f [n] exp − , k = 0, 1, · · · , N − 1. (53)
N
n=0

117
There is no factor in front of the summation. The inverse DFT associated with the this DFT is given
by
N −1  
1 X i2πkn
f [n] = F [k] exp , n = 0, 1, · · · , N − 1. (54)
N N
k=0

In MATLAB, the DFT and IDFT are denoted as follows,

F = fft(f),
f = ifft(F).

Using the orthogonality property of the complex exponential functions established earlier, i.e., huk , ul i =
N δkl , it can be shown (a simple modification of the derivation for DFT, Version 1, Eq. (45)) that this
particular version of the DFT satisfies the relation,

1
kf k22 = kF k22 , (55)
N

where k · k2 denotes the L2 /Euclidean norm on CN . From this point onward, we shall omit the
subscript 2 from the norm and write k · k, with the understanding that it represents the L2 norm.

118
Lecture 10

Discrete Fourier Transform (cont’d)

Matrix form of DFT

You’ll note that all the coefficients multiplying the f [n] elements in the DFT of Eq. (53) involve
powers of the complex number
     
i2π 2π 2π
ω = exp − = cos − i sin . (56)
N N N

A closer examination shows that if f and F are written as column N -vectors, f and F, respectively,
then the DFT relation in (53) may be written in matrix form as

F(f ) = Ff , (57)

where F is an N × N complex matrix having the form

 
1 1 1 ··· 1
 
1 ω ω2 ··· ω N −1
 
 
 
1 ω2 ω4 ··· ω 2N −2 (58)
 
 
.. .. .. ..
 
 

 . . . . 

1 ω N −1 ω 2N −2 · · · ω (N −1)(N −1)

The kth entry of the vector F is given by

F [k] = f [0] + ω k f [1] + ω 2k f [2] + · · · + ω (N −1)k f [N − 1]

= f [0] + f [1]ω k + f [2](ω k )2 + · · · f [N − 1](ω k )N −1 , (59)

where the second line indicates that F [k] is a polynomial in ω k . This suggests that it may be evaluated
recursively, as opposed to computing the terms separately and adding them up. The following is a
pseudocode version of “Horner’s algorithm” for computing the entire vector F :

z:=1
ω = e−i2π/N
for k=0,1,...,N-1 do
S:=f[N-1]

119
 for l=2,3,...,N do
S:=z*S+f[N-l]
od
F[k]:=S
z:=z*ω
od

In this form, the computation of the DFT F requires N 2 complex operations, which translates to
4N 2 real operations. For special values of N , the procedure can be optimized, utilizing the fact that
ω is a root of unity. This is the basis of the fast Fourier transform (FFT) which we may discuss a
little later in the course.

Further investigation of the DFT

We now examine the DFT a little further, with the help of some examples. As mentioned earlier, we
shall be using the DFT, Version 3 – the “MATLAB” formula – summarized again below:

DFT and IDFT, Version 3

N −1  
X i2πkn
F [k] = f [n] exp − , k = 0, 1, · · · , N − 1. (60)
n=0
N
N −1  
1 X i2πkn
f [n] = F [k] exp , n = 0, 1, · · · , N − 1. (61)
N N
k=0

An important note: As we know from before, the DFT and IDFT may be viewed as inner
products between appropriate N -vectors. In the special case k = 0, all of the complex exponentials in
Eq. (60) are equal to 1. This is because the particular element F [0] is the inner product between the
N -vector f and the unnormalized N -vector u0 = (1, 1, · · · , 1). As such,
N
X −1
F [0] = hf, u0 i = f [n] . (62)
n=0

120
Some examples:
For N = 4:

1. f = (1, 1, 1, 1), F = (4, 0, 0, 0) (This illustrates Eq. (62) above.)

2. g = (0, 1, 0, 1), G = (2, 0, −2, 0)

3. h = (1, 2, 1, 2), H = (6, 0, −2, 0).

4. a = (1, 2, 3, 4), A = (10, −2 + 2i, −2, −2 − 2i).

Comments:

• In 1, the signal f is a constant signal, i.e., no variation. This means that the only frequency
component is zero frequency, i.e., k = 0. This is why the first element, k = 0, corresponding to
the constant vector u0 is the only nonzero component of F . The signal f is orthogonal to all
other vectors uk , k 6= 0.

• In 2, the signal g has period 2, i.e., it oscillates with twice the periodicity of signal f (period 4).
This accounts for the nonzero entry G[2] = −2.

• Note that the third result is in accordance with the linearity of the DFT: h = f + g implies that

H = F(h) = F(f + g)

= F(f ) + F(g)

= F + G. (63)

• The result in 4 shows that a real-valued signal can have a complex-valued DFT.

Each of the four results above demonstrates the modified Parseval equality for the DFT, Version
3 mentioned earlier, i.e.,
1
kf k2 = kF k2 . (64)
N

121
Some more complicated examples:

5. We consider the function f (x) = cos(2x) defined on the interval 0 ≤ x ≤ 2π. From this function
we construct N = 256 equally-spaced samples,
2πn
f [n] = f (xn ) = cos(2xn ), xn = , n = 0, 1, · · · , N − 1. (65)
N
The samples are plotted on the left in the figure below.
2.5 150

125
1.5

1
100

0.5

|F(k)|
0 75

-0.5

50
-1

-1.5
25

-2

-2.5 0
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 5.5 6 6.5 0 50 100 150 200 250 300
x k

Sampled signal f [n] = cos(2xn ), n = 0, 1, · · · , 255 and magnitudes |F [k]| of its DFT.

Numerically, we find that all DFT coefficients F [k] are zero, except for two elements:

F [2] = 128, F [254] = 128. (66)

A plot of the magnitudes |F [k]| of the DFT coefficients is presented on the right in the figure
below. The nonzero entry for F [2] picks out the k = 2 frequency of the signal. We’ll see later
that the F [254] component does likewise.

Numerically, we also find that

255
X
kf k2 = |f [n]|2 = 128 (to two decimals), (67)
n=0

and
1 1 1
kF k2 = (F [2]2 + F [4]2 ) = (1282 + 1282 ) = 128. (68)
N 256 256
Thus, Eq. (55) is satisfied.

6. Now consider the function f (x) = sin(2x) defined on the interval 0 ≤ x ≤ 2π. From this function
we construct N = 256 equally-spaced samples,
2πn
f [n] = f (xn ) = sin(2xn ), xn = , n = 0, 1, · · · , N − 1. (69)
N

122
The samples are plotted on the left in the figure below.
1.5 150

1 125

0.5 100

|F(k)|
f[n]

0 75

-0.5 50

-1 25

-1.5 0
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 5.5 6 6.5 0 50 100 150 200 250 300
x k

Sampled signal f [n] = sin(2xn ), n = 0, 1, · · · , 255 and magnitudes |F [k]| of its DFT.

Numerically, we find that all DFT coefficients F [k] are zero, except for two elements:

F [2] = −128i, F [254] = 128i. (70)

Of course, there is a similarity between this spectrum and that of Example 5 in that the peaks
coincide at k = 2 (and 254), corresponding to the common frequency k = 2. As such, a plot
of the magnitudes |F [k]| of the DFT coefficients, presented on the right in the figure below, is
identical to the corresponding plot of Example 5.

On the other hand, the DFT coefficients for the sin(2x) function are complex. In fact, they are
purely imaginary. The coefficient F [2] for the sin(x) function is obtained from the coefficient of
the cos(x) function by multiplication by i = eiπ/2 . This might have something to do with the
fact that the sin(x) function is a shifted version of the cos(x) version. More on this later.

The moral of the story is that the magnitudes |F [k]| do not contain all of the information
about a signal. If we write
F [k] = |F [k]|eiφk , (71)

then the phases φk also contain information about the signal and cannot be ignored.

Numerically, we also find that

255
X
kf k2 = |f [n]|2 = 128 (to two decimals), (72)
n=0

and
1 1
kF k2 = (1282 + 1282 ) = 128. (73)
N 256

123
 Thus, Eq. (55) is satisfied.

7. We now consider the function f (x) = sin(2x) + 5 sin(5x) defined on the interval 0 ≤ x ≤ 2π. We
have added a higher-frequency term to the function of Example 5.

From this function we again construct N = 256 equally-spaced samples,

2πn
f [n] = f (xn ) = sin(2xn ) + 5 sin(5xn ), xn = , n = 0, 1, · · · , N − 1. (74)
N
The samples are plotted on the left in the figure below. Numerically, we find that all DFT
10 800

8
700

600

500
2 |F(k)|
f[n]

0 400

-2
300

-4

200

-6

100
-8

-10 0
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 5.5 6 6.5 0 50 100 150 200 250 300
x k

Sampled signal f [n], n = 0, 1, · · · , 255 and magnitudes |F [k]| of the DFT, for Example 7.

coefficients F [k] are zero, except for four elements:

F [2] = −128i, F [254] = 128i, (75)

as expected, corresponding to the sin(2x) component, and

F [5] = −640i, F [251] = 640i, (76)

corresponding to the sin(5x) component. From the linearity of the DFT, the DFT of the sum
of these two functions is the sum of their DFTs. Also note that the ratio of amplitudes of these
two sets follows the 1 : 5 ratio of the sin(2x) and sin(5x) components. A plot of the magnitudes
|F [k]| of the DFT coefficients is presented on the right in the figure below.

Numerically, we also find that

255
X
kf k2 = |f [n]|2 = 3328. (77)
n=0
and
1 1
kF k2 = (2 ∗ 1282 + 2 ∗ 6402 ) = 3328, (78)
N 256

124
 once again in accordance with Eq. (55).

8. Let us now generalize the results from the previous two examples. Suppose that we have the
complex-valued function f (x) = exp(ik0 x), defined on the interval [0, 2π], with k0 an integer.
For the moment, we assume that k0 ∈ {0, 1, 2, · · · , N − 1}. From this function, we extract N
equally-spaced samples at the sample points xn = 2πn/N , n = 0, 1, · · · , N − 1, i.e,
 
i2πk0 n
f [n] = f (xn ) = exp(ik0 xn ) = exp , n = 0, 1, · · · , N − 1. (79)
N

The function f (x) is 2π-periodic. But what is its DFT?

By definition, its DFT is given by

N −1  
X i2πkn
F [k] = f [n] exp −
N
n=0
N −1    
X i2πk0 n i2πkn
= exp exp − . (80)
n=0
N N

Now recall from the previous lecture that the discrete exponential functions uk ,
N −1  
X i2πkn
uk [n] = exp − , (81)
N
n=0

an orthogonal set on CN , i.e.,

huk , ul i = N δkl . (82)

This means that the F [k] = 0 in Eq. (80) unless k = k0 . In other words, the N -point DFT of
the exponential function exp(ik0 x) sampled on [0, 2π] is given by

F [k] = N δkk0 . (83)

The DFT consists of a single peak of magnitude N at k = k0 .

But wait just one minute!

The above result applies to the case k0 ∈ {0, 1, 2, · · · , N − 1}. What happens if it is not in the set
of frequencies {0, 1, 2, · · · , N − 1} covered by the DFT? For example, if N = 256, what happens if
k0 = 260? Will the function be oscillating too quickly to be detected?

125
 The answer is “No, it will be detected.” Somehow, one gets the feeling that everything here happens
“modulo N ,” because of the periodicity of the vectors. And that is what happens in frequency space
as well. Let us replace k0 in Eq. (80) with k0 + N . Then the RHS becomes

N −1     N −1      
X i2π(k0 + N)n i2πkn X i2πk0 n i2πkNn i2πkn
exp exp − = exp exp exp −
n=0
N N n=0
N N N
N −1    
X i2πk0 n i2πkn
= exp exp (i2πkn) exp −
n=0
N N
N −1    
X i2πk0 n i2πkn
= exp exp − . (84)
n=0
N N
In other words, the result in Eq. (80) is unchanged. Therefore the same result holds for k0 + pN ,
where p is an integer. This implies that a peak will show up at k = k0 mod N . So the final result is:

The N -point DF T of the sampled function exp(ik0 x), 0 ≤ x ≤ 2π, i.e., f [n] given in Eq.
(79), is given by a single peak:

 N, k = k0 mod N,
F [k] = (85)
 0, otherwise.

From this property, we may now go back and verify that the calculations of Examples 5-7 of the
previous lecture, involving sine and cosine functions, are correct.

1. Since
1 1
cos(k0 x) = eik0 x + e−ik0 x , (86)
2 2
it follows, from the linearity of the DFT, and the “modulo N ” property derived above, that the
N -point DFT of cos(k0 x) consists of two peaks of height N/2, i.e.,

 N2 ,

 k = k0 ,

F [k] = N (87)
 2, k = N − k0 ,


 0, otherwise.

The peak at k = N − k0 comes from the “modulo N” property. The second exponential in Eq.
(86) would produce a peak at k = −k0 which, in turn, because of the N -periodicity of the DFT,
produces a peak at k = −k0 + N = N − k0 .

Note that this result is in agreement with the computation in Example 5 above.

126
 2. Now consider the correspoinding sine function,

1 ik0 x 1
sin(k0 x) = e − e−ik0 x . (88)
2i 2i

It follows, once again from the linearity of the DFT, and the “modulo N ” property derived
above, that the N -point DFT of sin(k0 x) consists of the following two peaks,

N N
 2i = − 2 i, k = k0 ,



F [k] = −N N
2i = 2 i, k = N − k0 ,
(89)



 0, otherwise.

Note that these peaks have the same magnitudes as for the cosine case, but that they are now
complex. Moreover, the two peaks of the DFT of the sine function are complex conjugates of
each other. More on this later.

For the moment, we note that this result is in agreement with the computation in Example 6
above.

3. Finally, notice that if we add up the DFTs of the sine and cosine function appropriately, we
retrieve the DFT of the exponential, i.e.,

F(cos(k0 x) + i sin(k0 x)) = F(exp(ik0 x)). (90)

The peaks of the cosine and sine at k = k0 “constructively interfere” whereas their peaks at
k = N − k0 “destructively interfere.” The result is a single peak of height N at k = k0 .

We now consider a slightly perturbed version of Example 6 of the previous lecture, namely the function
f (x) = sin(2.1x) defined on the interval 0 ≤ x ≤ 2π. From this function we construct N = 256 equally-
spaced samples,

2πn
f [n] = f (xn ) = sin(2.1xn ), xn = , n = 0, 1, · · · , N − 1. (91)
N

The samples are plotted on the left in the figure below. Note that this signal is not 2π-periodic, but
the sampling and resulting DFT produces a 2π-periodic extension. As such, there is a significant jump
between f [255] and f [256] = f [0].
This time, we find that the DFT spectrum of coefficients F [k] is not as simple as in the first two
examples. First of all, with the exception of F [0] ≈ 3.41068, all DFT coefficients are complex, i.e.,

127
have nonzero imaginary part. A plot of the magnitudes |F [k] of the DFT coefficients is presented on
the right in the figure below. There is still a dominant peak at k = 2, but it is not a singular peak –
it is somewhat diffuse.
1.5 150

1 125

0.5 100

|F(k)|
f[n]

0 75

-0.5 50

-1 25

-1.5 0
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 5.5 6 6.5 0 50 100 150 200 250 300
x k

Sampled signal f [n] = sin(2.1xn ), n = 0, 1, · · · , 255 and magnitudes |F [k]| of the DFT, for Example 9.

In order to show the diffuseness of the DFT spectrum, the coefficients are plotted on a different
scale so that the enormous peaks at k = 2 and 254 do not mask their behaviour as in the previous
plot.
20

15
|F(k)|

10

0
0 50 100 150 200 250 300
k

Plot of magnitudes |F (k)| of DFT of sin(2.1xn ) signal of Example 9, magnified to show the diffuse
structure around the dominant peaks at k = 2 and 254.

Ideally, the DFT would like to place a peak at the frequency k = 2.1, but it doesn’t exist. As such,
the dominant peaks are found at k = 2 and 254. But all other frequencies are need to accomodate
this “nonexistent” or irregular frequency – note that their contribution decreases as we move away
from the peaks.

128
 If this appears to be a rather “bizarre” phenomenon, just go back and think about the Fourier
(sine) series of this function, i.e.,
∞
X
sin(2.1x) = bk sin(kx). (92)
k=1

In fact, the coefficients bk can be computed rather easily, and one observes that they produce a
somewhat “diffuse” Fourier spectrum that peaks at k = 2.

The reader may wish to examine the effect of further perturbing the frequency of the sampled
signal, i.e., the function f (x) = sin(2 + ǫ)x as ǫ is increased. For example, will the k = 3 (and 253)
components of the DFT increase in magnitude. And for ǫ > 0.5, does k = 3 “take over” in magnitude?

Finally, let us examine the DFT of the exponentially damped cosine signal f (x) = e−x cos(50x),
sampled with N = 256 equally-spaced points on [0, 2π]:

2πn
f [n] = f (xn ) = e−xn cos(50xn ), xn = n = 0, 1, · · · , N − 1 . (93)
N

A plot of the signal is shown at the left of the figure blow. The magnitudes |F [k]| of the DFT
coefficients of this signal are plotted at the right.
2.5 30

1.5

1
20

0.5
|F(k)|

-0.5

10
-1

-1.5

-2

-2.5 0
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 5.5 6 6.5 0 50 100 150 200 250
x k

Sampled signal f [n] = exp(−t) cos(50xn ), n = 0, 1, · · · , 255 and magnitudes |F [k]| of the DFT,

We know that the DFT of the undamped signal cos(50xn ) would have two single peaks of magni-
tude N/2 = 128 at k = 50 and k = 256 − 50 = 216. The DFT of the exponentially damped also has
dominant peaks at these locations, but they are much smaller in magnitude and more diffuse, as was
observed for the sin(2.1xn ) signal.

129
Some properties of DFTs

We now establish a few properties of DFTs which are discrete analogues of properties of Fourier
Transforms that you may have seen.

Linearity

F(f + g) = Ff + Fg. (94)

This is quite trivial to show. Let h = f + g, F = Ff and G = Fg. By definition, for 0 ≤ k ≤ N − 1,

N −1  
X i2πkn
H[k] = h[n] exp −
N
n=1
N −1  
X i2πkn
= (f [n] + g[n]) exp −
n=1
N
N −1   NX −1  
X i2πkn i2πkn
= f [n] exp − + g[n] exp −
N N
n=1 n=1
= F [k] + G[k], (95)

which proves Eq. (94).

Conjugate symmetry

Let F be the DFT of a sampled real signal f of N points. Then

F [k] = F [N − k], k = 0, 1, · · · , N − 1. (96)

Proof: By definition,
N −1  
X i2πkn
F [k] = f [n] exp − , (97)
N
n=1
so that
N −1  
X i2π(N − k)n
F [N − k] = f [n] exp −
n=1
N
N −1  
X i2πkn
= f [n] exp exp(−i2πn)
n=1
N
N −1  
X i2πkn
= f [n] exp
N
n=1
= F [k], (98)

and the desired result follows.

130
Shift Theorem

Let
f = (f [0], f [1], · · · , f [N − 1]) and g = (f [1], f [2], · · · , f [N − 1], f [0]). (99)

In other words, g is obtained from f by shifting the sequence of sample values one space to the left.
We may consider the first element f [0] to be “wrapped-around” and placed at the end, or we may view
it as coming from the next N data points, since f [N ] = f [0]. Then the DFT coefficients of G = Fg
and F = Ff are related as follows,
 
i2π
G[k] = ω −k
F [k], k = 0, 1, · · · , N − 1, where ω = exp − . (100)
N

Proof: By defnition,
N −1  
X i2πkn
G[k] = g[n] exp − ,
n=0
N
N −1  
X i2πkn
= f [n + 1] exp −
n=0
N
N −1    
X i2πk(n + 1) i2πk
= f [n + 1] exp − exp
N N
n=0
  N −1  
i2πk X i2πk(n + 1)
= exp f [n + 1] exp −
N n=0
N
= ω −k F [k]. (101)

Remarks:

1. The Shift Theorem may be applied repeatedly. For example, if gM is obtained by left-shifting
the entries of f M times, i.e.,

f = (f [0], f [1], · · · , f [N − 1]) and gM = (f [M ], f [M + 1], · · · , f [M − 2], f [M − 1]), (102)

then (Exercise)
GM [k] = ω −M k F [k] , k = 0, 1, · · · , N − 1 . (103)

2. The Shift Theorem may also be applied to right-shifted sequences. One suspects that if the
N -vector is obtained by right-shifting the entries of f once, i.e.,

f = (f [0], f [1], · · · , f [N − 1]) and g = (f [N − 1], f [0], f [2], · · · , f [N − 2]) , (104)

131
 then
G[k] = ω k F [k] , k = 0, 1, · · · , N − 1 . (105)

We leave this as an exercise. Furthermore, we may also apply the right-shift operator M times.
We also leave this as an exercise.

3. We also leave it as an exercise for the reader to show how the Shift Theorem may be used to
derive the DFT of the sin(2xn ) sequence of Example 2 of the previous lecture from the cos(2xn )
sequence of Example 1 (or vice versa).

132
Lecture 11

Discrete Fourier transform (cont’d)

Convolution Theorem

Let f and g be two N -periodic complex vectors. Define the (circular) convolution of these two vectors
as the vector h with components
N
X −1
h[n] = f [j]g[n − j], n = 0, 1, · · · , N − 1. (106)
j=0

Then the DFT of h is related to the DFTs of f and g as follows,

H[k] = F [k]G[k], (107)

in other words, the pointwise product – the discrete analogue of multiplying two functions together,
i.e., f (x)g(x).

Proof:
N −1  
X i2πkn
H[k] = h[n] exp − ,
n=0
N
 
N −1 N −1  
X X i2πkn
=  f [j]g[n − j] exp −

n=0
N
j=0
N −1 N −1    
X X i2πkj i2πk(n − j)
= f [j] exp − g[n − j] exp −
n=0 j=0
N N
N −1   NX
−1  
X i2πkj i2πk(n − j)
= f [j] exp − g[n − j] exp −
N N
j=0 n=0
 "
N −1   NX −1  #
X i2πkj i2πkl
=  f [j] exp −  g[l] exp −
N N
j=0 l=0
= F [k]G[k]. (108)

The second-to-last line in the sequence follows from the fact that the products f [j]g[n − j] exhaust all
possible pairs since the vectors are N -periodic.

133
Some consequences of properties on DFTs

We now examine some consequences of the properties of DFTs derived in the previous lecture.

Conjugate Symmetry

Recall that if f ∈ RN is a real-valued signal, then its DFT F has conjugate symmetry of the form

F [N − k] = F [k], k = 0, 1, · · · , N. (109)

Assuming that N is even, this implies that we need only compute at most the first N/2 coefficients,
F [0], F [1], · · · , F [N/2 − 1]. The other coefficients may be computed by complex conjugating these
coefficients. But there are some other interesting features.
First of all, note that for k = 0, conjugate symmetry implies that F [N ] = F [0]. But by definition,
N
X −1
F [0] = f [n], (110)
n=0

which is real-valued. Therefore F [N ] = F [0] ∈ R. If N is even, then setting k = N/2 implies that

F [N/2] = F [N/2], (111)

implying that F [N/2] ∈ R. All other DFT coefficients F [1], F [2], · · · F [N/2 − 1] are complex. If
we consider a real number to represent one degree of freedom and a complex number two degrees
of freedom, then the total number of “degrees of freedom” represented by the DFT coefficients
F [0], F [1], · · · , F [N/2 − 1] is 1 + 2 ∗ (N/2 − 1) + 1 = N . This is the number of degrees of freedom
represented by the original data vector f ∈ RN .

Finally, we note that Examples 1 and 2 of the previous lecture, the cos(2xn ) and sin(2xn ) data
sets, along with Example 3, where an additional sin(5xn ) set was added, demonstrated conjugate
symmetry.

High- and low-frequency DFT coefficients

From Eq. (109), it follows that

|F [N − k]| = |F [k]|, k = 0, 1, 2, · · · , N − 1. (112)

134
In other words, a plot of the magnitudes of DFT coefficients |F [k]| will be symmetric with respect
to the line k = N/2. The plots of DFT coefficient magnitudes presented in the previous lecture all
demonstrate this symmetry.
From our discussions of Fourier series expansions, we know that the magnitudes of the coefficients
an and bn of cos(nx) and sin(nx), respectively, decay to zero in the limit n → ∞. Since the DFT is
based on complex exponentials, we also expect the DFT coefficients |F [k]| to decay with increasing k.
But from the conjugate symmetry property, the region of high-frequency DFTs is centered about the
value k = N/2, as sketched below.

|F [k]|

k
0 N/2 N −1

increasing oscillation increasing oscillation

low frequency highest frequency low frequency

region region

General behaviour of magnitudes |F [k]| of DFT coefficients.

In applications, it is often more desirable to consider a slightly revised version of this plot of DFT
coefficient magnitudes. Recalling that the DFT values F [k] are N -periodic, the right-half of the above
figure is identical to the plot of DFT coefficients immediately to the left of the point k = 0. As such,
we may consider the plot of F [k] magnitudes centered at k = 0, as shown below. In this way, moving
away from k = 0, in either direction corresponds to higher frequencies.
We shall discuss the region of high-frequency DFT coefficients very shortly.

“Thresholding” of DFT coefficients as a method of data compression

Data compression is a fundamental area of research and development in signal and image processing.
Practically speaking, you’d like to get as many songs or images on a CD – or your iPhone - as you
can. But you probably know that in order to squeeze more songs on a device, you have to “compress”

135
 |F [k]|

k
−N/2 + 1 0 N/2
high frequency low frequency high frequency
region region region

A revised version of the above plot of the magnitudes |F [k]| of DFT coefficients.

the digital data sets representing the songs/images, i.e., reduce the storage space required for each
item. But reducing the storage space means throwing out some information, implying a reduction
in the quality or fidelity of the song or image. Because of the redundancy of signals and images,
some compression is possible without any noticeable changes in aural or visual quality. But as the
compression is pushed higher, noticeable distortions eventually appear – for example, echoing or hissing
in the case of audio and blockiness or blurring in the case of images.
It is not the purpose of this course to study data compression methods in any detail. However,
our study of Fourier, and later wavelet, transforms naturally takes us to some of the basic concepts
that underlie compression methods.
The truncation of the Fourier series expansion of a function f (x), i.e., approximating f (x) by the
partial sum SN (x) may be viewed as a compression method. In general, it is impossible to store all
Fourier coefficients an and bn . We know that for a function f ∈ L2 [a, b], the coefficients an and bn
decay to zero: for a sufficiently large N , all coefficients for which n > N will be “negligible.” (Of
course, as we have discussed, N will depend on the decay of the coefficients which, in turn, depends
on the regularity of the function f .

In the case of discrete data sets, i.e., the discrete Fourier transform, we already have a finite
number of coefficients representing our signal/image of interest. Given a signal f ∈ RN , the N DFT
coefficients |F [k]|, k = 0, 1, · · · , N − 1 permit a perfect reconstruction of f . We now wish to perform
compression on this data set.
That being said, it is probably useful to mention one simple, yet important, fact: You can’t

136
perform compression by simply deleting the signal values f [n]. That would be too brutal.
That is not to say that you couldn’t exploit the redundancy of the signal, e.g., that contiguous
elements – f [n] and f [n + 1] – of the signal are generally close in magnitude. You could do this by
keeping f [n] and perhaps using a single digit, say “1”, to indicate that f [n] is repeated in the next
data element. Or you could register the difference between f [n] and f [n + 1] which generally would
require less storage space. This is the essence of “predictive coding.” But this is getting us deeper
into the subject of data compression and away from the course. Here, we simply wish to show that
one can work on the discrete transforms – Fourier and wavelet – of signals.
One of the simplest methods of performing compression is thresholding, i.e., the rather brutal
removal of “insignificant coefficients” in a discrete Fourier or wavelet transform. (The words sig-
nificant and insignificant are heavily used in signal/image compression literature, especially with
regard to wavelets.) The idea is simple: You set a threshold value ǫ > 0, and delete all coefficients
with magnitudes less than ǫ. Let’s present this a little more mathematically:

Thresholding algorithm: Let f = (f [0], f [1], · · · , f [N − 1]) ∈ RN (or CN ) represent our signal of
interest, with DFT,
F = Ff = (F [0], F [1], · · · , F [N − 1]) ∈ CN . (113)

Let ǫ ≥ 0 be a threshold parameter. Now define the new sequence

F̃ǫ = (F̃ǫ [0], · · · , F̃ǫ [N − 1]), (114)

as follows, 
 F [k], if |F [k]| ≥ ǫ,
F̃ǫ [k] = (115)
 0, if |F [k]| < ǫ.
Then define
f˜ǫ = F −1 F̃ǫ . (116)

f˜ǫ is the “compressed signal” corresponding to the threshold parameter ǫ. Of course, f˜0 = f , since
you haven’t thrown away any DFT coefficients. We expect that for “small” values of ǫ (the question,
of course, is “What is small?” It will depend upon the signal, in particular, on entire ensemble of
DFT coefficients F [k].) f˜ǫ should well approximate f . This could be summarized in the statement

kf − f˜ǫ k → 0 as ǫ → 0. (117)

The quantity on the left may be viewed as the approximation error.

137
 Of course, the question that remains is, “What values of ǫ do we use?” This depends on the
signal and the nature of the DFT spectrum F [k]. One could, for example, sort the DFT coefficients
in decreasing order of magnitude and then decide where to “cut.” Instead of prescribing an ǫ-value,
one may also decide to throw away a prescribed percentage of DFT coefficients based on their
insignificance, e.g., the 10% most insignificant coefficients.

Example: We now illustrate this method with a simple example. Consider the following function

2 /10
f (x) = e−x [sin(2x) + 2 cos(4x) + 0.4 sin(x) sin(10x)], 0 ≤ x ≤ 2π. (118)

(This function was also used in the book of Boggess and Narcowich, p. 139.) We sample this function
on [0, 2π] with N = 256, i.e.,

2πn
f [n] = f (xn ), xn = , n = 0, 1, · · · , 255. (119)
N

This original sampled signal is plotted on the top left of the next figure. Consecutive data points f [n]
have been connected with straight lines so that the signal may be seen more clearly. On the top left
is a plot of the magnitudes of the DFT coefficients F [k] of this signal. Note that the 2π-extension of
this signal is not continuous at the endpoints, since f (0) 6= f (2π).
The sets of plots that follow show the results of thresholding for increasing values of ǫ: 1.5,
2.5, 10.0, 20.0 and 40.0. In each case, the signal f˜ and thresholded DFT spectrum F̃ are shown.
Also, the percentage of coefficients retained by the thresholding procedure is given, as well as the
L2 (Euclidean) distance kf − f˜k between f and f˜, as well as the percentage relative error of
approximation, computed as follows,
kf − f˜k
× 100. (120)
kf k
There are some noteworthy features demonstrated in these plots:

1. First of all, as expected, as ǫ increases, more and more of the inner “high frequency” region of
the DFT spectrum gets deleted.

2. Also as expected, the L2 (Euclidean) distance kf − f˜k increases as ǫ increases.

3. Note that at small ǫ values, e.g., ǫ = 1.5, the most significant error/distortion to the signal
occurs in the region 5 ≤ x ≤ 6. Indeed, one observes “ringing” there. This is because of the
discontinuity of the 2π-periodic extension of f : The value of f (x255 ) is close to zero, whereas

138
 Thresholding: Given an ǫ > 0, remove all DFT coefficients |F [k]| < ǫ.
2.5 125

1.5 100

0.5 75

|F(k)|
0

-0.5 50

-1

-1.5 25

-2

-2.5 0
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 5.5 6 6.5 0 50 100 150 200 250
x k

Left: Original signal f [n], n = 0, · · · , 255, obtained by sampling f (x) in Eq. (118). Right: Magnitudes |F [k]|

of DFT coefficients. kf k = 12.95.

2.5 125

1.5 100

0.5 75
|F(k)|

-0.5 50

-1

-1.5 25

-2

-2.5 0
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 5.5 6 6.5 0 50 100 150 200 250
x k

Threshold ǫ = 1.5. 44.9% of original coeffs retained. Reconstructed signal f˜[n] and magnitudes |F̃ [k]|. L2

error kf − f˜k2 = 0.83. Relative L2 error 6.3%.

2.5 125

1.5 100

0.5 75
|F(k)|

-0.5 50

-1

-1.5 25

-2

-2.5 0
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 5.5 6 6.5 0 50 100 150 200 250
x k

Threshold ǫ = 2.5. 25.4% of original coeffs retained. Reconstructed signal f˜[n] and magnitudes |F̃ [k]|. L2

error kf − f˜k2 = 1.19. Relative L2 error 9.0%.

139
 2.5 125

1.5 100

0.5 75

|F(k)|
0

-0.5 50

-1

-1.5 25

-2

-2.5 0
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 5.5 6 6.5 0 50 100 150 200 250
x k

Threshold ǫ = 10.0. 8.2% of original coeffs retained. Reconstructed signal f˜[n] and magnitudes |F̃ [k]|. L2

error kf − f˜k2 = 2.18. Relative L2 error 16.5%.

2.5 125

1.5 100

0.5 75
|F(k)|

-0.5 50

-1

-1.5 25

-2

-2.5 0
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 5.5 6 6.5 0 50 100 150 200 250
x k

Threshold ǫ = 20.0. 5.1% of original coeffs retained. Reconstructed signal f˜[n] and magnitudes |F̃ [k]|. L2

error kf − f˜k2 = 3.75. Relative L2 error 28.5%.

2.5 125

1.5 100

0.5 75
|F(k)|

-0.5 50

-1

-1.5 25

-2

-2.5 0
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 5.5 6 6.5 0 50 100 150 200 250
x k

Threshold ǫ = 40.0. 3.1% of original coeffs retained. Reconstructed signal f˜[n] and magnitudes |F̃ [k]|. L2

error kf − f˜k2 = 5.44. Relative L2 error 41.4%.

140
 f (x256 ) = f (x0 ) = 2. That being said, we must mention that the use of all 255 DFT
coefficients will reconstruct the data series f [n] perfectly. However, when terms of the
DFT are deleted, the Gibbs phenomenon will appear.

4. As ǫ is increased, the signal is altered in other regions as well, e.g., the relative minimum near
x = 0.6, and the relative maximum near x = 0.1.

Finally, let us return to the observation that the relative error in approximation of the signal
f by f˜ǫ increases with ǫ. In the next figure is plotted the relative error vs. ǫ for 0 ≤ ǫ ≤ 110. To
construct this plot, the relative errors were computed in increments of ∆ǫ = 0.5.

100

90

80

70
relative error

60

50

40

30

20

10

0
0 10 20 30 40 50 60 70 80 90 100 110
epsilon

Relative error of approximation of signal f with thresholded signal f˜ǫ vs. ǫ.

This figure is a simple example of a rate-distortion curve: a plot of the distortion, or error, vs.
the rate of thresholding. There are some noteworthy features in this plot:

1. For 0 ≤ ǫ < 1, there is no noticeable error. This might be because there are no coefficients with
magnitudes less than these ǫ values, or there are so few that their absence does not affect the
fidelity of reconstruction.

2. As ǫ is increased, there are flat regions. Once again, these might correspond to regions that are
not occupied by the DFT coefficients.

3. At ǫ = 107.5, 100% error is achieved. This is because the DFT coefficient with the highest
magnitude is F [4] = 107.36.

141
 In order to clarify the questions raised in points 1 and 2 above, it is probably more instructive to
plot the relative error vs. the percentage of coefficients removed: this is a more accurate indication of
the compression rate. This plot is shown below.
100

90

80

70

60
relative error

50

40

30

20

10

0
0 10 20 30 40 50 60 70 80 90 100
percentage of coeffs removed

Relative error of approximation of signal f with thresholded signal f˜ǫ vs. percentage of coefficients removed.

This plot is much more instructive. It shows that, in fact, significant compression can be performed
before the error becomes appreciable. When 70% of the coefficients are removed, the relative error is
10%.
For this simple example, the compression ratio, R, may defined as

number of DFT coefficients used in original signal

R= (121)
number of DFT coefficients used in thresholded signal

In this case, the numerator is N = 256. A plot of the relative error vs. compression rate is shown
below. This is the true rate-distortion curve for this experiment.
The point situated roughly at (3, 10) corresponds to the 70% removal rate mentioned for the
previous plot. The plot stops at R = 128 – after this rate, virtually all coefficients are removed, with
corresponds to R = ∞, where the asymptotic value of 100% relative error is achieved.

142
 100

90

80

70

60
relative error

50

40

30

20

10

0
0 10 20 30 40 50 60 70 80 90 100 110 120 130
compression rate

Relative error of approximation of signal f with thresholded signal f˜ǫ vs. compression rate.

143