Slides Seniales

Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering
PDS
2018-11-19
Slides mainly compiled from [1] and [2]
1 / 643
1 Introduction to digital signal processing

Signal
Discrete-time signals
Complex Exponentials
2 Vectors spaces and DSP
Signal Processing and vector spaces
Vector Spaces
Signal Spaces
Bases
3 Basic of Fourier Analysis
Introduction to Fourier Analysis
The Discrete Fourier Transform
Examples of DFT calculation
Interpreting a DFT plot
DFT analysis
DFT synthesis
2 / 643
THe STFT
The Spectrogram
Time-frequency tiling
4 Advanced Fourier Analysis

DFS
DTFT
Existence and properties of the DFTF
The DTFT as a change of basis
5 Introduction to Filtering
Linear Filters
Filtering: Intuition
Filter Stability
Frequency Response
Ideal Filters
3 / 643
Introduction to digital signal processing
4 / 643
Signal
Signal: Description of the evolution of a physical phenomenon.

weather: temperature
sound: pressure
sound: magnetic deviation
light intensity: gray level on paper
stock market
5 / 643
Signal

sound: pressure
stock market
6 / 643
Signal

sound: pressure
stock market
7 / 643
Signal

sound: pressure
stock market
8 / 643
Signal

sound: pressure
stock market
9 / 643
Processing
Analysis
understanding the information carried by the signal
extracting features from the signal
Synthesis
creating a signal to contain the given information (e.g. to transmit information)
10 / 643
Analog Signals (physics)
11 / 643
Analog Signals (electronics)
12 / 643
Analog Signals
f :R→R
13 / 643
Analog Signals (physics)
only 1 degree of freedom: vy
14 / 643
Analog Signals (electronics)
only 2 degrees of freedom: R, C
15 / 643
What about “interesting” signals?
f (t) =?
16 / 643
To each analog signal, a different device
17 / 643
From analog ...
18 / 643
... to digital
... 12 −45 33 41 −89 60 70 −99 ...
19 / 643
The Digital Paradigm
Key ingredients
discrete time
discrete amplitude
20 / 643
The Digital Paradigm
21 / 643
Discretizing time
What is time?
22 / 643
Two competing models for reality
23 / 643
Two competing models for reality
24 / 643
Discrete time is very practical! Computing the average
25 / 643
26 / 643
27 / 643
28 / 643
The discrete-time model
29 / 643
The discrete-time model
x[n] = . . . , 1.2390, −0.7372, 0.8978, 0.1498, −1.1501, −0.2642
30 / 643
How can we go from continuous to discrete time?
x[n] =?
31 / 643
from continuous to discrete time: the founding fathers
32 / 643
The Sampling Theorem (1920)
Under the appropriate “slowness” conditions for x(t) we have:
∞
X t − nTs
x (t) = x[n] sinc
n=−∞
Ts
33 / 643
Sinc: the building block
34 / 643
from continuous to discrete time (and back)
35 / 643
36 / 643
37 / 643
38 / 643
39 / 643
40 / 643
When can we do all this? Joseph Fourier will tell us
41 / 643
Digital Signals
Key ingredients
discrete time
discrete amplitude (Quantization)
42 / 643
Digital Signals
43 / 643
Digital Signals
44 / 643
Digital Signals
45 / 643
Digital Signals
46 / 643
Digital Signals
Quantization: Each sample can take values from a predetermined set of possible
levels.
The set of levels is countable.
47 / 643
Digital amplitude
Why it is important
storage
General purpose devices (e.g. computer memory)
processing
Does not require specific devices (e.g. hardware audio equalizers, analog
thermostats)
Processing is performed in software
transmission
48 / 643
Analog storage
49 / 643
Digital storage
{0, 1}
50 / 643
Processing
51 / 643
Data transmission
52 / 643
What happens to analog signals
53 / 643
54 / 643
55 / 643
56 / 643
We can amplify to compensate attenuation
57 / 643
Transmission of analog signals
58 / 643
59 / 643
60 / 643
61 / 643
62 / 643
Transmitting a signal overseas
63 / 643
Transmitting a signal overseas
For a long, long channel we need repeaters
64 / 643
65 / 643
66 / 643
67 / 643
68 / 643
69 / 643
70 / 643
71 / 643
In digital signals we can threshold
72 / 643
Transmission of quantized signals
73 / 643
74 / 643
75 / 643
76 / 643
Digital data throughputs
Transatlantic cable:
1866: 8 words per minute (≈ 5 bps)
1956: AT&T, coax, 48 voice channels (≈ 3 Mbps)
2005: Alcatel Tera10, fiber, 8.4 Tbps (8.4 × 1012 bps)
2012: fiber, 60 Tbps
Voiceband modems
1950s: Bell 202, 1200 bps
1990s: V90, 56 Kbps
2008 ADSL2+, 24Mpbs
77 / 643
Digital Signal Processing: key ideas
discretization of time
samples replace idealized models
simple math replaces calculus
discretization of amplitude:
general-purpose storage
general purpose processing (CPU)
noise can be controlled
78 / 643
Discrete-time signals
Discrete-time signal: a sequence of complex numbers

one dimension (for now)
notation: x[n]
two-sided sequences: x : Z → C
n is a-dimensional “time”
79 / 643
The delta signal
x[n] = δ[n]
80 / 643
How do you synchronize audio and video ...
81 / 643
How do you synchronize audio and video ...
82 / 643
The unit step
x[n] = u[n]
83 / 643
A switch
84 / 643
The exponential decay
x[n] = |a|n u[n], |a| < 1
85 / 643
How fast does your coffee get cold ...
86 / 643
Also, how fast a capacitor discharges ...
87 / 643
The sinusoid
x[n] = sen(ω0 n + θ)
88 / 643
Oscillations are everywhere
89 / 643
Four signal classes
finite-length
infinite-length
periodic
finite-support
90 / 643
Finite-length signals
notation: x[n], n = 0, 1, . . . , N − 1
vector notation: x = [ x0 x1 . . . xN−1 ]T
practical entities, good for numerical packages (e.g.matlab, numpy)
91 / 643
notation: x[n], n = 0, 1, . . . , N − 1
92 / 643
notation: x[n], n = 0, 1, . . . , N − 1
93 / 643
Infinite-length signals
sequences notation: x[n], n ∈ Z

abstraction, good for theorems
They do not depend on the length of the signal N
94 / 643
Infinite-length signals
sequences notation: x[n], n ∈ Z

abstraction, good for theorems
They do not depend on the length of the signal N
95 / 643
Periodic signals
N-periodic sequence: x̃[n] = x̃[n + kN], n, k, N ∈ Z

same information as finite-length of length N
“natural” bridge between finite and infinite lengths
96 / 643
Periodic signals

97 / 643
Periodic signals

98 / 643
Finite-support signals
Finite-support sequence:

x[n] if 0 ≤ n < N
x̄[n] =
0 otherwise
for n ∈ Z

anther bridge between finite and infinite lengths
99 / 643

x[n] if 0 ≤ n < N
x̄[n] =
0 otherwise
for n ∈ Z

100 / 643

x[n] if 0 ≤ n < N
x̄[n] =
0 otherwise
for n ∈ Z

101 / 643
Elementary operators
scaling:
y [n] = αx[n]
sum:
y [n] = x[n] + z[n]
product:
y [n] = x[n] · z[n]
shift by k (delay):
y [n] = x[n − k]
102 / 643
scaling:
y [n] = αx[n]
sum:
y [n] = x[n] + z[n]
product:
y [n] = x[n] · z[n]
shift by k (delay):
y [n] = x[n − k]
103 / 643
scaling:
y [n] = αx[n]
sum:
y [n] = x[n] + z[n]
product:
y [n] = x[n] · z[n]
shift by k (delay):
y [n] = x[n − k]
104 / 643
scaling:
y [n] = αx[n]
sum:
y [n] = x[n] + z[n]
product:
y [n] = x[n] · z[n]
shift by k (delay):
y [n] = x[n − k]
105 / 643
Shift of a finite-length: finite support
106 / 643
107 / 643
108 / 643
109 / 643
110 / 643
111 / 643
Shift of a finite-length: periodic extension
112 / 643
113 / 643
114 / 643
115 / 643
116 / 643
Energy and power
∞
X
Ex = |x [n]|2
n=−∞
N
1 X
Px = lim |x [n]|2
N→∞ 2N + 1
n=−N
117 / 643
Energy and power: periodic signals
Ex̃ = ∞
N−1
1 X
Px̃ = |x̃ [n]|2
N
n=0
A signal can be an energy signal, a power signal, or neither type:

power signals has finite Px and infinite Ex (e.g. periodic signals)
Energy signals has finite Ex and Px = 0
118 / 643
Digital vs physical frequency
Discrete time
n: no physical dimension (just a counter)
periodicity: how many samples before pattern repeats
Physical world
periodicity: how many seconds before pattern repeats
frequency measured in Hz
119 / 643
How your PC plays sounds
120 / 643
set Ts , time in seconds between samples

periodicity of M samples → periodicity of MTs seconds
real world frequency:
1
f = in Hz
MTs
121 / 643
usually we choose Fs the number of samples per second

Ts = 1/Fs
E.g. for a typical value Fs = 48000, Ts ≈ 20.8µs. if M = 110, then
1
f = MTs ≈ 440 Hz
122 / 643
Suppose the system clock of your computer has an associated frequency of 44 kHz.
What is the period of a discrete-time sinusoid stored on your computer that
corresponds to 880 Hz?
Answer: M = 50
123 / 643
Building blocks: Adder
124 / 643
Building blocks: Multiplier
125 / 643
Building blocks: Delay
126 / 643
The 2-point moving average
moving average: take a “local” average
x [n] + x [n − 1]
y [n] =
2
127 / 643
The 2-point Moving Average DSP Blocks
128 / 643
Moving average example
x[n] = δ[n]
129 / 643
What if we reverse the loop?
130 / 643
What if we reverse the loop? Recursion
y [n] = x[n] + αy [n − 1]
131 / 643
How we solve the chicken-and-egg problem
Zero initial Conditions

set a start time (usually n0 = 0)
assume input and output are zero for all time before n0
132 / 643
Recursion: the compound interest problem
Consider the following compound interest problem:

constant interest rate of 5% per year
deposits/withdrawals during year n: x[n]
balance at year n:
y [n] = 1.05y [n − 1] + x [n]
133 / 643
Accumulation of interest: first-order recursion
y [n] = 1.05y [n − 1] + x [n]
134 / 643
Example: the one-time investment
x [n] = 100δ [n]

y [n] = 1.05y [n − 1] + x [n]
135 / 643
An interesting generalization
y [n] = αy [n − M] + x[n]
136 / 643
Creating loops
137 / 643
Example
M = 3, α = 1, x[n] = δ[n] + 2δ[n − 1] + 3δ[n − 2]
138 / 643
We can make music with that!
build a recursion loop with a delay of M

choose a signal x̄ [n] that is nonzero only for 0 ≤ n < M
choose a decay factor
input x̄ [n] to the system
play the output
139 / 643
Playing a sine wave
M = 100, α = 1, x̄ [n] = sen(2πn/100) for 0 ≤ n < 100 and zero elsewhere.
140 / 643
Introducing some realism
M controls frequency (pitch)

α controls envelope (decay)
x̄[n] controls color (timbre)
141 / 643
A proto-violin
M = 100, α = 0.95, x̄ [n] : zero-mean sawtooth wave for 0 ≤ n < 100 and zero
elsewhere.
142 / 643
The Karplus-Strong Algorithm
M = 100, α = 0.9, x̄ [n]: 100 random values between 0 and 99, zero elsewhere.
143 / 643
Complex Exponentials
e jx
sustainable dynamic systems exhibit oscillatory behavior
intuitively: things that don’t move in circles can’t last:
bombs
rockets
human beings ...
144 / 643
Continuous-time oscillations
145 / 643
Continuous-time oscillations
146 / 643
Discrete-time oscillations
a frequency ω (units: radians)

an initial phase φ (units: radians)
an amplitude A
x[n] = Ae j(ωn+φ) = A[cos(ωn + φ) + j sen(ωn + φ)]
147 / 643
Why complex exponentials?
we can use complex numbers in digital systems, so why not?

it makes sense: every sinusoid can always be written as a sum of sine and cosine
math is simpler: trigonometric becomes algebra
148 / 643

149 / 643

150 / 643
The complex exponential
e jα = cos(α) + j sen(α)
151 / 643
The complex exponential
rotation: z0 = ze jα
152 / 643
The complex exponential generating machine
x[n] = e jωn ; x[n + 1] = e jω x[n]
153 / 643
x[n] = e jωn ; x[n + 1] = e jω x[n]
154 / 643
x[n] = e jωn ; x[n + 1] = e jω x[n]
155 / 643
x[n] = e jωn ; x[n + 1] = e jω x[n]
156 / 643
x[n] = e jωn ; x[n + 1] = e jω x[n]
157 / 643
x[n] = e jωn ; x[n + 1] = e jω x[n]
158 / 643
x[n] = e jωn ; x[n + 1] = e jω x[n]
159 / 643
x[n] = e jωn ; x[n + 1] = e jω x[n]
160 / 643
x[n] = e jωn ; x[n + 1] = e jω x[n]
161 / 643
x[n] = e jωn ; x[n + 1] = e jω x[n]
162 / 643
x[n] = e jωn ; x[n + 1] = e jω x[n]
163 / 643
x[n] = e jωn ; x[n + 1] = e jω x[n]
164 / 643
x[n] = e jωn ; x[n + 1] = e jω x[n]
165 / 643
Initial phase
x[n] = e jωn+φ ; x[n + 1] = e jω x[n], x[0] = e jφ
166 / 643
Initial phase
x[n] = e jωn+φ ; x[n + 1] = e jω x[n], x[0] = e jφ
167 / 643
Careful: not every sinusoid is periodic in discrete time
x[n] = e jωn ; x[n + 1] = e jω x[n]
168 / 643
x[n] = e jωn ; x[n + 1] = e jω x[n]
169 / 643
x[n] = e jωn ; x[n + 1] = e jω x[n]
170 / 643
x[n] = e jωn ; x[n + 1] = e jω x[n]
171 / 643
x[n] = e jωn ; x[n + 1] = e jω x[n]
172 / 643
M
e jωn periodic in n ⇔ ω = 2π, for M, N ∈ N
N
173 / 643
2π-periodicity
e jα = e j(α+2kπ) ∀k ∈ Z
174 / 643
2π-periodicity: one point, many names (ALIASING)
175 / 643
176 / 643
177 / 643
178 / 643
How “fast” can we go?
Wagon-wheel effect:
https://www.youtube.com/watch?v=jHS9JGkEOmA
179 / 643
Remember the complex exponential generating machine
180 / 643
0 ≤ ω < 2π
181 / 643
ω = 2π/12
182 / 643
ω = 2π/6
183 / 643
ω = 2π/5
184 / 643
ω = 2π/4
185 / 643
ω = 2π/2 = π
186 / 643
ω = 2π/2 = π
187 / 643
ω = 2π/2 = π
188 / 643
π < ω < 2π
189 / 643
Let’s go really too fast
ω = 2π − α, αsmall
190 / 643
Let’s go really too fast
ω = 2π − α, αsmall
191 / 643
Vector Spaces and DSP
192 / 643
Prerequisite Warning!
  
a0,0 a0,1 ··· a0,N−1 x0
 a1,0 a1,1 ··· a1,N−1  x1 
 = Ax
  
 .. .. .. ..  ..
 . . . .  . 
aM−1,0 aM−1,1 · · · aM−1,N−1 xN−1
193 / 643
Prerequisite Warning!
T
x+y = x0 + y0 x1 + y1
194 / 643
A generic discrete-time signal
x[n] = 1.1, 2.2., −1.0, 3.3, · · ·
finite length?
infinite length?
periodic?
finite support?
We need a common framework: vector space
195 / 643
Why using vector space in DSP?
Easier math and unified framework for signal processing:

same framework for different classes of signals
same framework for continuous-time signals
easy explanation of the Fourier Transform
easy explanation of sampling and interpolation
useful in approximation and compression
fundamental in communication system design
196 / 643
Why using vector space in DSP?
vector spaces are very general objects

vector spaces are defined by their properties
once you know the properties are satisfied, you can use all the tools for the space
197 / 643
Analogy #1: Object Oriented Programming
198 / 643
Analogy #1: Object Oriented Programming
199 / 643
Analogy #2: LEGO
basic building block:
200 / 643
Analogy #2: LEGO
scaling (4x2):
201 / 643
Analogy #2: LEGO
adding:
202 / 643
Vector Spaces
Some spaces should be very familiar:

R2 , R3 : Euclidean space, geometry
RN , CN : linear algebra
Others perhaps not so much . . .

`2 (Z): space of square-summable infinite sequences
L2 ([a, b]): space of square-integrable functions over an interval
yes, vectors can be functions!
203 / 643
Vector Spaces


204 / 643
Vector Spaces


205 / 643
Some spaces can be represented graphically
T
R2 :

x= x0 x1
206 / 643
Some spaces can be drawn
T
R3 :

x= x0 x1 x2
207 / 643
Some spaces can be drawn
L2 [−1, 1] : x = x(t), t ∈ [−1, 1]
208 / 643
Others can’t
RN , for N > 3
CN , for N > 1
209 / 643
Vector spaces: operational definition
Ingredients:
the set of vector V
a set of scalars (say C)
We need at least to be able to:

resize vectors, i.e. multiply a vector by a scalar
combine vectors together, i.e. sum them
210 / 643
Vector spaces: operational definition
Ingredients:
the set of vector V
We need at least to be able to:

resize vectors, i.e. multiply a vector by a scalar
combine vectors together, i.e. sum them
211 / 643
Formal properties of a vector space
For x, y, z ∈ V and α, β ∈ C
x+y =y+x
(x + y) + z = x + (y + z)
α(x + y) = αx + αy
(α + β)x = αx + βx
α(βx) = (αβ)x
∃0 ∈ V | x+0=0+x=x
∀x ∈ V ∃(−x) | x + (−x) = 0
212 / 643
2
Example: scalar multiplication in R
T
αx = αx0 αx1
213 / 643
2
Example: addition in R
x + y = [ x0 + y0 x1 + y1 ]T
214 / 643
Example: scalar multiplication in L2 [−1, 1]
αx = αx(t)
215 / 643
Example: Addition in L2 [−1, 1]
x + y = x(t) + y (t)
216 / 643
Example: Addition in L2 [−1, 1]
x + y = x(t) + y (t)
217 / 643
Vector spaces: we need something more
the set of vectors V

scalar multiplication
addition
We need something to measure and compare:
inner product (aka dot product)
218 / 643
Vector spaces: we need something more
the set of vectors V

scalar multiplication
addition
We need something to measure and compare:
inner product (aka dot product)
219 / 643
Inner product
h·, ·i : V × V → C
measure of similarity between vectors

inner product is zero? vectors are orthogonal (maximally different)
220 / 643
Formal properties of the inner product
For x, y, z ∈ V and α, β ∈ C
221 / 643
2
Inner product in R : the norm
hx, xi = x02 + x12 = kxk2
222 / 643
2
Inner product in R
hx, yi = x0 y0 + x1 y1 = kxk kyk cos(α)
223 / 643
2
Inner product in R : orthogonality
hx, yi = x0 y0 + x1 y1 = kxk kyk cos(α)
224 / 643
Inner product in L2 [−1, 1]
ˆ1
hx, yi = x (t) y (t) dt
−1
225 / 643
Inner product in L2 [−1, 1] : the norm
ˆ1
2
hx, xi = kxk = sen2 (πt) dt = 1
−1
226 / 643
Inner product in L2 [−1, 1] : the norm
ˆ1
2
kyk = t 2 dt = 2/3
−1
227 / 643
ˆ1 p
hx, yi = 3/2t sen (πt) dt ≈ 0.78
−1
228 / 643
x, y from orthogonal subspaces: hx, yi = 0
229 / 643
sinusoids with frequencies integer multiples of a fundamental
230 / 643
sinusoids with frequencies integer multiples of a fundamental
231 / 643
Norm vs Distance
p
inner product defines a norm: kxk = hx, xi
norm defines a distance: d (x, y) = kx − yk
232 / 643
Norm vs Distance
p
inner product defines a norm: kxk = hx, xi
norm defines a distance: d (x, y) = kx − yk
233 / 643
2
Norm and distance in R
q
kx − yk = (x0 − y0 )2 + (x1 − y1 )2
234 / 643
Distance in L2 [−1, 1] : the Mean Square Error
ˆ1
2
kx − yk = |x (t) − y (t)|2 dt
−1
235 / 643
Distance in L2 [−1, 1] : the Mean Square Error
ˆ1
2
kx − yk = |x (t) − y (t)|2 dt = 2
−1
236 / 643
Finite-Length Signals
finite-length and periodic signals live in CN

T
vector notation: x = x0 x1 . . . xN−1
all operations well defined and intuitive
space of N-periodic signals sometimes indicated by C̃N
237 / 643
Inner product for signals
N−1
X
hx, yi = x ∗ [n] y [n]
n=0
well defined for all finite-length vectors (i.e. vectors in CN )
238 / 643
What about Infinite-length signals?
∞
X
hx, yi = x ∗ [n] y [n]
n=−∞
careful: sum may explode! (e.g. u[n] )
239 / 643
What about Infinite-length signals?
∞
X
hx, yi = x ∗ [n] y [n]
n=−∞
|x [n]|2 < ∞
P
We require sequences to be square-summable:
Space of square-summable sequences: `2 (Z)
240 / 643
Infinite-Length Signals
“well-behaved” infinite-length signals live in `2 (Z)
T
vector notation: x = . . . x−2 x−1 x0 x1 x2 . . .
many interesting signals not in `2 (Z) unfortunately (x[n] = 1, x[n] = cos(ωn), etc)
241 / 643
Bases
linear combination is the basic operation in vector spaces:
g = αx + βy
can we find a set of vectors {w(k) } so that we can write any vector as a linear
combination of {w(k) }?
242 / 643
Bases
linear combination is the basic operation in vector spaces:
g = αx + βy
can we find a set of vectors {w(k) } so that we can write any vector as a linear
combination of {w(k) }?
243 / 643
2
The canonical R basis

1 0
e(0) = e(1) =
0 1

x0 1 0
= x0 + x1
x1 0 1
244 / 643
2

1 0
e(0) = e(1) =
0 1

x0 1 0
= x0 + x1
x1 0 1
245 / 643
2

2
x= = 2e(0) + e(1)
1
246 / 643
2
Another R basis

1 1
v(0) = v(1) =
0 1

2
x= = v(0) + v(1)
1
247 / 643
2
But this is not a basis for R

1 −1
g(0) = g(1) =
0 0

2
x 6= = α1 g(0) + α2 g(1)
1
248 / 643
What about infinite-dimensional spaces?
∞
X
x= αk w(k)
k=0
249 / 643
A basis for `2 (Z)
..
 
 . 

 0 


 0 

1
e(k)
 
=  k-th position, k ∈ Z

 0 


 0 


 0 

..
.
250 / 643
What about function vector spaces?
X
f (t) = αk h(k) (t)
k
251 / 643
A basis for the functions over an interval?
the Fourier basis for [−1, 1] :
1
√ , cos (πt) , sen (πt) , cos (2πt) , sen (2πt) , cos (3πt) , sen (3πt) , . . .
2
252 / 643
Using the Fourier Basis
N
X sen (2k + 1) πt
2k + 1
k=0
253 / 643
N
X sen (2k + 1) πt
2k + 1
k=0
254 / 643
N
X sen (2k + 1) πt
2k + 1
k=0
255 / 643
N
X sen (2k + 1) πt
2k + 1
k=0
256 / 643
Bases: formal definition
Given:
a vector space H
a set of K vectors from H: W = {wk }k=0,1,...,K −1
W is a basis for H if:

1 we can write for all x ∈ H:
K
X −1
x= αk w(k) , αk ∈ C
k=0
2 the coefficients αk are unique
257 / 643
Bases: formal definition
Unique representation implies linear independence:
K
X −1
αk w(k) = 0 ⇔ αk = 0, k = 0, 1, . . . , K − 1
k=0
258 / 643
Special bases
Orthogonal basis:
D E
w(k) , w(n) = 0 for k 6= n
Orthonormal basis:
D E
w(k) , w(n) = δ[n − k]
We can alway orthonormalize a basis via a Gram-Schmidt algorithm
259 / 643
Special bases
Orthogonal basis:
D E
w(k) , w(n) = 0 for k 6= n
Orthonormal basis:
D E
w(k) , w(n) = δ[n − k]
260 / 643
Special bases
Orthogonal basis:
D E
w(k) , w(n) = 0 for k 6= n
Orthonormal basis:
D E
w(k) , w(n) = δ[n − k]
261 / 643
Basis expansion
K
X −1
k=0
How do we find the α’s?
Orthonormal bases are the best:
D E
αk = w(k) , x
262 / 643
Basis expansion
K
X −1
k=0
How do we find the α’s?
Orthonormal bases are the best:
D E
αk = w(k) , x
263 / 643
Change of basis
K
X −1 K
X −1
x= αk w(k) = βk v(k)
k=0 k=0
(k)
if {v} is orthonormal:
D E
βh = v(h) , x
K −1
* +
X
βh = v(h) , αk w(k)
k=0
K
X −1 D E
βh = αk v(h) , w(k)
k=0
264 / 643
Change of basis
K
X −1 D E
βh = αk v(h) , w(k)
k=0
K
X −1
βh = αk chk
k=0
  
c00 c01 ... c0(K −1) α0
 c10 c11 ··· c1(K −1)  α1 
βh = 
  
.. .. .. ..  .. 
 . . . .  . 
c(K −1)0 c(K −1)1 · · · c(K −1)(K −1) αK −1
265 / 643
Change of basis: example
canonical basis E = {e(0) , e(1) }

x = α0 e(0) + α1 e(1)
266 / 643
canonical basis E = {e(0) , e(1) }

x = α0 e(0) + α1 e(1)
new basis V = {v(0) , v(1) } with

T
v(0) =

cos (θ) sen (θ)
T
v(1) =

− sen (θ) cos (θ)
v = β0 v(0) + β1 v(1)
267 / 643
new basis is orthonormal

in compact form:

β0 cos (θ) sen (θ) α0
= = Rα
β1 − sen (θ) cos (θ) α1
R: rotation matrix
key fact: RT R = I
268 / 643

in compact form:

= = Rα
R: rotation matrix
key fact: RT R = I
269 / 643

in compact form:

= = Rα
R: rotation matrix
key fact: RT R = I
270 / 643

in compact form:

= = Rα
R: rotation matrix
key fact: RT R = I
271 / 643
Basic of Fourier Analysis
272 / 643

sustainable dynamic systems exhibit oscillatory behavior
intuitively: things that don’t move in circles can’t last:
bombs
rockets
human beings ...
273 / 643
You too can detect sinusoids!
the human body has two receptors for sinusoidal signals:
air pressure sinusoids electromagnetic sinusoids

frequencies from 20Hz to 20kHz frequencies from 430THz to 790THz
274 / 643
The intuition
humans analyze complex signals (audio, images) in terms of their sinusoidal

components
we can build instruments that “resonate” at one or multiple frequencies (tuning
fork vs piano)
the “frequency domain” seems to be as important as the time domain
275 / 643
Fundamental question
can we decompose any signal into sinusoidal elements?
yes, and Fourier showed us how to do it exactly !
analysis synthesis
from time domain to frequency domain from frequency domain to time domain
find the contribution of different create signals with known frequency
frequencies content
discover “hidden” signal properties fit signals to specific frequency regions
276 / 643
analysis synthesis
frequencies content
277 / 643
analysis synthesis
frequencies content
278 / 643
analysis synthesis
frequencies content
279 / 643
The mathematical setup
let’s start with finite-length signals (i.e. vectors in CN )

Fourier analysis is a simple change of basis
a change of basis is a change of perspective
a change of perspective can reveal things (if the basis is good)
280 / 643
Mystery signal
281 / 643
Mystery signal in the Fourier basis
282 / 643
N
The Fourier Basis for C
Claim: the set of N signals in CN

2π
wk [n] = e j N
nk
, n, k = 0, 1, . . . , N − 1
is an orthogonal basis in CN .
283 / 643
N
In vector notation:
{w(k) }k=0,1,...,N−1
with
2π
wn (k) = e j N
nk
is an orthogonal basis in CN
284 / 643
Recall the complex exponential generating machine...
285 / 643
Recall the complex exponential generating machine...
286 / 643
(0) 64
Basis vector w ∈C
287 / 643
(1) 64
Basis vector w ∈C
288 / 643
(2) 64
Basis vector w ∈C
289 / 643
(3) 64
Basis vector w ∈C
290 / 643
(4) 64
Basis vector w ∈C
291 / 643
(5) 64
Basis vector w ∈C
292 / 643
(16) 64
Basis vector w ∈C
293 / 643
(20) 64
Basis vector w ∈C
294 / 643
(30) 64
Basis vector w ∈C
295 / 643
(31) 64
Basis vector w ∈C
296 / 643
(32) 64
Basis vector w ∈C
297 / 643
(33) 64
Basis vector w ∈C
298 / 643
(34) 64
Basis vector w ∈C
299 / 643
(60) 64
Basis vector w ∈C
300 / 643
(62) 64
Basis vector w ∈C
301 / 643
(63) 64
Basis vector w ∈C
302 / 643
Proof of orthogonality
D E N−1
X 2π ∗ 2π
w(k) , w(h) = e j N nk e j N nh
n=0
D E N−1
X 2π
w(k) , w(h) = ej N
(h−k)n
n=0

D E N n=k
w(k) , w(h) =
0 otherwise
303 / 643
Remarks
N orthogonal vectors → basis for CN

√
vectors are not orthonormal. Normalization factor would be 1/ N
304 / 643
Remarks

√
305 / 643
The Discrete Fourier Transform

DFT
306 / 643
N
2π
in “signal” notation: wk [n] = e j N
nk
, n, k = 0, 1, . . . , N − 1
2π
in vector notation: {w(k) }k=0,1,...,N−1 with wn (k) = e j N
nk
307 / 643
N
2π
in “signal” notation: wk [n] = e j N
nk
, n, k = 0, 1, . . . , N − 1
2π
in vector notation: {w(k) }k=0,1,...,N−1 with wn (k) = e j N
nk
308 / 643
N

√
will keep normalization factor explicit in DFT formulas
309 / 643
Basis expansion
Analysis formula:
D E
Xk = w(k) , x
Synthesis formula:
N−1
1 X
x= Xk w(k)
N
k=0
310 / 643
Change of basis in matrix form
2π
Define WN = e −j N
(or simply W when N is evident from the context)
Change of basis matrix W with W[n, m] = WNnm :
1 1 1 1 ... 1
 
 1 W1 W2 W3 . . . W N−1 
W2 W4 W6 . . . W 2(N−1)
 
W=
 1 

 1 ... 
2
1 W N−1 W 2(N−1) W 3(N−1) . . . W (N−1)
311 / 643
2π
Define WN = e −j N
(or simply W when N is evident from the context)
Change of basis matrix W with W[n, m] = WNnm :
1 1 1 1 ... 1
 
 1 W1 W2 W3 . . . W N−1 
W2 W4 W6 . . . W 2(N−1)
 
W=
 1 

 1 ... 
2
1 W N−1 W 2(N−1) W 3(N−1) . . . W (N−1)
312 / 643
Analysis formula:
X = Wx
Synthesis formula:
1 H
x= W X
N
313 / 643
Basis expansion (signal notation)
Analysis formula:
N−1
X 2π
X [k] = x[n]e −j N
nk
, k = 0, 1, . . . , N − 1
n=0
N-point signal in the frequency domain
Synthesis formula:
N−1
1 X 2π
x[n] = X [k]e j N nk , n = 0, 1, . . . , N − 1
N
k=0
N-point signal in the “time” domain
314 / 643
Basis expansion (signal notation)
Analysis formula:
N−1
X 2π
X [k] = x[n]e −j N
nk
, k = 0, 1, . . . , N − 1
n=0
N-point signal in the frequency domain
Synthesis formula:
N−1
1 X 2π
x[n] = X [k]e j N nk , n = 0, 1, . . . , N − 1
N
k=0
N-point signal in the “time” domain
315 / 643
DFT is obviously linear
DFT {αx[n] + βy [n]} = α DFT{x[n]} + β DFT{y [n]}
316 / 643
N
DFT of x[n] = δ[n], x[n] ∈ C
N−1
X 2π
X [k] = δ[n]e −j N
nk
=1
n=0
317 / 643
N
DFT of x[n] = 1, x[n] ∈ C
N−1
X 2π
X [k] = e −j N
nk
= Nδ[k]
n=0
318 / 643
64
DFT of x[n] = 3 cos(2π/16n), x[n] ∈ C

2π
x[n] = 3 cos 4n
64
3 h j 2π 4n 2π
i
x[n] = e 64 + e −j 64 4n
2
3 h j 2π 4n 2π
i
x[n] = e 64 + e j 64 60n
2
3
x[n] = (w4 [n] + w60 [n])
2
319 / 643
64

2π
x[n] = 3 cos 4n
64
3 h j 2π 4n 2π
i
x[n] = e 64 + e −j 64 4n
2
3 h j 2π 4n 2π
i
x[n] = e 64 + e j 64 60n
2
3
x[n] = (w4 [n] + w60 [n])
2
320 / 643
64

2π
x[n] = 3 cos 4n
64
3 h j 2π 4n 2π
i
x[n] = e 64 + e −j 64 4n
2
3 h j 2π 4n 2π
i
x[n] = e 64 + e j 64 60n
2
3
x[n] = (w4 [n] + w60 [n])
2
321 / 643
64

2π
x[n] = 3 cos 4n
64
3 h j 2π 4n 2π
i
x[n] = e 64 + e −j 64 4n
2
3 h j 2π 4n 2π
i
x[n] = e 64 + e j 64 60n
2
3
x[n] = (w4 [n] + w60 [n])
2
322 / 643
64
X [k] = hwk [n], x[n]i

3
X [k] = wk [n], (w4 [n] + w60 [n])
2
3 3
X [k] = hwk [n], w4 [n]i + hwk [n], w60 [n]i
2 2
3

2 64 = 96 for k = 4, 60
X [k] =
0 otherwise
323 / 643
64

3
X [k] = wk [n], (w4 [n] + w60 [n])
2
3 3
2 2
3

2 64 = 96 for k = 4, 60
X [k] =
0 otherwise
324 / 643
64

3
X [k] = wk [n], (w4 [n] + w60 [n])
2
3 3
2 2
3

2 64 = 96 for k = 4, 60
X [k] =
0 otherwise
325 / 643
64

3
X [k] = wk [n], (w4 [n] + w60 [n])
2
3 3
2 2
3

2 64 = 96 for k = 4, 60
X [k] =
0 otherwise
326 / 643
64
327 / 643
64
DFT of x[n] = 3 cos(2π/16n + pi/3), x[n] ∈ C

2π π
x[n] = 3 cos n+
16 3

2π π
x[n] = 3 cos 4n +
64 3
3 h j 2π 4n j π 2π π
i
x[n] = e 64 e 3 + e −j 64 4n e −j 3
2
3 jπ π

x[n] = e 3 w4 [n] + e −j 3 w60 [n]
2
328 / 643
64

2π π
x[n] = 3 cos n+
16 3

2π π
x[n] = 3 cos 4n +
64 3
i
x[n] = e 64 e 3 + e −j 64 4n e −j 3
2
3 jπ π

x[n] = e 3 w4 [n] + e −j 3 w60 [n]
2
329 / 643
64

2π π
x[n] = 3 cos n+
16 3

2π π
x[n] = 3 cos 4n +
64 3
i
x[n] = e 64 e 3 + e −j 64 4n e −j 3
2
3 jπ π

x[n] = e 3 w4 [n] + e −j 3 w60 [n]
2
330 / 643
64

2π π
x[n] = 3 cos n+
16 3

2π π
x[n] = 3 cos 4n +
64 3
i
x[n] = e 64 e 3 + e −j 64 4n e −j 3
2
3 jπ π

x[n] = e 3 w4 [n] + e −j 3 w60 [n]
2
331 / 643
64

π
 96e j 3

for k = 4
π
X [k] = −j
96e 3 for k = 60
0 otherwise

332 / 643
64
333 / 643
64
334 / 643
64
2π 2π 2π
6< < 7
64 10 64
335 / 643
64
336 / 643
N
DFT of length-M step in C
M−1
X
x[n] = δ[n − h], n = 0, 1, . . . , N − 1
h=0
337 / 643
N
N−1 M−1
−j 2π 2π
X X
X [k] = x[n]e N
nk
= e −j N
nk
n=0 n=0
sen Nπ Mk −j π (M−1)k

X [k] = e N
sen Nπ k
338 / 643
N
sen Nπ Mk −j π (M−1)k

X [k] = e N
sen Nπ k
X [0] = M, from the definition of the sum

X [k] = 0 if Mk/N integer (0 ≤ k < N)
∠X [k] linear in k (except at sign changes for the real part)
339 / 643
64
DFT of length-4 step in C
340 / 643
Wrapping the phase
Often the phase is displayed “wrapped” over the [−π, π] interval.

most numerical packages return wrapped phase
phase can be unwrapped by adding multiples of 2π
341 / 643
The DFT in Practice
342 / 643
343 / 643
344 / 643
345 / 643
346 / 643
x[n] = 1 (slowest signal)
only lowest frequency
347 / 643
x[n] = cos(πn) = (−1)n (fastest signal)
only highest frequency
348 / 643
Energy distribution
Recall Parseval: kxk2 = |αk |2

P
N−1 N−1
X 1 X
|x[n]|2 = |X [k]|2
N
n=0 k=0
square magnitude of k-th DFT coefficient proportional to signal’s energy at frequency

ω = (2π/N)k
349 / 643
x[n] = 3 cos(2π/16n)
energy concentrated on single frequency

(counterclockwise and clockwise combine to give real signal)
350 / 643
x[n] = u[n] − u[n − 4] (step)
energy mostly in low frequencies
351 / 643
DFT of real signals
For real signals the DFT is “symmetric” in magnitude
|X [k]| = |X [N − k]| for k = 1, 2, . . . , bN/2c
352 / 643
DFT of real signals
For real signals, magnitude plots need only bN/2c + 1 points
353 / 643
Mystery signal revisited
354 / 643
355 / 643
x[n] = cos (ωn + φ) + η[n]
with
φ=0
2π
ω = 1024 64
356 / 643
x[n] = cos (ωn + φ) + η[n]
with
φ=0
2π
ω = 1024 64
357 / 643

Cosine and noise components
358 / 643
Solar spots
sunspot number: s = 10 × # of clusters + # of spots

data set from 1749 to 2003, 2904 months
359 / 643
Solar spots
sunspot number: s = 10 × # of clusters + # of spots

data set from 1749 to 2003, 2904 months
360 / 643
Solar spots
361 / 643
Solar spots
362 / 643
Solar spots
363 / 643
Solar spots
DFT main peak for k = 22

22 cycles over 2904 months
2904
period: 22 ≈ 11 years
364 / 643
Daily temperature (2920 days)
365 / 643
Daily temperature: DFT
first few hundred DFT coefficients

(in magnitude and normalized by the length of the temperature vector)
366 / 643
Daily temperature: DFT
first few hundred DFT coefficients

(in magnitude and normalized by the length of the temperature vector)
367 / 643
Daily temperature
average value (0-th DFT coefficient): 12.3◦ C

DFT main peak for k = 8, value 6.4◦ C
8 cycles over 2920 days
2920
Period: 8 = 365 days
temperature excursion: 12.3◦ C ± 12.8◦ C
368 / 643
Daily temperature
if we know the “clock” of the system Ts

fastest (positive) frequency is ω = π
sinusoid at ω = π needs two samples to do a full revolution
time between samples: Ts = 1/Fs seconds
real-world period for fastest sinusoid: 2Ts seconds
real-world frequency for fastest sinusoid: Fs /2 Hz
369 / 643
Daily temperature

370 / 643
Daily temperature

371 / 643
Daily temperature

372 / 643
Daily temperature

373 / 643
Example: train whistle
374 / 643
32768 samples (the “clock” of the system Fs = 8000 Hz).

So, Ts = 1/8000 seconds
375 / 643

376 / 643

377 / 643
If we look up the frequencies:
378 / 643
Synthesis: the sinusoidal generator
2π
wk [n] = e j ( N kn+φk )
379 / 643
2π
wk [n] = e j ( N kn+φk )
380 / 643
381 / 643
DFT synthesis formula
382 / 643
Initializing the machine
|X [k]|
Ak =
N
φk = ∠X [k]
383 / 643
Example
x = [ 1 2 3 4 3 2 1 ]T
384 / 643
Example
385 / 643
DFT synthesis formula
386 / 643
Example
387 / 643
Example
388 / 643
Example
389 / 643
Example
390 / 643
Example
391 / 643
Example
392 / 643
Example
393 / 643
Running the machine too long ...
x[n + N] = x[n]
output signal is N-periodic!
394 / 643
Inherent periodicities in the DFT
the analysis formula:

N−1
X 2π
X [k] = x[n]e −j N
nk
, k = 0, 1, . . . , N − 1
n=0
produces an N-point signal in the frequency domain
the synthesis formula:
N−1
1 X 2π
x[n] = X [k]e j N nk , n = 0, 1, . . . , N − 1
N
k=0
produces an N-point signal in the “time” domain

395 / 643

N−1
X 2π
X [k] = x[n]e −j N
nk
, k∈Z
n=0
produces an N-periodic signal in the frequency domain
N−1
1 X 2π
x[n] = X [k]e j N nk , n = 0, 1, . . . , N − 1
N
k=0
produces an N-point signal in the “time” domain

396 / 643

N−1
X 2π
X [k] = x[n]e −j N
nk
, k∈Z
n=0
produces an N-periodic signal in the frequency domain
N−1
1 X 2π
x[n] = X [k]e j N nk , n∈Z
N
k=0
produces an N-periodic signal in the “time” domain

397 / 643
The Short-Time Fourier Transform

STFT
398 / 643
Dual-Tone Multi Frequency Dialing (DTMF)
399 / 643
DTMF Signaling
400 / 643
1-5-9 in time
401 / 643
1-5-9 in time (in detail)
402 / 643
1-5-9 in frequency (magnitude)
403 / 643
The fundamental tradeoff
time representation obfuscates frequency

frequency representation obfuscates time
404 / 643
Short-Time Fourier Transform
Idea:
take small signal pieces of length L
look at the DFT of each piece
L−1
X 2π
X [m; k] = x[m + n]e −j L
nk
n=0
405 / 643
Short-Time Fourier Transform
Idea:
take small signal pieces of length L
look at the DFT of each piece
L−1
X 2π
X [m; k] = x[m + n]e −j L
nk
n=0
406 / 643
Short-Time Fourier Transform (L=256)
407 / 643
408 / 643
409 / 643
410 / 643
The Spectrogram
color-code the magnitude: dark is small, white is large

use 10 log (|X [m; k]|) to see better (power in dBs)
plot spectral slices one after another
411 / 643
The Spectrogram

412 / 643
The Spectrogram

413 / 643
DTMF spectrogram
414 / 643
Labeling the Spectrogram
If we know the “system clock” Fs = 1/Ts we can label the axis

highest positive frequency: Fs /2 Hz
frequency resolution: Fs /L Hz
width of time slices: LTs seconds
415 / 643

416 / 643

417 / 643
DTMF spectrogram (Fs = 8000)
418 / 643
The Spectrogram
Questions:
width of the analysis window?
position of the windows (overlapping?)
shape of the window ?(weighing the samples)
419 / 643
The Spectrogram
Questions:
420 / 643
The Spectrogram
Questions:
421 / 643
Wideband vs Narrowband
Long window: narrowband spectrogram

long window ⇒ more DFT points ⇒ more frequency resolution
long window ⇒ more “things can happen” ⇒ less precision in time
Short window: wideband spectrogram

short window ⇒ many time slices ⇒ precise location of transitions
short window ⇒ fewer DFT points ⇒ poor frequency resolution
422 / 643


423 / 643


424 / 643


425 / 643


426 / 643
DTMF spectrogram (wideband)
N = 16800, L = 32
427 / 643
DTMF spectrogram
N = 16800, L = 256
428 / 643
DTMF spectrogram (narrowband)
N = 16800, L = 1024
429 / 643
Speech analysis
430 / 643
Speech analysis
8ms analysis window (125Hz frequency bins), 4ms shifts
431 / 643
Speech analysis
32ms analysis window (31Hz frequency bins), 4ms shifts
432 / 643
Time-Frequency tiling
L = 20
433 / 643
L = 10
434 / 643
L=4
435 / 643
Food for thought
time “resolution” ∆t = L
frequency “resolution” ∆f = 2π/L
∆t∆f = 2π
uncertainty principle!
436 / 643
Food for thought
∆t∆f = 2π
437 / 643
Food for thought
∆t∆f = 2π
438 / 643
Even more food for thought
more sophisticated tilings of the time-frequency planes

can be obtained with the “wavelet” transform
439 / 643
Advanced Fourier Analysis
440 / 643
Discrete Fourier Series
441 / 643
Discrete Fourier Series (DFS)
DFS = DFT with periodicity explicit
the DFS maps an N-periodic signal onto an N-periodic sequence of Fourier

coefficients
the inverse DFS maps an N-periodic sequence of Fourier coefficients a set onto an
N-periodic signal.
the DFS of an N-periodic signal is mathematically equivalent to the DFT of one
period.
442 / 643
Finite-length time shifts revisited
The DFS helps us understand how to define time shifts for finite-length signals.
For an N-periodic sequence x̃[n]:

x̃[n − M] is well-defined for all M ∈ N
2π
DFS{x̃[n − M]} = e −j N
Mk
X̃ [k] (easy derivation)
IDFS{X̃[k]} = x̃[n-M]
443 / 643

2π
DFS{x̃[n − M]} = e −j N
Mk
444 / 643

2π
DFS{x̃[n − M]} = e −j N
Mk
445 / 643

2π
DFS{x̃[n − M]} = e −j N
Mk
446 / 643
For an N-point signal x[n]:

x[n − M] is not well-defined
build x̃[n] = x[n mod N] ⇒ X̃ [k] = X [k]
2π 2π
IDFT{e −j N
Mk
X [k]} = IDFS{e −j N
Mk
X̃ [k]} = x̃[n − M] = x[(n − M) mod N]}
shifts for finite-length signals are “naturally” circular
447 / 643

2π 2π
IDFT{e −j N
Mk
Mk
X̃ [k]} = x̃[n − M] = x[(n − M) mod N]}
448 / 643

2π 2π
IDFT{e −j N
Mk
Mk
X̃ [k]} = x̃[n − M] = x[(n − M) mod N]}
449 / 643

2π 2π
IDFT{e −j N
Mk
Mk
X̃ [k]} = x̃[n − M] = x[(n − M) mod N]}
450 / 643
Periodic sequences: a bridge to infinite-length signals
N-periodic sequence: N degrees of freedom

DFS: only N Fourier coefficients capture all the information
451 / 643
Karplus-Strong revisited
y [n] = x[n] + αy [n − 1]
452 / 643
choose a signal x̄[n] that is nonzero only for 0 ≤ n < M

α = 1 (for now)
y [n] = x̄[0], x̄[1], . . . , x̄[M − 1], x̄[0], x̄[1], . . . , x̄[M − 1], x̄[0], x̄[1], . . .
453 / 643
choose a signal x̄[n] that is nonzero only for 0 ≤ n < M

α = 1 (for now)
y [n] = x̄[0], x̄[1], . . . , x̄[M − 1], x̄[0], x̄[1], . . . , x̄[M − 1], x̄[0], x̄[1], . . .
454 / 643
Example: 32-tap sawtooth wave
455 / 643
Example: DFT of 32-tap sawtooth wave
456 / 643
What if we take the DFT of two periods?
457 / 643
What if we take the DFT of two periods?
458 / 643
DFT of two periods: intuition
459 / 643
460 / 643
461 / 643
462 / 643
463 / 643
DFT of L periods

LX̄ [k/L] if k = 0, L, 2L, 3L, . . .
XL [k] =
0 otherwise
464 / 643
DFT and DFS
again, all the spectral information for a periodic signal is contained in the DFT
coefficients of a single period
to stress the periodicity of the underlying signal, we use the term DFS
465 / 643
Discrete-Time Fourier Transform
466 / 643
The situation so far
Fourier representation for signal classes:

N-point finite-length: DFT
N-point periodic: DFS
infinite length: ?
467 / 643

infinite length: ?
468 / 643

infinite length: ?
469 / 643
Karplus-Strong revisited, part 2
consider now α < 1

generated signal is infinite-length but not periodic:
y [n] = x̄[0], x̄[1], . . . , x̄[M − 1], αx̄[0], αx̄[1], . . . , αx̄[M − 1], α2 x̄[0], α2 x̄[1], . . .
what is a good spectral representation?
470 / 643
consider now α < 1

471 / 643
consider now α < 1

472 / 643
DFT of increasingly long signals
Start with the DFT. What happens when N → ∞

(2π/N)k becomes denser in [0, 2π] . . .
In the limit (2π/N)k → ω:
X
x [n] e −jωn ω∈R
n
473 / 643

X
n
474 / 643

X
n
475 / 643
Discrete-Time Fourier Transform (DFTF)
Formal definition:
x[n] ∈ `2 (Z)
define the function of ω ∈ R
∞
X
F (ω) = x[n]e −jωn
−∞
inversion (when F (ω) exists ):

ˆπ
1
x[n] = F (ω) e jωn dω n∈Z
2π
−π
476 / 643
Formal definition:
x[n] ∈ `2 (Z)
∞
X
−∞

ˆπ
1
2π
−π
477 / 643
Formal definition:
x[n] ∈ `2 (Z)
∞
X
−∞

ˆπ
1
2π
−π
478 / 643
DTFT periodicity and notation
F (ω) is 2π-periodic
to stress periodicity (and for other reasons) we will write
∞
X
X e jω = x[n]e −jωn
−∞
jω

by convention,X e is represented over [−π, π]
479 / 643
∞
X
−∞
jω

480 / 643
∞
X
−∞
jω

481 / 643
n
x[n] = α u[n] |α| < 1
482 / 643
n
x[n] = α u[n] |α| < 1
∞
X
−∞
∞
X
X e jω = αn e −jωn

n=0
jω
1
X e =
1 − αe −jω
483 / 643
n
x[n] = α u[n] |α| < 1
X e jω =
1
1 + α2 − 2α cos (ω)
484 / 643
Plotting a DTFT
485 / 643
Plotting a DTFT
486 / 643
Plotting a DTFT
487 / 643
Plotting a DTFT
488 / 643
n
x[n] = α u[n] |α| < 1
489 / 643
Remember the periodicity
490 / 643
491 / 643
492 / 643
493 / 643
Existence and properties of the DFTF
494 / 643
Existence easy for absolutely summable sequences
∞

jω
X −jωn

X e = x[n]e

n=−∞
∞
X
X e jω 6 x[n]e −jωn

n=−∞
X∞
X e jω =

|x[n]|
n=−∞
X e jω < ∞

495 / 643
Inversion easy for absolutely summable sequences
ˆπ ˆπ ∞
!
1 1 X
jω
e jωn dω = x [k] e −jωk e jωn dω

X e
2π 2π
−π −π k=−∞
ˆπ ∞ ˆπ
1 jω
jωn
X e jω(n−k)
X e e dω = x [k] dω
2π 2π
−π k=−∞ −π
ˆπ
1
X e jω e jωn dω = x[n]

2π
−π
496 / 643
A formal change of basis
formally DTFT is an inner product in C∞ :

∞
X
x[n]e −jωn = e jωn , x[n]

−∞
“basis” is an infinite, uncountable basis: {e jωn }ω∈R

something “breaks down”: we start with sequences but the transform is a function
we used absolutely summable sequences but DTFT exists for all square-summable
sequences (proof is rather technical)
497 / 643
Review: DFT
498 / 643
Review: DFS
499 / 643
Review: DTFT
500 / 643
DTFT properties
501 / 643
DTFT properties
502 / 643
Some particular cases:
503 / 643
The DTFT as a change of basis
504 / 643
DTFT as basis expansion
Some things are OK:

DFT{δ[n]} = 1
DTFT{δ[n]} = e jωn , δ[n] = 1

505 / 643
Some things are OK:

DFT{δ[n]} = 1
DTFT{δ[n]} = e jωn , δ[n] = 1

506 / 643
Some things aren’t

DFT{1} = Nδ[k]
∞
e −jωn =?
P
DTFT{1} =
−∞
problem: too many interesting sequences are not square summable
507 / 643

DFT{1} = Nδ[k]
∞
e −jωn =?
P
DTFT{1} =
−∞
508 / 643

DFT{1} = Nδ[k]
∞
e −jωn =?
P
DTFT{1} =
−∞
509 / 643
The Dirac delta functional
Defined by the “sifting” property:

ˆ∞
δ (t − s) f (t) dt = f (s)
−∞
for all functions of s, t ∈ R
510 / 643
The “pulse train”
∞
X
δ̃ (ω) = 2π δ (ω − 2πk)
k=−∞
just a technicality to use the Dirac delta in the space of 2π-periodic functions
511 / 643
Graphical representation
512 / 643
Now let the show begin!
ˆπ
1
IDTFT{δ̃ (ω)} = δ̃ (ω) e jωn dω
2π
−π
ˆπ
IDTFT{δ̃ (ω)} = δ (ω) e jωn dω
−π
jωn

IDTFT{δ̃ (ω)} = e ω=0
IDTFT{δ̃ (ω)} = 1
This is similar to the fact that the IDFT{Nδ[n]} = 1
513 / 643
Now let the show begin!
ˆπ
1
IDTFT{δ̃ (ω)} = δ̃ (ω) e jωn dω
2π
−π
ˆπ
IDTFT{δ̃ (ω)} = δ (ω) e jωn dω
−π
jωn

IDTFT{δ̃ (ω)} = e ω=0
IDTFT{δ̃ (ω)} = 1
This is similar to the fact that the IDFT{Nδ[n]} = 1
514 / 643
In other words
DTFT{1} = δ̃ (ω)
515 / 643
Using the same technique
IDTFT{δ̃ (ω − ω0 )} = e jωo n
So:
DTFT{1} = δ̃ (ω)
DTFT{e jωo n } = δ̃ (ω − ω0 )
h i
DTFT{cos (ω0 n)} = δ̃ (ω − ω0 ) + δ̃ (ω + ω0 ) /2
h i
DTFT{ sen (ω0 n)} = −j δ̃ (ω − ω0 ) + δ̃ (ω + ω0 ) /2
516 / 643
So:
DTFT{1} = δ̃ (ω)
h i
h i
517 / 643
So:
DTFT{1} = δ̃ (ω)
h i
h i
518 / 643
So:
DTFT{1} = δ̃ (ω)
h i
h i
519 / 643
Introduction to Filtering
520 / 643
Linear Filters
521 / 643
Linear time-invariant filters

A generic signal processing device
y [n] = H {x [n]}
522 / 643

Linearity
H {αx1 [n] + βx2 [n]} = αH {x1 [n]} + βH {x2 [n]}
523 / 643

Linearity
524 / 643

(Non) Linearity
525 / 643

Time Invariance
y [n] = H {x [n]} ⇔ H {x [n − n0 ]} = y [n − n0 ]
526 / 643

Time Invariance
527 / 643

Time Variance
528 / 643

Linear, Time-Invariant Systems
529 / 643

LTI and Causality
y [n] = H (x[n], x[n − 1], x[n − 2], . . . , y [n − 1], y [n − 2], . . .)
with H() a linear function of its arguments
530 / 643
Convolution
Impulse response
y [n] = H {x [n]}
h[n] = H {δ [n]}
Fundamental result: impulse response fully characterizes the LTI system!
531 / 643
Convolution
Example
532 / 643
Convolution
Example
x[n] = 2δ[n] + 3δ[n − 1] + δ[n − 2]

we know the impulse response h[n] = H {δ [n]}
compute y [n] = H {x [n]} exploiting linearity and time-invariance.
533 / 643
Convolution
Example
x[n] = 2δ[n] + 3δ[n − 1] + δ[n − 2]

534 / 643
Convolution
Example
x[n] = 2δ[n] + 3δ[n − 1] + δ[n − 2]

535 / 643
Convolution
Example
536 / 643
Linear Filters
537 / 643
The moving average filter

Typical filtering scenario: denoising
538 / 643

Denoising by Moving Average
idea: replace each sample by the local average

for instance: y [n] = (x[n] + x[n − 1])/2
more generally:
M−1
1 X
y [n] = x[n − k]
M
k=0
539 / 643


more generally:
M−1
1 X
y [n] = x[n − k]
M
k=0
540 / 643


more generally:
M−1
1 X
y [n] = x[n − k]
M
k=0
541 / 643

542 / 643

543 / 643

544 / 643

545 / 643

MA: impulse response
M−1
1 X
h[n] = δ[n − k]
M
k=0
= 1/M for 0 ≤ n < M
546 / 643

MA: impulse response
547 / 643

MA: analysis
smoothing effect proportional to M

number of operations and storage also proportional to M
548 / 643

Generalization: The Moving Average (MA) process
For a MA filter
1 1 1
y [n] = x[n] + x[n − 1] + · · · + x[n − (M − 1)]
M M M
and
1 1 −1 1 −(M−1)
M + Mz + ... M z
H(z) =
1
In general, for a MA process we have
b0 + b1 z −1 + . . . bQ z −Q
H(z) =
1
549 / 643
The Leaky Integrator
try the filter

y [n] = λy [n − 1] + (1 − λ)x[n]
filter is now recursive, since it uses its previous output value

it can be shown that for λ ≈ 1, the Leaky Integrator is similar to the moving
average with a large M.
550 / 643
try the filter

y [n] = λy [n − 1] + (1 − λ)x[n]
filter is now recursive, since it uses its previous output value

it can be shown that for λ ≈ 1, the Leaky Integrator is similar to the moving
average with a large M.
551 / 643

Denoising recursively
552 / 643

553 / 643

554 / 643

What about the impulse response?
555 / 643

Impulse response?
h[n] = (1 − λ)λn u[n]
556 / 643

Why the name?
Discrete-time integrator is a boundless accumulator:

n
X
y [n] = x[k]
k=−∞
We can rewrite the integrator as
y [n] = y [n − 1] + x[n]
557 / 643

Why the name?
558 / 643

Generalization: The AutoRegressive (AR) process
In general, for a AR process we have
b0
H(z) =
1 + a1 z −1 + . . . aP z −P
559 / 643
Filter Stability
560 / 643
Filter classification in the time domain

Filter types according to impulse response
Finite Impulse Response (FIR)

Infinite Impulse Response (IIR)
causal
noncausal
561 / 643


causal
noncausal
562 / 643


causal
noncausal
563 / 643

FIR
impulse response has finite support

only a finite number of samples are involved in the computation of each output
sample
564 / 643

FIR (example)
565 / 643

IIR
impulse response has infinite support

a potentially infinite number of samples are involved in the computation of each
output sample
surprisingly, in many cases the computation can still be performed in a finite
amount of steps
566 / 643

IIR (example)
h[n] = (1 − λ)λn u[n]
Leaky Integrator
567 / 643

Causal vs Noncausal
causal
impulse response is zero for n < 0
only past samples (with respect to the present) are involved in the computation of
each output sample
causal filters can work ”on line” since they only need the past
noncausal
impulse response is nonzero for some (or all) n<0
can still be implemented in a offline fashion (when all input data is available on
storage, e.g., in Image Processing)
568 / 643

Causal example
Moving Average filter
569 / 643

NonCausal example
Zero-centered Moving Average filter
570 / 643
Filter stability
Stability
key concept: avoid “explosions” if the input is nice

a nice signal is a bounded signal: |x[n]| < M for all n
Bounded-Input Bounded-Output (BIBO) stability: if the input is nice the output
should be nice
571 / 643
Filter stability
Stability

should be nice
572 / 643
Filter stability
Stability

should be nice
573 / 643
Filter stability
Fundamental Stability Theorem
A filter is BIBO stable if and only if its impulse response is absolutely summable
X
|h[n]| = L < ∞ ⇔ |y [n]| < ML and |x[n]| < M
n
574 / 643
Filter stability
The good news
FIR filters are always stable
575 / 643
Filter stability
Checking the stability of IIRs
Let’s check the Leaky Integrator:
576 / 643
Frequency Response
577 / 643
The convolution theorem

A remarkable result
578 / 643

A remarkable result
579 / 643

A remarkable result
complex exponentials are eigensequences of LTI systems, i.e., linear filters cannot
change the frequency of sinusoids
DTFT of impulse response determines the frequency characteristics of a filter
580 / 643

A remarkable result
complex exponentials are eigensequences of LTI systems, i.e., linear filters cannot
change the frequency of sinusoids
DTFT of impulse response determines the frequency characteristics of a filter
581 / 643

Magnitude and phase
582 / 643
In general:
DTFT{x[n] ∗ h[n]} =?
Intuition: the DTFT reconstruction formula tell us how to build x[n] from a set of
complex exponential “basis” functions. By linearity . . .
583 / 643
In general:
DTFT{x[n] ∗ h[n]} =?
Intuition: the DTFT reconstruction formula tell us how to build x[n] from a set of
complex exponential “basis” functions. By linearity . . .
584 / 643
585 / 643

Frequency response
H(e jω ) = DTFT{h[n]}
Two effects:
magnitude: amplification (|H(e jω )| > 1) or attenuation (|H(e jω )| < 1) of input
frequencies
phase: overall delay and shape changes
586 / 643

Frequency response
H(e jω ) = DTFT{h[n]}
Two effects:
magnitude: amplification (|H(e jω )| > 1) or attenuation (|H(e jω )| < 1) of input
frequencies
phase: overall delay and shape changes
587 / 643
Examples of frequency response

Moving Average revisited
588 / 643

Moving Average, magnitude response
589 / 643

590 / 643

591 / 643

Denoising revisited
592 / 643

Denoising revisited
593 / 643

Denoising revisited
594 / 643

Denoising revisited
595 / 643

Denoising revisited
596 / 643

Denoising revisited
597 / 643

Denoising revisited
598 / 643

By the way, remember the time-domain analysis
599 / 643

By the way, remember the time-domain analysis
600 / 643

What about the phase?
Assume |H(e jω )| = 1
zero phase: ∠H(e jω ) = 0
linear phase: ∠H(e jω ) = dω, where d ∈ R
nonlinear phase
601 / 643

nonlinear phase
602 / 643

nonlinear phase
603 / 643

Phase and signal shape: zero phase
604 / 643

Phase and signal shape: linear phase
605 / 643

Phase and signal shape: nonlinear phase
In all three cases (zero, linear and nonlinear), the spectrum magnitude
remains the same. 606 / 643

Linear phase
y [n] = x[n − d]
Y (e jω ) = e −jωd X (e jω )
H(e jω ) = e −jωd
linear phase term
607 / 643

Linear phase
y [n] = x[n − d]
linear phase term
608 / 643

Linear phase
y [n] = x[n − d]
linear phase term
609 / 643

Linear phase
y [n] = x[n − d]
linear phase term
610 / 643

Linear phase
611 / 643

Moving Average is Linear phase
1 sen ω2 M −j M−1 ω

jω

H e = e 2
M sen ω2
612 / 643

Leaky integrator revisited
h[n] = (1 − λ)λn u[n]
613 / 643

1−λ
H e jω =

1 − λe jω
Finding magnitude and phase require a little algebra ...
614 / 643

615 / 643

Leaky integrator, magnitude response
616 / 643

617 / 643

618 / 643

Leaky integrator, phase response
619 / 643

Leaky integrator, phase response
620 / 643

Phase is sufficiently linear where it matters
621 / 643

Phase is sufficiently linear where it matters
622 / 643
Ideal Filters
623 / 643
Ideal Filters
Filter types according to magnitude response
Lowpass
Highpass
Bandpass
Allpass
Moving Average and Leaky Integrator are lowpass filters
624 / 643
Ideal Filters
Lowpass
Highpass
Bandpass
Allpass
625 / 643
Ideal Filters
Lowpass
Highpass
Bandpass
Allpass
626 / 643
Ideal Filters
Lowpass
Highpass
Bandpass
Allpass
627 / 643
Ideal Filters
Lowpass
Highpass
Bandpass
Allpass
628 / 643
Ideal Filters
Filter types according to phase response
Linear phase
Nonlinear phase
629 / 643
Ideal Filters
Filter types according to phase response
Linear phase
Nonlinear phase
630 / 643
Ideal Filters
What is the best lowpass we can think of?
631 / 643
Ideal Filters
What is the best lowpass we can think of?
632 / 643
Ideal Filters
Ideal lowpass filter

jω
1 for |ω 6 ωc |
H e = (2πperiodicity implicit)
0 otherwise
perfectly flat passband

infinite attenuation in stopband
zero-phase (no delay)
633 / 643
Ideal Filters
Ideal lowpass filter:impulse response
634 / 643
Ideal Filters
Ideal lowpass filter:impulse response
635 / 643
Ideal Filters
The bad news
impulse response is infinite support, two-sided

cannot compute the output in a finite amount of time
that’s why it’s called “ideal”
impulse response decays slowly in time
we need a lot of samples for a good approximation
636 / 643
Ideal Filters
The bad news
impulse response is infinite support, two-sided

cannot compute the output in a finite amount of time
that’s why it’s called “ideal”
impulse response decays slowly in time
we need a lot of samples for a good approximation
637 / 643
Ideal Filters
Nevertheless ...
The sinc-rect pair:
638 / 643
Ideal Filters
Little-known fact
the sinc is not absolutely summable

the ideal lowpass is not BIBO stable
639 / 643
Ideal Filters
Nevertheless ...
The sinc-rect pair:
640 / 643
Ideal Filters
Idel higpass filter
641 / 643
Ideal Filters
Idel bandpass filter
642 / 643
References I
[1] Paolo Prandoni and Martin Vetterli.

Digital Signal Processing.
https://www.coursera.org/learn/dsp/, 2018.
[2] Martin Vetterli and Paolo Prandoni.

Signal Processing for Communications.
EPFL press, 2008.
643 / 643

Slides Seniales

Încărcat de

Informații document

Descriere originală:

Drepturi de autor

Formate disponibile

Partajați acest document

Partajați sau inserați document

Opțiuni de partajare

Vi se pare util acest document?

Este necorespunzător acest conținut?

Drepturi de autor:

Formate disponibile

Slides Seniales

Încărcat de

Drepturi de autor:

Formate disponibile

Agenda Introducción Vector Spaces and DSP Basic of Fourier Analysis Advanced Fourier Analysis Introduction to Filtering

Slides mainly compiled from [1] and [2]

1 Introduction to digital signal processing

4 Advanced Fourier Analysis

Introduction to digital signal processing

Signal: Description of the evolution of a physical phenomenon.

Signal: Description of the evolution of a physical phenomenon.

Signal: Description of the evolution of a physical phenomenon.

Signal: Description of the evolution of a physical phenomenon.

Signal: Description of the evolution of a physical phenomenon.

Analog Signals (physics)

Analog Signals (electronics)

Analog Signals (physics)

only 1 degree of freedom: vy

Analog Signals (electronics)

only 2 degrees of freedom: R, C

What about “interesting” signals?

To each analog signal, a different device

From analog ...

... 12 −45 33 41 −89 60 70 −99 ...

The Digital Paradigm

The Digital Paradigm

Two competing models for reality

Two competing models for reality

Discrete time is very practical! Computing the average

Discrete time is very practical! Computing the average

Discrete time is very practical! Computing the average

Discrete time is very practical! Computing the average

The discrete-time model

The discrete-time model

x[n] = . . . , 1.2390, −0.7372, 0.8978, 0.1498, −1.1501, −0.2642

How can we go from continuous to discrete time?

from continuous to discrete time: the founding fathers

The Sampling Theorem (1920)

Under the appropriate “slowness” conditions for x(t) we have:

Sinc: the building block

from continuous to discrete time (and back)

from continuous to discrete time (and back)

from continuous to discrete time (and back)

from continuous to discrete time (and back)

from continuous to discrete time (and back)

from continuous to discrete time (and back)

When can we do all this? Joseph Fourier will tell us

What happens to analog signals

What happens to analog signals

What happens to analog signals

What happens to analog signals

We can amplify to compensate attenuation

Transmission of analog signals

Transmission of analog signals

Transmission of analog signals

Transmission of analog signals

Transmission of analog signals

Transmitting a signal overseas

Transmitting a signal overseas

For a long, long channel we need repeaters

Transmission of analog signals

Transmission of analog signals

Transmission of analog signals

Transmission of analog signals

Transmission of analog signals

Transmission of analog signals

Transmission of analog signals