Digital Signal Processing Prandoni and Martin Vetterli © 2013

P
sing terli
s
e
c
t
Signal Processing
ro in VeDigital
P
l
a
t
n
r
g
a
i
tal S i and M
i
g
Module 8: Image Processing
i
D
on 013
d
n
Pra
2
aolo
Module Overview:
Module 8.1: Introduction to Images and Image Processing
Module 8.2: Affine Transforms
Module 8.3:
Module 8.4:
Module 8.5:
Module 8.6: The JPEG Compression Standard
sing terli
s
e
c
ro in Vet
P
l
a
art
2D Fourier Analysis l Sign
M
igita oni and 3
Image FiltersD
and 201
r
P
lo
Image Compression
Pao
sing terli
s
e
c
t
Signal Processing
ro in VeDigital
P
l
a
t
n
r
g
a
i
tal S i and M
i
g
Module 8.1: Image Processing
i
D
on 013
d
n
Pra
2
aolo
Overview:
8.1
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
2D signal representations
l
d
ta
Digi doni an 13
Basic signals and operators ran
20
P
o
l
Pao
Images as multidimensional digital signals
Overview:
8.1
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
ta
Digi doni an 13
20
P
o
l
Pao
Overview:
8.1
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
ta
Digi doni an 13
20
P
o
l
Pao
Please meet ...
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
ta
Digi doni an 13
ran 20
P
o
l
ao
8.1
Digital images
8.1
sing terli
s
e
c
ro in Vet
P
l
a
indices locate a point on a grid pixel
gn
art
i
S
M
l
d
ta
grid is usually regularly spaced
Digi doni an 13
0
Pran appearance
2
lo pixels
values x[n1 , n2 ] refer toao
the
P
two-dimensional signal x[n1 , n2 ], n1 , n2 Z
Digital images
8.1
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
ta
Digi doni an 13
0
Pran appearance
2
lo pixels
the
P
Digital images
8.1
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
ta
Digi doni an 13
0
Pran appearance
2
lo pixels
the
P
Digital images
8.1
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
ta
Digi doni an 13
0
Pran appearance
2
lo pixels
the
P
Digital images: grayscale vs color
grayscale images: scalar pixel values
color images: multidimensional pixel values in a color space

g (RGB, HSV, YUV, etc)
8.1
n
essi etterli
c
o
r
we can consider the single components separately:
V
nal P Martin
g
i
S
d
tal
Digi doni an 13
ran 20
P
o
l
Pao

8.1
n
essi etterli
c
o
r
V
nal P Martin
g
i
S
d
tal
Digi doni an 13
ran 20
P
o
l
Pao

8.1
n
essi etterli
c
o
r
V
nal P Martin
g
i
S
d
tal
Digi doni an 13
ran 20
P
o
l
Pa=o
+
+
Image processing
sing terli
s
e
c
ro in Vet
P
something still works
l
a
gn
art
i
S
M
l
d
ta
something breaks down
Digi doni an 13
ran 20
P
something is new
o
l
Pao
From one to two dimensions...
8.1
Image processing
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
ta
Digi doni an 13
ran 20
P
something is new
o
l
Pao
8.1
Image processing
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
ta
Digi doni an 13
ran 20
P
something is new
o
l
Pao
8.1
Image processing
sing terli
s
e
c
ro in Vet
P
linearity, convolution
l
a
gn
art
i
S
M
l
d
ta
Fourier transform
Digi doni an 13
n
20
interpolation, sampling lo Pra
Pao
What works:
8.1
Image processing
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
ta
Fourier transform
Digi doni an 13
n
20
Pao
What works:
8.1
Image processing
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
ta
Fourier transform
Digi doni an 13
n
20
Pao
What works:
8.1
Image processing
sing terli
s
e
c
ro in Vet
P
Fourier analysis less relevant
l
a
gn
art
i
S
M
l
d
filter design hard, IIRs rare igita
D
ni an 13
o
d
ran 20
P
linear operators only mildly
useful
o
l
Pao
What breaks down:
8.1
Image processing
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
D
ni an 13
o
d
ran 20
P
useful
o
l
Pao
What breaks down:
8.1
Image processing
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
D
ni an 13
o
d
ran 20
P
useful
o
l
Pao
What breaks down:
8.1
Image processing
Whats new:
sing terli
s
e
c
ro in Vet
P
l
a
images are finite-support signals Sign
art
M
l
ta
nd
Digi indotheir
ni aentirety
images are (most often) available
causality loses meaning
an 2013
r
P
lo
images are very specialized
Pao signals, designed for a very specific processing system, i.e.
new manipulations: affine transforms
the
human brain! Lots of semantics that is extremely hard to deal with
8.1
Image processing
Whats new:
sing terli
s
e
c
ro in Vet
P
l
a
art
M
l
ta
nd
Digi indotheir
ni aentirety
an 2013
r
P
lo
the
8.1
Image processing
Whats new:
sing terli
s
e
c
ro in Vet
P
l
a
art
M
l
ta
nd
Digi indotheir
ni aentirety
an 2013
r
P
lo
the
8.1
Image processing
Whats new:
sing terli
s
e
c
ro in Vet
P
l
a
art
M
l
ta
nd
Digi indotheir
ni aentirety
an 2013
r
P
lo
the
8.1
2D signal processing: the basics
sing terli
s
e
c
A two-dimensional discrete-space signal:
ro in Vet
P
l
a
gn
art
i
S
M
l
x[n
ta , n ], dn , n Z
Digi 1do2ni an 11 3 2
ran 20
P
o
l
Pao
8.1
10
2D signals: Cartesian representation
h[n1 , n2 ]
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
ta
Digi doni an 13
ran 20
P
o
l
ao
n1
8.1
n2
11
2D signals: support representation
sing terli
s
e
c
ro in Vet
P
l
a
amplitude may be written along Sign
art
M
l
d
ta
explicitly
Digi doni an 13 0
ran 20
P
example:
o
l
o
( Pa
n2
just show coordinates of nonzero

samples
[n1 , n2 ] =
1 if n1 = n2 = 0
0 otherwise.
n1
8.1
12
2D signals: image representation
510
n2
425 ing
ess etterli
c
o
r
V
nal P M340
rtin
g
a
i
S
image values are quantized (usually
ital toni and 255
g
i
D
o
8 bits, or 256 levels)
and 2013
r
P
lo in space
170
the eye does the interpolation
Pao
medium has a certain dynamic range

(paper, screen)
provided the pixel density is high

enough
85
0
0
85
170
255
340
425
510
n1
8.1
13
Why 2D?
8.1
sing terli
s
e
c
o
Vet
images could be unrolled (printers, fax) al Pr
n
i
t
n
r
l Sig nd Ma
a
t
i
but what about spatial correlation?
Dig doni a 13
ran 20
P
o
l
Pao
14
Why 2D?
8.1
sing terli
s
e
c
o
Vet
images could be unrolled (printers, fax) al Pr
n
i
t
n
r
l Sig nd Ma
a
t
i
but what about spatial correlation?
Dig doni a 13
ran 20
P
o
l
Pao
14
2D vs raster scan
20
10
sing terli
s
e
c
ro in Vet
10
P
l
a
gn
art
i
S
M
l
d
20
ta
Digi 20do10
ni a0n 1103 20
ran 20
P
o
l
o
0
Pa
bbbbbbbbbbbbbbbbbbbbb
bbbbbbbbbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb
0
8.1
b b b b b b b b b b b b b b b b b b b b b b b b b b b b b b b b b b b b b b b b b
205
410
615
820
bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb
1025
1230
1435
1640
15
2D vs raster scan
20
10
sing terli
s
e
c
ro in Vet
10
P
l
a
gn
art
i
S
M
l
d
20
ta
Digi 20do10
ni a0n 1103 20
ran 20
P
o
l
o
0
Pa
b b b b b b b b b b
bbb
b b b b b b b b b b
bbbbb
bbbbb
bbbbb
b b b b b b b b b b
b b b b b b
bbb
bbbbbbbbbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbb bbbbbbbbbbb bbbbbbbbbbb bbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb
0
8.1
b b b b b
205
410
615
820
1025
1230
1435
1640
15
2D vs raster scan
20
10
b b b
sing terli
s
e
c
ro in Vet
10
P
l
a
gn
art
i
S
M
l
d
20
ta
Digi 20do10
ni a0n 1103 20
ran 20
P
o
l
o
0
b b b
Pa
bb
b b b b b
bbb
b b b b b
bbb
bbb
b b b b b
bbb
b b b b b
bbb
bbb
b b b b b
bbb
b b b b b
bb
bbbbbbbbbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb
0
8.1
b b b b b
205
410
615
820
1025
1230
1435
1640
15
2D vs raster scan
20
10
b b
b b
b b
b b
b b
b b
b b
sing terli
s
e
c
ro in Vet
10
P
l
a
gn
art
i
S
M
l
d
20
ta
Digi 20do10
ni a0n 1103 20
ran 20
P
o
l
o
0
b b
Pa
b b
b b
b b
b b
b b
b b
b b
b b
b b
bbbbbbbbbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb
0
8.1
b b
b b
b b
205
410
615
820
1025
1230
1435
1640
15
2D vs raster scan
20
10
b
b
b
b
b
b
b
b
b
sing terli
s
e
c
ro in Vet
10
P
l
a
gn
art
i
S
M
l
d
20
ta
Digi 20do10
ni a0n 1103 20
ran 20
P
o
l
o
0
Pa
b
bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbbb
0
8.1
205
410
615
820
1025
1230
1435
1640
15
2D vs raster scan
20
b
b
b
b
b
b
b
b
b
10
b
b
b
b
b
b
b
sing terli
s
e
c
ro in Vet
10
P
l
a
gn
art
i
S
M
l
d
20
ta
Digi 20do10
ni a0n 1103 20
ran 20
P
o
l
o
b
b
b
b
b
b
b
b
b
b
b
Pa
b
bbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbb
0
8.1
b
b
205
410
615
820
1025
1230
1435
1640
15
2D vs raster scan
20
b
b
b
b
b
b
b
b
b
b
10
b
b
b
b
b
b
b
sing terli
s
e
c
ro in Vet
10
P
l
a
gn
art
i
S
M
l
d
20
ta
Digi 20do10
ni a0n 1103 20
ran 20
P
o
l
o
b
b
b
b
b
b
b
b
b
b
b
b
b
b
b
b
b
b
b
b
b
b
b
b
Pa
b
bbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbb
0
8.1
205
410
615
820
1025
1230
1435
1640
15
2D vs raster scan
20
b
b
b
b
b
b
b
b
b
b
10
b
b
b
b
b
b
b
sing terli
s
e
c
ro in Vet
10
P
l
a
gn
art
i
S
M
l
d
20
ta
Digi 20do10
ni a0n 1103 20
ran 20
P
o
l
o
b
b
b
b
b
b
b
b
b
b
b
b
b
b
b
b
b
b
b
b
b
b
b
b
Pa
b
bbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbbbbbbbbbbb bbbbbbbbbbb
0
8.1
205
410
615
820
1025
1230
1435
1640
15
Basic 2D signals: the impulse
(
1
[n1 , n2 ] =
0
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
if n1 = n2 =
0
d
ta
Digi doni an 13 0
otherwise.
ran 20
P
o
l
Pao
b
5
5
8.1
16
Basic 2D signals: the rect
sing terli
s
e
c
ro in Vet
P
l
a
1 if |n1 | < N1 ign
art
S

M
l
a N2 i and 0
and |nig2 |it<
n1 n2
D
rect
,
=
don 2013
n
2N1 2N2
a
olo P
0 Pa
otherwise;
b
5
5
8.1
17
Separability
sing terli
s
e
c
ro in Vet
P
l
a
g2 ]n= x1[nM
a2r[nt2]
i
S
x[n
,
n
]x
l
1
1
d
ta
Digi doni an 13
ran 20
P
o
l
o
Pa
8.1
18
Separable signals
[n1 , n2 ] = [n1 ][n1s]ing
erli
es
t
c
t
o
e
r
V
nal P Martin
g
i
S
d
tal
Digi doni an 13

rann2 20 n1 n2
P
n
o
1
l
rect
Pao2N1 , 2N2 = rect 2N1 rect 2N2 .
8.1
19
Separable signals
[n1 , n2 ] = [n1 ][n1s]ing
erli
es
t
c
t
o
e
r
V
nal P Martin
g
i
S
d
tal
Digi doni an 13

rann2 20 n1 n2
P
n
o
1
l
rect
Pao2N1 , 2N2 = rect 2N1 rect 2N2 .
8.1
19
Nonseparable signal
x[n1 , n2 ] =
1
0
sing terli
s
e
c
ro in Vet
P
l
a
ign d Mart
if |n1 | + |n2 |it<alNS
Dig doni an 13 0
otherwise
ran 20
P
o
l
Pao
b
b
b
b
5
5
8.1
20
Nonseparable signal
sing terli
s
e
c
ro in Vet
P
l
a
n Mart

al Sig
d
t
n1 n2
Digi dn1on, i na2n 13 0
x[n1 , n2 ] = rect
,
rect
2N1 2N2
2M2
20
ran2M1
P
o
l
Pao
b
5
5
8.1
21
2D convolution
sing terli
s
e
c
ro in Vet
P
l
X
X
a
a1r,tk2 ]h[n1 k1 , n2 k2]
x[n1 , n2 ] h[n1 , n2 ] = l Sign
x[k
M
ta
nd
Digi k d=
ni ka=
o
an 2013
r
P
lo
Pao
1
8.1
22
2D convolution for separable signals
If h[n1 , n2 ] = h1 [n1 ]h2 [n2 ]:
sing terli
s
e
c
ro in Vet
P
l
a
t
Xign
arX
M
x[n1 , n2 ] h[n1 , n2 ] =ital S h1 [nn
k
]
x[k1 , k2 ]h2 [n2 k2 ]
1d 1
a
i
Dig k =
n
k =
o
and 2013
r
P
lo
Pao = h1 [n1 ] (h2[n2 ] x[n1 , n2 ]).
1
8.1
23
2D convolution for separable signals
sing terli
s
e
c
ro in Vet
P
l
a
rt sample
per
non-separable convolution: M1 M2 operations
gn
aoutput
i
S
M
l
nd
igitaoperations
separable convolution: M1D+ M2 n
oni a 0per13output sample!
d
a
2
lo Pr
o
a
P
If h[n1 , n2 ] is an M1 M2 finite-support signal:
8.1
24
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
ta
Digi doni an 13
ran 20
P
o
l
ao
END OF MODULE 8.1

P
sing terli
s
e
c
t
Signal Processing
ro in VeDigital
P
l
a
t
n
r
g
a
i
tal S i and M
i
g
Module 8.2: Image Manipulations
i
D
on 013
d
n
Pra
2
aolo
Overview:
Affine transforms
Bilinear interpolation
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
ta
Digi doni an 13
ran 20
P
o
l
ao
8.2
25
Overview:
Affine transforms
Bilinear interpolation
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
ta
Digi doni an 13
ran 20
P
o
l
ao
8.2
25
Affine transforms
mapping R2 R2 that reshapes the coordinate system:

t1
a11 a12 t1 ssindg1
=
e d etterli
t2
a21 al22Protc2
V2
gna Martin
i
S
l
d
ta
Digi doni an 13
ran 20
P
o
l
ao

t1
t
=A 1 d
t2
t2
8.2
26
Affine transforms
mapping R2 R2 that reshapes the coordinate system:

t1
a11 a12 t1 ssindg1
=
e d etterli
t2
a21 al22Protc2
V2
gna Martin
i
S
l
d
ta
Digi doni an 13
ran 20
P
o
l
ao

t1
t
=A 1 d
t2
t2
8.2
26
Translation
sing terli
s
e
c
ro in Vet

P
l
a
1 0
gn
art
i
A=
=I
S
M
l
d
0 1
ta
Digi doni an 13 0

d
ran 20
P
d= 1
o
l
d2 Pao
0
8.2
27
Translation
sing terli
s
e
c
ro in Vet

P
l
a
1 0
gn
art
i
A=
=I
S
M
l
d
0 1
ta
Digi doni an 13 0

d
ran 20
P
d= 1
o
l
d2 Pao
0
8.2
27
Scaling
sing terli
s
e
c
a
0
ro in Vet
P
l
A= 1
a
0 a2
gn
art
i
S
M
l
d
ta
Digi doni an 13 0
d=0
ran 20
P
o
l
if a1 = a2 the aspect ratio
Paiso preserved

8.2
28
Scaling
sing terli
s
e
c
a
0
ro in Vet
P
l
A= 1
a
0 a2
gn
art
i
S
M
l
d
ta
Digi doni an 13 0
d=0
ran 20
P
o
l
if a1 = a2 the aspect ratio
Paiso preserved

8.2
28
Rotation
A=
d=0
cos
sin
sing terli
s
e
c
ro in Vet
P
l

a
gn
art
i
sin
S
M
l
d
ta
cos Digi
ni an 13 0
o
d
ran 20
P
o
l
Pao
0
8.2
29
Rotation
A=
d=0
cos
sin
sing terli
s
e
c
ro in Vet
P
l

a
gn
art
i
sin
S
M
l
d
ta
cos Digi
ni an 13 0
o
d
ran 20
P
o
l
Pao
0
8.2
29
Rotation
A=
d=0
cos
sin
sing terli
s
e
c
ro in Vet
P
l

a
gn
art
i
sin
S
M
l
d
ta
cos Digi
ni an 13 0
o
d
ran 20
P
o
l
Pao
0
8.2
29
Flips
Horizontal:

sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
ta
Digi doni an 13 0
ran 20
P
o

l
ao
1 0
A=
0 1
d=0
Vertical:
A=
1 P0
0 1
d=0
0
8.2
30
Flips
Horizontal:

sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
ta
Digi doni an 13 0
ran 20
P
o

l
ao
1 0
A=
0 1
d=0
Vertical:
A=
1 P0
0 1
d=0
0
8.2
30
Shear
Horizontal:

sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
ta
Digi doni an 13 0
ran 20
olo P
a

1 s
A=
0 1
d=0
Vertical:
A=
1 P0
s 1
d=0
0
8.2
31
Shear
Horizontal:

sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
ta
Digi doni an 13 0
ran 20
olo P
a

1 s
A=
0 1
d=0
Vertical:
A=
1 P0
s 1
d=0
0
8.2
31
Affine transforms in discrete-space
sing terli
s
e
c
et

l Pro
V
n
i
a
t
t1
gnn1
ar 2 2
t=
alASi n2i andd M R 6= Z
ti2g
i
D
don 2013
n
a
r
P
aolo
8.2
32
Solution for images
8.2
apply the inverse transform:

g
t1
din
s
li
1
1 m1e+
s
= A roc
; etter
t2
l P m2 + din2 V
gna Mart
i
S
l
d mid-point
interpolate from the originalig
grid
tapoint to the
D i doni an 13
ran1, 2 +22),0 1,2 Z,
(t1 , t2 ) =lo
(1P+
Pao
0 1,2 < 1
33
Solution for images
8.2
apply the inverse transform:

g
t1
din
s
li
1
1 m1e+
s
= A roc
; etter
t2
l P m2 + din2 V
gna Mart
i
S
l
d mid-point
interpolate from the originalig
grid
tapoint to the
D i doni an 13
ran1, 2 +22),0 1,2 Z,
(t1 , t2 ) =lo
(1P+
Pao
0 1,2 < 1
33
Bilinear Interpolation
x[1 + 1, 2 + 1]
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
ta
Digi doni an 13
ran 20
P
o
l
x[ , ]
ao
b
2 + 1
1
8.2
1 + 1
34
P a
2
x[1 , 2 ]
1
8.2
x[1 + 1, 2 + 1]
sing terli
s
e
c
(t , t )ro
Vet
P
l
n
i
a
t
n
r
l Sig nd Ma
a
t
i
Dig doni a 13
ran 20
P
o
l
o
2 + 1
1 + 1
34
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
ta
Digi donian 13
ran 20
P
o
l
ao
b
2 + 1
1
8.2
1 + 1
34
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
ta
Digi doni an 13
ran 20
P
o
l
ao
b
2 + 1
1
8.2
1 + 1
34
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
ta
Digi donian 13
ran 20
P
o
l
ao
b
2 + 1
1
8.2
1 + 1
34
sing terli
s
e
c
ro in Vet
P
l
a
g2n)x[1 , M
y [m1 , m2 ] = (1 1 )(1l
+rt1 (1 2 )x[1 + 1, 2 ]
i
2] a
S
nd
igita o1n, i2a+
+ (1 D
1 )2 x[
1] + x[ + 1, 2 + 1]
d
an 2013 1 2 1
r
P
lo
Pao
If we use a first-order interpolator:
8.2
35
Shearing
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
ta
Digi doni an 13
ran 20
P
o
l
ao
8.2
36
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
ta
Digi doni an 13
ran 20
P
o
l
ao
END OF MODULE 8.2

P
sing terli
s
e
c
t
Signal Processing
ro in VeDigital
P
l
a
t
n
r
g
a
i
tal S i and M
i
g
Module 8.3: Frequency Analysis
i
D
on 013
d
n
Pra
2
aolo
Overview:
8.3
sing terli
s
e
c
ro in Vet
P
DFT
l
a
gn
art
i
S
M
l
d
ta
Magnitude and phase
Digi doni an 13
ran 20
P
o
l
Pao
37
Overview:
8.3
sing terli
s
e
c
ro in Vet
P
DFT
l
a
gn
art
i
S
M
l
d
ta
Magnitude and phase
Digi doni an 13
ran 20
P
o
l
Pao
37
2D DFT
g j rnli k
e
c
Vett
n =0 n =0 l Pro
n
i
a
t
n
r
l Sig nd Ma
a
t
i
Dig doni a 13
20
r1anNX
1 NX
1
P
o
l
j n k j n k
x[n1P
, na2o
]=
X [k1 , k2 ]e
e
N1 N2
X [k1 , k2 ] =
NX
1 1 N
2 1
X
1
x[n1 , n2 ]eessNi1n1 1 e
j 2 n k
2
N2
2 2
2
N1
1 1
2
N2
2 2
k1 =0 k2 =0
8.3
38
2D DFT
g j rnli k
e
c
Vett
n =0 n =0 l Pro
n
i
a
t
n
r
l Sig nd Ma
a
t
i
Dig doni a 13
20
r1anNX
1 NX
1
P
o
l
j n k j n k
x[n1P
, na2o
]=
X [k1 , k2 ]e
e
N1 N2
X [k1 , k2 ] =
NX
1 1 N
2 1
X
1
x[n1 , n2 ]eessNi1n1 1 e
j 2 n k
2
N2
2 2
2
N1
1 1
2
N2
2 2
k1 =0 k2 =0
8.3
38
2D-DFT Basis Vectors
sing terli
s
e
c
ro in Vet
P
l
a
j n k tj n k
wk ,k [nl 1S, ing2n
] = e Mare
d
ta
Digi doni an 13
ran 20
P
o
l
o
There are N1 N2 orthogonal basis vectors for an N1 N2 image:
2
N1
1 1
2
N2
2 2
for n1 , kP1 = 0, 1, . . . , N1 1 and n2 , k2 = 0, 1, . . . , N2 1
8.3
39
2D-DFT basis vectors for k1 = 1, k2 = 0 (real part)

255
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
ta
Digi doni an 13
ran 20
P
o
l
ao
0
0
8.3
255
40

255
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
ta
Digi doni an 13
ran 20
P
o
l
ao
0
0
8.3
255
41

255
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
ta
Digi doni an 13
ran 20
P
o
l
ao
0
0
8.3
255
42

255
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
ta
Digi doni an 13
ran 20
P
o
l
ao
0
0
8.3
255
43

255
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
ta
Digi doni an 13
ran 20
P
o
l
ao
0
0
8.3
255
44

255
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
ta
Digi doni an 13
ran 20
P
o
l
ao
0
0
8.3
255
45

255
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
ta
Digi doni an 13
ran 20
P
o
l
ao
0
0
8.3
255
46

255
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
ta
Digi doni an 13
ran 20
P
o
l
ao
0
0
8.3
255
47

255
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
ta
Digi doni an 13
ran 20
P
o
l
ao
0
0
8.3
255
48

255
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
ta
Digi doni an 13
ran 20
P
o
l
ao
0
0
8.3
255
49
2D DFT
2D-DFT basis functions are separable, and so is the 2D-DFT:
sing terli
s
e
c
rno2 ]e j inn Vk eetj n k
P
X [k1 , k2 ] =
x[n
,
l
1
a
gn
art
S
M
na
=0
ni =0
l
d
t
Digi doni an 13
ran 20
1D-DFT along n2 (the columns)
P
o
l
Pao
NX
1 1 N
2 1
X
1
8.3
2
N1
1 1
2
N2
2 2
1D-DFT along n1 (the rows)
50
2D DFT

" 1
s2ing # ter2li
s
NX
1 1 N
2
e
X
c
rno2 ]e j Ni2nn2Vk2ete j N1 n1 k1
P
X [k1 , k2 ] =
x[n
,
l
1
a
t
n
r
l Sn ig=0 nd Ma
a
t
i
Dig doni a 13
ran 20
P
o
l
Pao
n1 =0
8.3
50
2D DFT
ng erli2
i
NX
NX
2 1
1 1
s
s
2
cej n2 k2 tt j n1 k1
x[n
X [k1 , k2 ] =
l P1 ,rno2 ]e N2n Ve e N1
na Marti
nig
=0
S
l
d
ta
Digi doni an 13
ran 20
P
o
l
Pao
n1 =0
8.3
50
2D DFT
ng erli
i
s
s
ocej nn Vk ette j
X [k1 , k2 ] =
x[n
P
1 ,rn2 ]e
l
a
n
arti
n =0
nig
=0
S
M
l
d
ta
Digi doni an 13
ran 20
P
o
l
Pao
NX
1 1
1
8.3
NX
2 1
2
N2
2 2
2
n k
N1 1 1
50
2D DFT in matrix form
finite-support 2D signal can be written as a matrix x
N1 N2 image is an N2 N1 matrix (n1 spans the columns,

g n2 spans the rows)
8.3
essi
terli
roc in Vet
recall also the N N DFT matrix (Module
4.2):
P
l
a
gn
art
i
S
M
l
1 igi1ta
1and
1
...
1
i
1 D W 1ndonW 2 013 W 3
. . . WNN1
N2
aN
lo Pr 2
2(N1)
4
6
o
1
W
W
W
.
.
.
W
WN =
a
N
N
N
P
...
2
2(N1)
3(N1)
(N1)
N1
1 WN
WN
WN
. . . WN
51

8.3
essi
terli
roc in Vet
4.2):
P
l
a
gn
art
i
S
M
l
1 igi1ta
1and
1
...
1
i
1 D W 1ndonW 2 013 W 3
. . . WNN1
N2
aN
lo Pr 2
2(N1)
4
6
o
1
W
W
W
.
.
.
W
WN =
a
N
N
N
P
...
2
2(N1)
3(N1)
(N1)
N1
1 WN
WN
WN
. . . WN
51

8.3
essi
terli
roc in Vet
4.2):
P
l
a
gn
art
i
S
M
l
1 igi1ta
1and
1
...
1
i
1 D W 1ndonW 2 013 W 3
. . . WNN1
N2
aN
lo Pr 2
2(N1)
4
6
o
1
W
W
W
.
.
.
W
WN =
a
N
N
N
P
...
2
2(N1)
3(N1)
(N1)
N1
1 WN
WN
WN
. . . WN
51
X [k1 , k2 ] =
NX
1 1
NX
2 1
x[n1 , n2 ]e
2
n k
N2 2 2
e j N1 n1 k1
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
ta
Digi doni an 13
ran 20
P
o
l
ao
n1 =0
n2 =0
8.3
52
X [k1 , k2 ] =
NX
1 1
NX
2 1
x[n1 , n2 ]e
2
n k
N2 2 2
e j N1 n1 k1
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
ta
DNigix doni an 13
V=W
ran 20
P
o
l
VPa
oRN N
n1 =0
n2 =0
8.3
52
X [k1 , k2 ] =
NX
1 1
NX
2 1
x[n1 , n2 ]e
2
n k
N2 2 2
e j N1 n1 k1
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
ta
DNigix doni an 13 X = V WN
V=W
ran 20
P
o
l
VPa
oRN N
X RN N
n1 =0
n2 =0
8.3
52
X [k1 , k2 ] =
NX
1 1
NX
2 1
x[n1 , n2 ]e
2
n k
N2 2 2
e j N1 n1 k1
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
ta
DNigix doni an 13 X = V WN
V=W
ran 20
P
o
l
VPa
oRN N
X RN N
n1 =0
n2 =0
X = WN2 x WN1
8.3
52
How does a 2D-DFT look like?
8.3
sing terli
s
e
c
ro in Vet
P
l
a
problem: the range is too big for theiggrayscale
range
n
art of paper or screen
S
M
l
d
anmax
igita o1 ,nni2]|/
try to normalize: |X [n1 , nD
2 ]| = |X [n
d
13|X [n1 , n2 ]|
n
0
a
2
r
P
but it doesnt work... aolo

P
try to show the magnitude as an image
53
DFT coefficients sorted by magnitude
107
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
ta
Digi doni an 13
ran 20
P
o
l
ao
105
103
101
0
8.3
5122
54
Dealing with HDR images
sing
if the image is high dynamic range we need to compress

ces the Vlevels
tter
o
e
r
P
P
P
al x[na1 ,rnt2in])
remove flagrant outliers (e.g. X [0, 0]ig
S =n
8.3
li
nd M
ital 1/3
a
g
i
i
D
n
use a nonlinear mapping: e.g. y =dxo after normalization
(x 1)
an 2013
r
P
lo
Pao
55
Dealing with HDR images
sing
if the image is high dynamic range we need to compress

ces the Vlevels
tter
o
e
r
P
P
P
al x[na1 ,rnt2in])
remove flagrant outliers (e.g. X [0, 0]ig
S =n
8.3
li
nd M
ital 1/3
a
g
i
i
D
n
use a nonlinear mapping: e.g. y =dxo after normalization
(x 1)
an 2013
r
P
lo
Pao
55
How does a 2D-DFT look like?

510
510
425
425
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
255
d
ta
Digi doni an 13
ran 20 170
P
o
l
ao
340
340
255
170
85
85
0
0
8.3
85
170
255
340
425
510
85
170
255
340
425
510
56
DFT magnitude doesnt carry much information
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
ta
Digi doni an 13
ran 20
P
o
l
ao
8.3
57
DFT phase, on the other hand...
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
ta
Digi doni an 13
ran 20
P
o
l
ao
8.3
58
Image frequency analysis
8.3
g
most of the information is contained in images edges
ssin
erli
e
t
c
t
o
e
r
tin V
edges are points of abrupt change inig
signals
nal P values
r
a
S
dM
ncaptured
ital ninot
a
edges are a space-domain
Digfeature
do 2013 by DFTs magnitude
n
a
r
lo P
phase alignment is important
Pao for reproducing edges
59
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
ta
Digi doni an 13
ran 20
P
o
l
ao
END OF MODULE 8.3

P
sing terli
s
e
c
t
Signal Processing
ro in VeDigital
P
l
a
t
n
r
g
a
i
tal S i and M
i
g
Module 8.4: Filtering
i
D
on 013
d
n
Pra
2
aolo
Overview:
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
ta
Digi doni an 13
ran 20
P
o
l
ao
Filters for image processing
Classification
Examples
8.4
60
Overview:
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
ta
Digi doni an 13
ran 20
P
o
l
ao
Classification
Examples
8.4
60
Overview:
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
ta
Digi doni an 13
ran 20
P
o
l
ao
Classification
Examples
8.4
60
Analogies with 1D filters
8.4
linearity
space invariance
impulse response
frequency response
stability
2D CCDE
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
ta
Digi doni an 13
ran 20
P
o
l
ao
61
The problem with LSI operators
interesting images contain lots of semantics: different information in different areas
n
space-invariant filters process everything in the same
essiway
g
erli
t
c
t
o
e
r
V
but we should process things differently
nal P Martin
g
i
S
d
tal
edges
Digi doni an 13
gradients
ran 20
P
o
l
Pao
textures
...
8.4
62
n
essiway
g
erli
t
c
t
o
e
r
V
nal P Martin
g
i
S
d
tal
edges
Digi doni an 13
gradients
ran 20
P
o
l
Pao
textures
...
8.4
62
n
essiway
g
erli
t
c
t
o
e
r
V
nal P Martin
g
i
S
d
tal
edges
Digi doni an 13
gradients
ran 20
P
o
l
Pao
textures
...
8.4
62
n
essiway
g
erli
t
c
t
o
e
r
V
nal P Martin
g
i
S
d
tal
edges
Digi doni an 13
gradients
ran 20
P
o
l
Pao
textures
...
8.4
62
n
essiway
g
erli
t
c
t
o
e
r
V
nal P Martin
g
i
S
d
tal
edges
Digi doni an 13
gradients
ran 20
P
o
l
Pao
textures
...
8.4
62
n
essiway
g
erli
t
c
t
o
e
r
V
nal P Martin
g
i
S
d
tal
edges
Digi doni an 13
gradients
ran 20
P
o
l
Pao
textures
...
8.4
62
n
essiway
g
erli
t
c
t
o
e
r
V
nal P Martin
g
i
S
d
tal
edges
Digi doni an 13
gradients
ran 20
P
o
l
Pao
textures
...
8.4
62
Filter types
8.4
sing terli
s
e
c
ro in Vet
P
causal or noncausal
l
a
gn
art
i
S
M
l
d
ta
highpass, lowpass, ...
Digi doni an 13
lowpass image smoothingran
20
P
o
l
highpass enhancement,
Pao edge detection
IIR, FIR
63
Filter types
8.4
sing terli
s
e
c
ro in Vet
P
causal or noncausal
l
a
gn
art
i
S
M
l
d
ta
Digi doni an 13
20
P
o
l
Pao edge detection
IIR, FIR
63
Filter types
8.4
sing terli
s
e
c
ro in Vet
P
causal or noncausal
l
a
gn
art
i
S
M
l
d
ta
Digi doni an 13
20
P
o
l
Pao edge detection
IIR, FIR
63
Filter types
8.4
sing terli
s
e
c
ro in Vet
P
causal or noncausal
l
a
gn
art
i
S
M
l
d
ta
Digi doni an 13
20
P
o
l
Pao edge detection
IIR, FIR
63
Filter types
8.4
sing terli
s
e
c
ro in Vet
P
causal or noncausal
l
a
gn
art
i
S
M
l
d
ta
Digi doni an 13
20
P
o
l
Pao edge detection
IIR, FIR
63
The problems with 2D IIRs
8.4
sing terli
s
e
c
ro in Vet
P
l
a
border effects
gn
art
i
S
M
l
igita ofonalgebra
i and doesnt
stability: the fundamental D
theorem
hold in multiple dimensions!
d
13
n
0
a
2
r
lo P
computability
Pao
nonlinear phase (edges!)
64
8.4
sing terli
s
e
c
ro in Vet
P
l
a
border effects
gn
art
i
S
M
l
igita ofonalgebra
i and doesnt
theorem
d
13
n
0
a
2
r
lo P
computability
Pao
64
8.4
sing terli
s
e
c
ro in Vet
P
l
a
border effects
gn
art
i
S
M
l
igita ofonalgebra
i and doesnt
theorem
d
13
n
0
a
2
r
lo P
computability
Pao
64
8.4
sing terli
s
e
c
ro in Vet
P
l
a
border effects
gn
art
i
S
M
l
igita ofonalgebra
i and doesnt
theorem
d
13
n
0
a
2
r
lo P
computability
Pao
64
A noncomputable CCDE
y [n1 , n2 ] = a0 y [n1 + 1, n2 ] + a1 y [n1 , n2 1] + a2 y [n1 1, n2 ] + a3 y [n1 , n2 + 1] + x[n1 , n2 ];
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
ta
Digi doni an 13
ran 20
P
o
l
ao
2
y[0, 1]
y[0, 0]
y[1, 0]
bc
y[1, 0]
y[0, 1]
2
8.4
2
65
A noncomputable CCDE
y [n1 , n2 ] = a0 y [n1 + 1, n2 ] + a1 y [n1 , n2 1] + a2 y [n1 1, n2 ] + a3 y [n1 , n2 + 1] + x[n1 , n2 ];
sing terli
s
e
c
ro in Vet
P
l
a
1
gn
art
i
S
M
l
d
ta
Digi doni an 13
ran 20
0
P
o
l
ao
2
bc
8.4
2
65
Practical FIR filters
8.4
generally zero centered (causality not an issue)

ing
odd number of taps in both directions
cess
erli
t
t
o
e
r
V
nal P Martin
per-sample complexity:
g
i
S
tal i and
M1 M2 for nonseparable impulse
Digi responses
don 2013
n
a
r
M1 + M2 for separable impulse
lo P responses
Pao
obviously always stable
66
8.4

ing
cess
erli
t
t
o
e
r
V
nal P Martin
g
i
S
tal i and
Digi responses
don 2013
n
a
r
lo P responses
Pao
66
8.4

ing
cess
erli
t
t
o
e
r
V
nal P Martin
g
i
S
tal i and
Digi responses
don 2013
n
a
r
lo P responses
Pao
66
8.4

ing
cess
erli
t
t
o
e
r
V
nal P Martin
g
i
S
tal i and
Digi responses
don 2013
n
a
r
lo P responses
Pao
66
8.4

ing
cess
erli
t
t
o
e
r
V
nal P Martin
g
i
S
tal i and
Digi responses
don 2013
n
a
r
lo P responses
Pao
66
Moving Average
x[n
sin1g k1t,enr2li k2]
s
e
c
k =N
kro
=N
Vet
P
l
n
i
a
t
n
r
l Sig nd Ma
a
t
i
Dig doni a 13
ran 20
P
o
l
o
n n
1
y [n1 , n2 ] =
N
X
1
(2N + 1)2
1
Pah[n1, n2 ] =
8.4
(2N +
N
X
1)2
rect
2N 2N
67
Moving Average
x[n
sin1g k1t,enr2li k2]
s
e
c
k =N
kro
=N
Vet
P
l
n
i
a
t
n
r
l Sig nd Ma
a
t
i
Dig doni a 13
ran 20
P
o
l
o
1
n n
y [n1 , n2 ] =
N
X
1
(2N + 1)2
1
Pah[n1, n2 ] =
8.4
(2N +
N
X
1)2
rect
2N 2N
67
Moving Average
sing
roces etter
P
a1l 1 a1 rt1in V
n
g
i
S
h[n
ta1l, n2] =i a9nd 1M 1 1
i
g
i
D
1 31 1
on
ran
P
o
l
ao
8.4
li
01
68
Moving Average
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
ta
Digi doni an 13
ran 20
P
o
l
ao
11 11 MA
8.4
51 51 MA
69
Gaussian Blur
sing terli
s
e
c
1
r,o |nin,Vn e|t< N
P
l
h[n1 , n2 ] =
e
a
g2 n
art 1 2
2
i
S
M
l
d
ta
Digi doni an 13
ran 20
P
o
l
Pao
with N 3
n2 +n2
1 22
2
8.4
70
Gaussian Blur
h[n1 , n2 ]
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
ta
Digi doni an 13
ran 20
P
o
l
ao
n1
8.4
n2
71
Gaussian Blur
12
8
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
0
l
d
ta
Digi doni an 13
4
ran 20
P
o
l
ao
P8
4
12
12 8
8.4
4
0
4
= 5, N = 14
12
72
Gaussian Blur
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
ta
Digi doni an 13
ran 20
P
o
l
ao
= 1.8, 11 11 blur
8.4
= 8.7, 51 51 blur
73
Sobel filter
approximation of the first derivative in the horizontal direction:
1 0 1
g
so [n1 , n2 ] = 2 e
0 ss2in
erli
t
c
t
o
e
r
al P1 0 1in V
gn
art
i
S
M
l
d
ta
Digi doni an 13
ran 20
P
o
l
ao
separability and structure:P
8.4

1

so [n1 , n2 ] = 2 1 0 1
1
74
Sobel filter
approximation of the first derivative in the horizontal direction:
1 0 1
g
so [n1 , n2 ] = 2 e
0 ss2in
erli
t
c
t
o
e
r
al P1 0 1in V
gn
art
i
S
M
l
d
ta
Digi doni an 13
ran 20
P
o
l
ao
separability and structure:P
8.4

1

so [n1 , n2 ] = 2 1 0 1
1
74
Sobel filter
approximation of the first derivative in the vertical direction:

sing terli
s
e
c
l Pro
Vet
n
i
a
t
ar 1
Snig] n= n1
M2
l
d
a
t
s
[n
,
0
i
Digv 1 do2 ni a 113 02 01
an
20
lo
Pao
8.4
Pr
75
Sobel filter
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
ta
Digi doni an 13
ran 20
P
o
l
ao
horizontal Sobel filter

8.4
vertical Sobel filter

76
Sobel operator
approximation for the square magnitude of the gradient: sing
erli
es
t
c
t
o
e
r
al1P, n2 ]|2a+rt|sinv [nV1, n2 ] x[n1 , n2 ]|2
|x[n1 , n2 ]|2 = |so [n1 , n2 ] g
n
x[n
i
S
dM
tal
Digi doni an 13
(operator because its nonlinear)
ran 20
P
o
l
Pao
8.4
77
Gradient approximation for edge detection
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
ta
Digi doni an 13
ran 20
P
o
l
ao
Sobel operator
8.4
thresholeded Sobel operator

78
Laplacian operator
sing terli
s
e
c
ro in Vet
P
l
a
t
gn 2M
i
f ar 2 f
S
l
f
ta(t , t ) = d +
Digi d1on2i ant112 3 t22
ran 20
P
o
l
o
Laplacian of a function in continuous-space:
Pa
8.4
79
Laplacian operator
approximating the Laplacian; start with a Taylor expansion
f (t + ) =
X
f (n) (t)
nn g
n!
s
es i
erli
t
c
t
o
e
r
V
nal P Martin
g
i
S
l
gita) andon(ti and):
and compute the expansion in D
(t i+
and 2013
r
P
lo
Paof (t + ) = f (t) + f (t) + 1 f (t) 2
n=0
2
1
f (t ) = f (t) f (t) + f (t) 2
2
8.4
80
Laplacian operator
by rearranging terms:
dM
tal
Digi doni an 13
ranh[n] =1202
P
which, on the discrete grid, is lthe
FIR
o
Pao
8.4
ssin
rli
e
e
1
r2foc(t)
Ve+tt ))
P
f (t) = 2 (f (t a
)l
+inf (t
t
Sign
ar
1
81
Laplacian
summing the horizontal and vertical components:

sing terli
s
e
c
ro in Vet
P
l
a
gn 0 M1art0
i
S
l
d
ta
1
Digih[n1d, no2n] i=an10 14
3
an
20 1 0
lo
Pao
8.4
Pr
82
Laplacian
sing terli
s
e
c
ro
Vet
P
l
1
1
1
n
i
a
t
n
ar
l ,Sni2g] = n1d M
h[n
8 1
1
a
t
i
Dig doni a 1 131 1
ran 20
P
o
l
o
If we use the diagonals too:
Pa
8.4
83
Laplacian for Edge Detection
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
ta
Digi doni an 13
ran 20
P
o
l
ao
Laplacian operator
8.4
thresholeded Laplacian operator

84
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
ta
Digi doni an 13
ran 20
P
o
l
ao
END OF MODULE 8.4

P
sing terli
s
e
c
t
Signal Processing
ro in VeDigital
P
l
a
t
n
r
g
a
i
tal S i and M
i
g
Module 8.5: Image Compression
i
D
on 013
d
n
Pra
2
aolo
Overview:
8.5
sing terli
s
e
c
ro in Vet
P
Redundancy in natural images
l
a
gn
art
i
S
M
l
d
Image coding ingredients Digita
ni an 13
o
d
ran 20
P
o
l
Pao
85
Overview:
8.5
sing terli
s
e
c
ro in Vet
P
Redundancy in natural images
l
a
gn
art
i
S
M
l
d
Image coding ingredients Digita
ni an 13
o
d
ran 20
P
o
l
Pao
85
A thought experiment
8.5
sing terli
s
e
c
ro in Vet
P
l
each image is 524,288 bits
a
gn
art
i
S
M
l
igita 524,288
i and 157,826
D
n
o
d
total number of possible images:
an2 201013
r
P
lo
Paouniverse: 1082
number of atoms in the
consider all possible 256 256, 8bpp images
86
8.5
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
igita 524,288
i and 157,826
D
n
o
d
an2 201013
r
P
lo
Paouniverse: 1082
86
8.5
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
igita 524,288
i and 157,826
D
n
o
d
an2 201013
r
P
lo
Paouniverse: 1082
86
8.5
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
igita 524,288
i and 157,826
D
n
o
d
an2 201013
r
P
lo
Paouniverse: 1082
86
How many bits per image?

Another thought experiment
8.5
take all images in the world and list them in an encyclopedia of images
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
ta
on the Internet: M = 50 billion
Digi doni an 13
n
20
raimage
P
raw encoding: 524,288 bits

per
o
l
Pao
to indicate an image, simply give its number
enumeration-based encoding: log2 M 36 bits per image
(of course, side information is HUGE)
87

8.5
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
ta
Digi doni an 13
n
20
raimage
P

per
o
l
Pao
87

8.5
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
ta
Digi doni an 13
n
20
raimage
P

per
o
l
Pao
87

8.5
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
ta
Digi doni an 13
n
20
raimage
P

per
o
l
Pao
87

8.5
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
ta
Digi doni an 13
n
20
raimage
P

per
o
l
Pao
87

8.5
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
ta
Digi doni an 13
n
20
raimage
P

per
o
l
Pao
87
Compression
sing terli
s
e
c
ro in Vet
P
l
a
gn edges)Mart
i
S
allocate bits for things that matter
(e.g.
l
igita oni and 3
D
use psychovisual experiments toafind
nd out what
201matters
r
P
o
l
Pao
some information is discarded:
lossy compression
Another approach:
8.5
exploit physical redundancy
88
Compression
sing terli
s
e
c
ro in Vet
P
l
a
gn edges)Mart
i
S
(e.g.
l
igita oni and 3
D
nd out what
201matters
r
P
o
l
Pao
lossy compression
Another approach:
8.5
88
Compression
sing terli
s
e
c
ro in Vet
P
l
a
gn edges)Mart
i
S
(e.g.
l
igita oni and 3
D
nd out what
201matters
r
P
o
l
Pao
lossy compression
Another approach:
8.5
88
Compression
sing terli
s
e
c
ro in Vet
P
l
a
gn edges)Mart
i
S
(e.g.
l
igita oni and 3
D
nd out what
201matters
r
P
o
l
Pao
lossy compression
Another approach:
8.5
88
Key ingredients
8.5
sing terli
s
e
c
ro in Vet
P
l
a
using a suitable transform (i.e., a change
gn of basis)
art
i
S
M
l
d
ta
smart quantization
Digi doni an 13
ran 20
P
o
l
entropy coding
Pao
compressing at block level
89
Key ingredients
8.5
sing terli
s
e
c
ro in Vet
P
l
a
gn of basis)
art
i
S
M
l
d
ta
smart quantization
Digi doni an 13
ran 20
P
o
l
entropy coding
Pao
89
Key ingredients
8.5
sing terli
s
e
c
ro in Vet
P
l
a
gn of basis)
art
i
S
M
l
d
ta
smart quantization
Digi doni an 13
ran 20
P
o
l
entropy coding
Pao
89
Key ingredients
8.5
sing terli
s
e
c
ro in Vet
P
l
a
gn of basis)
art
i
S
M
l
d
ta
smart quantization
Digi doni an 13
ran 20
P
o
l
entropy coding
Pao
89
Compressing at pixel level
8.5
sing terli
s
e
c
reduce number bits per pixel
ro in Vet
P
l
a
gn
art
i
S
M
l
equivalent to coarser quantization
d
ta
Digi doni an 13
in the limit, 1bpp
ran 20
P
o
l
Pao
90
8.5
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
ta
Digi doni an 13
in the limit, 1bpp
ran 20
P
o
l
Pao
90
8.5
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
ta
Digi doni an 13
ran 20
P
o
l
in the limit, 1bpp
Pao
90
Compressing at block level
8.5
sing terli
s
e
c
ro in Vet
P
l
a
art
code the average value with 8 bits l Sign
M
ita ni and
Diggives
o
3 3 blocks at 8 bits per block
andless 2013
r
P
than 1bpp
lo
Pao
divide the image in blocks
91
8.5
sing terli
s
e
c
ro in Vet
P
l
a
art
code the average value with 8 bits l Sign
M
ita ni and
Diggives
o
andless 2013
r
P
than 1bpp
lo
Pao
91
8.5
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
ta
code the average value with 8 ibits
D gi doni an 13
ran less 20
gives
P
o
l
than 1bpp
Pao
91
8.5
sing terli
s
e
c
ro in Vet
P
exploit the local spatial correlation
l
a
gn
art
i
S
M
l
d
ta
compress remote regions independently
Digi doni an 13
ran 20
P
o
l
Pao
92
8.5
sing terli
s
e
c
ro in Vet
P
exploit the local spatial correlation
l
a
gn
art
i
S
M
l
d
ta
compress remote regions independently
Digi doni an 13
ran 20
P
o
l
Pao
92
Transform coding
A simple example:
sing terli
s
e
c
ro in Vet
P
l
a
storing the signal requires NR bitsSign
art
M
l
d
ta
Digiti looks
ni an 13
o
now you take the DFT and
like
d
ran 20
P
this
o
l
Pao
take a DT signal, assume R bits per

sample
b
b
b
b
b
b
8.5
in theory, we can just code the two

nonzero DFT coefficients!
10
b
b
b
15
20
b
b
25
30
93
Transform coding
A simple example:
sing terli
s
e
c
ro in Vet
P
l
a
art
M
l
d
ta
Digiti looks
ni an 13
o
like
d
ran 20
P
this
o
l
Pao

sample
b
b
b
b
b
b
8.5

10
b
b
b
15
20
b
b
25
30
93
Transform coding
A simple example:
sing terli
s
e
c
ro in Vet
P
l
a
art
M
l
d
ta
Digiti looks
ni an 13
o
like
d
ran 20
P
this
o
l
Pao

sample
8.5

10
15
20
25
30
93
Transform coding
A simple example:
sing terli
s
e
c
ro in Vet
P
l
a
art
M
l
d
ta
Digiti looks
ni an 13
o
like
d
ran 20
P
this
o
l
Pao

sample
8.5

10
15
20
25
30
93
Transform coding
sing terli
s
e
c
o in na Vfew
et coefficients
captures the important features of an image
rblock
P
l
i
a
t
n
r
l Sig nd Ma
a
is efficient to compute
t
i
Dig doni a 13
ran 20
P
o
l
answer: the Discrete PCosine
ao Transform
Ideally, we would like a transform that:
8.5
94
Transform coding
sing terli
s
e
c
o in na Vfew
et coefficients
rblock
P
l
i
a
t
n
r
l Sig nd Ma
a
t
i
Dig doni a 13
ran 20
P
o
l
ao Transform
8.5
94
Transform coding
sing terli
s
e
c
o in na Vfew
et coefficients
rblock
P
l
i
a
t
n
r
l Sig nd Ma
a
t
i
Dig doni a 13
ran 20
P
o
l
ao Transform
8.5
94
2D-DCT
sing terli
s
e
c
ro in Vet
l

P
N1
X N1
X
a
1rt
1
n
g
a
i
S
C [k1 , k2 ] =
x[n1 , na2l] cos
ndn1M+ 2 k1 cos N n2 + 2 k2
it
N
a
i
n =0 n =0Dig
n
o
and 2013
r
P
lo
Pao
1
8.5
95
DCT basis vectors for an 8 8 image
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
ta
Digi doni an 13
ran 20
P
o
l
ao
8.5
96
Smart quantization
8.5
sing terli
s
e
c
ro in Vet
P
deadzone
l
a
gn
art
i
S
M
l
d
ta
variable step (fine to coarse)
Digi doni an 13
ran 20
P
o
l
Pao
97
Smart quantization
8.5
sing terli
s
e
c
ro in Vet
P
deadzone
l
a
gn
art
i
S
M
l
d
ta
variable step (fine to coarse)
Digi doni an 13
ran 20
P
o
l
Pao
97
Quantization
x[n]
2
sing 1terli
s
e
c
ro in Vet 00
P
l
Standard quantization:
a
gn
art
i
S
M
l
ita ni and
x = floor(x) + 0.5 Dig
o
013 1 10
and 22
r
P
lo
Pao
01
x[n]
1
1
11
2
8.5
98
Quantization
x[n]
2
Deadzone quantization:
sing 1terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
ta
00
Digi doni an 13
0
ran 22
1
P
o
l
ao
x = round(x)
01
x[n]
1
10
8.5
98
Entropy coding
8.5
ng
l Pro rtin Ve
a
n
g
a
i
associate short symbols to frequent
values
and vice-versa
tal S i and M
i
g
i
D
don 2013
n
a
r
if it sounds familiar its because
lo P it is...
Pao
si
li
minimize the effort to encode a certain amount cof
esinformation
tter
99
Entropy coding
8.5
ng
l Pro rtin Ve
a
n
g
a
i
values
and vice-versa
tal S i and M
i
g
i
D
don 2013
n
a
r
lo P it is...
Pao
si
li
esinformation
tter
99
Entropy coding
8.5
ng
l Pro rtin Ve
a
n
g
a
i
values
and vice-versa
tal S i and M
i
g
i
D
don 2013
n
a
r
lo P it is...
Pao
si
li
esinformation
tter
99
Entropy coding
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
ta
Digi doni an 13
ran 20
P
o
l
ao
8.5
100
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
ta
Digi doni an 13
ran 20
P
o
l
ao
END OF MODULE 8.5

P
sing terli
s
e
c
t
Signal Processing
ro in VeDigital
P
l
a
t
n
r
g
a
i
d M 8.6: The JPEG Compression Algorithm
tal S i anModule
i
g
i
D
don 2013
n
a
r
P
aolo
Key ingredients
8.6
sing terli
s
e
c
ro in Vet
P
l
a
gn of basis)
art
i
S
M
l
d
ta
smart quantization
Digi doni an 13
ran 20
P
o
l
entropy coding
Pao
101
Key ingredients
8.6
sing terli
s
e
c
ro in Vet
P
l
a
gn of basis)
art
i
S
M
l
d
ta
smart quantization
Digi doni an 13
ran 20
P
o
l
entropy coding
Pao
split image into 8 8 non-overlapping blocks
101
Key ingredients
8.6
sing terli
s
e
c
ro in Vet
P
l
a
compute the DCT of each block
gn
art
i
S
M
l
d
ta
smart quantization
Digi doni an 13
ran 20
P
o
l
entropy coding
Pao
101
Key ingredients
8.6
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
and
igita otonipsycovisually-tuned
quantize DCT coefficients D
according
tables
d
13
n
0
a
2
r
lo P
entropy coding
Pao
101
Key ingredients
8.6
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
and
igita otonipsycovisually-tuned
quantize DCT coefficients D
according
tables
d
13
n
0
a
2
r
lo P
run-length encoding and
PaoHuffman coding
101
DCT coefficients of image blocks (detail)
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
ta
Digi doni an 13
ran 20
P
o
l
ao
8.6
102
DCT coefficients of image blocks (detail)
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
ta
Digi doni an 13
ran 20
P
o
l
ao
8.6
102
Smart quantization
8.6
most coefficients are negligible captured by the deadzone

ing
ess etterli
c
o
r
P
some coefficients are more important than
tin V
nal others
r
g
a
i
S
tal i and M
find out the critical coefficients
Digi by dexperimentation
on 013
n
a
2
r
allocate more bits (or, equivalently,
finerquantization
levels) to the most important
lo P
o
a
P
coefficients
103
Smart quantization
8.6

ing
ess etterli
c
o
r
P
tin V
nal others
r
g
a
i
S
tal i and M
on 013
n
a
2
r
finerquantization
lo P
o
a
P
coefficients
103
Smart quantization
8.6

ing
ess etterli
c
o
r
P
tin V
nal others
r
g
a
i
S
tal i and M
on 013
n
a
2
r
finerquantization
lo P
o
a
P
coefficients
103
Smart quantization
8.6

ing
ess etterli
c
o
r
P
tin V
nal others
r
g
a
i
S
tal i and M
on 013
n
a
2
r
finerquantization
lo P
o
a
P
coefficients
103
Psychovisually-tuned quantization table

c[k1 , k2 ] = round(c[k1 , k2 ]/Q[k1 , k2 ])
sing51 ter61li
16 11 10 16 24 es40
c
o 58n V60
et 55
12 12 14 19 l Pr26
i
a
t
gn24 40Mar57 69 56
i
14 13 l 16
S
gita
i 17 22oni29and513 87 80 62
14
D
Q=
1
and37
r
22
56 2068 109 103 77
P
18
o
l
Pao24 35 55 64 81 104 113 92
49 64 78 87 103 121 120 101
72 92 95 98 112 100 103 99
8.6
104
Advantages of nonuniform bit allocation
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
ta
Digi doni an 13
ran 20
P
o
l
ao
uniform
8.6
tuned
105
Advantages of nonuniform bit allocation (detail)
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
ta
Digi doni an 13
ran 20
P
o
l
ao
uniform
8.6
tuned
106
Efficient coding
8.6
sing terli
Vet
P
l
n
i
a
t
n
r
use zigzag scan to maximize ordering
l Sig nd Ma
a
t
i
Dig doni a 13
quantization will create long series
20
ran of zeros
P
o
l
Pao
s
most coefficients are small, decreasing with index
roce
107
Efficient coding
8.6
sing terli
Vet
P
l
n
i
a
t
n
r
l Sig nd Ma
a
t
i
Dig doni a 13
20
ran of zeros
P
o
l
Pao
s
roce
107
Efficient coding
8.6
sing terli
Vet
P
l
n
i
a
t
n
r
l Sig nd Ma
a
t
i
Dig doni a 13
20
ran of zeros
P
o
l
Pao
s
roce
107
Zigzag scan
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
ta
Digi doni an 13
ran 20
P
o
l
ao
8.6
108
Example
100 60 0 6 0 0 0 0
0
0 0 0 0 0 0 0
0
0 0 0 0 e0ssi0ng0
tterli
c
o
e
0
r
0 0 l 0P 0 0 i0n V0
13 1
gna 0 a0rt 0 0
ital Si 0 n0d M
ig0
0 ni0a 0 0 0 0 0
D
103 0 0 0
0rando
0
0
0
0
2
lo P
0 0 0 0 0 0 0
Pao 0
100, -60, 0, 0, 0, 0, 6, 0, 0, 0, 13, 0, 0, 0, 0, 0, 0, 0, 0, 13, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,

0, 0, 0, 0, 0, 0, 0,0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
8.6
109
Example
100 60 0 6 0 0 0 0
0
0 0 0 0 0 0 0
0
0 0 0 0 e0ssi0ng0
tterli
c
o
e
0
r
0 0 l 0P 0 0 i0n V0
13 1
gna 0 a0rt 0 0
ital Si 0 n0d M
ig0
0 ni0a 0 0 0 0 0
D
103 0 0 0
0rando
0
0
0
0
2
lo P
0 0 0 0 0 0 0
Pao 0
100, -60, 0, 0, 0, 0, 6, 0, 0, 0, 13, 0, 0, 0, 0, 0, 0, 0, 0, 13, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,

0, 0, 0, 0, 0, 0, 0,0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
8.6
109
Runlength encoding
Each nonzero value is encoded as the triple
sing terli
s
e
c
ro in Vet
P
l
a
n before
rt current value
gzeros
athe
i
r is the runlength i.e. the number
of
S
M
l
igita oni and 3
D
s is the size i.e. the number of abits
nd needed2to
01encode the value
r
P
o
l
o
c is the actual value Pa
[(r , s), c]
8.6
(0, 0) indicates that from now on its only zeros (end of block)
110
Runlength encoding
sing terli
s
e
c
ro in Vet
P
l
a
n before
rt current value
gzeros
athe
i
of
S
M
l
igita oni and 3
D
nd needed2to
01encode the value
r
P
o
l
o
[(r , s), c]
8.6
110
Runlength encoding
sing terli
s
e
c
ro in Vet
P
l
a
n before
rt current value
gzeros
athe
i
of
S
M
l
igita oni and 3
D
nd needed2to
01encode the value
r
P
o
l
o
[(r , s), c]
8.6
110
Runlength encoding
sing terli
s
e
c
ro in Vet
P
l
a
n before
rt current value
gzeros
athe
i
of
S
M
l
igita oni and 3
D
nd needed2to
01encode the value
r
P
o
l
o
[(r , s), c]
8.6
110
Example
100 60 0 6 0 0 0 0
0
0 0 0 0 0 0 0
0
0
0
0
0
0
0
0
g
n erli
i
s
s
0
0 0 0 o0ce0 0 0ett
13 1 0al 0Pr 0 0 ti0n V0

n
ar
0tal S0ig 0 0d M
0
0
0
0
n
igi
D 0 do0 ni0a 0 103 0 0 0
an
20
0 0 0 0 0 0
lo P0r 0
Pao
[(0, 7), 100], [(0, 6), 60], [(4, 3), 6], [(3, 4), 13], [(8, 1), 1], [(0, 0)]
8.6
111
Example
100 60 0 6 0 0 0 0
0
0 0 0 0 0 0 0
0
0
0
0
0
0
0
0
g
n erli
i
s
s
0
0 0 0 o0ce0 0 0ett
13 1 0al 0Pr 0 0 ti0n V0

n
ar
0tal S0ig 0 0d M
0
0
0
0
n
igi
D 0 do0 ni0a 0 103 0 0 0
an
20
0 0 0 0 0 0
lo P0r 0
Pao
[(0, 7), 100], [(0, 6), 60], [(4, 3), 6], [(3, 4), 13], [(8, 1), 1], [(0, 0)]
8.6
111
The runlength-size pairs
8.6
sing terli
s
e
c
ro in Vet
P
l
a
in theory, 8 bits per pair
gn
art
i
S
M
l
nd
igita othan
some pairs are much moreDcommon
ni aothers!
d
an 2013
r
P
lo
a lot of space can be saved
Pao by being smart
by design, (r , s) A with |A| = 256
112
8.6
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
nd
igita othan
ni aothers!
d
an 2013
r
P
lo
Pao by being smart
112
8.6
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
nd
igita othan
ni aothers!
d
an 2013
r
P
lo
Pao by being smart
112
8.6
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
nd
igita othan
ni aothers!
d
an 2013
r
P
lo
Pao by being smart
112
Variable-length encoding
great idea: shorter binary sequences for common symbols
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
ta
Digi doni an 13
ran 20
P
o
l
ao
8.6
113
however: if symbols have different lengths, we must

singknow how
li to parse them!
8.6
tter
oces
Sign nd Mar
l
a
t
i
in Morse code, pauses separate
and words (wasteful)
Dig letters
oni a 013
d
n
a
can we do away with separators?
2
lo Pr
o
a
P
e
Pr (wasteful)
in English, spaces separate words extra
al symbol
tin V
114

singknow how
li to parse them!
8.6
tter
oces
Sign nd Mar
l
a
t
i
Dig letters
oni a 013
d
n
a
2
lo Pr
o
a
P
e
Pr (wasteful)
al symbol
tin V
114

singknow how
li to parse them!
8.6
tter
oces
Sign nd Mar
l
a
t
i
Dig letters
oni a 013
d
n
a
2
lo Pr
o
a
P
e
Pr (wasteful)
al symbol
tin V
114
Prefix-free codes
8.6
sing
al P artin V
n
g
i
S
can parse a bitstream sequentially
dM
tal with no look-ahead
Digi doni an 13
extremely easy to understand graphically...
ran 20
P
o
l
Pao
li
r
s
no valid sequence can be the beginning of another
ette
roce valid sequence
115
Prefix-free codes
8.6
sing
al P artin V
n
g
i
S
dM
Digi doni an 13
ran 20
P
o
l
Pao
li
r
s
ette
roce valid sequence
115
Prefix-free codes
8.6
sing
al P artin V
n
g
i
S
dM
Digi doni an 13
ran 20
P
o
l
Pao
li
r
s
ette
roce valid sequence
115
Prefix-free code
0
0
ng eCrli
i
s
s
0
roce in Vett
P
l
a
D
1
gn
art 1
i
S
M
l
d
ta
Digi doni 1an 13B
ran 20
P
o
l
ao
001100110101100
8.6
116
Prefix-free code
0
0
ng eCrli
i
s
s
0
roce in Vett
P
l
a
D
1
gn
art 1
i
S
M
l
d
ta
Digi doni 1an 13B
ran 20
P
o
l
ao
001100110101100
8.6
116
Prefix-free code
0
0
ng eCrli
i
s
s
0
roce in Vett
P
l
a
D
1
gn
art 1
i
S
M
l
d
ta
Digi doni 1an 13B
ran 20
P
o
l
ao
001100110101100
A
8.6
116
Prefix-free code
0
0
ng eCrli
i
s
s
0
roce in Vett
P
l
a
D
1
gn
art 1
i
S
M
l
d
ta
Digi doni 1an 13B
ran 20
P
o
l
ao
001100110101100
AA
8.6
116
Prefix-free code
0
0
ng eCrli
i
s
s
0
roce in Vett
P
l
a
D
1
gn
art 1
i
S
M
l
d
ta
Digi doni 1an 13B
ran 20
P
o
l
ao
001100110101100
AAB
8.6
116
Prefix-free code
0
0
ng eCrli
i
s
s
0
roce in Vett
P
l
a
D
1
gn
art 1
i
S
M
l
d
ta
Digi doni 1an 13B
ran 20
P
o
l
ao
001100110101100
AABA
8.6
116
Prefix-free code
0
0
ng eCrli
i
s
s
0
roce in Vett
P
l
a
D
1
gn
art 1
i
S
M
l
d
ta
Digi doni 1an 13B
ran 20
P
o
l
ao
001100110101100
AABAA
8.6
116
Prefix-free code
0
0
ng eCrli
i
s
s
0
roce in Vett
P
l
a
D
1
gn
art 1
i
S
M
l
d
ta
Digi doni 1an 13B
ran 20
P
o
l
ao
001100110101100
AABAAB
8.6
116
Prefix-free code
0
0
ng eCrli
i
s
s
0
roce in Vett
P
l
a
D
1
gn
art 1
i
S
M
l
d
ta
Digi doni 1an 13B
ran 20
P
o
l
ao
001100110101100
AABAABA
8.6
116
Prefix-free code
0
0
ng eCrli
i
s
s
0
roce in Vett
P
l
a
D
1
gn
art 1
i
S
M
l
d
ta
Digi doni 1an 13B
ran 20
P
o
l
ao
001100110101100
AABAABAD
8.6
116
Prefix-free code
0
0
ng eCrli
i
s
s
0
roce in Vett
P
l
a
D
1
gn
art 1
i
S
M
l
d
ta
Digi doni 1an 13B
ran 20
P
o
l
ao
001100110101100
AABAABADC
8.6
116
Entropy coding
sing terli
s
e
c
assign short sequences to more frequent symbols
ro in Vet
P
l
a
gn codeMfor
arat set of symbol probabilities
i
S
l
the Huffman algorithm builds the
optimal
d
ta
Digi doni an 13
0
in JPEG, you can use a general-purpose
code or build your own
ran 2Huffman
P
o
l
o
(but then you pay a side-information
price)
a
P
goal: minimize message length
8.6
117
Entropy coding
sing terli
s
e
c
ro in Vet
P
l
a
gn codeMfor
i
S
l
optimal
d
ta
Digi doni an 13
0
ran 2Huffman
P
o
l
o
price)
a
P
8.6
117
Entropy coding
sing terli
s
e
c
ro in Vet
P
l
a
gn codeMfor
i
S
l
optimal
d
ta
Digi doni an 13
0
ran 2Huffman
P
o
l
o
price)
a
P
8.6
117
Example
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
igita= 0.38
Dp(A)
ni an 13 p(B) = 0.32
o
d
ra) n= 0.1 20 p(D) = 0.2
P
o
p(C
l
o
four symbols: A, B, C, D
probability table:
Pa
8.6
118
Example
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
igita= 0.38
Dp(A)
ni an 13 p(B) = 0.32
o
d
ra) n= 0.1 20 p(D) = 0.2
P
o
p(C
l
o
four symbols: A, B, C, D
probability table:
Pa
8.6
118
Building the Huffman code
0.10 sCing
p(B) = 0.32
p(C ) = 0.1
erli
es
t
c
t
o
e
r
P
tin V
0.30ignal
r
a
S
dM
tal
Digi doni a1 n 0.20
an 2013 D
r
P
aolo
p(A) = 0.38
8.6
p(D) = 0.2
119
Building the Huffman code
singD terli
s
e
c
ro 1 in Vet
P
l
a
0.62
gn
art
i
S
M
l
ta
nd
Digi do1 ni a0.32
an 201B3
r
P
aolo
0
0.30
p(A) = 0.38
8.6
p(B) = 0.32
p(C + D) = 0.3
120
Huffman Coding
0.38 A
sing terli
s
e
c
ro in0Vet
P
l
a
1.00
C
art
1 l Sign
M
d
a
t
n
i
0
Dig doni a 13
D
1
ran 0.62 20
P
o
l
Pao
0
p(A) = 0.38
8.6
p(B + C + D) = 0.62
121
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
ta
Digi doni an 13
ran 20
P
o
l
ao
END OF MODULE 8.6

P
sing terli
s
e
c
ro in Vet
P
l
a
gn
art
i
S
M
l
d
ta
Digi doni an 13
ran 20
P
o
l
ao
END OF MODULE 8
P

Digital Signal Processing Prandoni and Martin Vetterli © 2013

Încărcat de

Informații document

Titlu original

Drepturi de autor

Formate disponibile

Partajați acest document

Partajați sau inserați document

Opțiuni de partajare

Vi se pare util acest document?

Este necorespunzător acest conținut?

Drepturi de autor:

Formate disponibile

Digital Signal Processing Prandoni and Martin Vetterli © 2013

Încărcat de

Drepturi de autor:

Formate disponibile

P

Module 8.1: Introduction to Images and Image Processing

Module 8.2: Affine Transforms

Module 8.6: The JPEG Compression Standard

Please meet ...

two-dimensional signal x[n1 , n2 ], n1 , n2 Z

two-dimensional signal x[n1 , n2 ], n1 , n2 Z

two-dimensional signal x[n1 , n2 ], n1 , n2 Z

two-dimensional signal x[n1 , n2 ], n1 , n2 Z

Digital images: grayscale vs color

grayscale images: scalar pixel values

color images: multidimensional pixel values in a color space

Digital images: grayscale vs color

grayscale images: scalar pixel values

color images: multidimensional pixel values in a color space

Digital images: grayscale vs color

grayscale images: scalar pixel values

color images: multidimensional pixel values in a color space

From one to two dimensions...

From one to two dimensions...

From one to two dimensions...

What breaks down:

What breaks down:

What breaks down:

human brain! Lots of semantics that is extremely hard to deal with

human brain! Lots of semantics that is extremely hard to deal with

human brain! Lots of semantics that is extremely hard to deal with

human brain! Lots of semantics that is extremely hard to deal with

2D signal processing: the basics

2D signals: Cartesian representation

2D signals: support representation

just show coordinates of nonzero

2D signals: image representation

medium has a certain dynamic range

provided the pixel density is high

Basic 2D signals: the impulse

Basic 2D signals: the rect

[n1 , n2 ] = [n1 ][n1s]ing

[n1 , n2 ] = [n1 ][n1s]ing

2D convolution for separable signals

If h[n1 , n2 ] = h1 [n1 ]h2 [n2 ]:

2D convolution for separable signals

If h[n1 , n2 ] is an M1 M2 finite-support signal:

END OF MODULE 8.1

Affine transforms in discrete-space

Solution for images

apply the inverse transform:

Solution for images

apply the inverse transform:

If we use a first-order interpolator:

END OF MODULE 8.2

2D-DFT Basis Vectors

There are N1 N2 orthogonal basis vectors for an N1 N2 image:

for n1 , kP1 = 0, 1, . . . , N1 1 and n2 , k2 = 0, 1, . . . , N2 1

2D-DFT basis vectors for k1 = 1, k2 = 0 (real part)

2D-DFT basis vectors for k1 = 0, k2 = 1 (real part)

2D-DFT basis vectors for k1 = 2, k2 = 0 (real part)

2D-DFT basis vectors for k1 = 3, k2 = 0 (real part)

2D-DFT basis vectors for k1 = 0, k2 = 3 (real part)

2D-DFT basis vectors for k1 = 30, k2 = 0 (real part)

2D-DFT basis vectors for k1 = 1, k2 = 1 (real part)

2D-DFT basis vectors for k1 = 2, k2 = 7 (real part)

2D-DFT basis vectors for k1 = 5, k2 = 250 (real part)