Image Pyramids

Image features at different resolutions require filters at

Image Representation different scales.

Edges (derivatives):
• Gaussian pyramids
• Laplacian Pyramids f(x)

• Wavelet Pyramids
• Applications
f (x)

Image Pyramids
Image pyramids
Image Pyramid = Hierarchical representation of an image
• Gaussian Pyramids
• Laplacian Pyramids
No details in image -
Low
Resolution (blurred image) • Wavelet/QMF
low frequencies

High Details in image -

Resolution low+high frequencies

A collection of images at different resolutions.

Image Pyramid Image Pyramid
Frequency Domain
Low resolution
Low resolution

High resolution High resolution

Image Blurring = low pass filtering

*
=

*
~
=
=

*
=
 Image Pyramid
Gaussian Pyramid
Low resolution

Level n
1X1

Level 1
2n-1 X 2n-1

Level 0
2n X 2n

High resolution

Gaussian Pyramid Gaussian Pyramid

Burt & Adelson (1981)

Normalized: Σwi = 1
Symmetry: wi = w-i

Unimodal: wi ≥ wj for 0 < i < j

Equal Contribution: for all j Σwj+2i = constant

w-2 w-1 w0 w1 w2

w-2 w-1 w0 w1 w2
 Gaussian Pyramid Gaussian Pyramid

Burt & Adelson (1981) Burt & Adelson (1981)

Normalized: Σwi = 1 Normalized: Σwi = 1

Symmetry: wi = w-i Symmetry: wi = w-i

Unimodal: wi ≥ wj for 0 < i < j Unimodal: wi ≥ wj for 0 < i < j

Equal Contribution: for all j Σwj+2i = constant Equal Contribution: for all j Σwj+2i = constant

w-2 w-1 w0 w-1 w-2 w-2 w-1 w0 w-1 w-2

w-2 w-1 w0 w-1 w-2 w-1 w-1

w0
w-2 w-2

Gaussian Pyramid
For a = 0.4 most similar to a Gauusian filter
Burt & Adelson (1981)
g = [0.05 0.25 0.4 0.25 0.05]

low_pass_filter = g’ * g =

c b a b c 0.0025 0.0125 0.0200 0.0125 0.0025

0.0125 0.0625 0.1000 0.0625 0.0125
0.0200 0.1000 0.1600 0.1000 0.0200
c b a b c
0.0125 0.0625 0.1000 0.0625 0.0125
0.0025 0.0125 0.0200 0.0125 0.0025

a + 2b + 2c = 1 0.2

a + 2c = 2b
0.15

0.1
a > 0.25
b = 0.25 0.05

c = 0.25 - a/2
0
5
4 5
3 4
3
2
2
1 1
 Gaussian Pyramid - MultiScale Pattern Matching
Computational Aspects
Option 1:
Scale target and search for each in image.
Memory:
2NX2N (1 + 1/4 + 1/16 + ... ) = 2NX2N * 4/3

Computation:

Level i can be computed with a single convolution

with filter: hi = g * g * g * .....

i times
Option 2:
Example:
Search for original target in image pyramid.

h2 = * =

g g

Hierarchical Pattern Matching Pattern matching using Pyramids - Example

image pattern correlation

search

search

search

search
 Laplacian Pyramid
Image pyramids Motivation = Compression, redundancy removal.
compression rates are higher for predictable values.
e.g. values around 0.
• Gaussian Pyramids
G0, G1, .... = the levels of a Gaussian Pyramid.
• Laplacian Pyramids
• Wavelet/QMF Predict level Gl from level Gl+1 by Expanding Gl+1 to G’l

Gl+1
Expand
Reduce

Gl G’l

Denote by Ll the error in prediction:

Ll = Gl - G’l

L0, L1, .... = the levels of a Laplacian Pyramid.

What does blurring take away? What does blurring take away?

original smoothed (5x5 Gaussian)

What does blurring take away? Laplacian Pyramid

Gaussian Laplacian
Pyramid Pyramid

expa
nd

ex -
pa
=
nd

ex -
pa
=
nd

- =

smoothed – original

Frequency Laplace Pyramid -

Gaussian
Pyramid Domain Laplacian
No scaling
Pyramid
 Reconstruction of the original image
from the Laplacian Pyramid

Laplacian Gl = Ll + G’l
Pyramid

expand

+ =
expand

+ =
expand

+ =

Original
= Image

from: B.Freeman

Laplacian Pyramid - Image Mosaicing

Computational Aspects

Memory:
2NX2N (1 + 1/4 + 1/16 + ... ) = 2NX2N * 4/3
However coefficients are highly compressible.

Computation:

Li can be computed from G0 with a single convolution

with filter: ki = hi-1 - hi

- =
Registration
hi-1 hi
ki

k1 k2 k3
 Image Blending Blending

Multiresolution Spline Multiresolution Spline

When splining two images, transition from one image to

the other should behave:

High Frequencies

High Frequencies

Middle Frequencies Middle Frequencies

Low Frequencies

Low Frequencies
Multiresolution Spline - Example Multiresolustion Spline - Using Laplacian Pyramid

Left Image Right Image

Left + Right Narrow Transition

Wide Transition

(Burt & Adelson)

Multiresolution Spline - Example Multiresolution Spline - Example

Left Image Right Image
Original - Left Original - Right

Left + Right Narrow Transition

Wide Transition Multiresolution Spline Glued Splined

(Burt & Adelson)

 Multiresolution Spline

laplacian level 4

laplacian level 2

laplacian level 0

left pyramid right pyramid blended pyramid

Multiresolution Spline - Example

Multi-Res. Blending

© prof. dmartin
 What is a good representation
Image pyramids
for image analysis?
• Gaussian Pyramids • Pixel domain representation tells you
• Laplacian Pyramids “where” (pixel location), but not
“what”.
• Wavelet/QMF – In space, this representation is too localized
• Fourier transform domain tells you
“what” (textural properties), but not
“where”.
– In space, this representation is too spread out.
• Want an image representation that gives
you a local description of image
events—what is happening where.
– That representation might be “just right”.

Space-Frequency Tiling Space-Frequency Tiling

Freq. Freq.
Standard basis Standard basis

Spatial Spatial
Freq. Freq.
Fourier basis Fourier basis

Spatial Spatial
Freq. Freq.
Wavelet basis Wavelet basis

Spatial Spatial
 Various Wavelet basis Wavelet - Frequency domain
Wavelet bands are split recursively

image

H L

H L

H L

Wavelet - Frequency domain

Wavelet decomposition - 2D
Wavelet - Frequency domain
Apply the wavelet transform separably in both dimensions

Frequency domain

Horizontal high pass, Horizontal high pass,

vertical high pass vertical low-pass

Horizontal high pass Vertical high pass

Horizontal low pass, Horizontal low pass,

vertical high-pass Vertical low-pass

Horizontal low pass Vertical low pass

 Pyramids in Frequency Domain
• Splitting can be applied recursively:
Gaussian Pyramid Laplacian Pyramid

Wavelet Decomposition Wavelet Transform - Step By

Step Example

Fourier Space
 Wavelet Transform - Example Wavelet Transform - Example

Application: Wavelet Shrinkage

Denoising

Clean image
Noisy image

From: B. Freeman From: B. Freeman

 Wavelet Shrinkage Denoising Wavelet Shrinkage Denoising
Wavelet coefficient For every Wavelet Band define
Noisy image Histogram
Shrinkage function:

New Coefficient Value

Wavelet coefficient
Clean image Histogram

Wavelet Coefficient Value

Get top histogram but want to get bottom histogram.

From: B. Freeman From: B. Freeman

Wavelet Shrinkage Pipe-line

More results

Mappingfunctions
Mapping functions
Transform
Transform xiw
I W
W
yiw

Inverse
Inverse
Transform
Transform
WTT
W

I clean
 Image Pyramids - Comparison

More results Image pyramid levels = Filter then sample.

Filters:

Gaussian Pyramid

Laplacian Pyramid

Wavelet Pyramid

Image Linear Transforms Convolution and Transforms

Transform Basis Characteristics
in matrix notation (1D case)

Localized in space r r
F = Uf
Delta Standard Not localized in Frequency

Not localized in space

Fourier Sines+Cosines
Localized in Frequency
transformed image
Vectorized image
Wavelet Localized in space
Wavelet Filters Basis vectors (Fourier, Wavelet, etc)
Pyramid Localized in Frequency

Delta Fourier Wavelet

space space space

 Fourier Transform Transform in matrix notation
(1D case)
Forward Transform:
r r
F = Uf
= * transformed image
Vectorized image
Basis vectors (Fourier, Wavelet, etc)

Fourier Fourier bases pixel domain

transform image Inverse Transform:
r r
−1
Fourier bases are global: each transform
U F=f
coefficient depends on all pixel locations.
Basis vectors Vectorized image
transformed image

From: B. Freeman

Inverse Fourier Transform Convolution

* = = *

Fourier pixel domain Convolution Circular Matrix pixel image

Fourier bases
transform image Result of
Filter Kernels

Every image pixel is a linear combination of the

Fourier basis weighted by the coefficient.

Note that if U is orthonormal basis then U −1 = U t

From: B. Freeman
 Pyramid = Pyramid =
Convolution + Sampling Convolution + Sampling
Pyramid Level 1

= = *
*
Pyramid Level 2

Convolution Matrix of pixel image

Result Filter Kernels = *

Pyramid Level 2

= * *
From: B. Freeman

Pyramid = Pyramid as Matrix

Convolution + Sampling U1 =
Computation - Example
1 4 6 4 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

Pyramid Level 2 0 0 1 4 6 4 1 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 1 4 6 4 1 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 1 4 6 4 1 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 1 4 6 4 1 0 0 0 0 0 0 0

= * * 0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
1
0
4
0
6
1
4
4
1
6
0
4
0
1
0
0
0
0
0
0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 4 6 4 1 0

- Next pyramid level

Pyramid Level 2 U2 =
1 4 6 4 1 0 0 0
0 0 1 4 6 4 1 0
0 0 0 0 1 4 6 4
= * 0 0 0 0 0 0 1 4

- The combined effect of the two pyramid levels

U2 * U1 =
1 4 10 20 31 40 44 40 31 20 10 4 1 0 0 0 0 0 0 0
0 0 0 0 1 4 10 20 31 40 44 40 31 20 10 4 1 0 0 0
0 0 0 0 0 0 0 0 1 4 10 20 31 40 44 40 30 16 4 0
0 0 0 0 0 0 0 0 0 0 0 0 1 4 10 20 25 16 4 0

from: B.Freeman
 Gaussian Pyramid Laplacian Pyramid

= = * = *
pixel image pixel image

Overcomplete representation. Overcomplete representation.

Gaussian Low-pass filters, sampled Laplacian Transformed pixels represent
pyramid appropriately for their blur. pyramid bandpassed image information.

From: B. Freeman From: B. Freeman

Wavelet Transform

= *
The End

Wavelet
Ortho-normal pixel image
pyramid
transform (like
Fourier transform),
but with localized
basis functions.

From: B. Freeman