Ravi Im Rest

Digital Image Processing
UNIT-4:
Image Restoration
Concept of Image Restoration
Image restoration is to restore a degraded image back to

the original image while image enhancement is to
manipulate the image so that it is suitable for a specific
application.
Degradation model:
g ( x, y ) f ( x, y ) h ( x, y ) ( x, y )
where h(x,y) is a system that causes image distortion and

(x,y) is noise. (Images from Rafael C. Gonzalez and Richard E.
Wood, Digital Image Processing, 2nd Edition.
Noise Models
Noise cannot be predicted but can be approximately described in
statistical way using the probability density function (PDF)
Gaussian noise:
1 ( z ) 2 / 2 2
p( z ) e
2
Rayleigh noise
2 ( z a ) 2 / b
( z a )e for z a
p( z ) b
0 for z a
Erlang (Gamma) noise

a b z b1
( z a )e az for z 0
p( z ) (b 1)!

0 for z 0
Noise Models (cont.)
Exponential noise
p( z ) ae az
Uniform noise
1
for a z b
p( z ) b - a
0 otherwise
Impulse (salt & pepper) noise
Pa for z a

p( z ) Pb for z b
0
otherwise
PDF: Statistical Way to Describe Noise
PDF tells how much

each z value occurs.
(Images from Rafael C. Gonzalez and Richard E.

Image Degradation with Additive Noise
g ( x, y ) f ( x, y ) ( x, y )
Degraded images
Original image
Histogram

Image Degradation with Additive Noise (cont.)
g ( x, y ) f ( x, y ) ( x, y )
Degraded images
Original image
Histogram

Periodic Noise
Periodic noise
looks like dots
In the frequency
domain
Estimation of Noise
We cannot use the image

histogram to estimate
noise PDF.
It is better to use the

histogram of one area
of an image that has
constant intensity to
estimate noise PDF.

Periodic Noise Reduction by Freq. Domain Filtering
Degraded image DFT
Periodic noise
can be reduced by
setting frequency
components
corresponding to
noise to zero.
Band reject filter Restored image

Band Reject Filters
Use to eliminate frequency components in some bands
Periodic noise from the

previous slide that is
Filtered out.

Notch Reject Filters
A notch reject filter is used to eliminate some frequency components.

Notch Reject Filter:
Notch filter
Degraded image DFT (freq. Domain)

Noise Restored image
Example: Image Degraded by Periodic Noise
Degraded image
DFT
(no shift)
DFT of noise Noise Restored image

Mean Filters
Degradation model:
g ( x, y ) f ( x, y ) h ( x, y ) ( x, y )
To remove this part
Arithmetic mean filter or moving average filter (from Chapter 3)
f ( x, y ) 1
g ( s, t )
mn ( s ,t )S xy
Geometric mean filter mn = size of moving window
1
mn
f ( x, y ) g ( s, t )
( s ,t )S
xy
Geometric Mean Filter: Example
Image
Original
corrupted
image
by AWGN
Image Image
obtained obtained
using a 3x3 using a 3x3
arithmetic geometric
mean filter mean filter
AWGN: Additive White Gaussian Noise (Images from Rafael C. Gonzalez and Richard E.
Harmonic and Contraharmonic Filters
Harmonic mean filter
mn
f ( x, y ) Works well for salt noise
1

( s ,t )S xy g ( s, t )
but fails for pepper noise
Contraharmonic mean filter mn = size of moving window
g ( s , t ) Q 1
Positive Q is suitable for
f ( x, y )
( s ,t )S xy eliminating pepper noise.
g ( s ,
( s ,t )S xy
t ) Q Negative Q is suitable for
eliminating salt noise.
Q = the filter order

For Q = 0, the filter reduces to an arithmetic mean filter.
For Q = -1, the filter reduces to a harmonic mean filter.
Contraharmonic Filters: Example
Image Image
corrupted corrupted
by pepper by salt
noise with noise with
prob. = 0.1 prob. = 0.1
Image Image
obtained obtained
contra- contra-
harmonic harmonic
With Q = 1.5 With Q=-1.5

Contraharmonic Filters: Incorrect Use Example
Image Image
corrupted corrupted
by pepper by salt
prob. = 0.1 prob. = 0.1
Image Image
obtained obtained
contra- contra-
harmonic harmonic
With Q=-1.5 With Q=1.5

Order-Statistic Filters: Revisit
Original image
subimage
Statistic parameters
Mean, Median, Mode,
Min, Max, Etc.
Moving
window
Output image
Order-Statistics Filters
Median filter
f ( x, y ) median g ( s, t )
( s ,t )S xy
Max filter
f ( x, y ) max g ( s, t ) Reduce dark noise

( s ,t )S xy
(pepper noise)
Min filter
f ( x, y ) min g ( s, t ) Reduce bright noise

( s ,t )S xy (salt noise)
Midpoint filter
f ( x, y ) 1 max g ( s, t ) min g ( s, t )
2 ( s ,t )S xy ( s ,t )S xy
Median Filter : How it works
A median filter is good for removing impulse, isolated noise
Salt noise
Pepper noise
Median
Sorted
Degraded image Moving array
window
Salt noise Filter output
Pepper noise
Normally, impulse noise has high magnitude
and is isolated. When we sort pixels in the
moving window, noise pixels are usually
at the ends of the array.
Therefore, its rare that the noise pixel will be a median value.
Median Filter : Example
1 2
Image
corrupted
by salt-
and-pepper
noise with
pa=pb= 0.1
3 4
Images obtained using a 3x3 median (Images

filter from Rafael C. Gonzalez and Richard E.
Max and Min Filters: Example
Image Image
corrupted corrupted
by pepper by salt
prob. = 0.1 prob. = 0.1
Image Image
obtained obtained
max filter min filter

Alpha-trimmed Mean Filter
Formula:
1
f ( x, y )
mn d
g ( s, t )
( s ,t )S xy
r
where gr(s,t) represent the remaining mn-d pixels after

removing the d/2 highest and d/2 lowest values of g(s,t).
This filter is useful in situations involving multiple types

of noise such as a combination of salt-and-pepper and
Gaussian noise.
Alpha-trimmed Mean Filter: Example
1 2
Image
additionally
Image corrupted
corrupted by additive
by additive salt-and-
uniform pepper
noise noise
Image 2 Image 2
obtained obtained

Alpha-trimmed Mean Filter: Example (cont.)
1 2
Image
additionally
Image corrupted
corrupted by additive
by additive salt-and-
uniform pepper
noise noise
Image 2
Image 2 obtained
obtained using a 5x5
using a 5x5 alpha-
median filter trimmed
mean filter
with d = 5
Alpha-trimmed Mean Filter: Example (cont.)
Image Image
obtained obtained
Image
Image obtained
using a 5x5 alpha-
median filter trimmed
mean filter
with d = 5
Adaptive Filter
General concept:
-Filter behavior depends on statistical characteristics of local areas
inside mxn moving window
- More complex but superior performance compared with fixed
filters
Statistical characteristics:
Local mean: Noise variance:
1
mL 2
g ( s, t )
mn ( s ,t )S xy
Local variance:
1

2
L (
mn ( s ,t )S xy
g ( s , t ) mL ) 2
Adaptive, Local Noise Reduction Filter
Purpose: want to preserve edges
Concept:
1. If 2 is zero, No noise
the filter should return g(x,y) because g(x,y) = f(x,y)
2. If L2 is high relative to 2, Edges (should be preserved),

the filter should return the value close to g(x,y)
3. If L2 = 2, Areas inside objects

the filter should return the arithmetic mean value mL
Formula:
2
f ( x, y ) g ( x, y ) g ( x, y ) m
L2 L
Adaptive Noise Reduction Filter: Example
Image Image
corrupted obtained
by additive using a 7x7
Gaussian arithmetic
noise with mean filter
zero mean
and 2=1000
Image
Image obtained
using a 7x7 adaptive
geometric noise
mean filter reduction
filter

Adaptive Median Filter
Purpose: want to remove impulse noise while preserving edges
Algorithm:Level A: A1= zmedian zmin
A2= zmedian zmax
If A1 > 0 and A2 < 0, goto level B
Else increase window size
If window size <= Smax repeat level A
Else return zxy
Level B: B1= zxy zmin
B2= zxy zmax
If B1 > 0 and B2 < 0, return zxy
Else return zmedian
where zmin = minimum gray level value in Sxy
zmax = maximum gray level value in Sxy
zmedian = median of gray levels in Sxy
zxy = gray level value at pixel (x,y)
Smax = maximum allowed size of Sxy
Adaptive Median Filter: How it works
Level A: A1= zmedian zmin Determine
A2= zmedian zmax whether zmedian
If A1 > 0 and A2 < 0, goto level B is an impulse or not
Else Window is not big enough
increase window size
If window size <= Smax repeat level A
Else return zxy
Level B: zmedian is not an impulse
Determine
B1= zxy zmin whether zxy
B2= zxy zmax is an impulse or not
If B1 > 0 and B2 < 0, zxy is not an impulse
return zxy to preserve original details
Else
return zmedian to remove impulse
Adaptive Median Filter: Example
Image corrupted Image obtained Image obtained

by salt-and-pepper using a 7x7 using an adaptive
noise with median filter median filter with
pa=pb= 0.25 Smax = 7
More small details are preserved

Estimation of Degradation Model
Degradation model:
g ( x, y ) f ( x, y ) h ( x, y ) ( x, y )
or
G ( u, v ) F ( u, v ) H ( u, v ) N ( u, v )
Purpose: to estimate h(x,y) or H(u,v)

Why? If we know exactly h(x,y), regardless of noise, we can do
deconvolution to get f(x,y) back from g(x,y).
Methods:
1. Estimation by Image Observation
2. Estimation by Experiment
3. Estimation by Modeling (Images from Rafael C. Gonzalez and Richard E.

Estimation by Image Observation
Original image (unknown) Degraded image
f(x,y) f(x,y)*h(x,y) g(x,y)
Observation
DFT Subimage
Estimated Transfer Gs (u, v ) g s ( x, y )
function
G ( u, v ) Restoration
H ( u, v ) H s ( u, v ) s process by
Fs (u, v ) estimation
DFT Reconstructed
This case is used when we
Fs (u, v) Subimage
know only g(x,y) and cannot
repeat the experiment! fs ( x, y )
Estimation by Experiment
Used when we have the same equipment set up and can repeat the
experiment.
Response image from
Input impulse image the system
System
H( )
A ( x, y ) g ( x, y )
DFT DFT
DFT A ( x, y) A G ( u, v )
G (u, v )
H (u, v )
A (Images from Rafael C. Gonzalez and Richard E.
Estimation by Modeling
Used when we know physical mechanism underlying the image
formation process that can be expressed mathematically.
Original image Severe turbulence Example:
Atmospheric
Turbulence model
k ( u2 v 2 )5 / 6
H (u, v) e
k = 0.0025
Mild turbulence Low turbulence
k = 0.001 k = 0.00025
Estimation by Modeling: Motion Blurring
Assume that camera velocity is ( x0 (t ), y0 (t ))
The blurred image is obtained by
T
g ( x, y ) f ( x x0 (t ), y y0 (t )) dt
0
where T = exposure time.

j 2 ( ux vy )
G ( u, v ) g ( x , y ) e dxdy

T

j 2 ( ux vy )
f ( x x0 (t ), y y0 (t )) dt e dxdy
0

T

f ( x x0 (t ), y y0 (t )) e j 2 ( ux vy )
dxdydt
0
Estimation by Modeling: Motion Blurring (cont.)

T

G (u, v ) f ( x x0 (t ), y y0 (t )) e j 2 ( ux vy )
dxdydt
0

T
F (u, v )e j 2 ( ux0 ( t )vy0 ( t )) dt
0
T
F (u, v ) e j 2 ( ux0 ( t )vy0 ( t )) dt
0
Then we get, the motion blurring transfer function:

T
H (u, v) e j 2 ( ux0 ( t )vy0 ( t )) dt
0
For constant motion ( x0 (t ), y0 (t )) (at, bt )

T
T
H (u, v ) e j 2 ( ua vb )
dt sin( (ua vb)) e j ( uavb )
0
(ua vb)
Motion Blurring Example
For constant motion
T
H ( u, v ) sin( (ua vb)) e j ( uavb )
(ua vb)
Original image Motion blurred image

a = b = 0.1, T = 1
Inverse Filter
From degradation model:
G ( u, v ) F ( u, v ) H ( u, v ) N ( u, v )
after we obtain H(u,v), we can estimate F(u,v) by the inverse filter:

G ( u, v ) N ( u, v )
F (u, v ) F ( u, v )
H ( u, v ) H ( u, v )
Noise is enhanced
when H(u,v) is small.
To avoid the side effect of enhancing
noise, we can apply this formulation
to freq. component (u,v) with in a
radius D0 from the center of H(u,v).
In practical, the inverse filter is not
Popularly used. (Images from Rafael C. Gonzalez and Richard E.
Inverse Filter: Example
Result of applying Result of applying

Original image the full filter the filter with D0=40
Blurred image Result of applying Result of applying

Due to Turbulence the filter with D0=70 the filter with D0=85
0.0025( u2 v 2 )5 / 6
H (u, v) e
Wiener Filter: Minimum Mean Square Error Filter
Objective: optimize mean square error: e2 E ( f f )2
Wiener Filter Formula:
H *
( u , v ) S ( u , v )
F (u, v ) f
G ( u , v )
S f (u, v ) H (u, v ) S (u, v )
2
H * ( u, v )
G ( u , v )
H (u, v ) S (u, v ) / S f (u, v )
2
1 H ( u, v )
2

G ( u , v )
H (u, v ) H (u, v ) S (u, v ) / S f (u, v )
2
where
H(u,v) = Degradation function
S(u,v) = Power spectrum of noise
Sf(u,v) = Power spectrum of the undegraded image
Approximation of Wiener Filter
Wiener Filter Formula:
1 H ( u , v )
2

F (u, v ) G(u, v )
H (u, v ) H (u, v ) S (u, v ) / S f (u, v )
2
Difficult to estimate
Approximated Formula:
1 H ( u , v )
2

F (u, v ) G ( u , v )
H (u, v ) H (u, v ) K
2
Practically, K is chosen manually to obtained the best visual result!

Wiener Filter: Example
Result of the Result of the inverse

Original image full inverse filter filter with D0=70
Blurred image Result of the

Due to Turbulence full Wiener filter
Wiener Filter: Example (cont.)
Result of the inverse

Original image filter with D0=70
Blurred image Result of the

Due to Turbulence Wiener filter
Example: Wiener Filter and Motion Blurring
Image Result of the Result of the
degraded inverse filter Wiener filter
by motion
blur +
AWGN 2=650
2=325
Note: K is
chosen
manually
2=130 (Images from Rafael C. Gonzalez and Richard E.

Constrained Least Squares Filter
Degradation model: Written in a matrix form
g ( x, y ) f ( x, y ) h ( x, y ) ( x, y ) g Hf
Objective: to find the minimum of a criterion function

M 1 N 1
C f ( x, y ) 2 2
x 0 y 0
Subject to the constraint

2
g Hf w wT w
2 2
where
We get a constrained least square filter
H *
( u , v )
F (u, v ) 2
G ( u, v )
H (u, v ) P(u, v )
2
where 0 1 0
1 4 1
P(u,v) = Fourier transform of p(x,y) =
0 1 0
Constrained Least Squares Filter: Example
Constrained least square filter
H *
( u , v )
F (u, v ) 2
G ( u, v )
H (u, v ) P(u, v )
2
is adaptively adjusted to achieve the best result.
Results from the previous slide obtained

from the constrained least square filter
Constrained Least Squares Filter: Example (cont.)
Image Result of the Result of the
degraded Constrained Wiener filter
by motion Least square
blur + filter
AWGN 2=650
2=325
2=130
Constrained Least Squares Filter:Adjusting
It can be shown that ( ) r r r

2
Define r g Hf
T
r a
2 2
We want to adjust gamma so that 1
1. Specify an initial value of where a = accuracy factor

2
2. Compute r
3. Stop if 1 is satisfied
Otherwise return step 2 after increasing if r a
2 2
or decreasing if r a
2 2
Use the new value of to recompute

H *
( u , v )
F (u, v ) 2
G ( u, v )
H (u, v ) P(u, v )
2
Constrained Least Squares Filter:Adjusting (cont.)
H *
( u , v )
F (u, v ) 2
G ( u, v )
H (u, v ) P(u, v )
2
2
R(u, v ) G (u, v ) H (u, v ) F (u, v ) For computing r
M 1 N 1
1
r ( x, y)
2 2
r
MN x 0 y 0
M 1 N 1
1
m
MN
( x, y)
x 0 y 0
M 1 N 1
( x, y) m
2

2 1 2 For computing

MN x 0 y 0

MN 2 m
2

Constrained Least Squares Filter: Example
Use correct noise
Original image Correct parameters:
parameters
Initial = 10-5
Correction factor = 10-6
a = 0.25
2 = 10-5
Blurred image Use wrong noise Wrong noise parameter

Due to Turbulence parameters
2 = 10-2
(Images from Rafael C.

Gonzalez and Richard E.
Wood, Digital Image
Processing, 2nd Edition.
Results obtained from constrained least square filters

Geometric Mean filter
This filter represents a family of filters combined into a
single expression
1

H ( u, v )
* *
H ( u, v )

F (u, v ) G (u, v )
2 S (u, v )
H ( u , v )
H ( u, v )
2
S f (u, v )
= 1 the inverse filter

= 0 the Parametric Wiener filter
= 0, = 1 the standard Wiener filter
= 1, < 0.5 More like the inverse filter
= 1, > 0.5 More like the Wiener filter
Another name: the spectrum equalization filter

Geometric Transformation
These transformations are often called rubber-sheet transformations:
Printing an image on a rubber sheet and then stretch this sheet according
to some predefine set of rules.
A geometric transformation consists of 2 basic operations:

1. A spatial transformation :
Define how pixels are to be rearranged in the spatially
transformed image.
2. Gray level interpolation :
Assign gray level values to pixels in the spatially
transformed image.
Geometric Transformation : Algorithm
( x, y ) ( x , y )
Image f to be 3 Distorted image g

1
restored
1. Select coordinate (x,y) in f to be restored

2. Compute
x r ( x, y ) 4. get pixel value at g ( x, y )
By gray level interpolation
y s( x, y )
5. store that value in pixel f(x,y)
3. Go to pixel ( x , y )
in a distorted image g
5
Spatial Transformation
To map between pixel coordinate (x,y) of f and pixel coordinate
(x,y) of g
x r ( x, y ) y s( x, y )
For a bilinear transformation mapping between a pair of

Quadrilateral regions
x r( x, y ) c1 x c2 y c3 xy c4
y s( x, y ) c5 x c6 y c7 xy c8 ( x , y ) ( x, y )
To obtain r(x,y) and s(x,y), we need

to know 4 pairs of coordinates
( x, y ) and its corresponding ( x , y )
which are called tiepoints.
Gray Level Interpolation: Nearest Neighbor
Since ( x , y ) may not be at an integer coordinate, we need to
Interpolate the value of g ( x, y )
Example interpolation methods that can be used:

1. Nearest neighbor selection
2. Bilinear interpolation
3. Bicubic interpolation

Geometric Distortion and Restoration Example
Original image and Tiepoints of distorted
tiepoints image
Distorted image Restored image
Use nearest
neighbor
intepolation
Wood, Digital Image
Geometric Distortion and Restoration Example (cont.)
Original image and Tiepoints of distorted
tiepoints image
Distorted image Restored image
Use bilinear
intepolation
Wood, Digital Image
Example: Geometric Restoration
Original image Geometrically distorted
image
Use the same
Spatial Trans.
as in the previous
example
Restored image
Difference between (Images from Rafael C.

2 above images Wood, Digital Image

Ravi Im Rest

Încărcat de

Informații document

Descriere originală:

Drepturi de autor

Formate disponibile

Partajați acest document

Partajați sau inserați document

Opțiuni de partajare

Vi se pare util acest document?

Este necorespunzător acest conținut?

Drepturi de autor:

Formate disponibile

Ravi Im Rest

Încărcat de

Drepturi de autor:

Formate disponibile

Digital Image Processing

Image restoration is to restore a degraded image back to

where h(x,y) is a system that causes image distortion and

Erlang (Gamma) noise

Impulse (salt & pepper) noise

PDF tells how much

(Images from Rafael C. Gonzalez and Richard E.

(Images from Rafael C. Gonzalez and Richard E.

(Images from Rafael C. Gonzalez and Richard E.

We cannot use the image

It is better to use the

(Images from Rafael C. Gonzalez and Richard E.

Band reject filter Restored image

Periodic noise from the

(Images from Rafael C. Gonzalez and Richard E.

(Images from Rafael C. Gonzalez and Richard E.

(Images from Rafael C. Gonzalez and Richard E.

DFT of noise Noise Restored image

To remove this part

Arithmetic mean filter or moving average filter (from Chapter 3)

Contraharmonic mean filter mn = size of moving window

Q = the filter order

(Images from Rafael C. Gonzalez and Richard E.

(Images from Rafael C. Gonzalez and Richard E.

f ( x, y ) max g ( s, t ) Reduce dark noise

f ( x, y ) min g ( s, t ) Reduce bright noise

Images obtained using a 3x3 median (Images

(Images from Rafael C. Gonzalez and Richard E.

where gr(s,t) represent the remaining mn-d pixels after

This filter is useful in situations involving multiple types

(Images from Rafael C. Gonzalez and Richard E.

2. If L2 is high relative to 2, Edges (should be preserved),

3. If L2 = 2, Areas inside objects

(Images from Rafael C. Gonzalez and Richard E.

Image corrupted Image obtained Image obtained

More small details are preserved

(Images from Rafael C. Gonzalez and Richard E.

Purpose: to estimate h(x,y) or H(u,v)

3. Estimation by Modeling (Images from Rafael C. Gonzalez and Richard E.

f(x,y) f(x,y)*h(x,y) g(x,y)

Then we get, the motion blurring transfer function:

For constant motion ( x0 (t ), y0 (t )) (at, bt )

Original image Motion blurred image

after we obtain H(u,v), we can estimate F(u,v) by the inverse filter:

Result of applying Result of applying

Blurred image Result of applying Result of applying

Practically, K is chosen manually to obtained the best visual result!

Result of the Result of the inverse

Blurred image Result of the

Result of the inverse

Blurred image Result of the

2=130 (Images from Rafael C. Gonzalez and Richard E.

Subject to the constraint

is adaptively adjusted to achieve the best result.

Results from the previous slide obtained

It can be shown that ( ) r r r

1. Specify an initial value of where a = accuracy factor

Use the new value of to recompute

Blurred image Use wrong noise Wrong noise parameter

(Images from Rafael C.

Results obtained from constrained least square filters

= 1 the inverse filter

Another name: the spectrum equalization filter