The University of

CS 4487/9587

Algorithms for Image Analysis

Ontario

Image Processing Basics

Acknowledgements: slides from Steven Seitz, Aleosha Efros, David Forsyth, and Gonzalez & Woods

A crash course on

The University of

CS 4487/9587 Algorithms for Image Analysis

Image Processing Basics

Ontario

Point Processing
gamma correction window-center correction histogram equalization

Filtering
mean, Gaussian, and median filters image gradients
Extra Reading: Gonzales and Woods, Sec.3

The University of

Image processing

Ontario

An image processing operation typically defines a new image g in terms of an existing image f.

Examples:

The University of

Image processing

Ontario

An image processing operation typically defines a new image g in terms of an existing image f.

Examples:
We can transform either the domain or the range of f.

g ( x, y ) f (t x ( x, y ), t y ( x, y )) Domain transformation: What kinds of operations can this perform?

The University of

Image processing

Ontario

An image processing operation typically defines a new image g in terms of an existing image f.

Examples:
We can transform either the domain or the range of f.

g ( x, y ) f (t x ( x, y ), t y ( x, y )) Domain transformation: What kinds of operations can this perform?

g ( x, y ) t x , y ( f Range transformation: What kinds of operations can this perform?

( x, y ))

The University of

Image processing

Ontario

An image processing operation typically defines a new image g in terms of an existing image f.

Examples:
We can transform either the domain or the range of f.

g ( x, y ) f (t x ( x, y ), t y ( x, y )) Domain transformation: What kinds of operations can this perform?

g ( x, y ) t x , y ( f Range transformation: What kinds of operations can this perform?

( x, y ))

Filtering also generates new images from an existing image

g ( x, y )

|u | |v|

h(u, v) f ( x u, y v) du dv
more on filtering later in the notes

The University of

Point Processing

Ontario

The simplest kind of range transformations are these independent of position (x,y) : g ( x, y ) t ( f ( x, y ))
for each original image intensity value I, function t() returns a transformed intensity value t(I).

This is called point processing. Important: every pixel is for himself spatial information is ignored! What can point processing do?

The University of

Point Processing:

Examples of gray-scale transforms t

Ontario

The University of

Point Processing:

Negative

Ontario

f ( x, y)

g ( x, y )

t(I )

255 I

g ( x, y) t ( f ( x, y)) 255 f ( x, y)

The University of

Point Processing:

Power-law transformations t

Ontario

The University of

Point Processing:

Gamma Correction

Ontario

Gamma Measuring Applet: http://www.cs.berkeley.edu/~efros/java/gamma/gamma.html

The University of

Point Processing:

Enhancing Image via Gamma Correction

Ontario

The University of

Point Processing:

Understanding Image Histograms

Ontario

Image Brightness

Image Contrast

probability of intensity i :

p(i )

ni n

---number of pixels with intensity i ---total number of pixels in the image

The University of

Point Processing:

Contrast Stretching

Ontario

The University of

Point Processing:

Contrast Stretching
Original images Histogram corrected images

Ontario

1)

2)

The University of

Point Processing:

Contrast Stretching
Original images Histogram corrected images

Ontario

3)

4)

The University of

One way to automatically select transformation t :

Histogram Equalization

Ontario

t (i )
j 0

p( j )
j 0

nj n

= cumulative histogram of image intensities

see Gonzalez and Woods, Sec3.3.1, for more details

The University of

Point processing

Histogram Equalization
i i

Ontario

t (i )
j 0

p( j )
j 0

nj n

= cumulative histogram of image intensities

Why does that work? Answer in probability theory:

I random variable with probability distribution p(i) over i in [0,1]

If t(i) is a cumulative distribution of I then

I=t(I) is a random variable with uniform distribution over its range [0,1]
That is, transform image I will have a uniformly-spread histogram (good contrast)

The University of

Point Processing:

Window-Center adjustment

Ontario

(monitors dynamic range)

Output gray level

256

(high dynamic range image)

input gray level

60000

The University of

Point Processing:

Window-Center adjustment

Ontario

(monitors dynamic range)

Output gray level

256

(high dynamic range image)

input gray level

60000

The University of

Point Processing:

Window-Center adjustment
256

Ontario

output gray level

window

center

60000

input gray level

The University of

Point Processing:

Window-Center adjustment

Ontario

Window = 4000 Center = 500

The University of

Point Processing:

Window-Center adjustment

Ontario

Window = 2000 Center = 500

The University of

Point Processing:

Window-Center adjustment

Ontario

Window = 800 Center = 500

The University of

Point Processing:

Window-Center adjustment

Ontario

Window = 0 Center = 500

If window=0 then we get binary image thresholding

The University of

Point Processing:

Window-Center adjustment

Ontario

Window = 800 Center = 500

Window = 800 Center = 1160

The University of

Neighborhood Processing (or filtering)

Ontario

Q: What happens if I reshuffle all pixels within the image?

A: Its histogram wont change. No point processing will be affected Images contain a lot of spatial information

Readings: Forsyth & Ponce, chapters 8.1-8.2

The University of

Neighborhood Processing (filtering)

Linear image transforms

Lets start with 1D image (a signal): f[i]

Ontario

A very general and useful class of transforms are the linear transforms of f, defined by a matrix M

The University of

Neighborhood Processing (filtering)

Linear image transforms

Lets start with 1D image (a signal): f[i]

Ontario

matrix M

The University of

Neighborhood Processing (filtering)

Linear image transforms

Lets start with 1D image (a signal): f[i]

Ontario

matrix M

The University of

Neighborhood Processing (filtering)

Linear shift-invariant filters

matrix M

Ontario

It is known as a linear shift-invariant filter and is represented by a kernel (or mask) h:

h [i]
k

[ a b c]

and can be written (for kernel of size 2k+1) as:

g [i]
u k

h [u ] f [i u ]

M f

The above allows negative filter indices. When you implement need to use: h[u+k] instead of h[u]

The University of

Neighborhood Processing (filtering)

2D linear transforms

Ontario

f [i]

f [ i / m , i%m ]

The University of

Neighborhood Processing (filtering)

2D filtering

Ontario

A 2D image f[i,j] can be filtered by a 2D kernel h[u,v] to produce an output image g[i,j]:
k k

g [i, j ]
u k v k

h [u, v] f [i u, j v]

This is called a cross-correlation operation and written:

h f

h is called the filter, kernel, or mask.

The University of

Neighborhood Processing (filtering)

2D filtering (see more in Gonzalez and Woods, e.g. Sec. 4.6.4)

Ontario

A convolution operation is a cross-correlation where the filter is flipped both horizontally and vertically before being applied to the image:
k k

g [i, j ]
u k v k

h [u, v] f [i u, j v]

It is written:

h f
u

k k v

h[ u, v] f [i u, j v]
k

Suppose h is a Gaussian or mean kernel. How does convolution differ from cross-correlation?
If

h[u, v]

h[ u, v] then there is no difference between convolution and cross-correlation

The University of

2D filtering

Noise
Filtering is useful for noise reduction...
(side effects: blurring)
Common types of noise: Salt and pepper noise: random occurrences of black and white pixels Impulse noise: random occurrences of white pixels Gaussian noise: variations in intensity drawn from a Gaussian normal distribution

Ontario

The University of

Practical noise reduction

Ontario

How can we smooth away noise in a single image?

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 110 0 0 90 0 0 100 90 80 0 0 0 90 0 0 100 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

100 130 110 120 110 130 100 90 0 110 0 130 110 120 100 0 0

120 100 130 110 120

0
0

0
0

0
0

0
0

0
0

0
0

0
0

0
0

0
0

0
0

The University of

Neighborhood Processing (filtering)

Mean filtering
0 0 0 0 0 0 0 0 0 0

Ontario

0
0 0

0
0 0

0
0 0

0 90 90 90 90 90 0 0 90 90 90 90 90 0

0
0 0

0
0 0

0 90 90 90 90 90 0
0 90 0 90 90 90 0 0 90 90 90 90 90 0

0
0 0

0
0 0

0
0

0
0

0
0

0
0 0

0
0 0

0
0 0

0
0 0

0
0 0

0
0 0

0 90 0

The University of

Neighborhood Processing (filtering)

Mean filtering
0 0 0 0 0 0 0 0 0 0

Ontario

0
0 0

0
0 0

0
0 0

0 10 20 30 30 30 20 10 0 20 40 60 60 60 40 20 0 30 60 90 90 90 60 30 0 30 50 80 80 90 60 30 0 30 50 80 80 90 60 30 0 20 30 50 50 60 40 20 10 20 30 30 30 30 20 10 10 10 10 0 0 0 0 0

0 90 90 90 90 90 0 0 90 90 90 90 90 0

0
0 0

0
0 0

0 90 90 90 90 90 0
0 90 0 90 90 90 0 0 90 90 90 90 90 0

0
0 0

0
0 0

0
0

0
0

0
0

0
0 0

0
0 0

0
0 0

0
0 0

0
0 0

0
0 0

0 90 0

The University of

Effect of mean filters

Ontario

The University of

Neighborhood Processing (filtering)

Mean kernel

Ontario

Whats the kernel for a 3x3 mean filter?

0 0 0 0 0 0 0 0 0

0
0 0

0
0 0

0
0 0

0 90 90 90 90 90 0 0 90 90 90 90 90 0

0
0 0

0
0 0

0 90 90 90 90 90 0
0 90 0 90 90 90 0 0 90 90 90 90 90 0

0
0 0

H [u, v]

0
0 0

0
0

0
0

0
0

0
0 0

0
0 0

0
0 0

0
0 0

0
0 0

0
0 0

0 90 0

F [ x, y ]

The University of

Neighborhood Processing (filtering)

Gaussian Filtering

Ontario

A Gaussian kernel gives less weight to pixels 0 0 0 0 0 0 0 0 0 0 further from the center 0 0 0 0 0 0 0 0 0 0 of the window
0 0 0 0 0 90 90 90 90 90 0 0 90 90 90 90 90 0 0 0

0
0 0

0
0 0

0 90 90 90 90 90 0
0 90 0 90 90 90 0 0 90 90 90 90 90 0

0
0 0

1 16

1 2 1

2 4 2
1 2 1

H [u, v]

0
0 0

0
0

0
0

0
0

0
0 0

0
0 0

0
0 0

0
0 0

0
0 0

0
0 0 This kernel is an approximation of a Gaussian function:

0 90 0

F [ x, y ]

The University of

Neighborhood Processing (filtering)

Mean vs. Gaussian filtering

Ontario

The University of

Neighborhood Processing (filtering)

Median filters

Ontario

A Median Filter operates over a window by selecting the median intensity in the window. What advantage does a median filter have over a mean filter?

Is a median filter a kind of convolution?

- No, median filter is an example of non-linear filtering

Comparison: salt and pepper noise

The University of

Ontario

Comparison: Gaussian noise

The University of

Ontario

The University of

Reading: Forsyth & Ponce, 8.1-8.2

Differentiation and convolution

Ontario

Recall

We could approximate this as

lim
0

f (x

, y) f ( x, y)
x

f ( xi 1 , y) f ( xi 1 , y) 2 x
x

Now this is linear and shift invariant, so must be the result of a convolution.

f (convolution)
1 2 x

0 0 0 1 0 -1 0 0 0
x

with kernel

[u, v]

sometimes this may not be a very good way to do things, as we shall see

The University of

Reading: Forsyth & Ponce, 8.1-8.2

Differentiation and convolution

Ontario

Recall

We could approximate this as

lim
0

f (x

, y) f ( x, y)
x

f ( xi 1 , y) f ( xi 1 , y) 2 x
x

Now this is linear and shift invariant, so must be the result of a convolution.

f (convolution)
1 2 x

0 0 0 1 0 -1 0 0 0
x

with kernel

[u, v]

sometimes this may not be a very good way to do things, as we shall see

The University of

Finite differences

Ontario

The University of

Finite differences responding to noise

Ontario

The University of

Finite differences and noise

Ontario

Finite difference filters respond strongly to noise

obvious reason: image noise results in pixels that look very different from their neighbours

What is to be done?
intuitively, most pixels in images look quite a lot like their neighbours this is true even at an edge; along the edge theyre similar, across the edge theyre not suggests that smoothing the image should help, by forcing pixels different to their neighbours (=noise pixels?) to look more like neighbours

Generally, the larger the noise the stronger the response

The University of

Smoothing and Differentiation

Ontario

Issue: noise smooth before differentiation two convolutions: smooth, and then differentiate? actually, no - we can use a derivative of Gaussian filter because differentiation is convolution, and convolution is associative x ( H f ) ( x H ) f

The University of

H) f
Ontario

The University of

Sobol derivative kernels

Ontario

f
1 8 y

1 8 x

1 0 -1

1 2 1
0 0 0 -1 -2 -1
y

2 0 -2
1 0 -1
x

[u, v]

[u, v]

The University of

Image Gradients

Ontario

Recall for a function of two (or more) variables f ( x, y)

f x f y
x y

x
f x

Gradient at point (x,y)

f f

y
f y

a two (or more) dimensional vector

The absolute value | f | is large at image boundaries

( fx ) 2 ( fy ) 2

f )2 (

f )2

small image gradients in low textured areas

The direction of the gradient corresponds to the direction of the steepest ascend - normally gradient is orthogonal to object boundaries in the image.

The University of

Vector f is independent of specific choice of coordinate system

Ontario

Equivalently, gradient of function f ( p) 2 at point p R can be defined as a vector f s.t. for any unit vector n

Gradient at point p

( f n)

f n

f (p

n ) f ( p)

dot product

directional derivative of function f along direction n

Pure vector algebra, specific coordinate system is irrelevant Works for functions of 2-, 3-, or larger number of variables Previous slide gives you a specific way for computing coordinates of vector

( , )

f x

f y

w.r.t. given orthogonal basis (axis X and Y).

The University of

Image Gradient

Ontario

Typical application where image gradients are used is image edge detection
find points with large image gradients

Canny edge detector suppresses non-extrema Gradient points

The University of

Second Image Derivatives

(Laplace operator
2 2

f)

Ontario

f
2

f
2

x y

f
divergence of gradient

rotationally invariant second derivative for 2D functions

0 0 0 -1 2 -1 0 0 0

0 -1 0 0 2 0

0 -1 0 -1 4 -1 0 -1 0

0 -1 0

The University of

Second Image Derivatives

(Laplace operator
2 2

f)

Ontario

f
2

f
2

x y

f
divergence of gradient

Application: Laplacian Zero Crossings are used for edge detection

(alternative to methods computing Gradient extrema)
image intensity along some scan line magnitude of image gradient along the same scan line

http://homepages.inf.ed.ac.uk/rbf/HIPR2/zeros.htm

Laplacian (2nd derivative) of the image along the same scan line

The University of

Laplacian of a Gaussian (LoG)

Ontario

G
LoG( x, y )

image should be smoothed a bit first

LoG
1
4

y
2

x2 y2 2 2

fspecial

http://homepages.inf.ed.ac.uk/rbf/HIPR2/log.htm

The University of

Laplacian of a Gaussian (LoG)

Ontario

LoG

image should be smoothed a bit first

http://homepages.inf.ed.ac.uk/rbf/HIPR2/log.htm

The University of

Unsharp masking

Ontario

I G*I

=
unsharp mask

+
unsharp mask

The University of

Unsharp masking

Ontario

(1

)I

G*I

[(1

)G

1
1 2

G 2] *I

I G*I

unsharp mask I U

+
unsharp mask

The University of

Unsharp masking
MATLAB
Imrgb = imread(file.jpg); im = im2double(rgb2gray(imrgb)); g= fspecial('gaussian', 25,4); imblur = conv2(im,g,same'); imagesc([im imblur]) imagesc([im im+.4*(im-imblur)])

Ontario

unsharp mask kernel can be seen as a difference of two Gaussians (DoG) with 1 2 .

unsharp mask kernel

http://homepages.inf.ed.ac.uk/rbf/HIPR2/unsharp.htm

DoG

G1 G

LoG

The University of

Reading: Forsyth & Ponce ch.7.5

Filters and Templates

Ontario

Applying a filter at some point can be seen as taking a dot-product between the image and some vector Filtering the image is a set of dot products

Insight
filters may look like the effects they are intended to find filters find effects they look like

The University of

Normalized cross-correlation

Ontario

Think of filters of a dot product

now measure the angle i.e normalized correlation output is filter output, divided by root sum of squares of values over which filter lies

Tricks:
ensure that filter has a zero response to a constant region (helps reduce response to irrelevant background) subtract image average when computing the normalizing constant (i.e. subtract the image mean in the neighbourhood) absolute value deals with contrast reversal

The University of

Ontario

The University of

Ontario