Segmentation

Segmentation subdivides an image into its

constituent regions or objects
• Segmentation is based on two basic
properties of gray-level values:

– Discontinuity, i.e. to partition the image

based on abrupt changes in intensity (gray
levels), e.g. edges

– Similarity, i.e. to partition the image into

similar (according to predefined criteria)
regions, e.g. thresholding, region growing,
region splitting and merging
 Detection of Discontinuities
• 3 basic types of gray-level
discontinuities:

Points
Lines
Edges

• Common method of detection: run a

mask through the image.
Averaging leads to blurring effect

Averaging analogous to INTEGRATION

Hence, deblurring or sharpening

 DIFFERENTIATION
or DISCONTINUITY
Image differentiation enhances edges

and other discontinuities (like noise)

and deemphasizes area with slowly

varying gray levels

1st order derivative of 1-D function:

f
 f ( x  1)  f ( x ) ; x is discrete
x
 Gray level

Position
Properties::

a) Must be zero in the flat region

b) Must be non-zero at the onset of a gray level

step or ramp

c) Must be non-zero along the ramps.

y-derivative x-derivative
 For 2nd derivative, expand a function by

Taylor series:

f h  f 2 2
f ( x  h)  f ( x )  h   ....
x 2! x 2

Again,

f h  f 2 2
f ( x  h)  f ( x )  h   ....
x 2! x 2
Add the two, for a discrete image, h=1

 f
2
 f ( x  1)  f ( x  1)  2 f ( x )
x 2

2nd order derivative of 1-D function

Properties::

a) Must be zero in the flat region

b) Must be non zero at the onset and at the

end of the gray level step or ramp

c) Must be zero along the ramps of constant

slope

Is 2nd order derivative isotropic???

In conclusion

• 1st order derivatives generally produce thicker

edges in an image.

• 1st order derivatives have stronger response to

the gray level steps in the image.

•2nd order derivatives have a stronger response

to fine details such as thin lines and isolated
points.

•2nd order derivatives produce a double response

at the step changes in gray level.
The 2nd derivative is better than the 1st

because of its ability to enhance finer

details in the image.

 Use of 2nd derivative for image enhancement

--- “The Laplacian”

2 f
 f ( x  1, y )  f ( x  1, y )  2 f ( x , y )
x 2

2 f
 f ( x , y  1)  f ( x , y  1)  2 f ( x , y )
y 2

 2
f  2
f
 f 
2
 2
x 2
y
 f ( x  1, y )  f ( x  1, y )  f ( x , y  1)  f ( x , y  1)  4 f ( x , y )
g  w( s, t ) f ( x  s, y  t )

0 1 0
w =
1 -4 1

0 1 0
The Laplacian derivative is used to highlight

gray level discontinuities and deemphasize

the region with slow varying gray levels.

Image that have grayish edge line and other

discontinuities , all superimposed on a dark

featureless background.
The image can be sharpen by adding or

subtracting the original image with the

Laplacian, i.e.,

g( x , y )  f ( x , y )   f ( x , y ) If the center coefficient of the

2

or Laplacian mask is negative

g( x , y )  f ( x , y )   2 f ( x , y ) If the center coefficient of the
Laplacian mask is positive
 Simplification

g( x , y )  f ( x , y )   2 f ( x , y )
 f ( x , y )   f ( x  1, y )  f ( x  1, y )  ....  4 f ( x , y )
 5 f ( x , y )   f ( x  1, y )  .....

0 -1 0
 w =
-1 5 -1

0 -1 0

Composite mask
Unsharp masking and high-boost filtering

The process in which the blurred version

of the image is subtracted from the original

image is known as UNSHARP MASKING

f E ( x , y )  f ( x , y )  f ( x, y )

fE(x,y) = edges

f(x,y) is the blurred version of f(x,y)

 f s ( x, y )  f E ( x, y )  f ( x, y )
f s ( x, y )  f ( x, y )  f ( x , y )  f ( x, y )
The generalization of unsharp masking is

high-boost filtering

f hb ( x, y)  Af ( x, y)  f ( x, y)

where A 2
f hb ( x, y)  ( A  2) f ( x, y)  2 f ( x, y)  f ( x, y)
f hb ( x , y )  ( A  2) f ( x , y )  f s ( x , y )

Recall: f s ( x, y)  f ( x, y)   f ( x, y)
2

f hb ( x , y )  ( A  1) f ( x , y )   2 f ( x , y ) Central coefficient is “-”

or
f hb ( x , y )  ( A  1) f ( x , y )   2 f ( x , y ) Central coefficient is“+”
Composite mask

0 -1 0
 w =
-1 (A- -1
1)+4
0 -1 0
 A=1

A=2.0 A=2.7
 9
R  g( x , y )   wi zi
i 1
 Point Detection
• T: nonnegative threshold: | R | T
9
R  w1 z1  w2 z2  ...  w9 z9   wi zi
i 1


• A point has been detected at the location
on which the mask is centered if: |R|>T

• The gray level of an isolated point will be

quite different from the gray levels of its
neighbors

– measure the weighted differences between

the center point and its neighbors
 Line Detection

Response R1 R2 R3 R4
 Edge Detection

edge  ( x, y ) | for all gray level pairs 100  255

•First derivative can be used to detect the
presence of an edge (i.e. to determine if the
point is on a ramp)

•The sign of second derivative can be used to

determine whether an edge point lies on the
dark or bright side of an edge
• Second derivative produces two values at every edge
which is not desirable

• Second derivative can be used to locate the center

of a very thick edge.
Zero crossing
Edge of the object

Either:

Positive peak
Negative peak

Or:

Zero crossing
Effect of noise on
edge detection!!!!!
 To suppress the effect of noise, the Laplacian is combined
with smoothing function as a precursor to find edges
via zero cross linking.
r2

Consider the Gaussian function h( r )  e 2 2

r2
 r  2
2 2
  2
 h(r )  
2
e 2

 
4


This function is known as Laplacian of a Gaussian (LOG)

The gradient of an image f(x,y) at a location (x,y)

 f 
G x   x 
f      
G y   f 
 y 

mag(f )  G  2
x G 2
y 
1/ 2

 Gy 
 ( x , y )  tan  
1

 Gx 
In vector analysis, the gradient points in the direction
of maximum rate of change of f at coordinates (x,y)
 Laplacian
 2
f  2
f
 f 
2
 2
x 2
y

0 1 0 1 1 1
1 -4 1 1 -8 1
0 1 0 1 1 1
Laplacian is generally not used in its original form
for edge detection for several reasons::

1. As a second order derivative, the Laplacian is

unacceptably sensitive to noise

2.The magnitude of Laplacian produces double

edges; an undesirable effect because it
complicates the result

3. Finally, the Laplacian is unable to detect the

edge direction. It is isotropic and has equal
effect in all direction.
The role of Laplacian in segmentation is

1. Using its zero crossing property for edge

location

2. Using it for the complementary purpose of

establishing whether a pixel is on the dark
or bright side of the edge.
The convolution of two function f ( x) and h( x )


g( x )  f ( x ) * h( x )   f (u)h( x  u)du


Discrete
M 1 N 1
1
g( x , y )  f ( x , y ) * h( x , y ) 
MN
  f (m, n)h( x  m, y  n)
m  0 n 0
 f(x)
h(x)

x
x

h(x-u) g(x)
f(u)

g(u)

u x
 How to represent an image as a SET????

In binary images, the sets in question are the members

of the 2D integer space Z2 where each element of a set is
a TUPLE (2D vector) whose coordinates are (x,y)
coordinates of the white (or the black pixels) in the image.

Binary image  x1 , y1 ,  x2 , y2 ,....., xn , yn 

Grayscale image
 x1 , y1 , f1 ,  x2 , y2 , f 2 ,....., xn , yn , f n 
The edges break due to

1) noise in the image

2) inhomogeneous illumination in the light

The edge detection should be followed by

edge linking procedure.
 Edge Linking &
Boundary Detection
• Ideal case:
– Techniques detecting intensity discontinuities
should yield pixels lying only on edges ( or the
boundary between regions).

• Real life:
– The detected set of pixels very rarely
describes a complete edge due to effects
from: noise, breaks in the edge due to non-
uniform illumination.
 Edge Linking &
Boundary Detection
• Solution:

– Edge-detection techniques are followed by

linking and other boundary detection
procedures which assemble edge pixels into
meaningful boundaries.
 Local Processing

• Analyze the pixel characteristics in a

small neighborhood (3x3, 5x5) about
every (x,y) in an image.

• Link similar points to form a

edge/boundary of pixels sharing
common properties.
 Local Processing

• Criteria used/Properties:

1. The strength of the response of the gradient

operator that produced the edge pixel.

2. The direction of the gradient vector.

 Local Processing

• In other words:
1. f  mag (F )  G 2
x G y
2 1/ 2
or f | Gx |  | G y |

(x’,y’) and (x,y) are similar if:

f ( x, y)  f ( x, y)  T

where T is a nonnegative threshold.

 Local Processing

• In other words (cont.):

 Gy 
2.  ( x, y)  tan  
1

 Gx 
(x’,y’) and (x,y) are similar if:

 ( x, y)   ( x, y)  A
where A is an angle threshold.
Local Processing
 Global Processing via
the Hough Transform
• Points are linked by determining whether
they lie on a curve of specified shape.

• Problem:

– Find subsets of n points that lie on straight

lines.
 Global Processing via
the Hough Transform
• Solution:
– Find all lines determined by every pair of
points
– Find all subsets of points close to particular
lines

– Involves: n(n-1)/2 ~ n2 lines

n(n(n-1))/2 ~ n3 computations
for comparing every point to all lines.
• Better solution: Hough Transform

– Equation of line passing through point (xi,yi):

yi = axi + b (a,b varies)

– But: b = -xia + yi 
equation of single line on ab
plane
Accumulator cell
• Since a,b approach infinity as a line
approaches the vertical, we can use the
normal representation of a line:
y

 

x cos  y sin   
 Summary

1. Compute the gradient of the image and

threshold is to obtain the binary image

2. Specify subdivisions in the  plane

3. Examine the counts of the accumulator

cells for high pixel concentration

4. Examine the relationship (principle of

continuity) between pixels in a chosen cell
• Hough transform is applicable to any
function of the form g(v,c) = 0.
– v: vector of coordinates, c: coefficients.

• e.g. points lying on a circle:

( x  c1 )  ( y  c2 )  c
2 2 2
3

c1,c2 are the coordinates of the center and

c3 is the length of the circle.

We need a 3D accumulator!!!!!!!
Global processing via Hough transformation

In this technique, the points are linked by

determining first if they lie on a curve of
specified shape.

Consider a point (xi,yi), then one can draw infinite

no. of lines (yi=axi+b) passing through it for
various values of a and b

(xi,yi)
 Global Processing via
Graph-Theoretic Techniques
• A global approach for edge detection
and linking based on representing edge
segments in the form of a graph and
searching the graph for low-cost paths
that corresponds to significant edges.
 Global Processing via
Graph-Theoretic Techniques
• Edge element:
– The boundary between two pixels p & q such
that p and q are 4-neighbors.

• Edge:
– A sequence of connected edge elements.
 Global Processing via
Graph-Theoretic Techniques

c( p, q)  H  [ f ( p)  f (q)]
H: the highest intensity value in the image
 Global Processing via
Graph-Theoretic Techniques
• Minimum cost path:
Image Segmentation
Image Segmentation
Image Segmentation
Thresholding can be viewed upon as function
of (x,y,p(x,y),f(x,y))

T  T x, y, p( x, y ), f ( x, y )

If T depends on
1. f(x,y), the threshold is global
2.f(x,y) and p(x,y), it is local
3.(x,y), it is dynamic or adaptive
Basic global thresholding
Role of illumination in thresholding
Generally we know f ( x, y)  i ( x, y)r ( x, y)

If we know the illumination (how the image is being

captured under what light condition), we can simplify our
life
g( x, y)  k i ( x, y)
where g is the image depicting the illumination

f ( x, y) r ( x, y)
   h( x , y )
g( x , y ) k
Basic Adaptive Thresholding
The following algorithm can be used to compute T
automatically

1. Select an initial estimate of T

2. Segment the image using T; this will produce two
groups of pixels: G1 consisting of all pixels with
graylevel value > T and G2 consisting of pixels T 
3.Compute the average graylevel values (m1,m2) for
the region G1 and G2 1
4.Compute the new threshold : T  ( m1  m2 )
2
5.Repeat the process 2 to 4 until difference T in
successive iteration is smaller than predefined
parameter eo
%This program automatically does the global thresholding by an
%iterative process.
clear;
close;
a=imread(‘iceage.jpg');
a=double(a);
T=input('Type the initial threshold value\n');
To=1; % tolerence
[m,n] =size(a);
Tnew=sum(sum(a))/(m*n);
Told=T;
count3=0; avg1=sum1/count1;
while abs(Told-Tnew)>To avg2=sum2/count2;
count3=count3+1;
count1=1; Tnew=0.5*(avg1+avg2);
count2=1; Told=T;
sum1=0; T=Tnew;
sum2=0; end

for i=1:m for i=1:m

for j=1:n for j=1:n
if a(i,j)>T if a(i,j)>T
count1=count1+1; a(i,j)=255;
sum1=sum1+a(i,j); else
else a(i,j)=0;
count2=count2+1; end
sum2=sum2+a(i,j); end
end end
end
end imshow(uint8(a));
Optimal global and adaptive thresholding
B

A
Use of boundary characteristics for
histogram improvement and local
thresholding

Local Thresholding
A

A
 Segmentation based on single and
double derivatives


f  mag(F )  G x2  G y2 
1/ 2
or f | G x |  | G y |

 2
f  2
f
 f 
2
 2
x 2
y

s( x , y )  0 if f  T
  if f  T and  2 f  0
  if f  T and  f  0 2
Region based segmentation
Basic formulation::
If R represent the entire image, we may view the segmentation process
that partition R into n sub-regions R1, R2, R3… Rn such that
n
1.  Ri  R
i 1

2. Ri is a connected region , i  1,2,...n

3. Ri  R j  , i  j
4. P ( Ri )  True for i  1,2,..,n
5. P ( Ri  R j )  False for i  j
Region growing:
 Segmentation based on color

Localization of facial region in digital images

Problems: Multiple faces and different background
Hair detection!!!!!
Misclassification of the
Non hair region can be
Removed by quantization
Ref:
 Segmentation by watershed !!!!!!

after the morphological processing

 The use of motion in segmentation

When the object is in motion, we will be

interested in segmenting the object and
finding the motion parameters
 Basic Approach

Difference image (DI) method:

d ij ( x , y )  1 if f ( x , y , t i )  f ( x , y , t j )  T
0 otherwise

where T is some specified threshold

 t1 t2

DI
Problem with deciding the value of T, as the
illumination may vary from one frame to the other

This can be solved by normalizing the gray level

values

Let f(x,y,ti) and f(x,y,tj) are the pixel values at

(x,y) in the previous and the current frame.

f ij ( x , y )  f ( x , y , t j )  f ( x , y , t i )

We will to minimize above eqn by iterative process

clear;
Illuminate.m
close all;

a=imread('one.jpg');
%a is my reference image

b=imread('two.jpg');
%want to bring the illumination of b same as a

[bw,ax,ay]=roipoly(a);
clear bw;
[bw,bx,by]=roipoly(b);
clear bw;

a=double(a);b=double(b);
a=a*(255/max(max(a)));

ax=uint8(ax);ay=uint8(ay);bx=uint8(bx);by=uint8(by);

delf=b(bx(1,1),by(1,1))-a(ax(1,1),ay(1,1));
sign1=sign(delf);
tol=1.0;
counter=0;

while delf>tol
if sign1==1
fj=(b-delf)*(255/max(max(b-delf)));
else
fj=(b+delf)*(255/max(max(b+delf)));
end
b=fj;
delf=b(bx(1,1),by(1,1))-a(ax(1,1),ay(1,1));
counter=1+counter;
end

imtool(uint8(b));
imtool(uint8(a));
Accumulative difference image (ADI)