Sunteți pe pagina 1din 40

Image Segmentation

Gray-level thresholding
Supervised vs. unsupervised
thresholding
Binarization using Otsus method
Locally adaptive thresholding
Maximally stable extremal regions
Color-based segmentation
Region labeling and counting
Region moments

Digital Image Processing: Bernd Girod, 2013-2014 Stanford University -- Image Segmentation 1

Gray-level thresholding
How can holes be filled?

Original image
Peter f [x,y]

Thresholded
Peter m [x,y]

Digital Image Processing: Bernd Girod, 2013-2014 Stanford University -- Image Segmentation 2

f x, y m x, y

How to choose the threshold?


pdf
Background

Foreground

Gray level

Digital Image Processing: Bernd Girod, 2013-2014 Stanford University -- Image Segmentation 3

Unsupervised thresholding

Idea: find threshold T that minimizes within-class variance of both foreground and
background (same as k-means)
2
within
T

N Fgrnd T
N

2
Fgrnd
T

N Bgrnd T
N

2
Bgrnd
T

Equivalently, maximize between-class variance


2
2
between
T 2 within
T
N Bgrnd 1
N Fgrd 1
1
2
2
2
2
f x, y
f x, y Fgrnd

N
N
N
N
x, y

Fgrd x , yFgrnd

N Bgrnd
N Fgrnd 2
N Bgrnd 2
N Fgrnd
N Bgrnd
2
2
2

Fgrnd
Bgrnd
Fgrnd N Bgrnd
N
N
N
N
2
T N Bgrnd T
Fgrnd

T
Fgrnd Bgrnd
N2

x , yBgrnd

f x, y Bgrnd

[Otsu, 1979]
Digital Image Processing: Bernd Girod, 2013-2014 Stanford University -- Image Segmentation 4

Unsupervised thresholding (cont.)

Algorithm: Search for threshold T to maximize

2
between

N Fgrnd T N Bgrnd T
N2

T T
Fgrnd

Bgrnd

Useful recursion for sweeping T across histogram:


N Fgrnd T 1 N Fgrnd T nT
N Bgrnd T 1 N Bgrnd T nT

Fgrnd T N Fgrnd T nT T
Fgrnd T 1
N Fgrnd T 1
Bgrnd T N Bgrnd T nT T
Bgrnd T 1
N Fgrnd T 1
[Otsu, 1979]
Digital Image Processing: Bernd Girod, 2013-2014 Stanford University -- Image Segmentation 5

Unsupervised thresholding (cont.)


T 112

Digital Image Processing: Bernd Girod, 2013-2014 Stanford University -- Image Segmentation 6

Unsupervised thresholding (cont.)


T 56

Digital Image Processing: Bernd Girod, 2013-2014 Stanford University -- Image Segmentation 7

Unsupervised thresholding (cont.)


T 168

Digital Image Processing: Bernd Girod, 2013-2014 Stanford University -- Image Segmentation 8

Unsupervised thresholding (cont.)


T = 144

Digital Image Processing: Bernd Girod, 2013-2014 Stanford University -- Image Segmentation 9

Sometimes, a global threshold does not work

Original image

Digital Image Processing: Bernd Girod, 2013-2014 Stanford University -- Image Segmentation 10

Thresholded with
Otsus Method

Locally adaptive thresholding

Slide a window over the image


For each window position, decide whether to
perform thresholding

Thresholding should not be performed in uniform


areas
Use variance or other suitable criterion

Non-uniform areas: apply Otsus method (based on


local histogram)
Uniform areas: classify the entire area as
foreground or background based on mean value

Digital Image Processing: Bernd Girod, 2013-2014 Stanford University -- Image Segmentation 11

Locally adaptive thresholding (example)

Non-uniform areas

Local threshold values

Digital Image Processing: Bernd Girod, 2013-2014 Stanford University -- Image Segmentation 12

Locally thresholded result

Maximally stable extremal regions

Extremal region: any connected region in an image with all pixel values above (or
below) a threshold
Observations:

Nested extremal regions result when the threshold is successively raised (or lowered).
The nested extremal regions form a component tree.

Key idea: choose thresholds such that the resulting bright (or dark) extremal
regions are nearly constant when these thresholds are perturbed by +/-

maximally stable extremal regions (MSER)

[Matas, Chum, Urba, Pajdla, 2002]

Digital Image Processing: Bernd Girod, 2013-2014 Stanford University -- Image Segmentation 13

MSERs: illustration
180
160
140
120
100

+
80
-60

A+
A
A-

40
20
0

A A
Local minimum of
MSER
A
[Matas, Chum, Urba, Pajdla, 2002]

Digital Image Processing: Bernd Girod, 2013-2014 Stanford University -- Image Segmentation 14

Level sets of an image


1

f [ x, y]

Image

Digital Image Processing: Bernd Girod, 2013-2014 Stanford University -- Image Segmentation 15

f [fxf[,[xxy,,]yy]]
807
6
2
4
1
3
5

Level Set

Component tree of an image


a

A2=30

A2=21

A4=12

A4=18

A5=12

A4=14
h

A5=11
k

d
A3=9

A3=32 A3=19

A2=36

A3=18

d
A2=32

A A
A

A1=36

A3=21

Local minima of sequence

A1=43

A1=67

A0=225

A4=5
l

i
A4=19
i

m
A5=10

A6=4
p

Digital Image Processing: Bernd Girod, 2013-2014 Stanford University -- Image Segmentation 16

, 1,

MSERs

MSER: examples

Dark MSERs, =15

Original image

Digital Image Processing: Bernd Girod, 2013-2014 Stanford University -- Image Segmentation 17

Bright MSERs, =15

MSER: examples

Dark MSERs, =15

Original image

Digital Image Processing: Bernd Girod, 2013-2014 Stanford University -- Image Segmentation 18

Bright MSERs, =15

Supervised thresholding
pdf
Background

Foreground

error
probability

Digital Image Processing: Bernd Girod, 2013-2014 Stanford University -- Image Segmentation 19

Gray level

Supervised thresholding
pdf

MAP (Maximum A Posteriori) detector

Background

Foreground

error
probability

Gray level

If errors BGFG and FGBG are associated with different costs:


Bayes minimum risk detector is optimal.

Digital Image Processing: Bernd Girod, 2013-2014 Stanford University -- Image Segmentation 20

Multidimensional MAP detector

Training

Provide labelled set of training data


Subdivide n-dimensional space into small bins
Count frequency of occurrence for each bin and class
in training set, label bin with most probable class
(Propagate class labels to empty bins)

For test data: identify bin, look up the most


probable class

Digital Image Processing: Bernd Girod, 2013-2014 Stanford University -- Image Segmentation 21

MAP detector in RGB-space


Non-skin Samples

Skin Samples

Original image

Skin color detector

Five training images


Digital Image Processing: Bernd Girod, 2013-2014 Stanford University -- Image Segmentation 22

Linear discriminant function

To segment image with n components fi, i=1,2,,n


into two classes, perform test

w f

i i

w0 0 ?

Categories are separated by hyperplane in n-space


Numerous techniques to determine weights
wi, i=0,1,2,,n, see, e.g., [Duda, Hart, Stork, 2001]
Can be extended to the intersection of several linear
discrimant functions
Can be extended to multiple classes

Digital Image Processing: Bernd Girod, 2013-2014 Stanford University -- Image Segmentation 23

Chroma keying

Digital Image Processing: Bernd Girod, 2013-2014 Stanford University -- Image Segmentation 24

Landsat image processing

Original Landsat image false color picture out


of bands 4,5,6

Water area segmented and enhanced to show


sediments

Source: US Geological Survey USGS, http://sfbay.wr.usgs.gov/

Digital Image Processing: Bernd Girod, 2013-2014 Stanford University -- Image Segmentation 25

Region labeling and counting

How many fish in this picture?

Original Fish image

after thresholding

Which pixels belong to the same object (region labeling)?


How large is each object (region counting)?

Digital Image Processing: Bernd Girod, 2013-2014 Stanford University -- Image Segmentation 26

4-connected and 8-connected neighborhoods

Definition: a region is a set of pixels, where each pixel can be reached from
any other pixel in the region by a finite number of steps, with each step starting
at a pixel and ending in the neighborhood of the pixel.

4-neighborhood

8-neighborhood

Typically, either definition leads to the same regions, except when


a region is only connected across diagonally adjacent pixels.

Digital Image Processing: Bernd Girod, 2013-2014 Stanford University -- Image Segmentation 27

Region labeling algorithm (4-neighborhood)

Loop through all pixels f [x,y], left to right, top to bottom


If f [x,y]=0, do nothing.
If f [x,y]=1, distinguish 4 cases

Generate new
region label

Copy label
from above

Copy label
from the left

Second pass through image to replace equivalent label by the


same label.

Digital Image Processing: Bernd Girod, 2013-2014 Stanford University -- Image Segmentation 28

Copy label from the left. If


labels above and to the left are
different, store equivalence.

Region labeling example (4-neighborhood)


1

List of Region Labels


1

1
2

2
3

2
3

Digital Image Processing: Bernd Girod, 2013-2014 Stanford University -- Image Segmentation 29

All three labels are


equivalent, so merge
into single label.

Region labeling example (4-neighborhood)


1

List of Region Labels


1

1
1

1
1

1
1

Digital Image Processing: Bernd Girod, 2013-2014 Stanford University -- Image Segmentation 30

Example: region labeling

Thresholded image

Digital Image Processing: Bernd Girod, 2013-2014 Stanford University -- Image Segmentation 31

20 labeled regions

Region counting algorithm

Measures the size of each region

Initialize counter[label]=0 for all label

Loop through all pixels f [x,y], left to right, top to bottom


If f [x,y]=0, do nothing.
If f [x,y]=1, increment counter[label[x,y]]

Digital Image Processing: Bernd Girod, 2013-2014 Stanford University -- Image Segmentation 32

Small region removal

Loop through all pixels f [x,y], left to right, top to bottom


If f [x,y]=0, do nothing.
If f [x,y]=1 and counter[label[x,y]]<S, set f [x,y]=0
Removes all regions smaller than S pixels

Digital Image Processing: Bernd Girod, 2013-2014 Stanford University -- Image Segmentation 33

Hole filling as dual to small region removal


Mask with holes

After NOT operation, (background)


region labeling, small region removal,
and second NOT operation

Digital Image Processing: Bernd Girod, 2013-2014 Stanford University -- Image Segmentation 34

Region moments

M pq

Raw moments

x p yq

x,yRegion

Central moments

pq

x x y y
p

x,yRegion

M10
M 01
with x
and y
M 00
M 00

Region orientation and eccentricity:


calculate eigenvectors of covariance
matrix 20 11

11

02

Digital Image Processing: Bernd Girod, 2013-2014 Stanford University -- Image Segmentation 35

Example: Detecting bar codes

Original Image

Digital Image Processing: Bernd Girod, 2013-2014 Stanford University -- Image Segmentation 36

Example: Detecting bar codes


Locally adaptive
thresholding

Digital Image Processing: Bernd Girod, 2013-2014 Stanford University -- Image Segmentation 37

Example: Detecting bar codes


Locally adaptive
thresholding
Filtering by
eccentricity

Digital Image Processing: Bernd Girod, 2013-2014 Stanford University -- Image Segmentation 38

Example: Detecting bar codes


Locally adaptive
thresholding
Filtering by
eccentricity

Filtering by major
axis length

Digital Image Processing: Bernd Girod, 2013-2014 Stanford University -- Image Segmentation 39

Example: Detecting bar codes


Locally adaptive
thresholding
Filtering by
eccentricity

Filtering by major
axis length
Filtering by
orientation

Digital Image Processing: Bernd Girod, 2013-2014 Stanford University -- Image Segmentation 40