Robust Object Segmentation

Using Adaptive Thresholding

Xiaxi Huang and
Nikolaos V. Boulgouris



International Conference
on Image Processing 2007

Outline
Introduction
Proposed algorithm
Experimental results
Conclusions
Introduction(1/2)
The extraction of moving objects from
video sequences is important!!
Object detection methods
SGM ( Single Guassian Model )
MGM ( Mixed Guassian Model )
BG substaction combined color and edge
information. (Aug. 2000)
Serious
drawbacks
Poor performance for indoor shadow, light
reflection, and high similarity of FG and BG.
Introduction(2/2)
Proposed algorithm
Adaptive thresholding detection
Shadow removal method

Proposed Algorithm
Procedures Of Algorithm
Updating
BG Image
Initial Mask
Estimate
FG Areas
Estimate
BG Areas
Confidence
Map From
Detection
Of RGB
Confidence
Map From
Detection
Of Edge
Maximum
Of The
Confidence
Maps
Minimum
Of The
Confidence
Maps
Combined
Confidence
Map
Hysteresis
Thresholding
FG Map
(Object and
Shadows)
Edge Map
(Boundary
Of objects)
Post-
processing
Final Object
Map
Next Frame
Background
Substraction
Shadow
Removal
Proposed Algorithm
Background Updating
Why?
In many background substraction method
update all pixels in a frame.
A serious drawback


To avoid this condition

Misclassfied a stop moving object.
( ) ( ) ( ) ( ) | o | o |
1 1
1

+ =
t t t
B I B
Proposed Algorithm
Initial Block-size Mask(1/5)
Roughly determine the foreground
areas.
Lower threshold
Calculate average different between
the current frame and background
frame in a block


Threshold it with T

( ) ( )

e
=
u |
| |
u
B I d
avg
1
Proposed Algorithm
Initial Block-size Mask(2/5)
Divide the blocks with larger difference
which are assumed to contain foreground
pixels into smaller size


Apply a higher threshold and detect sub-
blocks



4
1
/
n n
=u u
added n n
T T T + =
1
Proposed Algorithm
Initial Block-size Mask(3/5)
The block with
larger difference
Using a higher
threshold

Proposed Algorithm
Initial Block-size Mask(4/5)
How to get a foreground blocks map?
Median filter
Edge pixels of the objects might be lost
Apply two initial block-size, and combine
their foreground map



' | |
Map Map Map
block
=
Proposed Algorithm
Initial Block-size Mask(5/5)
Proposed Algorithm
Color Change Detection With Adaptive
Threshold(1/2)
Adaptive threshold






D : difference frame ( D = | I - B|)

( )
( ) ( )
( ) ( ) ( )
( ) ( ) ( )
2
0
2
2
0 0
2
0
2
2
2
1
1
1
1
/

+ e
+ e
+ e

+
=

+
=
+
=
+ =
|
|
|
| | | o
| | | o
| |
o o
curr curr
diff diff
diff
diff diff curr adapt
I
D
D
k k T
Local variance in the
current frame
Local variance in the
difference frame
Local mean value in the
difference frame
Proposed Algorithm
Color Change Detection With Adaptive
Threshold(2/2)
Get a threshold T by setting k
1
, k
2
Create confidence maps in three color
channels respectively
Maximum the confidence map CMap
color


Proposed Algorithm
Edge Detection(1/2)
In order to
Extraction of the foreground
Removal of shadow
Compute edge magnitude


G
x
and G
y
are the horizontal and vertical difference
in the difference frame D.
Sobel mask


2 2
y x magnitude
G G G + =
(
(
(

=
(
(
(

=
1 2 1
0 0 0
1 2 1
1 0 1
2 0 2
1 0 1
y x
H H
Proposed Algorithm
Edge Detection(2/2)
Proposed Algorithm
Combination(1/3)
Combine two confidence maps.
Estimate foreground area
Maximum the confidence map


Estimate background area
Minimum the confidence map
( )
block edge color
Map CMap CMap CMap = , max
1
( )
block edge color
Map CMap CMap CMap = , min
2
Proposed Algorithm
Combination(2/3)
Combination

2 1
CMap CMap CMap =
Proposed Algorithm
Combination(3/3)
(a) (b)
(c) (d)
Fig. (a) Original
image, (b) Confidence
map Of RGB change
Detection with adaptive
threshold, (c) Confidence
map of Sobel edge
detection, (d) Combined
confidence Map.
Proposed Algorithm
Hysteresis thresholding
Remove false positive
Set two thresholds T
0
, T
1
; T
1
/T
0
is
about 2 or 3
C(x) > T1 : High confidence region
T0 < C(x) < T1 : Check neighbors

Hysteresis
thresholding
Proposed Algorithm
Shadow Removal And Post-Processing(1/4)
Indoor environment
Soft colored illumination, light-reflect
floor, shadow
Hard to distinguish shadows from objects
by using color information.
How to solve this problem?
Combine FG and edge confidence map
Proposed Algorithm
Shadow Removal And Post-Processing(2/4)
Apply hysteresis thresholding to edge
confidence map
Set bounding boxes
Remove pixels out of bounding boxes


Proposed Algorithm
Shadow Removal And Post-Processing(3/4)
(a) (b)
(d) (c)
Fig. (a) Foreground map
before shadow removal,
(b) Hysteresis thresholding
result of edge confidence
Map, (c) Foreground map
after shadow removal, (d)
Binary map of extracted
objects.
Proposed Algorithm
Shadow Removal And Post-Processing(4/4)
Some temporal filters of offline detection



To achieve above

( )
( )
1 1
1 1
+
+
=
=
t t t t
TF
t t t t
TF
Map Map Map Map
Map Map Map Map
( )
1 1
, ,
+
=
t t t t
TF
Map Map Map median Map
Eliminate spurious points
Retrieve missed FG pixels
Experimental Results
Fig. (a) Origin image, (b)
Foreground maps created
by MGM plus HSV method,
(c) Foreground maps
created by mixture Gaussian
Model, (d) Foreground maps
created by proposed algori-
thm.
(a) (b)
(d) (c)
Experimental Results
Fig. (a) Origin image, (b)
Foreground maps created
by MGM plus HSV method,
(c) Foreground maps
created by mixture Gaussian
Model, (d) Foreground maps
created by proposed algori-
thm.
(a) (b)
(d) (c)
Experimental Results
Fig. (a) Origin image, (b)
Foreground maps created
by MGM plus HSV method,
(c) Foreground maps
created by mixture Gaussian
Model, (d) Foreground maps
created by proposed algori-
thm.
(a) (b)
(d) (c)
Experimental Results
Fig. (a) Origin image, (b)
Foreground maps created
by MGM plus HSV method,
(c) Foreground maps
created by mixture Gaussian
Model, (d) Foreground maps
created by proposed algori-
thm.
(a) (b)
(d) (c)
Conclusions
Compared with the popular MGM object
segmentation method and the HSV shadow
removal method, proposed method achieves
more robust performance
Considering that the proposed algorithm does
not involve future frames, it can be used in
real-time processing applications.
Furthermore, if it is used offline, a temporal
filter can be applied to further improve the
performance of the algorithm.