Indian Academy of Sciences, Bangalore

Summer Research Fellowship Programme 2010

Name of SRF:

SEJAL CHAUHAN

Registration No.:

ENGS628

Institution where working:

IIT, BOMBAY

Date of joining the project:

10th MAY,2010

Date of completion of the project:

10th JULY,2010

Name of the Guide:

Prof. SUBHASIS CHAUDHARI

Project Title:

VIDEO COMPOSTING UNDER VARYING

ILLUMINATION

Acknowledgement
IamextremelygratefultoProf.SubhasisChaudhuri,DepartmentofElectricalEngineering,IndianInstitute
ofTechnology,Bombay,forbeingtheprimesourceofguidanceandinspirationincompletionofthe
project.IextendmyheartfeltgratitudetoMr.R.Shanmuganathanforguidingmeandhelpingme
overcomealltheproblemsduringthecourseoftheproject.Hehasbeenaperennialsourceofmotivation
andhelpbehindmywork.IthankMr.VishalKumarforconsistentlychannelizingmyknowledgeand
helpingmeovercometheinitialhurdlesintheproject.IamthankfultoMr.RanjithA.Ramwithwhose
cooperationIwasabletodothedesiredproject.IamthankfultoMr.DevenPatelandMrs.RenuM.R.for
providingthenecessarysupportandhelpinthelab.IthankMr.RonakShahforhisinspirationandhelpin
completionofthework.Iamthankfultomycolleaguesandfriendsformotivatingmeforresearchand
providingconstantsupportindifficulttimes. Ithasindeedbeenagreatprivilegeformetoworkintheir
projectgroup.IamindebtedtoIndianInstituteofTechnology,BombayespeciallyVisionandImage
ProcessingLabforprovidingmewiththenecessaryandimperativeinfrastructureandsystemsupport
towardsthecompletionofmywork.IambeholdentoIndianAcademyofSciences,IndianNational
ScienceAcademy&theNationalAcademyofSciences,Indiaforprovidingmewiththeopportunityand
financialsupporttoundertakethisprojectwork.

Certificate
Thisistocertifythattheprojecttitled,VideoCompositingUnderVaryingIllumination,was
undertakenbySejalChauan2ndyearB.Tech,DepartmentofElectronicsandCommunicationEngineering,
NationalInstituteofTechnology,WarangalasapartofSummerResearchFellowshipProgrammeoffered
byIndianAcademyofSciences,IndianNationalScienceAcademy&TheNationalAcademyofSciences.
TheworkwascompletedundertheguidanceofProf.SubhasisChauduri,DepartmentofElectrical
Engineering,IIT,Bombayduringthetimespanof8weeksbetween10May201010July2010anda
reportonthesamewassubmittedtoProf.SubhasisChauduri,DepartmentofElectricalEngineering,Indian
InstituteofTechnology,Bombay.

Prof.SubhasisChauduri
DepartmentofElectricalEngineering
IndianInstituteofTechnology,Bombay

Abstract
DigitalComputationalPhotographyaimsatreplicatingtherealworldscenesfaithfully.Butthereexist
someshortcomingswhichhavetobeimproveduponregardingtheclarityofthevideo.Onesuchbeingthe
varyingilluminationcausingthecameratoselfadjusttheintensityoftheoverallimageandhence,causing
distortionsinclarityofdesiredimageoftheextractedobject.Thisusuallyhappensbecauseoftheautomatic
gaincontrol(AGC)ofthecamera.Trackingthemovingobjectandselfilluminatingithelpsusresolvethe
problem.ThealgorithmbeingusedheretoextracttheexactobjectisChanVeseactivecontourwith
StaufferGrimsonsalgorithmwhichhelpsinbackgroundsubtractionusingmixtureofGaussians.Chan
Veseactivecontourwithoutedgesusesgeometricalmodelsandvariationallevelsetmethodstofindthe
fittingenergyfunctionalwhichisthenminimisedandtheparametersupdatedtoextractthemovingobject.
Thecontoursevolvedependingonthekindofinputimagegiventothemandamaskwhichhelpsitdecide
itsstartingposition.Theinputimagehasaninfluenceofthelightfallingontheobjectwhichevendistorts
theforegroundobtained.Soforbetteringtheimages,weusekmeansclusteringmethodtoobtainthebest
possiblemaskswhichsegmenttheoriginalimagesandareusedintheactivecontourmodelofChanVese.
Themaskscanthenbeusedtoreilluminatetheobjectinthevideo.

Contents
Video Compositing Under Varying Illumination
Acknowledgements
Certificate
Abstract
1Introduction....6
1.1AlgorithmUsed...7
1.2RequiredKnowledge.......7
2Background....8
2.1ExistingWork.....8
2.2AGCEstimator.......9
2.3ChanVesemodel...10
3PresentWork..11
3.1Kmeansclustermethod...12
3.2ConvexHull.......13
4.ExperimentationandResults.......13
4.1Applicationofvariousalgorithmsused.13
4.2SampleProblem...18
5Conclusionsandfuturework.22
6Bibliography.....23

Video Composting Under Varying

Illumination
1 Introduction
Digitalimageprocessingisanyformofsignalprocessingforwhichtheinputisanimage,suchasa
photographorvideoframe;theoutputofimageprocessingmaybeeitheranimageor,asetof
characteristicsorparametersrelatedtotheimage.Mostimageprocessingtechniquesinvolvetreatingthe
imageasatwodimensionalsignalandapplyingstandardsignalprocessingtechniquestoit.Thereare
manyapplicationstothisfieldsuchasComputervision,Facedetection,Remotesensing,Morphological
ImageProcessing.ThevariousfieldswherethisisappliedareBiologicaldatavisualization,Chemical
imaging,Crimemapping,Datavisualization,Educationalvisualization,Geovisualisation,Information
visualization,Mathematicalvisualization,Medicalimaging,Productvisualization,Volumevisualization.

Inthisproject,wedealwithbackgroundsubtractionwhichisaprocesstodetectamovementorsignificant
differencesinsideofthevideoframe,whencomparedtoareference,andtoremoveallthenonsignificant
components(background).Thebasicrequirementfrombackgroundsubtractionalgorithmistheabilityto
detectmotionelementsaccuratelyandtoadjusttoanybackgroundillumination.
Backgroundsubtractionisappliedinmanyareas,suchassurveillancesystem(toeffectivelysegmentthe
onlymovingobject),poseorgesturerecognition.ForinstancewhileinspectinganATMrobbery.The
problemthatwearedealingwithistheclarityoftheobject,inthiscasearobber.Itisveryimportantto
viewhis/herimagewiththebestluminancewhichthepresentdigitalcamerafailstocaptureduetolimited
dynamicranging.
Inthepast,someworkhasbeendoneinthefieldofmotiontrackinginvideoswheretheforegroundis
separatedfromthebackgroundandanymovingobjectistracked.Themainaimistosegmentandextract
theobjectinsuchawaythatallitsfeaturesarevisibleandminimalnoiseispresent.Manyalgorithms
involvingspatialhomogeneity,Kalmanetal.[5]filterapplication,Watershedalgorithm,Bayesian
techniquehavebeenused.Themainreasonforthefailureofbackgroundsubtractionmethodsincertain
casesisthelackofexploitationofspatialcorrelationinimageintensity,whereascameraAGCaffectsthe
intensityatagloballevel.
Thenextstepwouldbetomakesuretheobjectisobtainedproperlyviaactivecontourevolutionaroundthe
object.Thesecontourstakeupthevideoframeandthepreviousmaskobtainedastheinput.Thegoalisto
evolvethecontourinsuchawaythatitstopsontheboundariesoftheforegroundregion.Therearevarious
waystodefinetheevolutionequation;forexample,thecontourmightmovewithavelocitythatdependson
thelocalcurvatureatagivenpointortheimagegradientatthatpoint.Afterextractionoftheobject,weuse
themasktobrightenupthepresentobjectinthevideowhichisonlypossibleiftheobjectisextracted
6

perfectly.TheartificialilluminationisdonebytheBhatetal.[10]algorithm.
Thereportisorganizedinsuchamannerthatsection1dealswiththeintroductionoftheproblemand
variousconceptsinvolved.Section2dealswiththepreviousexistingworkandsection3revolvesaround
mywork.Section4givesustheobservationsandresultsfordifferentcasesfollowedbyconclusionsand
references.
1.1 ALGORITHMUSED
TheStaufferGrimsonsetal.[1]mulitimodalalgorithmwhichusestheadaptivemixtureofGaussian
variablesalongwithpoissonsolveretal.[9]andKalmanfilter[5]toprocessestheimageingradientfield.
Themasksobtainedfromthisalgorithmhavealotofnoiseowingtoautomaticgaincontrolofthedigital
camera.SoarobustalgorithmwhichestimatestheAGCwhiletrackingisbuildsothatthiscanbe
eliminatedandthemaskshenceobtainedarefreedfromnoise.
Wethentaketheseimprovedmasksandwiththehelpofactivecontoursevolvingaroundtheobjectand
usinglevelsetmethodextractafineroutline.Afunction(i,j,t)(thelevelsetfunction)isdefined,where
(i,j)arecoordinatesintheimageplaneandtisanartificialtime.Atanygiventime,thelevelsetfunction
simultaneouslydefinesanedgecontourandasegmentationoftheimage.Theedgecontouristakentobe
thezerolevelset{(i,j)suchthat (i,j,t)=0},andthesegmentationisgivenbythetworegions{ >=0}
and{ <0}.Thelevelsetfunctionwillbeevolvedaccordingtosomepartialdifferentialequation,and
hopefullywillreachasteadystatelimt>? thatgivesausefulsegmentationoftheimage.
Wheneverwetakeavideowhichhasvarioustextureinthebackground,thisalgorithmfailswhileusing
ChanVeseetal.[3]activecontoursastheyeventaketheinputimageintoconsideration.Sothereisaneed
toimprovetheinputimagesuchthatthefinalmaskobtainedistheexactsilhouetteoftheobject.Thek
meansclusteringmethodisused.
Thismethodofclusteranalysiswhichaimstopartitionnobservationsintokclustersinwhicheach
observationbelongstotheclusterwiththenearestmean.Euclideandistanceisusedasametricand
varianceisusedasameasureofclusterscatter.Aweighteddistancemeasureutilizingpixelcoordinates,
RGBpixelcolorand/orintensity,andimagetextureiscommonlyused.
1.2REQUIREDKNOWLEDGE
StaufferGrimsonModel
ThealgorithmdiscussesmodelingeachpixelasamixtureofGaussiansandusinganonlineapproximation
toupdatethemodel.Ridderetal.[5]modeledeachpixelwithaKalmanFilterwhichmadetheirsystem
morerobusttolightingchangesinthescene.Forapixel(x0,y0)atatimet,whereIistheimage,itshistory
isgivenby:

Theprobabilityofobservingthecurrentpixelvalueisgivenasfollows,

(1)

(2)
whereKisthenumberofdistributions,i,tisanestimateoftheweight(whatportionofthedatais
accountedforbythisGaussian)oftheithGaussianinthemixtureattimet,i,tisthemeanvalueoftheith
7

Gaussianinthemixtureattimet,i,tisthecovariancematrixoftheithGaussianinthemixtureattimet,
andwhereisaGaussianprobabilitydensityfunction.

(3)
IfnoneoftheKdistributionsmatchthecurrentpixelvalue,theleastprobabledistributionisreplaced
withadistributionwiththecurrentvalueasitsmeanvalue,aninitiallyhighvariance,andlowpriorweight.
ThepriorweightsoftheKdistributionsattimet,k,t,areadjustedasfollows
(4)
Theandparametersforunmatcheddistributionsremainthesame.Theparametersofthedistribution
whichmatchesthenewobservationareupdatedasfollows
(5)
(6)
(7)
First,theGaussiansareorderedbythevalueof/.Thisvalueincreasesbothasadistributiongains
moreevidenceandasthevariancedecreases.ThenthefirstBdistributionsarechosenasthe
backgroundmodel,where
(8)
PreprocessingofGradients
Thiswaythesaturationisreducedinunderoroverexposedimages.Thisisdonebycarefullymodifying
theircorrespondinggradientfields.Thefollowingistheilluminationchangefunction.

v f * f *

(9)

Thisoperationgisdependentonthevaluesofandwhicharetheparametersinvolved,whereasfi
istheimage.
g i ( x, y ) (0.2 * i /( f i ( x, y ) )) ( i ) f ( x, y )

(10)

Intheaboveequation,i is the mean gradient magnitude of the ith image. [0.2,0.2] where the image
brightens up when the value is negative and is exposed to low illumination when is positive.
This is the basic equation required to change the illumination of an object in a video which is done in the
other project using the masks obtained from this experiment.

2.BACKGROUND
8

2.1EXISTINGWORK
StaufferandGrimson[1]modelledeachpixelasamixtureofGaussiansandanonlineEMAlgorithmwas
proposedbyP.KaewTraKulPongetal.[8]toupdatethemodel.Eventhough3to5Gaussiandistributions
areabletomodelamultimodalbackgroundwithdifficultsituationslikeshakingbranchesoftree,clutter
andsoforth,thereisafactthatthiskindofpixelbasedbackgroundmodellingissensitivetonoiseand
illuminationchange.Alotofpeoplepresentedanumberofimportantproblemswhenusingbackground
subtractionalgorithms,theyproposedasolutionusingpixel,regionandframelevelprocessing,their
algorithmiscapableofdealingwithquickilluminationchanges,buttheirtechniqueisbasedonacomplex
gradientbasedalgorithm. Foreground was analyzedasmovingobjects,shadowandghostbycombining
themotioninformation.Whereastheseworksareeffectivetosolvealltheproblemstheymentioned,the
computationcostisrelativelyexpensiveforrealtimevideosurveillancesystemsbecauseofthe
computationofopticalflow.
Thefollowingformulaehavebeenusedinthepastsofartoextracttheforegroundfromthebackground.
[11]
(11)

(12)

(13)

(14)
(15)
(16)
(17)
(18)
Thevariationinilluminationisdoneinthegradientfieldwhichisnonconservativeafterprocessing.To
retrievetheimagesbackfromtheprocessedgradients,possionequationsolveretal.[9]hasbeenused.
2.2AGCESTIMATOR
9

WhentheeffectofcameraAGCissignificantoverashortspanoftimethenStaufferGrimsonmethodfails
toupdatethebackgroundpixelscorrectly.AGCispositivewhentheobjectisdarkerthanthebackground
andthewholesceneisexposedtomorelight.Whilewhentheobjectisbrighterthanitssurroundingsthen
AGCisnegative.TheAGCandexposuretimeaffectthevideoframeataglobal levelsothetemporal
intensityvariationofthenonmotionpixelscanbeexpectedtovaryinanabruptfashionbutitispossibleto
parameterizetheintensityvariation.Todecreasethecomputationaloverheadweconsiderafewuniformly
spacedpixelsforthepurpose.Ascatterplotismadewhichsamplespointsinthebestbackgroundimage
andthecurrentframe.Allthepointsthatliebeyondacertainthresholdbandwidthaty=xlineare
eliminated(outliers).
Transferfunctiongraph

C
U
R
R
E
N
T
F
R
A
M
E

Backgroundframe
10

Fig1

Infigure1wecanseethereisalotofdeviationfromthebandthresholdwhichmeansAGChasamajorrolein
changingtheimage.Sotheparametersareupdatedaccordingly.Thebestfitcurvesforred,green,bluecolorsarealso
depicted.

Outliers=numberofpointslyingoutsidethethinbandofthescatterplot
Totalnumberofpoints

(19)

IftheoutliersexceedacertainthresholdthenAGCispresentandabestfitcurveisobtained.Thenthe
backgroundpixelsmeanandvarianceareupdatedsoastocompensatefortheAGC.Thetransferfunction
fb,tfivesusagoodestimateoftheintensitytransfercharacteristicsbetweenthebestbackgroundestimate
andthecurrentframe.SopreprocessingofthemeanofGaussiansisdonebeforematchingitwiththe
intensityofthecurrentpixelintensity.
(20)
'
(1 )
f (
)
k , t 1

k ,t 1

b ,t

k , t 1

Whereliesbetween0.5and0.7
Thevarianceundergoesaweightedupdate

(21)

k' ,t 1 [(1 ) f b',t ( k ,t 1)] k ,t 1

Whenthevideoistoonoisythentheparametersareupdatedasfollows

(22)

k'2,t 1 max[ k2,t 1 , ( k ,t 1 f b',t ( k ,t 1 )) 2 ]

2.3CHANVESEMODEL
Dependingonimplementation,activecontourshavebeenclassifiedasgeometricorparametricactive
contours.Parametriccontours,irrespectiveofrepresentation,areknowntosufferfromtheproblemof
irregularbunchingandspacingoutofcurvepointsduringthecurveevolution.Inasplinebased
implementationofactivecontours,thisleadstooccasionalformationofloopslocally,andsubsequentlythe
curveblowsupduetoinstabilities.Initializationisproblemwhichisovercomeinthisexperimentasthe
masksthatareobtainedfromtrackeralgorithmisgiventotheactivecontouralgorithm.
ChanVese[3]algorithmusesvariationalcalculusmethodstoevolvethelevelsetfunction.Variational
methodsworkbyseekingalevelsetfunctionthatminimizessomefunctional.Inthiscontext,byfunctional
wemeanamappingthattakesalevelsetfunction asinput,andreturnsarealnumber.
Euclideancurveshorteningequation
(23)
Energyofsnakesisgivenby
(24)

WhereEintrepresentstheinternalenergyofthecurve,positionvectorv(s)=(x(s),y(s)),andare
parametersthataresetbytheuserwithsubscriptsbeingthederivatives.
considerthecontourwiththepositionvectorw=v1,v2,.,vn,vi=(xi,yi).

11

(25)

(26)
Thefittingenergywheretheminimizinglevelsetfunction willdefinethesegmentation,isgivenby
(27)

(28)
WhereHisaHeavisidefunction,,,1, 2andpareparametersselectedbytheusertofitaparticular
classofimages.
TheEulerLagrangeequationsandthegradientdescentmethodareusedtoderivethefollowingevolution
equationforthelevelsetfunction thatwillminimizethefittingenergy

Theequationtoreinitializingtheenergyforthesnakestokeepevolvingisgivenby

Wheret(0)= att=0

FlowChart

12

3PRESENTWORK
Theexperimentaimsatachievingrobustnessandbeingabletoapplytoanyscene.Themaskspresently
obtainedfromvariousalgorithmsgiveusanideaonlyaboutthepositionoftheobjectandsomewhatthe
shape.Wehavetriedtoapproachtheexactextractionofthesilhouetteofthemovingobjectbythe
applicationofkmeansclustersegmentationtotheappropriateactivecontourmodelsuchthatthebestmask
isapproached.Wehaveevenincludedallthosesituationsinwhichaparticularalgorithmwouldwork.In
caseofavideowhereinthebackgroundisplain,thealgorithmalmostachievestheperfectmask.Butin
caseofamorenaturalbackground,thereareextraartifactsincludedinthemaskswhichdistortsthem.So
weeventrytoovercomeitbymorphologicaloperationsbeforetheyareinputtedtothefinalactive
contours.
3.1KMEANSCLUSTERSEGMENTATION
Thisisusedtosegmenttheoriginalimageinsuchamannerthattwooftheclusterscombinetocomplete
theobjectwhichneedstobeextracted.Theprocessedimagecanthenbeproperlyusedasaninputtothe
activecontourmodelwithoutthefinalmaskbeingdistorted.Clusteringisawaytoseparategroupsof
objects.Kmeansclusteringtreatseachobjectashavingalocationinspace.Itfindspartitionssuchthat
objectswithineachclusterareasclosetoeachotheraspossible,andasfarfromobjectsinotherclustersas
possible.Thergbimagesarefirstconvertedtol*a*bandtheapproachistochooseasmallsampleregion
foreachcolorandtocalculateeachsampleregion'saveragecolorin'a*b*'space.Weusethesecolor
markerstoclassifyeachpixel.
13

3.2CONVEXHULL
Inmathematics,andcomputationalgeometry,aDelaunaytriangulationforasetPofpointsintheplaneis
atriangulationDT(P)suchthatnopointinPisinsidethecircumcircleofanytriangleinDT(P).Delaunay
triangulationsmaximizetheminimumangleofalltheanglesofthetrianglesinthetriangulation;theytend
toavoidskinnytriangles.Convexhullgivesusthenonambiguousandefficientrepresentationofthe
requiredconvexshapeconstructed.

(25)
Thoughthiswasamethodwhichwillhelpusfindtheexactoutlineoftheobjectinaframebutitonlygave
usthepointswhichfellontheoutlineanditdidnothelpmuchtoachievetheexactmask.
4EXPERIMENTSANDRESULTS
4.1APPLICATIONOFVARIOUSALGORITHMSUSED
4.1.1ACTIVECONTOUR

Fig1.1

fig1.2

Inthefig1.1,animageistakewhoseobjectneedstobeextracted.Infig1.2themanwalkingisgraspedbythe
contouralongwithhisshadow.

Fig1.3

fig1.4

fig1.5

fig1.6

Fig1.3istheobjectobtainedafter180iterationswith=0.8, fig 1.4 is the object extracted after 180

iterations when =0.2. Fig 1.5 is the image of the object extracted after 1800 iterations with
=0.8, fig 1.6 is after 1800 iterations with =0.2 in the Chan Vese model of active contours.

14

Weobservethehigherthevalueofalpha,theloweritschancesoftheobjectboundarybeingdistorted.
Alsowenoticethatthenumberofiterationsalsoactivelycontributeinextractingtheobject.Thisisdueto
theenergyfunctionbeingminimizedandevolvedalargenumberoftimessuchthattheydistorttheoutput.
Thenthesurroundingbackgroundpixelsalsostartinvolvingintherobustalgorithm.

Fig1.7

fig1.8

Fig1.7isthemaskgiventotheChanVesemodelofactivecontourandfig1.8istheoutputobtained.

Ifwenoticecarefully,thecontourhasgrabbedonlythosepartsofthemaskwhicharenotveryilluminated
intheactualimage.

Fig1.9

fig1.10

Fig1.12

fig1.11

fig1.13

Figure1.9isthemaskobtainedfromStaufferGrimson[1],fig1.10,fig1.11,fig1.12,fig1.13aretheoutputsofactive
contoursrunatdifferentiterations.

Heretheoriginalmaskisbetterinthoseareaswhicharewellilluminated.Butthecontouriscoveringupall
theholesinthedarkerregionoftheactualregion.Thisshowsusthatactivecontourwillfailiftheinput
imagehasdifferentilluminationsontheobject.Sothekclustermethodshouldbeadoptedtosegmentthe
objectsimageandcombinethoseinwhichtheobjectbecomescompleteregardlesstheillumination
withoutincludingthebackground.
15

4.1.2CONVEXHULL

Fig2.1

Fig2.3

Fig2.5

fig2.2

fig2.4

fig2.6

Figure2showsustheimplementationofconvexhull.Fig2.1showsustheoriginalinputimage,fig2.2givesusthe
maskfromthetrackercode,fig2.3isthemorphologicallyimprovedimagemask,fig2.4istheforegroundextracted
whenfig2.1andfig2.3arefedtotheactivecontours,fig2.5isthecloseupversionoftheobject,fig2.6isthe
convexhullobtainedoftheobject.Note:fig2.6cannotbeusedtoretrievethebestpossiblemask.

4.1.3KmeansClusterSegmentation
Withfig2.1beingtheinput,weapplykmeansclustersegmentationandobtainthefollowingresults.

16

Fig3.1

Fig3.3

fig3.2

fig3.4

fig3.5

Fig3.1givesusthefirstsegmentedimageclusterwheretheobjectislowlyexposed,fig3.2givesthesecond
segmentedimageclusterwheretheobjectishighlyexposed,fig3.3givesusthethirdimageclusterwhichcomprises
ofthebackground,fig3.4isthecombinationofcluster1and2whichisgivenastheinputtoactivecontours,fig3.5
isthealmostperfectmaskobtainedfromtheimage.Note:Thedrawbackhereismissinghairinthefinalmask
obtained.

4.1.4ILLUMINATIONVARIATIONINAGIVENIMAGE

Fig4.1

17

fig4.2

Fig4.3

fig4.4

fig4.5

Inthesefigures,illuminationvariationhasbeenapplied.Fig4.1istheinputimage1,fig4.2istheimageobtainedby
processingimage1andsettingthevalueasnegativeforbrighterexposure;fig4.3,fig4.4,fig4.5areimages
obtainedwhilethevaluesof were positive.

4.1.5 Running of the Stauffer Grimson code with the AGC estimator

Fig 4
In fig 4, the top left corner has a grid which is divided into four parts where the top left grid shows the current frame
being processed, top right shows the image obtained after tracker algorithm was applied, bottom right shows the
output after the AGC estimator has rectified the mask by reducing the noise drastically and bottom left shows the best
background image; whereas the right image shows the transfer function graph of the current frame.

18

4.2 SAMPLE PROBLEM 1

Proper Evaluation of Stauffer Grimson with AGC estimator

Fig 4.2.1

Fig 4.2.2

Fig 4.2.3

Fig 4.2.4

Fig 4.2.5
In these figures the left side is the original frame and the right side is the mask obtained of the corresponding image.
Fig 4.2.1 the mask even has the shadow of the moving object creating a misconception of two objects being present,
fig 4.2.2 the object has merged with the shadow and the algorithm slowly gives a better output in fig 4.2.3. In fig
4.2.4 when the AGC becomes evident some artifacts start appearing in the mask. Fig 4.2.5 has intolerable noise level

19

which is due to the number of outliers exceeding the band threshold in the transfer function and the AGC more than it
can be compensated.
Application of Kmeans Cluster Segmentation

Fig 4.2.6

fig 4.2.7

Fig 4.2.8
`

Fig 4.2.9
fig 4.2.10
Fig4.2.6istheframeevaluated;fig4.2.7isthekmeansclustersegmentedimagewheretwooftheclustershave
beencombinedtomakesurethedarkandthelightpartsoftheobjectareincluded;fig4.2.8isthemaskobtained
fromthetrackercodewithAGCestimator;fig4.2.9isthefinalmaskobtainedbyusingtheoriginalframeand
previousmaskinactivecontourmodel;fig4.2.10isthefinalmaskobtainedbyusingthekmeansclusteredimage
andtrackercodemask

20

Comparison between backtracking and tracker code masks

Fig4.2.11

fig4.2.12

Infig4.2.11isthemaskobtainedbyfeedingthepreviousbettermasksintheactivecontouralongwiththecurrent
framewhichissegmentedbykmeanscluster.Fig4.2.12isthemaskobtainedbythetrackercodewithAGC
estimator.
Backtracking

Fig4.2.13

fig4.2.14

Fig4.2.13istheoriginalframeandfig4.2.14istheoutputobtainedbyapplyingbacktrackingwhichturnsouttobe
moreaccurate.Thelastmaskisfirstgivenastheinputtoactivecontouralongwiththeircorrespondingframeandthe
maskobtainedisiterativelyfedtotheactivecontourmodelwiththepreviousframe.Thismakessurethatthenoise
levelremainsincontrol.
Backtracking and K-means Clustering

Fig4.2.15

fig4.2.16

fig4.2.17

Fig 4.2.15 is the segmented image, fig 4.2.16 is the mask obtained via backtracking and fig 4.2.17 is the image mask
obtained when the algorithm is carried out sequentially from the beginning. Note: We can see the improvement in the
backtracking algorithm.

21

Shortcomingofthealgorithm

Fig4.2.18

fig4.2.19

Fig 4.2.18 is the mask obtained from the algorithm till where the noise is tolerable, fig 4.2.19 is the mask obtained
towards the end when AGC is too high.

We will notice that when a lot of pixels exceed the bandwidth of the transfer function graph, the algorithm
is not able to compensate for the gain and a distorted mask is obtained which even segmentation is not able
to overcome.

Fig 4.2.20

fig 4.2.21

Fig4.2.20isthemaskobtainedwhenthesegmentedclustersarenotchosenandanytworandomonescombinesto
givetheinputimagetoactivecontourmodel,fig4.2.21isthemaskobtainedafterselectingtheclusters.

22

CONCLUSIONSANDFUTUREWORK
Wehaveapproachedtowardsdevelopinganoveltechniqueforextractingthebestpossiblemaskfromthe
framesofavideo.Thekclustermethodusedhasgivenbetterresultsthanthepreviousmethodsexisting.
Buttheshortcomingisthatitrequiresuserinputforselectinganytwoclusterscompletingtheobject.The
kmeansclustersegmentationtakesO(n)timeandcanbeusedinrealtimeprograms.Furtherworkcanbe
doneonthisfieldsoastomakeitmorerobustandlessdependentontheuserinputs.Wenoticethatthe
noiseisintolerablewhentheAGCisveryhigh,failingthesegmentationwithcontourmethodinsuchcases.
Wealsonoticedthatconvexhullwouldnotbeabletohelpusinregainingthemasksastheyonlygive
somepointslyingontheoutlineoftheobjectmakingitmorevulnerabletodistortions.Another
shortcomingofthekmeansclustersegmentationisthatifthecoloroftheobjectmatcheswithsomeofthe
backgroundcomponentsthentheactivecontourwillalsograspotherartifactsdistortingthefinalmask
obtained.Wealsoinferthatthereisaneedformorphologicaloperationsincasethebackgroundismore
naturalwithalotofotherobjects.
Wehaveshownthatweareabletoproducesignificantimprovementsinthemasksbyapplyingactive
contourstotheclustersegmentedimages.Moreover,thecomputationalcomplexityistakenonlybythe
activecontouralgorithm.Infuture,wecanbuildupanalgorithmwhichwillclustertheimageinsucha
waythattheshadowalongwithotherstaticobjectscanbedetectedandeliminatedfromthefinalmasks.

23


BIBLIOGRAPHY
[1]Grimson,W.,&Stauffer,C.(1999).Adaptivebackgroundmixturemodelsforrealtimetracking.IEEE
ComputerSocietyConferenceonComputerVisionandPatternRecognition(CVPR'99).
[2]Agrawal,A.,&Raskar,R.(2007).GradientDomainManipulationTechniquesinVisionandGraphics.
InternationalConferenceonComputerVision2007course.
[3]T.ChanandL.Vese,Activecontourswithoutedges,IEEEtransactionsonimageprocessing
10(2)(2001),pp.266277
[4]S.Nandagopalan,Dr.B.S.Adiga,C.Dhanalakshmi,RDCS(USA)FIAE,FASE,N.Deepa:Automatic
SegmentationandVentricularBorderDetectionof2DEchocardiographicImagesCombiningKMeans
ClusteringandActiveContourModel.In:IEEESecondInternationalConferenceonComputerand
NetworkTechnology,2010.(447451)
[5]ChristofRidder,OlafMunkelt,andHaraldKirchner.AdaptiveBackgroundEstimationandForeground
DetectionusingKalmanFiltering,ProceedingsofInternationalConferenceonrecentAdvancesin
Mechatronics,ICRAM95,UNESCOChaironMechatronics,193199,
1995.
[6]Perez,P.,Gangnet,M.,&Blake,A.(2003).Poissonimageediting.SIGGRAPH2003,(pp.313318)
[7]TianciHuang,JingBangQiu,TakeshiIkenaga,"AForegroundExtractionAlgorithmBasedon
AdaptivelyAdjustedGaussianMixtureModels,"ncm,pp.16621667,2009FifthInternationalJoint
ConferenceonINC,IMSandIDC,2009
[8]P.KaewTraKulPongandR.Bowden,AnImprovedAdaptiveBackgroundMixtureModelforReal
timeTrackingwithShadowDetection,Proc.2ndEuropeanWorkshoponAdvancedVideoBased
SurveillanceSystems,AVBS01,2001.
[9]KhabouMA,HermiL,RhoumaMBH(2007)ShaperecognitionusingeigenvaluesoftheDirichlet
Laplacian.PatternRecognit40:141153
[10]Bhat,P.,Curless,B.,Cohen,M.,AndZitnick,L.,2008.Gradientshopimageandvideoprocessing.
http:/www.GradientShop.com,May2008
[11]S.ChandranandA.K.Potty,Energyminimizationofcontoursusingboundaryconditions.IEEE
Trans.PatternAnal.MachineIntell.205(1998),pp.546549.

24