An Intelligent Object Inpainting Approach For Video Repairing

International Journal of Advance Foundation and Research in Computer (IJAFRC)
Volume 1, Issue 5, Ma !"1#$ I%%& !'#( ) #(5'

## * + !"1#, IJAFRC All Ri,hts Reserved ---$i.afrc$or,
An Intelli,ent /0.ect Inpaintin, Approach For Video
Repairin,
Mr.Abhijeet A.Chincholkar.Prof.Salim A.Chavan.
PG Student, M.E. Digital Electronics, D!C"E# $avatmal, Maharashtra, %ndia.
Associate Professor and &ice Princi'al, D!C"E# $avatmal, Maharashtra, %ndia.
chincholkarabhijeet(gmail.com,salimsahil)*(rediffmail.com

A 1 % 2 R A C 2
2his paper improves techni3ue to speed up video Inpaintin, for Video Repairin,$ 2his approach
reduces search area 0 usin, !4 slices rather than '4 patches$ In this method RA&%AC al,orithm
for 5stimation of 2ransformation 0loc6, 782 Al,orithm for feature trac6in, and Ma9 compiler is
used to compile C pro,rammin, and reduces processin, time$ 2o overcome over)smoothin,
pro0lem method finds 0est match to ,ive the ,ood results in less time$ 2his method also tries to
maintain spatial consistenc and temporal motion continuit simultaneousl of an o0.ect$
:omemade video includes .itter (unintended motion) and some 6ind of intentional motion such
as pan or ;oom$ Vie-ers preferred al-as to see .itter free videos, created 0 smoothl movin,
camera$ %ometimes in video analsis contrast, ali,nin, to fi9 sta0le 0ac6,round is preferred$ 2his
paper presents simple al,orithm -hich is a0le to remove 0oth tpes of motion 0 usin, an
efficient -a to trac6 0ac6,round points -hile i,nore movin, fore,round points$ 2his approach is
mostl similar to ima,e mosaicin, 0ut result is video rather than an enlar,ed ima,e$ It is related
-ith multiple o0.ects trac6in, approach 0ut 3uite simpler due to movin, o0.ects trac6in, is not
needed$ 2he al,orithm receives a video input and returns one or multiple sta0ili;ed videos$ Video
0rea6s into parts -hen al,orithm detects 0ac6,round chan,e and it fi9 upon a ne- 0ac6,round$

Inde9 2erms <) Feature 59traction, Ima,e Mosaicin,, Feature %election, Video %ta0ili;ation, Matri9
2ransformation, 4e,ree of Fredom $

I$ I&2R/4=C2I/&

#here are t+o t,'es of motions +hich dominate the videos taken b, a 'erson +ith a hel' of hand-held
camera. .irst is intentional motion such as 'anning or /ooming and second are unintentional motions
+hich are un+anted shakes and jitters. Some recent +ork on video stabili/ation tries to remove shakes
and jitters +hile smoothing the most 'robabl, intended motion. #his is done b, firstl, identif,ing
common features in the scene and estimating frame-to-frame movement of these features. #hen,
algorithms +ork as follo+s0 12 #akes estimation of original camera motion, 32 fit 'ro'osed model for
ca'turing smoothed most 'robabl, intended camera motion, 42 solve for transformation matrices for
each frame merging them +ith ne+ stabili/ed video +ithout jitter and a''arentl, smooth camera
motion.

%n recent +ork done the same a''roach is b, Grundmann et al.567, demonstrate the conte8t of $ou#ube a
robust algorithm for stabili/ing the video to 'roduce a final video +hich a''ears a shot due to im'ro'er
handling b, a 'rofessional cameraman. %n some situation, associated +ith video anal,sis, +here goal
stabili/es a video in res'ective of its background and accordingl, removes clear observed camera motion
entirel,. Man, reasons are here to remove camera motions such as stabili/ation of videos +hich enables
background modeling5167, makes object9s motion more clear to human and observers both. #here are
some modes +here goal of removing camera motion is not 'ractical. :here cameras are mounted on
moving vehicles, more com'lications are involves multi'le object tracking and camera modeling is
Volume 1, Issue 5, Ma !"1#$ I%%& !'#( ) #(5'

#5 * + !"1#, IJAFRC All Ri,hts Reserved ---$i.afrc$or,
re;uired. %n the same manner this +ork on stabili/ation, +orks incor'orates +ith detection of salient
features 567, <ucas and =anade tracking56,117, >A!SAC 567 to test 'otential feature corres'ondences, and
at last motion modeling is done in terms of frame-to-frame alignment transforms 567. #he best t+o
as'ects in this algorithm are such as, it introduces staged multi-frame mechanism to introduce ne+
salient features in tracking 'rocedure and takes decisions that +hether a salient 'oints are 'art of stable
background based on measurements over multi'le frames. #ransformation matri8 that registers frame t
to the background is com'uted using features initiall, detected in frame t - 3. #hese features are tracked
to frame t - 1 b, using <ucas-=anade #omasi algorithm and their motion is com'ared +ith camera
motion at t - 1. #hose features +hich move consistentl, +ith camera and 'otential background 'oints are
for+arded to frame t, +hereas features +ith inconsistent motion are 'resumed to be foreground features
and are dro''ed. >esult consider transformation matri8 at time t is estimated b, a''l,ing >A!SAC
algorithm and motion modeling to set of features on background 'oints and have 'ersisted for t+o time
ste's. >es'ectivel, transformation at time t ? 1 is estimated from features 'reviousl, e8tracted at time t -
1. #his t,'e of feature selection 'rocess adds robustness +hich 'rovides more 'ositive a''roach to+ards
the 'rocedure. >esult sho+s that this method sort registration errors better than the techni;ues +hich
are 'reviousl, +orked. #he most 'rior +ork follo+s b, some difference relative directl, from goal of
stabili/ing relative to a fi8ed background. %n this a''roach, final ste' of generating the stabili/ed video
establishes common reference frame and ma''ing of all frames back to this reference one. %n this 'rocess
of ma''ing videos to a common reference frame for video re'airing is mostl, done as image mosiacing
547. #his 'a'er enlist some 'oints as follo+s0 - revie+ed video stabili/ation and image mosaicing related
+ork, our video stabili/ation algorithm, an e8'erimental evaluation com'aring this method +ith
alternative methods based on 'rior +ork for video stabili/ation and image mosaicing algorithms in the
literature.

II$ R58A254 >/R7

#he main aim of &ideo Stabili/ation is to remove un+anted motion. %t is almost im'ossible for an,
general 'erson to hold camera and ma, not be introduce small but ra'id movements such as jitters.
&ideo Stabili/ation algorithms are often takes an estimate of smooth camera motion 'ath from video
frame data 5117. Grundmann et al. 567 'resented a robust method for finding an <1- o'timal Camera 'ath
to generate stabili/ed videos. asicall, this algorithm is based on <inear Programming frame+ork +hich
finds o'timal frame 'artitions of smooth camera 'ath. Path modeling itself having fitting 'ortions of the
'ath to constants, linear and 'arabolic motion models. A cro''ed +indo+ of fi8ed as'ect ratio @A>2 is
moved along +ith that o'timal 'ath to include salient 'oints or regions +hen minimi/ing <1 smoothness
constraint. #he main goal is that to stabili/e video such that it might have been shot b, a 'rofessional
cinema 'hotogra'her using e8'ensive 'h,sicall, stabili/ed cameras. Such a video is feeling 'leasant for
human vie+ers.

III$ /V5RVI5> C/MM/&

Previous +ork on image alignment or image mosaicing is more closel, related to goal of creating
stabili/ed backgrounds is closer than the recent +ork on video stabili/ation for smooth camera motion
trajectories. "bjectives of both +ork shares common as'ects such as e8tracting matching features in
successive frames. .eatures like harris corners 547, S%.# features or Good features 513, A7 can be used for
feature selection. .eatures matching techni;ues as correlation of a +indo+ centered about to feature
'oint547, or tracking methods as <ucas =anade #racker 5137 or =alman .ilters 5A7 are mostl, used for
tracking features from frame to frame. .rame-to-frame motion models also 'la, a vital role in the same
algorithm. Man, #ransformations of degrees of freedom @D".2 are common, together +ith likeness
Volume 1, Issue 5, Ma !"1#$ I%%& !'#( ) #(5'

#? * + !"1#, IJAFRC All Ri,hts Reserved ---$i.afrc$or,
transform @A D".2 5147, affine transform @6 D".25*7 or homogra'h, transform @B D".25A,4,137. Such
transformations are regularl, estimates from the matching features bet+een 'airs of video frames. :hen
E8'eriment takes 'lace it is observed that balance must be struck bet+een the abilit, to model is more
com'le8 forms +here the need to find more matching features is sim'le. #o constrain the additional D".,
further higher D". motion models are used due to their much larger s'ace of 'ossible solutions and
sometimes the, are susce'tible to settling on unrealistic frame-to-frame motions. A major 'roblem in
this algorithm is that ho+ to decide +hich matched features is to be use. %f scene containing inde'endent
moving objects then all feature are not matches bet+een frames and that are associated +ith the
dominant motion. .eature 'oints on inde'endentl, moving object if included in calculation of the
dominant motion then estimate +ill thro+ off. #hen it is com'ulsor, to describe feature 'airs b, inliers
or outliers relative to motion estimation. #o se'arate outliers and inliers, >A!SAC 547 or <MeS @<east
Median of S;uares2 5137 are mainl, used.

IV$ %@5CIFIC @RI/R 5FF/R2%

>eal-time scene video stabili/ation demonstrated b, Cansen et al. 5*7, #heir &.E-1DD s,stem introduced
a multi-resolution iterative method using the <a'lacian ',ramid images. Each iteration contain o'tical
flo+ estimation is done b, using cross correlation of current image and 'revious image at 'articular
',ramid level. A linear motion model is fit to o'tical flo+ and 'revious image is merged +ith that model.
%n ne8t iteration o'tical flo+ estimation is done in bet+een merged 'revious image and the current one
image at higher resolution level of ',ramid. Morrimoto and Chella''a 5147 'ro'osed fast and robust
im'lementation of 3-D electronic image stabili/ation. #his method selects the features on hori/on b,
thresholding and dividing <a'lace of image into vertical /ones and selecting to'most feature of ever,
/one. #he selected features are tracked from ft - 1 to the frame ft b, a multi-resolution scheme and also
involving <a'lacian ',ramids. feature 'oints at ever, ',ramid level is in frame ft is searched over
+indo+ centered about 'oint in frame ft - 1 and 'oint +hich returns the minimum SSD is finest match.
Estimation obtained at coarse level is used to search for minimum SSD at finer level of <a'lace ',ramid.
At last <east s;uare solution is used for estimation of the similarit, matri8 from corres'onding 'oint.
Motion matrices are then combined from reference frame to current frame and current frame is merged
+ith accumulated motion model.

An image mosiacing algorithm em'lo,ing these techni;ues b, Ca'el and Eisserman547. %n that s,stem a
+indo+-based locali/ed correlation score used to match harris corners in bet+een t+o consecutive
frames. >A!SAC discards outlier 'oints and estimate homogra'h, that +as +ell defined matched inliers.
At last estimation of corres'onding 'oints and homogra'h, +ere advanced using non-linear o'timi/er
b, minimi/ing euclidean distance bet+een original feature 'oints and corrected feature 'oints of
corres'ondences. %n that method cost +as minimi/ed using the <evenberg-Mar;uardt algorithm. Censi et
al. 5A7 a''roaches for image mosaicing +ith the feature tracking. #his algorithm tracks best features 51*7
in each subse;uent frames of a se;uence using linear =alman filter. Predicted 'osition of feature 'oint is
e8tracted from 'redicted state of =alman filter and neighborhood of 'redicted 'osition is searched for
minimum SSD@sum of s;uare difference2 errors to find corres'onding feature 'oints. #his s,stem used
robust rejection rule 8BA 5F7 to identif, the outliers. #he residual of each feature is calculated and the
feature +hose residual differs b, more than F.3A MAD from the median residual discards an outlier.

Another image mosaics e8am'le +as used in +ide area surveillance. Mei et al. 5137 'resented background
retrieval s,stem +hich detect Good .eatures in the reference frame and tracks them over subse;uent
frames using =anade-<ucas-#omasi 51*7 tracker algorithm for obtaining frame-to-frame
corres'ondences. Comogra'h, estimation from those corres'ondences and <east Median of S;uares
Volume 1, Issue 5, Ma !"1#$ I%%& !'#( ) #(5'

#A * + !"1#, IJAFRC All Ri,hts Reserved ---$i.afrc$or,
@<MeS2 algorithm are used to remove outliers. Mi8ture of Gaussians @MoG2 5167, 'ro'oses background
modeling 'rocedure to se'arate stable background from 'oints of interest.

Ceikkila et al. 5B7, 'ro'oses an automatic image mosaicing method for the +ide area surveillance. %n that
method e8tracts the S%.# features from incoming images and constructs a mosaic image. >A!SAC method
used to reject outliers and 'arameters of homogra'h, are filtered b, using <evenberg Mar;uardt
algorithm for minimi/ing geometric cost function defined over inliers. %n another e8am'le of %mage
Alignment S%.# .lo+, <iu et al. 5)7, 'ro'osed s,stem +hich uses ',ramid based on discrete flo+
estimation algorithm to match S%.# descri'tors bet+een t+o consecutive images and defines
neighborhood for S%.# flo+. Dual la,er loo', believes 'ro'agation used to minimi/e objective functions
at each ',ramid level, +hich constrains S%.# descri'tor to matched flo+ vector and constrain flo+
vectors of adjacent 'i8els to be alike. #his s,stem9s 'rimar, estimates corres'onding images of different
scene categories. Co+ever it uses for registration of similar scenes satellite images.

V. A8IB&M5&2 2/ A C/MM/& R5F5R5&C5

Alignment of frames in video com'aring +ith reference frame +ill results in video +here frames
a''eared motionless e8ce't borders +here content background changes 537. #+o issues are arises +hen
alignment to reference frame. 12 Accumulating the errors in motion estimation can lead to unacce'table
errors relative to reference frame. 32 Amount b, +hich ne+ frame overla's to reference frame can gro+
too small. Man and Picard 51D7, solves this t,'e of 'roblem b, s'litting frames into a subsets +hich can
be best registered. Com'aring these algorithms that align video to common reference frame involves
both evaluating breaks and ;ualit, of aligned subsets of frames. Morimoto and Chella''a 51A7, 'ro'osed
fidelit, measure +hich corres'onds to Peak Signal to !oise >atio @PS!>2 in bet+een the stabili/ed
consecutive frames. #he PS!> is function of Mean s;uared error +hich is average de'artureG'i8el from
desire stabili/ed result.

VI$ %2A1I8ICA2I/& A8B/RI2:M%

#he 'revious algorithms described here take one video in'ut and return oneGmore videos, each of them
+ill be stabili/ed +ith res'ect to stationar, reference frame. %n another manner, goal is for background to
remaining fi8ed throughout +hich returned video. %n'ut videos broken +hen the camera motions so
alters objects visible in background. #here is no longer 'ossibilit, to align a video +ith a common
reference. Criterion for breaking video is discussed further. .our algorithms are described further. Each
carries out broadl, three o'erations such as0 12 .eature e8traction and feature ma''ing bet+een the
consecutive frames, 32 .rame to frame motion estimation from corres'ondences and 42 Motion
com'ensation. %n general these ste's are described further.

VII$ Frame to Frame Motion 5stimation

%n general, camera motions are estimated from feature 'oint corres'ondences. .eatures 'oints are
e8tracted, selected, and matched across frames, and then affine motions bet+een consecutive frames are
calculated from these tracked 'oints. .or frames ft and ft - 1 the affine transform ma, be e8'ressed as0

_
Xt
t
1
_ = _
A1,1 A1,2 A1,S
A2,1 A2,2 A2,S
u u 1
_ _
Xt -1
t -1
1
_

Volume 1, Issue 5, Ma !"1#$ I%%& !'#( ) #(5'

#( * + !"1#, IJAFRC All Ri,hts Reserved ---$i.afrc$or,
#his is a standard choice of motion model. #he 6 degrees of freedom affine has si8 unkno+n 'arameters,
and three 'oint corres'ondences generate si8 linear constraints e;uations. #hus match bet+een an,
three features in one frame to ne8t, t,'icall, define an alignment transformation. :hen BD".
e8'eriments +ith homogra'hies, +e found that the result sho+s +orse than the 6 D". transform.

>A!SAC algorithm is used to find affine transformation su''orted b, majorit, features. After each
iterations of >A!SAC, three 'oint-+ise corres'ondences are elected at random and used them for
estimation of affine motion matri8. Estimated motion matri8 checks ho+ man, 'oints are lies in
consensus set. %f 'ercentage of 'oints that are fits estimated affine model is more than threshold method
sto's and declares that model is good. #he 'oints those +ere fit this model kno+n as inliers and the
remaining are kno+n outliers. After obtaining inliers from >A!SAC algorithm, method 'erforms a linear
least s;uare solution for obtaining final affine inliner matri8.

>A!SAC algorithm 're assumes 'oint +ise corres'onding feature 'oints in bet+een t+o frames. %t can be
done b, t+o +a, of finding corres'onding match feature 'oints used in algorithms as follo+s. "ne is base
on feature similarit, means match measure similarit, bet+een feature 'oints e8'ressed in the feature
s'ace. #he same a''roach is used b, Ceikkila et al. 5B7, S%.# features to describe S%.# 'oints is used.
"ther is a tracker to move 'oints for+ard from frame t - 1 to the frame t. Mei et al. 5137, used <ucas
=anade #omasi feature tracker 51B7. Algorithm belo+ uses an iterative ',ramidal im'lementation of =<#
base on o'tical flo+ to 'rovide robustness to large dis'lacements 517.

VIII. M/2I/& C/M@5&%A2I/&

%n this a''roach one frame is taken as reference frame and ne8t frame is registered to this reference
frame. Su''ose if Ci re'resents affine transform bet+een the frames i and i - 1 and if reference frame is
CD, then frame i can be ma''ed to first frame b, com'osition of transformations0

C1HH.i I C1, C3, C4HHHCi E;uation 1

Given motion model of frames i +ith res'ect to reference frame, then this method merge frame i using
inverse of C1H.i. Merging frames means the 'rojection of each 'i8el coordinate to source image b,
motion matri8 to ne+ coordinate in destination image. .or estimated 'i8el value in destination image
sam'ling is done in reverse mode +hich is going from destination to source. .or each 'i8el @8, ,2 of
destination image functions com'ute coordinates of corres'onding Jdonor 'i8elK, in source image and
co', 'i8el value such as mentioned belo+.

dst @8,,2 I src @f8 @8,,2, f, @8,,22 E;uation 3

:here f8 @8,,2 and f, @8,,2 are merging functions. Since f8 @8,,2 and f, @8,,2 are seldom integers, bilinear
inter'olation +hich is used for obtaining ne+ inter'olated 'i8el values. :hen f8 @8,,2, f, @8,,2 or both fall
outside the source image, then dst@8,,2 is set to /ero.

IX. CRI25RIA F/R 1R5A7I&B VI45/%

#his algorithm introduces breaks in to t+o events. &er, first if e8cessive amount of 'anning is detected
means +here majorit, of non-visible 'i8els in reference frame, then break is created. Such a t,'e of break
is triggered b, monitoring fraction of 'i8els in current frame that lies outside reference frame +hen
ma''ed back to reference frame using accumulated motion model @C1H..i2. At certain condition this
Volume 1, Issue 5, Ma !"1#$ I%%& !'#( ) #(5'

#D * + !"1#, IJAFRC All Ri,hts Reserved ---$i.afrc$or,
algorithm t,'icall, set to break video, if more than half of the total 'i8els in the current frame are lies
outside reference frame. And in another condition this triggers for break as 'er e8cessive scaling in
transformation matri8. #his t,'e of trigger makes the sense in this domain because this algorithm +orks
+ith camera 'an but not +ith /oom inGout during single video. #riggers are im'lemented b, monitoring
determinant of accumulated motion model C1H.i. %f ratio drifts too far from 10D, means it start scaling
background relative to reference frame then break is created. %n general for this an algorithm, if
determinant falls outside range D0)F to 10DF then video are broken. %t is observed in e8'eriment that,
determinant of motion matri8 lies ver, close to 10D for frame to frame estimates. #he determinant al+a,s
remains +ithin 1 L M, +here M is in order of 1D-A. %t is also observed in cases +here errors accumulated
in accumulated motion model C1H..i, as 'er 'revious discussion and starting +ith a ne+ reference
becomes necessar,. E8'eriments test revealed +hich is used Comogra'h, transform +hich tends to
cause Jske+K at large instance and 'ers'ective errors. Due to this testing for these errors in turn leads for
more breaks. #he 'rocess becomes length, for stabili/ing videos using techni;ues in 5B,137, +hich used
for homogra'h, transforms. Co+ever, +e solve this led to large number of video breaks, and
subse;uentl, changed the im'lementations b, using affine models.

E$ @roposed Al,orithm

#he intelligent algorithm introduced here uses a t+o staged 'rocess to refresh set of S%.# feature 'oints
that are tracked using the =<# algorithm. Motivation behind this design a ne+ s,stem for background
in'ainting for video re'airing b, the use of <ucas-=anade #omasi algorithm and =<# #ransform, >A!SAC
algorithm and homogra'h, transform s'eedu' the in'ainting a''roach. %n this algorithm com'utational
need to create 'air +ise corres'ondences from scratch in each successive frame is solved. %t de'ends on
initiali/ation ste' +here 'oints +hich are to be tracked that established based on first frames of video. As
tracking 'roceeds, set of 'oints being tracked and shrinks in si/e. Points are getting dro''ed +hen =<#
algorithm cannot establish +ith confidence in ne+ 'osition in a ne+ frame. %f the scene got change then,
there is no mechanism for refreshing tracked 'oints.

Fi,ure 1< 4ifferent sta,es involves in video Inpaintin, for Video Repairin,$

Old Old
Inliers Inliers

H
Frame Fresh
Points Points

Fi,ure !< %in,le frame Ro0ust %ta,e RA&%AC 2rac6er Al,orithm.
KLT RANSAC Filter Inliers
KLT
KLT
%n'ut &ideo
Pre'rocessing
Model Sam'ling
Model Alignment
Motion Com'letion
&ideo %n'ainting
International Journal of Advance
5" * + !"1#, IJAFRC All Ri,hts Reserved
#his algorithm solve both 'roblems through an iterative 'rocess +hich constantl, refreshes grou' of
S%.# features +hich are being tracked. .igure 3 sho+s algorithm for a singl
'oints enters in frame in each iteration0 12 Set of inliers e8tracted from t+o frames 'revious to current
frame and 32 Set of fresh 'oints e8tracted from 'revious frame. %nliers t from 'revious frame is to be
used b, >A!SAC to estimate motion of current frame +ith res'ect to 'revious frame. "nce motion is
estimated then inliers are totall, discarded. .resh 'oints are then tested for com'atibilit, +ith estimated
motion bet+een current and 'revious one frame and 'oints +hich are c
on ne8t frame as ne+ set of inliers. .resh a''licant of S%.# features +ere e8tracted from current frame
and 'assed it as fresh 'oints to ne8t frame. Cere recent algorithm is described as >obust Staged >A!SAC
#racking algorithm. .or initiali/ation tracking algorithm in .igure 3, +e need bootstra' frame, S%.#
features e8traction is done from initial frame. =<# Algorithm u'dates 'osition of these features in ne8t
frame and >A!SAC com'ute motion and se'arate inliers from outlier
frame as inliers and ne+ set of S%.# features are e8tracted from first frame and 'assed as staged fresh
'oints for second frame.

Fi,ure '< Ro0ust %ta,ed RA&%AC 2rac6in, Al,orithm -ith motion compensation (:)$
EI$ 5E@5RIM5&2A8 R5%=82%

%n this method for each 'air of images it sorts frame to frame 'i8el differences b, magnitude and take
sum of all differences belo+ the median. #hose 'i8els are lies outside the video frame, are ignored. #his
t,'e of errors is normali/ed b, number of 'i8els +hich are a 'art of video content of the t+o frames. #his
error condition is slightl, com'le8 because score checks ho+ +ell the background is to be reconstructed
+hile ignoring the 'i8el differences occurred due to moving objects. !
accumulated for consecutive frames, e8ce't there are breaks and 'airs in frames across registration
+here breaks are ignored. Cere different threshold levels are used +hen creating breaks in video. Circled
'oints indicate default algorithm configurations +hich 'oints need to be transfer. #his algorithm checks
+hether the determinant of accumulated motion model C1H.i, falls outside the range D0)F to 10DF then
those errors does not accumulate in accumulated motion model. :hen range
numbers of breaks are increases. #he frames +hich are lies in bet+een breaks are finel, registered and
have lo+er overall errors. At a broader range, breaks are decrease but overall error scores are increase.

EII$ C/&C8=%I/& A&4 F=2=R5 %C/@5

#his 'a'er 'resents an intelligent object based video in'ainting techni;ue for video re'airing +hich
stabili/es video relativel, +ith fi8ed background. Cere staged multi frame video alignment method
Volume 1, Issue 5, Ma !"1#$

+ !"1#, IJAFRC All Ri,hts Reserved
S%.# features +hich are being tracked. .igure 3 sho+s algorithm for a single frame. #+o set of features
estimate motion of current frame +ith res'ect to 'revious frame. "nce motion is
motion bet+een current and 'revious one frame and 'oints +hich are consistent +ith motion are 'assed
thm. .or initiali/ation tracking algorithm in .igure 3, +e need bootstra' frame, S%.#
frame and >A!SAC com'ute motion and se'arate inliers from outliers. %nliers are 'assed to second ne8t


/ed b, number of 'i8els +hich are a 'art of video content of the t+o frames. #his
+hile ignoring the 'i8el differences occurred due to moving objects. !ormali/ed errors scores are
t algorithm configurations +hich 'oints need to be transfer. #his algorithm checks
those errors does not accumulate in accumulated motion model. :hen range
%C/@5
.rame
D
NE8tract S%.#
.rame
1
N#racked S%.# Points
NE8tract S%.#
.rame
3
N#racked %nliners
NE8tract S%.#
.rame
4
N#racked %nliners
NE8tract S%.#
Foundation and Research in Computer (IJAFRC)
Volume 1, Issue 5, Ma !"1#$ I%%& !'#( ) #(5'
---$i.afrc$or,
e frame. #+o set of features
estimate motion of current frame +ith res'ect to 'revious frame. "nce motion is
onsistent +ith motion are 'assed
thm. .or initiali/ation tracking algorithm in .igure 3, +e need bootstra' frame, S%.#
s. %nliers are 'assed to second ne8t
/ed b, number of 'i8els +hich are a 'art of video content of the t+o frames. #his
ormali/ed errors scores are
t algorithm configurations +hich 'oints need to be transfer. #his algorithm checks
those errors does not accumulate in accumulated motion model. :hen range of C is smaller, then
Volume 1, Issue 5, Ma !"1#$ I%%& !'#( ) #(5'

51 * + !"1#, IJAFRC All Ri,hts Reserved ---$i.afrc$or,
efficientl, tracks the inliers and tr,ing for maintaining number of tracked 'oints ma, not decrease over
time. #his techni;ue used first frame as the reference frame. %n future, challenge +ould be to select a
reference frame d,namicall, +hich o'timi/es number of breaks and stabili/ation error. As a result, frame
to frame motion is estimated and aligned more strongl, than in 'revious techni;ues. %n this techni;ue
algorithm tries to decrease the com'utation time. ut it is not suitable in such a domain, +here camera is
mobile or continuousl, /oom-inGEoom-out. %n such a critical condition, a goal of registering frame to
fi8ed reference frame is not suitable.

Fi,ure #< Inliners plot summari;ed result analsis of three sample videos -here camera motions are
unintentional$

Volume 1, Issue 5, Ma !"1#$ I%%& !'#( ) #(5'

5! * + !"1#, IJAFRC All Ri,hts Reserved ---$i.afrc$or,

Fi,ure 5< Mer,in, frames summari;ed result analsis of three sample videos of camera unintentional
motion$

2a0le 1<) @erformance Analsis of Al,orithm Results of 2hree %ample Videos$

EIII$ R5F5R5&C5%
517 %n 1))), O. $. ouguet 'ro'osed P,ramidal im'lementation of lucas-kanade feature tracker.
#echnolog, >e'., %ntel Cor'oration, Micro'rocessor >esearch <abs.
537 %n 3D13, M. ruder, G. A. >oitman and . Cernuschi-.rias 'ro'osed robust methods for
background e8traction in video. Argentine S,m'osium on Artificial %ntelligence, 'ages B4P)F.
547 %n 1))B, D. Ca'el and A. Eisserman 'ro'osed Automated mosaicing +ith su'er-resolution /oom.
%n Com'uter &ision and Pattern >ecognition, Proceedings. %EEE Com'uter Societ, Conference on,
'ages BBFPB)1.
5A7 %n 1))), A. Censi, A. .usiello and &. >oberto 'ro'osed %mage stabili/ation b, features tracking. %n
%mage Anal,sis and Processing, Proceedings. %nternational Conference on, 'ages 66FP66*.
5F7 %n 1)B6, P. O. >. .rank >. Cam'el, Elve/io M. >onchetti and :. A. Stahel 'ro'osed >obust Statistics0
#he A''roach ased on %nfluence .unctions. :ile, Series in Probabilit, and mathematical
statistics.
Calculated Parameters
Sam'le
&ideo 1
Sam'le
&ideo 3
Sam'le
&ideo 4
!umber of %terations taken b, Algorithm 36 3A 34
Ma8imum %nliners found in .rames 1B 4D 3D
Minimum %nliners found in .rames * 1A *
Ma8imum !umber of >eference frame %nliner 'oints )) )* 13D
Minimum !umber of >eference frame %nliner 'oints )) )6 11)
Ma8imum #ime taken to creat 'oints on >eference frame D.DF D.D6 D.DF
Minimum #ime taken to creat 'oints on >eference frame D.D3 D.D3 D.D4
Ma8imum !umber of Current frame %nliner 'oints 113 113 161
Minimum !umber of Current frame %nliner 'oints *B )D 141
Ma8imum #ime taken to creat 'oints on Current frame D.DA D.DA D.D6
Minimum #ime taken to creat 'oints on Current frame D.D3 D.D3 D.D4
Ma8imum #ime taken to match frames D.DD1 D.DD16 D.DD43
Minimum #ime taken to match frames D.DDD* D.DDDB D.DD14
Volume 1, Issue 5, Ma !"1#$ I%%& !'#( ) #(5'

5' * + !"1#, IJAFRC All Ri,hts Reserved ---$i.afrc$or,
567 %n 3D11, M. Grundmann, &. =+atra, and %. Essa 'ro'osed Auto-directed video stabili/ation +ith
robust o'timal camera 'aths. %n Com'uter &ision and Pattern >ecognition @C&P>2, %EEE
Conference on, 'ages 33FP343.
5*7 %n 1))A, M. Cansen, P. Anandan, =. Dana, G. &an der :al, and P. urt 'ro'osed >eal-time scene
stabili/ation and mosaic construction. %n A''lications of Com'uter &ision, Proceedings %EEE
:orksho', 'ages FAP63.
5B7 %n 3DDF, M. Ceikkil and M. Pietik ainen 'ro'osed an image mosaicing module for +ide-area
surveillance. %n Proceedings of third ACM international +orksho' on &ideo surveillance Q sensor
net+orks, 'ages 11P1B.
5)7 %n 3D11, C. <iu, O. $uen, and A. #orralba 'ro'osed Sift flo+0 Dense corres'ondence across scenes
and its a''lications. Pattern Anal,sis and Machine %ntelligence, %EEE #ransactions on, 44@F20)*B P
))A.
51D7 %n 1))F, S. Mann and >. :. Picard 'ro'osed &ideo orbits of the 'rojective grou'0 A ne+
'ers'ective on image mosaicing.
5117 %n 3DD6, $. Matsushita, E. "fek, :. Ge, R. #ang, and C.-$. Shum 'ro'osed .ull-frame video
stabili/ation +ith motion in'ainting. Pattern Anal,sis and Machine %ntelligence, %EEE
#ransactions on, 3B@*2011FDP1164.
5137 %n 3DDF, R. Mei, M. >amachandran, and S. =. Ehou 'ro'osed &ideo background retrieval using
mosaic images. %n %nternational Conference on Acoustics, S'eech, and Signal Processing, volume
3, 'ages AA1PAAA.
5147 %n 1))6, C. Morimoto and >. Chella''a 'ro'osed .ast electronic digital image stabili/ation. %n
Pattern >ecognition, %EEE Proceedings of the 14th %nternational Conference on, volume 4, 'ages
3BAP 3BB.
51A7 %n 1))B, C. Morimoto and >. Chella''a 'ro'osed Evaluation of image stabili/ation algorithms. %n
Acoustics, S'eech and Signal Processing, Proceedings of %EEE %nternational Conf. on, vol.F, 'ages
3*B)P3*)3.
51F7 %n 3DDD, C. Stauffer and :. E. <. Grimson 'ro'osed <earning 'atterns of activit, using real-time
tracking. Pattern Anal,sis and Machine %ntelligence, %EEE #ransactions on, 33@B20*A*P*F*.
5167 %n 1))1, C. #omasi and #. =anade 'ro'osed Detection and tracking of 'oint features. School of
Com'uter Science, Carnegie Mellon Sniversit,.
51*7 %n 1))A, C. #omasi and O. Shi Pro'osed Good features to track. Proceeding %EEE CS Conference
Com'uter vision and Pattern >ecognition, Pages F)4-6DD.

An Intelligent Object Inpainting Approach For Video Repairing

Încărcat de

Informații document

Drepturi de autor

Formate disponibile

Partajați acest document

Partajați sau inserați document

Opțiuni de partajare

Vi se pare util acest document?

Este necorespunzător acest conținut?

Drepturi de autor:

Formate disponibile

An Intelligent Object Inpainting Approach For Video Repairing

Încărcat de

Drepturi de autor:

Formate disponibile

International Journal of Advance Foundation and Research in Computer (IJAFRC)

Volume 1, Issue 5, Ma !"1#$ I%%& !'#( ) #(5'

S-ar putea să vă placă și