0 Voturi pozitive0 Voturi negative

6 (de) vizualizări12 pagini.

Aug 22, 2013

© Attribution Non-Commercial (BY-NC)

PDF, TXT sau citiți online pe Scribd

.

Attribution Non-Commercial (BY-NC)

6 (de) vizualizări

.

Attribution Non-Commercial (BY-NC)

- 08.13.Intelligent Scissors for Image Composition
- An Overview of Image and Video Segmentation
- Digital Image Processing 2
- 133_dragut_blaschke_geomorphology.pdf
- 3 Color Image Segmentation Using Distance Regularized Level Set
- Contour Free Level Set Method
- khashman2007.pdf
- IJ 281 Anbeek MSlesion Article Revised
- Across Expertise Exploitation Results Reality
- 1-a-staier
- 04_Segmentation_P1
- Rajeshwar
- IJAIEM-2014-06-16-36
- Using Compression Techniques to Streamline Image and Video Storage and Retrieval
- 20160421
- 1-s2.0-S2212017313003538-main
- Computing Theory (OCR GCSE)
- Advanced Applied Adjustment of Function Form of Thought to Assign Anything Clear Exactly True by Said Mchaalia
- 10.1.1.153.4666
- IJIREEICE3G a ragini RECOGNITION OF VEHICLE NUMBER PLATE USING MATLAB (1).pdf

Sunteți pe pagina 1din 12

Signal Processing Laboratory, Swiss Federal Institute of Technology, CH-1015 Lausanne, Switzerland Email: jordan@ltssg3.epfl.ch Tel: ++41 21 693 70 88 Fax: ++41 21 693 70 90

This paper presents a technique to segment and track a region of interest in a video sequence. The presented technique starts with a spatio-temporal contour extraction. This contour extraction is performed by mixing information from a spatial gradient and a temporal di erentiator. A block segmentation is then applied on the image, and within each block the contour part is approximated by a straight line segment. These contour approximations are then further quantized by a vector quantization technique, using a prede ned codebook. In order to close and smooth the region of interest contour, an interpolation based on Hermite polynomials is performed on the set of discrete quantized contour parts. Finally, the tracking process is performed using an addition-removal of blocks containing contour parts by testing their possible belonging to the polynomial contour. This testing is done by using a blurred version of the polynomial contour in order to be able to track deformations in the object. The initial guess for tracking is given manually by an end-user.

ABSTRACT

Keywords: Contour extraction, quantization, and interpolation; block based; tracking; deformable

objects; supervised segmentation; Hermite polynomials; shape coding; video processing.

1. INTRODUCTION

With the development of new multimedia applications and the future video standard MPEG-4, new image processing algorithms must be developed in order to reach high compression ratios as well as to provide various functionalities, such as the detection and manipulation of moving objects in video sequences. Classical block-based compression schemes which have been developed in the past can not easily provide the required functionalities, since this kind of segmentation has no way to detect or follow the real moving contours existing in the scene. Nevertheless, block-based approaches have widely proved their e ciency both in the rate distortion trade-o 1 and reasonable computing time 2 . This paper presents a way to upgrade a block-based coding scheme into a region-based one. Similar approaches can be found in 3, 4 . In order to reduce the blocking artifacts introduced by a pure block-based approach and to improve the quality of the predicted image, several techniques to split blocks into two or more regions have been proposed in 5, 6, 7, 8 . These motion eld re nement techniques de ne a new non-pure block-based segmentation. The block splitting is usually done using a prede ned codebook which describes the

patterns in which a block can be sub-segmented. Such a codebook is a vector-quantized representation of all possible ways to split a block in several sub-regions. It has been shown in 5, 6 that the new segmentation obtained by such a block splitting technique is more correlated to the contours existing in natural images, the latter being a by-product of the motion eld re nement which was originally designed for compression purposes by means of reduction of the displaced frame di erence DFD entropy. In this paper we propose a method which allows to de ne and track in time a closed region in a sequence. The closed region should correspond to a real moving object in the scene. To initialize the process, a manual selection of the region of interest is performed by the user. Then, the contours are extracted from this region of interest and approximated by a list of line segments. This list gives a rough approximation of the contour of the object we intend to segment and track. The tracking is performed by means of a matching process of line segments between the current frame and the previous one. Finally, the closed contour is computed using a Hermite polynomial. The implementation of the presented segmentation and tracking technique has been made with realtime constraints in a software-based system on a massively parallel super-computer. For this reason the proposed segmentation technique has been designed with the following additional constraints: Low computation load Backward compatible with already existing block-based codecs. This paper is organized as follows: Sec. 2 gives a brief overview of the complete technique and motivations for the performed investigations. The detailed description of the various tools used is given in later sections. First, Sec. 3 describes the spatio-temporal contour extraction scheme which has been developed. Then, Sec. 4 describes the technique used to quantize the contour parts. Sec. 5 explains how the quantized contour parts are interpolated in order to nd smooth and closed contours. Finally, Sec. 6 deals with the developed object contour tracking technique. Simulation results are shown and discussed in Sec. 7 and nally, Sec. 8 gives some concluding remarks.

2. DESCRIPTION OF TECHNIQUE

The motion eld re nement technique by means of block splitting proposed in 5 was based on properties of the DFD. This enabled to minimize the DFD energy, in other words to decrease its entropy, thus to increase the compression ratio for a given nal quality of the decompressed sequence when compared with a pure block-based technique like MPEG-2 for example. It has been shown in 5, 6 that the segmentation re nement of the blocks can roughly be correlated with moving object boundaries. The major drawback of this segmentation technique is the heavy computational cost. Thus it can not directly be used for the real-time constrained implementation we have in mind. In the proposed technique, a combined spatial and temporal contour extraction technique is chosen as a starting point. The obtained contours are then used to de ne the line segments. These line segments are taken from the prede ned codebook. In more details, the result of the contour extractor is split up into blocks. Thus, we can have a small piece of contour in some of the blocks. These small pieces of contour are then approximated using the best tting line segment that can be found in a given codebook. These codebook-based approximated contour segments are then used as guide points and slopes for a Hermite polynomial interpolator, which is used in order to produce a smooth closed contour of a region of interest in the sequence. This region of interest is de ned by the end-user, since he is the only one who is able to de ne what region he wants the system to extract from the sequence and track through time. The region tracking process is performed by means of contour tracking. More precisely, the contour segments are matched with the polynomial contour. A decision process adds or removes some of the

control blocks where interesting vector quantized contour parts are located. This adding and removing process allows the presented technique to track deformable objects through time.

3. CONTOUR EXTRACTION

The contour extraction technique we used in this work is based on a combined spatial and temporal contour extractor. The idea is to use a combination of a gradient-based spatial contour extractor and a temporal frame di erentiator in a weighted sum in order to get spatio-temporal contours. The spatial contour Si;j of pixel at location i; j is obtained by means of a gradient based contour extraction: Si;j = jIi,1;j , Ii+1;j j + jIi;j ,1 , Ii;j +1j : 1 The temporal contour Ti;j of pixel at location i; j is computed as the di erence between its value in the current frame and the one in the previous frame:

t , I t, : Ti;j = Ii;j i;j

1

2

Finally the spatial Si;j and temporal Ti;j components of our contour extractor are combined together in a weighted sum in order to get our spatio-temporal contour Ci;j : + Ti;j : Ci;j = Si;j + 3

The weights and are a description of the tradeo between spatial and temporal contours. Typically we used = 8 and = 2.

The vector quantization of the contours obtained after contour extraction is done in two steps. First, the contour part contained in a block is approximated by a straight line segment. In a second step the codebook element which is geometrically the closest to the straight line segment is chosen.

To approximate the contour within a given block with a straight line, we look for two contour points on the boundaries of the block, as illustrated in Fig. 1. Contour points are simply points with high gradient values. Thus, the Ci;j values Eq. 3 located in the gray shaded area of Fig. 1 are scanned for maximum values. One maximum location is selected per block side. The two biggest of the four maximas are chosen in order to nd the coordinates i; j k1 and i; j k2. These two coordinates give the two end points of the straight lines we intend to nd. This technique ensures us not to take two end points on the same block side, which would lead to an unusable result.

In order to reduce the transmission cost for the segmentation parts, the straight lines approximating the contours are quantized using a vector quantization technique. In such a technique a codebook describing a sub-set of all possible block segmentations is used on the coder and the decoder side. Thus, for each block, a code describing its segmentation is transmitted; a zero code indicating that the block is not part of the contour. Fig. 2 gives an example of a codebook.

(i , j )k1

(i , j )k2

If ek;l denotes an element of the codebook for the block at coordinate k; l, and since the block splitter used to generate the codebook used straight lines to separate a block in two regions, ek;l can be described by the end points i; j e1 and i; j e2 of the straight line segment. In order to select the closest codebook element for a given line approximated contour, a minimization of a distance function is performed. The distance d between two lines given by their end points is given by:

1 1 2 1 1 2 2 2 2 2 2 2

4

For each block of the image which contains a contour part, the minimization process of done using all the codebook elements. The result of this process is a block-based description of the contours, as shown in Fig. 3.

5. CONTOUR INTERPOLATION

The segments which have been computed in Sec. 4 represent a rough de nition of the object shapes as can be seen in Fig. 3. In order to de ne a closed region, it is necessary to connect" the segments. One way to achieve such a task consists in de ning the contour as a polynomial ps. The proposed method approximates contours using multiple Hermite polynomials. The curve is drawn between pairs m; m + 1 of consecutive blocks which belong to the contour and that we call control blocks". Each of these control blocks uses the line segment orientation of the codebook element to evaluate the derivative of the polynomial, as described in Fig. 4. The center i; j m of a segment m is called the control point" of the polynomial and de ned as: ip2 ; jp1 + jp2
: i; j m = ip1 + 5 2 2 The Hermite polynomials are computed for each pair of blocks m; m +1 using two constraints which are the derivatives and the control points. The two derivatives are both multiplied by the distance D between the two control points. This allows to adapt the derivative constraint when two control points are not equally spaced. Thus, when control points are distant, we can impose a hard constraint on the derivative. When the control points are close, better results are obtained when the derivative constraint is lowered. Actually this gives a rough control on the maximum curvature of the contour. The derivative

( i , j ) ( s) (i , j )m + 1 d (i , j )m + 1 ds

(i , j )m l d (i , j )m ds

Figure 4: Contour interpolation using Hermite polynomials of a polynomial can now be written as: di; j m = D ip1 , ip2 ; jp1 , jp2
; where l, the segment length is de ned as:

q

1 2 2

ds

6 7

l = ip , ip + jp , jp :

1 2 2

Each of the i; j s coordinates can now be written as a Hermite polynomial: ps = as3 + bs2 + cs + d: 8 The coe cients a, b, c and d of the Hermite polynomial can be found for is and j s using the following constraints:

Fig. 5 shows an example of a region closed using such a polynomial after line and codebook approximation of contours. It may be noticed that this technique can also be used for shape coding, as shown in 9 .

+1 +1

9

6. TRACKING

The object at time t is de ned with a list Lt of codebook elements and the corresponding k; l block coordinates: Lt = fk; lr g t: 10 The tracking process is initialized with a manual drawing of the region of interest, as illustrated in Fig. 6. The shape of the region of interest is updated from frame to frame using two processes: a removal of segments from the list according to a motion criteria and an addition of new segments to the list. Fig. 7 shows the processes.

L(t )

+ +

L ( t +1)

+ -

L ( t +1)

Interpolation

Contour Extraction

From frame to frame, and for each block de ning the contour, the distance d de ned in Eq. 4 between two codebook elements ek;l belonging to this block at time t , 1 and time t is computed. A threshold T is applied to this distance to decide whether the element should be removed or kept in the new list L:

11

The criteria then enables the tracking of objects whose motion is small lower than T . Since it is applied to each element of L, it allows the tracking of deformable object through time. This process creates a new list L0t +1, which has always less elements than the previous list Lt. With this removal technique, noisy elements or elements which move out of the current block are removed from the list:

12

In order to not completely destroy the list of elements, an additive" process is needed, which will add new segments to the new list Lt + 1. These elements will be created either from the motion of an element from another block or if a new contour appears. To select new elements, an assumption on the smoothness of the object's shape is done: if the smoothness is su cient, the polynomial ps created with L0 t + 1 should approximate correctly the shape. For this reason, we use the polynomial ps to select new blocks k; l where an element to be added to the list L0 t + 1 can be found. The blocks k; l whose codebook elements ek;l belong" to the polynomial ps constructed with Lt are added to the new list to create:

13

To give the belonging" property better chances than a trivial dual choice, it must be de ned such that it also works for contour approximations and not only perfect matches. The rst process in the belonging" computation is to get a blurred version of the polynomial contour in a block. First the polynomial contour is drawn in white on a black background, as shown in Fig. 5. We denote with P 0 the black and white image of the polynomial ps in one block, which is de ned by the following rule:

8m;

P0 =

255; 0;

14

The blurred version of the polynomial image in one block, denoted by P , is obtained by applying a 55 kernel operator. For the same block as for P 0 and P , we denote by E the black and white image of the codebook element line ek;l de ned for the block r. We can now de ne two values a and b de ning the sums of matched and unmatched contour pixels within the block r:

a =

b =

i;j 2r

X

Pi;j + Ei;j ;

i;j 2r

jPi;j , Ei;j j:

15

It is now possible to de ne an estimation of the belonging" or matching of codebook element ek;l with the polynomial ps within a given block with a normalized value c: where c simply expresses how many pixels have the same location between a codebook element line and the polynomial part within a given block. Fig. 8 shows an example of the superposition of a blurred polynomial contour and a codebook element line segment in a block.

b; c= a, a

16

Figure 8: Codebook polynomial superposition used for matching process in a 1212 block.

7. SIMULATION RESULTS The simulations have been made with a 352288 8 bits LTS" video sequence, with 1212 blocks. A

region is selected at frame 200. Fig. 9 shows the extracted region girl" for frame 201 and frame 225 of the test sequence LTS". The frame 225 shows the result after recursive tracking of 25 frames. Fig. 10 shows the extracted region hand" for frames 204, 213 and 225 of the test sequence LTS". The list Lt has been updated 25 times between these two frames, using the above described process. Fig. 11 shows in details the updating of the list Lt. During the tracking, it is possible to notice that the number of elements which de ne the shape does not vary very much, which indicates a good equilibrium between removed and added elements. The number of removed and added elements is irregular, which shows that the updating process is working correctly.

8. CONCLUSION

The segmentation technique described in this paper allows to obtain a joint detection and segmentation of objects on the basis of a block-based segmentation coding scheme. Since each stage of the algorithm works only at block level instead of the usual pixel level, the computational load remains very low. In practice, it requires about 0.2 second per image to obtain the segmentation and tracking on a Impact" Silicon Graphics workstation. The presented technique allows a smooth transition from a block-based to an object-based video processing approach, thus achieving the backward compatibility constraint. The quantization of extracted contours, combined with the contour reconstruction by means of Hermite polynomials allows a low cost

a

b

Figure 9: Extracted regions from the sequence LTS" at frame: a 201, b 225.

a

b

c

Figure 10: Hand tracked in the sequence LTS" at frame: a 204, b 213, c 225.

40 Number of elements

10

0 0 10 20 Frame number 30

Figure 11: Variation of added, removed and total number of elements in list Lt. representation of the contours in terms of data transmission. Furthemore, the contour parts updating scheme allows tracking of moving and deformable objects in video sequences.

9. REFERENCES

1 F. Dufaux, I Moccagatta, B. Rouchouze, T. Ebrahimi, and M. Kunt. Motion compensated generic coding of video based on multiresolution data structure". Optical Engeneering, Vol. 32, No. 7, pp. 1559 1570, July 1993. 2 H. Nicolas and F. Jordan. Interactive optimization and massively parallel implementations of video compression algorithms". In IEEE International Conference on Image Processing, Vol. I of III, pp. 229 232, Lausanne, Switzerland, September 1996. ICIP'96. 3 R. Nohre. Fragmentation-based image coding". Electronics Letters, Vol. 31, No. 11, pp. 870 871, 1995. 4 S. Desmet, B. Deknuydt, L. Van Eycken, and A. Oosterlinck. Initial segmentation of a scene using the results of a classi cation based motion estimator". In IEEE International Conference on Image Processing, Vol. I of III, pp. 559 562, Austin, Texas, November 1994. ICIP'94. 5 I. Moccagatta, F. Moscheni, M. Schutz, and F. Dufaux. A motion eld segmentation to improve moving edges reconstruction in video coding". In First IEEE International Conference on Image Processing, Vol. III of III, pp. 751 755, Austin, Texas, 1994. ICIP'94. 6 I. Moccagatta, F. Moscheni, M. Schutz, and F. Dufaux. A VQ-based motion eld re nement technique for image sequence coding". In European Symposium on Advanced Networks and Services, Conference on Compression Technologies and Standards for Image and Video Communications, Vol. 2451, pp. 156 167, Amsterdam, March 1995. SPIE.

7 F. Dufaux, I. Moccagatta, F. Moscheni, and H. Nicols. Vector quantization-based motion eld segmentation under the entropy criterion". Journal of Visual Communication and Image Representation, Vol. 5, No. 4, pp. 356 369, December 1994. 8 C. Deutsch, A. Zaccarin, Orchard, and T. Michael. Estimation theoric approach for robust predictive motion eld segmentation". In Proceedings of the SPIE - The International Society of Optical Engineering, Vol. 2186, pp. 210 221. SPIE, 1994. 9 C. Le Buhan, F. Bossen, S. Bhattacharjee, F. Jordan, and T. Ebrahimi. Shape representation and coding of visual objects in multimedia applications - An overview". To be published in Annales des Telecommunications, special issue on MPEG-4.

- 08.13.Intelligent Scissors for Image CompositionÎncărcat deAlessio
- An Overview of Image and Video SegmentationÎncărcat dePatricia Perrone
- Digital Image Processing 2Încărcat devenchinn4676
- 133_dragut_blaschke_geomorphology.pdfÎncărcat deMade Nandini
- 3 Color Image Segmentation Using Distance Regularized Level SetÎncărcat deVenkat Raman
- Contour Free Level Set MethodÎncărcat deInternational Journal for Scientific Research and Development - IJSRD
- khashman2007.pdfÎncărcat deAbdul Saboor Khan
- IJ 281 Anbeek MSlesion Article RevisedÎncărcat deaibk01
- Across Expertise Exploitation Results RealityÎncărcat deAnonymous 8erOuK4i
- 1-a-staierÎncărcat desaleemnasir2k7154
- 04_Segmentation_P1Încărcat deDoi Thay
- RajeshwarÎncărcat deRupesh Sushir
- IJAIEM-2014-06-16-36Încărcat deAnonymous vQrJlEN
- Using Compression Techniques to Streamline Image and Video Storage and RetrievalÎncărcat deCognizant
- 20160421Încărcat denaaz_pinu
- 1-s2.0-S2212017313003538-mainÎncărcat deRajib Deyana
- Computing Theory (OCR GCSE)Încărcat deOrkryx
- Advanced Applied Adjustment of Function Form of Thought to Assign Anything Clear Exactly True by Said MchaaliaÎncărcat deAnonymous 8erOuK4i
- 10.1.1.153.4666Încărcat deNguyễn Trọng Duy
- IJIREEICE3G a ragini RECOGNITION OF VEHICLE NUMBER PLATE USING MATLAB (1).pdfÎncărcat deMuhammad Azhar Iqbal
- Topics in Signal Processing Volume 10 - Computer Vision Analysis of Image Motion by Variational Methods (2014) [Amar Mitiche, J.K. Aggarwal]Încărcat deAchilles
- PinakiÎncărcat deMD SHAHJAD
- UCLA Image ProcessingÎncărcat deDhruv Ramani
- Eiko Nal Diffusion i Jc mÎncărcat deCarmen Haidee Pelaez
- 20120316Încărcat deSenapathi Teja
- Customer Lifecycle ValueÎncărcat deamsom121212
- 04368197Încărcat deAdemar Alves Trindade
- CHARACTER RECOGNITION BASED ON SKELETONIZATION: A SURVEY.Încărcat deIJAR Journal
- Vipimage Bernard 2013Încărcat deozanoktay_imperial
- cnn hssÎncărcat deVishwanath Shervegar

- 64 Interview QuestionsÎncărcat deshivakumar N
- WERA-2008Încărcat deDeni Chan
- OKR GuideÎncărcat depapyrus test1
- UTM-directory.-June-2018.-v1.pdfÎncărcat deDeni Chan
- Reflections on Trusting TrustÎncărcat demanjunathbhatt
- SQL TuningÎncărcat detaiwo44
- Preliminary Study in Managing Safe and Efficient Low-Altitude UAV OperationsÎncărcat deRajat Rajgaria
- p761 ThompsonÎncărcat deDeni Chan
- 2010 June NodeXL Book FlyerÎncărcat deDeni Chan
- Danelec ECDIS Whitepaper 160216 WebÎncărcat deDeni Chan
- Ids Odbc BookmapÎncărcat deDeni Chan
- ECDIS Legal Aspects NumericChart ENC RNC - CopiaÎncărcat deBalasi Simona
- marineÎncărcat deSatish Gupta
- InformixPerformanceTuning-ArtKagel_TakeIIÎncărcat deDeni Chan
- Performance Assessment of DC-free Multimode CodesÎncărcat deDeni Chan
- Analog vs Digital FilterÎncărcat dePavan Kumar
- Crane SlidesÎncărcat deDeni Chan
- ADMIRALTY Digital Catalogue HelpÎncărcat deDeni Chan
- BATOS Weather Station, Provence Sea IOPsÎncărcat deDeni Chan
- java_dip_tutorial.pdfÎncărcat derphmi
- Defence Software TrainingÎncărcat deDeni Chan
- 07Încărcat deDeni Chan
- An LP-based Algorithm for the Data Association Problem in MTTÎncărcat deDeni Chan
- Lc07 SL Estimation.ppt_0Încărcat deDeni Chan
- 8570Încărcat deDeni Chan
- 42 ExtendedÎncărcat deDeni Chan
- 18Încărcat deDeni Chan
- 3_datafus_kalman_selfloc_v08_ajsÎncărcat deDeni Chan

- File_InfoÎncărcat deanimationeasy
- Tese - Dissertação de doutorado - Thales Teixeira - University of MichiganÎncărcat deMarcelo Augusto Mascarenhas
- CS1354- GRAPHICS AND MUTIMEDIA IMPORTANT TWO MARKSÎncărcat deSumathi Bas
- An Introduction to MPEG Video CompressionÎncărcat dethuhienptit2003
- COMPRESSION final.pptxÎncărcat deBalakrishnan Natarajan
- Mc 1701 Computer NetworksÎncărcat deSantosh Kumar Thiyagarajan
- Image Compression meidcal preodceÎncărcat dekrrishshrish14
- DptÎncărcat deCường Mậm
- 54284953 Linear Prediction for Speech EncodingÎncărcat deRamana Reddy
- Mac OSXÎncărcat desafadga
- RT41043022018.pdfÎncărcat deJanardhan Ch
- Ncct Vb Net Project Titles 2008 - 2009 - Latest, New, InnovativeÎncărcat deesskayn16936
- BE Computer EnggÎncărcat deMani Kant
- Final Year Project IdeasÎncărcat deMuhammad Ashraf
- Sole DadosÎncărcat deumetani
- 2_FPGA Implementation of Pipelined 2D-DCT and Quantization Architecture for JPEG Image Compression.Încărcat deMahesh Mahi
- ADSP 10 AC Psycho Acoustics EC623 ADSPÎncărcat deAli Mirza
- lsb stegoÎncărcat deSreekanth BG
- Zip ScriptsÎncărcat deRaj Mohan
- Paper-4 a Novel Methodology Proposed for Compact & Secure Logging System Towards Cyber ForensicÎncărcat deRachel Wheeler
- Cie as Computer Science 9608 ZnotesÎncărcat deNouhiya Khadaroo
- FactSheetWashington State University CSÎncărcat deAnmol Gupta
- Poser 8 ReadmeÎncărcat deCarlos De Luise Saurrë
- Sony DSR 2000 BrochureÎncărcat deJJ2cool
- Compiler DesignÎncărcat deGagAn SinGh
- Deleted FilesÎncărcat deGiorgioMontesi
- 1814621 - Faq Dbif_rsql_sql_error Short Dump, Ora-14400 ErrorÎncărcat deSreeni Choudary
- mtech-vlsiÎncărcat deMukesh Maheshwari
- Advance JstegÎncărcat deJaideep Singh Rathore
- Image Processing and Pattern MatchingÎncărcat deharitha_1225

## Mult mai mult decât documente.

Descoperiți tot ce are Scribd de oferit, inclusiv cărți și cărți audio de la editori majori.

Anulați oricând.