Detection of Left Luggage and Theft

Instructor: Prof. Amitabha Mukerjee (amit@iitk.ac.in ) Mitesh Gupta

Dept of Computer Science and Engineering Indian Institute of Technology Kanpur

Shishir Jain
Dept of Computer Science and Engineering Indian Institute of Technology Kanpur

mitesh@iitk.ac.in Abstract: In recent years the demand on

video analysis application such as video surveillance is growing rapidly. Video surveillance is commonly used in security systems, but requires more intelligent and more robust technical approaches. Such systems, used in airports, train stations or other public spaces, can bring security to a higher level. In this context, we present a simple method to detect left luggage and theft in public places. We first detect moving objects by background subtraction. Then heuristic is used to detect owners and luggage items left by their owners. Mean shift algorithm is used to track objects. An alarm is triggered when the person moves too far away from his luggage.

shijain@iitk.ac.in Definitions
Left luggage: Left luggage is defined as the items that had been abandoned by their owner. [1] Attended and Unattended luggage: 3 rules to identify the attended luggage [1, 2]: The luggage is owned and attended to by a person when it enters the scene with the luggage until such point that the luggage is not in physical contact with the person. At this point the luggage is attended to by a person if the person is less than a meter from the luggage. The luggage item is unattended if the owner is further than b meter (b>=a) away from the luggage. The region in between a and b meters is referred as warning zone where the luggage is neither attended to nor left unattended.

Motivation
Increasing need of security in public places leads to the detection of suspicious things while they occur rather recording them up. Surveillance applications are developed such that they can view a wide area through multiple cameras, typically in the public areas which are susceptible to be crowded. Tracking people in crowded place is a big problem due to occlusion.

Theft: The luggage has been attended by the third party and it moves the luggage b meters away from the owner. [1]

Our Approach
Fig 1.a We present an algorithm to detect abandoned luggage and left luggage in real world public environment. We will use PETS [1] datasets for testing purposes. The following figure (Fig 2) shows our surveillance process.

Fig 1.b

Fig 1.c Fig 2 Background subtraction: For background subtraction we have used foreground detection technique (Implementation of [4]) which internally uses another algorithm for change detection as described in [5]. In this the model is updated for a certain set of data referred as training set and after this model is further not updated. So if a person is static it will be

Fig 1.d Abandoned luggage: The luggage which has been abandoned by the owner for consecutive t seconds. [1, 2]

in foreground for a sufficient amount of time.

height ratio should be between 0.3 to 0.8 and minimum height of 120 pixels and minimum width of 40 pixels. For a blob if the height to width ratio is matched with the pre defined ratio than that blob can be considered as luggage/person. After luggage is identified then the nearest blob (having width to height ratio in between 0.3 to 0.8 and minimum height of 120 pixels and minimum width of 40 pixels) can be considered as the owner of the luggage. This assumption helps us in simplifying our model. Distance is measured in 3D using the camera calibration data available with the dataset.

Fig 3.a: A sample image (luggage and foreground object enclosed in rectangle)

Mean Shift approach for tracking: This approach is mainly based on maximization of a similarity measure (Implementation of [6]). Mean Shift algorithm as implemented in OpenCV [7] was used which iterates to find the object center given its back projection and initial position of search window. The iterations are made until the search window center moves by less than the given value and/or until the function has done the maximum number of iterations. In this method the color histogram of the object is calculated and is used to obtain back projection of the image. The calculated back projection is given as an input to the mean shift algorithm.

Fig 3.b: Background subtraction of Fig 3.a

Heuristic approach for detection of owner and its luggage: We have defined height to width ratio (1.1) for the luggage range of height (55, 75) pixels and range of width (60, 85) pixels. For a blob to be blob of a person width to

subtraction openCV [7] implementation of Foreground Detection Algorithm [4] was used. Various parameters were adjusted to obtain good results. Parameters for this module are given in the CvFGDStatModelParams struct: typedef struct CvFGDStatModelParams { int Lc, N1c, N2c, Lcc, N1cc, N2cc, is_obj_without_holes, perform_morphing; float alpha1, alpha2, alpha3, delta, T, minArea; } CvFGDStatModelParams; Fig 4.a: A sample image with foreground object enclosed in rectangle.
Parametersforforeground detection
MY_BGFG_FGD_LC MY_BGFG_FGD_N1C MY_BGFG_FGD_N2C MY_BGFG_FGD_LCC MY_BGFG_FGD_N1CC MY_BGFG_FGD_N2CC MY_BGFG_FGD_ALPHA_1 MY_BGFG_FGD_ALPHA_2 MY_BGFG_FGD_ALPHA_3 MY_BGFG_FGD_DELTA

Values Default wehave Values used 128 15 25 64 25 40 0.1f 0.005f 0.1f 2 0.9f 15.f 128 20 30 64 30 45 0.1f 0.005f 0.3f 2 0.9f 50.f

Fig 4.b: Color histogram of the foreground object in Fig 4.a

MY_BGFG_FGD_T MY_BGFG_FGD_MINAREA

Simulation
Dataset was obtained from the PETS 2007 website [1] and our approach was simulated on Dataset S8. Firstly, a background subtraction [4] was performed on the images. For doing the background Then on the obtained blobs heuristic approach (as described in the section above) was used to identify person and object in the images. Also, a tracking is initiated on a new person or object as it is detected. When a luggage is identified the person closest to

the luggage is made owner of that luggage then in all subsequent images the distance between the luggage and its owner is measured and checked again the criteria described for left/unattended luggage.

Results
Run sequence from Image No. 1000 to 1150 is as shown:
OurResult Lugg age dete cted N N Y N Y Y GroundTruth Left Luggag luggag Left e luggage e detecte detect detect ed ed d N N N N Y Y N Y Y Y Y Y N N N N N Y

Fig 6: Showing left luggage in our results (image no. 1099)

ImageNo. 10001025
1026-1041 1042-1081 1081-1098 1099-1111 1112-1150

Here we have assumed that when owner leaves 2 meter circle left luggage is detected.

Fig 7: Showing the ground truth when there is left luggage (image no. 1112)

Conclusions
Foreground Detection algorithm used for background subtraction gives good results but a lot of fine tuning for various parameters is required. A better technique for automatically adjusting the parameter depending on the time of the day may be a good approach. Also the heuristic approach used to identify person, owner and luggage item can not be rely upon and a more sophisticated technique is needed. Occlusion

Fig 5: Showing the scene when the luggage is detected in our results (image no. 1042)

is a problem in tracking and a better technique for handling occlusions is needed. Also the algorithm for background subtraction is very time inefficient and can not be used in real time. A more efficient technique for background subtraction is required. Using our approach the accuracy is low but can be improved by doing an improved tracking on the objects.

[6] Y. Cheng. Mean shift, mode seeking, and clustering. IEEE Transactions on Pattern Analysis and Machine Intelligence, 17(17), 1995. [7] OpenCV. Open source computer vision library: http://sourceforge.net/projects/opencvlibrary/

Acknowledgement
We are highly thankful to Prof. Amitabha Mukerjee for granting us an opportunity to work under this project. We are thankful to the OpenCV community.

References
[1] PETS 2007, http://pets2007.net/ , as on October,2007 [2] PETS 2006, http://www.cvg.rdg.ac.uk/PETS2006/data.html , as on October,2007 [3] Edouard Auvinet, Etienne Grossmann, Caroline Rougier, Mohamed Dahmane and Jean Meunier, Left-Luggage Detection using Homographies and Simple Heuristics, In Proceedings 9th IEEE International Workshop on PETS, New York, 2006, pages 51-58 [4] Liyuan Li, Weimin Huang, Irene Y. H. Gu and Qi Tian, Foreground object detection from videos containing complex background , Proceedings of the eleventh ACM international conference on Multimedia,2003, Berkeley, CA, USA, pages 2-10 [5] P.Rosin, Thresholding for Change Detection, ICCV, 1998, pages 274-279