International Journal of Graphics JOURNAL and Multimedia (IJGM), ISSN 0976 AND 6448(Print), INTERNATIONAL OF GRAPHICS ISSN 0976

6 6456(Online) Volume 4, Issue 1, January - April 2013, IAEME

MULTIMEDIA (IJGM)

ISSN 0976 - 6448 (Print) ISSN 0976 -6456 (Online) Volume 4, Issue 1, January - April 2013, pp. 20-30 IAEME: www.iaeme.com/ijgm.asp Journal Impact Factor (2013): 4.1089 (Calculated by GISI) www.jifactor.com

IJGM
IAEME

ENTROPY FEATURES TRAINED SUPPORT VECTOR MACHINE BASED LOGO DETECTION METHOD FOR REPLAY DETECTION AND EXTRACTION FROM SPORTS VIDEOS
Vilas Naik1, Raghavendra Havin2 1 Department of CSE, Basaveshwar Engineering College, Bagalkot, India 2 Department of CSE, Basaveshwar Engineering College, Bagalkot, India

ABSTRACT In many sports, the majority of highlights are confined to relatively short durations of intense action. In some sense these segments capture the essence of a game and summarize the moments of important action. Automatic detection of these highlights could provide an important browsing mechanism in a video library of sports games. These replays often correspond to highlights in a game and can be used as indices of a sports video. The proposed mechanism employs support vector machine (SVM) for detection of logos that are flashed at beginning and end of every reply action. The algorithm is composed of logos detection and replay segment extraction. First SVM is trained with features of all possible logos normally used in various sports videos. Then the SVM is used for detection of logos that sandwich replay segment further that segment is automatically extracted to produce replay clip. The SVM classifier is trained with histogram features of logos is utilized. Experiments conducted on IPL and Soccer videos demonstrate the effectiveness of this method. Moreover, algorithm can be easily applied to other sports videos where replay is sandwiched by pair of logos. Keywords: Logo-based detection, Support vector machine (SVM learning (ML), Sports replay detection, Video summarization. 1. INTRODUCTION

In a sports video library, one may find thousands of hours of recordings, representing many hundreds of individual games. Fortunately, in any given game, not every minute commands the same interest. A majority of the excitement is contained in a small fraction of the coverage of the game. These highlights are the essence of the game and present a succinct
20

International Journal of Graphics and Multimedia (IJGM), ISSN 0976 6448(Print), ISSN 0976 6456(Online) Volume 4, Issue 1, January - April 2013, IAEME summary of the significant actions and events. For many viewers of sports video these highlights are all they are really interested in seeing. Detection of these highlights is therefore of interest in itself, but might also serve as an important indexing mechanism for searching for more substantial sections of material from large archives. With the rapid increase of media data, it is in urgent need of an efficient and effective method for information management and retrieval. Semantics based event detection and indexing has been attracted much attention recently for interesting events are highly useful for video browsing, indexing and highlights generation. Replay is an important video editing way in broadcasting programs. Commonly, an important or interesting segment will be played with a slow-motion pattern in order to let audiences explore and enjoy the detail. Replay is a significant indication for highlights and often taken as a key factor in event detection. The sports video summarization is widely investigated since sports holds a large amounts of audiences worldwide. As an important compiled clue, the replay shows the details of an important video segment with a slower speed. Thus it is widely employed in sports video analysis especially for event detection, highlight creation and summarization etc. Replays are related to the sports, which are shown more than once when any important event in sports takes place. To get more information from replay, generally it is shown with slow motion. Hence it is also called slow motion replay (SMR). Replay is which is necessary for highlights generation. Highlight is a shorter version of sports video that is also called as sports video summarization. Replay is important for all spectators and referees to judge and take final decision in most of the sports. When an important event happen like goal in football or hockey cricket. When referee is in doubtful condition, or strong appeal is made by players, putting all such events in an original temporal order we can have highlights of any sports video. Replay contains all-important events. Replays are used for highlight making. Replays can be used as a summary of game for searching it in digital video library. Replay is an important video editing way in broadcasting programs. Commonly, an important or interesting segment will be played with a slow-motion pattern in order to let audiences explore and enjoy the detail. Replay is a significant indication for highlights and often taken as a key factor in event detection. In recent years, researchers have reported many approaches of replay detection in literatures. In this paper, an automatic and effective logo based replay detection method is proposed. It is well known that in most cases there exists a special transition at both start and end of a replay, in which a highlighted logo comes in and out gradually. This transition is called as logo-transition. Logos often keep the same during the whole program. Our motivation is to use the logos to assist replay segments detection. Firstly some logo-transitions are detected and further extract logo-samples from them at the beginning of a program. After that, the logo-template is extracted from these samples. Further, this template is employed to detect the other logos. After all logos are obtained, the video can be partitioned into segments with taking logos as boundaries. Then, shot and motion features are extracted in each segment. Finally, based on these cues, an SVM classifier is used to replay segment identification. 2. RELATED WORK

Slow motion replays in broadcast sports video are important and reliable clues for highlights and key events, since they are usually chosen by human experts. Therefore, detection of replays could facilitate semantic-based high level video processing tasks such as highlight generation [1], event detection [2], video summarization [3], etc.
21

International Journal of Graphics and Multimedia (IJGM), ISSN 0976 6448(Print), ISSN 0976 6456(Online) Volume 4, Issue 1, January - April 2013, IAEME Some slow motion replay detection methods have been proposed in the literature. They can be broadly classified into three categories. The first category of method directly analyzes the inherent attributes of replay video segments and tries to use those attributes to differentiate the slow motion replays from the normal plays. Before high speed cameras are widely used, slow motion replays were usually produced by repeating frames recorded from standard cameras. A few detection methods are based on this assumption. A replay detection method based on the macroblock type, motion vector and bit rate information is presentd in [4]. The detection is performed in the MPEG compressed domain to find the locations of repeating frames. The algorithm in [1] calculates the image differences between adjacent frames and search fluctuations in the differences to identify replays. These methods usually get low performance on replays that captured by high speed cameras and they can not provide accurate boundaries of replays. The second category of method employs statistical techniques to perform shot classification for the purpose of detecting replay shots. The author of [5] presented a method based on the difference of motions between replays and normal plays. A support vector machine (SVM) was trained with features of color ratio, shot length, mean color value and motion related feature to detect replay shots. The proposal in. [6] applied scene transition structure analysis on the shot classification results. Replays were extracted based on the generated shot label sequences by using some predefined rules. A HMM to classify replay and non-replay shots is employed in [7]. The performance of this category is usually not satisfied and results are still not precise enough. The third category of method tries to find the differences between replays and normal plays in terms of specific production behaviors, such as special video effects or logo patterns. Replays are then determined according to the existence of such behaviors. The work presented in [8] captures a digital video effect (DVE) l interactively for given videos. It is described by color and motion of the gradually changing boundary between adjacent shots. A replays are located by two DVEs detected with the model. Based on the observation that more and more replays that are sandwiched in between two certain logo transition sequences, some researchers try to use logo transition locations to identify replays. The mean-square differences of intensity and color histogram as frame similarity measure to detect logo template and to search logos in videos is discussed in[9]. A method in [10] employed the spatiotemporal mode seeking on selected logo sequences to capture the dominant color mode. This color mode is then used to perform EMD-based similarity matching for logo searching purpose. Tong et al. [11] used frame-to-frame difference with MSD to select candidate logo sequence. They calculated the average image of those candidate logo frames near to the center to obtain logo template. Template matching was performed based on color and shape features. Huang et al. [12] first extracted gradual transitions and employed motion features to learn the logo template. A color representation was then computed from the logo template and was used to search logos. Dang et al. [13] semiautomatically extracted logo template sequences and detected replays based on template sequence matching. Han et al. [14] learned gradual transition patterns based on motion vector reliability classification from a database by SVM. Replay detection was performed by fusing the pattern matching results and slow motion detection results based on motion vector information. The third category of method can usually obtain high accuracy on replay boundaries and the performance is better than above two categories. However, these methods often make assumptions on the logo transition sequences such as logo locations, moving patterns and speed, types of digital effects, etc. And

22

International Journal of Graphics and Multimedia (IJGM), ISSN 0976 6448(Print), ISSN 0976 6456(Online) Volume 4, Issue 1, January - April 2013, IAEME nearly all of them assume that the two logo transition sequences before and after the replay are identical. The Proposed solution is a methodology of determining the replay detection in a sports video using support vector machine for classifying frames as Logo frame and non logo frame. In most of sports video a logo swaps at beginning and end of the replay, the frames inbetween the start logo frame to its corresponding end logo frame of a videos construct a replay event. Applying same procedure for total video it results into collection of replays of sports video. These replays can be used to form a summary of video. 3. PROPOSED ALGORITHM FOR REPLAY DETECTION

The proposed method employs a Support Vector Machine based classifier for detecting the logos that will swap across the screen in broadcasted video at start and end of the replay action. The proposed algorithm consist of mainly TWO stages i. Train the SVM classifier with logo patterns and non logo patterns for classifying the frames in the video as logo frames and non logo frames. ii. Detecting logos at start and end of the replay action and extracting the frames between them. The flow charts for two phases of proposed algorithm are presented in Fig. 1.
Start

Start

Extract Logo frames from the Input Video and take some Non logo frames

Read the Input Video

Extract the Entropy Features of a frame

Extract the Entropy Features of each frames

Train the SVM Classifier with Entropy Features of Logo frame and Non logo

Test the frame on trained SVM Classifier

frame
END Detection of Two logos at beginning and end of each replay event

Extraction of frames between two logos

END

Fig. 1 Flow diagram of proposed Algorithm, (a) Flow diagram for Training SVM Classifier (b) Flow diagram for Replay Detection and Extraction.
23

International Journal of Graphics and Multimedia (IJGM), ISSN 0976 6448(Print), ISSN 0976 6456(Online) Volume 4, Issue 1, January - April 2013, IAEME In most of the sports video typical logo-transition lasts 0.5-0.8 seconds or 5-24 frames. In logo-transition, there is an image frame that contains the clearest and the most complete logo. Usually, the logo is highlighted and located at the middle part of a frame. With these prior knowledge, the logo-samples from video can be detected and extracted. 3.1 Logo-transition Detection A typical logo-transition lasts 0.5-0.8 seconds or 15-24 frames. The frame-to-frame difference is measured by intensity mean square difference (MSD). The difference sequence after median filtering is a plateau-like shaped pattern (Fig. 3). Therefore, the logo-transition detection can be performed through plateau-like shape detection over the frame-to-frame difference sequence. The proposed algorithm is i. Compute frame-to-frame difference with MSD. Check the frame difference. Proceed to step 3 When the difference exceeds a ii. threshold, otherwise go back to step 1) for next frame. iii. Count the number of consecutive frame-differences that all exceed the framedifference-threshold until encounter several consecutive frame-differences that drop below this threshold. If the counter exceeds a certain threshold, a wipe transition can be determined. Otherwise go back to step 1) for next frame.

Fig. 2 A logo-transition (8 frames are displayed)

Fig. 3 Frame difference sequences. Original(left) and after median filtering(right)

3.2

Logo-sample Determination In a logo-transition, there is an image frame that contains the clearest and the most complete logo. Usually, the logo is highlighted and located at the middle part of a frame. With these prior knowledge, the logo-samples can be extracted.

24

International Journal of Graphics and Multimedia (IJGM), ISSN 0976 6448(Print), ISSN 0976 6456(Online) Volume 4, Issue 1, January - April 2013, IAEME Feature Extraction The color, texture, shape and motion estimation have been commonly used features in the literature, It is observed that color and motion features play a dominant role in the extraction of characteristics from the videos and hence the color information is adopted in this work. Normally color distribution is estimated in the form of histogram, the proposed work estimates the information of color content at different color intensity planes of images in terms of entropy values. The entropy computed for a color plane in an image gives the average information conveyed by an image. In the pr oposed work the entropy of three layers of RGB are computed and feature vector is written as in equation (1) (1) Where : Feature vector of frame from video sequence : Entropy of Red plane : Entropy of green plane : Entropy of blue plane of an RGB frame from video sequence. The feature vectors as above are constructed for Logo templates extracted from various videos and some Non logo frames and are used for training SVM. 3.4 Support Vector Machine (SVM) The support vector machine (SVM) algorithm seeks to maximize the margin around a hyperplane that separates a positive class from a negative class [20]. Given a training dataset with n samples (x1, y1); (x2,y2).(xn, yn), where xi is a feature vector in a vdimensional feature space and with labels yi {-1,1}belonging to either of two linearly separable classes C1 and C2. Geometrically, the SVM modeling algorithm finds an optimal hyper plane with the maximal margin to separate two classes, which requires to solve the optimization problem, as shown in equations (2) and (3).
(2)

3.3

(3)

where, i is the weight assigned to the training sample xi. If i > 0, xi is called a support vector. C is a regulation parameter used to balance the training accuracy and the model complexity so that a superior generalization capability can be achieved. K is a kernel function, which is used to measure the similarity between two samples. Different choices of kernel functions have been proposed and extensively used in the past and the most popular are the gaussian radial basis function (RBF), polynomial of a given degree, and multi layer perceptron. These kernels are in general used, independently of the problem, for both discrete and continuous data.

25

International Journal of Graphics and Multimedia (IJGM), ISSN 0976 6448(Print), ISSN 0976 6456(Online) Volume 4, Issue 1, January - April 2013, IAEME 3.5 SVM classifier based Replay detection Algorithm The proposed algorithm model uses priorly retrieved logos of sports video for the training, then its features i.e its entropy values are extracted and trained to SVM classifier. SVM is used as a classifier due to its clear classification method. Some randomly selected frames without logo are also trained to SVM classifier as Non Logo templates.It classify two Class vectors by a linear hyper plane , by which classifier can easily classify Logo and Non Logo frames.

Start

READ Video file (.avi,.mpeg) frame wise LOGO_COUNT =0

Extract Entropy features of current frame

Test the Frame with trained SVM for

LOGO or NON LOGO frame For LOGO frame LOGO_COUNT =1 for NON LOGO LOGO_COUNT =2

Is LOGO frame detected

Extract all the frames between Current LOGO detected and previous detection write them to file which gives a replay event.

End

Fig. 4 Flow chart of replay detection Algorithm

26

International Journal of Graphics and Multimedia (IJGM), ISSN 0976 6448(Print), ISSN 0976 6456(Online) Volume 4, Issue 1, January - April 2013, IAEME Algorithm: Step 1: Extract the Logo frames from the input videos as explained in above sections. Step 2: Extract the entropy features of a frames as given in section 3.3. Step 3: Store these entropy features into vectors as feature patterns. Step 4: Train the SVM Classifier with the feature patterns. Step 5: Include non logo frames features in SVM training set. Step 5: The pair of logos can be detected as described in flowchart of Fig. 4. Step 6: Replay is Detected between a pair of logos. Flow Diagram of Testing Phase to Identify an Logo Image in a Sports Video Detection Phase is shown in Fig. 4 4. EXPERIMENTAL RESULTS AND DISCUSSIONS and Replay

In the proposed work experimentation is conducted on data set of ten videos comprising of soccer and IPL videos. The dataset of all videos logo frames are extracted from all ten videos and are shown in next page in Fig.5, the features of which are used to train support vector machine. The SVM trained with these features is employed for detection of pairs of logos at the beginning and end of each replay action in soccer and cricket videos. The summary of experimentation is presented in Table 1. .

27

International Journal of Graphics and Multimedia (IJGM), ISSN 0976 6448(Print), ISSN 0976 6456(Online) Volume 4, Issue 1, January - April 2013, IAEME

Fig. 5 Logo frames trained to SVM classifier

The Table 1 summarizes the performance of proposed algorithm for replay detection over ten sample sports video out of which 3 are soccer video and 7 are cricket video. The proposed mechanism for replay detects the two logos that sandwich the replay action. The logos are detected by SVM classifier by testing every frame of the video. The video segment between two consecutive logos is picked as replay. The average values of performance parameters recall and precision up to 94% and 97% for the algorithm indicates that the proposed mechanism is suitable for extraction of replay actions in sports video and construct summary of sports videos.
28

International Journal of Graphics and Multimedia (IJGM), ISSN 0976 6448(Print), ISSN 0976 6456(Online) Volume 4, Issue 1, January - April 2013, IAEME TABLE1 Results of proposed algorithm for 10 test video samples
Video File Name Actual Number of Replays Total Number of replay correctly detected replays 9 56 20 31 7 14 19 10 23 12 Undetected replays False detected replays Recall Precision

Video1.avi Video2.avi Video3.avi Video4.avi Video5.avi Video6.avi Video7.avi Video8.avi Video9.avi Video10.avi

10 60 20 34 8 15 19 10 25 12

1 4 0 3 1 1 0 0 2 0

1 2 0 1 0 1 0 1 0 0

90.00% 93.33% 100.00% 91.17% 87.50% 93.33% 100.00% 100.00% 92.00% 100.00%

90.00% 96.55% 100.00% 96.85% 100.00% 93.33% 100.00% 90.90% 100.00% 100.00%

5.

CONCLUSION

The proposed algorithm for SVM based method for replay detection and extraction from sports video is designed and experimented with sufficient number of soccer and cricket clips. The algorithm is implemented in Matlab 2010b and executed on Pentium P6200 processor with 2GB RAM memory. The replay detection and extraction in soccer and cricket games are tested well with a sufficient number of video clips. The Proposed algorithms identify a replay by finding a pair of logos and hence extract the replay by extracting all frames in-between the pair of logos. The proposed method identify a pair of logos by matching a entropy features of trained logos with the each frames histogram values of input videos. The proposed work uses support vector machine to classify between true logo and non logos. The replays are used as highlights of a game, and by which we formed a summary of sports video by these highlights. REFERENCES [1] H. Pan, P. van Beek, and M.I. Sezan, Detection of slowmotion replay segments in sports video for highlights generation,in Proc. of ICASSP'01, vol. 3, pp. 16491652, 2001. [2] X. F. Tong, H. Q. Lu, and Q. S. Liu, A three-layer event detection framework and its application in soccer video, in Proc.of ICME'04, vol. 3, pp. 15511554, 2004. [3] A. Ekin, A. M. Tekalp, and R. Mehrotra, Automatic soccer video analysis and summarization, IEEE Transactions on Image Processing, vol. 12, no. 7, pp. 796807, 2003. [4] V. Kobla, D. DeMenthon, and D. Doermann, Detection of slow-motion replays for identifying sports videos, in Proc. of MMSP'99, pp.135140, 1999. [5] L. Wang, X. Liu, S. Lin, G. Y. Xu, and H.-Y. Shum, Generic slow-motion replay detection in sports video, in Proc. of ICIP'04,vol.3, pp. 15851588, 2004. [6] J. J. Wang, E. Chng, and C. S. Xu, Soccer replay detection using scene transition structure analysis, in Proc. of ICASSP'05,vol.2, pp. 433436, 2005. [7] Y. Yang, S. X. Lin, Y. D. Zhang, and S. Tang, A statistical framework for replay detection in soccer video, in Proc. Of ISCAS'08, pp. 35383541, 2008.
29

International Journal of Graphics and Multimedia (IJGM), ISSN 0976 6448(Print), ISSN 0976 6456(Online) Volume 4, Issue 1, January - April 2013, IAEME [8] N. Babaguchi, Y. Kawai, Y. Yasugi, and T. Kitahashi, Linking live and replay scenes in broadcasted sports video, in Proc. of MULTIMEDIA'00, pp. 205208, 2000. [9] H. Pan, B. Li, and M. I. Sezan, Automatic detection of replay segments in broadcast sports programs by detection of logos in scene transitions, in Proc. of ICASSP'02, pp. 3385 3388, 2002. [10] L.-Y. Duan, M. Xu, Q. Tian, and C.-S. Xu, Mean shift based video segment representation and applications to replay detection, in Proc. of ICASSP'04, pp. 709712, 2004. [11] X. F. Tong, H. Q. Lu, Q. S. Liu, and H. L. Jin, Replay detection in broadcasting sports video, in Proc. of ICIG'04, pp.337340, 2004. [12] Q. Huang, J. M. Hu, W. Hu, T. Wang, H. L. Bai, and Y. M. Zhang, A reliable logo and replay detector for sports video, in Proc. of ICME'07, pp. 16951698, 2007. [13] Z. H. Dang, J. Du, Q. M. Huang, and S. Q. Jiang, Replay detection based on semiautomatic logo template sequence extraction in sports video, in Proc. of ICIG'07, pp. 839 844, 2007. [14] B. Han, Y. Yan, Z. H. Chen, C. Liu, and W. G. Wu, A general framework for automatic on-line replay detection in sports video, In Proc. of MM'09, pp. 501504, 2009. [15] Reeja S R and Dr. N. P Kavya, Motion Detection for Video Denoising The State of Art and the Challenges, International Journal of Computer Engineering & Technology (IJCET), Volume 3, Issue 2, 2012, pp. 518 - 525, ISSN Print: 0976 6367, ISSN Online: 0976 6375.

30