1

Bahar Berna Akko and Ahsen kbal Yergk Department of Computer Engineering TOBB Economics and Technology University

DYNAMIC HAND GESTURE RECOGNITION

Abstract The purpose of this report is detailing the period of progress until the end of the previous term. During this period we amended our L-1 matching algorithm exploiting threshold. Also we implemented Haussdorff distance algorithm besides L-1 distance algorithm. Haussdorff distance gives the similarity between two color histograms computing maximum of the minimum difference values. Finally we completed the animation part of our project creating 3D animation models using openGL which perform the suitable movement fitting to the hand gesture performed in front of the camera. I.INTRODUCTION In this period of time, after time performance augmentation, histogram equalization, and experimental studies for the previous period, we exploited from threshold for the inconvenience of L-1 distance. Even though cumulative calculation and histogram equalization, still light conditions affect the results badly. Due to noises [1] which affect the results, we calculated the threshold between the hand colors and the background. We separated hand cutout at certain verges and applied L-1 distance on this cutout, obtained much more satisfying results. Secondly, we have implemented Haussdorff distance algorithm as a second algorithm to be compared with L-1 distance. In Haussdorff distance, we calculate the minimum of difference of intensity values between two color histograms and the compute the maximums of these minimums. This new distance will say how similar these two histograms. Eventually, we modeled and joined the animation part of the project. We created an athlete performing the movements fitting to the query gesture. Besides studying on different methods, we decided to work on virtual reality tool while creating the animation with openGL. After the comparison and detection process, this animated video appears and moves according to your hand gestures. That is if you do the come here movement, the athlete begin to come closer to the screen. II.PROGRESS At this time of progress we have worked on improving our system as the matter of threshold and animation besides demonstrating hand gestures. Threshold: At this time period we evaluate the most convenient threshold value to separate hand and the background. Due to easily changeable light conditions, even applying histogram equalization and cumulative L-1 distance algorithm, results were not as expected.

Figure 1

baharberna@hotmail.com ahsenikbal@gmail.com

As you see the figure 1, the first frame is pure frame captured under natural light, however the heterogeneous brightness

2 throughout the hand shape will create a misunderstanding and will have a possibility of resembling to another hand shape. We calculate the threshold to convert the skin tones to white and background to black, and compute the L-1 distance over the second frame above. Furthermore, we obtain more sharp edges than the previous one removing the noise. Eventually our results are at approximately 100% percent accuracy at detection process. Animations: In order to demonstrate the results of the program, a moving model is needed. Firstly a 2D model is decided to be animated whereas 2D modeling tools such as Adobe Flash cannot be linked to MATLAB. Therefore this, implementing a three dimensional model [2] which is moving by the commands of MATLAB is chosen to show the query result. After this decision, a 3D modeling tool should be selected or the model is created with OpenGL [2] codes. In the same manner of the farmer experience, 3D modeling tools like Maya and 3DMax cannot be binned to our project. Additionally the idea of OpenGL based modeling need to be coded in C/ C++ and the query results of the MATLAB should be written in a text file which will be read by C/C++ program and run OpenGL. Finally a 3D modeling tool a plug-in service of MATLABs Virtual Reality Toolbox- is found and the model is implemented via VRealm Builder 2.0.

Figure 3 As shown in Figure 3, an athlete figure is modeled and animated with the correlation between hand gestures. For instance, the hand gesture of good bye is modeled as a hand shaking athlete.

Figure 2 VRealm Builder 2.0 (Figure 2) is a modeling tool that creates virtual worlds and objects of these virtual worlds via VRML (Virtual Reality Modeling Language). These objects are constructed as Transform one of which has rotation, color, dimension variables, etc. By using these variables and key frame mode of the tool, animations are modeled and saved as separate, infinite moving objects.

Figure 4 The animations are called in the program and opened via VRML Viewer which is shown in Figure 4. The viewer provides us to view the animation, change the rotation and specify viewpoint. Haussdorf Distance: Another algorithm to overcome the separation of hand gestures is Haussdorff distance which uses the maximum value of the candidate minimum differences. This algorithm is applied onto frames of a query gesture and the database by using histogram information of these frames. Haussdorff distance [4] is a metric that finds the maximum of the

3 minimum difference between two histograms as shown in Figure 4. Statistic, O.R. and Computer Science, University of La Laguna. [4] Colton M., Indyk P., Approximate Nearest Neighbor Algorithms for Hausdorff Metrics via Embeddings Rutgers University - Stanford University, 1999.

Figure 5 A and B are two data sets of the intensity values of two distinct histograms and a, b are the values of separate bins of this histograms each of which includes approximately 50 intensity values between 0 and 255. Haussdorff distance is evaluated by subtracting number of the intensity values of similar bins [3] of the query gesture and database.

Figure 6 Two different histogram values which are shown above are subtracted in order to calculate the distance cell array. After this evaluation, the minimum value of these differences is selected as one of the candidate. Hence the maximum value of these candidates is the desired gesture. III.CONCLUSION At this period of timeline, we worked on L-1 distance augmenting it by exploiting threshold values to discriminate background and hand shapes which also give the facility to ignore features of the light. Also we implement Haussdorff distance algorithm which finds the most similar frame by finding maximum of the minimum difference of the intensity value between two color histograms. Finally we bind our detection level to the animation level with a model created using openGL. This model performs the desired movement according to the hand gesture you perform in front of the camera. IV.REFERENCES [1] Vladimir Crnojevic, Impulse Noise Filter with Adaptive MAD-Based Threshold, University of Novi Sad Trg Obradovi ca, 21125 Novi Sad, Serbia and Montenegro, 2005. [2] Carr, M., Visualization with OpenGL: 3D made easy, Rad. Lab., EECS Dept. University of Michigan, 1997 [3] E. S-Nielsen, L.A-Canals, M. H-Tejera, Hand Gesture Recognition for Human-Machine Interaction, Department of