Dung, L., and Mizukawa, M.

Paper:

Fast Hand Feature Extraction Based on Connected Component Labeling, Distance Transform and Hough Transform
Le Dung and Makoto Mizukawa
Department of Electrical Engineering, Shibaura Institute of Technology 3-7-5, Toyosu, Koto-ku, Tokyo 135-8548, Japan E-mail: {m706502@, mizukawa@sic.}shibaura-it.ac.jp [Received May 11, 2009; accepted September 30, 2009]

In hand gesture recognition or hand tracking systems relied on hand modeling methods, it is usually required to extract from a hand image some hand features. This paper presents a new robust method based on connected component labeling (CCL), distance transform (DT) and Hough transform (HT) to fast and precisely extract the center of the hand, the directions and the ngertip positions of all outstretched ngers on a skin color detection image. First, the method uses a simple but reliable technique that is performed on both the connected component labeling image and the distance transform image to extract the center of the hand and a set of features pixels, which are called distance-based feature pixels. Then, the Hough transform is calculated on these feature pixels to detect all outstretched ngers as lines. From the line detection result, the nger directions and the ngertip positions are determined easily and precisely. This method can be carried out fast and accurately, even when the skin color detection image includes hand, faces and some noise. Moreover, the number of distance-based feature pixels is usually not so high; therefore, the line detection process based on the Hough transform can be performed very fast. That can satisfy the demands of a real-time human-robot interaction system based on hand gestures or hand tracking. Keywords: hand feature extraction, ngertip positioning, distance transform, Hough transform, connected component labeling

Fig. 1. Overview of the proposed method to exact hand features.

1. Introduction
Hand gestures are a form of non-verbal communication that can be used not only for human-human interaction but also for human-computer interaction (HCI) and human-robot interaction (HRI). With a camera attached to a robot or a computer, the images of the hand can be collected easily; however, it is not easy to understand the meanings of the hand gestures in these images. Some processing steps, such as hand segmentation, hand feature extraction and hand gesture recognition, should be preformed to bring out the meaning of the hand gesture. 726

In recent years, to respond to the demands of smart realtime systems [14] that can help robots or computers to interact with humans in natural ways, many researchers have focused on nding fast yet highly accurate methods of extracting hand features corresponding to the palm, the ngers and the ngertips [3, 1622]. These features are very important parameters for both hand gesture recognition and hand tracking. This paper also focuses on hand feature extraction problems after segmenting the hand from the image frame through skin color detection. A new robust hand feature extraction method will be proposed in this paper that can fast and precisely determine the center of the hand, the directions and the ngertip positions of all outstretched ngers. The originality of the proposed method is a simple but reliable technique, which can extract the center of the hand and a set of features pixels based on the distance transformation image [5] and the connected component labeling image [6] that can be made from the binary skin color detection image [23] fast and easily. These feature pixels called distance-based feature pixels, correspond to the positions of the ngers and ngertips. Even when the skin color detection image includes hand, faces and some segmentation noise, this feature pixel extraction technique can still perform fairly well (Fig. 1). If all open ngers are outstretched ngers, they can Journal of Robotics and Mechatronics Vol.21 No.6, 2009

Fast Hand Feature Extraction Based on CCL, DT and HT

be detected as lines by using a line detection technique on the set of distance-based feature pixels. The line detection technique selected for the proposed method uses the Hough transform with the conventional voting process [7, 8]. This paper has also proposed an improvement in the voting process to reduce the line detection time. This improvement relies on the hand center position to signicantly reduce the number of quantized angles that will be calculated in the Hough transform. Moreover, the number of distance-based feature pixels is usually not so high; therefore, the line detection process based on the Hough transform can be performed very fast. From the line detection result, the nger directions and the ngertip positions will be determined easily and accurately. Once the center of the hand, the directions of the ngers and the positions of the ngertips can be determined exactly, the hand gesture recognition process and the hand tracking process will be performed more easily and accurately. Based on connected component labeling, distance transform and the Hough transform, which do not require complex calculations, the proposed method allows a hand gesture recognition or hand tracking system to achieve realtime performance. That is very important in a real-time human robot interaction system based on hand gestures or hand tracking. The rest of this paper is organized as follows: Section 2 discusses some related methods of extracting hand features. Section 3 describes the process of making the distance transformation image and the connected component labeling image from an image frame. Section 4 presents the technique of extracting the center of the hand and the set of distance-based feature pixels. Section 5 describes the process of determining the directions of the ngers and the positions of the ngertips based on the Hough transform. Some experimental results and discussion are presented in Section 6. Finally, we conclude and make proposals for future work.

elastic matching for the classication of hand postures against complex backgrounds in grey-level images. This method uses elastic graphs to represent hands with local image descriptions called jets, which are computed by a Gabor lter at each vertex. The main problem with this method is the large variations in the hand shapes that refer to the same gesture. There are some methods [1619] based on the contour of the hand segmentation region to determine the ngertip positions as local maxima, and the center of the hand is determined from the rst moment of the 2-D probability distribution calculated on the hand segmentation region. These methods usually require good hand segmentation results to have a good hand contour for ngertip detection. Xabier Iturbe [4] used the Hough transform on an edge detection image and movement information to detect only one ngertip in the image. Because the number of pixels on the contour or edges of the hand is usually not small, the computation cost is considerably high. Template matching can be used to detect ngertips and palms in some methods [3, 20, 21]. However, in systems using these methods, the distance from camera to hand is xed and good hand segmentation results are also required. Ko-Jen Hsiao [22] proposed a method for fast ngertip positioning by combining particle ltering with particles random diffusion. However, the number of particle used in the algorithm is still higher than the number of distance-based feature pixels in the proposed method. Based on the distance-based feature pixels extracted from the distance transformation image and connected component labeling image, the proposed method in this paper can make a hand model with the center of the hand, the directions and the ngertip positions of all outstretched ngers fast and precisely, even when the image frame includes faces and a complex background.

2. Related Works
In hand feature extraction related to building a hand model [9], there are some researchers who have proposed methods that have achieved satisfactory accuracy and processing time. Lars and Lindberg [10] used multiscale color features to detect multi-scale blob and ridge features corresponding to the palm and nger structures. The multi-scale color features detection method [11] has been widely used for geometric feature detection, but the computation costs are high, as it involves a great deal of large-scale Gaussian convolution. Inspired by the work in [14], Fang Yikai proposed a method [12] that utilizes a simple integral image [13] to replace the complex Gaussian derivatives in the multi-scale color features detection method of Lars and Lindberg. Although this method can speed up hand gesture recognition, the accuracy seems to be improved and the number of blob and ridge feature points is still high in the case of the hand images with complex backgrounds. Triesh and Malsburg [15] proposed a method based on Journal of Robotics and Mechatronics Vol.21 No.6, 2009

3. Distance Transformation Image and Connected Component Labeling Image

From an image frame taken by a camera, after using skin color detection, the distance transformation image [5] and connected component labeling image [6] will be made for the hand feature extraction process. The skin color model selected in this paper is the elliptical boundary skin color model, which can make a skin color detection image very fast considerable accuracy [23]. The connected component labeling image is used not only for rening the binary skin color detection image but also for the hand selection process.

3.1. Skin Color Detection This is an important step that separates all skin color pixels from the image taken by the camera. These may include those of the hand and face as well as other skin color regions. Most skin color detection methods are based on a skin color model and classier. There are many skin color
727