Face

Aim of Our Project

The aim of this project was to develop an algorithm that detects human facial features like the mouth, nose and eyes in a full frontal face image. We adapted a multi-step process in order to achieve the goal. To detect the face region a skin-color segmentation method was used. Morphological techniques were used to fill the holes that were created after the segmentation process. From the skeletoni ation process, a skeleton of the face was o!tained from which face contour points were e"tracted. Facial features can !e located in the interior of the face contour. We used several different facial-images to test our method. We achieved detection.

Introduction
Face feature e"traction is considered to !e a key requirement in many applications such as #iometrics, Facial recognition systems, video surveillance, $uman computer interface etc. Therefore relia!le face detection is required for the success of these applications. The task of human facial feature e"traction is not easy. $uman face varies from person to person. The race, gender, age and other physical characteristics of an individual have to !e considered there!y creating a challenge in computer vision. Facial feature detection aims to detect and e"tract specific features such as eyes, mouth and nose.

Previous work & Research in the field of face recognition

There are many face e"traction methods reported !y researchers in the field of face recognition. To com!ine image information and knowledge of the face is the main trend. %n order to discriminate face candidates from the neck, ears and the incorrect segmented parts, shape analysis of the color segmentation result is necessary. &ne method '(a!er and Tekalp, )**+, -ee et al., )**./ involves first filling an ellipse to the segmented face for registration. Most of them do not take into consideration the misalignment caused !y ear and neck. %an-0ang Wang, 1ric sung in their article have proposed a morphological procedure to analy e the shape of segmented face region. This is an une"plored approach in face detection research. We have incorporated some of the methods proposed in this article. (everal rules have !een formulated for the task of locating the contour of the face. These rules were primarily !ased on the facial skeleton and knowledge of the face. The searching region of the face features reduces. The facial features such as mouth, nostrils and eyes can !e more accurately located within the face contour. Terrillon et al., )**+ mentions the pro!lem of how other !ody parts such as neck may lead to face locali ation error. 2 different approach was presented in '(o!ottka and 3itas, )**+/ where the features are first detected and then the contour was tracked using a deforma!le model. #runelli and 3oggio,)**4 use dynamic programming to follow the outline in a gradient intensity map of elliptical projection of face image. $aalick and (hapiro, )**4 demonstrate how morphological operations can simplify the image data while preserving their essential shape characteristics and can eliminate irrelevances.

Method5 Face feature detection is a multi step process. %n our project in order to detect the face region a skin-color segmentation method was used. This method was developed !y 6ang and Wai!el')**7/.

To segment the face region we used morphological processes of dilation and erosion. The skeletoni ation process was used to e"tract face contour points. The (keleton thus o!tained is used in the process of line segmentation which includes line tracing and line merging. Facial features can !e located in the interior of the face contour. The process of Facial feature e"traction is descri!ed in section 8.4. Flowchart depicting the entire process

Section 2.1 Skin Segmentation

This is one the important steps in face feature e"traction process. The efficiency of the color segmentation of a human face depends on the color space that is selected. We used the finding !y 6ang and Wai!el ')**7,)**./ who noted that the skin colors of different people are very closely grouped in the normali ed r-g color plane. (o we took some seed pi"els from the face area o!tained the normali ed r-g spread and classified the pi"els that were close enough to this spread as !elonging to the face. The Figures !elow demonstrate our results.

Fig1 : Skin color segmentation a)Original image b) Skin color segmentation

Section 2.2 !or"hological Image Processing

(Process of filling interior holes of the face region and extraction of the face skeleton for determining contour points) Morphological image processing is a collection of techniques for digital image processing !ased on mathematical morphology. (ince these techniques rely only on the

relative ordering of pi"el values, not on their numerical values, they are especially suited to the processing of !inary images and grayscale images whose light transfer function is not known. 98: ;efinitions of some morphological operations that we used in our project5 Erosion5 1rosion is an operation that thins or shrinks the o!jects in the !inary image.1rosion are performed !y %3T function imerode'/. Dilation5 ;ilation is an operation that grows or thickens the o!jects in the !inary image. ;ilation is performed !y %3T function imdialate'/. For successful facial feature e"traction the accurate computation of the contour points is very important. This helps in locating searching regions. #oth neck and ears have the same color as that of the face. $ence they are connected to the face region. Therefore we need to separate them so as to !etter locate the facial features. %n order to detect the skeleton, the face region was filled !y applying multiple morphological dilation operations. Then the same num!er of erosion operations was applied. There e"ist holes that corresponds to the eyes, nose and mouth etc. We fill these holes using the dilation operation. The erosion operation is applied to the dilation result in order to restore the shape of the face. Then we o!tained the skeleton of the face. We used the %3T function !wmorph'/ in M2T-2# to e"tract the skeleton. The ears and the neck can !e separated from the segmented face region !y analy ing the skeleton. The figures on the ne"t page show the output we o!tained after performing operations.

Figure a/ The output after performing skin color segmentation

Figure !/ The output after filling the interior holes of the face region using dilation and erosion operations.

Figure c/ &verlay skeleton with face region.

#ontour $racing
The points for fitting the contour of the human face can !e collected while e"cluding the points that lie on the ear and neck.

Section 2.% &eature e'traction within the ROI

2fter the contour has !een found we o!tained the <egion of interest. Features such as eyes, nose and mouth can !e found within the <&%. We locate the mouth first. &nce the mouth is detected we can determine the eyes and nose easily. For finding the lips we tried several methods. We performed edge detection using (o!el operator that finds the hori ontal line within the <&% .Finding vertical position of the line !etween the lips was done !y using hori ontal integral projection ph of the !inary image in the search region. ph is o!tained !y summing up the pi"els values in each row of the search. #ecause lip line will have the longest hori ontal line, its vertical position can !e located where ph has ma"imum value. We also used some prior knowledge a!out the face in order to determine the lips. We knew that lips are centered hori ontally in the lower half of the face and are isolated .We took advantage of this information in order to constrain the vertical position of the search area. &nce we have located the lips finding the eyes and nose was done in similar fashion, each time constraining the search area in vertical direction.

Figure: Contour points extracted from earlier process Figure: !dges remo"ed for detecting FF.

Figure : The Region of nterest Figure: Final Results

Results & Anal(sis

=um!er of images tested5 4> (uccess5 *>? The following Ta!le shows the results of the first )4 images. Image no ) 8 4 @ Name
c1m.jpg c2m.jpg c37m.jpg c4m.jpg

Lips

Nose

Mouth

7 . A + * )> )) )8 )4

c5m.jpg c6m.jpg c7m.jpg c8m.jpg c9m.jpg c10m.jpg c11m.jpg c12m.jpg c13m.jpg

The <esults and analysis of the e"periments conducted are demonstrated in the ne"t page.

&ne of the e"perimental results with steps is shown in figures !elow. The e"perimental face images were downloaded from www.faceresearch.org site. 2 total of 4> images of different people were selected randomly. The e"perimental results shown a!ove illustrate that our method is quite good. 2 correct recognition rate of *>? is o!tained using out method. There were some images which showed wrong detection of facial features. 2n analysis of these images shows that our method failed to e"tract enough contour points from these images which resulted from ill skeleton-e"traction. -ack of contour points resulted into restricted <&% in which needed features failed to appear. &ne way of improving the skeleton e"traction could !e to more generali e the face region e"traction process followed !y sufficient dilation and erosion process. #etter the effect of skin-region e"traction process, !etter will !e the resulting skeleton o!tained and thus !etter will !e the contour points e"tracted. &ther issue that limits given e"periment is the image si e. (ince the e"periment is performed using M2T-2# %3- tool!o", running time of the program is a major issue. Therefore large si e images take longer time to compute the result and some may not give desired results cause of image comple"ity.

Figure: a) Original Face mage

b) Skin Color Segmentation

Figure: c) #fter $ilation % !rosion

d) Skeleton of the image

Figure: e) Contour points

f) RO

Figure: g) #fter appl&ing !dge $etection to the RO

h) Final image 'ith features detected

References
9):Frontal-view face detection and facial feature e"traction using color and morphological operations !y Bian-0ang Wang, 1ric (ung C 98: 2 Model-#ased 0a e Tracking (ystem !y <ainer (tiefelhagen, Bie 6ang, 2le" Wai!el 94: ;igital %mage 3rocessing Dsing M2T-2# !y 0on ale , Woods E1ddins,3rentice 9@: %mages taken from 5 www.faceresearch.org 97: 3rof. 0a!orskiFs lecture slides. 9@: www.wikipedia.com