APOLLO INSTITUTE OF TECHNOLOGY, KANPUR

(Department of Computer Science and Engineering)

A Project Report
On
Face Recognition in Uncontrolled Environment
Submitted in partial fulfillments
For the award of degree of

BACHELOR OF TECHNOLOGY
In
COMPUTER SCIENCE AND ENGINEERING
(2012 2016)
SUBMITTED TO: SUBMITED BY: -

UNDER GUDANCE: -

Er. Mohit Gupta

Er. Amrit Agrwal

CSE Department

CSE Department

Kumar[1135310903]

CSE 4th Year

Group No :-

Dhruv Mishra
[1235310016]
Rajat Awasthi
[1235310032]
Sandeep

APOLLO INSTITUTE OF TECHNOLOGY, KANPUR

Department of Computer Science & Engineering

Certificate
This is to certify that Dhruv Mishra (1235310016) bonafide students of
VII semester,
branch Computer Science and Engineering of this college have successfully
completed the
project entitled An Evaluation Of Face Recognition System, in partial
fulfillment of the
requirements for the award for degr ee of Bachelor of Engineering in
Computer Science &
Engineering of the Uttar Pradesh Technical University, Lucknow during the
academic year
2012 2016. The project report has been approved as it satisfies the academic
requirements
in respect of project work prescribed for the said degree.

Signature of Guide
(Er. Amrit Agrwal)

Signature of HOD
(Er. Amrit Agarwal)

ACKNOWLEDGEMENT

Our first sincere appreciation and gratitude goes to respected Er. Amrit
Agrwal, Project guide for his guidance, constructive comments, valuable
suggestions and inspirations. During the entire development session, we have
received endless help from him. And also thanks to Er. Saurabh Srivastva for
the valuable guidance and for permitting us to carry out this project.
We also take opportunity to express our sincere thanks and full appreciation to
Er Amrit Agarwal (Head of Department Of Computer Science &
Engineering) Apollo institute of technology Kanpur, who extended their
wholehearted cooperation, moral support and rendering ungrudging assistance
whenever and where ever need aroused .We are very Thankful to them.
Finally, we wish to say thanks to all people of the institute for their kind
cooperation.

ER. Amrit Agrwal

(Project guide)
(CSE Depatrment)

Er Amrit Agarwal
(Head of Department)
(C.S.E)

INDEX

Abstract:
Human face detection and recognition play important roles in many
applications such as video surveillance and face image database
management. In our project, we have studied worked on both face
recognition and detection techniques and developed algorithms for
them. In face recognition the algorithm used is PCA (principal
component analysis), MPCA(Multi-linear Principal Component
Analysis) and LDA(Linear Discriminant Analysis) in which we
recognize an unknown test image by comparing it with the known
training images stored in the database as well as give information
regarding the person recognized. These techniques works well under
robust conditions like complex background, different face positions.
These algorithms give different rates of accuracy under different
conditions as experimentally observed.
In face detection, we have developed an algorithm that can detect
human faces from an image. We have taken skin colour as a tool for
detection. This technique works well for Indian faces which have a
specific complexion varying under certain range. We have taken real
life examples and simulated the algorithms in MATLAB successfully.

1 INTRODUCTION:
The face is our primary focus of attention in social life playing an
important role in conveying identity and emotions.
We can recognize a number of faces learned throughout our lifespan and
identify faces at a glance even after years of separation. This skill is
quite robust despite of large variations in visual stimulus due to
changing condition, aging and distractions such as beard, glasses or
changes in hairstyle.
Computational models of face recognition are interesting because they
can contribute not only to theoretical knowledge but also to practical
applications. Computers that detect and recognize faces could be
applied to a wide variety of tasks including criminal identification,
security system, image and film processing, identity verification,
tagging purposes and human-computer interaction. Unfortunately,
developing a computational model of face detection and recognition is
quite difficult because faces are complex, multidimensional and
meaningful visual stimuli.
Face detection is used in many places now a days especially the
websites hosting images like picassa, photobucket and facebook. The
automatically tagging feature adds a new dimension to sharing pictures

among the people who are in the picture and also gives the idea to other
people about who the person is in the image. In our project, we have
studied and implemented a pretty simple but very effective face
detection algorithm which takes human skin colour into account.
Our aim, which we believe we have reached, was to develop a method
of face recognition that is fast, robust, reasonably simple and accurate
with a relatively simple and easy to understand algorithms and
techniques. The examples provided in this thesis are real-time and taken
from our own surroundings.

Biometrics:
Biometrics refers to metrics related to human characteristics. Biometrics authentication
(or realistic authentication) is used in computer science as a form of identification
and access control.[1] It is also used to identify individuals in groups that are under
surveillance.
Biometric identifiers are the distinctive, measurable characteristics used to label and
describe individuals. Biometric identifiers are often categorized as physiological versus
behavioral characteristics. Physiological characteristics are related to the shape of the
body. Examples include, but are not limited to fingerprint, palm veins, face
recognition, DNA, palm print, hand geometry, iris recognition, retina and odour/scent.
Behavioral characteristics are related to the pattern of behavior of a person, including
but not limited to typing rhythm, gait, and voice. Some researchers have coined the
term behaviometrics to describe the latter class of biometrics.
More traditional means of access control include token-based identification systems,
such as a driver's license or passport, and knowledge-based identification systems, such
as a password or personal identification number. Since biometric identifiers are unique
to individuals, they are more reliable in verifying identity than token and knowledgebased methods; however, the collection of biometric identifiers raises privacy concerns
about the ultimate use of this information.

As modern society increasingly depends on systems to provide secure

environments and services to people, it becomes paramount to ensure the
security of a system through means to identify the validity of an individual
requesting access to it. This is usually established by extracting some form of
information from the individual to check against information held by the system

about valid users.

This ITU-T Technology Watch Report spotlights biometric recognition as a key
form of authentication, one which is increasingly used in a wide range of
applications made possible by advanced pattern recognition algorithms applied
through powerful information and communication technologies (ICT).
Biometric recognition can be described as automated methods to accurately
recognize individuals based on distinguishing physiological and/or behavioral
traits. It is a subset of the broader field of the science of human identification.
Technologies used in biometrics include recognition of fingerprints, faces, vein
patterns, irises, voices and keystroke patterns (See Figure 1). In the subfield of
telebiometrics, these recognition methods are applied to telecommunications.
In a non-automated way and on a smaller scale, parts of the human body and
aspects of human behavior have been used ever since the dawn of mankind as a
means of interpersonal recognition and authentication. For example, face
recognition has been used for a long time in (non-automated) security and access
applications, e.g., as a method to verify that the owner of a passport and the
person showing the passport are the same, by comparing the persons face and
the passport photo.
The Digital Revolution added ICT as a means to fulfill recognition and
authentication
processes,
often
through
PCs
and
computerized
telecommunication devices, such as cash dispensers. Users authenticate
themselves to the machine by entering a secret knowledge-based authenticator,
such as a PIN or passphrase, or by the possession of a token, like a bank card or
key, and sometimes authentication requires a combination of knowledge and
possession.
The 1960s also saw the first automated biometric recognition applications.
However, the biometric industry did not take off at that time, due to high cost,
low recognition accuracy and the lack of standards and testing benchmarks with
which the different approaches could be compared and quality ensured.
To further the use of biometric systems, issues of security and privacy will need
to be carefully addressed, as well as the high levels of expectation in accuracy,
reliability, performance

Biometrics can be sorted into two classes:

Physiological:
Examples: face, fingerprint, hand geometry and iris recognition

Behavioral:
Examples: signature and voice

Common:

Figure 1:

Others:

Fingerprint Recognition
Face Recognition
Speaker Recognition
Iris Recognition
Hand Geometry
Overview
of some biometrics
Signature
verification

Fingerprint

Iris

DNA
Retina recognition
Thermo grams
Gait
Keystroke
Ear recognition
Skin reflection
Lip motion
Body odor

DNA

Images uploaded to Flickr by (1) Fazen, (2) Sarah Cartwright, (3) ynse, (4) Ben Harris-Roxas.

Keystroke pattern

Others:
Vein Pattern
Sweat Pores
Fingernail Bed
Hand Grip
Brain Wave Pattern
Footprint and Foot Dynamics

Fingerprint Recognition:
An extremely useful biometrics technology since fingerprints have long been
recognized as a primary and accurate identification method.

2. Face Recognition:

Uses an image or series of images either from a camera or

photograph to recognize a person.

Principle: analysis of the unique shape, pattern and

positioning of facial features.

Features:
Passive biometrics and does not require a persons
cooperation
Highly complex technology and largely software based.
Primary advantage is that the biometric system is able
to operate hands-free and a users identity is
confirmed by simply staring at the screen.

Details:
Source of data: Single image, video sequence, 3D image and
Near Infrared
Models: weak models of the human face that model face shape
in terms of facial texture
Face appearance
Examples: eigenfaces
Face geometry
Examples: feature-based

method

Example:

3.

Voice Recognition:
Voice recognition is not the same as speech recognition, it is speaker recognition
Considered both physiological and behavioral
Popular and low-cost, but less accurate and sometimes lengthy enrollment

4.

Iris recognition:
Analysis of the iris of the eye, which is the colored ring of tissue that surrounds
the pupil of the eye.
Based on visible features, i.e. rings, furrows, freckles and the corona. Features
and their location are used to form the Iriscodes, which is the digital template.
Widely regarded as the most safe, accurate biometrics technology and capable
of performing 1-to-many matches at extraordinarily high speeds, without
sacrificing accuracy.

Example:

Iris recognition is a
highly mature
technology with a
proven track record in a
number of application
areas.
Used very effectively all
over the world.

5.

Hand geometry:
Hand geometry systems are commonly available in two main forms. Full
hand geometry systems take an image of the entire hand for comparison
while Two Finger readers only image two fingers of the hand.
Hand recognition technology is currently one of the most deployed
biometrics disciplines world wide

2 FACE RECOGNITION:
The face recognition algorithms used here are Principal Component
Analysis(PCA), Multilinear Principal Component Analysis (MPCA) and
Linear Discriminant Analysis(LDA).
Every algorithm has its own advantage. While PCA is the most simple
and fast algorithm, MPCA and LDA which have been applied together
as a single algorithm named MPCALDA provide better results under
complex circumstances like face position, luminance variation etc. Each
of them have been discussed one by one below.

2.1 PRINCIPAL COMPONENT ANALYSIS (PCA):

Principal component analysis (PCA) was invented in 1901 by Karl
Pearson. PCA involves a mathematical procedure that transforms a
number of possibly correlated variables into a number of uncorrelated
variables called principal components, related to the original variables
by an orthogonal transformation. This transformation is defined in such
a way that the first principal component has as high a variance as
possible (that is, accounts for as much of the variability in the data as
possible), and each succeeding component in turn has the highest
variance possible under the constraint that it be orthogonal to the
preceding components. PCA is sensitive to the relative scaling of the

original variables. Depending on the field of application, it is also

named the discrete KarhunenLove transform (KLT), the Hotelling
transform or proper orthogonal decomposition (POD).
The major advantage of PCA is that the eigenface approach helps
reducing the size of the database required for recognition of a test
image. The trained images are not stored as raw images rather they are
stored as their weights which are found out projecting each and every
trained image to the set of eigenfaces obtained. 6

2.1.1 The eigenface approach:

In the language of information theory, the relevant information in a face
needs to be extracted, encoded efficiently and one face encoding is
compared with the similarly encoded database. The trick behind
extracting such kind of information is to capture as many variations as
possible from the set of training images.
Mathematically, the principal components of the distribution of faces
are found out using the eigenface approach. First the eigenvectors of the
covariance matrix of the set of face images is found out and then they
are sorted according to their corresponding eigenvalues. Then a
threshold eigenvalue is taken into account and eigenvectors with
eigenvalues less than that threshold values are discarded. So ultimately
the eigenvectors having the most significant eigenvalues are selected.
Then the set of face images are projected into the significant
eigenvectors to obtain a set called eigenfaces. Every face has a
contribution to the eigenfaces obtained. The best M eigenfaces from a
M dimensional subspace is called face space
Each individual face can be represented exactly as the linear
combination of eigenfaces or each face can also be approximated
using those significant eigenfaces obtained using the most significant
eigen values.

Figure1: eigen faces

Now the test image subjected to recognition is also projected to the

face space and then the weights corresponding to each eigenface are
found out. Also the weights of all the training images are found out
and stored. Now the weights of the test image is compared to the set of
weights of the training images and the best possible match is found
out. The comparison is done using the Euclidean distance
measurement. Minimum the distance is the maximum is the match.
The approach to face recognition involves the following initialisation
operations:
1. Acquire an initial set of N face images (training images).
2. Calculate the eigenface from the training set keeping only the M
images that correspond to the highest eigenvalues. These M images
define the facespace. As new faces are encountered, the eigenfaces
can be updated or recalculated accordingly.
3. Calculate the corresponding distribution in M dimensional weight
space for each known individual by projecting their face images onto
the face space.
4. Calculate a set of weights projecting the input image to the M
eigenfaces.
5. Determine whether the image is a face or not by checking the
closeness of the image to the face space.
6. If it is close enough, classify, the weight pattern as either a known
person or as an unknown based on the Euclidean distance measured.
7. If it is close enough then cite the recognition successful and provide
relevant information about the recognised face form the database
which contains information about the faces.

2.1.2 Mathematical approach:

Let 1, 2, , m be the set of train images.
Average face of set can be defined as
=(1/ M ) n=1 M ( n)

Each face differs from the average by the vector

i = i

when subjected to PCA, this large set of vectors seeks a set of M orthogonal
vectors Un, which best describes the distribution of data.
The kth vector Uk is chosen such that

k=(1/M) n=1M[(Uk) T * n]2.

is maximum, subject to:

The vector Uk and scalar k are the eigenvectors and eigenvalues respectively of the
covariance matrix

Where the matrix A= [1 2 .. M].

2.2 Experimental analysis:

We created a student database of AIT Kanpur which contains their photographs and
information (name, age, roll no).

2.2.1 Student database..

2.2.1.1 Student information:

Name
Dhruv Mishra

Age

Sex

Roll No

9 eigenfaces were obtained from the 10 test images. Since the database is not large enough, the
threshold eigenvalue was kept low.

How Face Recognition works:

Introduction to Face Recognition: Structure and Procedure
In this report, we focus on image-based face recognition. Given a picture taken from
a digital camera, wed like to know if there is any person inside, where his/her face
locates at, and who he/she is. Towards this goal, we generally separate the face
recognition procedure into three steps: Face Detection, Feature Extraction, and Face
Recognition.