MORPHO WHITE PAPER

FACE
RECOGNITION

1
 PROPRIETARY RIGHTS
This document contains information of a proprietary nature to Morpho and is submitted in confidence for a specific purpose. The recipient assumes
custody and control and agrees that this document will not be copied or reproduced in whole or in part, nor its contents revealed in any manner or
to any person except to meet the purpose for which it was delivered.

This caveat is applicable to all the pages of this document.

2
TABLE OF CONTENTS

INTRODUCTION....................................................................................... 4

I. FACE RECOGNITION............................................................................. 5
1. DETECTION.................................................................................................................... 5
2. IMAGE ENHANCEMENT................................................................................................. 5
3. FEATURE EXTRACTION.................................................................................................. 5
4. COMPARISON................................................................................................................. 6
5. DECISION....................................................................................................................... 7

II. PERFORMANCE................................................................................... 8
1. CHALLENGES................................................................................................................ 8
2. CAPTURE EXPERTISE.................................................................................................... 8
3. MATCHING PERFORMANCE.......................................................................................... 9
4. FUTURE........................................................................................................................ 11

III. USE CASES....................................................................................... 12

1. PUBLIC SECURITY....................................................................................................... 12
2. CIVIL IDENTITY............................................................................................................. 13
3. BORDER CONTROL...................................................................................................... 14
4. AND MANY MORE...................................................................................................... 15

IV. IMPLEMENTATION GUIDELINES....................................................... 16

1. QUALITY CHECK.......................................................................................................... 16
2. CAMERA SET UP.......................................................................................................... 16
3. EXPERTISE IN CHECKING SEARCHES........................................................................ 17
4. MANAGING THE EXPECTATIONS................................................................................. 17

CONCLUSION......................................................................................... 18

3
 INTRODUCTION
From the very advent of photography, both government
agencies and private organizations have kept
collections of portraits and ID photos have gradually
made their way onto all personal identification
documents, from the most official passports to informal
membership cards issued by sports clubs.

Before the use of computers to recognize faces

was even considered a possibility, face recognition
was already the subject of a great deal of research.
Examples include:
n  itness interview: development of identification
W
parade or line-up techniques in the United Kingdom,
in which a witness is confronted with a group of
physically similar people, one of whom is a suspect.
The witness must decide whether one of the persons
in the group was present at the scene of the crime.
n  ace classification: in order to recognize delinquents
F
who are repeatedly arrested, without having to
resort to large collections of portraits, Bertillon
suggested that the portraits be sorted by common
morphological characteristics, i.e. the specific shapes
of the different parts of the face. This classification is
known as the spoken portrait.
Morpho has been active in the field of face recognition
since early 2000s. As a leader in biometric technology,
Morpho has been developing new systems from the
beginning of the automatic face recognition history.

In an effort to answer the increasing number of

questions being put forward, this document presents
face recognition in general, its performance, its
potential in operations and provides some clues about
implementation guidelines.

4
I. FACE RECOGNITION
Recognizing faces is the most natural thing to do for This stage is performed with a classifier that indicates
the human brain. After a few days, even newborns whether an image of fixed size represents a face.
can recognize their mothers face. Besides, capturing
portraits digitally is easy: contactless, with no need for The classifier learns from a database of faces and non
specific equipment now that cameras are everywhere, faces.
in the streets, in computers, even integrated in the
smartphone in your pocket.
2. IMAGE ENHANCEMENT
But when it comes to identifying someone from his or
Once the faces have been found and adjusted to the
her face, it is not because it is natural that it is simple.
same scale and position, they need to be enhanced.
The face is a three dimensional ever changing object
This involves minimizing the effects of compression,
always in motion. For an efficient identification system,
correcting inconsistent lighting or detecting and
face recognition technology starts well before image
excluding unusable zones (masked by clothes for
comparison.
example). Morpho developed a 3D morpheable model
to generate frontal images, correcting the orientation
of the face and the effects of expressions. While
1. DETECTION many enhancements can be made automatically, the
The first step is to detect the face in the images assistance of an operator may prove very useful when
collected from the source. It can be easy in static working on difficult images.
images such as identity documents with standardized
pose, lighting and plain background. It can be a huge
challenge in video streams with multiple persons in
movement and a busy background. The goal is then to
detect faces successfully, which means maintaining a
low rate of false face detection.

Feature extraction: Hierarchical graph matching

5
 3. FEATURE EXTRACTION
The image is processed to extract information and
convert it into a digital description for computer-based
comparison. Every face has numerous, distinguishable
traits, such as the distance between the eyes, the width
of the nose or the shape of the cheekbones. Face
recognition algorithms rely on those feature points but
also on mathematical information not identifiable by the
human eye.

To increase accuracy, multiple approaches are used,

amoung others the following are implemented:
n  ierarchical graph matching: creates a dynamic link
H
architecture that projects the face onto an elastic grid.
n  kin texture analysis: compares minute skin wrinkles,
S
pores, scars and other artifacts.
n  rincipal component analysis: removes superfluous
P
information (reduces data dimension) by breaking
down the face structure into uncorrelated
components known as eigenfaces.

4. COMPARISON
Once you have extracted the face features, you
can compare them to the ones in the database or
a watchlist and identify whether the person you are
looking for is already in it or at least get a list of potential
candidates that looked like him/her.

The comparison, or matching, is achieved through a

succession of algorithms in a multi-stage architecture:
the first ones are conservative and fast, the last ones
are slower but very selective. Each matching step
provides a candidate list of which only the top is used
for the next step. This approach narrows down the list
to be able to use more demanding algorithms efficiently
on a smaller amount of data. This process ensures both
high accuracy and fast matching.

The outcome of the comparison is a matching score

per candidate measuring the similarity between two
sets of face features and reflecting the confidence level
that they are coming from the same person. The final
step involves score normalization, in order to guarantee
Feature extraction: eigenfaces
that the matching score remains stable.

6
5. DECISION
In an identity management scenario, the goal is
prevent identity fraud by checking the uniqueness of
applicants biometric data. Based on huge biometric
databases, comparisons will be much easier by using
reliable thresholds that do not require adjustment as the
database grows. In case of automatic hit, an operator
can verify to confirm the result and proceed with the
investigation on the fraud attempt.

In law enforcement scenarios, a human reviewer

is usually employed to systematically review the
candidates returned from an identification search.
Usually, the reviewer inspects the suggested candidates
ordered in descending matching score, stopping when
he is able to positively confirm a mate. The length of
the candidate list may be fixed or variable by applying
a threshold. In this case, he onlyreviews the candidates
with a matching score above the threshold. Thanks to
accurate face recognition algorithms, with images of
reasonably good quality (such as mugshot), the use of
such a threshold allows for a dramatical reduction of the
reviewer workload. Indeed, the reviewer will only receive
a small number of images that stand a high chance of
matching the wanted person, thus preserving his time
and attention to process more cases or spend more
time on critical cases.

WHAT ACCURACY MEANS

To assess the success of this step, measured in terms of accuracy, we look at:
n FALSE ACCEPTANCE: You get a match but the images are not from the same person.

n FALSE REJECTION: You do not get a match while the person is in the database.

The effect of false acceptance and false rejection depends on how a system is used. This can be:
n NEGATIVE: You want to stop a person if there is a match (criminal watch list).

n POSITIVE: You want to allow a person if there is a match (frequent traveler program).

In negative use cases, a false rejection means the system failed to detect a person of interest, failing in its
mission. False acceptance means that you mark an authorized user as a suspect, causing discomfort and
distress. It also wastes money by triggering an alarm for no reason.
In positive use cases, a false acceptance means someone unknown has been granted access. False rejection
will lead to discomfort for the user, and costly manual processing. You will have to establish the persons identity
manually.
False rejection in a frequent traveler program will cause irritation. In a criminal watch list, it could help a criminal
leave the country.
Accuracy impacts the complete system, beyond the biometric technology. Whatever the use case, if the system
is not performant enough, people will eventually lose their trust in it.

7
 II. PERFORMANCE
In order to provide trustworthy and accurate results, face recognition cannot be performed without some precautions.

1. CHALLENGES
Indeed, a face image can be altered in many ways and affected by many parameters.

Resolution should be the highest

Lighting and contrast must be
possible, but depends on
the optimal, but depends on the
acquisition equipment and distance
acquisition environment
of the camera

The individuals pose should Ideally, facial expression should be

be frontal, but depends on the neutral but again, this depends on
conditions of the face capture the willingness of the individual

Occlusion can make the image Face recognition is sensitive to

unusable ageing

2. CAPTURE EXPERTISE n  3D modeling tool to rebuild a frontal image of a side

A
view picture, with a maximum of 30
The capture step is fundamental in the face recognition
process. If the reference database is populated with
poor quality images, then results will be much less
relevant. In the same way, the better the image to be In multi camera system, such as automatic border
compared is, the better the results will be. control gates, Morpho proposes an advanced workflow
to improve the image quality. It builds a synthetic
In order to improve image quality, Morpho has image based on multiple face captures. This allows the
developed various tools such as: generation of a frontal image with high resolution and a
n image enhancement tools to optimize brightness and neutral expression.
contrast
n Image processing tools to move eye position or
characteristic points

8
Use of the 3D modeling tool to rebuild a frontal picture
3. MATCHING PERFORMANCE
Biometric matching is all about accuracy and speed. Luckily, a customer can rely on independent benchmarks such
as the ones executed by the National Institute of Standards and Technology (NIST) which test the accuracy of different
vendors.
ABOUT NIST
The NIST is a laboratory commissioned by the
US Department of Homeland Security. One of the
laboratories is the Information Technology Laboratory,
which is in charge of Biometrics Standards. The project
develops biometric data exchange formats, biometric
sample quality, biometric acquisition and processing
protocols. Even more importantly: they test the
matching algorithms.
You measure accuracy by looking at the False Rejection
Rate (FRR) and False Acceptance Rate (FAR). The
two are closely related. A lower FRR will always result
in a higher FAR and vice versa. A small difference in
accuracy, either FRR or FAR, has a direct and significant
impact on the system efficiency and operational cost.
Morpho and its predecessors (Sagem, Visionics, Identix,
Viisage, and L-1 Identity Solutions) have participated
in the NIST tests from their inception in 2002. In every
single test, the companys technology was ranked first
or second.
FACE RECOGNITION VENDOR TEST (FRVT)
FRVT 2013 (Face Recognition Vendor Test) that ended
in May 2014 is the latest and largest scale test of face
recognition technology. It is an independent test, using
a large real-world dataset that provides an objective and
quantitative comparison between vendors. Results are
publicly made available on the website: http://www.nist.
gov/itl/iad/ig/frvt-home.cfm.
The test addresses two common use cases from two
different operational datasets:
n Law enforcement images, mostly mugshots with
some webcam images
n Identity document images from a smaller visa
database
A future test will address video sequences.
FRVT 2013 confirmed the outstanding performance
of Morphos technology. Morphos algorithm achieves
the second best result across all tests, and performs
particularly well at the lowest false alarm levels, which
corresponds to the operational setting with a multi-
million gallery and limited resources for visual inspection.
9
 The NIST evaluation recognizes the significant
improvement of face recognition technology over the
past years. As one of the only vendors, Morpho was
able to demonstrate a miss rate lower than 10% at rank
one on a 1.4 million record mugshot dataset.
Morpho technology is also shown to have an increasing
accuracy advantage as the database grows, illustrated
by the limited loss in accuracy while the False
Acceptance Rate (FAR) gets lower. Indeed, when
database grows, the probability to find look-alike people
increases as well, corresponding to an increase of the
FAR. FRVT demonstrates that with a ten-fold population
size increase (from 160 000 to 1.6 million), the miss
rate increases by small factors, Morpho achieving the
smallest one (1.1).
Another important aspect is the threshold stability. The
threshold is the value of the similarity score between
two portraits, above which they will be considered as
coming from the same person. This value is usually set
with the objective of obtaining a predictable FAR. The
threshold often needs to be adjusted as the size of the
database increases, which is not possible in real life
Threshold stability Accuracy (FAR, FRR) with different database sizes for Morpho (algorithm D31C) and
competitor J
10
systems. Our technology demonstrates stable scoring
that allows users to set reliable thresholds that do not
require adjustment as the database grows. Extracted
from NIST FRVT report, the following figure shows that
Morpho (letter D) stabilizes FAR to limit the impact of
the database size, while most competitors (such as J)
exhibit a linear dependence of FAR with population size.
Threshold relates also directly with operator workload.
In face identification systems, a human reviewer is
usually employed to review the candidates returned
from the search. In law enforcement scenarios, the
reviewer inspects the candidates either from a list with
a defined length (up to a fixed rank, e.g. 50) or only
the ones above the threshold. NIST assesses that with
Morpho technology, using a threshold can reduce the
workload by 60% for only a 5% increase in miss rate. In
this aspect, Morphos algorithm exhibits the best trade-
off between accuracy and human workload.
While all other providers algorithms exhibit a quasi
linear dependence of search duration on the enrolled
population size, Morpho appears as a noticeable
exception. With Morphos algorithm, the search duration
increases modestly: a 10 fold increase in database size
gives only a 25% increase in search duration, and thus
in computational power. Such behavior makes a key
difference on large scale systems.
Ranking of performance across algorithms must be
weighed by application-specific requirements. In
regards of real-world scenarios, Morpho delivers top
performance, enlarging the gap with other competitors.
ACCURACY AND SPEED
NIST evaluations provide participants with the
opportunity to compete under the same conditions on
accuracy, but system efficiency must be kept in mind.
Biometric systems are always the result of a trade-
off between accuracy, speed and cost, both for the
hardware platform and human resources involved.
Morpho provides algorithms with operating points
(in terms of accuracy and speed) adapted to real-life
implementations. Morpho is also committed to making
all technical improvements available to customers by
integrating the latest algorithms in its products very
quickly.
4. FUTURE
During the last decade, face recognition research
methodology has matured, sizable training databases
have been collected, and the feedback from practical
deployments has been incorporated, which has led to a
dramatic jump in accuracy.
The last NIST benchmarks conclusions are clear: face
recognition technology is mature enough for operational
use in identification systems with frontal images in a
controlled environment.
Efforts do not stop here. Research is moving forward on
accurately processing more difficult images and video
sequences, with lower resolution and a less controlled
environment. Some of the challenges include external
factors (such as lighting) or subject cooperation (for
instance facial expression or occlusions).
Addressing those challenges will lead to an increased
performance and an extended range of applications for
face recognition.
11
 III. USE CASES
Face recognition is used for both authentication
(checking a persons identity) and identification (finding
a person among a group of known people). Because
face images can be collected either overtly or covertly,
and since people are increasingly willing to give their
portrait than before, this technology offers a wide range
of applications.
1. PUBLIC SECURITY
Like fingerprints or DNA, faces are now leaving
marks that can be used for investigation. Indeed, a
witness can take a picture with his smartphone, video
surveillance cameras are able to record suspects,
images and videos can be fetched from media related
to an investigation. In any case, those marks can be
used to identify a person already known and recorded
in police portrait gallery.
Police mugshots databases are populated with images
taken at the police station during custody. After
police officers make an arrest, they usually collect the
fingerprints, palmprints and mugshots of all suspects,
offering the opportunity to build a database of good
quality to which latent faces can be submitted.
Once the database is created, face recognition can be
useful to various purposes:
Morpho Face Expert with 3D modeling tool
12
INVESTIGATION
Face recognition is well known to be used for
investigation purposes. It allows linking a crime to a
suspect or different crimes together. Another use case
is to help investigators to make a witness identify a
suspect, using face recognition software to speed up
the creation of efficient mugbooks and to organize
lineups.
As an example, Morpho offers its customers the
capacity to process face matching thanks to its
dedicated software: Morpho Face Expert. Its high level
of accuracy and the numerous tools it offers make it
one of the best solutions available on the market. The
Morpho Face Detective software allow police agencies
to build line ups easily and thus identifying suspects in a
more efficient way.
IDENTITY CHECKS IN THE FIELD
Mobile devices can now be equipped with very good
cameras and high performance data transmission
capability allowing identity checks on the field. Police
officers can directly submit search requests to local or
remote face databases and quickly determine whether
an individual is already known to the police forces.
IN JAILS
Face recognition can help control visitors, and verify
that no prisoner escapes at the end of the visit (e.g.
switching with a visitor). It can also be used within the
jail, for instance to make sure prisoners going out to
work areas or to court are the same people who are
coming back.
INTELLIGENCE
Pictures and video streams can reveal a lot of
information to investigators: not only suspects identities
but also other hints such as accomplices or at least
links between individuals.
PREVENTION
Identifying dangerous people is key to prevent incidents
in public places. By setting up surveillance cameras at
the stadium entrance equipped with face recognition
software, blacklisted individuals can be identified on
the spot. In the same way, unlawful demonstrations
recorded by CCTV cameras can be submitted to a face
recognition system.
Comparisons may then lead to the identification
of suspects or at least provide investigators with
information to track them.
Albanian woman taking back her biometric passport
2. CIVIL IDENTITY
Face recognition is particularly well suited to check the
uniqueness of applications for identity papers: ID cards,
passports, driving licenses. All of these documents
display a photo of the holder used to recognize him.
With the development of automatic face recognition
systems, pictures can be used not only to personalize
an identity document but also to fight against multiple
application fraud. Hence, biometric identification
has been implemented to secure identity document
issuance.
As an example, driver licenses include a photograph,
but rarely include any other biometric data. Face
recognition applications can be used to guarantee
that a single motorist cannot possess several driving
licenses.
Since 2008, Morpho is deploying a full identity
management system in Albania with mass enrolment of
3.2 million citizens within five months and the building
of a complete infrastructure for biometric ID cards and
passports production. Face recognition is used for
deduplication purpose and it is a frank success.
13
 3. BORDER CONTROL
SECURITY
Airports are often targeted by terrorists and face
recognition can help a lot in improving their security. A
global surveillance system integrating face recognition
technology and connected to several cameras is able to
detect wanted people registered in black lists.
IDENTITY CHECK
The International Civil Aviation Organization (ICAO)
has been promoting for years Machine Readable
Travel Documents to facilitate and secure border
crossing. The specifications (ICAO document 9303)
require the passports chip to store a picture of the
holder. The chip can be used to automatically check
the passports authenticity while the traveller is going
Morpho Smartgates in New Zealand
14
through the border control gates. Biometric systems
can simulatenously verify that the passenger does not
belong to a list of people who cannot leave or enter the
country.
Following the success met in Australia, Morpho has
deployed in 2009 automatic border control gates
with biometric capacities in New Zealand. The
governements goals were to protect the community
from risks related to international trade and travel
and to facilitating the legitimate movement of people
and goods across the borders. All eligible e-Passport
holders travelers cross the border faster and simpler.
The number of passengers can increase within the
existing facilitation standards and without major
infrastructure changes to airports or increase in staff
numbers. It allows maintaining or improving the integrity
of passenger clearance process.
FAST BOARDING
To ensure fluidity in the transit of airport users, tracking
passengers through the terminal can be facilitated by
face recognition. By measuring the duration of each
step from check-in to boarding, the airport authorities
can fine tune staffing and line opening at each stage.
Furthermore this kind of checking can be used to
ensure that people getting onto the planes have been
through check-in. Face recognition can therefore speed
up identity controls and decrease the passengers
waiting time.
4. AND MANY MORE
Face recognition is becoming increasingly common. It
is used in day-to-day life by everybody, sometimes in
unnoticeable ways. For example, it is used to:
n index media archives,
n recognize VIP customers,
n restrain access to casinos,
n offer personalized pictures to leisure park visitors,
n help security guards limit the access to a shop or
bank agency to safe visitors,
n display the most attractive advertisement depending
on age and gender,
n secure access to electronic devices.
15
 IV. IMPLEMENTATION GUIDELINES
Face recognition is a very challenging technology.
Images vary a lot depending on scale and resolution,
in sharpness and in lighting, notwithstanding artifacts
like compression, interlacing or overlay for identity
documents, red eyes when a flash light is used...
Another element to take into account is the constant
variations of the human face: it is a very mobile and
deformable 3D object, which can change in seconds,
and varies over time with age and physical condition. It
even varies in color So intrinsically, face images are
extremely variable.
This is the reason why the deployment of a face
recognition system must take both technical and
human factors into consideration to be used in the most
efficient way. The environment must be controlled and
its quality checked to get the best results.
1. QUALITY CHECK
Having a good quality database is one of the keys to
get the best results. When the portrait acquisition is
done in a controlled environment, the enrolment should
follow best practices:
- Frontal pose
- Neutral expression
- Clear face to limit occlusion (glasses, headwear, etc)
A camera implementation advice
16
As assessed by the NIST: improvement of image quality
is the largest contributing factor to recognition accuracy.
When possible, the system shall conform to the ISO/
IEC 19794-5 standard, with two complementary
approaches: by design using proper devices and
illumination, and by detection of non conformant images
at the collection stage.
2. CAMERA SET UP
The results of identification in video streams depend
a lot on the setting up of cameras in the environment.
The stakeholders should define in advance how to
implement the cameras. They must be set up at choke
points, where people move at a controlled pace, flow
and direction. Cameras should also fit the following
requirements:
n Control lighting,
n Be placed at a suitable height to correctly acquire the
faces of people of different sizes,
n Capture peoples attention to make them look at the
camera (mirror, advertisement, weather forecast,
news etc.).
3. EXPERTISE IN CHECKING SEARCHES
In investigation use cases, visually recognizing people
can require some training. A number of methods have
been developed to improve the visual recognition
of persons. As an example, the Federal Bureau of
Investigation (FBI) has proposed a facial comparison
and identification training program to provide students
with awareness and understanding of the facial
comparison discipline. This training aims at facilitating
the expanded use of face technologies, interpretation
of the output of face recognition systems, and better
integration of face biometrics into law enforcement and
intelligence work.

4. MANAGING THE EXPECTATIONS

Face recognition seems so simple and intuitive that
expectations relative to this technology are sometimes
out of proportion. For example, for the purpose of
video screening, cameras must provide high resolution
pictures to be used by a face recognition system. Every
potential customer should conduct tests to assess the
suitability of the technology to operational applications
before proceeding with deployment. It is possible to
test usage scenarios, check the results that may be
obtained and measure the workload required. Morpho
can provide support in this assessment process.

17
 CONCLUSION
Morpho has demonstrated the high accuracy of its
algorithms in all NIST evaluations, however the most
valuable demonstration of Morpho performance is
provided by its experience in implementing real life
biometric solutions all around the world.
Empowered by successful deployments, face
recognition still has considerable headroom for
improvement. For instance, these systems could add
3D sensors, recognition of moving faces, processing of
images captured from above or the side, development
of models to integrate ageing, and much more. In a
near future, we expect to see a strong development
of the use of such a technology since solutions will be
more adapted, efficient and performant:
18
n s ocial media with more and more pictures
n trained examiners
n improvement in CCTV and consumers cameras
n capture standardization allowing an easy use of
pictures
But in any case, the performances of face recognition
will definitely depend on the way the system is used
and how it has been implemented. Relying on a partner,
such as Morpho, with extensive experience in biometric
system deployment is one of the key factors for the
success of your system.
 KEY MISSIONS, KEY TECHNOLOGIES, KEY TALENTS

SAFRAN MORPHO

Phone: +33 (0)1 58 11 25 00 - Fax: +33 (0)1 58 11 25 50 - www.morpho.com

Registered office: Le Ponant de Paris - 27, rue Leblanc - F-75512 PARIS CEDEX 15 - FRANCE
Socit anonyme au capital de 159.876.075
440 305 282 RCS PARIS