DIS - 19989-3 - Criteria and Methodology For Security Evaluation of Biometric Systems

DRAFT INTERNATIONAL STANDARD
ISO/IEC DIS 19989-3
ISO/IEC JTC 1/SC 27 Secretariat: DIN

Voting begins on: Voting terminates on:
2019-12-30 2020-03-23
Information security — Criteria and methodology for

security evaluation of biometric systems —
Part 3:
Presentation attack detection
ICS: 35.030
THIS DOCUMENT IS A DRAFT CIRCULATED

FOR COMMENT AND APPROVAL. IT IS
THEREFORE SUBJECT TO CHANGE AND MAY
NOT BE REFERRED TO AS AN INTERNATIONAL
STANDARD UNTIL PUBLISHED AS SUCH.
IN ADDITION TO THEIR EVALUATION AS
BEING ACCEPTABLE FOR INDUSTRIAL, This document is circulated as received from the committee secretariat.
TECHNOLOGICAL, COMMERCIAL AND
USER PURPOSES, DRAFT INTERNATIONAL
STANDARDS MAY ON OCCASION HAVE TO
BE CONSIDERED IN THE LIGHT OF THEIR
POTENTIAL TO BECOME STANDARDS TO
WHICH REFERENCE MAY BE MADE IN Reference number
NATIONAL REGULATIONS.
ISO/IEC DIS 19989-3:2019(E)
RECIPIENTS OF THIS DRAFT ARE INVITED
Licensed to: Plessis, Patrice M.
TO SUBMIT, WITH THEIR COMMENTS,
Downloaded:
NOTIFICATION OF ANY RELEVANT PATENT 2020-03-09
RIGHTS OF WHICH THEY ARE AWARE
SingleAND TO licence only, copying
user and networking prohibited
PROVIDE SUPPORTING DOCUMENTATION. © ISO/IEC 2019
ISO/IEC DIS 19989-3:2019(E)

COPYRIGHT PROTECTED DOCUMENT

© ISO/IEC 2019
All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may
be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting
on the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address
below or ISO’s member body in the country of the requester.
ISO copyright office
CP 401 • Ch. de Blandonnet 8
CH-1214 Vernier, Geneva
Phone: +41 22 749 01 11
Fax: +41 22 749 09 47
Email: copyright@iso.org
Website: www.iso.org
Published in Switzerland Licensed to: Plessis, Patrice M.
Downloaded: 2020-03-09
Single user licence only, copying and networking prohibited
ii © ISO/IEC 2019 – All rights reserved
ISO/IEC DIS 19989-3:2019(E)

Contents Page
Foreword......................................................................................................................................................................................................................................... iv
Introduction...................................................................................................................................................................................................................................v
1 Scope.................................................................................................................................................................................................................................. 1
2 Normative references....................................................................................................................................................................................... 1
3 Terms and definitions...................................................................................................................................................................................... 1
4 Symbols (and abbreviated terms)....................................................................................................................................................11
5 General remark....................................................................................................................................................................................................12
6 Overview of PAD testing in Class ATE and Class AVA.....................................................................................................13
6.1 Objectives and principles............................................................................................................................................................. 13
6.1.1 Class ATE.............................................................................................................................................................................. 13
6.1.2 Class AVA............................................................................................................................................................................... 13
6.2 PAIs used in testing activities................................................................................................................................................... 13
6.2.1 Class ATE.............................................................................................................................................................................. 13
6.2.2 Class AVA............................................................................................................................................................................... 14
6.3 Testing activities.................................................................................................................................................................................. 14
6.3.1 Class ATE.............................................................................................................................................................................. 14
6.3.2 Class AVA............................................................................................................................................................................... 14
6.4 Criteria of pass/failure................................................................................................................................................................... 14
7 Supplementary activities to ISO/IEC 18045 on Tests (ATE)..................................................................................15
7.1 Testing approach toward PAD.................................................................................................................................................. 15
7.2 Metrics for PAD testing.................................................................................................................................................................. 15
7.2.1 General................................................................................................................................................................................... 15
7.2.2 Metrics used for PAD subsystem TOEs....................................................................................................... 16
7.2.3 Metrics used for data capture subsystem TOEs.................................................................................. 16
7.2.4 Metrics used for other TOEs................................................................................................................................ 17
7.3 Minimum test sizes and maximum error rates......................................................................................................... 18
8 Supplementary activities to ISO/IEC 18045 on Vulnerability Assessment (AVA)...........................18
8.1 Penetration testing using PAI variations........................................................................................................................ 18
8.2 Potential vulnerabilities................................................................................................................................................................ 19
8.3 Rating of vulnerabilities and TOE resistanc................................................................................................................. 19
Annex A (Informative) Examples of calculations of attack potential................................................................................20
Bibliography.............................................................................................................................................................................................................................. 25

© ISO/IEC 2019 – All rights reserved iii
ISO/IEC DIS 19989-3:2019(E)

Foreword
ISO (the International Organization for Standardization) and IEC (the International Electrotechnical
Commission) form the specialized system for worldwide standardization. National bodies that are
members of ISO or IEC participate in the development of International Standards through technical
committees established by the respective organization to deal with particular fields of technical
activity. ISO and IEC technical committees collaborate in fields of mutual interest. Other international
organizations, governmental and non-governmental, in liaison with ISO and IEC, also take part in the
work. In the field of information technology, ISO and IEC have established a joint technical committee,
ISO/IEC JTC 1.
The procedures used to develop this document and those intended for its further maintenance are
described in the ISO/IEC Directives, Part 1. In particular the different approval criteria needed for
the different types of document should be noted. This document was drafted in accordance with the
editorial rules of the ISO/IEC Directives, Part 2 (see www.iso.org/directives).
Attention is drawn to the possibility that some of the elements of this document may be the subject
of patent rights. ISO and IEC shall not be held responsible for identifying any or all such patent
rights. Details of any patent rights identified during the development of the document will be in the
Introduction and/or on the ISO list of patent declarations received (see www.iso.org/patents).
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation on the meaning of ISO specific terms and expressions related to conformity assessment,
as well as information about ISO's adherence to the World Trade Organization (WTO) principles in the
Technical Barriers to Trade (TBT) see the following URL www.iso.org/iso/foreword.html.
The committee responsible for this document is ISO/IEC JTC 1, Information technology, SC XXX
This second/third/... edition cancels and replaces the first/second/... edition (), [clause(s) / subclause(s)
/ table(s) / figure(s) / annex(es)] of which [has / have] been technically revised.
ISO XXXX consists of the following parts. [Add information as necessary.]

iv © ISO/IEC 2019 – All rights reserved
ISO/IEC DIS 19989-3:2019(E)

Introduction
Biometric systems may be vulnerable to presentation attacks where attackers attempt to subvert the
system security policy by presenting their natural biometric characteristics or artefacts holding copied
or faked characteristics. Presentation attacks can occur during enrolment or identification/verification
events. Techniques designed to detect presentation artefacts are generally different from those to detect
attacks where natural characteristics are used. Defence against presentation attacks with natural
characteristics typically relies on the ability of a biometric system to discriminate between genuine
enrolees and attackers based on the differences between their natural biometric characteristics. This
ability is characterised by the biometric recognition performance of the system. Biometric recognition
performance and presentation attack detection have a bearing on the security of biometric systems.
Hence the evaluation of these aspects of performance from a security viewpoint will become important
considerations for the procurement of biometric products and systems.
Biometric products and systems share many of the properties of other IT products and systems which
are amenable to security evaluation using ISO/IEC 15408 series and ISO/IEC 18045 in the standard
way. However, biometric systems embody certain functionality that needs specialised evaluation
criteria and methodology which is not addressed by ISO/IEC 15408 series and ISO/IEC 18045. Mainly
these relate to the evaluation of biometric recognition and presentation attack detection. These are the
functions addressed in this document.
ISO/IEC 19792 describes these biometric-specific aspects and specifies principles to be considered
during the security evaluation of biometric systems. However, it does not specify the concrete criteria
and methodology that are needed for security evaluation based on ISO/IEC 15408 series.
ISO/IEC 19989 series provide a bridge between the evaluation principles for biometric products and
systems defined in ISO/19792 and the criteria and methodology requirements for security evaluation
based on ISO/IEC 15408 series. The multi-part standard ISO/IEC 19989 supplements ISO/IEC 15408
series and ISO/IEC 18045 by providing extended security functional requirements together with
assurance activities related to these requirements. The extensions to the requirements and assurance
activities found in ISO/IEC 15408 series and ISO/IEC 18045 relate to the evaluation of biometric
recognition and presentation attack detection which are particular to biometric systems.
This document provides guidance and requirements to the developer and the evaluator for the
supplementary activities on presentation attack detection specified in ISO/IEC 19989-1. It builds on
the general considerations described in ISO/IEC 19792 and the presentation attack detection testing
methodology described in ISO/IEC 30107-3 by providing additional guidance to the evaluator.
In this document, the term "user" is used to mean the term "capture subject" used in biometrics.

© ISO/IEC 2019 – All rights reserved v
DRAFT INTERNATIONAL STANDARD ISO/IEC DIS 19989-3:2019(E)
Information security — Criteria and methodology for

security evaluation of biometric systems —
Part 3:
Presentation attack detection
1 Scope
For security evaluation of biometric verification systems and biometric identification systems,
this document is dedicated to security evaluation of presentation attack detection applying the
ISO/IEC 15408 series. It provides:
— Guidance and requirements to the developer and the evaluator for the supplementary activities on
presentation attack detection specified in ISO/IEC 19989-1.
This document is applicable only to TOEs for single biometric characteristic type. However, the selection
of a characteristic from multiple characteristics in SFRs is allowed.
2 Normative references
The following documents are referred to in the text in such a way that some or all of their content
constitutes requirements of this document. For dated references, only the edition cited applies. For
undated references, the latest edition of the referenced document (including any amendments) applies.
ISO/IEC 15408-3:2008, Information technology — Security techniques — Evaluation criteria for IT
security — Part 3: Security assurance components
ISO/IEC 18045:2008, Information technology — Security techniques — Methodology for IT security
evaluation
ISO/IEC 19989-1, Information Technology — Security techniques — Criteria and methodology for security
evaluation of biometric systems – Part 1: framework
ISO/IEC 30107-3, Information technology — Biometric presentation attack detection — Part 3: Testing
and reporting
3 Terms and definitions

For the purposes of this document, the following terms and definitions apply.
ISO and IEC maintain terminological databases for use in standardization at the following addresses:
— IEC Electropedia: available at http://w ww.electropedia.org/
— ISO Online browsing platform: available at http://w ww.iso.org/obp
3.1
activity
application of an assurance class of ISO/IEC 15408-3
[SOURCE: ISO/IEC 18045: 2008, 3.2]

© ISO/IEC 2019 – All rights reserved 1
ISO/IEC DIS 19989-3:2019(E)

3.2
assurance
grounds for confidence that a TOE meets the SFRs
[SOURCE: ISO/IEC 15408-1: 2009, 3.1.4]
3.3
attack potential
measure of the effort to be expended in attacking a TOE, expressed in terms of an attacker's expertise,
resources and motivation
[SOURCE: ISO/IEC 15408-1: 2009, 3.1.5]
3.4
attack presentation classification error rate
APCER
proportion of attack presentations using the same PAI species incorrectly classified as bona fide
presentations in a specific scenario
[SOURCE: ISO/IEC 30107-3: 2017, 3.2.1]
3.5
attack presentation non-response rate
APNRR
proportion of attack presentations using the same PAI species that cause no response at the PAD
subsystem or data capture subsystem
EXAMPLE A fingerprint system may not register or react to the presentation of a PAI due to the PAI’s lack of
realism.
[SOURCE: ISO/IEC 30107-3: 2017, 3.2.3]

3.6
attack type
element and characteristic of a presentation attack, including PAI species, concealer or impostor attack,
degree of supervision, and method of interaction with the capture device
[SOURCE: ISO/IEC 30107-3: 2017, 3.1.3]
3.7
biometric (adjective)
of or having to do with biometrics
EXAMPLE 1 Incorrect usage #1: ICAO resolved that face is the biometric most suited to the practicalities
of travel documents.
EXAMPLE 2 Correct usage #1: ICAO resolved that face recognition is the biometric mode most suited to the
practicalities of travel documents.
EXAMPLE 3 Incorrect usage #2: The biometric recorded in my passport is a facial image.
EXAMPLE 4 Correct usage #2: T he biometric characteristic recorded in my passport is a facial image.
Note 1 to entry: The use of biometric as a noun, to mean for example, biometric characteristic, is deprecated.
Note 2 to entry: Since the late 19th century, the designations ‘biometrics’ and ‘biometry’ have been used with
the general meaning of counting, measuring and statistical analysis of any kind of data in the biological sciences
including the relevant medical sciences.
[SOURCE: ISO/IEC 2382-37:2017, 3.1.1]

2 © ISO/IEC 2019 – All rights reserved
ISO/IEC DIS 19989-3:2019(E)

3.8
biometric capture
obtain and record, in a retrievable form, signal(s) of biometric characteristic(s) directly from
individual(s), or from representation(s) of biometric characteristic(s)
Note 1 to entry: Representation’ is used in the natural language sense, e.g. a photograph.
Note 2 to entry: Retrievable’ refers to the record and not the original signal.
Note 3 to entry: A signal can be generated by the biometric characteristic or generated elsewhere and affected by
the biometric characteristic. For example, face illuminated by incident light.
[SOURCE: ISO/IEC 2382-37:2017, 3.6.3]

3.9
biometric capture device
device that collects a signal from a biometric characteristic and converts it to a captured biometric sample
Note 1 to entry: A signal can be generated by the biometric characteristic or generated elsewhere and affected by
the biometric characteristic, for example, face illuminated by incident light.
Note 2 to entry: A biometric capture device can be any piece of hardware (and supporting software and firmware).
Note 3 to entry: A biometric capture device may comprise components such as an illumination source, one or
more biometric sensors, etc.
[SOURCE: ISO/IEC 2382-37:2017, 3.4.1]

Note 4 to entry: In this document, the expression "capture device" is sometimes used instead of this term.
Note 5 to entry: Biometric sample is defined as analog or digital representation of biometric characteristics prior
to biometric feature extraction at ISO/IEC 2382-37:2017, 3.3.21.
3.10
biometric characteristic
biological and behavioural characteristic of an individual from which distinguishing, repeatable
biometric features can be extracted for the purpose of biometric recognition
EXAMPLE Examples of biometric characteristics are: Galton ridge structure, face topography, facial skin
texture, hand topography, finger topography, iris structure, vein structure of the hand, ridge structure of the
palm, retinal pattern, handwritten signature dynamics, etc.
[SOURCE: ISO/IEC 2382-37:2017, 3.1.2]

Note 1 to entry: Biometric feature is defined as numbers or labels extracted from biometric samples and used for
comparison at ISO/IEC 2382-37:2017, 3.3.11.
3.11
biometric concealer
subversive biometric capture subject who performs a biometric concealment attack
[SOURCE: ISO/IEC 2382-37:2017, 3.7.21]
Note 1 to entry: In this document, the expression "concealer" is sometimes used instead of this term.
3.12
biometric enrolment
DEPRECATED: registration
act of creating and storing a biometric enrolment data record in accordance with an enrolment policy
Note 1 to entry: Registration has a different meaning in the signal processing community and its use is therefore
deprecated in biometrics in favour of enrolment.

ISO/IEC DIS 19989-3:2019(E)

Note 2 to entry: Enrolment in a biometric system might, in some cases, not involve storage of biometric data, for
example, when biometric data from an enrolee cannot be acquired.
[SOURCE: ISO/IEC 2382-37:2017, 3.5.3]

Note 3 to entry: In this document, the expression "enrolment" is sometimes used instead of this term.
3.13
biometric identification
process of searching against a biometric enrolment database to find and return the biometric reference
identifier(s) attributable to a single individual
Note 1 to entry: Use of the term “authentication” as a substitute for biometric identification is deprecated.
[SOURCE: ISO/IEC 2382-37:2017, 3.8.2]

Note 2 to entry: Biometric enrolment database is defined as database of biometric enrolment data record(s) at
ISO/IEC 2382-37:2017, 3.3.9.
Note 3 to entry: Biometric reference is defined as one or more stored biometric samples, biometric templates
or biometric models attributed to a biometric data subject and used as the object of biometric comparison at
ISO/IEC 2382-37:2017, 3.3.16.
3.14
biometric impostor
subversive biometric capture subject who performs a biometric impostor attack
Note 1 to entry: COED defines impostor as person who assumes a false identity in order to deceive or defraud.
Note 2 to entry: COED defines impersonate as pretend to be (another person) for entertainment or fraud.
[SOURCE: ISO/IEC 2382-37:2017, 3.7.13]

Note 3 to entry: In this document, the expression "impostor" is sometimes used instead of this term.
3.15
biometric recognition
biometrics
automated recognition of individuals based on their biological and behavioural characteristics
Note 1 to entry: In the field of biometrics (as defined in this document), “Individual” is restricted in scope to refer
only to humans.
Note 2 to entry: The general meaning of biometrics encompasses counting, measuring and statistical analysis of
any kind of data in the biological sciences including the relevant medical sciences.
Note 3 to entry: Biometric recognition encompasses biometric verification and biometric identification (3.8.2).
Note 4 to entry: Automated recognition implies that a machine based system is used for the recognition either for
the full process or assisted by a human being.
Note 5 to entry: Behavioural and biological characteristics cannot be completely separated which is why
the definition uses ‘and’ instead of ‘and/or’. For example, a fingerprint image results from the biological
characteristics of the finger ridge patterns and the behavioural act of presenting the finger.
Note 6 to entry: Use of ‘authentication’ as a synonym for “biometric verification or biometric identification” is

deprecated; the term biometric recognition is preferred.
[SOURCE: ISO/IEC 2382-37:2017, 3.1.3]

ISO/IEC DIS 19989-3:2019(E)

3.16
biometric system
system for the purpose of the biometric recognition of individuals based on their behavioural and
biological characteristics
Note 1 to entry: A biometric system will contain both biometric and non-biometric components.
[SOURCE: ISO/IEC 2382-37:2017, 3.2.3]

3.17
biometric verification
process of confirming a biometric claim (3.6.4) through biometric comparison
Note 1 to entry: Use of the term “authentication” as a substitute for biometric verification is deprecated.
[SOURCE: ISO/IEC 2382-37:2017, 3.8.3]

3.18
bona fide presentation
interaction of the biometric capture subject and the biometric data capture subsystem in the fashion
intended by the policy of the biometric system
Note 1 to entry: Bona fide is analogous to normal or routine, when referring to a bona fide presentation.
Note 2 to entry: Bona fide presentations can include those in which the user has a low level of training or
skill. Bona fide presentations encompass the totality of good-faith presentations to a biometric data capture
subsystem.
[SOURCE: ISO/IEC 30107-3: 2017, 3.1.2]

3.19
bona fide presentation classification error rate
BPCER
proportion of bona fide presentations incorrectly classified as presentation attacks in a specific
scenario
[SOURCE: ISO/IEC 30107-3: 2017, 3.2.2]
3.20
bona fide presentation non-response rate
BPNRR
proportion of bona fide presentations that cause no response at the PAD subsystem or data capture
subsystem
[SOURCE: ISO/IEC 30107-3: 2017, 3.2.4]
3.21
class
set of ISO/IEC 15408 families that share a common focus
[SOURCE: ISO/IEC 15408-1: 2009, 3.1.9]
3.22
comparison
estimation, calculation or measurement of similarity or dissimilarity between biometric probe(s)
[SOURCE: ISO/IEC 2382-37:2017, 3.5.7]

ISO/IEC DIS 19989-3:2019(E)

3.23
component
smallest selectable set of elements on which requirements may be based
[SOURCE: ISO/IEC 15408-1: 2009, 3.1.12]
Note 1 to entry: Element is defined as indivisible statement of a security need at ISO/IEC 15408-1: 2009, 3.1.24.
3.24
concealer attack presentation non-identification rate
CAPNIR
<full-system evaluation of an identification system> proportion of concealer presentation attacks using
the same PAI species in which the reference identifier of the concealer is not among the identifiers
returned or, depending on intended use case, in which no identifiers are returned
Note 1 to entry: In a negative identification system, such as a black-list, the concealer could intend that no
identifiers are returned to avoid scrutiny by a human operator.
[SOURCE: ISO/IEC 30107-3: 2017, 3.2.9]

3.25
concealer attack presentation non-match rate
CAPNMR
<full-system evaluation of a verification system> proportion of concealer attack presentations using
the same PAI species in which the reference of the concealer is not matched
[SOURCE: ISO/IEC 30107-3: 2017, 3.2.7]
3.26
describe
provide specific details of an entity
[SOURCE: ISO/IEC 15408-1: 2009, 3.1.21]
3.27
determine
affirm a particular conclusion based on independent analysis with the objective of reaching a particular
conclusion
Note 1 to entry: The usage of this term implies a truly independent analysis, usually in the absence of any previous
analysis having been performed. Compare with the terms “confirm” or “verify” which imply that an analysis has
already been performed which needs to be reviewed
[SOURCE: ISO/IEC 15408-1: 2009, 3.1.22]

Note 2 to entry: Verify is defined as rigorously review in detail with an independent determination of sufficiency
at ISO/IEC 15408-1: 2009, 3.1.84.
3.28
developer
organisation responsible for the development of the TOE
[SOURCE: ISO/IEC 15408-1: 2009, 3. 4.15]
3.29
development
product life-cycle phase which is concerned with generating the implementation representation of the
TOE
Note 1 to entry: Throughout the ALC: Life-cycle support requirements, development and related terms (developer,
develop) are meant in the more general sense to comprise development and production.

ISO/IEC DIS 19989-3:2019(E)

[SOURCE: ISO/IEC 15408-1: 2009, 3. 4.16]

Note 2 to entry: Life-cycle is defined as sequence of stages of existence of an object (for example a product or a
system) in time at ISO/IEC 15408-1: 2009, 3.4.19.
3.30
enrol
create and store a biometric enrolment data record in accordance with the biometric enrolment policy
[SOURCE: ISO/IEC 2382-37:2017, 3.5.8]
3.31
ensure
guarantee a strong causal relationship between an action and its consequences
Note 1 to entry: When this term is preceded by the word “help” it indicates that the consequence is not fully
certain, on the basis of that action alone.
[SOURCE: ISO/IEC 15408-1: 2009, 3.1.25]

3.32
evaluation
assessment of a PP, an ST or a TOE, against defined criteria
[SOURCE: ISO/IEC 15408-1: 2009, 3.1.26]
3.33
examine
generate a verdict by analysis using evaluator expertise
Note 1 to entry: The statement that uses this verb identifies what is analysed and the properties for which it is
analysed.
[SOURCE: ISO/IEC 18045: 2008, 3.7]

3.34
exploitable vulnerability
weakness in the TOE that can be used to violate the SFRs in the operational environment for the TOE
[SOURCE: ISO/IEC 15408-1: 2009, 3.5.3]
3.35
failure-to-acquire rate
FTAR
proportion of a specified set of biometric acquisition processes that were failures to acquire
Note 1 to entry: The results of the biometric acquisition processes may be biometric probes or biometric
references.
Note 2 to entry: The experimenter specifies which biometric probe (or biometric reference) acquisitions are in
the set, as well as the criteria for deeming a biometric acquisition process has failed.
Note 3 to entry: The proportion is the number of processes that failed divided by the total number of biometric
acquisition processes within the specified set.
[SOURCE: ISO/IEC 2382-37:2017, 3.9.4]

3.36
failure-to-enrol rate
FTER
proportion of a specified set of biometric enrolment transactions that resulted in a failure to enrol
Note 1 to entry: Basing the denominator on the number of biometric enrolment transactions may result in a
higher value than basing it on the
Licensed to:number
Plessis, of biometric
Patrice M. capture subjects.
ISO/IEC DIS 19989-3:2019(E)

Note 2 to entry: If the FTER is to measure solely transactions that fail to complete due to quality of the submitted
biometric data the denominator should not include transactions that fail due to non-biometric reasons (i.e. lack of
eligibility due to age or citizenship).
[SOURCE: ISO/IEC 2382-37:2017, 3.9.7]

3.37
false match rate
FMR
proportion of the completed biometric non-mated comparison trials (3.9.2) that result in a false match
Note 1 to entry: The value computed for the false match rate will depend on thresholds, and other parameters of
the comparison process, and the protocol defining the biometric non-mated comparison trials.
Note 2 to entry: Comparisons between:
— identical twins;
— different, but related biometric characteristics from the same individual, such as left and right hand
topography will need proper consideration (see ISO/IEC 19795-1).
Note 3 to entry: “Completed” refers to the computational processes required to make a comparison decision, i.e.
failures to decide are excluded.
[SOURCE: ISO/IEC 2382-37:2017, 3.9.9]

Note 4 to entry: Threshold is defined as numerical value (or set of values) at which a decision boundary exists at
ISO/IEC 2382-37:2017, 3.3.36.
3.38
false-negative identification-error rate
FNIR
proportion of identification transactions by users enrolled in the system in which the user’s correct
identifier is not among those returned
[SOURCE: ISO/IEC 19795-1:2006, 4.6.8]
3.39
false non-match rate
FNMR
proportion of the completed biometric mated comparison trials that result in a false non-match
Note 1 to entry: The value computed for the false non-match rate will depend on thresholds, and other parameters
of the comparison process, and the protocol defining the biometric mated comparison trials.
Note 2 to entry: “Completed” refers to the computational processes required to make a comparison decision, i.e.
failures to decide are excluded.
[SOURCE: ISO/IEC 2382-37:2017, 3.9.11]

3.40
false-positive identification-error rate
FPIR
proportion of identification transactions by users not enrolled in the system, where an identifier is
returned
[SOURCE: ISO/IEC 19795-1:2006, 4.6.9]
3.41
family
set of components that share a similar goal but differ in emphasis or rigour
[SOURCE: ISO/IEC 15408-1: 2009, 3.1.34]
ISO/IEC DIS 19989-3:2019(E)

3.42
impostor attack presentation identification rate
IAPIR
<full-system evaluation of an identification system> proportion of impostor attack presentations using
the same PAI species in which the targeted reference identifier is among the identifiers returned or,
depending on intended use case, at least one identifier is returned by the system
Note 1 to entry: An attacker might be both an impostor (trying to match an existing non-self enrolee) and a
concealer (obscuring his real biometric sample with a PAI).
[SOURCE: ISO/IEC 30107-3: 2017, 3.2.8]

3.43
impostor attack presentation match rate
IAPMR
<full-system evaluation of a verification system> proportion of impostor attack presentations using the
same PAI species in which the target reference is matched
[SOURCE: ISO/IEC 30107-3: 2017, 3.2.6]
3.44
match (noun)
comparison decision stating that the biometric probe(s) and the biometric reference are from the
same source
Note 1 to entry: Historically, the word match has been used as a verb to indicate the act of comparison and
decision making. As ‘match’ is the decision coming out of the comparison process, its use as a verb is deprecated
in favour of compare.
[SOURCE: ISO/IEC 2382-37:2017, 3.3.31]

3.45
methodology
system of principles, procedures and processes applied to IT security evaluations
[SOURCE: ISO/IEC 18045: 2008, 3.9]
3.46
non-standard PAI
PAI not corresponding to a standard PAI species.
3.47
PAI species
class of presentation attack instruments created using a common production method and based on
different biometric characteristics
EXAMPLE 1 A set of fake fingerprints all made in the same way with the same materials but with different
friction ridge patterns would constitute a PAI species.
EXAMPLE 2 A specific type of alteration made to the fingerprints of several data capture subjects would
constitute a PAI species.
Note 1 to entry: The term “recipe” is often used to refer to how to make a PAI species.
Note 2 to entry: Presentation attack instruments of the same species may have different success rates due to
variability in the production process.
[SOURCE: ISO/IEC 30107-3: 2017, 3.1.6]

ISO/IEC DIS 19989-3:2019(E)

3.48
penetration testing
testing used in vulnerability analysis for vulnerability assessment, trying to reveal vulnerabilities of
the TOE based on the information about the TOE gathered during the relevant evaluation activities
Note 1 to entry: In ISO/IEC 15408 series, this term is used without definition.
3.49
presentation attack
presentation to the biometric data capture subsystem with the goal of interfering with the operation of
the biometric system
Note 1 to entry: Presentation attack can be implemented through a number of methods, e.g. artefact, mutilations,
replay, etc.
Note 2 to entry: Presentation attacks may have a number of goals, e.g. impersonation or not being recognized.
Note 3 to entry: Biometric systems may not be able to differentiate between biometric presentation attacks with
the goal of interfering with the systems operation and non-conformant presentations.
[SOURCE: ISO/IEC 30107-1: 2016, 3.5]

Note 4 to entry: Bimetric presentation is defined as interaction of the biometric capture subject and the biometric
capture subsystem to obtain a signal from a biometric characteristic at ISO/IEC 2382-37:2017, 3.6.7.
3.50
presentation attack detection
PAD
automated determination of a presentation attack
Note 1 to entry: PAD cannot infer the subject’s intent. In fact it may be impossible to derive that difference from
the data capture process or acquired sample.
[SOURCE: ISO/IEC 30107-1: 2016, 3.6]

3.51
presentation attack instrument
PAI
biometric characteristic or object used in a presentation attack
Note 1 to entry: The set of PAI includes artefacts but would also include lifeless biometric characteristics (i.e.
stemming from dead bodies) or altered biometric characteristics (e.g. altered fingerprints) that are used in
an attack.
[SOURCE: ISO/IEC 30107-1: 2016, 3.7]

3.52
Protection Profile
implementation-independent statement of security needs for a TOE type
[SOURCE: ISO/IEC 15408-1: 2009, 3.1.52]
3.53
report
include evaluation results and supporting material in the Evaluation Technical Report or an
Observation Report
[SOURCE: ISO/IEC 18045: 2008, 3.14]

ISO/IEC DIS 19989-3:2019(E)

3.54
scheme
set of rules, established by an evaluation authority, defining the evaluation environment, including
criteria and methodology required to conduct IT security evaluations
[SOURCE: ISO/IEC 18045: 2008, 3.15]
3.55
Security Target
implementation-dependent statement of security needs for a specific identified TOE
[SOURCE: ISO/IEC 15408-1: 2009, 3.1.63]
3.56
standard PAI
PAI in standard PAI species.
3.57
standard PAI species
PAI species determined and specified as standard by a certification body for the purpose of conducting
evaluations
Note 1 to entry: Standard PAI species may not be specified by the certification body. In such a case, the developer
as well as the evaluator prepares non-standard PAIs to use in evaluation activities.
3.58
target of evaluation
set of software, firmware and/or hardware possibly accompanied by guidance
[SOURCE: ISO/IEC 15408-1: 2009, 3.1.70]
3.59
vulnerability
weakness in the TOE that can be used to violate the SFRs in some environment
[SOURCE: ISO/IEC 15408-1: 2009, 3.5.5]
4 Symbols (and abbreviated terms)

APAR
Attack Presentation Acquisition Rate
APCER
Attack Presentation Classification Error Rate
APNRR
Attack Presentation Non-Response Rate
ADV
SAR (Security Assurance Requirement) class of DeVelopment
NOTE The class name is defined in ISO/IEC 15408-3. Here A of ADV stands for Assurance require-
ment, DV for DeVelopment. The class name is defined in this way in ISO/IEC 15408.
ATE
SAR (Security Assurance Requirement) class of tests
AVA
SAR (Security Assurance Requirement) class of Vulnerability Assessment
AVA_VAN
SAR (Security Assurance Requirement) family for Vulnerability Analysis in Class AVA
ISO/IEC DIS 19989-3:2019(E)

BPCER
Bona fide Presentation Classification Error Rate
BPNRR
Bona fide Presentation Non-Response Rate
CAPNIR
Concealer Attack Presentation Non-Identification Rate
CAPNMR
Concealer Attack Presentation Non-Match rateI
IAPIR
Impostor Attack Presentation Identification Rate
IAPMR
Impostor Attack Presentation Match Rate
FMR
False Match Rate
FNIR
False-Negative Identification-error Rate
FNMR
False Non-Match rate
FPIR
False-Positive Identification-error Rate
FTAR
Failure To Acquire Rate
FTER
Failure To Enrol Rate
PAD
Presentation Attack Detection
PAI
Presentation Attack Instrument
PP
Protection Profile
SFR
Security Functional Requirement
ST
Security Target
TOE
Target Of Evaluation
5 General remark
In addition to the requirements and recommendations provided in this document, those in
ISO/IEC 15408-3 and ISO/IEC 18045 shall be applied.

ISO/IEC DIS 19989-3:2019(E)

6 Overview of PAD testing in Class ATE and Class AVA
6.1 Objectives and principles
6.1.1 Class ATE
The activities in Class ATE focus on the question whether the provided PAD mechanisms work as
specified. Functional testing can demonstrate the existence of PAD vulnerabilities in the TOE (i.e. non
zero error rates) but it cannot prove that no vulnerabilities exist.
Functional testing of the effectiveness of the PAD capability of the TOE is done by checking for the
successes and failures of detection by the TOE of PAIs using a statistically based test methodology (i.e.
the measurement of PAD success and error rates), in order to demonstrate that PAD capability exists
and that the PAD error rates meet the specification in the ATE_FUN documentation. ATE_IND may or
may not include statistical testing depending on the evaluation context.
Note that the functional testing described in this document is distinct from the functional testing
of biometric recognition performance using the natural biometric characteristics of test subjects
described in ISO/IEC 19989-2.
6.1.2 Class AVA
Penetration testing involves investigating the potential vulnerabilities of a TOE to presentation attacks
which might not have been uncovered by previous functional testing (class ATE). This could include
standard PAIs and variants of standard PAIs used in functional testing, and new PAIs which are created
to expose possible PAD weaknesses in the capture hardware or software algorithms used in e.g. signal
processing and biometric comparison. Penetration testing does not involve the statistical testing
approach used for functional testing (class ATE).
Note that testing with PAIs is subject to presentation variability and PAI preparation variability.
Testing should be continued until an appropriate level of confidence in the results of the test is achieved
corresponding to the level of the assurance family AVA_VAN specified in the ST of the TOE.
6.2 PAIs used in testing activities
6.2.1 Class ATE
Standard PAIs shall be prepared and used in accordance with specifications and instructions, if
provided by the relevant certification body. Standard PAIs may be supplied to the developer and the
evaluator by the certification body, or prepared by the developer and the evaluator in accordance with
specifications and instructions provided by the certification body.
NOTE The use of natural biometrics as PAI is included in testing activities if SFR(s) such as FPT_BCP.1, FIA_
EBR.1, FIA_BVR.4 and FIA_BID.4 specified in ISO/IEC 19989-1 are selected in the ST. Even if those SFRs are not
selected, natural biometric PAIs could be part of the standard PAIs. As described in 6.4.2.1, 7.5.1.1, 7.5.6.1, and
7.5.10.1 of ISO/IEC 19989-1, natural biometric PAIs include natural biometric characteristics presented with
movements, rotations, or distances against the specification of the capture device. This also applies to Class AVA.
If the certification body does not provide the standard PAIs, non-standard PAIs shall be prepared by the
developer and supplied to the evaluator by the developer.
The PAIs used by the evaluator for ATE will vary with the information available to the evaluator. By
default the evaluator should rely on the standard PAI species. Additionally, the evaluator may rely on
state-of-the-art to determine whether the tests that are conducted are representative of the PAIs that
can be used on the TOE. Overall, the representativeness of the PAIs used during ATE has an impact on
the interpretation of a pass result in ATE with respect to TOE robustness, and the usefulness of ATE for
AVA activities.

ISO/IEC DIS 19989-3:2019(E)

6.2.2 Class AVA
Non-standard PAIs shall be created and used by the evaluator in penetration testing. Note that the
accuracy and repeatability of the test results are subject to statistical limitations e.g. number of test
presentations and variability between the presentations.
6.3 Testing activities
6.3.1 Class ATE
It is the objective of any functional testing activity conducted in Class ATE to determine whether the
PAD mechanism is able to detect PAIs with sufficient reliability. If ATE_FUN.1 or ATE_FUN.2 is chosen
in the ST, the developer shall conduct functional testing using standard PAI species at least if they are
provided. The developer may prepare non-standard PAIs to conduct additional functional testing which
will give information on the nature of the PAIs the evaluator should focus on in order to reduce the
evaluator’s evaluation activities.
The evaluator shall conduct independent testing using PAIs selected from standard PAIs, if provided, or
non-standard PAIs.
The values for maximum error rates to be validated in the evaluation are specified in the TOE ATE_FUN
documentation and the implications of the values to the test sizes are further discussed in 6.3.
The error rates shall be reported independently for each PAI species tested. The maximum error rate
measured for all PAI species tested is considered as the main indicator on how well the TOE performs in
detecting given PAI species.
NOTE ADV documents are disclosed only to the evaluator and the certification body while the ST is
publicised when the TOE is certified.
6.3.2 Class AVA
While the testing activities in context of ATE allow an overall assessment of the performance of the
PAD mechanism for the tested PAIs, they do not provide any confidence about the PAD effectiveness for
other PAIs. It falls to the vulnerability assessment to evaluate whether the use of additional PAIs that
have not been part of the standard PAI species or variations of PAIs from the standard PAI species can
lead to exploitable vulnerabilities.
During the vulnerability analysis the evaluator will use information and knowledge gained during the
evaluation of the other assurance classes for penetration testing. Any information found in the previous
evaluation activities shall be made available as input to the activities for the AVA evaluation activities
described in this document.
Penetration testing depends on the evaluator’s expertise, skill, and knowledge on potential PAD
vulnerabilities, such as identification of possible areas of weakness, iteratively probing these areas
using specially prepared PAIs, refining the PAIs, and the presentation techniques to attempt to find
vulnerabilities, based on public and private sources of information available on vulnerabilities and PAIs.
Penetration testing is characterized as an activity based on knowledge, expertise, skill, and learning
to penetrate the TOE using specific PAIs, for which the attack potential is calculated from associated
information such as expertise ,effort, time, cost, etc. needed by the evaluator to identify and exploit the
vulnerabilities
NOTE Penetration testing cannot prove that no vulnerabilities exist even if it fails to uncover any PAD
vulnerabilities in the TOE.
6.4 Criteria of pass/failure

The TOE shall pass the evaluation only if:
ISO/IEC DIS 19989-3:2019(E)

The functional testing shows that the TOE is able to recognize the PAIs within the maximum error rates
stated in the TOE ATE_FUN documentation;
And
The vulnerability analysis shows that dedicated variations of standard PAIs or any other innovative
non-standard PAIs designed by the evaluator do not lead to vulnerabilities with an attack below the
considered attack potential.
7 Supplementary activities to ISO/IEC 18045 on Tests (ATE)
7.1 Testing approach toward PAD

The main objective of the testing activity for PAD systems is to demonstrate that the PAD mechanism
is able to detect presentation attacks with sufficient reliability. To achieve this, the developer shall
determine the rate at which the TOE fails to detect PAIs of a given PAI species - the Attack Presentation
Classification Error Rate (APCER) for that PAI species.
In order to determine the APCER of the PAD mechanism, the developer shall prepare PAIs using
information provided by the certification body.
In the course of their testing activity the PAIs prepared by the developer shall be presented to the PAD
system and the resulting PAD decision (presentation attack detected / presentation attack not detected)
shall be recorded. The developer shall prepare and test using the standard PAI species applicable to the
TOE and may extend the testing to include non-standard PAIs.
In order to judge whether or not a PAD mechanism is working adequately, the maximum values for
APCER and definitions for the minimal number of attack types and PAI species shall be defined in the
TOE ATE_FUN documentation.
As required by ATE_IND.2 and ATE_IND.3, the evaluator shall repeat a subset of the developer tests and
also devise their own tests in order to gain confidence in the testing activity of the developer. For the
repetition of developer tests the developer shall provide the description of their PAIs to the evaluation
body. Additionally, the evaluation body shall create their own PAIs based on the more detailed
information from the complete documentation of the standard PAI species provided by the certification
body. Therefore, independence and a required degree of variation is given.
The maximum APCER and minimal test sizes as introduced above should also be considered for ATE_
IND. Note that the standard PAI species describes different classes of PAIs which shall be prepared
according to the preparation manuals presented by the certification body. The maximum APCER value
shall separately be assigned to every PAI species and not only for the set of all prepared PAIs.
Summarizing, this means that the developer as well as the evaluator are obliged to use the standard
PAI species for their testing activities. In this way it can be ensured that a representative set of PAIs is
used to test the TOE. The documentation of the standard PAI species defines a minimum set of attack
types that every system for PAD shall be able to detect. The document should be maintained by the
certification body and developed along with the needs of the market and the further development of
systems in this field.
It is important to note that the testing approach described here is not sufficient to claim that the PAD
mechanism cannot be circumvented with any other PAIs which are not prepared for the TOE during the
functional testing. This aspect is a part of the vulnerability analysis (AVA_VAN) which is discussed in 7.
7.2 Metrics for PAD testing
7.2.1 General
ISO/IEC 30107 (all parts) classifies presentation types in terms of the intention of the presenter, i.e. bona
fide presentations and attackto:presentations.
Licensed Plessis, Patrice However,
M. PAD systems are normally unable to determine
ISO/IEC DIS 19989-3:2019(E)

a presenter’s intent and PAD techniques are based on the measurement of physical and/or behavioural
attributes associated with a presentation plus a decision scheme that classifies the presentation as
either a bona fide presentation or an attack presentation. The PAD decision is not wholly deterministic
and decision errors may occur in operational biometric systems where attack presentations are
misclassified as bona fide or bona fide presentations are misclassified as attacks.
Recognising this, ISO/IEC 30107-3 specifies a set of PAD metrics including error metrics that are defined
for the bona fide and attack presentation types.
The metrics specified in ISO/IEC 30107-3 shall be used in ADV documentation, functional testing for
PAD, and its documentation. The appropriate metrics depend on the functionality provided by the TOE.
The error rates measured with the metrics shall be independently reported for each PAI tested.
ISO/IEC 30107-3 provides a number of metrics which can be used for testing the performance of PAD
systems. Subclause 6.2 specifies the metrics of ISO/IEC 30107-3 that shall be used for PAD testing. The
appropriate metrics depend on the functionality provided by the TOE.
7.2.2 Metrics used for PAD subsystem TOEs
PAD testing shall include the metrics APCER, BPCER, APNRR, and BPNRR, which are mandatory. In
addition, PAD subsystem processing duration (PS-PD) from the PAD subsystem may be measured and
reported as mean duration. Table 1 summarizes the relation among error rate, presentation type, and
attack classification. Note that the two metrics APNRR and BPNRR shall be evaluated under the PAD
subsystem processing duration.
BPCER could be relevant to the performance and usability of a system because occurrences could cause
usability problems and delays for affected users. Though ISO/IEC 15408 security evaluation primarily
focusses on security rather than usability/performance issues, testing of the BPCER is necessary to
determine whether the TOE is adequate for its purpose. As APCER and BPCER are dependent metrics
and are usually tuned using specific parameters, the developer could easily decrease the APCER while
increasing the BPCER. BPCER test conditions shall be the same as those for APCER.
The number of test crew to calculate BPCER is in general almost the same as the number of PAIs used
to calculate APCER. The performance testing activity in ISO/IEC 19989-2 usually uses test crew of the
order of 1 000. There is a big difference in number. If both biometric recognition performance and PAD
are evaluated for the TOE, BPCER should be calculated in biometric recognition performance testing
and written as supplementary information in the document for the ATE_FUN activity.
NOTE APCER for a given PAI species PAIS is defined and denoted as APCER PAIS in ISO/IEC 30107-3.
Table 1 — Relation among error rates, presentation type, and attack classification for PAD
subsystem
PAD Result
Presentation Type (Output)
(Input)
Attack Bona-fide No–response
Attack --- APCER APNRR
Bona Fide BPCER --- BPNRR
The symbol "---" means out of consideration in the above table.
7.2.3 Metrics used for data capture subsystem TOEs
The used error metrics are APCER, BPCER, APNRR, BPNRR, APNCR, APAR, FTER, and FTAR which are
mandatory. In addition, data capture subsystem processing duration, may be optionally used. Table 2
summarizes the relation among error rate, presentation type, and attack classification. Note that the

ISO/IEC DIS 19989-3:2019(E)

above metrics should be evaluated within the time interval of the data capture subsystem processing
duration.
NOTE A data capture subsystem, consisting of capture hardware or/and software, couples PAD mechanisms
and quality checks which could be opaque to the evaluator. Therefore the evaluator may not always know whether
a failure results from a detection of a presentation attack or a poor quality of the biometric characteristics.
Acquisition may be for the purpose of enrolment or recognition, but no comparison takes place in the data
capture subsystem.
Table 2 — Relation among error rates, presentation type, and attack classification for Data
Capture subsystem
Presentation PAD Result

Type (Output)
(Input) Attack Bona-fide No–response Capture Failure Capture Success
Attack --- APCER APNRR --- APAR
Enrolment FTER ---
Bona Fide BPCER --- BPNRR Verification/
FTAR ---
Identification
7.2.4 Metrics used for other TOEs
Other TOEs correspond to the third case in 5.3.2 of ISO/IEC 19989-1. Such a TOE contains at least the
comparison and decision subsystems for biometric verification or identification. This category of TOEs
includes a full system. PAD evaluation to a TOE of this category, even if the TOE itself is not a full system,
shall be done for a full system complementing other components to the TOE, if any, where the other
components shall be specified in the ST.
When the TOE is for biometric verification, the metrics are FNMR, FMR, IAPMR for biometric impostors,
and CAPNMR for biometric concealers. When the TOE is for positive identification, the metrics are FPIR
and IAPIR. When the TOE is for negative identification, the metrics are FNIR and CAPNIR. All these
metrics are mandatory (see ISO/IEC 19795-1 for FNMR, FMR, FNIR, and FPIR). In addition, full system
processing duration is optionally used. The mandatory metrics should be evaluated within the time
interval of the full system processing duration. Table 3 and Table 4 summarize the relation among error
rate, presentation type, and attack classification.
If Concealer produces a match against a different subject, it shall be considered as match by Impostor.
NOTE 1 If the TOE outputs the result of PAD in addition to the decision in some way, APCER and BPCER may
be used.
Table 3 — Relation among error rates, presentation type, and attack classification for full
system of biometric verification
Decision
(Input)
Match No–match
PAI (natural biometric Impostor IAPMR ---
characteristic/artefact) Attack presentation Concealer --- CAPNMR
Impostor FMR ---
Natural biometric charac-
teristic Bona fide pres-
Genuine --- FNMR
entation

NOTE 2 The last column expresses decision and relevant error rates.
ISO/IEC DIS 19989-3:2019(E)

Table 4 — Relation among error rates, presentation type, and attack classification for full
system of biometric identification
Decision
System
(Input)
Candidate Not candidate
Attack IAPIR ---
Positive identification
Bona Fide FPIR ---
Attack --- CAPNIR
Negative identification
Bona Fide --- FNIR
7.3 Minimum test sizes and maximum error rates

The test size requirements for PAD testing depend on the magnitude of the error rates to be measured
and the acceptable error bounds for the test results. As error rates and acceptable error bounds reduce,
statistical considerations require that the test size increases.
For PAD testing various PAIs representing PAI species and attack types are subject to testing and the
results shall be reported for each PAI species and attack type. The test size shall be assessed for each
PAI species and attack type.
It is the purpose of functional testing to assess that the PAD is working properly on the TOE, but not to
validate that there is no vulnerability on the TOE. The number of tests performed may be reduced to a
minimum at functional testing.
Thus, functional testing can be restricted to a subset of standard PAIs if any or non-standard PAIs
created by the developer if standard PAIs are not provided. The evaluator should investigate other
possible PAIs during AVA. The minimum number of PAI species, minimum test sizes, and maximum
error rates for functional testing may be defined either in the standard PAI species by the certification
body or in in the evaluation guidance of the considered Protection Profile. If any recommendation from
the certification body is given, the evaluator should consider them to determine the adequate test size
per PAI species. If they are not defined, the evaluator shall use at least 10 different samples for each PAI
species considered. They shall be agreed between the developer and the evaluator. The minimal test
size for each PAI species should be 10. Therefore, the maximum error rate (APCER) for each PAI species
should not be greater than 0,1
Evaluator should also consider recommendations from other international standards or non-
international standards for specific use cases. For instance, in the case of mobile biometric system, the
evaluator should use as a baseline the recommendations given in ISO/IEC 30107-4 for the number of
different PAI species and test size
8 Supplementary activities to ISO/IEC 18045 on Vulnerability Assessment (AVA)
8.1 Penetration testing using PAI variations

In contrast to the functional testing, the evaluator should consider specific aspects of the TOE in the
context of penetration testing. The evaluator shall use information obtained from the evaluation of
other assurance classes such as ADV to find potential weaknesses in design or implementation of the
TOE. With the results of the analyses, the evaluator shall identify attack types that are potentially able
to circumvent the PAD mechanism of the TOE. The evaluator shall look for materials, material mixtures,
and techniques that could be used to create PAIs that might defeat the PAD mechanisms of the TOE.
The evaluator shall try candidate PAIs that are relevant to the specified level of attack potential for
the TOE looking for presentation misclassifications. If a misclassification occurs the evaluator shall
record all relevant details of the PAI and the number of previous correct classifications for the PAI
prior to the misclassificationLicensed
occurring. Trials using the same PAI shall continue looking until further
to: Plessis, Patrice M.
ISO/IEC DIS 19989-3:2019(E)

misclassifications are found or the evaluator is satisfied that misclassification occurrences are not
readily repeatable. No rigid rules can be given on how much time should be spent on a typical evaluation
by a competent evaluator, however as a guidance, the time spent on this activity for AVA_VAN.1 should
be around 1 week, while it should be around 2 months for AVA_VAN.5.
A further source of information on relevant attack types that are created for penetration testing is the
functional testing. Functional testing could reveal that particular PAIs or attack types result in PAD
error rates that are higher than normal for the TOE. These PAIs / attack types may be candidates
for further exploration as part of penetration testing. Even if functional testing does not reveal that
particular PAIs or attack types result in PAD error rates that are higher than normal for the TOE, there
may be different/variant PAIs/attack types that could circumvent the PAD mechanism of the TOE in the
penetration testing stage of the evaluation.
Penetration testing in vulnerability analysis has a creative aspect. The effectiveness of a presentation
attack will depend on the preparation of the PAIs and their presentation. In the case of fingerprint PAIs,
the material used will affect the success of an attack as will presentation details such as the temperature
and thickness of the PAI and the lubrication of its surface with water or oil. For modalities such as face,
iris, and voice, other influencing factors could include sample rate, video resolution, frame rate, colour
space, and physical size used in making PAIs. The evaluators should build and maintain the requisite
level of preparation and presentation skills through a combination of the monitoring of public domain
and other information on presentation attacks against biometric systems and practical training and
experiment.
The evaluator shall determine the attack approach investigating attack methods relevant to the TOE
and any potential vulnerabilities that have been identified for the TOE, andacquiring or preparing
suitable PAIs prior to commencing the penetration testing. During penetration testing, the evaluator
shall present all the prepared PAIs to the TOE multiple times using suitable variations of presentation.
If no PAI is found that is able to circumvent the PAD subsystem, the evaluator may conclude that the
TOE is resistant to these PAIs.
If a TOE fails to detect a PAI presented during vulnerability assessment then the TOE is shown to
be vulnerable to the PAI/presentation and by implication to other PAI/presentations of the attack
approach. The evaluator should seek to reproduce the vulnerability to be able to estimate the
difficulty of reproducing it. Practical constraints such as time and variability will probably mean that
reproducibility can only be estimated on a fairly coarse scale of difficulty (e.g. easy, moderately difficult,
very difficult). The level of difficulty of reproducing the vulnerability will be a factor in determining the
risk involved in using the TOE in an application scenario and in calculating the attack potential. The
APCER for a PAI species in the functional testing can be used to inform the penetration testing, for
example to highlight PAI species that the TOE might be vulnerable to for further investigation.
Note that some attack scenarios may not need to be covered by penetration testing if the required
attack potential of the attack scenario is higher than specified by the AVA_VAN component of the TOE.
The attack potential for for an attack using a PAI against the TOE PAD mechanism shall be performed
calculated according to the guidance in Annex D of ISO/IEC 19989-1. Examples are provided in Annex A.
Annex C.1 of ISO/IEC 19989-1 gives information on other potential TOE vulnerabilities and guidance on
penetration testing where applicable to a particular TOE.
8.2 Potential vulnerabilities

The vulnerabilities that the evaluator shall at least take into account are described in 5.1 of
ISO/IEC 19989-1. Additionally, the evaluator should consider the combination of those vulnerabilities
with other IT-related vulnerabilities.
8.3 Rating of vulnerabilities and TOE resistanc

The rating shall be done according to D.1.5 of ISO/IEC 19989-1, as well as the consideration of the attack
potential (see Annex D.1.2 in ISO/IEC 19989-1). Examples related to PAD vulnerabilities are provided in
Annex A. Licensed to: Plessis, Patrice M.
ISO/IEC DIS 19989-3:2019(E)

Annex A
(Informative)
Examples of calculations of attack potential
This Annex provides several examples that include various systems that could be evaluated (access
control device for a building, an office, etc., access control to a personal device, etc.) and the "classical"
attacks that could be applied. Refer to Annex D.1.2 in ISO/IEC 19989-1 and Annex B.4 of ISO/IEC 18045.
A.1 Example 1 – Simple system without presentation attack detection

Two examples are here rated, 2D face recognition and fingerprint-based systems, unattended
and without any restriction to access the TOE. The difference in factors is only Access to biometric
characteristics.
It is considered that:
Elapsed time: 1 day is enough to define the method to build an attack type (identification) and to
generate a PAI targeting a dedicated person (exploitation). For 2D images, a simple print of a picture
could be enough, for fingerprints a moulding with easy to get material (glue, silicon, latex, and so forth)
is efficient.
Expertise: A lot of publications explain how to perform. No specific expertise is required (layman value
is enough).
Knowledge of the TOE: No specific knowledge of the TOE is required.
Window of opportunity (Access to the TOE): It does not make any problem to access to the TOE both in
identification (easy to buy without control) or in exploitation (a fingerprint PAI is easy to present: for
example by "gluing" the PAI to the real finger, for 2D face, a picture is presented to the camera).
Window of opportunity (Access to biometric characteristics): The level is Immediate for 2D face and
Easy for fingerprint.
Equipment: There is no speciffic requirement on equipment.
Table A.1 — Calculation of attack potential for example 1 (2D face)

Window of opportunity Total
Knowledge of Access to biom- 0
Elapsed Time Expertise Equipment
TOE Access to TOE etric character-
istics
Id Exp Id Exp Id Exp Id Exp Id Exp Id Exp Id Exp
0 0 0 0 0 - 0 0 - 0 0 0 0 0
Table A.2 — Calculation of attack potential for example 1 (Fingerprint)

istics
0 0 0 0 0 - 0 0 - 2 0 0 0 2
ISO/IEC DIS 19989-3:2019(E)

The attack potential for the attack is Basic.

The system fails any level of evaluation assuming that the described attack can be performed
successfully.
A.2 Example 2 – Fingerprints with presentation attack detection

The system is typically an access control system in an open environment. Consecutive presentations
are possible but "strange" behaviour of the user would be detected.
The system includes PAD implying to find the right material for making the attack type (glycerine,
gelatin for example) and the application to a real finger is not immediate (thin film, leaving part of the
skin in contact with the capture device, a print with specific ink directly on a real finger, etc.).
Elapsed time: Finding the right material for the attack type and how to present it to the system is not
evident and will require consecutive presentations. 2 weeks for identification is realistic for an example.
Once defined, producing the PAI for a dedicated person and applying with the predefined method to the
real TOE is immediate (1 day for exploitation).
Expertise: A lot of publications explain how to perform. However, the attacker will have to understand
(and even to find) what is the principle of the PAD, and to derive a specific strategy both for creating the
PAI and to apply it. A proficient level for identification is realistic, for exploitation a layman is enough
(following a script).
Knowledge of the TOE: It is assumed that no specific knowledge is required. The existence of PAD is
probably advertised (either by the developer or the user). With some time, the attacker (proficient level, so
knowing what is offered by industrial systems), will probably find the method this detection is based on.
Window of opportunity (Access to the TOE): Being a security system, it is assumed that it is not possible
to simply buy the system without any control, but that its distribution is controlled (for example by
requiring an identification of the buyer and potentially to sign a non-disclosure agreement). Moderate
level is adapted for the identification phase and for the exploitation phase (detection of a strange
behaviour).
Window of opportunity (Access to biometric characteristics): The level is Easy.
Equipment: There is no specific requirement on equipment.
Table A.3 — Calculation of attack potential for example 2

istics
2 0 0 2 0 - 2 4 - 2 0 0 4 8
The attack potential for the attack is Enhanced-Basic.

If the attack can be performed successfully and no other successful attack with a lower attack potential
is found, the resistance of the TOE is Basic.

ISO/IEC DIS 19989-3:2019(E)

The system is compatible with the AVA VAN.2 component.

NOTE The attack potential of the attack is close to the limit between Basic and Enhanced-Basic. This means
that any reduction, for example less time in identification, having no control in the distribution of the system
(moving from Moderate to Easy in Window of opportunity (Access of the TOE), being in a fully unsupervised
environment, could reduce the attack potential of the attack to Basic and then reduce the resistance of the system
to "No rating".
A.3 Example 3 – Fingerprints with Advanced presentation attack detection

The system is typically an access control system in an open environment. Several tries are possible but
"strange" behaviour of the user would be detected.
The system includes an advanced PAD subsystem. Advanced means that without a detailed knowledge,
it is impossible to find in a reasonable time the method to make undetected attack types (multi detection
methods, needing a specific presentation strategy). In addition, no details can be found on the public
domain, nor explained to buyers. It is also considered that the system is only sold to well identified
users, under non-disclosure agreement.
Elapsed time: Finding the right material for the attack type and how to present it to the system is not
evident and will require multiple tries. 1 month for identification is realistic. Once defined, producing
the PAI for a dedicated person and applying with the predefined method to the real TOE is immediate (1
day for exploitation).
Expertise: PAD strategy is not described in the public domain, detailed measures are kept confidential
and a specific strategy to make the PAI and present it has certainly to be invented. An expert level
for identification is realistic, for exploitation a proficient is enough (following a script, but applying a
complex strategy requiring good knowledge and understanding of the PAD mechanism).
Knowledge of the TOE: Without detailed knowledge of the PAD mechanisms, it is assumed that
it is impossible in a reasonable time to create an accepted PAI. This information is protected and a
compromising is necessary. So, a Sensitive knowledge is required.
Window of opportunity (Access to the TOE): Being a security system, it is assumed that it is not
possible to simply buy the system without any control, but that its distribution is controlled (for
example by requiring an identification of the buyer and potentially to sign a non-disclosure agreement).
It is assumed that the protection is more efficient than in the previous example, so a Difficult level is
probably adapted for the identification phase. For exploitation, Access of the TOE is rated to Moderate.
Window of opportunity (Access to biometric characteristics): The level is Easy.
Equipment: There is no specific requirement on equipment.

istics
4 0 4 4 4 - 4 4 - 2 0 0 16 10
The attack potential for the attack is Moderate.

is found, the resistance of the TOE is Enhanced-Basic.

ISO/IEC DIS 19989-3:2019(E)


NOTE A significant part of the calculating attack potential is due to non-technical measures (Knowledge of
the TOE, Winodw of opportunity (Access to the TOE)). This means that if such measures are not implemented,
the resistance of the TOE will have to be moved down to Basic.
A.4 Example 4 – 3D Face with presentation attack detection and try counter
This example is for a 3D face based system with (relatively simple) PAD, but for which the detection is
hard to circumvent with a chance greater than 10% and a try counter which raises an alert (and triggers
some subsequent corrective actions) when more than 3 presentation attacks have been detected in a
row of attempts without any successful acceptance. The system is typically an access control system
in an open environment. The system includes PAD implying to find the right material for making the
attack type.
Elapsed time: Finding the right material for the attack type and how to present it to the system is
not evident and will require multiple tries. 1 month for identification is realistic for an example. Once
defined, producing the PAI for a dedicated person and applying with the predefined method to the real
TOE is immediate (1 day for exploitation).
Expertise: Some publications may explain how to perform. However, the attacker will have to
understand (and even to find) what is the principle of the PAD, and to derive a specific strategy both for
creating the PAI and to apply it. A proficient level for identification is at least needed, for exploitation a
layman is enough (following a script).
Knowledge of the TOE: It is assumed that a restricted knowledge of the TOE is required to understand
what is used for PAD in case of 3D system.
Window of opportunity (Access to the TOE): Being a security system, it is assumed that it is not possible
to simply buy the system without any control, but that its distribution is controlled (for example by
requiring an identification of the buyer and potentially to sign a non-disclosure agreement). Moderate
level is probably adapted for the identification phase. For exploitation, Access to the TOE is rated as
difficult due to the try counter.
Window of opportunity (Access to biometric characteristics): The level is Moderate.
Equipment: There may be specific equipment needed for producing 3D PAI during identification and
exploitation.

istics
4 0 2 0 2 - 2 8 - 4 2 4 12 16
The attack potential for the attack is Moderate.

is found, the resistance of the TOE is Enhanced-Basic.

ISO/IEC DIS 19989-3:2019(E)

A.5 Example 5 – Wolf Attack

This example is for a biometric system for a given modality, operating in a fully uncontrolled
environment (for example, protecting the access to a device or an equipment). Assume that it is a purely
software-based system, for which it is thus straightforward to connect a computer just before the
feature extractor. Assume also that there is a flaw in the comparison algorithm such that there is a way
to construct an image (not necessarily representative of a biometric capture) for which the acceptance
rate is significantly higher than any randomly chosen biometric data, whatever the enrolled data are.
This corresponds to a system with a high rate of successful attack. The attack will consist in finding the
particular (or one of) image(s) that leads to a high chance of being accepted.
The identification phase corresponds to find the weakness in the matching algorithm and therefore to
generate an image which exploits this flaw. The exploitation phase is just inputting the image to get access.
Note There are few who have rare special biometric characteristics with the above features. Such
special biometric characteristics are called human wolves.
Elapsed time: More than one month for identification might be needed while exploitation is immediate.
Expertise: An expert attacker should be needed to find the flaw in the algorithm. A layman will be able
to input the image once generated.
Knowledge of the TOE: Sensitive knowledge of the TOE is required to learn the detail of the matching
algorithm.
Window of opportunity (Access to the TOE): The TOE operates in a fully uncontrolled environment.
Window of opportunity (Access to biometric characteristics): It is rated as Immediate as no access is
needed for the attack.
Equipment: A specific equipment may be required to generate an image, acceptable for the feature
extractor.

istics
8 0 4 0 4 - 0 0 - 0 2 0 18 0
The attack potential for the attack is: Enhanced-Basic

is found, the resistance of the TOE is Basic.

ISO/IEC DIS 19989-3:2019(E)

Bibliography
[1] ISO/IEC 2382-37:2017, Information technology — Vocabulary — Part 37: Biometrics

[2] ISO/IEC 15408-1:2009, Information technology — Security techniques — Evaluation criteria for IT
security — Part 1: Introduction and general model
[3] ISO/IEC 19792, Information technology — Security techniques — Security evaluation of biometrics
[4] ISO/IEC 30107-1, Information technology — Biometric presentation attack detection — Part 1:
Framework
[5] Bundesamt für Sicherheit in der Informationstechnik, Fingerprint Spoof Detection Protection
Profile based on Organisational Security Policies FSDPP_OSP v1.7, November 2009.
[6] Ellingsgaard J., Sousedik C., Busch C., Detecting Fingerprint Alterations by Orientation
Field and Minutiae Orientation Analysis, in Proceedings of the 2nd International Workshop on
Biometrics and Forensics 2014 (IWBF 2014), 27-28th March 2014, Valletta, Malta, (2014).
[7] Gomez-Barrero M., Galbally J., Morales A., Ferrer M., Fierrez J., Ortega-Garcia J., A
novel hand reconstruction approach and its application to vulnerability assessment. Information
Sciences, 268:103–121, 2014.
[8] Gottschlich C., Mikaelyan A., Olsen M., Bigun J., Busch C., Improving Fingerprint Alteration
Detection, in Proceedings 9th International Symposium on Image and Signal Processing and
Analysis (ISPA 2015), 7-9 September, Zagreb, Croatia, (2015).
[9] Haraksim R., Anthonioz A., Champod C., Olsen M., Ellingsgaard J., Busch. C. Altered
fingerprint detection - Algorithm performance evaluation, in Proceedings of the 4th
International Workshop on Biometrics and Forensics 2016 (IWBF 2016), 3-4th March 2016,
Limassol, Cyprus, (2016).
[10] Martinez-Diaz M., Fierrez J., Galbally J., Ortega-Garcia J., An evaluation of indirect
attacks and countermeasures in fingerprint verification systems. Pattern Recognition Letters,
32(12):1643–1651, 2011.
[11] Tekampe N., Merle A., Bringer J., Gomez-Barrero M., Fierrez J., Galbally J., D6.5: Towards
the Common Criteria evaluations of biometric systems, March 2016.
[12] Yoon S., Feng J., Jain A. K., Altered Fingerprints: Analysis and Detection, IEEE Trans. Pattern
Anal. Mach. Intell., vol. 34, no. 3, pp. 451–464, (2012)


DIS - 19989-3 - Criteria and Methodology For Security Evaluation of Biometric Systems

Încărcat de

Informații document

Titlu original

Drepturi de autor

Formate disponibile

Partajați acest document

Partajați sau inserați document

Opțiuni de partajare

Vi se pare util acest document?

Este necorespunzător acest conținut?

Drepturi de autor:

Formate disponibile

DIS - 19989-3 - Criteria and Methodology For Security Evaluation of Biometric Systems

Încărcat de

Drepturi de autor:

Formate disponibile

DRAFT INTERNATIONAL STANDARD

ISO/IEC DIS 19989-3

ISO/IEC JTC 1/SC 27 Secretariat: DIN

Information security — Criteria and methodology for

THIS DOCUMENT IS A DRAFT CIRCULATED

COPYRIGHT PROTECTED DOCUMENT

Licensed to: Plessis, Patrice M.

Licensed to: Plessis, Patrice M.

Licensed to: Plessis, Patrice M.

Information security — Criteria and methodology for

3 Terms and definitions

Licensed to: Plessis, Patrice M.

[SOURCE: ISO/IEC 30107-3: 2017, 3.2.3]

[SOURCE: ISO/IEC 2382-37:2017, 3.1.1]

Licensed to: Plessis, Patrice M.

[SOURCE: ISO/IEC 2382-37:2017, 3.6.3]

[SOURCE: ISO/IEC 2382-37:2017, 3.4.1]

[SOURCE: ISO/IEC 2382-37:2017, 3.1.2]

Licensed to: Plessis, Patrice M.

[SOURCE: ISO/IEC 2382-37:2017, 3.5.3]

[SOURCE: ISO/IEC 2382-37:2017, 3.8.2]

[SOURCE: ISO/IEC 2382-37:2017, 3.7.13]

Note 6 to entry: Use of ‘authentication’ as a synonym for “biometric verification or biometric identification” is

[SOURCE: ISO/IEC 2382-37:2017, 3.1.3]

Licensed to: Plessis, Patrice M.

[SOURCE: ISO/IEC 2382-37:2017, 3.2.3]

[SOURCE: ISO/IEC 2382-37:2017, 3.8.3]

[SOURCE: ISO/IEC 30107-3: 2017, 3.1.2]

Licensed to: Plessis, Patrice M.

[SOURCE: ISO/IEC 30107-3: 2017, 3.2.9]

[SOURCE: ISO/IEC 15408-1: 2009, 3.1.22]

Licensed to: Plessis, Patrice M.

[SOURCE: ISO/IEC 15408-1: 2009, 3. 4.16]

[SOURCE: ISO/IEC 15408-1: 2009, 3.1.25]

[SOURCE: ISO/IEC 18045: 2008, 3.7]

[SOURCE: ISO/IEC 2382-37:2017, 3.9.4]

[SOURCE: ISO/IEC 2382-37:2017, 3.9.7]

Note 2 to entry: Comparisons between:

[SOURCE: ISO/IEC 2382-37:2017, 3.9.9]

[SOURCE: ISO/IEC 2382-37:2017, 3.9.11]

[SOURCE: ISO/IEC 30107-3: 2017, 3.2.8]

[SOURCE: ISO/IEC 2382-37:2017, 3.3.31]

[SOURCE: ISO/IEC 30107-3: 2017, 3.1.6]

Licensed to: Plessis, Patrice M.

[SOURCE: ISO/IEC 30107-1: 2016, 3.5]

[SOURCE: ISO/IEC 30107-1: 2016, 3.6]

[SOURCE: ISO/IEC 30107-1: 2016, 3.7]

Licensed to: Plessis, Patrice M.

4 Symbols (and abbreviated terms)

Licensed to: Plessis, Patrice M.

6 Overview of PAD testing in Class ATE and Class AVA

6.1 Objectives and principles

6.1.1 Class ATE

6.1.2 Class AVA

6.2 PAIs used in testing activities

6.2.1 Class ATE

Licensed to: Plessis, Patrice M.

6.2.2 Class AVA

6.3 Testing activities

6.3.1 Class ATE

6.3.2 Class AVA

6.4 Criteria of pass/failure

7 Supplementary activities to ISO/IEC 18045 on Tests (ATE)