See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/225744225

Validation of Innovative Food Microbiological Methods According to the EN

ISO 16140 Standard

Article  in  Food Analytical Methods · June 2011

DOI: 10.1007/s12161-010-9154-4

CITATIONS READS

8 4,177

2 authors:

Bertrand Lombard Alexandre Leclercq

Agence Nationale de Sécurité Sanitaire de l'Alimentation, de l'Environnement et d… Institut Pasteur
53 PUBLICATIONS   930 CITATIONS    119 PUBLICATIONS   2,122 CITATIONS   

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Surveillance of Listeria View project

Taxonomy of Listeria genus View project

All content following this page was uploaded by Bertrand Lombard on 16 May 2014.

The user has requested enhancement of the downloaded file.

Food Anal. Methods
DOI 10.1007/s12161-010-9154-4
Validation of Innovative Food Microbiological Methods
According to the EN ISO 16140 Standard
Bertrand Lombard & Alexandre Leclercq
Received: 7 April 2010 / Accepted: 27 May 2010
# Springer Science+Business Media, LLC 2010
Abstract Reference methods in food microbiology are
mostly based on conventional microbiology. Innovative
methods have been developed and commercialised, with
several advantages, but their users need guarantees on their
performance. Validation schemes have thus been estab-
lished to assess whether these methods perform at least as
well as the corresponding reference methods. In addition, a
European and International Standard, EN ISO 16140, has
been developed to provide a common reference protocol for
the validation of alternative methods, as well as to
determine general principles for their possible subsequent
certification. The content of this standard is summarised, a
technical protocol is defined separately for qualitative and
quantitative methods and the validation study, which
comprises two phases, a comparative study and an inter-
laboratory study, is described. This standard is currently
being revised and the main directions taken are presented.
This paper then briefly introduces the validation/certifica-
tion systems before addressing several aspects related to the
links with laboratory accreditation and European regula-
tions on microbiological criteria, as well as the use of
alternative methods.
B. Lombard (*)
AFSSA-LERQAP (French Food Safety Agency—Laboratory
for Study and Research on Food Quality and Food Processing),
23 avenue du Général de Gaulle,
94706 Maisons-Alfort, France
e-mail: b.lombard@afssa.fr
A. Leclercq
WHO Collaborative Center for Foodborne Listeriosis,
National Reference Center for Listeria, Institut Pasteur,
28 rue du Docteur Roux,
75724 Paris cedex 15, France
e-mail: alexlec@pasteur.fr
Keywords Food Microbiology . Method Validation .
Innovative Methods . Alternative Methods
Introduction
Reference methods in food microbiology, for the analysis of
food, animal feeding stuffs, samples from primary produc-
tion and food processing and handling, are standardised at
the international level by International Organization for
Standardization (ISO) and at the European level by
European Committee for Standardization (CEN) (Lombard
et al. 1996; Lombard 2004; Leclercq 2002; Leclercq et al.
1999; Lombard and Leclercq 2010). The standardisation
process consists of establishing reference documents in
agreement with all parties and countries concerned. These
reference methods are mostly based on conventional
microbiology, enabling the major characteristic of a
standard to be met: the largest consensus and the broadest
applicability in food microbiology worldwide, including
developed countries themselves. Application of reference
methods requires skilled staff and is labour-intensive and
time-consuming, due to the time needed by microorganisms
to grow in or on culture media.
Innovative methods in food microbiology, so called
‘rapid methods’, have been developed to achieve results
within a shorter time and sometimes with higher perfor-
mance than ‘classical’ methods based on conventional
microbiology. They can be classified into three main types:
(1) solid culture media other than those prescribed in the
standard reference methods, mostly based on chromogenic
substrates so as to enable easier visual recognition of the
target bacteria on the plates, (2) immunoenzymatic kits,
such as ELISA, and (3) molecular-based methods, such as
hybridisation probes or PCR, either conventional or real-

time. They are often based on proprietary components (e.g.
the chromogenic substrates, the ELISA antibodies, the
DNA probes or DNA primers). Their main advantage is
to achieve quicker, more specific and sensitive results than
with the conventional methods. They are also designed to
be more convenient to use, requiring less labour, in
particular when they are automated. Automation also
enables large sets of samples to be analysed at the same
time. They are also better suited to hazard control in the
food chain (HACCP systems), to the control of food
products with short shelf life, to the demand of legal
experts or insurance companies in the event of litigation
and finally to the management of food crises (Leclercq et
al. 1999).
Some new reference methods under development, such
as detection of verotoxigenic Escherichia coli, or some
recently revised CEN/ISO reference methods, such as on
the detection and enumeration of Listeria monocytogenes
(EN ISO 11290-1 and 2), use innovative products (here,
respectively, real-time PCR and a chromogenic medium)
when conventional microbiology alone is unable to provide
satisfactory solution for analysing the given target: conven-
tional enrichment/isolation procedures perform poorly or do
not allow identification of the targeted microorganisms. An
essential requirement for the standardisation of these
products is that they should be ‘open’ in their description
(composition of culture media, sequence of the DNA
probes and primers, etc.), so as to avoid giving exclusivity
in the standard to one manufacturer. But the main drawback
in standardising innovative methods is the slow stand-
ardisation process, especially at the European and interna-
tional levels, which takes at least 3 years, a pace which is
not easily compatible with the development of new
technologies.
Given this situation, systems for the validation of
innovative methods against reference methods were estab-
lished in the 1990s, to enable quicker recognition of these
alternative methods and at the same time to protect the
know-how of the test kit manufacturers (the method can be
validated as a ‘black box’) (Lombard et al. 1996). In
addition, these validation schemes, operated by third-party
organisations independent from the manufacturers, meet the
needs of the users of these methods (food industry, private
and public laboratories, public authorities, accreditation
bodies) who require guarantees on their performance. Such
innovative methods are proprietary in the sense that they
are given a trade name, sold by their manufacturer or
distributors and are usually protected by licences. They are
often called ‘alternative methods’. An alternative method
can be defined as a method which analyses, for a given
scope, the same target analyte as that measured by the
corresponding reference method and which meets the
following requirements: rapid analysis and/or response,
Food Anal. Methods
easy to carry out and/or automation, analytical perform-
ances at least as good as those of the reference method. In
other words, an alternative method is designed and intended
to be used instead of the corresponding reference method.
The term ‘alternative method’ refers to the combination of
product, equipment and complete operating procedure,
from sample preparation to final result. It includes all the
elements necessary for implementing the alternative meth-
od: equipment, culture media, software for analysis of
results, etc.
In general, validation of a method consists of confirm-
ing, by examination and provision of objective evidence,
that the particular requirements for a specific intended use
are fulfilled (Leclercq 2002). For a specific intended use, a
laboratory has to reply to the question: why am I analysing
the samples and for what purpose are the resulting data to
be used? For objective evidence, laboratories need to have
or refer to validation studies. For confirmation, laboratories
have to compare information obtained from validation
studies with customer's requirements.
In order to have a common reference protocol to conduct
the validation of these innovative methods, CEN has
developed a standard on the validation of alternative
methods in food microbiology, EN ISO 16140 (Anonymous
2003), a standard adopted in parallel by ISO. According to
the standard on general requirements and guidance for
microbiological examinations (EN ISO 7218 (Anonymous
2007) Clause 14.2), the validation of alternative methods
shall be performed according to EN ISO 16140.
This paper will mainly present the protocol for the
validation of alternative methods according to the EN ISO
16140 Standard, then will give the revision status of this
standard and introduce validation/certification systems, as
well as their relation with laboratory accreditation and
regulatory aspects, and the use of validated methods.
Protocol for the Validation of Alternative Methods
According to the EN ISO 16140 Standard
Origin
The EN ISO 16140 Standard originated from the ‘Micro-
Val’ project of the European Eureka R&D programme
(Rentenaar 1996). This project, which ran from 1993 to
1998, aimed at setting up a European protocol for the
validation of alternative (proprietary) methods in food
microbiology. It included 20 partners: test kit manufac-
turers, food manufacturers, food testing laboratories and
standardisation/certification bodies (AFNOR and NEN,
respectively, the French and Dutch standardisation bodies,
the latter having managed the project). Based on an
experimental phase, it developed two reference documents,
Food Anal. Methods
the first one defining general rules and a certification
scheme, the second one technical rules.
Standardisation Process
Based on the MicroVal technical protocol, CEN/TC 275/
WG 61, the group in charge of European standardisation in
food microbiology, decided to develop a standard on the
validation of alternative methods in food microbiology. For
this purpose, in 1996, it established a group, Task Group 2
‘Validation of alternative methods’ of WG 6, chaired by
Roy HOLBROOK (UK) and comprising 25 members from
MicroVal and WG 6. After a rather long process (7 years),
due to the complexity of the topic which was being
standardised for the first time in food microbiology, the
EN ISO 16140 Standard was published in 2003. It has been
adopted in parallel as an ISO Standard by ISO/TC 34/SC 92,
the group in charge of international standardisation in food
microbiology.
Presentation of EN ISO 16140
Scope
The standard's objectives are to define:
(a) The general principles and technical protocol for the
validation of alternative (mainly proprietary, innova-
tive) methods, with the idea of allowing the use of
alternative methods instead of the corresponding
reference methods, for the reasons given in the
introduction
(b) The general principles for third-party certification of
alternative methods
It applies to the microbiological analysis of any food
intended for human and animal consumption, as well as to
the analysis of samples from primary production (the
breeding environment) and from the food production and
handling environment.
Overview
As for any standard, a clause of definitions can be found,
including what can be regarded as the main one, ‘validation
of an alternative method’. According to the standard, this
consists in ‘demonstrating that results obtained with the
alternative method are comparable to the results which
would have been obtained with the reference method’. In
1
Working Group 6 “Microbiology” of Technical Committee 275
‘Food analysis—Horizontal methods’.
2
Sub-Committee 9 “Microbiology” of the ISO Technical Committee
34 ‘Food Products’.
this definition, the word ‘comparable’ can be understood as
‘at least equivalent’. A note then explains that the word
‘comparable’ is defined later in the standard in a technical
protocol adapted to each type of method (qualitative or
quantitative).
In the first part of the standard, the principles of the
validation protocol are established. It comprises two
phases: (1) a comparative study of the alternative method
(AM) with the corresponding reference method (RM),
conducted by one laboratory, and (2) an inter-laboratory
study on both methods in parallel, organised by the
laboratory in charge of the first phase. The standard defines
technical rules for these two phases, according to each type
of method, either qualitative or quantitative.
If certification of methods is sought in addition to their
technical validation, general principles for this certification are
given. We may recall at first a definition of certification,
derived from ISO/IEC 17000 (Anonymous 2004) and adapted
to our context: it is a process through which a recognised
body acting independently, the certification body, with no ties
to the parties involved, gives written assurance that a product
(here test kits) meets the baseline requirements set out in a
reference standard (here the Standard EN ISO 16140). The
certification principles of proprietary methods set out in EN
ISO 16140 consist of (1) requirements for the quality
management system of the test kit's production line, based
on EN ISO 9001 (Anonymous 2008), and (2) regular audits
of the test kit manufacturer's quality system and of the test
kit's production control requirements. Details on how certifi-
cation is organised (management of the two phases of the
validation protocol, all the different partners involved includ-
ing the expert laboratory, the reviewers) are provided by the
certification bodies in their certification rules (see for example
the certification rules of AFNOR Validation (http://www.
afnor-validation.com/afnor-validation-food-industry/food-
industry.html; MicroVal, http://www.microval.org/rules.html).
The standard then details the validation technical rules,
which consist of the selection of a set of analytical
performance criteria. For each criterion, the standard gives
a definition, an experimental design (type/number of
samples to be tested), calculations and an interpretation.
If the AM has already been validated prior to the
implementation of the standard, previous results may be
taken into account if they have been obtained according to
the same experimental design. Annex A of the standard
gives specific rules for the acceptance of such external
results, dealing with two scenarios:
(a) The method has already been validated against a CEN/
ISO Standard RM. If the method has been modified
since its initial validation, the degree of change to the
method needs to be assessed, and if minor, the results
can be accepted.

(b) The method has already been validated against a RM
which is not a CEN/ISO Standard. The results can be
accepted if the study was performed by a recognised
validation system (see below) and if the differences
between the RM used and the corresponding CEN/ISO
Standard are judged by the validation/certification
body to be minor.
An overview of the technical protocol is now given.
Technical Protocol for Qualitative Methods
Method Comparison Study
The first phase, called the ‘methods comparison study’ in
the standard (Clause 5.1), enables three different sets of
criteria to be assessed, trueness, limit of detection and
inclusivity/exclusivity, so as to characterise the performance
of the AM in comparison to the RM.
The trueness (called ‘relative accuracy’ in the standard)
is defined as the degree of correspondence between the AM
and RM on identical samples. The experimental design
gives priority to naturally contaminated samples, from a
large variety of matrices likely to be contaminated by the
microorganism of interest. If not enough naturally contam-
inated samples are available, artificial contamination can be
performed, according to a protocol defined in Annex C of
the standard, with the purpose of mimicking natural
contamination. If a method is intended to be validated for
all foods, five food categories are tested (these categories
are defined in Annex B of the standard, and consist of the
main food families, such as milk and dairy products, meat
and meat products). A method can also be validated for
specific food categories (less than five). Samples from the
primary production and food production environments are
briefly treated as separate categories. For each category and
each method, 60 test portions are analysed, belonging to
three food types defined in Annex B (they generally
correspond to different processes, such as heat treatment,
fermentation, etc.). The results are presented in the usual
2×2 table for qualitative methods, except that the terms
‘false positives’ and ‘false negatives’ have been replaced by
‘positive deviations’ and ‘negative deviations’. It indicates
that the AM has been compared exactly to the RM, without
attempting to assess whether the results of the RM are true,
i.e. the positives (negatives) of the AM are true or false
positives (negatives), in the event of deviation from the
RM. Three criteria are calculated: (1) relative accuracy, i.e.
the number of pairs of identical results for both methods
divided by the total number of results, (2) relative
specificity, i.e. the number of pairs of negative results for
both methods divided by the number of negative results for
the RM and (3) relative sensitivity, i.e. the number of pairs
Food Anal. Methods
of positive results for both methods divided by the number
of positive results for the RM. Confidence intervals can be
calculated, as described in Annex E of the standard. The
degree of difference between the two methods is assessed
by a χ2 test (McNemar) or by comparison to a binomial
law, as detailed in Annex F of the standard. Only these tests
in the standard enable the equivalence of qualitative
methods to be assessed.
The limit of detection (called ‘relative detection level’ in
the standard) is defined as the smallest number of cultivable
microorganisms which can be detected on 50% of occa-
sions. It is determined using artificially contaminated
samples from one food matrix chosen from each category
tested (see above), preferably at five (minimum three)
contamination levels, including a negative control, using
one target microorganism per category and six replicates
per method. The calculation is quite basic: the limit of
detection is characterised by a range between the two
contamination levels giving, for the first one, less than 50%
of positive results (i.e. less than three positives out of six
replicates) and for the second one more than 50% of
positive results (i.e. more than three positives out of six
replicates). The ranges for the AM and RM are then
compared.
The inclusivity and exclusivity are combined criteria;
they assess the AM's ability to detect the target microor-
ganism and its lack of interference with a range of non-
target microorganisms. Pure strains are tested: at least 50
target strains (except for Salmonella: 30) and 30 non-target
strains. Criteria for selecting the strains are given in Annex
G of the standard. Inoculation methods are also described.
Inter-laboratory Study
The second phase, the inter-laboratory study (Clause 5.2 of
the standard) aims at assessing the variability of the AM in
an inter-laboratory context (‘transferability’ of the method).
The experimental design requires at least ten collabora-
tive laboratories having obtained results which can be taken
into account for precision and trueness estimation. The
samples are prepared with one matrix, contaminated at
three levels (including a negative control). Eight blind
replicates are prepared at each level, to be analysed by both
the AM and RM. Annex H of the standard provides detailed
guidance on the organisation of the inter-laboratory study.
Laboratory results are excluded only in the event of
observed technical problems: samples damaged, use of
inappropriate culture media, transport conditions (time,
temperature) not respected, acceptability rules for colony-
counting/enumeration values not respected and deviations
from SOP. These are detailed in Annex K of the standard.
The trueness criteria (accuracy, sensitivity and specific-
ity) are calculated for each contamination level, and their
Food Anal. Methods
values are compared to those obtained in the first phase.
Annex L of the standard gives further optional criteria
based on the works of Langton et al. (2002), which attempt
to better address the random variability of a qualitative
method by introducing parameters (accordance, concor-
dance and concordance odds ratio) equivalent to repeat-
ability and reproducibility for quantitative methods.
Technical Protocol for Quantitative Methods
Method Comparison Study
This first phase of the study (Clause 6.2 of the standard)
enables four sets of criteria to be assessed: linearity and
trueness, limit of detection/quantification, sensitivity and
inclusivity/exclusivity.
Linearity is defined as the ability of the AM, for a given
matrix, to yield results that are in proportion to the amount of
analyte present in the sample. For trueness, the term ‘relative
accuracy’ is also used with the same definition as for
qualitative methods (see above). The standard clarifies that,
for instrumental methods, which are commonly used as AMs
in quantitative microbiology (especially for total flora or
hygienic indicators), the calibration curve between the
response/signal of the instrumental method and the unit of
the RM (colony-forming units) is not part of the validation
process; it should be established in a preliminary stage.
The experimental design requires a minimum of five
contamination levels with priority given to natural contam-
ination. At least duplicates (preferably five to ten replicates)
are prepared. The rules defined above for qualitative
methods on the categories to be studied also apply here.
Once the results have been obtained, a regression
analysis is conducted. Two methods for linear regression
are suggested in Annex R of the standard: orthogonal and
least-squares linear regressions. The linearity of the AM
against the RM is then assessed (lack-of-fit test), as well as
its trueness (bias in the linear equation).
The limit of detection (LOD) is defined as the smallest
amount of analyte which can be detected but not quantified
with a high probability, i.e. 95%, whereas the limit of
quantification (LOQ) is the smallest amount of analyte
which can be measured and quantified with defined
precision and trueness.
As for analytical chemistry, the experimental design and
calculation for indirect instrumental methods refer to blank
sample analyses and their standard deviation (sO): LOD=
3.3sO and LOQ=10sO. For colony-count or most probable
number techniques, the standard suggests an approach for
LOD derived from the approach for qualitative methods,
but which is not detailed.
The sensitivity, i.e. the AM's ability to detect two
(slightly) different amounts of analyte measured by the
RM, refers for the experimental design and calculation to
linearity.
Inclusivity and exclusivity are defined as for qualitative
methods; the experimental design is similar except that the
minimum number of target strains to be tested is set at 30,
and 20 for non-target strains.
Inter-laboratory Study
The inter-laboratory study (Clause 6.3 of the standard)
addresses the inter-laboratory precision and trueness of the
AM against the RM, by estimating the well-known parameters
of repeatability (minimum random variability), reproducibility
(maximum random variability) and bias (systematic error).
The experimental design prescribes at least eight
laboratories having obtained results without outliers. The
samples are prepared from one matrix, contaminated at four
levels, including a negative control, and analysed in blind
duplicates by both the AM and RM.
As for the qualitative methods, results are excluded only
on the basis of reported technical deviations. The laboratory
results are at first log10 transformed, a common step in
quantitative microbiology in order to approach a Normal
distribution of the data and to stabilise the variance over
contamination levels (Lombard et al. 2005).
The precision characteristics are then calculated with
robust statistics, based, for the repeatability standard
deviation on the median of the duplicate standard devia-
tions, and for the reproducibility standard deviation on the
Rousseuw recursive median Sn (Rousseeuw and Croux
1993). Robust statistics are less sensitive to outlying data
than parametric statistics; they accommodate with these
extreme data and make it possible to avoid the use of
statistical tests to exclude outliers (Lombard et al. 2005).
For each contamination level, the bias (relative accuracy)
of the AM against the RM is estimated by the median of the
laboratories' differences between duplicate means for the
AM and the RM.
Revision of EN ISO 16140
In 2005, both CEN/TC 275/WG 6 and ISO/TC 34/SC 9
decided to launch the revision of EN ISO 16140, based on
its implementation in particular by AFNOR Certification.
The main topics for revision are : (1) to cover more
comprehensively the topic of method validation in food
microbiology (not only the case of alternative/proprietary
methods), (2) to improve the statistical approaches, and (3)
to include acceptability criteria for the analytical perfor-
mance characteristics, most of them only being calculated
in the current version. Acceptability criteria are a require-
ment of technical guidelines or rules of accreditation bodies

in the field of food microbiology, in the context of Clause
5.4.5 of the EN ISO 17025 Standard (Anonymous 2005a).
In addition, the scope of EN ISO 16140 was to be enlarged.
Indeed, since the publication of the existing version of this
standard, the respective scopes of ISO TC34/SC9 and CEN
TC275/WG6 have been enlarged to include the analyses of
food-borne viruses and parasites: the new version of EN
ISO 16140 should also cover them, and not be restricted to
bacteria, yeasts and moulds. In addition, details would be
needed on the category of primary production samples. The
validation of PCR methods is also expected to be explicitly
covered.
Given the CEN/ISO cooperation agreement and in order
to have an international standard on validation of AMs fully
recognised by all certification/validation bodies worldwide,
the leadership for the revision of EN ISO 16140 has been
transferred to ISO/TC 34/SC 9, which has set up a new
group, the WG 3 ‘Method Validation’, chaired by Paul In’t
Veld (Netherlands), who is also the current project leader
for the dormant TAG 2 at the CEN TC275 WG 6 level.
Statistical aspects related to the revision of the standard are
covered by another group of ISO/TC 34/SC 9, the WG 2
‘Statistics’, which is providing recommendations on these
aspects to WG 3.
The revised standard will comprise several parts,
covering different aspects of method validation, and at
least the following ones:
– Part 1 on terminology
– Part 2 on validation of proprietary methods
– Part 3 on in-house method validation, distinguishing in
particular the case where the method to be validated is
compared or not to a RM, and the case of cultivable/
non-cultivable microorganisms (e.g. viruses)
– Part 4 on method verification, i.e. how a laboratory can
verify that it is correctly implementing a validated
method, in particular in the context of laboratory
accreditation
Because the case of alternative/innovative methods
(called ‘proprietary’ methods in the draft revision of the
standard) involves the most comprehensive validation
process, with an in-house study and an inter-laboratory
study, part 2 will represent the basis for the other parts of
the revised standard.
Compared to the existing standard presented earlier, part
2 will include the following key modifications:
– The table defining the classification according to food
categories and food types (Annex B of the current
standard) will be reviewed and restructured.
– The experimental design for the inter-laboratory study
for both qualitative and quantitative methods would
allow the possibility of a reduced number of partici-
Food Anal. Methods
pating laboratories (less than eight or ten). It is
envisaged to have several participants in the same
laboratory/geographical site, with a minimum of five
different laboratories.
– For qualitative methods, it is intended to combine the
experimental design for trueness and LOD. A protocol for
artificial contamination better mimicking the natural stress
conditions of the targeted bacteria would be introduced:
instead of a spiking technique with evaluation of injury
efficiency given in a normative annex of the current
standard, a seeding protocol, without injury efficiency
evaluation, would be provided in an informative annex,
thus, a less stringent requirement which may introduce
noticeable differences between the technical protocols of
validation/certification systems. It is intended to report the
AM results on two forms: non-confirmed (screening) and
confirmed: compared to the existing version, the estima-
tion of trueness would not be limited to a strict comparison
of the AM and RM. It is envisaged to assess the
performance of the qualitative methods on the basis of
two parameters: (1) mainly relative LOD (RLOD, a ratio
between the LODs of the AM and RM), at 50% and 95%
levels, with calculations based on a complementary log
−log model (a generalised linear model) developed by
ISO/TC 34/SC 9/WG 2 (Wilrich and Wilrich 2009) and
(2) inclusivity/exclusivity, as in the current standard. An
acceptability limit would be defined for RLOD, to assess
equivalence between the AM and RM. RLOD would be
a performance characteristic for both the method com-
parison study and the inter-laboratory study. The other
parameters of the existing standard (accuracy, sensitivity
and specificity) are considered less valuable than RLOD
since their values are dependent upon the contamination
level of the samples used in the experimental design, and
they would be maintained for information only. The
RLOD approach is currently suited to unpaired results
(results obtained from two different test portions, each
analysed by AM and RM), but investigations are under
way within ISO/TC 34/SC 9/WG 2 to expand its use to
paired results (results obtained from one test portion, the
first step of the analysis—generally pre-enrichment—
being common to AM and RM). In case of a successful
outcome of this work, the McNemar test, specific for use
with paired results, would be removed.
– For quantitative methods, a new approach would be
introduced for the performance characteristics: the
accuracy profile, already implemented in the pharma-
ceutical field (Hubert et al. 2003, 2006a, b), which
jointly interprets precision and trueness. It avoids one
of the drawbacks of the usual approach to validation of
the AM against a RM: the test for bias significance
may conclude, for a given bias value, an acceptable
bias with an AM having poor precision, and a
Food Anal. Methods

significant bias with an AM having good precision.
The accuracy profile approach would replace the
current approach to linearity/trueness based on regres-
sion analysis, keeping only a graphical representation.
The LOQ would be maintained, but with a new
estimation, directly derived from the graphical repre-
sentation of the accuracy profile.
For any calculations, WG 2 is to provide calculation
tools which could be easily implemented by microbiolog-
ical laboratories, without the need to ask a statistician to
perform them. An added value would be to improve the
standardisation of the data analysis. This goal should be
met by developing Excel™ spread-sheets or by providing
calculation tools on the ISO website.
A draft for part 1 and part 2 was circulated in July 2009
to ISO/TC 34/SC 9 and CEN/TC 275/WG 6 for an initial
vote to include these two parts in the official work
programme. The outcome was positive, and comments
received have been considered by ISO/TC 34/SC 9/WG 3
to prepare a new draft to be submitted to the next vote (ISO
Committee Draft vote) in the course of 2010.
Validation/Certification Systems Based on EN ISO
16140
A test kit's certification based on the EN ISO 16140
Standard obviously requires that the technical validation
follow the entire protocol defined in the standard, providing
the assessment of a kit's performance at a given time. The
added benefit of certification is that the validation study is
performed and evaluated by a certification body, a third
party independent from the test kit's manufacturer. It also
ensures that the method's performance remains constant
over time, with quality assurance requirements and audits
on the production site. The organisational aspects of
certification are defined by the certification body itself
and described in documents called certification rules.
Today, only two certification systems are known to apply
EN ISO 16140 in full:
– AFNOR Validation scheme managed by AFNOR
Certification, which has long experience in test kit
certification in food microbiology (since 1989) and has
certified to date around 80 methods (http://www.afnor-
validation.com/afnor-validation/afnor-validation.html)
– MicroVal, managed by NEN, currently comprising one
certification body, Lloyd's Register QA (http://www.
microval.org), and derived from the MicroVal project
(see above)
Other systems to validate innovative methods in food
microbiology can be found, the main ones being the
Official Methods scheme of AOAC International, Perfor-
mance Tested Methods scheme of AOAC Research Institute
in the USA and NordVal of NMKL in the European Nordic
countries. The Official Methods scheme and NordVal
follow a technical protocol similar but not identical to that
of EN ISO 16140. In contrast, the technical protocol of the
Performance Tested Methods scheme does not include an
inter-laboratory study. In addition, the organisations
concerned do not operate as certification bodies, lacking
in particular the control of the manufacturer's quality
assurance.
In the future, official reference laboratories, such as
Community or National Reference Laboratories in Europe,
may become new actors in the field of validation of AMs,
depending on possible new regulatory duties given by their
prescribers, the Directorate-General for Health and Con-
sumers (DG SANCO) of the European Commission, or
national competent authorities in charge of official food
control.
Another point is that certification bodies contract with
expert laboratories to conduct the two parts of the
validation study described in EN ISO 16140: the methods
comparison study and the inter-laboratory study. The two
certification schemes (AFNOR Validation and MicroVal),
as well as NordVal, require that the expert laboratory be
accredited according to EN ISO 17025 for the relevant RM,
whereas this is not the case with AOAC. However, such an
accreditation does not appear to cover an expert laboratory's
entire field of competence necessary to implement the
experimental designs, calculations and interpretation of
the different phases of the validation study. Such a
competence includes (1) the organisation of the inter-
laboratory study, which is particularly difficult in food
microbiology, given the living nature of the bacteria, and
(2) the quality assurance of the expert laboratory's strain
collection (traceability, characterisation), a key element to
conduct inclusivity and exclusivity testing, as well as to
perform artificial contamination. It is true that a single
reference document for validation of AMs, the EN ISO
16140, is crucial, but the guarantee of its correct application
with reliable results is no less important. It would require in
the future the development of the accreditation of expert
laboratories for the application of EN ISO 16140, in the
same way that proficiency testing (PT) scheme providers
can be accredited according to EN ISO 17043 (Anonymous
2010) to organise these PT schemes.
Validation of Confirmation/Identification Step
The EN ISO 16140 Standard is intended to be used to
validate a complete method (from sample preparation to
final result) but not to validate a separate step of an AM,

such as the identification or confirmation of the targeted
bacteria by biochemical galleries, nucleic probes, PCR 16 S
DNA and other molecular biology methods. To our
knowledge, no protocol is currently available to validate
an identification/confirmation step, and we consider that the
development of such a protocol, with the subsequent
validation/certification of commercial kits, is necessary.
According to the EN ISO 7218 Standard (Anonymous
2007), dealing with the possible use of biochemical
galleries (Clause 12.4) or nucleic probes (Clause 12.5) as
alternatives to the biochemical confirmation tests described
in standard RMs, an evaluation file established by the
manufacturer, based on studies published in the interna-
tional scientific literature, or the recommendations of a
reference laboratory for this microorganism is sufficient,
but guidance on the content of this validation file is still
lacking.
Relations Between EN ISO 16140 and Laboratory
Accreditation
According to the EN ISO 17025 (Anonymous 2005a)
Standard (Clause 5.4.5.1 on laboratory quality manage-
ment), accredited laboratories shall validate the non-
standardised methods they use, such as test kits. If an AM
has been validated/certified according to EN ISO 16140, it
is obvious that the EN ISO 17025 requirement has been
met and the laboratory does not need to conduct any more
validation itself when using this method. Method validation
can be considered as one of the three pillars of quality
assurance of test results, together with proficiency testing
and reference materials.
The EN ISO 17025 Standard (Clause 5.4.2) also requires
that laboratories verify that the methods they use, once
validated, are correctly implemented in the particular
context of their laboratory, and in particular that pre-set
performance criteria are met. The new part of EN ISO
16140 on method verification, under development, will
allow us to have a common reference document to define in
food microbiology the notion of method verification, with
the purpose to harmonise the practices and requirements of
accreditation bodies.
Relations Between EN ISO 16140 and European
Regulation 2073/2005 on Microbiological Criteria
According to Codex Alimentarius (Anonymous 1997), a
microbiological criterion shall be based in particular on a
detection or quantification method which is relatively easy
to implement but also reliable and validated. Microbiolog-
ical criteria are strictly linked to the methods used to check
Food Anal. Methods
product conformity with these criteria. When a regulatory
text defines a microbiological criterion, it should also
define the RMs and AMs to be used.
As regards the situation in Europe, Commission Regu-
lation (EC) 2073/2005 (Anonymous 2005b) gathers and
updates microbiological criteria for food in the European
Union (EU). This regulation is formally applicable to food
business operators, along the whole food chain from
production to distribution, but it is also indirectly applicable
to official controls performed by public authorities of EU
Member States. It defines, for each microbiological
criterion, a RM, generally an EN ISO Standard, but states
in an article of the main text (Article 5) that other methods
can be applied, under the following conditions: ‘when the
methods are validated against the Reference Method in
Annex I and if a proprietary method, certified by a third party
in accordance with the protocol set out in EN ISO standard
16140 or other internationally accepted similar protocols’.
From this text, it is clear that, if an AM has been validated
according to EN ISO 16140 or if a proprietary method has
been validated and certified according to this standard, in both
cases in comparison to the corresponding RM cited in the
regulation (mostly CEN/ISO Standards), it can be used under
this regulation for the operator's own checks and for official
controls, so as to assess whether food products meet the
microbiological criteria defined in the regulation.
Risk managers should be aware that, if an innovative
method has been validated, it offers at least the same
performance as the corresponding RM, and it may even
perform better. A difference in performance between the
two methods may be a source of litigation between two
countries. For example, a product testing positive for
Salmonella with a PCR method by the importing country
may be rejected, whereas it may have been tested negative
by the exporting country using the EN ISO 6579 reference
culture method.
Relations Between EN ISO 16140, Its Revision and New
Standards
The rapid development of techniques, such as molecular
methods, and their proprietary nature will make it difficult
in a growing number of cases to fully describe the methods
in ISO/CEN Standards. With the current revision of EN
ISO 16140, a new type of standard could be developed in
food microbiology, which is already the case for food
chemistry: standards composed, apart from a precise scope,
of analytical performance criteria. In this case, any
innovative methods could be applied if they met the scope
and performance criteria described in these standards. For
example, a standard for the detection of staphylococcal
enterotoxins is being developed according to this new
Food Anal. Methods
approach, which appears to be suited to the detection step
which currently needs to rely upon proprietary ELISA kits:
setting performance criteria will avoid the need to refer to
specific kits in the standard.
Another impact of the EN ISO 16140 revision on
European and International Standards is the preparation,
by the same WG 3 of ISO/TC 34/SC 9, of a new CEN ISO
publication on the validation of RMs. To achieve full
recognition at worldwide level and to better play its role of
anchor method for the validation of AMs, a RM should be
validated during its development process and before its
approval. A RM should be also validated if used outside its
intended scope or if technically amplified or modified, with
major changes. It is intended that the new CEN ISO
publication will define the technical requirements (type of
studies and performance criteria) to standardise a new RM
or to revise an existing standard method. Validation of RMs
is a big challenge, since these methods are of common
interest to all countries, but their validation requires
financial support at either national or regional levels.
Different Users of EN ISO 16140
The main users of EN ISO 16140 (Part 2 for its revision)
are the validation/certification bodies and their partners, in
particular the expert laboratories, which conduct validation
of test kits at the request of kit manufacturers. The second
type of user is expected to be public authorities who are to
refer to this standard in order to accept AMs, which could
be used for official controls. Regarding the other parts of
EN ISO 16140 under revision, users would be (1)
laboratory networks, such as in Europe the networks of
National Reference Laboratories, each coordinated by a
European Union Reference Laboratory for different bacteria
or viruses, or (2) individual laboratories, which have
developed their own methods and intend to validate them
against the corresponding RMs, or verify their correct
implementation in the specific context of the laboratory, for
accreditation purposes. Depending on the method and its
scope, the estimated cost of implementing EN ISO 16140 is
expected to lie between 50,000 and 150,000 Euros for a test
kit manufacturer in the context of a validation/certification
scheme.
Choice and Use of Validated/Certified Commercial
Methods
According to Thompson et al. (2002), to validate a method
is to investigate whether the analytical purpose of the
method is achieved, that is the acquisition of analytical
results with an acceptable uncertainty level. Once an
innovative method has been validated, the certification
body issues a certificate. Laboratories using such validated
methods often do not read these certificates carefully and
think that AMs are systematically fully validated; this is not
always the case. For example, the certificate includes the
validation scope which underlines the food matrices tested:
not all food matrices are covered and the method may
perform differently with specific matrices than with the one
reported in the validation study. Another aspect is the
possible restriction of use of the AM, to be clearly
stipulated on the certificate, for example, stating that the
validated method only detects motile Salmonella, and not
all Salmonella spp. as in the EN ISO 6579 Standard RM.
Validation certificates also describe performance criteria,
accompanied by comments which should be read carefully:
some limitations or values of performance criteria may be
acceptable for the technical board of the certification body
but not for customers of the laboratory. For example, in the
exclusivity criterion, some strains (species or serotypes or
types) may not be detected by the AM, but this is not
stipulated in the restrictions of use.
When a food microbiology laboratory, especially if
accredited according to EN ISO 17025 (Anonymous
2005a), conducts a contract review with its customers, it
should mention all these restrictions. In this case, the
performance of a method matches the criteria agreed
between the analyst and the end-user of the data (Thompson
et al. 2002). Moreover, EN ISO 17025 requires that the range
and accuracy of the values obtained from a validated method
be relevant to the customer's needs.
A laboratory's choice of an AM should not only be based
on its technical validation. Other aspects should be taken
into account: (1) logistic and laboratory aspects (sample
preparation, staff competence, availability and safety of
reagents, time to achieve the final result, productivity,
automation, laboratory space needed, quality assurance,
verification and metrology of the equipment and technical
assistance provided by the manufacturer), (2) financial
aspects (purchase cost of the equipment and/or reagents)
and (3) strategic aspects (recognition by customers, distrib-
utors, public authorities and accreditation bodies).
Once the proprietary method has been selected, the
laboratory needs to verify the correct implementation of this
validated method: this is a requirement for accredited
laboratories and for any method used, according to Clause
5.4.2. of EN ISO 17025. Verification is the confirmation,
through the provision of objective evidence, that specified
requirements have been fulfilled (Anonymous 2008). It
differs from validation since objective evidence is obtained
not to fulfil the intended use but the specified requirements.
It is an additional procedure that confirms that the
laboratory has achieved the performance characteristics of
the method. Given the lack of harmonised rules on method

verification, it has been decided that a separate part of the
revised EN ISO 16140 would be dedicated to this topic (see
above).
Finally, during the use of a given method, some method
parameters may have to be changed or adjusted if the
values of the performance characteristics appear to fall
outside their acceptance limits. The question is whether
such changes require revalidation. For accredited laborato-
ries, according to Clause 5.4.5.2 Note 3 of EN ISO 17025,
when changes are made to validated non-standard methods,
the influence of such changes should be documented and, if
appropriate, a new validation should be carried out. In order
to deal with this question, the laboratory should define an
operating range for each method, either based on its
experience with similar methods or investigated during
method development. This range should be verified during
method validation in trueness studies and should be part of
the method characteristics. Revalidation is also required if
the scope of the method has been changed or extended.
This question will be covered in the forthcoming part 2 of
the revised EN ISO 16140.
Conclusion
In numerous cases, laboratories performing routine food
microbiological analyses do not use the RMs based on
conventional microbiology but increasingly often choose
proprietary methods, based on innovative technologies,
which provide results in a shorter amount of time, are
easier to perform, require fewer resources in terms of
skilled staff, and are more automated. The validation of
these proprietary methods according to EN ISO 16140, and
their subsequent certification, give assurance to the labora-
tories and their customers (in the food industry, food
distribution, public control authorities) that the results
obtained with these methods are valid and at least as good,
if not better, than those obtained with the corresponding
RMs, the ‘gold standards’.
In certain cases, laboratories may also use in-house
methods that they have developed. The revised EN ISO
16140 will provide a tool to the laboratories to validate
these methods and to give confidence in their reliability,
both to their customers and to accreditation bodies.
The revised EN ISO 16140 will also define what method
verification in food microbiology is, guiding the laborato-
View publication stats
Food Anal. Methods
ries in its implementation and allowing a harmonised
assessment by accreditation bodies.
To further ensure the reliability of results obtained with
these innovative methods, ISO and CEN should develop
general requirements and guidance to use these methods in
food microbiology laboratories, such as the EN ISO 22174
Standard (Anonymous 2005c) for PCR.
References
Anonymous (1997) CAC/GL 21 Principles for the establishment and
application of microbiological criteria for foods, Codex Alimen-
tarius, Rome
Anonymous (2003) EN ISO 16140: Microbiology of food and animal
feeding stuffs—Protocol for the validation of alternative meth-
ods, ISO, Geneva
Anonymous (2004) ISO/IEC 17000: Conformity assessment—Vocab-
ulary and general principles, ISO, Geneva
Anonymous (2005a) EN ISO 17025: General requirements for the
competence of testing and calibration laboratories, ISO, Geneva
Anonymous (2005b) Commission Regulation (EC) 2073/2005 of 15
November 2005 on microbiological criteria for foodstuffs.
Official Journal of the European Union, L338/1-26
Anonymous (2005c) EN ISO 22174: Microbiology of food and
animal feeding stuffs—Polymerase chain reaction (PCR) for the
detection of food-borne pathogens—General requirements and
definitions, ISO, Geneva
Anonymous (2007) EN ISO 7218: Microbiology of food and animal
feeding stuffs—General requirements and guidance for microbi-
ological examinations, ISO, Geneva
Anonymous (2008) EN ISO 9001: Quality management systems—
Requirements, ISO, Geneva
Anonymous (2010) EN ISO 17043: Conformity assessment—General
requirements for proficiency testing, ISO, Geneva
Hubert Ph et al (2003) STP Pharma Pratiques 13:3
Hubert Ph et al (2006a) STP Pharma Pratiques 16:1
Hubert Ph et al (2006b) STP Pharma Pratiques 16:2
Langton SL, Chevennement R, Nagelkerke N, and Lombard B (2002)
International Journal of Food Microbiology 79
Leclercq A (2002) Accredit Qual Assur 7:299
Leclercq A, Lombard B, Mossel DAA (1999) Sciences des Aliments
20
Lombard B (2004) ISO Focus 1:8
Lombard B, and Leclercq A (2010) The role of standardization bodies
in the harmonization of analytical methods in Food Microbiol-
ogy. In: Global Issues in Food Science and Technology, IUFoST,
Elsevier ed (in press)
Lombard B, Gomy C, Catteau M (1996) Food Control 7:1
Lombard B, Feinberg MH, Lahellec C (2005) J AOAC Int 88:3
Rentenaar IMF (1996) Food Control 17:1
Rousseeuw PJ, Croux C (1993) J Am Stat Assoc 88:424
Thompson M, Ellison SLR, Wood R (2002) Pure Appl Chem 74:835
Wilrich C, Wilrich PT (2009) J AOAC Int 92:6