SIX SIGMA RISK ANALYSIS

Six Sigma Tools and Metrics

Learn what ISO standards, CLSI guidelines, and Joint
Commission protocols recommend for Risk Analysis
Identify hazards and failure modes in your own processes
Apply the appropriate ranking scale (qualitative, semi-
quantitative, and quantitative) to your process
Use Risk Analysis tools to assess and judge the
acceptability of risks in your processes.

Risk Analysis is coming to medical laboratories. But for too many

labs, Risk Analysis is a buzzword without meaning, an approach
without defined technique. In this book, Dr. Westgard surveys the
ISO standards (ISO 14971, ISO 22367) as well as the CLSI guidelines
(EP18, EP23) and the Joint Commission methodology for Proactive
Risk Reduction.
After providing an overview of the general approach to Risk Analysis,
Dr. Westgard explains how to adapt the principles for the medical
laboratory, using data-driven tools and practical implementation tips:

Process maps, flowcharts and fishbone diagrams

Risk Acceptability matrices
Assessment of hazards through Failure Mode Effect Analysis (FMEA)
Fault Tree Analysis (FTA) and Failure Reporting, Analysis and Corrective
Actions System (FRACAS)
Six Sigma metric integration into the Risk Analysis techniques

WESTGARD QC, INC.

Using Six Sigma metrics, Dr. Westgard shows how Risk Analysis
can be converted from an arbitrary and qualitative technique, into
something concrete, quantitative, and relevant to medical laboratories
and the patients they serve.
For laboratories serious about adopting Risk Analysis in their
operations - and manufacturers eager to provide industry-leading
support of their instruments - this is an essential reference.

7614 Gray Fox Trail Madison WI 53717

Copyright 2011 Westgard QC Inc.
http://www.westgard.com
 Six Sigma
Risk Analysis
Designing Analytic QC Plans
for the Medical Laboratory

James O. Westgard, PhD

Copyright 2011
7614 Gray Fox Trail, Madison WI 53717
Phone 608-833-4718 HTTP://WWW.WESTGARD.COM
 Library of Congress Control Number: 2011906056

ISBN 1-886958-27-0
ISBN-13 978-1-886958-27-2
Published by Westgard QC, Inc.
7614 Gray Fox Trail
Madison, WI 53717

Phone 608-833-4718

Copyright 2011 by Westgard QC, Inc. (WQC). All rights reserved.

No part of this publication may be reproduced, stored in a retrieval
system, or transmitted in any form or by any means, electronic,
mechanical, photocopying, recording, or otherwise, without prior
written permission of Westgard QC, Inc..
 Westgard QC, Inc. Copyright 2011

Preface
With manufacturers building control mechanisms into their
analytical systems, laboratories are interested in customizing their
quality control systems on the basis of risk analysis and the remaining
failure modes, i.e., errors that still may occur and affect the quality
of laboratory testing. The focus in this book is on the development
of Analytic Quality Control Plans to fit the needs of a particular
analytic instrument as operated in an individual medical laboratory.
The QC issues discussed in this book trace their origin to
CMSs 2004 interpretative guidelines for Equivalent QC (EQC),
which allow laboratories to reduce the frequency of QC from 2 lev-
els of controls per day to 2 per week or even 2 per month. There is
no scientific evidence to support the equivalence of these practices
to traditional QC procedures. There is no proof that these reduced
QC frequency protocols provide adequate error detection or patient
safety, even though CMS prescribed validation protocols in order
to qualify for reduced QC frequency. Those validation protocols
themselves are not valid.
The EQC guidelines seem to be driven by a desire to simplify
QC practices, particularly for Point-of-Care applications where op-
erators have little experience in doing laboratory tests and minimal
knowledge of traditional QC practices. Because of concerns and
urgings of both manufacturers and laboratories, CLSI initiated a
project to develop a new guideline for QC procedures based on risk
analysis. That guideline is known as EP23 Laboratory Quality
Control based on Risk Management.
Manufacturers are generally familiar with risk analysis be-
cause of practices recommended in ISO 14971 Application of risk
management to medical devices. Medical laboratories, on the other
hand, have little or no experience with formal risk analysis. We,
like others, had to start from scratch to study risk analysis in order
to understand its potential application for developing Analytic QC
Plans. In the process of learning about risk analysis, we examined
the risk models and the reliability of particular techniques for es-
timation or calculation of risk. We assessed the practicality of the
risk analysis methodology that was being recommended. On the

Page i
 Six Sigma Risk Analysis

basis of our studies, we concluded there could be serious problems

in implementing the CLSI guidance unless additional educational
materials and training programs are made available to laboratories.
This book is part of our efforts to provide more practical and
quantitative guidance for the application of risk analysis in medical
laboratories. In Part I, First, do no harm!, we consider the problems
and issues that must be addressed within the overall framework for
managing analytical quality in a medical laboratory. In Part II, ISO
and CLSI Guidance, we review several international and national
consensus standards, including ISO 14971, ISO 22367, ISO 15198,
CLSI C24, CLSI EP18, and CLSI EP23. In Part III, Methodology
and Tools, we recommend adoption and adaptation of the JC (Joint
Commission) Proactive Risk Reduction methodology and illustrate
the application of many of the tools that are useful for developing
Analytic QC Plans. Our resulting methodology employs a more
rigorous risk analysis model and a more quantitative approach for
assessment of the residual risk of an Analytic QC Plan.
As quality management systems evolve, risk analysis should
be integrated with existing practices, particularly with Six Sigma
concepts, principles, tools, and metrics. That is the basis of our
approach in Six Sigma Risk Analysis. Six Sigma is inherently risk
oriented in its definition of tolerance limits, estimation of defects,
and characterization of defect rates. Risk models, when properly
applied, can provide estimates of the number of defective test results
that may produced by a laboratory. Such defective test results are
potentially harmful or hazardous to the health of our patients.
Given that Six Sigma Risk Analysis is a more advanced and
complex subject, it will require more extensive and more thorough
study than some of our previous books. This book assumes a basic
knowledge of quality control, such as found in our book on Basic QC
Practices that describes the principles and procedures for imple-
menting Statistical QC to monitor the performance of the methods
in your laboratory. It assumes knowledge of the experimental and
statistical techniques for evaluating the analytical performance of
measurement procedures, such as found in our book Basic Method
Validation. It also builds on Six Sigma concepts, principles, and
tools, as found in our book on Six Sigma Quality Design and Control

Page ii
 Westgard QC, Inc. Copyright 2011

and in our book on Assuring the Right Quality Right, which focuses
on the design of Statistical QC procedures to verify the attainment
of the intended quality of test results (the technical requirement for
QC as stated in ISO 15189).

A Note on ISO standards and CLSI guidelines

As of the time of this printing, the CLSI EP23 was not yet finalized
and accepted as a guideline. Nevertheless, the final shape of the
document is complete.
But in that spirit, readers should note that each standard,
guideline, and regulation is inevitably a moving target. ISO and CLSI
continuously review and attempt to improve their documents, and
every few years they issue an update. The regulatory and accredi-
tation bodies (CLIA, CAP, JC, etc.) do the same. Thus, the specific
language of some of these standards will change. However, as you
are probably well aware, large changes in regulatory policy are rare.
It is unlikely that major changes will occur that change the goals of
these organizations and their recommendations.
It is also important to note that this book is NOT meant
to replace or substitute for ISO standards or CLSI guidelines.
Laboratories are strongly encouraged to purchase the spe-
cific documents that they intend to use in their operations.
For a manufacturer or laboratory that intends to implement Risk
Analysis, it will not be sufficient to read just this book.
Where this book can be helpful is to give an overview and a
comparison of ISO, CLSI, and JC recommendations. Laboratories
may be able to decide which documents to purchase, as well as how
to reconcile the differences between the different standards and
guidelines.
The other unique feature of this book is the Sigma-metric ap-
proach. While the other recommendations tend to be vague on how to
rank and judge the acceptability of risk, this book is very quantitative
and data-driven. Assessing your risk on the Sigma-scale will give
you a concrete estimate of its residual risk and impact on patient
care. The combination of Six Sigma and Risk Analysis can provide
powerful tools and techniques to the medical laboratory.

Page iii
 Six Sigma Risk Analysis

Acknowledgments
First and foremost, Sten Westgard made this book possible
through his perspective on risk analysis in the world outside the
laboratory, his interest that we find a way to help laboratories do a
better job of risk analysis, and his commitment to supporting pub-
lication of these new materials, both as Internet courses and this
book. I am fortunate that once these words are typed into a word
processor, they can be transformed into educational materials and
quickly made available to the laboratory community. Sten makes
that happen and these materials wouldnt exist without him.
In developing Six Sigma Risk Analysis, we have been stimu-
lated by discussions with many people, including Jan Krouwer, Don
Powers, Greg Cooper, Tina Krenc, Mike Noble, and Jim Nichols. We
have also been stimulated by the evolving standards and guidelines
for application of risk analysis with medical devices and medical
laboratories, and appreciate the time, effort, and hard work that
has gone into their development. We hope that our distillation of
the current guidelines and our recommendations for adapting the
JC methodology will support the implementation of a more rigorous
and objective approach for risk analysis in medical laboratories,
particularly for the design of Analytic QC Plans that will verify the
attainment of the intended quality of results, as recommended in
ISO 15189.

But wait, theres more!

This book is not long enough to contain all the tools and lessons that
we want to make available to you. So weve put some more online.
For readers who have purchased this book, additional tools
available online. For example, a set of power function curves that
show the error detection and false rejection capabililites of Average
of Normals (AoN) QC procedures.
Visit http://www.westgard.com/risk-extras.htm for more de-
tails. Youll be given a link to a special location, along with password
instructions.

Page iv
 Westgard QC, Inc, Copyright 2011

Table of Contents

1. Controlling Quality...................................................................................................................................... 1

2. Managing Analytical Quality . .....................................................................................................13

3. Analyzing and Assessing Risk..................................................................................................25

4. A Safety Net to Catch Analytical Errors.........................................................................51

5. ISO 14971 Risk Management for Medical Devices. ........................................67

6. ISO 15198 and CLSI C24 Guidance for Safe Use and QC........................95

7. EP18 & EP22 Guidance for Risk Analysis and QC Plans......................113

8. ISO 22367 Guidance for Risk Management........................................................... 137

9. Adopting the JC Risk Analysis Methodology..................................................... 147

10. Diagramming a Laboratory Process........................................................................... 163

11. Identifying Failure Modes......................................................................................................... 177

12. Prioritizing Failure Modes. ...................................................................................................... 189

13. Determining Root Causes. ...................................................................................................... 203

14. Mitigating Risks with an Analytical QC Plan.................................................... 213

 Six Sigma Risk Analysis

15. Estimating Detection and Evaluating Residual Risks.......................... 233

16. Monitoring Failures and Measuring Performance..................................... 259

17. Implementing Analytical QC Plans............................................................................... 275

18. Integrating Six Sigma into Risk Analysis . .......................................................... 283

Index .................................................................................................................................................
291
 Westgard QC, Inc. Copyright 2011

3: Analyzing and Assessing Risk

with Sten Westgard, MS

What is Risk Management?

Existing ISO guidelines and emerging CLSI guidelines recom-
mend that risk management be applied in healthcare laboratories.
Laboratories face a learning curve that includes basic concepts and
terminology, as well as practical applications for the total testing
process. Our particular interest here is the application of risk man-
agement to the analytical portion of the testing process, particularly
the development of an Analytical QC Plan that takes into account
the potential risk of a particular test, its analytical methodology,
and its application in a laboratory environment.

We want to be forthright about our concern about the ap-

plication of risk management in healthcare laboratories. We agree
with the principles of risk management to prevent problems from
occurring and reduce harm to patients when problems do occur. It
would be ideal if errors can be prevented by manufacturers in their
design of analytic systems and minimized by built-in controls and
instrument checks, but laboratories are still responsible to verify
the attainment of the intended quality of test results, according to
ISO 15189 [1]. We believe that means Statistical QC should be a
major part of any Analytical QC Plan. Also, we have reservations
about practice guidelines that are emerging, particularly in the US
where the drive has been to reduce the amount of QC performed,
rather than to optimize QC to guarantee the quality of test results.
Our purpose here is to demonstrate that the principles of risk
management can be related and applied quantitatively as part of
analytical quality management and with the application of tradi-
tional SQC. To accomplish this, we will review the principles of
risk management, define the terms to clarify the risk management
Process, focus on the steps involved with risk assessment and a

Page 25
 Six Sigma Risk Analysis

commonly-used tool Failure Mode Effects Analysis (FMEA), then

demonstrate how its application to analytical testing can be related
to the Sigma performance of analytical methods and systems.

An all-too-brief history of Risk Management

Peter Bernstein, author of Against the Gods: The Remarkable Story
of Risk [2], places the birth of Risk (as we know it) in a coffee house in
London around the year 1696. Edward Lloyd, the enterprising owner
of the Starbucks of his day, noticed his customers mostly sailors
and ship captains, since he was near the docks had an insatiable
thirst for information about the arrivals and departures of ships, as
well as details on sea conditions and news in foreign countries. He
published all of this information in what became known as Lloyds
List. All of this information, assembled together, dramatically
improved the ability to launch new ventures.
[A]nyone who was seeking insurance would go to a broker, who
would then hawk the risk to the individual risk-takers who gathered
in the coffee houses or in the precincts of the Royal Exchange. When
a deal was closed, the risk-taker would confirm the agreement to
cover the loss in return for a specified premium by writing his name
under the terms of the contract; soon these one-man insurance
operators came to be known as underwriters.
In 1771, nearly one hundred of the underwriters formally
created the Society of Lloyds. Over 300 years since Edward Lloyd
began serving information along with coffee, Lloyds of London has
grown into a leading supplier of specialist insurance.
Of course, you might protest, thats the birth of insurance, not
risk management. But the two concepts are intertwined. Insurance
is an answer to the problem of risk. Its illustrative to see that at
the very birth of the risk and risk management, information was
critical to the process. Even if sea merchants lacked the informa-
tion or ability to mitigate their hazards by redesign (i.e. change the
weather so their ships would avoid storms), they gained some idea
of what to avoid.

Page 26
 Westgard QC, Inc. Copyright 2011

As we return to the history of risk, we need to connect the birth of

risk and risk management to its formal adoption and implementation
in healthcare. To do that, we have to leap forward several hundred
years to the malpractice crisis of the 1970s and 1980s. During that
time, verdicts and settlements against hospitals and other health-
care institutions threatened to overwhelm their financial reserves.
Risk management became a way to address the threat of litigation.
While at first this was a defensive, reactive move, a way to shield
the institution from litigation and losses, eventually healthcare
institutions began to use risk management proactively [3]:
Professionals with clinical experience were hired with the hope that
they could identify the systemic problems in specific clinical areas
(primarily obstetrics, anesthesia, and the emergency department),
engage clinicians and educate them about the need to modify specific
behaviors, and work collaboratively with others on the clinical and
administrative teams to help design environments that would be
more conducive to the delivery of safe care.
When the reports of the Institute of Medicine, To Err is Hu-
man [4] and Crossing the Quality Chasm [5] were issued in 2000 and
2001, respectively, both the public and healthcare professionals alike
were shocked by the frequency and severity of medical errors. That
heightened awareness motivated the healthcare field to search for
new tools to combat and prevent medical errors. Risk management
became one of the new tools to address the problem.
In healthcare, the Joint Commission recommended the use of
risk management as part of the Patient Safety Movement. Its ac-
creditation guidelines for 2002 included a requirement that health-
care organizations should perform at least one Failure Mode Effects
Analysis (FMEA) each year [6]. Also in response to patient safety
issues, the Institute for Healthcare Improvement (IHI) began provid-
ing education, training, and support for FMEA via its website [7].
In addition, the Veterans Affairs National Center for Patient Safety
supported the use of FMEA throughout its healthcare institutions[8].

Page 27
 Six Sigma Risk Analysis

Finally, still another thread of history helped bring risk man-

agement into healthcare institutions global standards courtesy
of ISO. Long accepted by industry, ISO sets rigorous guidelines
for processes and products marketed worldwide. Adherence to ISO
standards is often a de facto requirement for businesses to compete
globally. As ISO standards were expanded and applied to more and
different segments of industry, they developed standards for risk
management in Medical Devices (ISO 14971)[9]. The medical device
industry, already an industry where litigation worries mandated a
robust analysis of potential design flaws and device hazards, found
that the risk management techniques married well with their ex-
isting efforts to improve quality. From the medical device industry
to the medical device marketplace was only a small step. Already
ISO 15189 had specified particular requirements for quality and
competence in medical laboratories. A further standard, ISO 22367
[10], specified techniques for the Reduction of error through Risk
Management and continual improvement.
For laboratories outside the US, ISO standards often replace,
supplement, or substitute for local government regulations. Some
countries simply point to ISO standards and adopt them in their
entirety for accreditation of medical laboratories. In the US, how-
ever, ISO standards have not been widely adopted because the CLIA
regulations have been dominant. With the CLIA Final Rule in 2003
and the subsequent proposal for Equivalent Quality Control prac-
tices, the door opened wide for alternative quality regulations. Faced
with scientific and professional debate about the adequacy of the
new EQC guidelines, CLSI began to develop an alternate approach
for defining QC Plans based on risk management guidelines that
adhered to the ISO standards. These CLSI guidelines are intended
to supplement, if not replace, the CLIA guidelines for EQC. CMS
will decide whether the new risk-based QC guidelines can be used
to provide equivalent quality testing.

What is Risk?
Risk is both a noun and verb, a concept and an action. We can take
risks and we can risk disaster. We can speculate that analytical
error is one of the biggest risks in laboratory testing. We can try to

Page 28
 Westgard QC, Inc. Copyright 2011

11. Identifying Failure Modes

What tools can be used to identify failure modes?

An essential part of risk analysis is identifying what might go
wrong, i.e., the potential failure modes in the process under study.
The process should first be diagrammed to clarify how the process
works. Next, each step of the process should be examined to iden-
tify what might go wrong. The technique for doing this begins with
brainstorming by the project team to identify potential failure modes,
then summarizing and organizing those potential failure modes us-
ing a cause-and-effect diagram, also known as a fishbone diagram.

In any risk analysis methodology, the process of interest must

first be diagrammed to identify the critical steps or operations, as
described in the previous chapter. The next step in the methodology
is to identify potential failure modes, i.e., what might go wrong and
cause a delay or an error in reporting a test result. The identifica-
tion of failure modes often makes use of brainstorming, followed by
a graphical summary by fishbone diagram two tools that will be
described in this chapter. Later chapters will consider the prioritiza-
tion of failure modes to assess their relative importance and guide
efforts to reduce the risks of failures, the identification of root causes
of the high priority failure modes, and the utilization of risk control
option analysis (ISO/CLSI terminology) [1-4] or process redesign
strategies (JC terminology) [5] to eliminate causes when possible,
detect failures and implement corrective actions for recovery, and
reduce the risk of patient harm by providing information for the
safe use of test results.

Page 177
 Six Sigma Risk Analysis

Patient testing process

The overall patient testing process involves many steps and can be
extremely complicated. To simplify any risk analysis project, it will be
critical to confine the project to a part of the patient testing process,
otherwise the project may become unmanageable (i.e. the scope will
be so large as to be overwhelmed by a multitude of failure modes).
An overview of the patient testing process is shown in Figure 11-1,
which outlines two major processes the clinical testing process
and the laboratory testing process, the latter often called the total
testing process by the personnel in a medical laboratory. This
laboratory process consists of the pre-analytic, analytic, and post-
analytic phases of the patient testing process. It typically begins
with the receipt of an order for a test and ends with the report of
results. This is the way personnel outside the laboratory usually
see the laboratory a department or black box with an input for
test requisitions and an output for test results. Their focus is on
the clinical testing process, or the pre-pre-analytic and post-post-
analytic phases of the patient testing process, though they wont
necessarily describe it using this terminology.
In applying risk analysis in a medical laboratory, it will be
critical to delineate the scope of the project, particularly whether
it crosses the departmental boundaries between the clinical and
laboratory processes. Across-department projects will require a
more diverse and larger team. For example, projects on patient
identification and test turnaround time are inherently difficult be-
cause of the number of people and departments that are involved.
Even within the laboratory, it is necessary to carefully delineate the
start and stop steps to narrow the focus when possible to a primary
phase. For example, when the purpose of the risk analysis project
is to develop an Analytic QC Plan, the focus will be primarily on the
analytic phase of the laboratory testing process. Still, there could be
overlap with the pre-analytic phase related to sample quality and
overlap with the post-analytic phase related to implementation of
control mechanisms (e.g., delta checks), review of test reports, and
provision of information for safe use via the LIS or HIS. The proj-
ect team may need some representation from different areas in the
laboratory department.

Page 178
 Westgard QC, Inc. Copyright 2011

Clinical Testing Process

START

Pre-Pre-Analytic Post-Post-Analytic
Collect patient info Need Review test reports
Select test YES Laboratory Interpret test results
Prepare patient Test? Plan treatment
Enter test orders Treat patient

NO

END

Pre-Analytic
Receive test order
Identify patient
Collect specimen
Transport specimen
Process specimen
Prepare samples
Distribute samples
Analytic
Receive & Inspect samples
Post-Analytic
Prepare reagents & controls
Review test results
Setup analyzer
Monitor quality
Calibrate method
Add safety information
Analyze samples
Enter in patient record
Verify analyzer operation
Check QC Transmit test reports
Release test results

Laboratory Testing Process

Figure 11-1. Diagram of the patient testing process, showing the clinical
testing process (composed of the pre-pre-analytic and post-post-analytic
phases) and the laboratory testing process (composed of the pre-analytic,
analytic, and post-analytic phases).

This big picture of the patient testing process should provide

the frame of reference for narrowing the focus of any risk analysis
project. Next, a more detailed description in the form of a flowchart
is needed of the specific process of interest. This chapter focuses on
the analytic phase of the patient testing process and the identifica-
tion of potential failure modes that lead to delayed test results and
delayed patient diagnosis or treatment.

Page 179
 Six Sigma Risk Analysis

Analytic testing process

The analytic phase begins with the receipt of samples and ends with
the release of test results, as shown in Figure 11-2.

Receive Samples

Prepare reagents ,
calibrators, & Inspect samples
controls

Setup analyzer &

Samples Re-process,
Perform function NO
adequate? Re-collect
checks

YES

Calibrate &
Analyzer
Verify analyzer NO
Ready?
performance

YES

Analyze samples &

Repeat tests
controls

Analyzer Trouble-shoot,
performance NO Take corrective
OK? action

YES

Release test
results

Post-analytic
review

Figure 11-2. Flowchart for the analytic phase of an example laboratory testing
process.

Page 180
 Westgard QC, Inc. Copyright 2011

15. Estimating Detection and Evaluating

Residual Risks

How should the risk mitigations be tested and evaluated?

After identifying the risk mitigation strategies or redesign
options for addressing the high priority failure modes, the effective-
ness of the redesigned process should be tested and evaluated. For
laboratory applications where the redesigned process is actually an
Analytic QC Plan, the residual risks should be evaluated to determine
acceptability. The Joint Commission and ISO provide some guid-
ance on how to test the process, but little useful advice for judging
the acceptability of residual risks. The laboratory should assure
that the risks are acceptable for the particular clinical or medical
applications of the tests performed. In the case of laboratory tests,
quality requirements can be defined for the intended use of each
test, the detection of control mechanisms can be characterized by
their probabilities of rejection, risk can be estimated in the form of
a defect rate, and the residual risks can be described in terms of the
number of potentially harmful patient test results.

This step is described as analyzing and testing the redesigned

process in the Joint Commission (JC) methodology [1]. The gen-
eral approaches recommended include pilot testing, simulations,
and paper testing which in this case means preparing a second
FMEA. All these approaches have some utility in the evaluation
of an Analytic QC Plan in a medical laboratory if they are focused
on estimation of detection and evaluation of the risks that remain
after implementation of the controls, i.e., residual risks. ISO 15198
[2] provides additional guidance on how this might be done, recom-
mending studies that include the introduction of failure modes or
error conditions, or simulated challenges, and in the case of SQC
procedures, validation may be based on statistical evaluation of the
simulated effects of imprecision and/or bias on actual performance
data obtained in routine operating conditions.

Page 233
 Six Sigma Risk Analysis

The combined JC and ISO guidance suggests the following

approaches for analyzing and testing an Analytic QC Plan:
1. Evaluate a manufacturers information on the performance
of built-in controls, which should be based on experimental
studies to characterize their detection.
2. Characterize QC performance for stable sample and patient
data control procedures on the basis of simulated effects of
errors and estimate detection for medically important errors.
3. Prepare a second FMEA using a 3-factor risk model that
includes the estimates of detection for the various controls in
order to evaluate residual risks.

It is important to distinguish the two uses of FMEAs. In the

earlier step, a FMEA was used to prioritize the failure modes that
were then addressed by risk mitigation actions. In the latter step,
the purpose is to evaluate the effectiveness of those risk mitigation
actions and determine the acceptability of the residual risks that
remain. JC provides little guidance on how to make this decision.
ISO 14971 [3] and CLSI [4,5] describe risk acceptability matrices
that are qualitative (and somewhat arbitrary). And though they
talk about clinically acceptable risk or medically acceptable risk,
the guidelines provide no quantitative relationship between the
acceptability matrix and definitions of the quality needed for the
intended clinical and medical use of laboratory tests. Despite the
risk numbers, it boils down to a personal decision (or a committee
vote). You (or the team) decide that the risk is okay, based on your
judgment.
There should be a more quantitative, objective way to do this.
Analytic QC Plans should incorporate a definition of the allowable
total error for the quality required for a test. The expected defect
rate provides a rational metric for estimating the risk due to delayed
and/or erroneous test results. Defect rate can be used to calculate
the number of test results that are potentially harmful to patients,
which provides an understandable estimate of residual risk. In this
way, laboratories can make an informed judgment on the accept-
ability of residual risks, rather than resorting to an arbitrary risk
acceptability matrix.

Page 234
 Westgard QC, Inc. Copyright 2011

1. Evaluate the manufacturers information about built-in

controls
Ideally, a manufacturer will disclose the risk analysis studies that
describe the effectiveness of its built-in controls, but remember a
manufacturer is not required to do this. If such information is not
available, it may be possible for a medical laboratory to estimate
detection for some of these controls by performing the appropriate
experimental studies. For example, the effectiveness of serum indices
for detection of abnormal sample conditions could be determined
in the laboratory by preparing a series of samples having different
levels of hemolysis, lipemia, and bilirubin. Such experiments would
be a natural part of the method validation studies that should be
performed in the laboratory. Experimental testing of other built-in
controls might be possible but will be more difficult. It would be
much better if the manufacturer performed that testing and provided
clear documentation.
If this information is not available from the manufacturer, then
the laboratory must implement at least one independent control.
And if this risk information is not presented in a practically useful
manner (for instance, in terms of a defect rate), laboratories will have
little ability to understand the residual risks again pointing to a
need for at least one independent control mechanism whose detec-
tion can be determined and optimized for the intended use of the
test and the performance of the method. The easiest independent
control mechanism in your laboratory is SQC. With a statistical QC
procedure, you can determine its detection and verify the attainment
of the intended quality of results in your laboratory.

2. Characterize QC detection of statistical control

procedures
In evaluating the effectiveness of a QC Plan, a laboratory should
optimize the probability of detecting medically important errors (true
alarms) and minimize the probability of false error detection (false
alarms) [5]. Manufacturers are expected to provide information
about the performance of their electronic checks, built-in controls,
and integrated liquid controls. Laboratories, likewise, must consider

Page 235
 Westgard QC, Inc. Copyright 2011

17. Implementing Analytical QC Plans

How do you implement a QC Plan?

Implementation of an Analytic QC Plan may be done using
manual tools, offline computer tools, or online data processing tools.
The available tools will depend on software that resides in an ana-
lytic system, computerized work stations, middleware, Laboratory
Information System (LIS), or Hospital Information System (HIS).
Implementation may involve qualitative, semi-quantitative, or
quantitative applications of Six Sigma Risk Analysis. The complex-
ity of the analytic systems and the capabilities of the laboratory
analysts will affect how risk analysis is applied and how QC Plans
are implemented in the medical laboratory.

Applications of Six Sigma Risk Analysis depend on the re-

sources available in an individual laboratory, particularly the data
analysis programs and information systems for managing the testing
processes. Manual systems may be sufficient for low volume labo-
ratories and Point-of-Care applications, but computer support will
be needed for high volume testing and complex analytic systems.
Computer support may be found in the analytic system itself, of-
fline and/or online programs. As the volume and complexity of the
testing processes increase, online tools that are integrated into the
quality management process become increasingly important. Given
the many different analytic systems and the many different data
processing configurations, the capabilities for implementing QC
Plans will vary with the systems available to your laboratory. In
the discussion here, we use the terms qualitative, semi-quantitative,
and quantitative to refer to the rigor of the risk analysis methodol-
ogy and the objectivity in the estimation of risk, not to identify the
type of laboratory tests (i.e. qualitative or quantitative lab tests).

Page 275
 Six Sigma Risk Analysis

Qualitative risk applications

Lets start with the simplest case of a manual method that performs
only one test. This category applies to many Point-of-Care devices and
Physician Office Laboratories. The test systems are often waived,
which means there are minimal regulatory requirements, usually
only that the laboratory must follow the manufacturers instructions
for use. Nevertheless, a good quality system requires that method
performance be validated, which should include the definition of
the quality requirements for the intended clinical use of the test.
With the information on the quality requirement for the test and the
precision and accuracy observed for the method, the SQC procedure
should be designed to assure detection of medically important errors.
Next particular failure modes and events that require testing of
control materials should be identified to determine when controls
need to be analyzed (the basic safety net for catching errors).
The manufacturers recommended control mechanisms should be
reviewed, along with regulatory and accreditation requirements, to
define a QC Plan. The Plan should specify frequency of monitoring,
corrective action and recovery procedures, and reporting guidelines
that include information for safety.
For simple applications, method validation should include
experiments for reportable range, replication, comparison of meth-
ods, and verification of reference intervals. Data analysis may be
performed using an electronic spreadsheet or Internet tools, such as
the method validation calculators provided on the Westgard website
(www.westgard.com). Decisions on acceptable performance depend
on the definition of the quality required for the intended use, which
may be different in Point-of-Care applications and Physician Office
Laboratories. Calculation of a Sigma-metric helps the decision-
making process, as well as the selection of an appropriate SQC
procedure using the Sigma-Metric QC Selection tool from CLSI C24
[1]. Control charts can be prepared for manual plotting of control
measurements and should include a tabular record of any changes
to the process and all corrective actions. Other control mechanisms
recommended by the manufacturer should be integrated into the
QC Plan and documented in the control records. Operator training
should include training on all of the control mechanisms included
in the QC Plan.

Page 276
 Westgard QC, Inc. Copyright 2011

Index CLSI EP18 9, 34, 36, 45, 67, 113

123, 153, 186, 214, 278, 280
CLSI EP22 83, 132
CLSI EP23 9, 34, 45, 67, 97, 113-114,
A 124-132, 163, 183, 199, 213,
219, 280
Acceptability matrix, Cognitive error 139
see risk acceptablity matrix Competency evaluation 268269
AdvaMed 7 Continual improvement 138
Allowable total error (TEa) 21, 58, 222, 285 Control procedure
Analytical performance characteristics 52 definition 96
Analytical QC Strategy 19, 22 Corrective action 138
Analytical quality requirements 56 Corrective Actions and Preventive Actions
Analytic errors 143 (CAPA) 264
Analytic QC Plan 19, 21, 151, Criticality 31, 36, 117, 118, 153, 191
178, 191, 222- 226, 233, 259261 definition 117
Developing 213230 Criticality matrix 36, 117
Implementing 275281 Critical systematic error 240241
The plan, the plan, the plan, the plan! 270 Crossing the Quality Chasm 27
Analytic QC strategy 53, 283
Analytic system checks 130 D
Analytic testing process 180181
Default QC (DQC) 5
A safety net to catch analytical errors 5166
Defect rates 285, 287
Autoverification programs 280
converting ISO/CLSI ratings 247
B Delta checks 216
Deming 14
Block diagrams and Top-Down Flow- Design for Six Sigma 17
charts 166169 Detailed process flowchart 172173
Blood transfusion process 150 Detailed process outline 169
Brainstorming 33, 152, 168 Detection 33, 42, 43, 132, 153, 189
basics 181183 200, 214, 216220, 245, 246, 286
Brooks 99 estimating 233256
Developing Analytical QC Plans and Quality
C Systems 1924
Device risk mitigation features 132
Carey 131
Diagramming a Laboratory Process 152,163
Cassidy 284
174
Cembrowski 131
Disclosure 221, 262263
CLIA 1, 2, 7, 10, 18, 53, 124, 156, 157,
219, 261 E
QC possibilities 35
CLIA criteria for acceptable performance 21 Electronic QC 18
CLIA Final Rule 28 Eliminating or reducing occurrence 215216
clinical decision interval 21 Equivalent QC (EQC) 5, 28, 124, 219
CLSI 5, 9, 17, 28, 32 options 7
CLSI C24 10, 92, 95 Option 4 7
CLSI C24 Guidance for SQC Proce- Equivalent quality testing 5, 28
dures 101111 Error assessment 57
CLSI EP15 266 Error budgets 16

Page 291
 Six Sigma Risk Analysis

Estimating Detection 233256 Hazard 29, 69

Examination failure 86 definition 71
External Quality Assessment (EQA) 56, related to performance 8788
217, 267 Hazard analysis 31, 140
Hazard Identification 127
F Hazard matrix 117
Hazard score 117
Failure Mode and Effects Analysis
Healthcare FMEA (HFMEA) 117
(FMEA) 17, 27, 32-33, 114-115,
hemolysis 157, 197, 221, 235
143, 147, 148, 153-154, 189
HFEMA risk scoring matrix 117
200, 272
High risk methods 66
3-factor model 245
Hubbard 286
definition 115
example for Turnaround Time 194 I
example tables 250253
preparing a second FMEA 234 Identification and control of non-conformi-
Preparing a table 192 ties 138
Failure modes 33, 168, 189-200, 213, 270 Improving detection 216220
Identifying 177187 Inadequate volume 197
List of potential failures and specific Information for safety 9091
causes 120123 Inspection 16
Potential failure modes causing a delay in Instructions for use
reporting test results 182 definition 97
Failure Reporting, Analysis, and Corrective Integrating Six Sigma into Risk Analy-
Action Systems (FRACAS) 114, 263 sis 283289
definition 115 Intended purpose 71
False rejection rate 38 Intended use 71, 86
Fault Tree Analysis (FTA) 114, 155, 207 ISO 17, 34
212 ISO 14971 6, 28, 29, 34, 36, 67
definition 115 93, 129, 143, 213, 221
example FT for identifying root cause of General Guidance and Approach for Risk
delayed test analysis 211 Management 70
FDA 1, 6, 7 Informative annexes 6869
Fishbone diagram 33, 177187, 183186 ISO 15189 3, 9, 22, 25, 28, 36, 52, 67, 9
Five-Whys 155, 204, 206207 2, 137, 221, 262, 266
Flowcharts 15, 33 ISO 15198 6, 91, 9599, 233
basics 164165 Validation of QC Procedures 98100
Formulation of an Analytical Quality Control ISO 22367 6, 28, 35, 36, 153, 264, 269
Strategy 6166 ISO/TS 22367 131, 137145
Fraser 262 IVD failure 86
Frequency of controls 260
J
G
JC Risk Analysis Methodology 147161
Guidance to Manufacturers on Validating Joint Commission 27, 147, 233
User QC Procedures 96 rating scale for severity, occurrence, and
detection 154
H Juran 14
Harm 29

Page 292
 Westgard QC, Inc. Copyright 2011

K Pareto diagrams 15
Parvin 46, 99, 261
Koshy 132 Patient data QC 130, 242, 280
Krouwer 32, 207, 261 Patient Safety 27
Patient testing process 178179
L
Peer comparison programs 267, 280
Laboratory error 139 Performance specifications 53
Latent error 139 Periodic review of QC data 267
Lean 17, 159 Post-analytic errors 143
Lipemia 235 Power function graph 236-237
Lloyds 26 Powers 32, 35, 36, 45
Low risk methods 64 Pre-analytic errors 142
Prevention 214
M Preventive action 138
Prioritizing Failure Modes 168, 189200
Management Responsibilities 140 Proactive Risk Reduction 147
Manufacturers Risk Report 131 161, 163, 166, 221, 259
McDermott 45 Probability for error detection 236
Measurement uncertainty 262, 266 Probability for false rejection 236
Measuring Performance 259272 Process capability index, Cpk 39
Medically important systematic errors 240 Process redesign options 214
Method Decision 285 Proficiency Testing (PT) 217, 267
Moderate risk methods 65 Protective measures 90
Monitoring Failures 259272
Multistage QC 46 Q

N QC Design and Planning 15, 102105

QC Tools
Need for objective and quantitative estimates example assessment of feasibility of differ-
of risk 285287 ent tools 227
Need for scientific management of qual- Qualitative ranking 34
ity 284286 Qualitative risk applications 276277
Need for uniform standards of quality in labo- Quality Assessment (QA) 15
ratory testing 287289 quality by design 266
Non-cognitive error 139 Quality Compliance 2
Non-statistical QC procedures 53 Quality Control (QC)
definition 2, 101
O
event-driven 46
Occurrence 31, 33, 38, 39, 117, 128, 153, Quality Control Plan (QCP or QC Plan) 114,
189200, 216, 245, 246, 286 169, 216, 235
qualitative ranking 78 Quality Control rule (QC procedures) 53
semi-quantitative ranking 78 definition 101
Operating point 61 Quality Control Strategy (QC strategy)
OPSpecs chart 61, 285 definition 102
high error detection 62
P low error detection 63
moderate error detection 62
Pareto analysis 31 Sigma-metric relationship 63
definition 117 Quality Goals (QG) 15, 52, 5466

Page 293
 Six Sigma Risk Analysis

Quality Improvement (QI) 15 Risk management 17, 26-50

Quality Indicators 17, 269 comparison of different models 44
Quality Laboratory Processes (QLP) 15 Risk management 17
Quality Requirements Risk Management QC Plan (QCP) 5
Biologic variation 56 Risk matrix
European recommendations for biologic qualitative 79
goals 56 semi-quantitative 80
Medically important changes 57 Risk Mitigation 213230, 286
Stockholm Consensus Conference 54 Risk Monitoring 32
Quality triology 14 Risk Priority Number (RPN) 31, 33,
Quantitative risk applications 278279 118, 132, 153, 191, 248
Risk Reduction 32
R Risk Score 191
Robust Process Improvement 159
Recovery 214, 261
Root Cause Analysis (RCA) 155, 168, 183,
Redesigning the process 156157
203212, 215
Redesign options 156
differences between JC and CLSI 204205
Reducing severity 221222
Roubini 287, 288
Re-engineering 17
Reference change value 262 S
Residual risk 19, 30, 191, 199, 248
249, 254 Safety by design 8990, 156
definition 73 Selection of Statistical QC procedures 5966
evaluating 233256 Semi-quantitative ranking 34
Risk 28 Semi-quantitative risk applications 277278
definition 71 Sentinel event 151
Risk acceptability matrices 36, 82, 118, 143, Severity 29, 31,
191, 234, 255 33, 37, 117, 118, 128, 153, 189
comparison of ISO and CLSI 129 200, 245, 246, 286
qualitative 80 qualitative ranking 77
semi-quantitative 81, 144 reducing 221222
Risk analysis 30-31, 143 semi-quantitative ranking 78
comparison of JC and EP23 methods 168 Shadow testing 288
definition 72 Sigma-metric 40, 79, 223225, 240
integrating Six Sigma 283289 241, 245, 285
outline of integrated JC/CLSI/ISO ap- equation 58
proach 170172 priority of QC tools in relation to Sigma
Risk assessment 30-31, 35 performance 225
number of factors 34 QC recommendations 59
prioritization 35 Sigma-Metrics QC Selection Tool 60,
ranking scales 34 105109, 285
Risk control 30, 32, 168 Six Sigma 16, 17, 275
definition 73 Specific monitors or quality indicators
Risk control option analysis 155, 213 (list) 265
Risk estimation 31, 168 Stable sample controls 130
definition 72 Stamatis 32, 45, 236
Risk evaluation 30, 35, 36 State Operations Manual (SOM) 7
definition 72 Statistical QC 18
Risk information 45, 285 Statistical QC (SQC) 2, 5, 219

Page 294
 Westgard QC, Inc. Copyright 2011

220, 235, 261, 272, 285

definition 102
Surrogate QC 3
Surrogate sample 3

Tague 207
To Err is Human 27
Total Quality Management (TQM) 14
Total testing process 178
Trueness 10
Trueness controls 217, 266
Trueness controls 268
Truth in labeling 1
Turbid sample 197

Uncertainty 10
Use error
definition 71
Use errors 8687

Veterans Administration National Center for

Patient Safety 117

Wrong specimen type 197

Young 262

Page 295
 Westgard QC Order Form Westgard QC, 7614 Gray Fox Trail, Madison WI 53717
CALL 1-608-833-4718 if you wish to pay by purchase order or other means. Fax: 608-833-0640 or 203-389-6753

Item Price (US$) Quantity Subtotal

Westgard software and reference manuals

EZ Rules 3 QC Design software $695.00

NEW! Six Sigma Risk Analysis $90.00

(save $20 if you order before September 30th) $70.00

Six Sigma Quality Design & Control, 2nd Edition $90.00

Assuring the Right Quality Right (Advanced Quality Management) $90.00

Basic QC Practices manual, 3rd Edition (most popular) $80.00

Basic Method Validation, 3rd Edition $80.00

Basic Planning for Quality $60.00

CLIA Final Rules for Quality manual $60.00

Westgard Online Courses

Introduction to Risk Analysis for the Laboratory (4 credits) $155.00

Quality Management and Design of Analytical Systems (5 credits) $75.00

"Westgard Rules" and Levey-Jennings Charts mini-course (3 credits) $75.00

Basic QC Practices complete online course (14 credits) $135.00

Basic Method Validation complete online course (15 credits) $175.00

Grand Subtotal

Shipping & Handling: Within US = $9

Canada & Mexico = Add 10%; Europe, Asia, & all other countries = Add 20%

Sales Tax (Add 5.5% in WI, 6.0% in CT)

GRAND TOTAL

Save more when you order online! Visit http://www.westgard.com/store.htm

to see specials and coupons available only online

Your Name

Institution

Department

Street Address

City State Zip Code

Country

Business Phone/Business Fax

E-mail Address

Credit Card Type (circle one) VISA Mastercard American Express

Credit Card Number Exp. Date

Signature

**Westgard QC ships domestic orders via US Priority Mail with Delivery Confirmation.
For other shipping methods, please call. Fax this to 608-833-0640 or 203-389-6753.