Documente Academic
Documente Profesional
Documente Cultură
by Duke Okes
The Opportunity
It’s a given that not all audit nonconformities (NCs) are equal. Some indicate system weaknesses that
could create life ending outcomes (e.g., hygiene issues in the food or healthcare industries), while others
may simply be a minor documentation error (e.g., a class roster that was not signed by the course
instructor).
Recognizing the relative degree of risk related to each NC allows an organization to better allocate
resources where it makes more sense … another application of the Pareto principle. Even ISO 9001
indicates such when saying in the audit section 9.2.2.e) “take appropriate correction and corrective
action …” and in the nonconformity and corrective action section 10.2.1.b) “evaluate the need for action
…” and 10.2.1 “Corrective actions shall be appropriate to the effects …”
So corrective action might not be warranted for some NCs (e.g., correct the problem only), physical
causes level for others, and root cause level for still others. Risk ranking can also help determine the
relative timing allowed for corrective actions to be carried out as well as who should be involved in the
investigation.
Unfortunately, too many organizations have simply adopted the binary classification of NCs used by
most registrars, Major and Minor. While this is better than simply reporting it as a NC, it still does not
provide much information. Of course, organizations may add other classifications to deal with audit
findings where it is unclear whether the system needs work (Observation) or where there are
Opportunities for improvement, but these do not help in ranking actual NCs.
The introduction of the term “risk-based thinking” in the ISO standard goes beyond simply replacing the
preventive action requirements of the previous edition of the standard. It implies that throughout the
management system risks can/should be considered when making decisions. This also concurs with
senior management thinking that efforts should be placed where greater value can be achieved.
Some Examples
As part of courses on risk-based thinking and risk-based quality audits the author developed a simple NC
ranking system (see Table 1). Such an evaluation could consider risks to product/service quality,
customer satisfaction, regulatory compliance, and/or other objectives or stakeholders.
A Google search uncovered a similar four-level NC rating system used by the Finance Division at the
University of Scranton in Pennsylvania. The levels are Nominal, Notable, Significant and Major, with the
level at which the issues should be resolved ranging from the staff of the department where found to
involvement of Deans, and communications to the Board ranging from not at all to “in a timely manner.”
Seriousness considers the financial impact, whether it involves a violation of laws or regulations, fraud,
reputation and others.
Some of the terms in Table 1 are likely to be interpreted differently by each individual so in risk
management programs a risk appetite table is often used to define risk levels in more specific terms. An
example that could help auditors determine the appropriate level for each NC is shown in Table 2. It
requires a deeper evaluation of the potential impact of the NC on each specific objective.
A similar concept has been applied to external audits in some industries. In 2012 the Global
Harmonization Task Force released a guidance document (GHTF/SG3/N19:2012) describing a NC grading
system that is being adopted by the Medical Device Single Audit Program (MDSAP). It identifies
processes within ISO 13485 that are likely to have a direct impact on device safety and performance
(e.g., product realization) versus that that would have an indirect impact (e.g., documentation). It also
considers whether the NC is a first occurrence or a repeat of a NC found during recent previous audits.
It then uses a 2x2 matrix of Occurrence and Impact to score the NC as 1, 2, 3 or 4. Additions to the score
(called escalations) can also be made if the process is not adequately documented or if a nonconforming
Quality audit managers who want to develop their own process-risk-focused ranking system might want
to perform a risk assessment of processes in the QMS. Such an evaluation can also help prioritize other
elements of the audit plan (ISO 9001:2015 indicates that audits should “take into consideration the
importance of the processes concerned, …). Table 3 is a partial example, which demonstrates that for
this organization some processes inherently have greater risk, meaning that they not only should be
audited more frequently, but also that NCs in these processes have greater risk. As ISO 9001 points out,
organizational context has a big impact on risk (in this organization all raw materials were supplied by
the customer, which means Purchasing had little impact on quality performance). Such an assessment
would ideally be conducted with input from process owners.
TYPES OF RISK
QMS Process Delivery Regulatory
Product Function Performance Compliance Cost of Quality
Probability
Probability
Probability
Probability
Impact
Impact
Impact
Impact
Risk
Risk
Risk
Risk
Customer contracting/ordering 1 4 H 3 4 H 2 4 H 1 1 L
Product design 4 5 H 1 2 M 4 4 H 5 5 H
Process design 2 4 H 3 3 M 1 3 M 5 5 H
Purchasing 2 1 L 2 1 L 2 2 M 2 2 M
Order fulfillment
Calibration
Nonconforming material
Document control
Internal audit
…
Aggregating Risks
Of course, looking at single NCs may also cause the organization to not see the big picture. A way to
aggregate risks might then add additional value. This is often done by department or process (see
“Internal Audit Scorecards” in Okes, 2017), but a better way might be to show a matrix of all NCs found
during the audit and the degree to which each aligns to objectives (see Table 4). In this case rather than
using words a number is assigned to each level using a nonlinear scale of 1, 3, 7 and 10. This helps
better differentiate when the number of potential levels is low. Note that NC#s 2 & 4 are potentially
Objectives
Customer
NCs Product Quality Satisfaction Cost of Quality Total
1 1 1 1 3
2 3 6 10 19
3 1 3 3 7
4 6 6 1 13
5 3 1 3 7
6 1 3 1 5
Total 15 20 19 54
Another factor that could be considered for each NC is velocity. That is, if the potential impact on the
objective turns into an actual impact, how long is it likely to take for it to show up? For example, will the
impact on product functional performance show up at a final test station? Will it only show up after the
customer purchases the car? Or will it be years before the degradation is evident?
Cautions
Internal financial auditors (often called GRC auditors – Governance, Risk & Compliance) have typically
rated either the entire audit or each specific finding according to risk. Richard Chambers of the Institute
of Internal Auditors (IIA, 2017) indicates that this helps draw the attention of the board, as well as that
of executive management. However, he also cautions about potential downsides:
Process owners may take the ratings personally, especially if their performance reviews are
impacted by audit ratings
The ratings add time to generating the audit report
Ratings can draw attention to some factors, with other factors perhaps being ignored or
downplayed
Given that there are likely to be GRC, environmental, occupational health & safety, IT and other audits
conducted in the organization, it would then be useful for quality auditors to consider how NCs are
handled in these audits. After all, a fully integrated management system would include a fully
integrated audit function. But if this integration has not already occurred, each group should at least be
studying/benchmarking the others.
Summary
ISO 31000:2018 defined risk as the “effect of uncertainty on objectives.” When an audit is conducted it
is evaluating the controls that have been put in place to reduce risk, and if those controls are not
The process of risk ranking and reporting NCs also needs to have its own controls, and it is hoped that
the ideas presented herein will be useful in helping the reader evaluate the process at his/her own
organization, as well as suppliers, customers and/or clients.
References
Bio
Duke Okes has been in private practice for 34 years as a trainer, consultant, writer and speaker on
quality management topics. His book titled “Musings on Internal Quality Audits: Having a Greater
Impact” was published by ASQ Quality Press in 2017. He is an ASQ Fellow and holds certifications as a
CMQ/OE, CQE and CQA.