Quality Assurance in Education

A critical evaluation of academic internal audit

Jacqueline Ann Blackmore

Article information:
To cite this document:
Jacqueline Ann Blackmore, (2004),"A critical evaluation of academic internal audit", Quality Assurance in Education, Vol. 12 Iss
3 pp. 128 - 135
Permanent link to this document:
http://dx.doi.org/10.1108/09684880410548753
Downloaded on: 09 March 2015, At: 18:56 (PT)
References: this document contains references to 32 other documents.
To copy this document: permissions@emeraldinsight.com
The fulltext of this document has been downloaded 1911 times since 2006*

Users who downloaded this article also downloaded:

Dominic S.B. Soh, Nonna Martinov-Bennie, (2011),"The internal audit function: Perceptions of internal audit roles, effectiveness
and evaluation", Managerial Auditing Journal, Vol. 26 Iss 7 pp. 605-622 http://dx.doi.org/10.1108/02686901111151332
Faudziah Hanim Fadzil, Hasnah Haron, Muhamad Jantan, (2005),"Internal auditing practices and internal control system",
Managerial Auditing Journal, Vol. 20 Iss 8 pp. 844-866 http://dx.doi.org/10.1108/02686900510619683
Jenny Stewart, Nava Subramaniam, (2010),"Internal audit independence and objectivity: emerging research opportunities",
Managerial Auditing Journal, Vol. 25 Iss 4 pp. 328-360 http://dx.doi.org/10.1108/02686901011034162

Access to this document was granted through an Emerald subscription provided by 273599 []

For Authors
If you would like to write for this, or any other Emerald publication, then please use our Emerald for Authors service
information about how to choose which publication to write for and submission guidelines are available for all. Please visit
www.emeraldinsight.com/authors for more information.

About Emerald www.emeraldinsight.com

Emerald is a global publisher linking research and practice to the benefit of society. The company manages a portfolio of
more than 290 journals and over 2,350 books and book series volumes, as well as providing an extensive range of online
products and additional customer resources and services.
Emerald is both COUNTER 4 and TRANSFER compliant. The organization is a partner of the Committee on Publication Ethics
(COPE) and also works with Portico and the LOCKSS initiative for digital archive preservation.
*Related content and download information correct at time of download.

Introduction

A critical evaluation of
academic internal audit
Jacqueline Ann Blackmore

The author
Jacqueline Ann Blackmore is Faculty Quality Officer at
Liverpool John Moores University, Liverpool, UK.

Keywords
Internal auditing, Higher education, Standards

Abstract
This account of internal audit is set within the context of higher
education in the UK and a fictitiously named Riverbank
University. The study evaluates the recent introduction of
Internal Academic Audit to the University and compares the
process with that of the internationally recognised ISO 19011
Guidelines for Auditing Quality Management Systems, used both
in the private and public sectors. A thorough review of the
literature on audit theory was conducted in order to gauge best
and worst practices. The aim was to identify any theory-practice
gaps between the British Standard guidelines and actual internal
academic audit performance, as well as any possible reasons for
them. Finally, it is suggested that when compared with the
British Standards model the internal academic audit process is
somewhat less robust, particularly in terms of auditor selection
and training.

Electronic access
The Emerald Research Register for this journal is
available at
www.emeraldinsight.com/researchregister
The current issue and full text archive of this journal is
available at
www.emeraldinsight.com/0968-4883.htm

Quality Assurance in Education

Volume 12 Number 3 2004 pp. 128-135
q Emerald Group Publishing Limited ISSN 0968-4883
DOI 10.1108/09684880410548753

The Quality Assurance Agency (QAA) was

established in 1997 as an independent body
funded by subscription from Higher Education
Institutions (HEIs) and through the contracts with
the main HE funding bodies including the Higher
Education Funding Council for England
(HEFCE). Until recently, the QAA assessed the
quality and standards of all 180 HE institutions in
the UK at two levels. First, Institutional Audit,
which concerned an institutions effectiveness at
managing the academic standards of its awards
and the quality of its programmes. Second,
Universal Subject Reviews, which assessed the
performance of academic programmes.
Between 1993 (when they were carried out by
HEFCE) and 2001 over 2,500 reviews were
carried out in more than 60 subjects at an
estimated cost to all HEIs of between 45m and
50m per year (Better Regulation Task Force,
2002). This massive cost caused a number of
complaints from Universities, particularly those
who did not rely on HEFCE for the majority of
their funding, and a decision was taken to reduce
the cost burden on Universities by combining the
two. Thus, in January 2002, a revised method of
Academic Review was introduced which includes
Institutional Audits. Part of this deal was that
HEIs publish information on quality and
standards of teaching and learning (with a deadline
of December 2004 at the latest) on a national
Teaching Quality Information (TQI) Web-site
available from March 2004 (HEFCE, 2003). In
order to facilitate an informed choice of
University, the Website will be available for use by
interested stakeholders including, potential
students, parents, employers and potential
employees.
Of particular importance to this paper is an
additional requirement by HEFCE for HEIs to
undertake their own periodic reviews of
departments periodic being defined as at least
once every six years. They are then required to
produce summaries of such reviews for publication
on the TQI Website, together with the actions
taken in response to any needs identified as a
result. Internal academic audit was introduced
within the case study University to satisfy this
criterion as well as acting as a preparatory exercise
for its next Institutional Audit.
The aim of this paper is to determine the
robustness of internal academic audit within HE,
when compared with the British Standard
guidelines on auditing (BS EN ISO 19011, 2002)
These comments were assembled from the responses
from the members of the review team and represent
their consensus view.

128

A critical evaluation of academic internal audit

Quality Assurance in Education

Jacqueline Ann Blackmore

Volume 12 Number 3 2004 128-135

(BSI, 2002). This aim will be achieved by

comparing the guidelines on auditing a quality
management system with the approach to internal
academic audit within the case study University.

registration and by 2002 over 550,000 certificates

worldwide had been issued (Farmer, 2003).
Indeed, certification seems to have become a
business passport for small to medium
enterprises, (Yamada et al., 2003) apparently,
because of external pressures, particularly from
customers who wanted some sort of assurance that
they were dealing with a quality organisation.
ISO 9000 is based on the premise that the
creation of products and services is the result of a
system with inputs and outputs that can be
measured at their various stages of adding value to
the system as opposed to an inspection-based
philosophy that comes too late at the end of the
process. The focus is on processes and the
documentation produced on the various stages of
these processes, e.g. the quality policy and
procedures manuals, not solely on the quality of
the end products, and is more in line with
Demings Plan, Do, Study, Act model of
continuous improvement (Deming, 1986).
Although originally intended for the
manufacturing industry the Standards were
identified as being suitable for organisations
delivering services including education (Shutler
and Crawford, 1998) although some adaptations
were required. In 1994 the Standards were revised
to make them more user friendly for service
industries, and by the year 2000 a major revision
was completed creating the current generic series
of Standards. The emphasis of this major revision
was less on compliance and more on continual
improvement and greater customer focus (Farmer,
2003). Zuckerman (1999) believed that the
Standards in any case were similar to other
national quality standards, such as the Malcolm
Baldrige National Quality Award (MBNQA) in
the US and the European Foundation for Quality
Management (EFQM) Excellence Award, with
more emphasis on continuous improvement and
processes, rather than outcomes.
There were a number of criticisms of the
Standards prior to the 2000 revision, not least of
all that it created a mountain of paperwork, and
was itself based on an inspection of procedures
(therefore not real quality). In addition to being
costly it needed to be interpreted first by a quality
consultant then by external auditors (Seddon,
1997). Also, post-2000, although much improved,
with the emphasis on the auditing of processes
rather than procedures, there are still some
criticisms that the Standards have not gone far
enough along the Total Quality Management
(TQM) path. According to Poksinska (2003) the
Standards are still too general, needing to be
interpreted, and according to Farmer (2003) still
create too much work. However, a number of
benefits have also been cited within the plethora of

Quality management systems: history and

evolution
Pre-1940s the method of ensuring delivery of
quality products to customers was based on what
was known as quality control, i.e. inspection took
place at the end of, not during the process. This
was costly in terms of both money and time. At the
end of the 1940s the Department of Defence in the
US recognised the benefits of a quality assurance
system that had already transformed the Japanese
manufacturing industry. This system involved
establishing procedures to manage all the
functions that affected the quality of the
manufactured products within an organisation;
thereby, building quality into the process itself
(BSI, n.d.).
In 1979, the British Standards Institute (BSI)
published its first edition of the BS 5750 series of
quality assurance standards. These were intended
for general use by any manufacturer and enabled
organisations to become certified and so advertise
the mark of registration in order to gain a
competitive edge. In 1987 the International
Organisation for Standardisation based the ISO
9000 family of standards on BS 5750. The ISO
9000 standards were adopted across what was then
the European Community. These standards are
now known as the BS EN ISO 9000 series of
standards for simplification called ISO 9000 in
the rest of this paper. The ISO 9000 series consists
of three core Standards, namely, ISO 9000:2000
Quality Management Systems Fundamentals and
Vocabulary (BSI, 2000a), ISO 9001:2000
Quality Management Systems Requirements (BSI,
2000b) and ISO 9004:2000 Quality
Management Systems Guidance (BSI, 2000c) and
a series of supporting Standards including ISO
19011:2002 Guidelines for Quality and
Environmental Management Systems Auditing
(BSI, 2002). A series of guidance notes have also
been produced by BSI, for example, Guidance
Notes on the Application of ISO 9001 for Quality
Management Systems in Education and Training
(BSI, 1999) and these help sector specific
organisations to understand the requirement of the
Standards.
Within the field of study concerning quality
management systems, the ISO 9000 series is
recognised as the currently established, global
standard (Beckford, 1998). Since 1987
thousands of businesses worldwide have sought

129

A critical evaluation of academic internal audit

Quality Assurance in Education

Jacqueline Ann Blackmore

Volume 12 Number 3 2004 128-135

literature in this area (see for example, Dale,

1999). Stevenson and Barnes (2001) argued that
aligning an organisations quality systems to the
ISO 9000 Standard, not necessarily seeking
certification, would contribute to the systematic
improvement of overall quality and profit. Due to
the permitted three-year changeover to the 2000
revision of the Standard from the 1994 version,
granted to all certified organisations, which ended
in December 2003, there is a lack of both
practitioner and researcher information on the
effects of the new Standard. Only time will tell if its
degree of impact on quality will be any greater than
previous iterations of the Standard.

example, Beeler (2003) argues that audits cannot

themselves drive continuous improvement, but
Peters (1998) believes that dynamic and
proactive auditing can indeed induce
improvement, whereas static auditing will only
serve to ensure compliance to a minimum
standard, and therefore improvement will not
necessarily be forthcoming as a result. Audit, if not
used with care however, can have negative effects
on those involved (Douglas et al., 2003) for
example the creation of raised stress levels among
individuals.
Auditing is seen as a tool to aid effective
decision-making as well as a source of information
for managers (Chan et al., 1993). The purpose of
quality audit is to determine whether quality
assurance activities and related results comply with
the planned arrangements, i.e. that the company is
doing what it says it is doing, and whether these
arrangements are implemented effectively and are
suitable to achieve objectives (fitness for purpose)
(BSI, 2002). Rice (1994) recognised that quality
auditors themselves are not usually the most
welcomed of colleagues. Therefore, how they
conduct the audit (based on their personal
characteristics and training) is of great importance.
Rice (1994) identified a number of skills that
quality auditors should employ before, during and
after an audit, including effective communication,
sound preparation, empathy, the use of objective
criteria, to focus on the system and its problems
and not on individuals, to consult before making
decisions, and the ability to build working
relationships.
More recently Beeler (2003) emphasised that
audit should provide information about three
aspects of systems, adequacy, implementation and
effectiveness, and that traditional compliance
audits did not meet this requirement. He also
challenged the requirement that auditors should
focus on facts, suggesting that although facts are
important the focus of the audit should be on
systems in order to provide valid feedback on the
adequacy, implementation and effectiveness of the
systems. This is the information that management
require.

The role of audit in quality assurance

One of the principal clauses in ISO 9001: 1994
(BSI, 1994) was concerned with internal quality
audits. It provided a focus on a number of aspects
of audit including: planning, auditor responsibility
and independence, procedures that ensure that the
audit process is clear, compliance with the
documented system, reporting of discrepancies
and results to those being audited, assessment of
the effectiveness of action taken, and review and
implementation of corrective action by
management (Dale, 1999). It explicitly stated that
the aim of audit was to determine the
effectiveness of the quality system. The latest
version (BSI 2000 series) still emphasises the need
for an audit programme to be planned, that the
auditors selected should ensure objectivity, that
auditors should not audit their own work and that
they should be trained.
According to Karapetrovic and Willborn (2001)
quality audit measures the effectiveness (fitness of
purpose) and achieved improvement (fitness for
purpose) of an organisations quality system
against the requirements of a selected model.
It is implicit within the quality literature that
evaluating performance and implementing
improvements are essential to organisational
success, (see for example Crosby (1979) and Juran
(1988)). However, without some form of objective
measurement an organisation will be unable to
determine if it has improved or not. BS EN ISO
19011:2002 is an International (and British)
Standard that provides a set of guidelines for the
effective audit of quality management systems. It
defines audit as a systematic and independent
examination, with an evidence-based approach
(BSI, 2002).
Quality audit is meant to facilitate and support
improvement (Karapetrovic and Willborn, 2001)
by, inter alia, identifying any shortfalls and/or
effective practice for sharing. However, there is a
debate within the literature as to whether quality
audit can actually cause improvement. For

The British standards guidelines for auditing

Among the competencies required for auditors of
quality management systems, the British Standard
for Auditing (BS 19011:2002) (BSI, 2002) lists
that they should be ethical, open-minded,
diplomatic and perceptive.
Training of internal auditors generally involves a
one-day course concerning the scope and the
objectives of the audit, identifying nonconformance and the degree of non-conformance,
recommended actions, and generating feedback.

130

A critical evaluation of academic internal audit

Quality Assurance in Education

Jacqueline Ann Blackmore

Volume 12 Number 3 2004 128-135

Summary of auditing best practice

Based on the British Standard for Quality
Management Systems (ISO 9001:2000) and the
Guidelines for Quality Auditing (ISO
19011:2002) as well as the received wisdom from a
review of the research literature on the subject, a
number of best practice criteria have been
identified:
(1) Auditors should be independent of the area
being audited.
(2) Auditors should receive appropriate training.
(3) Not everyone is suitable to be an auditor.
(4) To maximise the benefits of auditing,
processes as well as outcomes should be
audited.
(5) Audit should be based on facts, i.e. objective
evidence.
(6) The focus should be on systems and
processes, not people.
(7) For an audit to contribute to continuous
improvement it has to go beyond compliance.
(8) The criteria against which the audit is being
conducted should be clearly indicated.

funding of higher education should be developed.

Dearings view seems to have been drawn from the
business (private) sector in line with the
Conservative philosophy that private was best.
One of the main outcomes of the Dearing Report
(HMSO, 1997) was the establishing of the QAA to
review academic quality standards within
Universities. The term review is used in much of
the QAA documentation. However, their system of
review is based on audit. In the past these were
akin to the old ISO 9001:1994 audits (BSI, 1994),
i.e. an inspection focused on outputs and
procedures. Changes in the QAAs methodology
have been much publicised in recent years and the
move to academic review which includes an
Institutional Audit with some Discipline Audit
Trails is supposed to be less inspection oriented,
(Jackson, 2003) than in the past, with the old style
subject reviews.

Quality assurance in higher education

institutions
It was not until the mid-1980s that significant
formal attention was paid to the quality of degrees
and other programmes of study at Universities.
Until then it was generally assumed that excellence
was guaranteed because of the external examiner
system introduced to ensure comparability within
and between University programmes. As the
Universities largely determined their own student
entry levels and therefore their graduate rates, they
were able to remain complacent and confident that
there would be no externally organised scrutiny of
their standards.
Since World War II higher education in the UK
has changed radically with the expansion of
tertiary education worldwide, leading to a demand
for more public accountability as well as less
unconditional trust in the HEIs.
The Higher Education Funding Council for
England (HEFCE) became responsible under the
Further and Higher Education Act 1992 for
assessing the quality of higher education.
Previously, an Academic Audit Unit had been
established by Universities themselves in order to
monitor quality. Its methods were based on the
model of financial audit and monitored and
commented on the structures of HEIs and their
own internal quality mechanisms.
In 1997 a review of higher education in the UK
was undertaken by the Dearing Committee,
commissioned in 1996, originally by John Majors
second Conservative Administration with
bipartisan support, to make recommendations on
how the purposes, shape, structure, size and

The role of audit in quality assurance in

higher education institutions
The use of ISO 9000 certification in higher
education represents a challenge, not least in
determining what is the product of higher
education. For example, according to Shutler and
Crawford (1998) the actual learning of the student
is the product, but it could be argued that the
product is the actual provision of learning
opportunities. There is little evidence of literature
on the application of ISO 9000 within the higher
education sector generally, perhaps due in part to
the challenge of identifying the product of
education, but more likely due to the requirements
already imposed by its external funding body,
HEFCE, via the QAA. Dancing to one tune is
probably enough for most institutions. In 1995
(and revised in 1999), the British Standards
Institute produced guidance notes on the
application of the standards for education, based
on the assumption that the product of an
education institution is the enhancement of
competence, knowledge, understanding or
personal development of the student resulting
from the learning experience, and that quality
management in this provision must also include
administrative and supportive services which
contribute to the effectiveness of the institution as
a whole and should be considered part of the
service (BSI, 1999). A small number of
educational institutions have achieved certification
to the Standard (Lundquist, 1997 in Shutler and
Crawford, 1998), but for the most part ISO 9000
has been ignored by HEIs.
HE audits are usually conducted against the
requirements of an overseeing body such as the
QAA and are generally based on the submission of
evidence by the auditee (The HEI) to the

131

A critical evaluation of academic internal audit

Quality Assurance in Education

Jacqueline Ann Blackmore

Volume 12 Number 3 2004 128-135

auditor (QAA). Such submissions vary greatly in

sophistication from the basic unresponsive
through descriptive and responsive to the
more sophisticated evaluative submissions and
not ignoring those that were deceptive (Carroll,
2003). This is often the typology of submissions
for internal audits also and these are discussed in
the next section.

part of the timetable for preparation and planning

for the Institutional Audit by the Quality
Assurance Agency programmed for autumn 2004.
Each of the six faculties was scheduled for audit.
The purpose of this audit process had three
strands, explicitly:
(1) To seek to ensure that with the devolution of
the quality assurance processes to Faculty
Quality Committees there is clarity of
responsibility regarding documentation and
the storage of key items of documents.
(2) To measure compliance with HEFCE
guidelines on quality and standards (HEFCE,
2003) where the Institution is required to have
available summaries of reports of internal
reviews by the departments.
(3) To get the house in order before the external
audit by the QAA in 2004.

Internal quality audit in HE

Audit itself is certainly not new to education,
particularly external audit by the QAA at
institutional and subject levels, as discussed
previously. Some Universities have yet to
experience first-hand what the changes in
methodology mean for their organisations when
they undergo Institutional Audit in the coming
years. In the same vein, the application of internal
quality audit processes within higher education is
not a new phenomenon, but rather has been
considered since the pre-QAA period (Reid and
Ashelby, 2002). The policy of one such audit
undertaken in Swansea Institute of Higher
Education (Reid and Ashelby, 2002) was that the
auditors should be both appropriately trained and
should not have any direct involvement with the
area being audited, in line with the British
Standard for auditing. The purpose of this audit
was to exchange good practice, to promote
understanding and ownership of quality and
standards, and to aid enhancement. The selection
of auditors was based on set criteria and included,
inter alia that they be independent, impartial and
good listeners. Training of such audit panels
included audit methodology, triangulation of
evidence, and confidentiality issues. Such auditors
would act as mentors to future teams. It is clear
from Reid and Ashelbys (2002) report that the
focus of the audit was on processes, not outcomes,
again in line with the BSI guidelines.

Case study Riverbank University

A fictitiously named Riverbank University has
approximately 20,000 full-time equivalent
students and 700 full-time equivalent academic
staff supported by 550 administrative staff and 196
technical staff. The structure is based around six
faculties each containing a number of different
schools, for example the Faculty of Business
contains five schools, Law, Languages,
Management, Finance and Business Information.
The University acted on the QAA requirement for
periodic review of departments via the
introduction of Internal Academic Audit for the
academic cycle 2003/2004. This was one of the
Quality Support Departments key objectives as

The audit focused on the availability at faculty,

school and programme level of designated pieces
of information relating to the internal procedures
for assuring academic quality and standards.

Internal academic auditors

Auditing panels were drawn from each of the
faculties and could consist of both academic and
administrative staff. Faculties were asked to
nominate three appropriate panel members.
However, the definition of appropriate was not
provided. There was no consistency in how the
panel members were chosen in each of the areas
but they appeared to be drawn mainly from the
teaching staff, with one exception. When the six
panels were drawn up each included one
representative from the faculty being audited. This
was clarified to the nominees at the audit training
session as a way of aiding transparency. It soon
became apparent that not all auditors were entirely
independent with one panel including a Faculty
Quality Officer (FQO) who would audit her own
area. However, to ensure some consistency
throughout the various audits, it was decided that
the same person would chair each audit panel, i.e.
assume the role of Lead Auditor.

Auditor training
A two-hour training session was scheduled for the
selected panel members. At this session each
received a copy of the Process Guide for
Participants an internal Riverbank University
document. The role of the faculty member was
discussed in more depth and it was clarified that
their role was one of critical friend. At the
training session it was emphasised that the focus of
the audit would be solely on outcomes, not inputs
and processes.

132

A critical evaluation of academic internal audit

Quality Assurance in Education

Jacqueline Ann Blackmore

Volume 12 Number 3 2004 128-135

Audit events
A series of six audits was scheduled, one per
faculty, commencing December 2003 and
continuing into January 2004. A three-day audit
process was conducted as follows: Day one The
audit team met one another for the first time to
formulate the agenda and proposed programme
for the audit and reviewed the evidence provided
by each faculty. The evidence included, inter alia,
minutes of meetings; external examiner reports;
faculty and school policies on appointing and
responding to external examiners, teaching and
learning, and peer observation schemes; list of
programmes and external examiners; and an
example of programme self assessment documents
(annual reports from programme leaders) for each
school. Procedures were not asked for.
Based on their consideration of such evidence,
the audit team determined what additional
documentation they wished to see and which
personnel they wanted to meet on days two and
three of the audit.
In the lead-up to day one, the faculty quality
officers within each faculty collated the
documentation required for their submission. This
culminated in the submission of three box files of
paperwork containing examples of programme
annual reports, all external examiner reports,
minutes of various committee meetings, a
statement on the process of appointment of
external examiners, and responses to external
examiner reports, a statement from each school on
any peer observation schemes, and lists of current
programmes and external examiners. Each faculty
was also invited to provide a statement on any
areas of good practice they wanted to disseminate
to other faculties. However, this was not an
absolute requirement.
Some of the processes, although in operation,
had to be formalised so that they could be
submitted as evidence, for example, the school
processes on responding to external examiner
reports. This enquiry regarding processes was out
of alignment with the originally stated scope of the
audit, which was to examine outcomes only.
The list of programmes required an indication
of whether a definitive programme document
was available for scrutiny if required. This resulted
in various activities across one particular faculty,
ranging from one school arranging for programme
leaders to meet with the FQO and indicate by use
of a programme box checklist that all the necessary
information was available to another school
making their own, less public, arrangements. From
another school, one colleague enquired as to
whether the University would be open at the
weekend so that she could finish writing the

definitive document for a recently approved

programme.
In the limited time available for all parties
concerned, and a nave reliance on email
communication, i.e. if an instruction has been
emailed it will have been read and understood,
some confusion as to the exact requirements arose
and the final submissions from some of the
faculties contained some gaps. Thus, typically,
audit submissions were a combination of
unresponsive (for whatever reason), descriptive,
responsive and evaluative as per Carrolls (2003)
typography, with deceptive missing or at least
undetected. The full implications of this variation
in submissions are yet to be realised, although
faculties were given a second chance to supply
missing information and documentation in time
for days two and three.
Days two and three, which occurred in early
January 2004, focused on reading the additional
documentation requested and meeting with
faculty and school personnel. The aim was to
follow up in more detail, issues or queries raised by
the desk-top analysis of the submitted
documentation. This follow-up was divided into
four quality related themes: Learning and teaching
including staff development and peer review;
quality systems including all aspects of programme
management and evaluation; all aspects of the
external examiner process including appointment,
induction, and the management of reports; and
finally, student input, including student
attendance at various committees and student
feedback at module, programme and institutional
level. Each faculty nominated relevant personnel
to attend the appropriate meeting to discuss those
issues and answer any questions regarding their
submissions.
Conclusions and recommendations were fed
back to staff following day three at a closing
meeting. A written report of findings was
prepared for each faculty and within two months of
receiving the report containing the outcomes, the
faculties will have to submit an action plan to the
quality support team and have any corrective
action completed within twelve months of the
audit.

Conclusions
Earlier in this paper, it was stated that the role of
quality audit was to measure the achieved
improvements as well as the effectiveness of the
quality system. There is limited evidence to
suggest that some improvements had been
identified within the case study University during
the internal audit process. For example, schools

133

A critical evaluation of academic internal audit

Quality Assurance in Education

Jacqueline Ann Blackmore

Volume 12 Number 3 2004 128-135

were compelled to formalise procedures on

responding to external examiners and on their peer
observation schemes. The main focus of the audit
seems to have been on the effectiveness of the
quality system by checking that schools and
faculties did what they said they did, i.e.
compliance to the minimum standard. However,
as suggested by Beeler (2003) this falls far short of
the information required by management. Where
non-compliances had been identified
recommendations for corrective action were made.
Experience of the Internal Academic Audit
certainly concurred with previous views on
working to Standards, such as the creation of
paperwork (three box files for each audit panel)
and creating work for people, for example the
formalising of some of the schemes already in
operation in order to provide evidence.
One of the aims of the audit was to determine
whether committees were working within the
terms of reference set by the University, this is
clearly in keeping with ISO 19011 (BSI, 2002)
which defined the purpose of quality audit as
ensuring that activities and results comply with the
planned arrangements.
However, in terms of the procedures for the
selection, training and criteria for auditors, the
audit is somewhat less robust than those outlined
by the British Standard for audit, which gives clear
guidance on the essential characteristics of an
auditor. For example, ISO 19011 defines audit as
independent, yet each panel consisted of a faculty
representative and some were directly involved in
the areas being audited.
It is too early to tell if the Internal Academic
Audit process is going to be classed as Peters
(1998) dynamic and proactive audit and so
induce any improvement given that the outcomes
of days two and three of each audit have not yet
been published. However, early indications show
that in terms of a tool for checking whether the
University is ready for a QAA engagement, i.e. is
the house in order, then Internal Academic Audit
may well be a useful one. However, its impact on
the experience of some of those involved seems to
have been a negative one. Many of those under
scrutiny were in a state of near panic, despite the
fact that people were not the focus of the audit.
This probably says more about the culture in the
University that it does about the conduct of the
audit. Moreover, when mapped against audit
per se, clearly, the process is barely comparable,
particularly in terms of reviewing systems and
processes, which was not the focus of the Internal
Academic Audit. Rather, the focus was on
outcomes. How those outcomes were achieved was
not considered relevant but they were objective
evidence as far as the audit was concerned. Again,

this concurs with Beeler (2003) and confirms the

limitations of focussing on facts instead of systems.
In quality evolutionary terms, Internal Academic
Quality Audit is closer to inspection and quality
control than it is to quality assurance and TQM,
despite reassurances from the QAA that they are
more interested in auditing processes than
outcomes. Clearly, as far as HEIs and their
quality people are concerned old habits die
hard. Attitudes and methods will have to change
if institutions are to achieve any value-added
return for the amount of resources they are still
putting in to conducting internal audits or it will be
a case of back to normal after the auditors have
gone.

References
Beckford, J. (1998), Critical Introduction, Routledge, London.
Beeler, D.L. (2003), Internal auditing: the big lies, Quality
Progress, Vol. 32 No. 5, pp. 73-8.
Better Regulation Task Force (2002), Higher Education: Easing
the Burden, The Cabinet Office and Publicity Team,
London.
BSI (1994), BS EN ISO 9001: Quality Systems: Model for Quality
Assurance in Design, Development, Production,
Installation and Servicing, BSI, London.
BSI (1999), Education and Training: Guidance Notes on the
Application of ISO 9001 for Quality Management Systems
in Education and Training, BSI, London.
BSI (2000a), BS EN ISO 9000: Quality Management Systems
Fundamentals and Vocabulary, BSI, London.
BSI (2000b), BS EN ISO 9001: Quality Management Systems
Requirements, BSI, London.
BSI (2000c), BS EN ISO 9004: Quality Management Systems
Guidelines for Performance Improvements, BSI, London.
BSI (2002), EN ISO 19011:2002: Guidelines for Quality and/or
Environmental Management Systems Auditing, BSI,
London.
BSI (n.d.), available at: www.BSI-global.com (accessed
1 March 2004).
Carroll, M. (2003), Does auditing HE against standards
encourage masterpieces or paint-by-numbers?,
Assessment & Evaluation in Higher Education, Vol. 28
No. 3, pp. 297-307.
Chan, F.Y., Willborn, W., Xiao, H. and Li, H. (1993), An expert
system for quality management auditing, Asia Pacific
Journal of Quality Management, Vol. 2 No. 1, pp. 65-72.
Crosby, P.B. (1979), Quality Is Free, McGraw-Hill, New York, NY.
Dale, B.G. (1999), Managing Quality, 3rd ed., Blackwell, Oxford.
Deming, W.E. (1986), Out of the Crisis, MIT Press, Cambridge,
MA.
Douglas, A., Coleman, S. and Oddy, R. (2003), The case for ISO
9000, TQM Magazine, Vol. 15 No. 5, pp. 316-24.
Farmer, F. (2003), ISO 9001:2000, to deadline and beyond,
Quality World, December, pp. 10-12.
HEFCE (2003), HEFCE Information on Quality and Standards in
Higher Education: Final Guidance, HEFCE Publications,
Bristol.
HMSO (1997), Higher education in the learning environment,
Report of the National Committee of Inquiry into Higher
Education, HMSO, London.

134

A critical evaluation of academic internal audit

Quality Assurance in Education

Jacqueline Ann Blackmore

Volume 12 Number 3 2004 128-135

Jackson, S. (2003), The reviews group, interview with Steve

Jackson, Higher Quality Bulletin, QAA for HE Lancaster.
Juran, J.M. (1988), Jurans Quality Control Handbook, 4th ed.,
McGraw-Hill, New York, NY.
Karepetrovic, S. and Willborn, W. (2001), Audit and selfassessment in quality management: comparison and
compatibility, Managerial Auditing Journal, Vol. 16 No. 6,
pp. 366-77.
Lundquist, R. (1997) Quality in higher education: approaches to
its management and improvement, Licentiate thesis,
Division of Quality Technology and Statistics, Lulea
University of Technology, Sweden,.
Peters, J. (1998), Some thoughts on auditing, TQM Magazine,
Vol. 10 No. 1, pp. 4-5.
Poksinska, B. (2003), ISO 9000:2000: the emperors new
clothes, Proceedings of the 6th International Conference
of Quality Management and Organisational Development,
Paris.
Reid, K. and Ashelby, D. (2002), The Swansea internal quality
audit processes: a case study, Quality Assurance in
Education, Vol. 10 No. 4, pp. 237-46.
Rice, C.M. (1994), How to conduct an internal quality audit and
still have friends, Quality Progress, June, pp. 39-41.
Seddon, J. (1997), Ten arguments against ISO 9000, Managing
Service Quality, Vol. 7 No. 4, pp. 162-8.
Shutler, P.M.E. and Crawford, L.E.D. (1998), The challenge of
ISO 9000 certification in higher education, Quality
Assurance in Education, Vol. 6 No. 3, pp. 152-61.

Stevenson, T.H. and Barnes, F.C. (2001), Fourteen years of ISO

9000: impact, criticisms, costs, and benefits, Business
Horizons, Vol. 44 No. 3, pp. 45-57.
Yamada, S., Chiba, M. and Seguchi, H. (2003), Implementation
of ISO 9000 in medium- and small-sized enterprises,
Proceedings of the 6th International Conference on
Quality Management Organisational Development, Paris.
Zuckerman, A. (1999), ISO 9000 revision might miss year-2000
deadline, Purchasing, 8 April, p. 35.

Appendix. Glossary
FQO
HE
HEFCE
HEI
QAA
TQI
UK

135

Faculty Quality Officer

Higher Education
Higher Education Funding Council
for England
Higher Education Institution
Quality Assurance Agency
Teaching Quality Information
United Kingdom

This article has been cited by:

1. Ellen J. Dumond, Thomas W. Johnson. 2013. Managing university business educational quality: ISO or AACSB?. Quality
Assurance in Education 21:2, 127-144. [Abstract] [Full Text] [PDF]
2. Aura emien. 2011. Vidaus audito taka studij kokybei. Verslo ir teiss aktualijos 6, 367-387. [CrossRef]
3. Jeanne Keay, Christine Lloyd. 2009. Highquality professional development in physical education: the role of a subject association.
Professional Development in Education 35, 655-676. [CrossRef]