Available online at www.sciencedirect.

com

Studies in Educational Evaluation 34 (2008) 1523

www.elsevier.com/stueduc

School self-evaluations and school inspections in Europe:

An exploratory study
Frans J.G. Janssens a,*, Gonnie H.W.C.H. van Amelsvoort b
a

Universtity of Twente, Faculty of Behavioral Sciences, PO Box 217, NL 7500 AE Enschede, The Netherlands
b
Netherlands Education Inspectorate, PO Box 2730, NL 3500 GS Utrecht, The Netherlands

Abstract
This article presents the results of an exploratory study into the effects of School Self-Evaluation (SSE) used by eight Education Inspectorates in
seven European countries. This study reveals that in the countries where SSE is strongly incorporated into the school inspection system, there is a
rather substantial degree of steering by the Inspectorate with respect to the form and content of the SSE; it also tends to be both oriented towards
improvement and accountability (England, Scotland, the Netherlands and Northern Ireland).
On the other hand, in countries where SSE holds a moderate or weak position in the school inspection structure, the degree of steering varies
from open (Hesse, Denmark) to pre-structured (Belgium, Lower Saxony) and is more improvement-oriented.
# 2008 Elsevier Ltd. All rights reserved.

Introduction
In the Netherlands, proportional supervision has played an
important role in the practice of inspection since 2002
(Janssens, 2007). Proportional supervision is conceptualised
in two ways. Firstly, the frequency and form of school
inspections are based on the quality of the school and the risks
of a decline in quality. This means that poorly performing
schools or schools that can be expected to suffer from a serious
decline in quality are inspected sooner and more often than the
schools that perform better. A proportionality of this kind can
also be found within other European inspectorates (van
Amelsvoort & de Wolf, 2006). Secondly and this is how it
is formulated in the Dutch Supervision Act of 2002 the extent
of the actual school inspection is based on data reported in a
schools self-evaluation. This means that the inspectorate
should not re-investigate aspects of the quality of education that
have already been evaluated properly by the school itself.
From an international perspective, it can be said that the
Dutch Inspectorate takes an extreme position in valuing School
Self-Evaluation (SSE). In the Netherlands, a proper selfevaluation may result in a less radical external evaluation by the
inspectorate; in recognition of the autonomy of schools and to

* Corresponding author.
E-mail addresses: f.j.g.janssens@utwente.nl, fjanssens@planet.nl
(F.J.G. Janssens), G.vanAmelsvoort@owinsp.nl (G.H.W.C.H. van Amelsvoort).
0191-491X/$ see front matter # 2008 Elsevier Ltd. All rights reserved.
doi:10.1016/j.stueduc.2008.01.002

encourage improvements in education. The same approach to

developments in governance within education is anticipated for
the near futurei.e., adjusting inspection evaluation to the
principles of increased school autonomy (Janssens, 2007).
However, since the new Supervision Act was implemented
only recently, it is unclear whether the current Dutch approach
will result in effective self-evaluation and quality assurance in
schools. Initial research findings are rather critical (Emmelot,
Kartsen, Ledoux, & Vermeulen, 2004). It is still unclear
whether or not this approach has undesirable side effects. Based
on these questions regarding proportional supervision, the
Dutch Inspectorate has initiated this explorative, international
comparative study.
The main goal of this study is to explore the conditions for
and the use of self-evaluations by schools and what the effects
are in the school inspectorates of countries with similar SSE
developments (the Netherlands, Belgium, Denmark, England,
Germany, Northern Ireland and Scotland).
Definitions
School inspection, quality assurance, school self-evaluation
and school improvement are important themes in current
educational policy-making and they have been given increased
attention in research. School inspection is defined as the process
of periodic, targeted scrutiny carried out to provide independent
verification, and to report on whether the quality of schools is

16

F.J.G. Janssens, G.H.W.C.H. van Amelsvoort / Studies in Educational Evaluation 34 (2008) 1523

meeting national and local performance standards, legislative

and professional requirements, and the needs of students and
parents (Janssens, 2007).
Quality assurance is a term that describes the active focus of
schools in ensuring the quality of the education they provide
and, if possible, improving its quality (Hendriks, 2001;
Visscher, 2002). Quality assurance is a cyclical process
(Schildkamp, 2007). School self-evaluation is closely related
to quality care. Based on definitions formed by McBeath
(1999), Nevo (2001) and Scheerens, Glas and Thomas (2003),
school self-evaluation was defined by Schildkamp as
a procedure involving systematic information gathering that
is initiated by the school itself and intends to assess the
functioning of the school and the attainment of its
educational goals for purposes of supporting decisionmaking and learning and for fostering school improvement
as a whole. (Schildkamp, 2007, p. 4)
Past research indicates that in most European countries an
official definition of school self-evaluation (SSE) and quality
assurance (QA) is not available (SICI, 2003; van Amelsvoort
et al., 2006). Nevertheless, various informal definitions do exist
and are being applied. In most definitions, SSE is referred to as
a process, directly or indirectly aimed at school improvement.
In some cases SSE is also regarded as a product, in respect of
the results of the SSE process. The concept of SSE as a process
may be narrowly defined as the verification or measurement
phase within a QA system or school development plan. This is
the case in places such as Lower Saxony and the Netherlands.
Even narrower definitions speak of a single measurement
instrument, such as a satisfaction survey, as being a selfevaluation.
SSE can also be more broadly defined as a systematic
process, which includes cyclic activities such as goal-setting,
planning, evaluation and defines new improvement measures.
Within these broad definitions, SSE is virtually synonymous
with the definitions for QA or school development planning.
This is the case in places such as Belgium, Denmark, England,
Hesse, Northern Ireland and Scotland. In both the narrow and
the broad definitions, the process of SSE is clearly seen as a
function or aspect of school improvement.
Another common feature of these definitions is that SSE
means assessing quality, as well as judging and valuing
learning, teaching and performance. Hence, SSE should always
yield more than information alone. It should also yield
evaluative (judgmental) information (van Amelsvoort et al.,
2006).
Finally, SSE can also be regarded as a product. Either as a
comprehensive document (as in the form of self-evaluation
used in England and in the Netherlands), or as a (short) listing
of SSE results referring to or supposedly based on underlying
SSE documents or sources (Denmark and Scotland). SSE as a
product is usually seen as a source for accountability purposes.
Inspectorates make use of SSE results in some way or
another. The better the content of an SSE corresponds with the
frameworks used for school inspections, the better use
inspectorates can make of the SSE results in their assessments

of the quality of schools. Optimal correspondence can be

achieved if the inspectorate provides guidelines, instructions
and examples to schools. In some countries, inspectorates even
provide training in SSE methods, which may enhance their
control over the shape of the SSE documents produced by
schools. The degree to which inspectorates prescribe the form
and content of an SSE is referred to as product steering in the
context of this study.
External and internal evaluation
Since the 1960s, both internal (QA and SSE) and external
evaluations (school inspections) have been seen as integral
parts of much broader school reform initiatives. External and
internal evaluation can be regarded as two interrelated areas of
one strategy for school reform. Although school reform and the
development of external and internal evaluation systems are
common to all countries participating in this study, the reasons
giving rise to the development of these strategies are rooted in
particular economic, political and social contexts. As such,
different approaches can be distinguished. A study carried out
by the International Network for Innovative School Systems
(Leithwood, Edge, & Jantzi, 1999) makes a distinction, for
example, between market-oriented approaches, the decentralisation of decision-making, professionalisation and managerial
approaches. Each of these approaches is based upon a unique
set of basic ideas and assumptions about schools and how they
can be changed and each of them draws on a unique set of
accountability tools.
In the Netherlands, for example, the development of external
and internal evaluation fits in with a trend towards decentralisation, which has evolved over the last 3040 years (van
Amelsvoort & Scheerens, 1997). In Scotland the same trend can
be discerned. However, it is different from the situation in the
Netherlands in that, together with the decentralisation process,
the views of parents, staff and pupils are surveyed and members
of the community are involved as members of inspection teams
as part of the evaluation processes. In Hesse and Lower Saxony,
two other examples, the development of evaluation practices
must be regarded more from the perspective of school
development at both the school level and the level of the
school support system.
In a Eurydice study (Eurydice, 2004) several characteristics
of external and internal evaluations inherent to internal and
external evaluation practices were investigated to clarify certain
issues of common interest in the evaluation of schools. In this
study it is assumed that the processes of internal and external
evaluations always include four stages, namely, the gathering of
relevant information, the assessment, the drafting of the
evaluation report and the implementation of changes. The main
responsibility of internal evaluation is the identification of
means to improve quality. External evaluation is meant to
monitor the quality of performance and ensure that improvements are indeed introduced.
An ideal relationship between external and internal
evaluation would be achieved if there was a certain balance
between the two. This balance, however, differs from one

F.J.G. Janssens, G.H.W.C.H. van Amelsvoort / Studies in Educational Evaluation 34 (2008) 1523

country to the next, depending on its specific context and the

function of both kinds of evaluations (Janssens, 2005;
McBeath, 1999; SICI, 2003). The Effective School SelfEvaluation (ESSE) study (SICI, 2003) made it clear that the
complexity of the concepts of accountability and improvement,
as functions of both external and internal evaluation, and their
interrelations, needed more clarification. An important aim of
our joint international study was to detect more or less
favourable ways and (national) conditions to create an effective
balance between external and internal evaluation. The study is
exploratory because the (side) effects of the respective
countries approach in school inspections are being explored
on a small scale and have not as yet been evaluated.

17

an account of the quality of education they provide through SSE

(van Amelsvoort & Scheerens, 1997).
The right balance between the accountability and the
improvement functions of both internal and external evaluations depends greatly on the organisational and political
context in the country. The ESSE study (SICI, 2003)
generated five national context variables, which proved to
be generally supportive of effective SSE in a country. On the
other hand, it was shown that the ideal balance between
external and internal evaluations had not been found in most
countries. SSE could be placed in five categories that show an
increasing degree of combination with external accountability-oriented motives (Scheerens, van Amelsvoort, &
Donoughue, 1999):

Balancing accountability and improvement

Inspectorates in Europe have a range of functions. Van
Bruggen (2001, 2006) identified three main groups of functions
that inspectorates have: (1) giving a public account concerning
the quality of education; (2) providing a guarantee of
compliance with regulations, and (3) providing an imposed
service for quality management.
The predominant perspective or function of external
evaluations which are conducted by inspectorates is accountability, in the sense of providing a public account of the quality
of schools and the education they provide (Janssens, 2007;
Kyriakides & Campbell, 2004; van Amelsvoort et al., 2006). At
a national level, external evaluations by inspectorates provide
information to policy makers and the public about the state of
the education system, value for money, compliance with
regulations, and quality differences among schools. On the
other hand, governments or inspectorates can regard an
external evaluation as a vehicle for schools to improve. The
second function of an external evaluation, therefore, is
improvement. It is widely recognised that inspectorates that
carry out independent evaluations make a significant contribution to the improvement of the education sector (Janssens,
2005, 2007; Matthews & Sammons, 2004; Ofsted, 2004; van
Bruggen, 2006). Inspection reports provide feedback to
schools about their strengths and weaknesses, and indicate
ways to develop or trigger the internal evaluation of the school.
Also, inspection reports may mirror the SSE findings of the
school.
In their literature review, Maes, Vereecke and Zaman (2002)
concluded that it is widely accepted that, for the schools, the
primary function of SSE is to focus on improvement. SSE is
regarded as a tool to facilitate school improvement (see also:
Janssens, 2005; Schildkamp, 2007). This conclusion is
supported by our analysis of existing definitions for SSE in
seven countries (van Amelsvoort et al., 2006). However, SSE
also has an accountability function. Schools are supposed to
provide insight into their quality, accessibility, efficiency and
freedom of choice to governments and inspectorates. This
function is seen as the concomitant of increased school
autonomy and is also called vertical accountability. Since
decentralisation processes in education have proceeded all over
Europe, an increased appeal has been issued to schools to give

 tailor-made SSE of individual schools;

 SSEs that are part of improvement programmes that involve a
number of schools;
 SSEs that are explicitly aimed at providing information to
external constituencies as well as using information for
school improvement processes;
 SSEs that serve internal and external purposes and are subject
to meta-evaluations by inspectorates;
 SSEs that are spin-offs of national or district level assessment
programmes, where school results are fed back to individual
schools.
An inherent imbalance between the two functions of an
external evaluation would generally occur when the urge to
advise, support or stimulate improvement would hinder the
external evaluator in achieving an independent judgment.
This could be the case when the evaluation report contains too
many detailed recommendations and suggestions for
improvement, or when the quality-evaluation function and
the quality-support function are combined within one institute
in a country. This would place inspectors in a situation in
which they must evaluate their own advice. It could also
involve external evaluations which hinder the internal
motivation of schools to improve the education they
provide.
On the other hand, an inherent imbalance between the two
functions of an internal evaluation or SSE would occur when
the accountability demands impede the improvement-function
of SSE. This could lead to undesirable side effects. Examples
are self-evaluations that are written for the inspectorate which
no longer serve the goal of improving education.
Tensions particularly occur when there is an attempt to
combine the two basic forms in one way or the other, as in forms
of proportional supervision. When the stakes are high in
external evaluation, schools might feel threatened and turn to
strategic or opportunistic behaviour, such as withholding data,
distorting data, manipulating pupils school careers, etc. An
obsession with rules and procedures is to be expected as a side
effect (Janssens, 2005; de Wolf & Janssens, 2007). Supporters
of the combination of SSE and external evaluation, therefore,
make a number of demands for the dialogue between the two
(Nevo, 2001).

18

F.J.G. Janssens, G.H.W.C.H. van Amelsvoort / Studies in Educational Evaluation 34 (2008) 1523

In the Netherlands, the decentralisation process is rapidly

proceeding towards governance in education (Janssens, 2005).
Governance increases and spreads the appeal made to schools
to be accountable. Next to the vertical (hierarchical) external
accountability function, two new forms of accountability can be
distinguished: (1) horizontal accountability, in which schools
should provide their community and stakeholders with insight
into their processes, choices and results, and (2) vertical
accountability, in which schools are supposed to provide their
boards of supervision with insight into the adequacy of their
management, policy and steering. In England and Scotland, we
also find this kind of vertical accountability to Local Education
Authorities.
To focus on the research questions of this exploratory study,
we have approached SSE and the inspection of the education
sector from the perspective of both accountability and
improvement in order to detect features of balance and
imbalance between these two interrelated concepts.

National SSE context

In the various European countries, there is an ongoing search
for the right balance between the role of schools and the role of
inspectorates when it comes to determining how to monitor and
improve the quality of education. This balance has to be found
within the country-specific evaluation context. The role that
SSE plays within a country and the way in which it is formed
depend very much on the political, historical and organisational
context in which schools operate.
In this study we build upon the findings from the ESSE study
(SICI, 2003). The contextual factors which proved to influence
the development of SSE in a country are:






the extent and method of legal anchoring;

the presence of national reference data;
the presence of a general framework with quality indicators;
provision for training in and support for self-evaluation, and
the nature of external evaluation/inspection.

Theoretical framework
The main research focus of this study is to explore the effects
of the conditions for and the use of SSE in the school
inspectorates in European countries undergoing similar SSE
developments. A theoretical framework was applied to
systematically acquire detailed knowledge on the use and
effects of SSE for inspection purposes. Approached from the
perspective of the functions of SSE, positive effects would
indicate a proper balance in which improvement is clearly the
initial function of SSE. Negative effects point to a less effective
SSE when the accountability demands clearly impede the initial
improvement-function of SSE.
The main research question was subdivided into several
exploratory research questions about the positioning of SSE in
inspection systems and questions about steering mechanisms
that potentially influence the form and content of SSE
documents (see Fig. 1).

These context factors were regarded as background

variables.
Position of SSE in the inspection system
In some countries, such as the Netherlands, a proper selfevaluation may result in a less drastic external evaluation by the
inspectorate. This study investigated how strongly the SSE is
embedded in each inspection system, what kind of inspection
decisions are taken based on an SSE and what the estimated
impact of an SSE in the inspectors final assessment of the
quality of the school. These aspects are conceptually
interrelated. The most important aspect is the way inspectors
use the SSE results in their assessment of school quality; the
position of SSE is rated more strongly if the SSE results are
used for assessment purposes, while they are rated more weakly
if the results are used for discussion purposes only.

Fig. 1. Exploratory theoretical framework.

F.J.G. Janssens, G.H.W.C.H. van Amelsvoort / Studies in Educational Evaluation 34 (2008) 1523

Steering of SSE products by the inspectorate

It is assumed that inspectorates make use of SSE results in
some way or another. This implies a demand on schools to
record the results of SSE data and submit this information to
the inspectorate. There may be some requirements concerning
the form, content and delivery of the SSE documents.
This so-called product steering can be characterised by the
amount of requirements (prescriptions or guidelines), the
openness of the requirements, or the items or subjects to
which the requirements refer. In some countries inspectorates
provide training in SSE methods, which may enhance their
control over the shape of the SSE documents produced by
schools.
Effects and side effects
The present study does not allow for effectiveness research,
in which the dependent variable would be the real quality of the
SSE itself. The latter would require an internationally
comparable measurement of SSE quality, which would only
be possible in a joint international comparative research
project. However, for exploratory purposes, two proxy
variables were chosen:
 the characteristics of the SSE documents;
 key inspectors opinions about characteristics of SSE and
about the consequences of utilising and steering SSE.
Three variables are relevant with regard to the characteristics
of these SSE products. First, the SSE may be delivered as a
separate document or as an integrated part in a school
development plan. Next, there may or may not be a systematic
approach clearly underlying the document. Third, the orientation or function of the SSE document could be more
accountability-oriented or more improvement-oriented.
The second proxy for the effect variable is constituted by the
opinions of key inspectors in a country. They are in a position to
characterise SSE products and to give an indication of the
function SSE fulfils within the schools in their country. They
may also have observed positive and negative effects on SSE
development that could be related to the national constellation
of internal and external evaluations.
Method and data
The main goal of this study is to explore the effects of the
conditions for and the use of SSE in the school inspectorates in
European countries undergoing similar SSE developments. In
total, eight school inspectorates from seven European countries
took part in the study: the Netherlands, Belgium, Denmark,
England, Germany, Northern Ireland and Scotland. From
Belgium, the Flemish inspectorate participated and from
Germany the inspectorates of two states participated in the
study: i.e., Hesse and Lower Saxony. The study was conducted
by a research team from the Netherlands inspectorate of
schools.

19

The data for this study were collected in different ways. The
eight inspectorates were asked to make documents available on
the role of QA and SSE in school inspections and provide
examples of SSEs from their country. The documents were
analysed with respect to the role and position of the SSE in
school inspections and the degree to which inspectorates gave
schools guidelines for the content and the structure of an SSE.
The examples of SSEs (N = 25) were analysed for their content
and the degree to which the SSE fit into the school inspections
in the different countries.
Finally, the participating inspectorate was asked to indicate
key inspectors who were interviewed about the role and use of
the SSE and about the degree to which the SSE is pre-structured
by the inspectorate. These key inspectors (N = 17) also
participated in a meeting of experts in which additional data
were gathered and the draft findings of the study were validated.
Results
National evaluation context
The national evaluation context refers to each country
creating a context for SSE that takes account of national and
local circumstances. This involves striking a dynamic balance
between responsibility for accountability and improvement,
external and internal pressures for change and the support and
challenge provided to schools (SICI, 2003, p. 65). Within a
productive interplay, the SSE product is designed to be a
summary of the schools self-evaluation for internal use
(improvement) and for the purposes of inspection (accountability). However, the balance is disturbed when accountability
goals impede school improvement. An imbalance can result in
undesirable side effects.
The presence of external support factors in the seven
countries is briefly presented below. The focus is placed on the
extent to which they might stimulate either the accountabilityorientation (AO) or the improvement-orientation (IO) of the
SSE.
 The legislative position of SSE in all countries investigated is
aimed at school improvement. However, in some countries
(Belgium, Northern Ireland, Scotland) legislation also
prescribes reporting on SSE or the external evaluation of
SSE (England, Lower Saxony, the Netherlands) which might
increase the AO of the SSE.
 The presence of adequate benchmarking data contributes to
good self-evaluation and school improvement, which is
predominantly the case in England, the Netherlands, Northern Ireland and Scotland. In Denmark some benchmarking
data are available at a regional level. In some places
(Denmark, England, Lower Saxony, the Netherlands, Northern Ireland, Scotland) schools are held accountable for their
performance based on benchmarking data; this might
increase the AO of the SSE that draws heavily on data
interpretation skills in schools.
 In all countries frameworks or guidelines aimed at school
improvement are available to support the SSE. The IO of SSE

20

F.J.G. Janssens, G.H.W.C.H. van Amelsvoort / Studies in Educational Evaluation 34 (2008) 1523

might be increased if guidelines provide abundant freedom

for the schools own priorities, such as in Belgium, Denmark,
Hesse and the Netherlands. In England and Scotland, quality
frameworks for inspection and for SSE are based on the same
evaluation schedule and quality criteria, but schools are at
liberty to choose any method for SSE or choose their own
priorities (Scotland). The AO of the SSE might be increased if
the quality frameworks for inspection and for SSE are largely
the same, as in Denmark, England, Northern Ireland and
Scotland. In Lower Saxony, developments are proceeding in
the same direction. However, a general tendency can be
perceived of schools (voluntarily) using the inspection
framework as their framework for SSE.
 The availability of SSE training facilities varies across
countries. A range of providers and an offer of different open
models (Belgium, Denmark, England, Hesse, Lower Saxony,
the Netherlands) might be favourable for the IO of an SSE,
while centrally (or locally) offered training in one model
(England, Northern Ireland, Scotland) might be more
favourable for the AO of an SSE (especially when the
training is offered by those to whom the schools are
accountable). In England, a mixed training market is in
operation.
 In all participating countries, an education inspectorate exists
that more or less pays attention to the SSE. However, if the
position of SSE is firmly embedded in the inspection system
and a lot of the inspectorates decisions (including
judgments) are based on the SSE, this might stimulate the
AO of an SSE. This is increasingly the case in England,
Lower Saxony, Northern Ireland and the Netherlands.
 The three main functions of European inspectorates are
evaluating the quality of education, advising schools about
improvement and checking compliance with regulations (van
Bruggen, 2001). Countries differ to the extent that these
functions exist. In many countries a mixture of all three
functions exists. In Hesse (Germany), Denmark and Lower
Saxony (Germany), the predominant function of inspections
is to advise and support schools and to promote improvement,
which might increase the IO of the SSE. In Belgium,
England, the Netherlands, Northern Ireland and Scotland, the
predominant function of inspections is quality assessment
and accountability, which might increase the AO of the SSE.
However in Denmark, Hesse and Lower Saxony, the function
of inspections is rapidly changing from advising and
controlling to providing external quality evaluation.
Position of SSE in the inspection system
An analysis was made into how and to what extent
inspectorates make use of SSE. It was studied how strongly the
SSE is embedded in each inspection system, what kind of
inspection decisions are taken based on the SSE, and what the
estimated impact of the SSE is on the inspectors ultimate
assessment of the quality of the school.
In all countries, the SSE is regarded as a source of
information for the inspector and this information plays a role
in setting the agenda for the inspection visit. Moreover, in two

countries (Lower Saxony [Germany] and the Netherlands), the

SSE is regarded as a sufficient basis for inspectoral judgments.
In Denmark, Hesse and Northern Ireland, the SSE is also seen
as a basis for advice. In nearly all countries, there is some kind
of reality check with respect to the SSE data. However, this
check ranges from very informal to more formal. In Belgium,
England, Hesse and Northern Ireland, the existence of the SSE
or similar forms of school planning/bidding for resources
serves as a basis for part of the school funding.
Frameworks for inspection differ across countries and these
frameworks correspond with the functions that these inspectorates have. In places where the predominant function of an
inspection is quality assessment (Belgium, England, Lower
Saxony, the Netherlands, Northern Ireland and Scotland),
frameworks are standardised and contain quality indicators and
effectiveness assessment standards. In places where the main
function of the inspectorate is to advise and support
(Denmark and Hesse), frameworks appear in the form of
checklists and are less standardised. This study shows that SSE
is more or less present in the frameworks of all participating
countries. SSE is strongly embedded in the frameworks of the
inspectorates of the Netherlands and of Lower Saxony: the
frameworks contain detailed indicators and standards for the
assessment of QA, including indicators for the SSE.
Furthermore, SSEs are indirectly present in the inspection
frameworks of England, Northern Ireland and Scotland, where
an SSE is one of the indicators for strategic management,
leadership and quality assurance (Scotland). In England and
Scotland, indicators regarding SSE are also present in the
frameworks for inspection of Local Education Authorities and
School Governing Boards and in England for all inspection
activities in schools and colleges. In Belgium, the SSE is one of
the indicators for QA, which are described in the inspection
report but are not assessed. In Denmark and Hesse, parts of an
SSE are used as checkpoints in the checklist that inspectors use
during their inspection or while approving the school plan.
In all countries, inspectors select items for discussion from
the SSE. In five countries, inspectors assess and describe the
quality of SSE in their reports, either directly (the Netherlands,
Scotland) or indirectly through indicators for management and
leadership, or indicators for the capacity of the school to
improve and for overall effectiveness (England, Northern
Ireland). In Belgium, inspectors describe the characteristics of
the SSE in their reports.
Comparisons between the SSE data and the inspection data
or reality are made by all inspectorates, but the thoroughness of
these comparisons variesfrom very thorough in the Netherlands to much less so in Denmark and Hesse. In Denmark and
Hesse, the few decisions that are based on an SSE can be
labelled as advice and agreements with the school about
improvements. In England, Lower Saxony, the Netherlands,
Northern Ireland and Scotland, aspects of the SSE and QA may
lead to decisions such as changing the content, form or
frequency of the inspection.
SSE products are not intended to have any impact on the
inspectors ultimate judgment of the school in Belgium,
Denmark, England, Hesse and Scotland. SSE products are

F.J.G. Janssens, G.H.W.C.H. van Amelsvoort / Studies in Educational Evaluation 34 (2008) 1523

Fig. 2. Position of the SSE in school inspections across European inspectorates

(1 = weak and 8 = strong).

intended to have an impact in the Netherlands and in Lower

Saxony.
Our analysis shows that the SSE occupies a strong position
in the inspections in England, Northern Ireland, the Netherlands, Lower Saxony and Scotland because inspectors use SSE
results in their school quality assessment, they take some or
many decisions based on the SSE and the presence of SSE in the
inspectorates framework is strong (see Fig. 2). In Denmark, the
position of the SSE in school inspections is rather weak. The
position of the SSE is weak to moderate in Belgium and Hesse.
The inspectorate in the Flemish part of Belgium only uses the
SSE to take some decisions. No judgments are based on the SSE
and the SSE is only indirectly present in the frameworks for
inspection.
Product steering
In all participating countries, SSE is used or intended to be
used in the preparation phase of an inspection visit. Thus, in all
countries inspectorates ask schools to send a type of SSE (if
available) in order to prepare their inspection visit, or they use a
SSE document that was sent on another occasion (such as for
the approval of a school plan).
Nowhere are there legal prescriptions for the form and
content of the SSE, but sometimes these prescriptions are either
partly or indirectly present in the legal prescriptions for the
form and content of school development plans (Hesse). But
only in the case of vocational education are there legal
prescriptions for the content of the SSE reportin Denmark,
Lower Saxony and the Netherlands.
In some countries, the form and content of the SSE products
are influenced by inspection demands to summarise the SSE
data in a highly structured format. This is the case in Belgium,
England, Lower Saxony and Northern Ireland.
In Denmark and Hesse, open or semi-structured guidelines
are provided at a national level to shape the form and content of
the SSE reports. In Hesse, these guidelines are embedded in the

21

guidelines for the school development plan. In the Netherlands,

there is only indirect steering of the form and content of SSE
products (through validation rules), but one may argue that the
assessment indicators for QA and the principle of proportionality influence the form and content of the SSE products. In
Scotland, the inspection framework How Good is Our School
also indirectly influences the form and content of the SSE
products.
An analysis of the results shows that in most countries some
kind of steering is given to the content of the SSE product. In
Hesse, many other guidelines are provided, such as the ones
referring to the underlying SSE process, the evaluation
instruments, target setting, evaluation cycles, reporting
requirements and guidance (Hessisches Kultusministerium,
2001). In Belgium, no steering is given to the underlying SSE
leading to the Information file. In England, the new SSE form
(implemented in mid-2005) greatly pre-structures the SSE
summary reporting method on content, systematic approach,
target setting and evaluation methods.
In England, Hesse and Scotland, in addition to the training
offered at regional or local levels, training in SSE is provided
centrally or provided by the inspectorate. This might have a
great deal of influence on the form and content of the SSE
products, especially when the training is set up on the basis of
one model. The direction of this influence (accountability or
improvement) is as yet unclear. In the other countries, training
in SSE is less centrally provided or completely market driven.
Characteristics of SSE
There appeared to be great variety among the analysed
samples of SSE. For instance, detailed school development
plans containing a small SSE element were found in Denmark,
Hesse, Lower Saxony and the Netherlands. Separate comprehensive SSE documents were found in Belgium, England, the
Netherlands and Scotland. Some formats contained results of
SSE in Belgium, England and Scotland. There were also
overviews of results on stakeholder satisfaction surveys in
Denmark and Lower Saxony. SSE overviews linked to certain
projects were found in Belgium, Lower Saxony and the
Netherlands. According to the key inspectors of the countries
where considerable product steering exists, the SSEs usually
follow the advised or prescribed structure, but the quality of the
elaboration varies widely from school to school (England and
Hesse). In countries where little steering is given, both the
structure and the elaboration of the SSE vary (the Netherlands).
SSE products were judged to be systematic if they were
evaluative rather than descriptive, took several respondents into
account, used adequate measurement instruments and referred
to the school as a whole. In all countries good, fair and weak
examples of systematic SSE products were found. In the
comprehensive SSE, one can detect some components of the
PDCA circle of Deming (1986). However, SSE differs
considerably in the degree to which this systematic approach
is elaborated. A remarkable finding was the fact that often
parallel evaluation cycles and goals exist in combination with a
weak connection between school development goals and

22

F.J.G. Janssens, G.H.W.C.H. van Amelsvoort / Studies in Educational Evaluation 34 (2008) 1523

evaluation goals. Other characteristics of the SSE analysed

were:
 improvement measures were mentioned, but often without a
plan for implementation;
 judgments that lack evidence, or descriptions without
judgment;
 often references were made to underlying documents, which
may look systematic but which do not always contribute to
the transparency of the SSE;
 in the majority of cases, the satisfaction or opinion survey
was used as the only instrument;
 for the purpose of measurement, several stakeholder groups
were taken into account as respondents in these surveys.
The cycle of inspections greatly influences the cyclic
approach of the SSE. Many SSEs are descriptive rather than
judgmental and the connection between goals and measurement
is rather weak.
From the document analyses, a clear impression emerges
that both the position of the SSE in the inspection system and
product steering greatly influence the form and content of the
SSE products that schools deliver. In other words, schools
deliver the kind of SSE that was requested by the inspectorate
or that they trained for in a project. In countries where the
function of inspection is external evaluation, the content of the
SSE is derived from the inspection indicatorssometimes
entirely. And sometimes the schools own priorities are
evaluated in addition to these indicators.
Whether there is a format or not, there are many similarities
between the SSE products within one country, with the
exception of the Netherlands, where a great variety exists.
Despite these similarities in appearance, the quality of the
different products varies widely.
Conclusions and discussion
When combining the characteristics of national evaluation
contexts with the current position SSE occupies in inspection
systems, one may conclude that the position of the SSE is
stronger in countries where more accountability demands are
imposed on the SSE. The position of the SSE is weaker when
the national evaluation context encourages improvement
more. The first group of countries consists of England, the
Netherlands, Northern Ireland and Scotland. In these
countries the national evaluation context is equally supportive
of accountability-oriented and improvement-oriented SSEs.
In these four countries, the position of the SSE in the
inspection system is strong. In the other group of countries
(Belgium, Denmark, Hesse-Germany, Lower Saxony-Germany), the national evaluation context is more supportive of
improvement-oriented SSEs. In these countries the position of
the SSE in the inspection system can be rated as weak to
moderate.
A diverse pattern emerges when we relate the steering of
form and content of SSE products to the context and position of
the SSE. In some countries, a pre-structured format for

reporting on an SSE is supplied, but schools are free to choose

their SSE methods. In other countries, no steering of format
exists, but schools voluntarily choose to adopt the inspection
framework as the format used for reporting on an SSE.
The exploratory character of this study does not allow
characteristics of SSE products to be clearly attributed to the
context, position or steering given in the investigated countries.
However, the findings of the study do point in a direction in
which steering seems to be the key factor determining the
characteristics and function of SSE products. In other words,
accountability demands imposed on the SSE generate
accountability-oriented SSEs, while improvement demands
generate improvement-oriented SSEs.
Alongside advanced decentralisation processes throughout
Europe, a clear tendency towards placing increased accountability demands on schools and their SSE has emerged. In all
countries, inspectorates ask schools to deliver their SSE and the
results are utilised in one way or the other. But in all countries a
mixture of improvement and accountability demands are
imposed on the SSE. In none of the countries was the national
evaluation context exclusively supportive of accountability
purposes. Fluctuations are evident that illustrate the process of
balancing accountability and improvement. For example, the
evaluation context in Scotland used to be quite accountabilityoriented but is shifting towards more incentives for improvement. In the Netherlands, the context is rapidly shifting towards
forms of governance, where horizontal accountability will
replace parts of the external accountability function of the SSE.
Denmark and the two German states are shifting from the
historic advisory function of their inspectorate to the external
evaluation function. But these countries have not yet clearly
separated the external evaluation function from the advisory
function for the newly constituted inspectorates.
When reviewing the international literature (Kyriakides &
Campbell, 2004; Maes et al., 2002), the working definitions in
all countries and Dutch research (Hofman, Dijkstra, Hofman,
& de Boom, 2004; Janssens, 2005; Schildkamp, 2007), it
becomes clear that SSE is seen as a tool to facilitate school
improvement. Either it is conceptualised as a systematic
process of quality assurance or as the measurement phase
within such a process of quality assurance. It is rather strange
that no country has an official definition of SSE, although every
school is expected to perform such a self-evaluation. This may
cause confusion and actually hinder school improvement.
Accountability is seen as the natural counterpart of school
autonomy. SSE is regarded as a tool for making schools
accountable for their performance. At the same time, its main
function is improvement. This combination is full of contradictions and inherent tensions.
There is a general tendency for schools to use the inspection
framework as their framework for the SSE, either mandatorily
or voluntarily. This is also the case in countries where we found
little steering of the form and content of the SSE, such as in the
Netherlands (Schildkamp, 2007). It is not easy, however, to
interpret this tendency. One interpretation may be that schools
are turning towards more accountability-oriented SSEs.
Another interpretation may be that schools need help and

F.J.G. Janssens, G.H.W.C.H. van Amelsvoort / Studies in Educational Evaluation 34 (2008) 1523

structure to perform their SSE. Yet another interpretation could

be that schools value the inspection framework as a suitable
framework for measuring quality, since they have difficulty in
formulating their own evaluation goals.
There are advantages: schools and inspectors use the same
performance indicators, which makes the communication about
the quality of schools more transparent and comparisons easier.
However, using a framework originally designed for external
evaluation for SSE purposes is not without its problems. A
tailor-made SSE should always contain the schools own goals,
priorities and improvement areas. These are not necessarily the
same as the quality indicators in an inspection framework.
Constraints on innovation and less ownership can be expected
(SICI, 2003). Another problem is that external evaluation
frameworks are designed more for accountability, judgment
and assessing quality, while a framework for SSE should
contain ways to measure and deal with growth (Hofman et al.,
2004; Janssens, 2005).
One might, therefore, be tempted to opt for a participatory
approach in which evaluation criteria are drawn up by the
school on an entirely autonomous basis, though naturally with
due regard given to local and national objectives. External
evaluators, for their part, should also take the evaluation criteria
and objectives drawn up by schools during internal evaluation
into account, so that they are able, if necessary, to qualify their
own judgments (Eurydice, 2004). Thus, an open inspection
framework that leaves space for the schools own priorities and
audit trails seems to be a way to combine the advantages and
disadvantages of using the inspection framework for SSE
purposes.
From our analyses, the impression emerges that a mixture of
a strong position for the SSE in the inspection system, an open
inspection framework, and considerable support for schools in
steering towards improvement is the most promising combination for bringing about an effective SSE.
References
van Amselsvoort, G. H. W. C. H. , & Scheerens, J. (1997). Policy issues
surrounding processes of centralisation and decentralisation in European
education systems. Educational Research and Evaluation, 3(4), 340
364.
van Amelsvoort, G. H. W. C. H., Bos, K. Tj., Janssens, F. J. G., Klaver, L.,
Lelyveld, J., & Pol, M. (2006). Proportional supervision and school
improvement from an international perspective. Utrecht: Inspectie van
het Onderwijs (Education Inspectorate).
van Amelsvoort, H. W. C. H. , & de Wolf, I. (2006). Risk analysis in European
Inspectorates. Utrecht: Netherlands Education Inspectorate.
van Bruggen, J. C. (2001). Functions of inspectorates in Europe. Paper
presented at the International Inspection, Academy, Berlin.
van Bruggen, J. C. (2006). Schulinspektion im internationalen Vergleich.
Vortrag fur die Bilanztagung der Hessischen Schulinspektion [International
comparison of school inspections: Paper presented to the meeting of the
Hesse Education Inspectorate]. Frankfurt: IQ.
Deming, W. E. (1986). Out of the crisis. Quality, productivity and competitive
position. Cambridge, MA: Cambridge University Press.

23

Emmelot, Y., Kartsen, S., Ledoux, G., & Vermeulen, A. (2004). Ervaringen met
het vernieuwde toezicht [Experiences with the new Dutch Supervision Act].
Amsterdam: SCO Kohnstamm Instituut.
Eurydice. (2004). Evaluation of schools providing compulsory education.
Brussels: Eurydice.
Hendriks, M. (2001). Systemen voor kwaliteitszorg in het primair en secundair
onderwijs [Systems for quality care in primary and secondary education]. In
R. J. Bosker (Ed.), Kwaliteitszorg. Onderwijskundig lexicon III (pp. 4772).
Alphen aan den Rijn: Samsom.
Hessisches Kultusministerium. (2001). Schulprogramme und Evaluation in
Hessen [School programme and evaluation in Hesse]. Wiesbaden: Hessisches Kultusministerium.
Hofman, R. H., Dijkstra, N. J., Hofman, W. H. A., & de Boom, J. (2004).
Kwaliteitszorg in het Onderwijs. Deelstudie 1 BOPO [Quality Assurance in
Education]. Groningen: RUG.
Janssens, F. J. G. (2005). Toezicht in discussie: Over onderwijstoezicht en
Educational Governance [School Inspections and Educational Governance]. Enschede: Universiteit Twente (Inaugural Lecture).
Janssens, F. J. G. (2007). Supervising the quality of education. In W. Bottcher &
H. G. Kotthoff (Eds.), Schulinspektion: Evaluation, Rechenschaftsleging
und Qualitatsentwicklung [School inspection: Evaluation, accountability
and quality development] Munster: Waxman.
Kyriakides, L., & Campbell, R. (2004). School self-evaluation and school
improvement: A critique of values and procedures. Studies in Educational
Evaluation, 30(1), 2337.
Leithwood, K., Edge, K., & Jantzi, D. (1999). Educational accountability: The
state of the art. Gutersloh: Bertelsman Foundation Publishers.
Maes, B., Vereecke, E., & Zaman, M. (2002). School self-evaluation and
inspection: A review of the literature. Belfast: SICI.
Matthews, P., & Sammons, P. (2004). Improvement through inspection: An
evaluation of the impact of Ofsteds work. London: Ofsted/University of
London.
McBeath, J. (1999). Schools must speak for themselves. London: Routledge.
Nevo, D. (2001). School evaluation: internal or external? Studies in Educational
Evaluation, 27(2), 95106.
Ofsted. (2004). A new relationship with schools: Improving performance
through school self-evaluation. London: DfES Publications.
Scheerens, J., van Amelsvoort, H. W. C. H. , & Donoughue, C. (1999). Aspects
of the organisational and political context of school evaluation in four
European countries. Studies in Educational Evaluation, 25, 79108.
Scheerens, J., Glas, C., & Thomas, S. M. (2003). Educational evaluation,
assessment and monitoring. Studies in Educational Evaluation, 29(1), 79
108.
Schildkamp, K. (2007). The utilisation of a self-evaluation instrument for
primary education. Enschede: University of Twente.
SICI. (2003). Effective School Self-Evaluation (SSE). Belfast: SICI.
Visscher, A. J. (2002). A framework for studying school performance feedback
systems. In A. J. Visscher & R. Coe (Eds.), School improvement through
performance feedback (pp. 4172). Lisse: Swets & Zeitlinger.
de Wolf, I. F., & Janssens, F. J. G. (2007). Effects and side-effects of inspections
and accountability in education: An overview of empirical studies. Oxford
Review of Education, 33(3), 379396.
Frans J.G. Janssens is professor of school inspections and public accountability at the University of Twente. He is also the co-ordinating inspector of
schools for the Netherlands Education Inspectorate. His field of research and
interest is educational evaluation and the effectiveness of school inspections and
accountability systems.
Gonnie H.W.C.H. van Amelsvoort is the inspector of schools and program
manager for international affairs at the Netherlands Education Inspectorate.
Her fields of interest are the connection between internal and external school
evaluation and international comparisons of school inspection systems.