0021.9681/85$3.00+ 0.

00
Copyright 0 1985Pergamon Press Ltd

J Chron Dis Vol. 38, No. 4, pp. 339-351, 1985

Printed in Great Britain. All rights reserved

THE MEASUREMENT
ITERATIVE LOOP: A
FRAMEWORK
FOR THE CRITICAL APPRAISAL
OF NEED, BENEFITS AND COSTS OF
HEALTH INTERVENTIONS
PETER TUGWELL,

Departments

KATHRYN J. BENNETT, DAVID L. SACKETT

and R. BRIAN HAVNES

of Clinical Epidemiology and Biostatistics, and Medicine, McMaster University,

Hamilton, Ontario Canada LSN 325
(Received

in revised form

27 June 1984)

Abstract-A

framework for organizing health services data is presented that subdivides the
spectrum
of health information
into subgroups
that constitute
a logical progression
from
quantifying
the burden of illness, through identifying its likely causes, to validating interventions
that prevent or ameliorate it and evaluating their efficiency, to monitoring
the application
of these
interventions
and coming full-circle to determine whether the burden of illness has been reduced.

INTRODUCTION
WHAT CLINICAL manoeuvres

for the management

of osteoarthritis
would most effectively
and efficiently help the patients in my practice?
What health services for the detection
and control
of hypertension
would most
effectively and efficiently help the people in my region?
Those who provide, plan or pay for health services must decide which health services
should be provided to whom in order to effectively and efficiently reduce the burden of
illness, disability and untimely death. Such decisions demand the identification
and critical
appraisal of existing evidence and, if it is insufficient,
the generation
of new evidence.
The approach described in this paper, an outgrowth of work by Cochrane [l], Sackett
[2] and Evans [3], provides a framework
for assembling
the specific subset of health
information
that is most likely to tell us how to reduce the burden of both morbidity
(symptoms;
physical, emotional
and social functional
impairment)
and mortality. This is
accomplished
by sub-dividing
the spectrum
of health information
into groups that
constitute
a logical progression
from quantifying
the burden of illness, through identifying its likely causes, to validating
interventions
that prevent or ameliorate
it and
evaluating their efficiency, to monitoring
the application of these interventions
and, coming
full-circle, to determining
whether the burden of illness has been reduced. Each of the
seven steps in the measurement
loop (Fig. 1) poses a different type of research or
evaluation
question and, if new information
is required, calls for a specific set of methods.
The loop format emphasizes the importance
of monitoring
after implementing
a health
intervention
to determine
whether the planned
reduction
in the burden
of illness is
achieved. This process is iterative since in almost all health care situations,
the burden of
illness is only reduced by a small proportion
and repeated cycles of the loop are needed
to eradicate even that proportion
for which beneficial interventions
exist.
Address all correspondence and reprint requests to: P. Tugwell, Department of Epidemiology and Biostatistics,
McMaster University, 1200 Main Street West, Hamilton, Ontario, Canada LSN 325.
339

340

PETER

TUGWELL

et

al.

BURDEN OF ILLNESS
Determine
health
status using health
status indicators

REASSESSMENT
Reassessment
of
magnitude
of
burden qf illness

AETIOLOGY
OR CAUSATION
Identify and as?.es.s
possible
causes of
burden of illness

THE
MEASUREMENT
ITERATIVE
LOOP

6
MONITORING
OF PROGRAMME
Ongoing
monltorlng
using
markers selected
to
indicate SUCCESS

COMMUNITY
EFFECTIVENESS
Assess benefltiharm
ratlo
of potentially
feasible
Interventions
and estimate
reduction
of burden of illness
11 programmes
successful

SYNTHESIS
& IMPLEMENTATION
lntegrabon
of feasfbilrty.
Impact and effnency
to make recommendatmns

EFFICIENCY
DetermIne
relatlonshlps
between costs and effects
of options
w,thln and
acrOss programmes

FIG.

1.

In the discussion that follows we show how a critical appraisal of the need, benefits and
costs of health interventions
using the loop can contribute
to health decisions. The loop
approach
provides a practical guide on research methods useful to both: (i) the consumers of research-health
professionals
and policy makers who wish to decide whether
to apply the results of research investigations
to health care decisions; and (ii) the doers
of research-those
individuals
involved in the planning
and implementation
of health
research. Osteoarthritis
and hypertension
are used to illustrate the loop steps.
MEASUREMENT

LOOP

STEP

NO.

l-THE

BURDEN

OF

ILLNESS

The first step determines

current levels of morbidity
(physical, emotional
and social
function; symptoms; disability days) and mortality (death rates, healthy years of life lost).
Our osteoarthritis
and hypertension
examples illustrate the various indicators that can be
used to measure the burden associated with these two health problems.
The 1974 U.S. National Health Interview Survey found that 4.5 million people suffered
reduction of activities due to pain and limitation of movement from chronic arthritis and
rheumatism
[4]. This was second only to heart conditions
among those 6.5 years of age or
more. The Canada Health Survey reported back, limb and joint disorders as the most
prevalent cause of health problems and disability [5]. In a community
survey in England
it was shown that 20% of those X-rayed showed radiological
abnormalities
of the hip [6].
Unfortunately,
most of this burden is unavoidable
but surgery of the hip is a good example
of an intervention
that can relieve a substantial
amount of pain and disability.
Several studies have consistently
shown us that sizeable fractions of U.S. and Canadian
adults have inadequately
controlled
hypertension
[7-91. Hypertension
provides a good
example of an asymptomatic
condition
contributing
to the burden of illness due to its
characteristic
as a risk factor for cardiac failure, stroke and death. This emphasizes the
temporal component
of the natural history of disease when assessing burden.
To make judgements
about the quality of burden information,
three guidelines are
applied. First, is the attribute selected for the measurement
relevant? For example, the use
of mortality rates as an indicator of burden due to arthritis and musculo-skeletal
problems
tells us nothing about the magnitude
of the associated pain and disability.
Even when

The Measurement

Iterative

Loop

341

mortality
is relevant it may give an incomplete
picture. For example, for hypertension,
burden includes distress and disability due to cardiac failure and stroke.
Second, are the measurement
methods accurate? Measurement
of the health attribute
or health status indicator must be accurate. Cause of death is often inaccurate and most
components
of morbidity require specially designed surveys. Caution should be exercised
when surrogate measures such as utilization
or supply are used as indicators
of burden
because they have been shown to be at variance with good survey data in some situations
[lo]. A recent report on the need for hip replacement
was based on utilization rates in one
county in the U.S. [l 11.Unfortunately,
it is uncertain what proportion
of all those in need
of total hip arthroplasty
is reflected by utilization
rates. In blood pressure studies
conducting
only one blood pressure assessment per person, the observed rates may be an
overestimate
since it is well known that many persons found hypertensive
at the first
reading will be normotensive
on subsequent
ones. Where good evidence is not available,
multiple sources of data should be checked.
Third, are the results easy to interpret and apply? Summary statistics for indicators such
as mortality rates, the proportion
suffering specific disability or mean blood pressure (with
ranges or confidence
limits as appropriate),
without sophisticated
mathematical
transformation
or statistics, are usually sufficient.
Health indices that combine several attributes or indicators of a disease into one number
are often used to express disease burden. However, the assumptions
and relative weights
assigned to factors that make up the index need to be explicitly stated so that users can
interpret the index and decide whether they can apply it in their setting. For example, in
Stason and Weinsteins
[12] important
economic analysis of resource allocation
in the
management
of hypertension,
morbidity (quality of life and side effects) and mortality due
to hypertension
were combined
in an index called quality
adjusted life expectancy.
Information
on the assumptions
and weighting used to adjust life expectancy for morbid
events are provided in the paper.
The use of indices that combine morbidity
and mortality
have the potential
to be
exceedingly helpful in quantifying
the burden of illness and assessing the relative impact
of different diseases. Such indices quantify and then combine the healthy days of life lost
due to both the disability and death associated with a specific disease. As a result they allow
one to summarize the overall impact of a disease in one number and the use of a common
unit of measure allows easy comparison
between different diseases. Weighting
of the
disability component
according to severity can be used to increase the sensitivity of the
index to the true impact of an illness on an individual.
For example, although two persons
with different diseases may spend the same duration of time in an unwell state, the burden
of one disease may be much greater for one than for the other-weighting
adjusts for this
difference in severity.
However, the operationalization
and use of such indices is often not straightforward.
Adequate data on disability days and years of life lost may not be easily obtainable
and
determining
weights is a complex and controversial
task requiring a number of assumptions to be made [13, 141.
Burden of illness can be sub-divided
into that which is avoidable and that which is not
(Fig. 2).
(a) Unavoidable

burden

of illness

Unavoidable
burden
consists of disability,
symptoms
and mortality
for which no
prevention or cure exists. Resource allocation here should focus on research into
etiology, prevention
and cure, and not just on care of current victims.
eficacious

(6) Avoidable

burden

of illness

Avoidable
burden consists of disability, symptoms and mortality for which eficacious
prevention
or therapy exists. Resource allocation
here should focus on health care of
proven efficacy or research into community
effectiveness
and efficiency of a known,
efficacious intervention.

PETER TUGWELL et al.

342

BURDEN
Avoidable

ENVIRONMENT

OF ILLNESS
vs Unavoidable

FIG. 3.

FIG. 2.

However, to make this distinction

requires knowledge
about possible
burden and whether efficacious preventive,
therapeutic,
or rehabilitation
available. The next two steps of the loop address these issues.

MEASUREMENT

LOOP

STEP

NO.

causes of the
strategies are

2-AETIOLOGY

The second step focuses on determining

the causes of health problems contributing
to
the burden identified in Step 1.
Elucidation
of causal factors requires a careful review of all the biological
and
behavioural
attributes
that might contribute
to the problem; the hypothesis
space is
explored and the most likely potential causes assessed (Fig. 3). This involves a wide array
of techniques ranging from the laboratory
to the clinic and the community.
For multiple
causal factors, the relative contribution
of each to a health problem is difficult to estimate,
but is crucial given the commitment
of time and money to developing
interventions
to
reverse their effects.
Guidelines
for assessing evidence for causation have been developed. These originated
from the work of Robert Koch in infectious
diseases [15]. Virologists
subsequently
modified
Kochs guidelines,
introducing
the key concept
of using an experimental
approach wherein the investigator
controls exposure to the putative cause. Subsequently
these guidelines were modified for use in non-infectious
disease by Bradford Hill [16] and
further developed [17] by the Department
of Clinical Epidemiology
and Biostatistics
at
McMaster
University
(Table 1).
There is currently
substantial
research into the pathogenesis
of hypertension
and
osteoarthritis
that may lead to prevention or early detection. However, some of the burden
associated with each is potentially avoidable and the subsequent steps in the loop will focus
on identifying
interventions
for minimizing
the burden once these conditions
develop.

TABLE I. THY N,NE D,AG~OST,CTESTSFOR CAUSAT,ON

I.

2.
3.
4.
5.
6.
7.
8.

9.

Is there evidence from true experiments in humans?

Yes= Class I Evidence: randomized controlled tnal
No = Class 2 Evidence: cohort studies; before-after studies
Class 3 Evidence: case-control studies
Class 4 Evidence: descriptive studies
Were the major sources of bias avoided or if present measured?
iti
Were the sampling, assessment of exposure and analysis at an acceptable level?
Is the association strong? Is the association stronger than for alternative explanations?
Do other investigators consiszenfl~~find this sxne result?
Is the remporal relationship in the proper direction?
Is there a gradient or dose-response relationship?
Does the association make epidemiologic sense.~
Is the association biologically smsible~
Is the association specific?
Is the relationship analagous to another, well accepted relationship?

The Measurement

MEASUREMENT

LOOP

STEP

NO.

Iterative

Loop

3-COMMUNITY

343
EFFECTIVENESS

This step looks at information

on how well an intervention
with potential for reducing
burden (by preventing
or modifying factors causing illness) will work when applied in the
community.
Disability
and disease for which effective interventions
exist can then be
identified as avoidable.
Community
effectiveness is determined
by five factors.

Efficacy asks the question Can it work? It is defined as the extent to which a health
intervention
does more good than harm to patients (citizens) who are diagnosed correctly,
appropriately
cared for and fully comply with recommendations
for treatment.
That is,
evaluation of efficacy assumes optimal diagnostic accuracy and health provider and patient
compliance.
Careful attention
to these factors is required when deciding whether a study
is truly assessing efficacy. Efficacy is therefore the anchor point when estimating
the
benefit of interventions
applied in a community
setting, giving us the maximum benefit that
can be achieved since it is more stable and less liable to fluctuate in different circumstances
than the other four components
of community
effectiveness.
Do we have evidence of efficacy for the treatment
of hypertension
and osteoarthritis?
Such studies should satisfy basic methodological
criteria before they are used for health
policy decisions [I 81.
Hypertension
is one of the best examples of a condition
studied in randomized
trials.
Five such studies demonstrate
that antihypertensive
drug treatment does more good than
harm to patients under optimal conditions of diagnostic accuracy and patient and provider
compliance
[19-231. In all, the morbidity
and mortality
associated with elevated blood
pressure was demonstrated
to be substantially
reduced. Furthermore,
these studies met
most of the methodologic
guidelines alluded to above: they were all randomized
controlled
trials; outcomes were measured appropriately
(blind assessment of blood pressure control;
morbidity,
mortality and side effects assessed); patients were well described; clinical and
statistical significance were demonstrated;
the therapeutic
manoeuvre
was described with
sufficient detail; the studies were analysed appropriately.
On the other hand, there are no randomized
trials available for joint replacements
in
osteoarthritis
and we must rely on before-after
follow-up studies. Total hip arthroplasty
in severely disabled patients (60 years and older) results in dramatic improvements
in pain
and activities of daily living (good or excellent results in over 86% of patients compared
with prior to surgery) and this type of evidence is regarded by many as sufficient [24-261.
However, in patient groups that are younger or at an earlier stage of disease, randomized
trials would be important
to carry out given that disability in this group may be minimal,
pain may be relieved by analgesics and the complications
due to surgery, loosening and
other failures of the joint may be substantial.
(h) Screening

and diagnostic

accuracy

Information
on the accuracy of screening and diagnosis can be obtained from studies
of the extent to which patients with the condition
of interest are correctly discriminated
from those without
it. This is not confined
to tests involving
technology
or the
laboratory
and includes the assessment of the accuracy of clinical signs (history, physical
examinations,
etc.) or other para-clinical
investigations
(laboratory,
X-ray, etc.). For
example, in identifying
patients who need hip replacement,
diagnosis involves finding
patients with symptoms unresponsive
to analgesics and radiographic
evidence of structural
damage to the joint, whilst the diagnosis
of hypertension
needing treatment
requires
multiple blood pressure readings taken over a period of time and assessment of clinical
and laboratory
evidence of target organ damage.
Unavoidable
health needs (i.e. those for which no therapy is currently available) should
not be the focus of screening, not only because of expense but also because of the negative

PETER TUGWELL et al

344

health consequences
of the labelling
that results. A study involving careful follow-up of
steelworkers
screened for hypertension
revealed that absenteeism
rose among previously
unaware hypertensives
labelled as a result of the screening process [27]; this led the authors
and three Canadian task forces on hypertension
to recommend that hypertension
detection
should only be carried out in settings where adequate therapy and long-term follow-up was
ensured.
In this way the disadvantages
associated
with being labelled hypertensive
could be countered
by the long term benefits of blood pressure control.
(c) Evaluation

of health provider

compliance

This focuses on whether the appropriate

diagnostic
and management
(prevention,
therapy and rehabilitation)
actions are complied with by the health provider. The literature
on Quality of Care calls these actions clinical
process
to discriminate
them from
structure
(the supply of facilities and qualified personnel)
and outcome
(patients
health status). Studies of health provider compliance
should be restricted to situations
where the causal relationship
between the process of care and patient
outcome
is
established
i.e. demonstration
of efficacy.
The Burlington Nurse Practitioner
Trial assessed quality of care in a primary health care
setting in the treatment
of hypertension
[28]. Adequacy
was assesed using indicator
conditions.
The care given by the nurse practitioner
was compared with that of the private
practice physician and it was found that 56% of all episodes of hypertension
handled by
the nurse practitioner
were judged adequate compared with 67% for the physicians.
These studies are particularly
important
for monitoring
harmful aspects of care; for
example, it has been noted that facilities performing
fewer than 50 hip replacements
per
year have a higher mortality
than those performing
higher numbers [29].
(d) Evaluation

of patient

compliance

This focuses on whether patients comply with the health providers recommendations
and treatment.
For hip replacement
surgery, patient non-compliance
with recommendations for surgery may occur due to fear or lack of understanding
of the potential benefits
~241.
For hypertension,
strategies to measure patient compliance
in taking their medication
include pill counts conducted in the clinic or home setting. Riboflavin tablets, which cause
the urine to fluoresce were used in the V.A. trials to screen out potentially
non-compliant
patients
[19,20]. A number
of studies have been conducted
to assess strategies for
improving
patient compliance
such as those conducted
among steelworkers
[30].
(e) Evaluation

qf coverage

Coverage refers to the extent to which the efficacious manoeuvre,

procedure or service
is being appropriately
utilized by all those who could benefit from it. Coverage should be
discriminated
from patient compliance--coverage
describes whether or not the individual
makes contact with the health professional,
whilst patient compliance
encompasses
the
adherence of the patient to the subsequent advice received. The determination
of coverage
assumes that efficacy is established
and should not be done in its absence.
Coverage evaluation
requires that the use of health services be related to the need for
them in a defined population
during a specified time period. Utilization
of services in the
form of activity to population
ratios, although very popular, are rarely accurate as a
measure of availability
since they fail to incorporate
information
about true need.
Appropriate
utilization
depends upon availability
and acceptability
of effective health
services.
Availability of efective health services. This focuses on whether efficacious health services
are accessible (get-at-able)
to those in need and they are aware of them being available.
This can be measured by estimating the supply of services (the resource/population
ratio)
taking into account distribution.
Awareness of the availability
of services is also relevant
here. For example, there may be a limitation
in surgical resources available to perform
joint replacements
in some settings. Availability
of health services is also important
in

The Measurement

Iterative

Loop

34.5

screening studies of hypertension

to ensure that there is appropriate
linkage of positive
screenees to the health provider for treatment
and not just labelling
[27].
Acceptability
of health services. Quantification
of this factor can best be obtained from
surveys carried out in the general population
(and should not be confined to users of health
services, since the latter does not tell us how many individuals
in need of an intervention
fail to receive it). Measurement
of acceptability
of health services can usefully be divided
into patients perceptions
of (i) the resources or facilities (ii) the behaviour
of health
professionals
and their staff and (iii) the expected benefits of the health service.
Prediction of the magnitude of community effectiveness. It is important
to be able to
estimate the impact of specific treatment interventions
when assessing whether a program
is achieving its full potential,
or when assessing the economic efficiency of the whole
program
or looking at alternatives
for improving
the program.
All of the economic
approaches
described in the next step of the loop except for cost-minimization
require an
estimate
of community
effectiveness.
The relationship
between the five factors that
determine community
effectiveness is most accurately estimated by using a multiplicative
conditional
probabilities
model (See Appendix).
Unfortunately
the necessary information
on conditional
probabilities
is rarely available in which case an acceptable alternative
[12]
is to use a simple multiplication
formula. This assumes that all the factors are independent.
It is unlikely that the factors are highly correlated, (e.g. many patients given optimal care
do not comply [31]), but research is needed to confirm the robustness
of the simple
multiplicative
formula.
Table 2 shows some sample calculations
for our two examples using the simple
multiplication
formula. The purpose of these sample calculations
is to illustrate the effect
on efficacy estimates
(obtained
under ideal circumstances)
of the other factors that
influence
community
effectiveness.
These calculations
are best estimates
based on
currently
available evidence and we fully appreciate
that they may be subject to error
particularly
for hip replacement
given the current lack of necessary community-based
data.
However, we feel that they serve the purpose of illustrating
the relative magnitude
of the
difference between efficacy estimates for an intervention
and its impact when implemented
under community
conditions.
Furthermore,
by examining
each of the individual
components of community
effectiveness we can identify the ones that, if improved would have
the greatest impact on increasing community
effectiveness.
For hypertension,
our estimate of efficacy comes from the V.A. trials of antihypertensive
drugs against placebo [19,20]. The results of these studies suggest we could expect a 76%
TABLE2. SAMPLECALCULATKJNS
FOR COMMUN~YEFFECTWENESS

Example

% of efficacy
achieved in
community

Efficacy

Diagnostic
accuracy

Provider
compliance

Patient
compliance

Coverage

Community
effectiveness

76/*
/o

95%

66%

65%

90%

28%1

$37%

76%:

95%

90%

90%

90%

53%1

$=70%

60/**0

75%

98%

87%

70%

27%~

27
G=45%

60/**
0

90%

98%

87%

90%

41%T

Z=68%

Hyperrension

under
currenl
conditions

(a)

(b) under
conditions
of improved

patient and
provider
compliance
Osteoarrhritis
(a) under
current

conditions
(b) under
conditions
of improwd
diagnostic
accuracy and
coverage

*Reduction in ail morbid events.

**Improvement in function.
CD 38iGE

346

PETERTUCWELL et al.

reduction in all morbid events associated with hypertension

in patients with a diastolic
blood pressure of > 90 mmHg. Estimates for the other components
of the equation are
based on recently published reports of hypertension
in the community
[8,9,30-321.
For
others making estimates for use in their own setting it would often be worth while carrying
out local checks on compliance
and coverage to ensure accuracy (given that the cost of
the expansion of health services warranted the expense of conducting
the check). Efficacy
should be reasonably
stable and therefore generalisable
to most populations.
Assuming that the diagnosis of hypertension
is based on 3 consecutive
blood pressure
readings
>95 mmHg (as it was in the efficacy trials) we could expect that 95% of all
hypertensives
could be correctly identified (that is, 95% sensitivity of 3 blood pressure
readings). Provider compliance
in administering
treatment
adequate for the control of
hypertension
is estimated at 65% (this includes the decision to treat plus provision of a
treatment
regimen of sufficient vigour to control the patients hypertension).
Adequate
patient compliance (defined as > 80% of prescribed medication
consumed) is estimated at
66% of hypertensives
treated. Finally, coverage of medical care to hypertensive
patients
is estimated at 90%. Using these figures the current impact of treatment for hypertension
on hypertensives
in the community
is estimated to be a 28% reduction in all morbid events
associated with hypertension
(See Table 2). This represents only 37% of the maximum
possible impact estimated by the efficacy component.
In other words 63% of the potency
of hypertensive
therapy is lost due to the other factors that influence its impact on
hypertensives
in the community.
When we examine the individual
components,
patient and provider compliance appear
to be the major limiting constraints
to community
effectiveness. If these could be raised
to a more optimal level (90% for each), and it seems reasonable that this could be achieved
given the appropriate
motivation
and commitment
of those involved,
community
effectiveness could be improved to 53 or 69% of the maximum possible impact based on
efficacy estimates. This finding is consistent with the work by Stason and Weinstein [12]
showing that strategies to improve patient compliance
have the greatest potential
for
increasing
the impact of treatment
for hypertension
on the community.
For hip replacement
in osteoarthritis
published studies suggest that a 60% improvement
in function can be expected after hip replacement
[24-261. Diagnostic accuracy (based on
X-ray and clinical symptoms)
is judged to be about 75%. The source of the 25% false
negative rate is thought to be at the primary care level as this is where OA patients are
first seen. Provider compliance
is estimated to be adequate in 87% of cases (this includes
appropriate
referral at the primary care levels as well as care at the tertiary level) [41];
patient compliance
is thought to be around 98% based on experience at the tertiary care
level [41]. However, it is recognized that it could be somewhat less given that patients seen
at the primary care level may not accept their physicians recommendation
for referral to
a specialist for possible surgery. Coverage of medical services to osteoarthritis
patients is
estimated at 70% [41]. Using these figures, the current impact of hip replacement
on OA
patients in the community
is estimated to be a 27% improvement
in function (See Table
2). This represents only 45% of the maximum
possible impact estimated by the efficacy
component-or
55% of the potency of hip replacement is lost due to the other factors that
influence its impact on OA patients in the community.
Again, when we examine the individual
components,
diagnostic accuracy and coverage
appear to be the major limiting constraints
to community
effectiveness. If these could be
raised to a more optimal
level (again, 900/, for each seems reasonable)
community
effectiveness could be improved to 41 or 68% of the maximum possible impact based on
efficacy estimates.
It should be noted that for simplicity we have ignored the iatrogenic effects of diagnostic
labelling and the complications
of therapy in these examples. For example, hypertensive
drug side effects leading to decreased quality of life would be subtracted from the benefit
if expressed as quality-adjusted
life-years.
These estimates also do not tell us to what extent the community as a whole would be
better off if an intervention
were implemented.
The reduction
in the overall burden of

The Measurement

Iterative

Loop

347

illness experienced by the community

will depend upon two factors. First, the proportion
of the total burden accounted
for by hypertension
or osteoarthritis
(for example) will
determine
the overall community
impact of interventions
targeted at these disorders.
Second, the effect of competing risks will determine the overall influence on the health of
the community.
Individuals
who avoid hypertension
related disability
and death may
simply succumb to some other disease such as cancer.

MEASUREMENT

LOOP

STEP

NO.

4-EFFICIENCY

This step provides information

on whether the intervention
is being delivered to those
who would benefit from it with an optimal use of resources and involves the relationship
between costs and effects.
EFFICIENCY

= PATIENT

BENEFIT

(OUTCOME)/COST

or
NET

COSTS($)

- NET

BENEFITS(S)

Efficiency is expressed as the effects (number of lives saved, number of disability days
avoided) obtained
for a specific cost (expressed in dollars). Again, the assessment
of
efficiency should not be conducted in the absence of evidence of efficacy and community
effectiveness. Other issues that should be considered in the assessment of efficiency have
been discussed by Stoddart
[33,34]. The most widely used approaches
are: (i) Costeffectiveness-effectiveness
is measured in a common unit of health such as lives saved,
levels of function or proportion
of patients in whom symptoms are controlled. This may be
derived from ejicacy studies in a tightly controlled situation or from studies incorporating
variable numbers
of the other components
that comprise community effectiveness;
(ii)
Cost-benefit-effectiveness
(benefit) is measured in the appropriate
unit of health and then
converted into monetary
units, i.e. dollars and cents and (iii) Cost-utility-effectiveness
(utility) is measured in the appropriate
unit of health (lives saved, functional improvement,
symptom control) and then converted into utility units that are measures of the relative
social value (importance)
of these outcomes.
The cost-effectiveness
of alternative
hypertension
treatment
strategies has been investigated in the Canadian
setting comparing work site care with regular care from family
physicians. This randomized
trial looked at both the effectiveness and efficiency of the two
modes and showed that work site care was substantially
more cost-effective
[35]. The
techniques of cost-effectiveness
analysis have been applied to the problem of how best to
allocate resources to manage hypertension.
The results of this work have suggested that
the most cost-effective
allocation
of health care dollars is to strategies which improve
adherence to treatment
rather than screening procedures
[12].
The issues involved in conducting
an analysis of the cost-effectiveness
of hip replacement
in arthritis have been reviewed by Bentkover
et al. [4]; although they conclude that a
cost-effectiveness
study is indicated this has not yet been commissioned.
MEASUREMENT

LOOP

STEP

NO.

5-SYNTHESIS

AND

IMPLEMENTATION

The fifth step integrates feasibility with the estimates of community

effectiveness and
efficiency obtained in the previous steps to make recommendations
for implementing
an
intervention.
This step identifies the possible limiting constraints
on the effectiveness and efficiency
estimates currently
existing in the setting where the intervention
will be implemented,
whether and to what extent these constraints
can be removed or reduced, and comes to
a conclusion as to the likely impact of the intervention
or program on the burden of illness.
Constraints
include
social, cultural
and political
barriers
as well as adequacy
and
availability
of facilities and manpower
and budgetary
considerations.
For example,
increased efforts to identify osteoarthritis
patients who could benefit from surgery would

PETER TLJGWELL et al

348

be futile if there were inadequate

manpower and surgical facilities to perform the implants
in a reasonable
period of time.
Inherent
in this step is the setting of targets, important
for both defining realistic
objectives
and for monitoring
success. The estimates of community
effectiveness
and
efficiency can be used later as a target to assess the success of the intervention.

MEASUREMENT

LOOP

STEP

NO.

6pMONITORING

The sixth step is ongoing monitoring

of a health program.
Monitoring
needs to be
tailored to the individual
program and may consist of short, intermediate
and long term
criteria of success selected to profile progress. These monitoring
methods usually focus on
one or more of the following categories: (i) structure: buildings built and equipped and
qualifications
of health workers; (e.g. availability
of facilities and personnel
for blood
pressure control and hip surgery); (ii) health care/administrative
process: the appropriateness of case finding and care of hypertensive
patients and patients with osteoarthritis
of the hip; and (iii) patient/citizen
health outcomes: changes in symptoms, disability and
mortality (e.g. quality of life of patients with treated hypertension
and hip replacements).
The selection of markers needs to take into account representativeness
of the spectrum of
disease and age groups, accuracy and feasibility [36]. Rutstein et al. have suggested a
promising strategy for monitoring
quality of care and patient outcomes by identifying
2
classes of marker diseases or disability that can be used as indices of quality of care. In
the first class, a single incident of the marker disease or disability would be cause for
concern; in the second, critical increases in rates of disease or disability serve as indicators
[371.
MEASUREMENT

LOOP

STEP

NO.

7-REASSESSMENT

Reassessment
of health needs and the burden of illness is essential to assess the overall
success of the components
of the measurement
loop, thus closing the loop by returning
again to the residual burden of illness for hypertension
and hip osteoarthritis.
This entails
periodic surveys to determine
the adequacy
of blood pressure control or number of
disabled in the population
from hip osteoarthritis.
Such a survey to reassess blood pressure
control in a Canadian
community
several years after recommendations
made by the
Canadian
Hypertension
Task Force were published
showed considerable
improvement
over prior surveys [32].
Can the loop be usefully applied to clinical and policy decisions?
The demonstrated
usefulness of the loop approach in clinical and policy decisions can
be illustrated
in two ways. First, the loop can be applied to the available evidence on
hypertension
and osteoarthritis
to draw clinical and policy conclusions
and make
recommendations
for needed research. Hypertension
is one example where adequate
information
exists for each step in the loop to the extent reasonable and possible. Currently
available evidence should provide a sound foundation
in most settings for clinical and
policy conclusions.
Community
surveys consistently
show that between 5 and 15% of
adults have high blood pressure depending
on age and race [9]. Efficacy has been
studied in several rigorous randomized
controlled
trials. The estimated benefit ranges
upwards from a 76% reduction
in all morbid events depending
on the initial level of
diastolic pressure. These studies of efficacy are probably generalizable
to most settings in
industrialized
countries. Estimates of the other components
of community
effectiveness are
more likely to vary with the local type of health care system but it should be possible to
approximate
these or carry out quick surveys to obtain this information.
Since the
economic analysis for hypertension
has been done, the situation specific assumptions
such
as relative changes in costs could be modified to assess the cost-effectiveness
of concentrating on compliance
rather than screening. The development
of newer, more efficient
ways of monitoring
the control of hypertension
and disability will allow more frequent

The Measurement

Iterative

Loop

349

reassessments of a representative sample of the population but at present it is probably

reasonable to use a yearly assessment of a sample of the population as is done in the
U.S. National Health Survey.
Osteoarthritis is one example where, by applying the loop to assemble the available
information we can identify important gaps in our knowledge and make recommendations
for research. For joint replacement, the major gap is the lack of efficiency data.
Information from such studies will contribute greatly to clinical and policy decisions in
this area.
Second, several examples exist where substantial problems could have been avoided
through the use of the loop in formulating clinical and policy decisions about the
management of hypertension and hip osteoarthritis. For example, the planning of a
shopping plaza screening programme for hypertension could have benefitted significantly
from the loop. Although a significant number of previously unidentified hypertensives were
diagnosed, the failure to link these individuals to a source of care resulted in very little
health benefit due to screening [8]. In fact the programme may have done more harm than
good due to the deleterious effects of being labelled hypertensive [27].
Another example from the hypertension literature is a recent report showing that the
recommendations of the various Canadian task forces in hypertension were not being
followed by many physicians [38].
Finally, a programme for rheumatic disability (including osteoarthritis) in the community might have improved its effectiveness by applying the loop in its planning phases.
This program identified persons with rheumatic disability and assisted these individuals
in obtaining health services and co-ordinated care [39]. Although utilization of services and
referrals increased, no improvement in function was observed. It is possible that the
programme might have had more impact had individuals been linked with specific
interventions known to be efficacious in the treatment of their specific disability.
CONCLUSIONS

It is fully appreciated that only rarely (or never) will ideal information be available.
However, the amount of evidence available to decision-makers can be substantial as our
two examples illustrate. Nevertheless, because there are gaps in the evidence it is all the
more important to systematically organize available information and explicitly identify the
potential gaps so that an informed decision may be made, whether it be a recommendation
for funding or further study.
Acknowledgements-We
wish to thank Professor Robin S. Roberts, Department
of Clinical Epidemiology
and
Biostatisticsfor his contribution to the development of the community effectiveness multiplicative model.

REFERENCES

5.
6.
7.
8.
9.

Cochrane AL: Effectiveness and Efficiency; Random Reflections on Health Services. The Neuffield Provincial
Hospitals Trust, 1972
Sackett DL: On the Evaluation
of Health Services. In Preventive Medicine and Public Health, Last J (Ed.)
11th edn. New York: Appleton Century Cofts, 1980
Evans JR: Measurement and Management in Medicine and Health Services. Mimeo New York: Rockefeller
Foundation
1981
Bentkover JD, Drew PG, Little AD: Cost Benefit/Cost
Effectiveness of Medicine Technologies:
A Case
Study of Orthopedic
Joint Implants Case Study No. 4, Background Paper No. 2 Case Studies of Medical
Technologies. Washington,
D.C: Congress United States, Office of Technology
Assessment,
1981
Health and Welfare Canada and Statistics Canada: The Health of Canadians: Report of the Canada Health
Survey, June 1981. Publications
Distribution,
Statistics Canada (Catalogue
82-538E)
Kellgren JM: Osteoarthritis
in patients and populations.
Br Med J July 1, l-6, 1961
Fodor JG, Abbott EC, Rusted IE: An epidemiologic
study of hypertension
in Newfoundland.
Can Med
Ass J 108: 136551368, 1973
Baitz T, Shimizu A, Johnson AL, Taylor DW: The Hamilton
High Blood Pressure Screening Projects.
Personal Communication.
Hypertension
Detection and Follow-up
Program
Co-Operative
Group: Five year findings of the hypertension detection and follow-up program
I: Reduction
in mortality
of persons with high blood pressure
including mild hypertension.
JAMA 242: 2562-2571,
1979

350

10.
11.
12.
13.
14.
15.
16.
17.
18.

19.
20.

21.
22.
23.
24.
25.
26.
27.
28.
29.
30.

31.
32.
33.
34.
35.
36.
37.
38.
39.
40.
41.

PETER TUGWELL

et al

Kohn R, White KL (Eds): Health Care: An International Study. Toronto: Oxford University Press, 1978
Melton LJ, Stauffer RN, Chao EY, Ilstrup DM: Rates of total hip arthroplasty:
A population
based study.
N Engl J Med 307: 124221245, 1982
Stason WB, Weinstein MC: Allocation
of resources to manage hypertension.
N Engl J Med 296: 7322739,
1977
Rosser RM. Kind P: A scale of valuations
of states of illness: Is there a social consensus.
Int J Epid 7:
347-358, 1978
Rosser R: Recent studies using a global approach to measuring illness. Med Care 14 (Suppl.): 138-147, 1976
de Raoville W: Medical Classics, 2: 8.53-880, 1973-1978. [English Translation of original paper: Koch R:
Etiologic de Tuberculose.
Berlin Kiln Wochenschir 19: 221-250, 1882
Hill AB: Principles of Medical Statistics, 9th edn. New York: Oxford University Press, 1971
Department
of Clinical Epidemiology
and Biostatistics,
McMaster
University,
Hamilton:
How to read
clinical journals
IV: To determine eiiblogy or causation.
Can Med Ass J 1241 985990,
1981
Department
of Clinical Epidemiology
and Biostatistics,
McMaster
University,
Hamilton:
How to read
clinical journals
V: To distinguish
useful from useless or even harmful therapy. Can Merl Ass J 124:
11561162,
1981
Veterans Administration
Co-Operative
Study Group on Antihypertensive
Agents I: Results in patients with
diastolic blood pressure averaging
115 through
129 mmHg. JAMA 202: 102881034, 1967
Veterans Administration
Co-Operative
Study Group on Antihypertensive
Agents: Effects of treatment of
morbidity
in hypertension:
II. Results in patients with diastolic blood pressure averaging
90 through
114mmHg.
JAMA 213: 1143-1152,
1970
Wolff FW, Lindeman RD: Effects of treatment in hypertension:
Results of a controlled study. J Chron Dis
19: 227-240, 1966
Hypertension-Stroke
Co-Operative
Study Group: Effect of antihypertensive
treatment on stroke recurrence.
JAMA 1974 229: 4099418, 1974
Smith WMcF: The Public Health Service Hospitals Study. Annual Meeting of the Council for High Blood
Pressure Research,
1976
Liang MH, Cullen KE, Poss R: Primary total hip or knee replacement:
Evaluation of patients. Ann Int Med
97: 735-739, 1982
Harris WH: Current concepts: Total joint replacement.
N Engl J Med 279: 650-651, 1977
Beckenbaugh
RD, Instrup DM: Total hip arthroplasty:
A review of 333 cases with long follow-up. J Bone
Joint Surg (AM) 60-A: 306-313, 1978
Haynes RB, Sackett DL, Taylor DW, Gibson ES, Johnson AL: Absenteeism
from work following the
detection and labelling of hvpertensives.
N Engl J Med 299: 741-744, 1978
Spitzer WO, Sackett DL, Sibley JC, Roberts RS, Gent M, Kergin DJ et (11: The Burlington
Randomized
Trial of the Nurse Practitioner.
N Enul J Med 290: 251-256, 1974
Luft HS, Bunker LP, Enthoreng
AC: Should operations
be regionalized?:
The empirical relation between
surgical volume and mortality.
N Engl J Med 301: 13641369,
1979
Haynes RB: A review of tested interventions
for improving compliance
with antihypertensive
treatment.
In Patient Compliance to Prescribed Antihypertensive Medication Regimens: A Report to the National Heart
Lung and Blood Institute, Haynes RB, Mattson ME, Engebretson
TD (Eds). US Department
of Health and
Human Services NIH Publication
No. 81-2102, October 1980. pp. 83-l 11
Haynes RB, Gibson ES, Taylor DW, Bernholtz CD, Sackett DL: Process versus outcome in hypertension:
A positive result. Circulation 65: 28-33, 1982
Birkett NJ, Donner A: Effect of follow-up visits on estimates of the prevalence and degree of control of
hypertension
in the community.
Clin Res 31: 294A, 1983
Stoddart
GL, Drummond
MF: Clinical epidemiology
rounds. How to read clinical journals VII: To
understand
an economic evaluation
(Part A). Can Mid Ass J 130: 142881434, 1984
Stoddart
GL, Drummond
MF: Clinical epidemiology
rounds. How to read clinical journals
VII: To
understand
an economic evaluation
(Part B). Can Med Ass J 130: 1542-1549,
1984
Logan AG: Cost-effectiveness
of a worksite hypertension
treatment program. Hypertension 3: 21 I-218, 1981
Tugwell P: A methodologic
perspective on process measures of the quality of medical care. Clin Invest Med
2: 113-119, 1979
Rutstein DD, Berenberg W, Chalmers TC, Child CG, Fishman AP, Perrin EB: Measuring the quality of
medical care: A clinical method. N Engl J Med 294: 582-588, 1976
Evans CE, Haynes RB, Gilbert JR, Taylor PW, Sackett DL, Johnston
M: Educational
package on
hypertension
for primary care physicians.
C Med Ass J 130: 719-722, 1984
Liang MH, Phillips EE, Scamman
CS, Lurye CS. Keith A, Cohen L, Taylor G: Evaluation
of a pilot
program for rheumatic disability m all urban community.
Arth Rheum 24: 937-943, 1981
Colton T: Statistics in Medicine. Boston: Little, Brown and Co. 1974. pp. 6673
Liang MH: Personal communication

The Measurement

Iterative

Loop

351

APPENDIX
CONDITIONAL

PROBABILITIES

Multiplicative law of combining conditional probabilities [40]

This states that the chance of two events x and y both happening P(x and y) = P(x) Z(J) assuming both events
are independent.
When one event is dependent
upon the other such as y is dependent upon x this is expressed
as P(x and y) = P(y/x) P(x). Community
effectiveness can therefore be stated as follows:
=
x
x

Community
effectiveness*
(or probability
of benefit)

Under

the assumption

of independence

Community
effectiveness

Efficacy

*Determinants
of community
effectiveness
of delivering and receiving health care.

P(Coverage)
P(Diagnostic
accuracy/coverage)
P(Health
provider compliance/
coverage and diagnostic accuracy)
P(Efficacy of treatment/coverage
and diagnostic
accuracy and health
provider compliance)
P(Patient
compliance/coverage
and diagnostic
accuracy and health
provider compliance
and efficacy
of treatment)

this can be simplified

x

Diagnostic
accuracy

are organized

to:
Health
provider
compliance

according

Patient
compliance

to the sequence

Coverage

they occur in, in the process