What is Best Practice?

Caroline Badger
Conference Presentation

British Lymphology Society

Bristol
October 2001

1
In this paper I am going to talk about what we mean by the term Best Practice
and then go on to explore whether we know what best practice is when it comes
to managing lymphoedema.

Over the last 20 years or so, an array of measures have sprung up in the Health
Service as the result of the move to develop evidence-based health-care. We
have had the introduction of Quality Assurance with its setting of standards and
the encouragement to conduct clinical audit of these. We have seen the
establishment of NICE (The National Institute for Clinical Excellence), whose
focus is the development of Clinical Guidelines; the function of which, to use
their own words, is to help health professionals and patients make the right
decisions about health care in specific clinical circumstances. There has been
the setting up of The Commission for Health Improvement with a remit to
develop clinical governance. In case like me you struggle with the term, here is
the Commissions own definition of what is meant by clinical governance:

A framework through which NHS organisations are accountable for continuously

improving the quality of their services and safeguarding high standards of care
by creating an environment in which excellence in clinical care will flourish.

The Commission intends to conduct reviews of every NHS trust and Health
Authority to assess the degree to which they meet their obligations to do this.
Talking of frameworks we also have the development of National Service
Frameworks, a definition of these couched in the Department of Healths own
inimitable style is as follows:

(The national service frameworks) set national standards, define service models
for a specific service or care group, put in place programmes to support
implementation, and establish performance measures against which progress
within an agreed time-scale will be measured.

2
The idea is that they will ensure that patients will receive greater consistency in
the availability and quality of services across health and social services. They
define explicit standards and principles for pattern and level of services
required.

The point about all these initiatives is that they reflect our desire as Health
Professionals to improve the quality and effectiveness of the services we provide
for our patients. There is a mass of weighty literature on this subject and I am not
going to bore you by trawling through it but if we are going to answer the
question of what is meant by best practice there are certain things that we must
appreciate.

In an excellent book on the subject of evidence-based healthcare, Muir Gray 1

explained that interventions can be categorised as:
1. Doing more good than harm
2. Doing more harm than good
3. Of unknown effect

Interventions of unknown effect should, he says, be discontinued, until proven to

cause more good than harm. He goes on to point out that the concept of good
includes safety and acceptability as well as effectiveness. And that harm is often
overlooked as a possible consequence when professionals champion a new
service or intervention. Finally, he highlights the word more and argues that the
magnitude of any difference described by the term is as important as the
existence of a difference. This notion of magnitude is reflected in the phrase
best practice - we are not interested merely in good practice but better or even
best practice.

1
Evidence-based Healthcare How to Make Health policy and Management Decisions. J.A. Muir
Gray. Churchill Livingstone 1997

3
As a beginning to answering our question what is meant by best practice?, we
could say that it is practice which produces the greatest good for the least harm
and, because we know that no health care system has unlimited resources, we
also have to add in, at the most affordable price.

There is a key factor that we havent as yet mentioned, but which underpins all of
the initiatives cited above, including the concept of best practice and that is, of
course, evidence. In order to determine the best way of doing something we
need to move away from the individual whims, opinions and wishes of those
involved in delivering and receiving care, towards evidence of the effectiveness
of that care. And evidence of its effectiveness not just under controlled
experimental conditions but in the real live clinical setting. It is here that we get
to the heart of the matter, for what constitutes evidence?

There is an accepted hierarchy of evidence which is graded from strongest to

weakest, starting with level 1.

Level 1 is strong evidence from at least 1 systematic review of multiple, well-

designed, randomised controlled trials.

Systematic Reviews were introduced into medicine as a way of sifting,

simplifying and synthesizing information. The idea is that you formulate a specific
question that needs answering; you set out explicit methods for identifying
research that might help you answer the question; you then search for, select
and appraise the evidence in a systematic way; having selected the studies that
you can use in the review you collect the data from them and analyse it.
Essentially the process itself is research and as such should be conducted
according to a pre-defined protocol. This systematic approach is needed to
minimise biases and random errors. Let us look an example of a possible source
of bias. There is evidence that positive results tend to get published while
negative findings do not, and that positive results tend to get published in English

4
language papers rather than foreign publications. So that only looking at
published papers in English journals might lead you to conclude that a particular
intervention was more effective than it really was.

A major force in this approach to establishing evidence is the Cochrane

Collaboration. Cochrane is an international project set up to prepare, maintain
and promote the accessibility of systematic reviews. There are Cochrane
Centres all around the world and the collaboration provides online access to its
library and databases. Working together with Cochrane, the RCN Institute has
established a yearly systematic review fellowship to develop systematic reviews
in nursing. Last year I applied for and was awarded such a fellowship to carry out
3 reviews concerning the effectiveness of treatments for lymphoedema. The first
concerns the use of benzo-pyrones in the treatment of lymphoedema. The
second looks at the use of prophylactic antibiotics in preventing recurrent acute
inflammatory episodes in lymphoedema and the third looks at the use of physical
therapies in treating lymphoedema. I have just reached the stage of analysing
data from the studies included in the first of these reviews and so dont as yet
have any results to give you but all 3 should be completed and published by the
end of the year and will also be accessible online through the Cochrane Library.
The important point about systematic reviews, is that the evidence from them is
only as good as the quality of the trials included in them. As the excellent,
evidence-based newsletter Bandolier 2 so succinctly puts it: Garbage in,
garbage out: if the studies included have been poorly designed or poorly
conducted then the chances are that they will be biased, which means that the
overall results of the review will be biased.

How does this bias occur? Well, lets look at the second and third levels of
evidence.
Level II is defined as strong evidence from at least 1 properly designed
randomised, controlled trial of appropriate size.

5
While Level III evidence is that from well-designed trials without randomisation,
single group pre-post, cohort, time series or well-matched case-controlled
studies.

The key words here are properly designed randomised controlled trials and well-
designed non-randomised trials. Not all randomised trials are what they seem
and one of things you have to do when assessing evidence from them is to
establish the quality of the trial. This assessment can be made using objective
criteria. The main questions to ask are:
Was the trial properly randomised ? i.e. using a truly random method such
as roll of a dice or random number tables, not birth-dates or alternate
patients.
Was the allocation of treatment concealed? i.e. the person entering the
patient into the trial should not know in advance which treatment the next
person will get.
And finally, if appropriate, such as in drug trials, was the study blinded ? i.e.
the participant does not know what preparation they are receiving.

There is plenty of statistical evidence to show that there is a relationship

between the design and conduct of a trial and the results it produces: in effect ,
poorly designed or poorly conducted trials tend to overestimate the effect of
treatment, in some cases by as much as 41% [see Bandolier]. It is possible for
completely different results to be arrived at over the same treatment depending
on whether the trial was randomised or not.

The fourth level of evidence concerns that from well-designed non-experimental

studies from more than 1 centre or research group.

2
Bandolier, Oct 2000;80-2. www.jr2.ox.ac.uk/bandolier

6
These are observational studies where the researcher is studying what is
happening without intervening in any way. Note that once again the words well-
designed appear and that we are looking for the results to have been replicated.

The final, fifth and weakest category is evidence derived from the opinions of
respected authorities, based on clinical evidence, descriptive studies or reports
of expert committees.

One of the ways in which such evidence may be assessed is through the
process of benchmarking. This is a word you may have heard in connection with
best practice. It is a process borrowed from industry and consists of comparing
what you do against what others in the same field do, to determine who is
producing the best results. In a very clear exposition of the idea in the Nursing
Times, Emily Lam 3 points out that the process is different to clinical audit. The
latter is usually focused inwards and consists of a retrospective comparison of a
process and outcomes against pre-determined standards. Benchmarking on the
other hand looks outwards at what others are doing and tries to identify what
currently works best. It depends upon co-operation and a willingness to share
information. Once again, for the process to be meaningful it has to be conducted
properly using a systematic approach: the question you are trying to answer has
to be carefully formulated, you then collect the available data, analyse them and
implement the findings.

With evidence-based care still in its infancy very few areas of medicine are in a
position to produce Level 1 evidence for their practice. For most, the greatest
proportion of the assembled evidence will be graded much lower. The full answer
then to our first question What is meant by best practice? is that best practice is
that which produces the greatest good with least harm and at the most
reasonable cost in the light of the best available evidence.

3
Benchmarking Best Practice. Emily Lam,Nursing Times, 1994 November 16, Vol 90, No.46:
pp48-51

7
If we turn our attention now to what is best practice in the management of
lymphoedema, the short answer is that we dont know. Those of us specialising
in this field have only just begun to take the first tentative steps in analysing the
evidence. It is not an edifying experience. The published clinical evidence is
generally of a very poor quality. Not only are trials poorly designed and poorly
conducted but they are also very badly reported. Vital information necessary to
form ones own opinion of the reliability of the findings is usually missing. There
have been a few rallying calls from distinguished members of the International
Society of Lymphology for an improvement in the scientific rigour of studies
conducted in this field 4 5. But these appeals seem to have fallen on deaf ears.
Those of you attending the recent ISL meeting in Genoa will know what I mean
after listening to some of the papers concerning treatment one could be forgiven
for wondering whether the scientific committee ever turns an abstract down.

Lymphology is not alone in producing bad research. In 1994 the distinguished

medical statistician Doug Altman wrote a piece for the BMJ entitled The Scandal
of Poor Medical Research 6. He pointed out that (and I quote) much poor
research arises because researchers feel compelled for career reasons to carry
out research that they are ill-equipped to perform, and nobody stops them.
According to Altman many journals use little or no statistical refereeing bad
papers (he says) are easy to publish!

As an aside here, you might be interested to know something about the

CONSORT statement 7. This statement developed in the mid-1990s consists of a
checklist and flow-diagram for reporting an RCT. They can be used when writing
up, reviewing or assessing reports of an RCT to make sure that you havent left
out vital information. CONSORT has been supported by a growing number of

4
Improving the Scientific Image of Lymphology. W Olszewski. Editorial in Lymphology 28 (1995)
pp55-56
5
Benzopyrones and the Placebo Arm. C Witte. Correspondence in Lymphology 32 (1999) 86-89
6
The Scandal of Poor Medical Research. D Altman. BMJ 1994, 29 Jan, Vol 308:pp 283-4

8
medical and health-care journals and there is even data available to show that
the use of CONSORT does indeed help to improve the quality of reports of
RCTs.
Perhaps unsurprisingly, the nursing literature is no better than the medical
literature. Nursing articles tend to fall into one of two categories the What is
lymphoedema article? and case studies. The first of these consists usually of a
description of the patho-physiology underlying lymphoedema and an outline of
the principles of treatment. It is very easy to be persuaded to write these kinds
of article (I myself am guilty of writing several of them) and indeed they provided
useful information and raised awareness in the early days of specialisation in this
field. But there is a limit to the number of ways you can say the same thing and I
think we have reached it.

The problem with the second type of article, detailing case studies, is that as a
recent paper in Nurse Researcher 8 pointed out , they are inherently biased. I
quote: Case reports, unique stories of individual cases, are principally written to
draw attention to interesting outliers the unusual situations. By their very nature
they exclude the ordinary, the usual, the common, and therefore provide for
interest rather than the commonality that is necessary when relating ones own
clinical decisions to those that have gone before. (I am sure we have all done
the same thing when giving talks about lymphoedema we try to find slides or
illustrations of the worst possible cases to ensure that we grab peoples attention
but these cases are not representative of our workload.)

Much of the evidence concerning the management of lymphoedema then is low

grade and so we have to proceed with great caution when assessing it but
proceed we must. Treating patients brings with it responsibilities and as
professionals we have to be sure of the evidence on which we base our clinical
decisions. If you are feeling daunted at this prospect, take heart. As an arts

7
CONSORT Statement. www.thelancet.com

9
graduate with little or no understanding of the scientific process when I first
started in this field, I have struggled at times to get my head around a different
way of thinking but I can confirm that there is a great deal of help out there in
the form of courses and literature as well as knowledgeable and approachable
experts eager to encourage you.

As well as taking advantage of the wealth of information and help available to all
of us, we should wake up to the fact that chronic conditions are news at the
moment. Last year, for example, the BMJ in partnership with an American
medical journal ran a whole issue on the theme of chronic conditions. You wont
be surprised to hear that there were no papers on lymphoedema. They are going
to be doing the same thing in October next year, so you have plenty of time to
plan a contribution.

Experts in chronic conditions have long realised that while some problems are
disease specific, there are many that are common across different chronic
conditions. In our search for evidence we should not forget to look at the
literature surrounding conditions such as arthritis, diabetes, asthma etc. It is
possible that some of these areas of care have a larger body of high quality
evidence that we can draw on, in the absence of any specific to lymphoedema. I
believe firmly that only good can come of aligning ourselves with other chronic
conditions; for too long lymphoedema has stood isolated from the mainstream of
care. We do our patients a disservice if we persist in standing on the fringes -
this slightly odd speciality that no-one has ever heard of and cant take seriously.
Even when lymphoedema services are sited in general hospitals (and we know
that this is not a common setting for them) they are seldom integrated into the
hospital in any real sense. The reality is that chronic oedema affects an
enormously wide range of people and, if looked for, can be detected in a wide
range of clinical settings besides that of oncology. The epidemiology study being

8
Quality of evidence: problems with analysing low-grade case series studies. G Louw et al.
Nurse Researcher Vol 6 No 1: pp39-46

10
conducted by Professor Christine Moffatt and her team is providing valuable
evidence of this fact.

That people who actually have lymphoedema are an untapped and important
source of evidence is still not fully recognised. Studying patients who are
particularly successful in controlling their oedema could provide a valuable
insight into what works. The Department of Health is starting to acknowledge that
the problem of chronic conditions is a huge one and have produced a report
which came out at the end of August entitled The Expert Patient: A new
approach to Chronic Disease Management for the 21 st Century. It makes
interesting reading and I would urge you all to download it from the D.O.H.
website9. It is refreshingly clear and, at around 34 pages, reasonably succinct. It
seems to me that we should be representing lymphoedema in the Departments
plans.

In addition to a climate in healthcare that is receptive to initiatives that focus on

the problem of chronic conditions and there are other bright spots on our
horizon. These include the existence of a body of practitioners who have now
been specialising in this field for a good number of years and have built up
considerable experience and skill, observing, measuring, assessing and treating
a wide variety of people with chronic oedema; the existence, as I have already
hinted at, of an enthusiastic, proactive, and, I think, exceptionally motivated
group of patients who are eager to help in any way they can and who possess
among them a formidable amount of knowledge and expertise; the existence of a
professional body, The British Lymphology Society, with a committed national
network of regional groups whose true potential has, in my view, yet to be
exploited.

My final point is this: if we take up the challenge and arm ourselves with the
necessary skills to appraise evidence, co-operate with each other and widen our

9
www.doh.gov.uk/healthinequalities

11
outlook, we have an opportunity to set in motion a process that will determine
the current best practice which will be continually updated and will attract
respect and credibility. Exposing the gaps in the evidence will help to determine
the priorities for clinical research world-wide. It may take years to see the real
effects of this approach but by starting now, by acting responsibly and planning a
long-term strategy, we stand a real chance of achieving that, up till now, elusive
goal of creating and developing permanent, effective, and affordable services for
all people with lymphoedema.

Caroline Badger, October 2001

12