EBM - Therapy

GITA SEKAR PRIHANTI

Adaptasi dari Iwan Dwiprahasto

Dept. Pharmacology & Toxicology/Clinical Epidemiology
& Biostatistics, Faculty of Medicine GMU
 Learning Objectives

Understand meaning of evidence-based

medicine (EBM) and the 5-step EBM process

Understand the mechanics of formulating

clinically relevant questions
• Using P-I-C-O to formulate questions

Understand basics of articles related to

Diagnosis and articles related to Therapy
Drugs withdrawn from the market

Ticrynafen (US, May 79 - May 80):

hepatocelullar injury (> 500 reports)

Benoxaprofen (April 82-August 82): 61 death

(cholestatic jaundice) di UK (BMJ)

Zomepirac (Oct 80-March 83): anaphylac-

toid reaction (NEJM 1981): 1100 reports

Suprofen (jan 86-May 87): flank pain synd (>

300 reports).

Temafloxacin (US, Feb 1992) hemolytic

anemia, renal disfunction, death
 Fenomena experience based treatment
intensitas

Ketrampilan klinik

Medical errors

Evidence-based
knowledge

waktu
Innovation, development, & diffusion of
Medical technology

Established technology
Late adopters

Early adopters Obsolete

technology
Clinical trials
Abandoned
First medical use technology

Innovation Development Diffusion Evaluation

“Half of what you are
taught as medical
students will in 10
years have been shown
to be wrong.
And the trouble is,
none of your teachers
knows which half.”
(Dr. Sydney Burwell,
Dean of Harvard Medical School).
Why Is EBM So Important?

EBM is an essential skill needed for a physician’s life-

long learning (example of “self-directed learning”) so
that patient care can be delivered in the most
appropriate manner

EBM skills can be learned, practiced, and improved

upon to enhance a physician’s LLL habits
 Evidence-based Medicine

The integration of the best research evidence with

your clinical expertise and patient values

5 steps EBM

Formulate • a clinically relevant question

Find • the best available evidence

• this evidence for its validity and applicability,

Appraise strengths and weaknesses
• the best research evidence with your clinical
Integrate expertise and patient’s values

Evaluate • your efforts and seek ways to improve

From: Evidence Based Health Care Newsletter, Department of Clinical Epidemiology &
Biostatistics, McMaster University, November 1997
(http://hiru.mcmaster.ca/ceb/newslett/ceb_17.htm).
Background Questions
What is the best treatment for pneumonia

How do you diagnose diabetes mellitus?

What are the causes/risk factors for stroke?

What is the prognosis for AMI in an old age?

What is the differential diagnosis of...?

What are the indications for...?

Foreground Questions

P • = Patient Population or Problem

I • =Intervention

C • =Comparison

O • =Outcome
Therapy
“In patients with an acute MI, are
antiarrhythmic drugs more effective than
standard therapy in preventing sudden
cardiac death?”

P • = Acute myocardial infarction

I • = Antiarrhythmic drugs

C • = Standard therapy

O • = Sudden cardiac death

 3 macam efek terapi

 Menurunkan resiko terjadinya hasil yang buruk

 Meningkatkan probabilitas terjadinya hasil yang baik

 Meningkatkan probabiloitas terjadinya hasil yang buruk

yang tidak diinginkan

Murti, 2011
 Terapi yang terbaik

 Karakteristik terapi yang terbaik :

– Memberikan lebih banyak manfaat daripada
kerugian
– Cost effective
– etis
Murti, 2011
Diagnosis
“In patients with suspected pulmonary
fibrosis, how does high-resolution CT
compare with lung biopsy for
establishing the diagnosis?”

P • = Pulmonary fibrosis

I • = High-resolution CT

C • = Lung biopsy

O • = Sensitivity/specificity/PVs/LRs
What is critical appraisal?

Process of examining research evidence to

evaluate its validity, results, and relevance
before making an informed decision

Foundation to practicing thoughtful

“evidence-based” or “evidence-informed”
medicine

Hill, et al. Bandolier Volume 3 (2). http://www.evidence-based-medicine.co.uk

Why is it necessary?

Not all publications are equally convincing or reliable

(even if published in a reputable journal).
Incorrect interpretation
Fraud and misrepresentation of results
Data dumping

Sometimes clinical experience and theory based on

pathophysiology can be misleading.

Systematically examining the literature increases our

confidence in our strengths and shed light on areas of
weakness
Critical appraisal requires that you ask
yourself:
Is this relevant?
Is this valid?
Is this reliable?
Is this important and meaningful?
Is this applicable or generalizable?
1. Is this article even relevant?

• Title
• Abstract
• Introduction
Should I • Does this study identify a gap in the evidence?
read it? • Were the objectives clear and focused?

• Methods
• Were inclusion and exclusion criteria clearly stated?
• Are the outcomes patient-oriented or surrogate
Should I markers for long-term health outcomes?
• How long was the study? (study duration)
continue?
2. Is this valid?

INTERNAL validity or study quality

Was the design, methods, and conduct of a study
likely to have prevented or minimized bias in such
a way that I can trust the findings?
• With the information provided, could I reproduce
this study and observe similar findings?
Validity : Menilai Validitas (Kebenaran) efek
terapi Murti, 2011
Pengaruh Bias, Confounding, dan
Peran Peluang terhadap efek terapi
Murti, 2011
Importance : Menilai Kemaknaan
Klinis efek terapi
Murti, 2011
Penelitian tentang terapi
 CER = Control event rate = proposal
pada kelompok kontrol
 EER = experiment event rate = proporsi
pada kelompok eksperimen
 RR = relative risk
 RRR = relative risk education
 ARR = absolute risk reduction
 NNT = number needed to treat
 Comparing efficacy of 2 drugs:
Randomized Controlled Trial (RCT)

Cured
Treatment
Drug A
effect
Not cured

Cured
Drug B (control Treatment
group) effect
Not cured
 Analgesic effect of Drug A vs.
Example Drug B
for dysmenorrhoea

Cured Not Cured

Drug A 87 (87%) 13 (13%)
Drug B 74 (74%) 26 (26%)

Why Drug A is better than drug B?

Drug A Drug B
Less severe More severe case
Younger Older
Better compliance More non compliance
Cheaper More expensive
.... etc .....etc
 Reports of individual studies

I. Are the results of the study valid?

II. What were the results?

III. Are these valid, important results

applicable to our patient?
 I. Are the results of the study
valid?
Primary Guides:

Was the assignment of patients to

treatments randomized?

Was follow-up of patients sufficiently

long and complete?

Were patients analyzed in the groups to

which they were randomized?
 1. Was the assignment of patients to
treatments randomized?

RCT-parallel design

Patient
treatment A O
U
T
C
eligible Random O
M
E

Treatment B
 RCT cross-over design

Patient washed
out

eligible
O O
Treatment A U Treatment B U
T T
C C
Random O O
M M
Treatment B Treatment A
E E
 1. Was the assignment of patients to
treatments randomized?

Preventing bias

Objective of Equal chance

randomisation

Balancing Subject
characteristics
 1. Was the assignment of patients to
treatments randomized?

Toast/coin

Table random

Random allocation
How to random
Simple random sampling

Cluster random sampling

Multistage random
sampling
Unaccepted randomization: Unethical

ANALGESICS

random

PLACEBO

random
 2. Was follow-up of patients
sufficiently long and complete?

Lost to follow up no more than

20%

journals like Evidence-Based Medicine and ACP Journal Club won’t

publish trials with < 80% follow-up.
 3. Were patients analyzed in the groups
to which they were randomized?

“intention-to-treat analysis”

all patients are analyzed in the groups to which

they were initially assigned

A strategy for analyzing data in which all participants are included

in the group to which they were assigned, whether or not they
completed the intervention given to the group.
 I. Are the results of the study valid?

Secondary Guides:

Were patients, health workers, and study

personnel "blind" to treatment?

Were the groups similar at the start of the trial?

Aside from the experimental intervention, were

the groups treated equally?
 1. Were patients, health workers, and study personnel
"blind" to treatment?

Why blinding? To prevent bias

blinding

Single blind Double blind Tripple blind

Placebo-controlled multicentre randomised trial of interferon b-1b in
treatment of secondary progressive multiple sclerosis
 3. Aside from the experimental intervention,
were the groups treated equally?

Similar
• form
Drug/intervention • colour
• taste
• drug administration
 II. What were the results?

• What is the magnitude of the

treatment effect?

• How precise was the estimate of the

treatment effect?
 • What is the magnitude of the treatment
effect?
20% of the control
group (X)
Example: To express
results in a study where died
15% of the treatment
group (Y)

absolute risk X-Y =

reduction (ARR or risk 0.20 – 0.15 = 0.05
difference):
Y/X =
relative risk (RR):
0.15/0.20 = .75
relative risk reduction (1- RR) x 100 =
(RRR): (1- 0.75) x 100 = 25%
 Relative Risk
Reduction

RRR = (|CER – EER|/CER)

CER : Control Event Rate (without treatment/placebo)

EER : Experimental Event Rate (with treatment)
 In the trial:

12,562 subjects with coronary syndrome were randomized

to
aspirin alone or aspirin plus clopidogrel.

Patients were followed for 9 months.

The primary endpoint was to prevent cardiovascular

(CV) death, myocardial infarction (MI), or stroke.
 To calculate the number needed to treat (NNT),

first calculate the absolute risk reduction (ARR).

This is the absolute difference between the event rate in

the control group (CER) minus the event rate in the
experimental group (EER).

The NNT is the inverse of the ARR.

 Aspirin alone
group (control 11.47%
group)
the ocurrence of
MI, Stroke, or
CV death
the aspirin plus
clopidogrel
(experimental 9.28%.
group)
• What is the magnitude of the treatment
effect?

The greater the relative risk reduction

the more effective the therapy
 Significance of Relative Risk Reduction

key points:

1. negative RRR (- 38%): treatment may do harm:

pts given the new treatment might be 38% more
likely to die than the control pt
2. RRR of 0%: no treatment effect or benefit
3. positive RRR (50%): pts receiving the new
treatment might have less than 1/2 risk of dying
compared to not treated
 Interferon as secondary prevention of
multiple sclerosis

CER (placebo) 50%

EER (interferon) 39%
RRR (CER-EER/CER) ?
ARR (|CER-EER|) ?
NNT (1/ARR) ?

Event rate= progression of disability within 33 months

CER = control event rate
EER = experimental event rate
RRR = relative risk reduction
ARR = absolute risk reduction
NNT = number needed to treat
 Interferon as secondary prevention of multiple
sclerosis (33 months of treatment)

CER (placebo) 50%

EER (interferon) 39%

RRR (CER-EER/CER) (50% - 39%)/50% = 22%

ARR (|CER-EER|) 50% - 39% = 11 %

NNT (1/ARR) 1/11% = 9

NNT 10 month = 9 x (33/10) = 29.7

Table 5.4 EBM-Sacket
Benefit vs Risk of antidiarrhoeal drugs

Antidiarrhoea NNT NNH

Kaolin 12 5
Pektin 14 9
Atapulgit 9,2 3.2
Neomisin 10.7 5.4
Loperamida 5.3 2.1
TMP/SMX 7.5 3.8
Siprofloksasin 13.4 4.4
Ofloksasin 11.8 3.2
 • How precise was the estimate of the
treatment effect?

95% Confidence interval

NSAID Gastric ulcer

2 RR = 2,45; 95% CI (1,85 - 2,92)

2 RR = 1,85; 95% CI (0,95-2,25)
 95% Confidence interval

Prophylactic antibiotics in SC to prevent endometritis

RR for elective SC : 0.24 (95% CI: 0.11 - 0.48)

RR for non-elective SC : 0.30 (95% CI: 0.25 -
0.35)
 III. Are
these valid, important results
applicable to our patient?

1. Is our patient so different from those in

the study that its results cannot apply?

2. Is the treatment feasible in our setting?

3. What are our patient’s potential benefits

and harms from the therapy?

4. What are our patient’s values and

expectations for both the outcome we
are trying to prevent and the treatment
we are offering?
 Weight reduction of orlistat
Table 1. Demographics of the patients.
Sex Number Age Weight at start BMI at start of
(years) of orlistat orlistat
Female 553 49 + 13 94.3 + 15.8 35 + 6
Male 236 52 + 12 112.1 + 15.5 35 + 6
Total 789 50 + 12 99.3 + 17.7 35 + 6
Normal weight = 18.5-24.9
Overweight = 25-29.9
Obesity = BMI of 30 or greater

Are patients characteristics similar to our patients?

 Obsolete & abandoned clinical practice

Negative appendectomy: 20-40%

Reducing negative appendectomy: evaluation of USG and computer

tomography in acute appendicitis

Study n Technique* Sensitivity Specificity

Men 268 CT Scan 92 100
Women 342 CT Scan 83 93
Men 47 USG 87 98
Women 67 USG 76 96
Styrud et al, , Intl J for Quality in Health Care, 2000
 Obsolete & abandoned clinical practice

Negative appendectomy: 20-40%

Summary of CT Diagnostic Sensitivity and Specificity for Appendicitis in

Children

Study n Technique* Sensitivity Specificity

Sivit et al (18) 153 Mixed 95 93

Garcia Pen˜a (11) 108 Rectal contrast
material only 95 95
Lowe et al (9) 76 Mixed 100 100 Mullins
et al (10) 199 Rectal contrast
material only 97 99
 Weight reduction of orlistat
Table 1. Demographics of the patients.
Sex Number Age Weight at start BMI at start of
(years) of orlistat orlistat
Female 553 49 + 13 94.3 + 15.8 35 + 6
Male 236 52 + 12 112.1 + 15.5 35 + 6
Total 789 50 + 12 99.3 + 17.7 35 + 6
Normal weight = 18.5-24.9
Overweight = 25-29.9
Obesity = BMI of 30 or greater

Apakah karakteristik demografi sesuai dgn pasien kita?

 Weight reduction of orlistat

Table 2. Weight changes in patient after 8 month

Berat badan turun karena orlistat atau karena diet?

 BEST PRACTICES WITH GREATEST STRENGTH OF EVIDENCE
Patient safety target
1. Venous thromboembolism
(VTE)
2. Infeksi akibat central
venous catheter
3. Surgical site infection
4. Ventilator associated
pneumonia
5. Pressure ulcer
6. Morbidity due to catheter
insertion
7. Adverse event due to
warfarin
Patient safety practice
1. VTE prophylaxis promptly &
appropriately
2. Maximum sterility during
catheter insertion
3. Prophylactic antibiotics
4. Continuous aspiration of
subglottic secretions
5. Nursing care
6. Use real-time US guidance
7. Self management-Home
monitoring devices
 Steps Evidence-Based Medicine

See a patient Ask a question Seek the best evidence

Monitor the Apply the evidence Appraise that evidence

change
 CONSORT checklist
CONSORT Checklist of items to include
when reporting a study on therapy (page
22)

Three day versus five day treatment with

amoxicillin for non severe pneumonia in
young children: a multicentre randomised
controlled trial (page 32)
Critical Appraisal for Therapy

Using the worksheet (page 13)

Three day versus five day treatment

with amoxicillin for non severe
pneumonia in young children: a
multicentre randomised controlled trial
CONSORT
 CONSORT
CONSORT = Consolidated Standards Of
Reporting Trial

22 items 
• Title/abstract
• Introduction
• Methods
• Results
• Discussion

http://www.consort-statement.org
BMJ 2004; 328 : 791
1. Title / Abstract
2. INTRODUCTION - background
3. METHODS – participants
4. METHODS – Interventions
5. METHODS – Objectives
6. METHODS – Outcomes
7. METHODS – sample size
8-10. METHODS – randomization
11. METHODS – Blinding
12. METHODS – Statistical Methods
13. RESULTS – Participant flow
14. RESULTS – recruitment
15. RESULTS –
Baseline data
16. RESULTS – Number analyzed
INTENTION TO TREAT ANALYSIS

a type of analysis of clinical trial data in

which all patients are included in the
analysis based on their original assignment
to intervention or control groups,
regardless of whether patients failed to
fully participate in the trial for any reason,
including whether they actually received
their allocated treatment, dropped out of
the trial, or crossed over to another group.
16. RESULTS – Number analyzed
17. RESULTS – Outcomes
18.
RESULTS –
Ancillary
analysis
19. RESULTS – Adverse events
21. DISCUSSION – Interpretation
21-22. DISCUSSION – generalizability and
overall evidence
CRITICAL APPRAISAL
BMJ 2004; 328 : 791
 Methods-1
Randomized, placebo controlled trial, multicentre,
double blind

2188 children 2-59 months

cough, rapid respiration or

Subjects : difficulty in breathing

Non severe pneumonia:

respiratory rate ≥ 50/min
(2-11) or ≥ 40/min (12-59)

Interventions:
Oral amoxicillin 31-54mg/kg/day in 3 divided dose.
1095 subject given 3 days and 1093 subjects given
five days
Methods-2
Severe pneumonia

Condition requiring AB

Excluded Congenital heart disease

Chronic systemic disorder

Using AB and allergy penicillin

Asthma & measles

Methods-3
OUTCOMES : treatment failure

development chest indrawing, convulsions,

drowsiness, or inability to drink, respiratory
rate above age specific cut points on day-3 or
later, or oxygen saturation by pulse oximetry <
90% on day 3

3 and 5 days after enrollment

FOLLOW UP
12 and 14 days after enrolment
 EBM Therapy Worksheet

I. Are the results of the trial valid ?

II. What were the results?

III. Will the results help me in caring

for my patient?
 I. Are the results of the trial valid?
What question did the study ask?

P I C O
Patient Intervention Comparison Outcomes
Or Problem

Non Antibiotic Cure rate

Antibiotics
severe for 3 for 5 days
pneumonia days
in children
I. Are the results of the trial valid?

1a. Was the assignment of patients to treatments

randomized?

1b. Were the groups similar at the start of the

trial ?

2a. Aside from the allocated treatment, were the

groups treated equally?

2b. Were all patients who entered the trial

accounted for? And were they analyzed in the
groups to which they were randomized?
I. Are the results of the trial valid ?

3. Were patients measure objective or

were the patients and clinicians and kept
"blind" to which treatment was being
received?
1a. Was the assignment of patients
to treatments randomized?

Block randomization with variable

sized block, to avoid unblinding

11 doses of 8 doses amoxicillin

amoxicillin for 3 or placebo for next
days 2 days
1b. Were
the groups
similar at
the start of
the trial ?
2a. Aside from the allocated treatment, were the
groups treated equally?

ANALGETIKA

random

PLASEBO

random
 2a. Aside from the allocated treatment, were the
groups treated equally?

11 doses amoxicillin
for 3 days
5 days of
treatment
(control) 8 doses amoxicillin
for next 2 days
Treatment
11 doses amoxicillin
for 3 days
5 days of
treatment
(experimental) 8 doses placebo
for next 2 days
2b. Were all
patients who
entered the trial
accounted for?
And were they
analyzed in the
groups to which
they were
randomized?

Lost to follow
up no more
than 20%
2b. Were all patients who entered the trial accounted
for? And were they analyzed in the groups to which
they were randomized?
 INTENTION TO TREAT ANALYSIS

a type of analysis of clinical trial data in

which all patients are included in the
analysis based on their original assignment
to intervention or control groups,
regardless of whether patients failed to
fully participate in the trial for any reason,
including whether they actually received
their allocated treatment, dropped out of
the trial, or crossed over to another group.
3. Were patients measure objective or
were the patients and clinicians and kept
"blind" to which treatment was being
received?

Double blind (participants and

researcher)
II. What were the results?

1. How large was the treatment effect

2. How precise is the estimate of the

treatment effect?
 Risiko relatif (RR = Relative Risk)

Disease a
------
+ - a+b
RR = --------------
A a b c
Treatment
------
B c d
c+d
 (RR = Relative Risk)

Cure EER 89.5

RR = --------------
+ - CER 89.9
3 days 980 115
RR = 0.996 = 1.00
5 days 983 110
 Control Event Rate (CER)

CER : the proportion of outcome in the control group

(treatment standard or placebo groups)

Experimental Event Rate (EER

CER : the proportion of outcome in the experimental group

(treatment standard or placebo groups)
EER CER
Absolute Risk Reduction (ARR)

ARR = (|CER – EER|)

ARR = (89.90-89.50)

ARR = 0.40
Relative Risk Reduction (RRR)

RRR = (|CER – EER|/CER)

RRR = (89.9-89.5/89.9)

RRR = 0.005
Number Needed to Treat (NNT)

NNT = 1/ARR

NNT = 1/0.4%

NNT = 250
Confidence Interval (CI) ARR

CER (1 CER) EER (1 EER)

 1.96 
Ncontrol Nexp

95% CI = ± (1.96 x 1.30)

95% CI = ± 2.26
95% CI ARR

95% CI ARR = 0.40 ± 2.26

95% CI ARR = -1.86 - 2.66