PRINCIPLES OF

DIAGNOSTIC RESEARCH
Siti Setiati
Objective
At the end of the lecture you will understand:

What is Diagnostic Research?
Why Performing Diagnostic Research?
How to Apply Diagnostic Research in Practice?
How to Design Diagnostic Research?


WHAT?
Type of Research
Question
Descriptive/Causal Aim
Diagnostic research Descriptive
Predict the probability of presence of target
disease from clinical and non-clinical profile
Prognostic research Descriptive
Predict the course of disease from clinical an
d non-clinical profile
Etiologic research Causal
Causally explain occurrence of target disease
from determinant
Intervention research Causal & Descriptive
(1) Causally explain the course of disease as
influenced by treatment
(2) Predict the course of disease given
treatment (options) and clinical and non-
clinical profile
Major Types of
Epidemiological Research

To assess the value of combinations of
diagnostic determinants in diagnosing a
particular disease
To assess the value of novel diagnostics tests
in addition to readily available tests (such as
signs and symptoms)

WHAT?
Aim of Diagnostic Research
Challenges of Diagnostics in Practice
Physicians are faced with multiple diagnostic
challenges
Diagnostic approach is always done when dealing
with patient complaints (symptoms and signs)
The aim is to interpret the symptoms, signs, and
results of other diagnostic test so that a diagnosis
can be established
Diagnosis will directs the physician in making
decisions for appropriate management

WHAT? WHY?
Why diagnose?
Basic medical handling
Determines treatment choice
Gives information about prognosis

Physician should:
Quickly and efficiently determine the correct
diagnosis

WHAT? WHY?
Steps in Making Diagnosis in Practice
WHAT? WHY?
Assess which diagnostic determinants (and in
what order) best predict the presence of
disease?
Starts with patient with clinical problem
(presenting a complaint symptom and/or
sign)
Symptom/sign --> suspicion of particular
disease
Eg: a 27 years old man with acute fever
and ptechiae DHF?



Steps in Making Diagnosis in Practice
(2)
WHAT? WHY?
Starts with a patient presenting a complaint
suggestive of a certain disease to be diagnosed
The subsequent work up is a multivariable
process involving multiple diagnostic
determinants
Ruling in or out a diagnosis is a probabilistic
action, the probability is continuously updated
based on subsequent diagnostic results



Common Diagnosis Process:
9
Diagnosis Process multivariable concern
= continuous adding info + updating probability

History + physical
examination
Simple lab
(blood/urine)
Electrophysiology
(ECG/EEG)
imaging
DIAGNOSIS
WHAT? WHY?
Why not performing all tests?
WHAT? WHY?
Invasive (for patient and budget)
Unnecessary: different test results give same info
However: In practice often more tested than
necessary!

What diagnostics truly necessary ?
Thats why we need scientific
DIAGNOSTIC RESEARCH!!!



Diagnostic Research
WHAT? WHY?

The aim is to predict (not to explain why certain
symptoms/signs lead more to a certain diagnosis)
the probability of presence of target disease from
clinical and non-clinical profile (diagnostic
determinants)
Determinants: element of clinical and non clinical profiles (signs,
symptoms, and possible test results)
Outcome : the diagnosis of the disease


Example
WHAT? WHY?
Research question:
What is the value of
Procalcitonin (PCT)
in addition to signs and
symptoms when diagnosing
sepsis in elderly presenting with
fever in primary care?
PCT GOLD STANDARD
Sepsis+ Sepsis -
Sepsis
+
a c
Sepsis
-
b d
Question: what are Determinants and Outcome?
Sepsis in elderly = f ( PCT, body temperature, symptoms, signs...... )
DIAGNOSTIC RESEARCH DIAGNOSTIC TEST
Start with a patient come with
symptoms & signs suggesting the
disease
Multivariable (involve multiple
tests)
Domain: patients suspected of a
particular disease based on
symptoms and signs presented
Presence of hierarchy of diagnostic
testing in practice
Start with non-symptomatic
individual
Univariable, to assess whether
a single diagnostic test able to
discriminate the presence or
absence of certain disease
Does not include
representatives of relevant
patients domain (patient with
symptoms and signs suggestive
of disease)



Julius
Centrum
Jakarta April 2009
13
WHAT? WHY?
Diagnostic Test VS Diagnostic Research
No information for doctor
or patient
No predictive value
Not important in clinical
practice
The aim is to estimate the
probability of disease from the
diagnostic results, Predictive
Values is important.


14
WHAT? WHY?
Sensitivity
If the disease present, what is the probability that the test result will be
positive
Specificity
If the disease is absent what is the probability that the test result will be
negative
Predictive value
If the test positive, what is the probability that the disease will be present


S
O
M
E

I
M
P
O
R
T
A
N
T

T
E
R
M
S

Before designing Diagnostic Research
One should ...
Understand clinical practice
Know clinical practice
Preferably both
WHAT? WHY? HOW TO APPLY?
Understanding application of diagnostic research in clinical
practice is important requirement before designing one
Clinical scenario
A 27-year old male came with acute fever and
ptechiae in both arms.
What are the most possible diagnosis for this
patient?
WHAT? WHY? HOW TO APPLY?
Differential diagnosis
- Dengue Hemorrhagic Fever
- Malaria
- Chikunguya
- Disseminated Intravascular Coagulation
- Idiopathic Thrombocytopenic Purpura

What is the most important diagnosis?
Which one does the physician does not want to miss?




WHAT? WHY? HOW TO APPLY?
Most important diagnosis:
DENGUE HEMORRHAGIC FEVER


If missed: often fatal
WHAT? WHY? HOW TO APPLY?
Suppose: 20% of all patient on the ER with
ptechiae have DHF
20% with disease in that population = prevalence
Prior-probability : probability of the presence of the target
diagnosis (ie. DHF) before any tests (incl. Symptoms & signs)
are performed.


What is your decision for the patient in this
case?
WHAT? WHY? HOW TO APPLY?
Decision for patient in case
Prior-probability too low to treat
Prior-probability too high to send home

Decision: reduce uncertainty diagnostics

What is the best test?

WHAT? WHY? HOW TO APPLY?

Best test:
VIRUS CULTURE

WHAT? WHY? HOW TO APPLY?
Diagnostics in Practice
Gold standard:
Expected to detect the true disease status; as the truth (to
detect the disease correctly)
However, the 24 carat status is not exist
Used as the referrence standard/test

How to perform a reference test for
everybody (=every patient in ER with
ptechiae)?

WHAT? WHY? HOW TO APPLY?
Reference test for everybody?
Reference test for everybody?
Unethical too invasive/risky
Inefficient too expensive
Do not perform unnecessarily

How should we then determine the
probability of disease presence and what
would be ideal?

WHAT? WHY? HOW TO APPLY?
How then?
Simpler diagnostics:
Usually anamnesis, physical exam, simple lab
tests, imaging, etc.
Ideal: diagnosis without reference test

Diagnostic process in practice:
Stepwise process: less more invasive
Not one diagnosis based on 1 test
Each item: separate test
WHAT? WHY? HOW TO APPLY?
Suppose:
after anamnesis & physical examinations: 10% probability of DHF

Probability of disease given test results = posterior-probability
The bigger the difference between prior and posterior probability, the
better the diagnostic value of the tests

Our decision for patient in case: probability is too high to
send home --> next step?



WHAT? WHY? HOW TO APPLY?
Next step
Additional research, e.g.
blood tests (leucocytes,
CRP, sedimentation, etc.)
WHAT? WHY? HOW TO APPLY?
Suppose: 1% posterior-probability after anamnesis, PE+
simple lab tests posterior probability low enough to
send home

Ideal diagnostic process: simple tests reduce posterior probability
to 0 or 100% (without reference)
Most often physician continues testing until sufficiently sure
(approximation of 0 or 100%)
Choose when sufficiently sure: depends on prognosis of disease if
untreated + risks/costs treatment

WHAT? WHY? HOW TO APPLY?
Diagnostics in Practice
To Summarize:

What does diagnosing involve in practice?

Estimation of probability of disease presence based on test
results of the patient

Study design: three components

1. Theoretical design

2. Design of data collection

3. Design of data analysis


WHAT? WHY? HOW TO APPLY?
HOW TO
DESIGN?
Three Components of Research Design

Theoretical design:
Design of the occurrence relation
Design of data collection:
design of the conceptual collection of data
Design of data analysis:
a description of the data and quantitative
estimates of association

WHAT? WHY? HOW TO APPLY?
HOW TO
DESIGN?
1. Theoretical Design

The theoretical design of a study starts from a research
question
Formulating the research question is of critical importance as
it guides the theoretical design
The occurrence relation is central to the theoretical design, is
the object of research and relates one or multiple
determinants to an outcome
The occurrence relation of diagnostic research:

P(D)= f(T1,T2,T3,----Tn)
P(D): the probability of the disease
T1 to Tn: the multiple diagnostic test results
WHAT? WHY? HOW TO APPLY?
HOW TO
DESIGN?
Theoretical design
32
Example:
Which simple tests contribute to the estimation of DHF
present in an adult coming to an ER with ptechiae

% DHF = F ( fever, myalgia, platelet count, haematocrite,
etc)

No confounding! (causality is of no concern in diagnostic
research, the motive is not to explain the disease
occurrence, but only to predict)






WHAT? WHY? HOW TO APPLY?
HOW TO
DESIGN?
Theoretical design outcome
Outcome

Define criteria presence/absence target disease




33
Requires gold standard
none 24 carat ( none have 100% sensitivity and
specificity)
the one used when doubt remains (conform practice)
better name: reference test (best available test in
current practice )

Theoretical design outcome

Case: virus culture
Other: NS-1 Antigen
Blind observer of reference test

34
WHAT? WHY? HOW TO APPLY?
HOW TO
DESIGN?
Theoretical design domain
35

Domain :
Pertains to the applicability/generalisability of
the results targeted population (NOT the
study population)
The population in which the results of the
study will be applied
WHAT? WHY? HOW TO APPLY?
HOW TO
DESIGN?
Domain
Implications for in-/exclusion criteria
how detailed (pragmatic research)?

Typically:
patients suspected of ........
patients presenting with..... (ie. Symptoms)
patients presenting with symptoms suggestive of ....
in whom additional tests are being considered?

36
Theoretical design
domain
WHAT? WHY? HOW TO APPLY?
HOW TO
DESIGN?
2. Design of Data Collection
a. Proper study design
b. In-/exclusion criteria patients
c. Assessment determinants
d. Assessment outcome
e. Sample size

37
WHAT? WHY? HOW TO APPLY?
HOW TO
DESIGN?

Answer The Following Questions
1. What is the Study Design ?
2. What is the inclusion/exclusion criteria of the
patients?
3. What is the assessment of the determinants?
4. What is the outcome?
5. Sample size?
38
WHAT? WHY? HOW TO APPLY?
HOW TO
DESIGN?
Design of Data Collection

a/. Proper study design: Cross sectional

Typical study design of diagnostic research

Diagnostic determinants and reference test are
determined at t0 (same time)

Even when all tests and definitive diagnosis
(outcome) takes some time (interest is at t0 !)

Natural working method of doctors !




39
3. Design of Data Analysis

Step 1: Estimate the Prior Probability


Step 2: Univariable Analysis
(to keep in touch with data)

Step 3: Multivariable analysis
40
Design of Data-Analysis

STEP 1 : Prior probability ?

200 patients with ptechiae at ER
Virus culture positive (DHF+) n=40
Virus culture negative (DHF-) n=160



41
Design of Data-Analysis
STEP 2: univariable analysis per test result


42
10 C
40 160
80 D 90
200
30 A 80 B 110
DHF+ DHF- Total
Y (+)
ptechiae
N (-)
Predictive value ?
27%
Design of Data-Analysis
STEP 2: univariable analysis per test result




Julius Centrum

Jakarta April 2009
43
10 C
40 160
80 D 90
200
30 A 80 B 110
BM+ BM- Total
Y (+)
ptechiae N (-)
Sensitivity ?
Specificity ?
Design of Data-Analysis

STEP 3: Multivariable analysis
Logistic regression modeling (multivariate)
Sum of regression coefficients for each patient
Leads to risk score (clinical score)

Calculate the risk score
- Which variables did the researchers select for the multivariate
analysis
- For patients with mialgia and fever?
- For patients with headache ?
- Calculate predictive value for a patient with headache. (Use table 4
Rietveld paper)







44
Design of Data-Analysis

ROC (Receiver Operating Characteristic) area
- the area under the the ROC curve reflects the overal discriminative
value of the model,
exhibits the extent to which the model can discriminate
between subjects with and without disease

- the more the ROC curve is in the left upper corner, the higher
the AUC

no clinical meaning, ROC area only reflects the overal
discriminative value of a model, no absolute disease probabilities

no info on individual probabilities of D (predictive values)

requires threshold (weighing false
positives and negatives)


45
Design of data-analysis



Julius Centrum

Jakarta April 2009
46
ROC Curve
1 - Specificity
1, 00 , 75 , 50 , 25 0, 00
S
e
n
s
i
t
i
v
i
t
y
1, 00
, 75
, 50
, 25
0, 00
47
*STARD initiative, Bossuyt etc. BMJ 2003
Take-Home Messages
Diagnosis = estimating probability disease presence
Absolute risk
Stepwise multivariable proces (none based on 1 test)
Added value of test is relevant
Diagnostic research is prediction (descriptive) research
Confounding is no issue
Subjects selected on symptoms or signs (suspicion)
Probability disease given combinations of test results
48
WHAT? WHY? HOW TO APPLY?
HOW TO
DESIGN?
TAKE HOME
MESSAGE

Thank You