Sunteți pe pagina 1din 7

Why every psychologist should know Bayes’ Theorem ....

and how easy it actually is

Bea Tiemens, Renée Wagenvoorde, Cilia Witteman

All psychologists have learned Bayes’ Theorem. But what was it about again? Wasn’t that
one of those complicated formulae? Oh well, that is far from practice. But beware, the
Bayes’ rule has everything to do with practice. By ignoring the rule, serious mistakes can
be made. We will explain this with an example and show how easy to use the rule is.

Probabilities?

Suppose you work in community mental healthcare and a patient was referred by the general
practitioner (GP) because of an increased score on a depression questionnaire. In this general
practice, 11% of the patients have a depression (the prevalence). If a patient has a depression, the
chance that the patient scores positively on this depression questionnaire is 83% (sensitivity). A
patient who does not have a depression, has an 80% chance of a negative score on the list
(specificity) and a 20% chance of a positive score (chance of a false positive result). What is the
chance that the referred patient has a depression? (give an answer before you continue reading).

All psychologists have learned at university how to answer this question. You must indeed take into
account the sensitivity and specificity of a psychological test, but what should you do with that
prevalence of 11%? Filling in Bayes' rule, as learned; but wasn’t that a complicated formula? Well,
if this patient has a high score on a depression questionnaire, the chance of a depression is fairly
high, isn’t it? You do not have to calculate that, and certainly not using such a formula that has
nothing to do with practice. Bayes’ rule, however, has everything to do with practice. By ignoring
the rule, serious mistakes will be made. Probably this indeed happens regularly and there may be a
lot of overdiagnosis and overtreatment. We will explain this using the above example, both by
applying Bayes’ formula, but also in two simpler ways.

Bayes' Theorem in words

Bayes' theorem helps determine the probability of an individual's disorder, given the prevalence of
that disorder in the population of that individual, given the outcome of a test and the psychometric
characteristics of that test. The population to which the individual belongs is essential in Bayes'
theorem, but it is often overlooked. If a disorder is rare in a population (the prevalence of that
disorder is low), the probability of the disorder will be higher with a positive test than before the
test was taken, but still very low. This means that the same test result in different populations
gives a different probability of the disorder. This is intuitively very difficult to understand, even if
you are familiar with the rule. Consider, for example, testing for a tropical disorder in the tropics or

1
in the Netherlands. In the Netherlands, the probability that a positive test result is due to
coincidence is much higher because the prevalence is much lower. It helps to understand this by
making the calculation every time a test result has to be interpreted. Below we show that Bayes’
formula might be difficult, but that the calculation is actually childishly simple.

Does the patient have a depression?

Difficult formula?

In our example, the probability that a patient the GP sees has a depression is 11%. This is the
prevalence (prev) in the consultation-hour population. This is also called the base rate, the basic
probability or pre-test probability; the probability before being tested. Do not drop out now, it will
really become easy. The left part of figure 1 first shows this probability of depression and below
that, it shows the probability that a patient who actually has a depression, has a positive score on a
depression questionnaire (sensitivity, 83%) and the probability that a patient who does not have
depression, scores negatively on that list (specificity, 80%). These probabilities are based on
research by Cameron and colleagues who tested three depression questionnaires in GP practices1.
From this study, we use the most favourable values for the sensitivity (Se) and specificity (Sp) of
the Brief Depression Inventory II (BDI-II)2.

Figure 1. The probability of depression with a positive depression test: Bayes' formula

Next to the diagram in Figure 1 on the left, Bayes’ formula is on the right. The P stands for
'probability' or chance. The vertical line means 'under the condition that' or 'given'. So P(D+|T+)
means the probability (P) that the patient has a depression (D+) given (|) a positive test result
(T+) on a depression questionnaire. We want to know that probability and we can use Bayes’
formula for this:

P(T+|D+) is the probability that a patient with a positive test has depression, so the
sensitivity, here 0.83.

P(D+) is the probability that a patient in this population has a depression, the prevalence,
here 0.11.

2
P(T+|D-) is the probability that a patient with a positive test has no depression, i.e. 1-
specificity (the probability that a patient with a negative test does not have depression),
here 1-0.8 = 0.2

P(D-) is the probability that a patient in this population has no depression, 1-prevalence,
here 1-0.11 = 0.89.

So we have all the required information to fill in Bayes’ formula:

Thus, the probability (P) that the referred patient from the concerning general practice population
has a depression (D+) given (|) a positive test result (T+) on a depression questionnaire,
P(D+|T+), is about a third, 34%. This is also called the after-chance or post-test probability; the
probability after the test has been taken and given the outcome of the test.

It can be easier

When applying Bayes’ formula, you have to do some puzzling and the formula is hard to
remember. However, it is much simpler if the same information is presented in absolute numbers
instead of in probabilities3, as is done in Figure 2.

Figure 2. The probability of depression for a positive depression test in numbers

On the left, the information is in the same flowchart as in figure 1 and on the right in a cross
tabulation, but now in absolute numbers. Using the sensitivity of the test we are able to calculate
how many patients with a depression will have a positive score: 0.83 x 110 = 91.3, rounded to 91.
So 110 - 91 = 19 patients with depression had a negative score on the depression test. The
specificity has been used to calculate how many patients who do not have a depression score
negatively on the test: 0.80 x 890 = 712. So, 890 - 712 = 178 patients who do not have
depression will have a positive score on the depression test. This way, it becomes clear at a glance

3
how many patients have a positive test score (91 + 178) and how many of them actually have a
depression (91). The probability that the patient has a depression given a positive test is therefore:

The probability that the patient does not have a depression given a negative test is:

Do not rely only on sensitivity and specificity

Although the sensitivity of the BDI in this study by Cameron et al. is 83% and the specificity 80%,
the probability that the patient with a positive test result actually has a depression is much lower,
namely only a third. This is much too low for a treatment indication. A negative test result, on the
other hand, gives a relatively high level of certainty in this population about the absence of
depression, a probability that is higher than the specificity. This test could therefore be used to
exclude depression.

From the above it becomes obvious that it is not recommended simply to rely on the score of a
test. A positive test does not necessarily mean that the disorder is present and a negative test does
not necessarily indicate that it is absent. Neither can we rely on the sensitivity or 'chance of being
caught' and the specificity of an instrument only. Especially when the prevalence is very low, a
positive test result will never give a very high probability of the disorder even when the sensitivity
and specificity of a test are good.

This becomes clear when we use the same depression test, with the same sensitivity and
specificity, but now in the general population, where the prevalence of depression is lower than in
general practice: 5%. Then the probability of depression with a positive test result is less than 20%
(figure 3): 42 / (42 + 190) = 0.18. Suppose that in the situation of figure 3 the specificity is not
80% but 95%. The number of non-depressed patients with a positive score will be 48. The a post-
test probability is much higher now, 42 / (42 + 48) = 0.47, but is still lower than 50%.

4
Figure 3. The probability of depression for a positive depression test in the general population

Research versus practice

The figures with the examples show why it is necessary in practice to always keep Bayes’ theorem
in mind. When calculating the sensitivity and specificity of a test in scientific research, it is known
which respondents do and do not have the disorder. In practice, this is unknown, and that is
precisely why the test is administered. The psychologist only sees a positive test result! In the
above example, this is the case with 23% (42 + 190/1000) of the people tested, while only 5%
have a depression. The psychologist does not know which of those 23% do or do not have a
depression. But if the psychologist is aware of Bayes' rule, it is clear that extra diagnostics is
needed in addition to taking a BDI.

Bayes’ theorem, in formula or in one of the other simpler forms, helps to determine the posterior
probability of a disorder, given the test result, the pre-test probability (the prevalence) and the
sensitivity and specificity of the diagnostic instrument. Clinicians often forget to include the pre-test
probability in their interpretation, they only consider the sensitivity of a test and a positive score is
interpreted as the presence of the disorder. How easy but incorrect this conclusion is, shoud be
clear now.

Bayes in practice

What does it mean in practice when one is aware of Bayes’ theorem? It means that when
interpreting a test result, the setting in which the test was taken must always be considered. The
assessment of the probability of a disorder always starts with the determination or estimation of
the prevalence in the population or setting in which the test was performed.

You may wonder how you can know what the prevalence is or where to find it. That is exactly the
right question. The prevalence of the most common mental illnesses at general population level can
be found in the national Nemesis study4,5, the publication of which can be downloaded free of

5
charge. Most psychologists, however, do not work at general population level, so the prevalences
in general practice or in mental health care are more relevant.

Most numbers on prevalence in general practice are based on registrations by the GP. We know,
however, that this prevalence on the basis of registrations is generally an underestimation. The
prevalence as mentioned in the general practice guidelines6 is, for example, lower than the
prevalence at population level. Because almost everyone in the Netherlands has a GP, the
prevalence in the general practice population should be comparable to the prevalence at population
level. However, patients with depression visit their GP more often than patients without depression.
The prevalence is higher among the patients who come to the consultation hour. A rough guideline
is to double the prevalence found at general population level7.

The prevalence of a mental illness in the setting in which you work is best determined on the basis
of registrations in your own organization. Suppose that about a third of the patients who are
referred to your organization have a depression. If you enter that prevalence in the flowchart as in
figure 2 and keep the same sensitivity and specificity, the probability of depression of a random
patient who scores positively on the BDI is increased to two thirds.

Conclusion

Ignoring Bayes’ theorem leads to overdiagnosis mainly of disorders that do not occur so often (with
a low pre-test probability). This can be clearly understood by looking at the flowcharts. Because
the 'no disorder' part becomes very large - even with very good specificity - the post-test
probability will remain low. This can have major consequences for the treatment policy and thus
the course of the symptoms of a patient. We therefore strongly recommend remembering Bayes'
principle, namely that given a positive test result the probability of a disorder depends on the
population and setting in which is tested. If you have forgotten Bayes’ formula, do not panic. The
calculation with the absolute numbers is easy to do. The steps are summarized in the box.
Calculating using a group of 100 or 1000 patients is very illuminating and provides quick insight
into the probability of a disorder given a score on a specific test.

Steps in assessing a test result:


1. Determine the population to which the person tested belongs and the setting in which the test
was taken
2. Find the prevalence of the disorder that is being tested in that population and setting
3. Find the sensitivity and specificity of the test in the manual of the test
4. Enter the values in a flowchart or crosstabulation as in figure 2
5. Calculate the probability of the presence of the disorder given a positive test result, or the
probability of absence of the disorder given a negative test result.

6
References

1
Cameron, I.M, Cardy, A., Crawford, J.R. et al. (2011). Measuring depression severity in general practice:
discriminatory performance of the PHQ-9,HADS-D, and BDI-II. British Journal of General Practice, July, 419-426,
DOI:10.3399/bjgp11X583209
2
Beck, A.T., Steer, R.A., Ball, R., et al. (1996). Comparison of Beck Depression Inventories-IA and -II in
psychiatric outpatients. J Pers Asses 67(3): 588–597.
3
Gigerenzer, G. (2011) What are natural frequencies? Doctors need to find better ways to communicate risk to
patients. British Medical Journal 343:d6386.
4
De Graaf, R. et al. (2012). Prevalentie van psychische aandoeningen en trends van 1996 tot 2009; resultaten
van NEMESIS-2. Tijdschrift voor Psychiatrie, 54; 27-38.
5
http://www.tijdschriftvoorpsychiatrie.nl/issues/449/articles/9311
6
https://www.nhg.org/standaarden/volledig/nhg-standaard-depressie-tweede-herziening#idm10758112
7
Űstün, T.B., Sartorius, N. (eds). Mental illness in general health care. An international study. Chichester, John
Wiley & Sons Ltd, 1995.