Sunteți pe pagina 1din 9

COMMENTARIES 553

J Epidemiol Community Health: first published as 10.1136/jech.57.8.558 on 25 July 2003. Downloaded from http://jech.bmj.com/ on October 25, 2019 by guest. Protected by copyright.
Psychosocial epidemiology health can only be assessed if these
................................................................................... pathways are modelled simultaneously,
something that has not yet been at-

Psychosocial factors and public tempted. The causal sequence would be


A (social position) leading to X (various
pathways: social, cultural, psychological,
health and economic) that in turn leads to B (ill
health). The authors accept the existence
A Singh-Manoux of this causal chain: both the link
between A and psychosocial-X, and that
...................................................................................
between psychosocial-X and health. At-
tempts to assess the impact of the
Another point of view (see page 565) psychosocial pathway, or any other path-
way, on health needs to be carried out

M
acleod and Davey Smith state which “poor people feel about their pov- within this sequential causal framework.
that the aim of their paper is to erty”. This restrictive view of psychoso- Neglect of temporal order by treating
critically examine the role of cial variables negates the importance of psychosocial variables as another subset
psychosocial factors in health.1 Unfortu- the ubiquitous association between so- of factors along with measures of social
nately, what could have been an interest- cial disadvantage and a host of psychoso- position in multiple regression type
ing discussion is compromised by the cial variables in the developed world. We analysis has been shown to systemati-
authors implicit assumption that there is still know very little about the mecha- cally underestimate their role in disease
a single pathway linking social position to nisms that create and sustain this link, aetiology.5 It is therefore necessary to
health. The authors seem to equate and when in the lifecourse this link is envisage new ways of examining the
parsimonious causal analysis with a nar- established. links between social structure and
row, reductionistic perspective, subse- In considering the importance of psy- health.
quently devoting most of their paper to a chosocial variables to health Macleod Figure 1 shows a simple example of
discussion of “psychosocial versus mate- and Davey Smith create a false di- the way in which the relative importance
rial explanation”, while ignoring evi- chotomy between “objective disease” of different pathways linking social
dence showing multiple pathways link- and “misery”. They themselves acknowl- structure to health could be modelled.
ing social position to health.2 3 edge “misery” to be a legitimate public This causal model respects the sequen-
health issue, particularly in the devel- tial relation between the variables, pay-
This commentary widens the debate
oped world with increasing life expect- ing heed to the importance of distin-
by considering three issues:
ancies. It may be important to examine guishing between proximal and distal
the links between “misery” and lifestyle variables in a causal chain.5 6 Structural
(1) PSYCHOSOCIAL VARIABLES:
in light of the World Health Organisation equation modelling (SEM) would allow
DEFINITION AND THEIR claim that “lifestyle-related diseases and
IMPORTANCE TO HEALTH the relative size of each of these path-
conditions are responsible for 70–80% of ways to be assessed. SEM has the added
Psychosocial variables encompass two deaths in developed countries”.4
categories of variables. The first consists advantage of allowing latent constructs
The authors also discuss the part to be modelled, enabling a comprehen-
of psychological attributes like hostility, played by “reporting bias” (people who
depression, hopelessness, etc, which sive assessment of all variables in the
report feeling miserable also report feel-
exist at the individual level, and are model. There are some recent examples
ing ill) in explaining the association
likely to be a result of the process of of SEM7 8 and alternative approaches to
between psychosocial exposure and ill-
socialisation. The second category is modelling pathways in the literature,9 10
ness. However, one feels that this is a
more structural in nature, work condi- demonstrating the way in which com-
diversion as the authors go on to cite evi-
tions for example. These two categories plex analytical techniques can be used to
dence showing psychosocial exposures
work synergistically at the individual answer complex questions.
to be associated with “objective” health
level, as can be seen from social support To assess the “independent effect” of
outcomes.
at work, which is a function of both work psychosocial variables, Macloed and
conditions and personal social interac- (2) SOCIAL STRUCTURE AND Davey Smith put their faith in the coun-
tion skills. Although the authors start HEALTH: ARE PSYCHOSOCIAL terfactual model of causation. The basis
of establishing causality here is the prob-
out with a similar definition of psychoso- VARIABLES ON THE PATHWAY?
ability of disease in the exposed group
cial variables, in fact they interpret them The relative importance of different
that would have occurred had they not
rather narrowly as being the way in pathways linking social position to
been exposed. As random assignment of
psychosocial variables is not feasible, the
authors recommend an examination of
the impact of psychosocial interventions.
However, psychosocial intervention stud-
ies are unlikely to shed any light on the
importance of psychosocial variables.
This is primarily because the counterfac-
tual contrast being set up is meaningless
if social structure is inextricably associ-
ated with psychosocial variables.11 Let us
take the example of a “psychosocial
intervention” set up to improve social
support at work for the socially disadvan-
taged group. This would involve
achieving a minimum of two things: fun-
damental changes in the structure of
Figure 1 Relation between social position and health. work, and instant learning of appropriate

www.jech.com
554 COMMENTARIES

J Epidemiol Community Health: first published as 10.1136/jech.57.8.558 on 25 July 2003. Downloaded from http://jech.bmj.com/ on October 25, 2019 by guest. Protected by copyright.
social skills normally acquired over the structure, work-life balance, etc. Psycho- 2 Davey Smith G, Hart C, Hole D, et al.
Education and occupation social class: which
lifecourse. The near infeasibility of such social variables are important both be-
is the more important indicator of mortality
an intervention is clear. The way ahead cause they affect quality of life (“mis- risk? J Epidemiol Community Health
entails choosing appropriate statistical ery”) and are on the causal pathway to 1998;52:153–60.
models that reflect advances in concep- somatic disease.12 As public expenditure 3 Marmot M, Theorell T. Social class and
cardiovascular disease: the contribution of
tual and theoretical models. on health encompasses both these out- work. Int J Health Serv 1988;18:659–74.
comes, policy implications need to ad- 4 World Health Organisation. The world
(3) PSYCHOSOCIAL VARIABLES dress them both. health report. Geneva: WHO, 1995.
5 Weitkunat R, Wildner M. Exploratory causal
AND POLICY IMPLICATIONS In conclusion, any discussion on psy-
modeling in epidemiology: are all factors
Macleod and Davey Smith are quite right chosocial variables is welcome as it is created equal? J Clin Epidemiol
in stating that amelioration of social likely to promote development of both 2002;55:436–44.
inequality in health is a priority for pub- theory and method aimed at under- 6 Singh-Manoux A, Clarke P, Marmot M.
Multiple measures of socioeconomic position
lic health policy in most economically standing the links between social struc- and psychosocial health: proximal and distal
developed countries. However, they be- ture and health. effects. Int J Epidemiol 2002;31:1192–9.
lieve that “psychosocial solutions do not 7 Chandola T, Bartley M, Sacker A, et al.
J Epidemiol Community Health Health selection in the Whitehall II study, UK,
necessitate fundamental social change”, 2003;57:553–554 Soc Sci Med 2003;56:2059–72.
while accepting the causal link between 8 Eamonn F, James D, O’Hehir F, et al. Pilot
social disadvantage and psychosocial study of the roles of personality, references,
..................... and personal statements in relation to
adversity. It seems difficult to under-
Author’s affiliations performance over the five years of a medical
stand how psychosocial change would degree. BMJ 2003;326:429–32.
A Singh-Manoux, International Centre for
work without a change in social inequal- 9 Didelez V, Pigeot I, Dean K, et al. A
Health and Society, Department of
ity to which it is causally linked. This Epidemiology and Public Health, University comparative analysis of graphical interaction
commentary calls for a push in social and logistic regression modelling: self-care
College London, 1–19 Torrington Place, London
and coping with a chronic illness in later life.
epidemiology towards understanding WC1E 6BT, UK
Biometrical J 2002;44:410–32.
the mechanisms by which social struc- Correspondence to: Dr A Singh-Manoux; 10 Greenland S, Brumback B. An overview of
ture influences psychosocial variables. relations among causal modelling methods. Int
A.Singh-Manoux@public-health.ucl.ac.uk
J Epidemiol 2002;31:1030–7.
Socialisation agents may be responsible, 11 Kaufman JS, Cooper RS. Seeking causal
and the part played by parents, schools, explanations in social epidemiology. Am J
and other agents needs to be elucidated. REFERENCES Epidemiol 1999;150:113–20.
Policy should also be directed towards 1 Macleod J, Davey Smith G. Psychosocial 12 Marmot M, Wilkinson RG. Psychosocial and
factors and public health: a suitable case for material pathways in the relation between
improving the structural aspects of psy- treatment? J Epidemiol Community Health income and health: a response to Lynch et al.
chosocial variations, in terms of work 2003;57:565–70. BMJ 2001;322:1233–6.

................................................................................... with social position, and hence poten-


tially with health, as health is socially

Authors’ reply patterned. Our central concern, as public


health scientists, should be to establish
which of these associations are causal,
................................................................................... rather than merely correlational. We
need to make this distinction because
non-causal associations will not form
the basis for effective interventions to

W
e are sorry that Dr Singh- to sunny places, thus increasing the risk improve population health and reduce
Manoux felt our discussion of melanoma or death in plane crashes, health inequalities. We have made no a
was not as interesting as it explains why these two causes of death priori implicit assumption that only
could have been, and while we recognise often show a strong positive social material pathways link social position to
that this is necessarily true, her rejoinder gradient.1 While recognising the fact that health, and have only argued that the
leaves us unclear as to why she feels this psychosocial experiences reflect events evidence should be examined critically.
way. Most of the points she raises in her in the external world impacting on the But we make no apology for continuing
commentary relate to areas we discussed micro-processes of brains of individuals, to emphasise the probable key role for
in some detail. we will refer to “material” causes in this material factors. Across all the different
Dr Singh-Manoux accuses us of “ig- response as those aspects of the world classification schemes what, fundamen-
noring evidence showing multiple path- that will influence health independent of tally, defines differences in social
ways linking social position to health”. the psychological response they engen- position?6 We suggest, differences in the
On the contrary, we did exactly the power to access material assets and,
der.
opposite. In our view there are multiple linked to this, the power to make healthy
Recognition of these (and many
but specific pathways between social choices. Wealth is required to convert
other1) specific pathways is explicitly
position and health outcomes, as we knowledge to health.7
have discussed in depth elsewhere.1 opposed to the “general susceptibility” However, as we clearly stated, the
Examples include childhood living con- theories that underlie much of the main purpose of our paper was not to
ditions that predispose to Helicobacter psychosocial discourse2; it is in the consider the evidence for a material
pylori acquisition and (many decades psychosocial literature that one reads of causal hypothesis in relation to social
later) adult stomach cancer risk. The how psychoneuroendocrine pathways health inequalities. Rather it was to
current social patterning of adult stom- mediate between the external psychoso- consider the evidence for the psycho-
ach cancer risk is thus the outcome of cial environment and nearly every health social causal hypothesis. Most of this
material processes acting in the early problem imaginable.3–5 In our paper we evidence is observational, and is there-
years of life.1 Conversely an adult income acknowledge that many factors (includ- fore subject to considerable problems of
that allows the purchase of airline tickets ing psychosocial factors) are associated interpretation.8 One of these is reporting

www.jech.com
COMMENTARIES 555

J Epidemiol Community Health: first published as 10.1136/jech.57.8.558 on 25 July 2003. Downloaded from http://jech.bmj.com/ on October 25, 2019 by guest. Protected by copyright.
bias. It is interesting that Dr Singh-
Manoux feels that this is a “diversion”. Table 1 Associations between perceived stress and job control and
In fact, much of the evidence in this area subjective and objective outcomes in the West of Scotland Collaborative
of research, such as that from the White- Study and the Whitehall II Study
hall II study, has been based on links Outcome type Effects in Collaborative Study11 Effects in Whitehall II Study9 10

between psychosocial factors and physi-


Fully subjective*
cal health outcomes assessed from self
High exposure 2.66 (1.61 to 4.41) 2.02 (1.22 to 2.34)
reports.9 A comparison of these relations Medium exposure 1.37 (0.91 to 2.08) 1.44 (0.86 to 2.39)
with those involving the few objective Low exposure 1.00 1.00
physical health measures thus far re- Fully objective**
ported from Whitehall shows the latter High exposure 0.67 (0.36 to 1.26) 1.17 (0.8 to 1.8)
to be considerably weaker.10 We repro- Medium exposure 1.03 (0.71 to 1.49) 1.16 (0.8 to 1.7)
Low exposure 1.00 1.00‡
duce these data in table 1, alongside our
own from the West of Scotland Collabo- *Rose angina in both studies; †ECG abnormalities (Minnesota coding system) in both studies. All
rative Study11—which provide clear evi- estimates adjusted for age, social position, and cardiovascular risk factors other than ‡ (only
dence of reporting bias—to allow readers unadjusted estimates were reported in the paper).
to make their own assessment. Other
than as a reflection of reporting bias,
how should we explain these findings? in the US smoked, compared with 40% of the work of Weitkunat and Wildner, who
Perhaps even more important, is the all adults.16 Lifestyle thus seems an basically develop the ongoing argument
issue of confounding—are psychosocial incomplete explanation, as suggested by as to whether it is appropriate to adjust
exposures themselves health damaging the above evidence, for current social for covariates that may be causal
or are they merely markers for other fac- health gradients. The determinants of intermediates—rather than con-
tors that are causally related to physical behaviour are complex and the generally founders—in statistical models. 18 They
health?12 As Dr Singh-Manoux notes, unimpressive effects of individually tar- show that such adjustment will tend to
there currently appears an almost ubiq- geted interventions aimed at modifying accentuate apparent effects of factors
uitous association between general so- behaviour should remind us of this. more proximal to the outcome. In other
cial disadvantage and a host of psychoso- In our paper we discuss general words in the case of psychosocial factors
cial variables in the developed world. She strategies for drawing causal inference in that may mediate the relation between
then seems to chastise us for our neglect health science. We are happy to agree social position and health adjustment
of the question as to why such factors with Dr Singh-Manoux that, when pro- will tend to lead to the psychosocial
may be linked to social position. Are we spective observational data are all that measure appearing to have an effect
the only readers of the JECH who feel are available, there may well be a place “independent” of that of the more distal
that it is scarcely mysterious that a for greater use of the graphical ap- (and perhaps determining) social
lifetime of social disadvantage and dis- proaches, including structural equation position measure. Psychosocial expo-
enfranchisement may be associated with modelling, that she suggests. However sures are amenable to experimental
negative feelings in the individual expe- we reiterate our points regarding the manipulation.19–21 If they weren’t how
riencing such hardship? However, simply limitations of analytical sophistication in could they form the basis for useful
because the basis of the relation between resolving these issues, as exemplified by health interventions? Experiment re-
disadvantage and bad feeling is self the recent cases of antioxidant vitamins mains the most powerful means of
evident it does not follow that bad reducing the risk of being misled by con-
and hormone replacement therapy,
feeling self evidently causes objective
where strong observational evidence of founding and selection bias (with “Men-
physical disease. Bad feelings are clearly
protective effects against heart disease delian randomisation”—in essence a
a bad thing, but they may not be on the
has been overturned by randomised natural experiment—a close second).22
pathway between social disadvantage
controlled trial evidence.17 We disagree We doubt that Dr Singh-Manoux is really
and objective physical disease as Dr
with Dr Singh-Manoux’s dismissal of the suggesting that we abandon randomised
Singh-Manoux claims.
role of experimental studies in this controlled trials in favour of observa-
Dr Singh-Manoux then raises the
regard, and with her interpretation of tional studies analysed using structural
issue of the behavioural or “lifestyle”
pathway between negative feelings and
poorer health. We are far more accus-
tomed to hearing the argument that
neuroendocrine pathways represent the
main mechanism by which psychosocial
factors “get under the skin”.13 Social gra-
dients in heart disease in Whitehall
were, after all, equally apparent among
lifelong non-smokers.14 Furthermore, ad-
justment for lifestyle measures only
partly attenuated most of the social
inequalities in physical health reported
from Whitehall.14 15 In our own data from
Scotland, higher stress was indeed asso-
ciated with less healthy lifestyle but not
with poorer health.11 12 So we agree with
Dr Singh-Manoux, that negative feelings
may, depending upon context, feed into
unhealthy lifestyles. However the coinci-
dence of some unhealthy behaviours
with social disadvantage is compara- Figure 1 Income inequality (Gini) and sex specific, age adjusted, all cause mortality USA,
tively recent: in 1950 53% of physicians 1968–1998.

www.jech.com
556 COMMENTARIES

J Epidemiol Community Health: first published as 10.1136/jech.57.8.558 on 25 July 2003. Downloaded from http://jech.bmj.com/ on October 25, 2019 by guest. Protected by copyright.
equation modelling as an approach to areas of population science. And to 14 Marmot M. Occupational therapy or the
major challenge? Int J Epidemiol
determining causation and the loci for reiterate, the human misery generated
2002;31:1122–4.
health interventions. It seems unlikely by unfair and unequal societies is un- 15 Marmot MG, Bosma H, Hemingway H, et al.
that the methods she proposes, if applied questionably a bad thing. However, Contribution of job control and other risk
to observational data on, say, hormone whether it is also a significant cause of factors to social variations in coronary heart
disease incidence. Lancet 1997;350:235–9.
replacement therapy and heart disease physical disease seems unclear; clarify- 16 Tate C. Cigarette wars: the triumph of “the
risk would have led to reaching the right ing this issue is important because it has little white slaver”. Oxford: OUP, 1999.
conclusions either. implications for how policy might effec- 17 Davey Smith G, Ebrahim S. Data dredging,
Dr Singh-Manoux states that, “psy- tively improve peoples’ health in both bias or confounding BMJ 2002;325:1437–8.
18 Weitkunat R, Wildner M. Exploratory causal
chosocial intervention studies are un- relative and absolute terms. modelling in epidemiology: are all factors
likely to shed any light on the import- J Epidemiol Community Health created equal? J Clin Epidemiol
ance of psychosocial variables.” We think 2003;57:554–556 2002;55:436–44.
19 Patel C, Marmot MG, Terry DJ, et al. Trial of
this is an ill considered assertion based relaxation in reducing coronary risk: four year
on her idea that changing psychosocial ..................... follow-up. BMJ 1985;290:1103–6.
exposure, without changing the social Authors’ affiliations 20 Orth-Gomer K, Eriksson I, Moser V, et al.
structure that it is imbedded within, will J Macleod, Department of Primary Care and Lipid lowering through work stress reduction.
General Practice, University of Birmingham, UK Int J Behav Med 1994;1:204–14.
be difficult. This is of course true; in fact 21 Linden W, Stossel C, Maurice J. Psychosocial
we made this very point in our paper and G Davey Smith, Department of Social
interventions for patients with coronary artery
Medicine, University of Bristol, UK
elsewhere.23 But such difficulty notwith- disease: a meta-analysis. Arch Intern Med
standing, we agree with Kuper, Marmot, 1996;156:745–52.
REFERENCES 22 Davey Smith G, Ebrahim S. “Mendelian
and others, that intervention studies are 1 Davey Smith G, Gunnell D, Ben-Shlomo Y. randomisation”: can genetic epidemiology
the bullet that psychosocial epidemiol- Life-course approaches to socio-economic contribute to understanding environmental
ogy has to bite if it is to influence differentials in cause-specific adult mortality. determinants of disease? Int J Epidemiol
In: Leon D, Walt G. Poverty, inequality and 2002;32:1–22.
policy.24–26 Experimental studies in this health. Oxford: Oxford University Press, 23 Lynch JW, Davey Smith G, Kaplan GA, et al.
area will provide better evidence on true 2000:88–124. Income inequality and mortality: importance to
causality than observational studies, 2 Syme SL, Berkman LF. Social class, health of individual income, psychosocial
however cleverly the latter are analysed. susceptibility and sickness. Am J Epidemiol environment, or material conditions. BMJ
1976;104:1–8. 2000;320:1200–4.
More importantly they will tell us how, if 3 Selye H. The stress of life. New York: 24 Kuper H, Marmot M. Job strain, job
at all, these causal relations might lead to McGraw Hill, 1956. demands, decision latitude, and risk of
effective public health policy. 4 Brunner E. Stress and the biology of coronary heart disease within the Whitehall II
inequality. BMJ 1997;314:1472–6. study. J Epidemiol Community Health
We did not touch upon population 5 Garrsen B. Psycho-oncology and cancer: 2003;57:147–53.
health in our paper, but one of the key linking psychosocial factors with cancer 25 Pickering T. Job stress, control and chronic
issues with respect to viewpoints that see development. Ann Oncol 2002;13 (suppl disease: moving to the next level of evidence.
4):171–5. Psychosom Med. 2001;63:734–36.
a primary psychosocial determination of 6 Wright EO. Class counts. Cambridge: 26 Burg M, Berkman L. Psychosocial
health is that it makes little sense in Cambridge University Press, 2000. interventions in coronary heart disease. In:
regard to trends in overall population 7 Johansson SR. Death and the doctors: Stansfeld SA, Marmot MG, eds. Stress and
medicine and elite mortality in Britain from the heart: psychosocial pathways to coronary
health. Factors such as income inequal- 1500 to 1800. Cambridge: Cambridge
ity (and presumably the feelings associ- heart disease. London: BMJ Books,
Group for the History of Population and Social
2002:278–93.
ated with it), and indices of “social capi- Structure Working Paper Series number 7,
27 Lynch J, Davey Smith G. Rates and states:
1999.
tal” such as rates of participation in the 8 Rellman AS, Angell M. Resolved:
reflections on the health of nations. Int J
electoral process have all deteriorated Epidemiol (in press).
psychosocial interventions can improve
clinical outcomes in organic disease (Con). 28 Szreter S. The population health approach in
over a period when mortality rates have historical perspective. Am J Public Health
declined (fig 1).27–31 Of course the contri- Psychosom Med 2002;64:558–63.
9 Bosma H. Marmot MG. Hemingway H, et al. 2003;93:421–31.
bution of psychosocial factors may differ Low job control and risk of coronary heart 29 Lindert PH. When did income inequality rise
by particular outcomes and may be com- disease in Whitehall II (prospective cohort) in Britain and America? J Income Distribution
study BMJ 1997;314:558–65. 2000;9:11–25.
plicated by differing time lags between 30 US Census Bureau. Reported voting and
10 Stansfeld SA, Fuhrer R, Shipley MJ, et al.
exposure and disease. Nevertheless, per- Psychological distress as a risk factor for registration by race, Hispanic origin, sex and
spectives that take into account the life coronary heart disease in the Whitehall II age groups, November 1964–2000. [Web
study. Int J Epidemiol 2002; 31:248–55. page]. 2002. Available at
course influences of particular material http://www.census.gov/population/www/
11 Macleod J, Davey Smith G, Heslop P, et al.
factors on specific health outcomes are Psychological stress and cardiovascular socdemo/voting.html. (Accessed 20 Mar
largely congruent with population disease: empirical demonstration of bias in a 2003).
health trends.1 32 prospective observational study of Scottish 31 US Census Bureau. Reported voting and
men. BMJ 2002;324:1247–51. registration by region, educational attainment
We argued for the need to critically 12 Macleod J, Davey Smith G, Heslop P, et al. and labor force: November 1964 to 2000.
examine the evidence supporting a Are the effects of psychosocial exposures [Web page]. 2002. Available at
causal role for psychosocial exposures on attributable to confounding? Evidence from a http://www.census.gov/population/www/
prospective observational study on socdemo/voting.html. (Accessed 20 Mar
objective disease and raised issues of psychological stress and mortality. J Epidemiol 2003).
reporting bias and confounding in that Community Health 2001;55:878–84. 32 Leon DA. Common threads: underlying
regard. Considering these issues is 13 Brunner E. Stress mechanisms in coronary components of inequalities in mortality
standard practice in epidemiology, we heart disease. In Stansfeld SA, Marmot MG, between and within countries. In: Leon D,
eds. Stress and the heart: psychosocial Walt G. Poverty, inequality and health.
ask nothing more from the study of psy- pathways to coronary heart disease. London: Oxford: Oxford University Press,
chosocial exposure than is asked in other BMJ Books, 2002:181–99. 2000:58–87.

www.jech.com
COMMENTARIES 557

J Epidemiol Community Health: first published as 10.1136/jech.57.8.558 on 25 July 2003. Downloaded from http://jech.bmj.com/ on October 25, 2019 by guest. Protected by copyright.
Suicide naturally occurring workers’ rage, and
................................................................................... deprive them of the emotional energies
useful to carry on conflicts for employ-

Unemployment and suicide ment, as perhaps the poorest protocols


provide for, a sympathetically lead pro-
gramme could permit maintaining an
A Preti adequate psychosocial functioning and
the early identification of the most
...................................................................................
severe disorders, thus preventing their
How is your work going? worst outcomes.
Moreover, as it implies a contraction of
a person’s social network and a relevant
change in the time structure in daily life,

T
he prevention of suicidal behaviour suicide risk. However, unemployment
is still a land of hopes and promises adds independently to suicide risk in job loss may lead to a reduction in
but not of certainties. In fact, West- both men and women.2 5 Other recent surveillance that, together with the
ern countries are facing a general decline studies found that exposure to unem- availability of lethal means, is another
in suicide rates that seems unrelated to ployment is related to suicidal ideation key element in suicide, particularly
any national plan aimed at obtaining the and behaviour, even when taking into among mentally troubled people. An
desired outcomes in those situations that account known psychosocial confound- effort to provide families with adequate
are known to be associated to suicidal ing factors and reverse causality.6 Unem- information on this topic could be
behaviour.1 General improvement in liv- ployment, therefore, should be consid- implemented through first level health
ered a true risk factor for suicide. resources—that is, the network of gen-
ing conditions, better access to care, and
To exploit this increased awareness of eral practitioners.
more effective treatments of mental dis-
the role of unemployment in the path- Unemployment is also a considerable
orders are the most probable reasons for
ways to suicide, however, we need to source of social stress leading to increased
the recent decrease in suicide rates in
infuse a creative effort that may take us a family tensions, increased isolation from
many countries. However, the most
little ahead of common sense. others, and the loss of self esteem and
recent financial-economic turmoil and
At a first glance, it would seem that confidence. The loss of employment,
the current threatening climate of per-
the role of clinicians and researchers in indeed, implies the loss of social contact
manent war will have a foreseeable and activity, and often leads to the sever-
fostering public awareness on the role of
impact on the standard of living, the ing of social ties. A well integrated social
social factors in negative psychological
consequences of which are still to be network plays an important protective
outcomes would merely end in support-
evaluated. part in maintaining mental health, offer-
ing public welfare programmes. How-
Socioeconomic events are known to ing support, guidance and assistance,
ever, suicide rates were found to increase
produce important fluctuations in suicide favouring compliance with medical or
over time in the states that had reduced
mortality. Unemployment, in particular, psychiatric treatment and offering swift
their per capita expenditure for public
seems related to suicide risk along direct welfare; conversely, states that spend aid in the case of a self destructive act.
and indirect pathways. Blakely and co- more on public welfare also have lower Again, increasing access to health services
workers’ paper in this issue2 adds to suicide rates.7 and resources might reduce the negative
evidence indicating a causal association This is not, however, the whole story. A impact of job loss. Multiplying the points
between unemployment and suicide. closer look at the pathways from unem- of entry to the health network, even using
Their results indicate that this association ployment to psychological maladjust- the still unexplored potentiality of the
is not attributable to confounding factors ment and—hence—to suicide could per- internet, ought to favour access to treat-
linked to the socioeconomic status and mit the definition of reasonably ment when necessary.
that it is only partly related to health practicable strategies aimed at prevent- A different set of explanations,
selection or mental disorders. Statistical ing the most negative outcomes. grouped under the “health selection”
analyses permit the authors to calculate Job loss usually comprises a whole hypothesis, asserts that poorer health by
that mental illnesses account for about sequence of stressful events, from antici- itself, including poorer mental health,
half of the deaths, however the effect of pation of job loss, to job search, and increases the risk of unemployment:
unemployment cannot be discounted training for re-employment, when possi- thus, having a disorder that implies a
solely on this basis. In longitudinal stud- ble. Exclusion from ordinary living pat- higher risk of suicide would also lead to
ies unemployment predates symptoms of terns, customs, and activities arising unemployment. Even assuming this ex-
depression.3 Moreover, the lack of eco- from a lack of resources adds independ- planation, which Blakely and coworkers’
nomic independence as a result of unem- ently to the stress caused by job loss, and paper seems to discount, providing sup-
ployment reduces the possibility of using further increases the risk of depression port and working opportunities to men-
social and health services appropriately: and subsequent suicide. It is therefore tally suffering patients would protect
this may prejudice compliance with mandatory, whenever a lasting period of them from the risk of suicide. In a 20 year
therapeutically prescribed treatments, unemployment is foreseeable, particu- prospective study on a large sample of
contributing to a worsening in the course larly when middle aged people encoun- psychiatric outpatients, unemployment
of a mental disorder. ter job loss because of factory closure, to was the most evident social factor that
The most disruptive effect of unem- supply a psychological counselling serv- had an impact on suicide risk together
ployment, however, acts on social ties at ice that may replace the informative, with clinical ones, such as suicide idea-
both individual and community level. emotional, and material supportive re- tion, and major depressive and bipolar
Measures of social fragmentation, in- sources diverted by unemployment. disorders.9 Whenever possible, any effort
deed, were found to predict the risk of Some pioneering studies found that should be done to keep all the patients
death by suicide and alcohol related psychological counselling programmes with a mental disorder employed.
diseases.4 could prevent the decline in self esteem Paying attention to the immediate
Socioeconomic variables are likely to and mood that generally occurs after health consequences of unemployment
contribute to the impact of employment being made unemployed.8 Although also could produce lasting positive ef-
status on suicide. In the USA, the lower such a service might be seen by trade fects on public spending. It is interesting
the socioeconomic status, the higher the unions as an attempt to counteract to see that growing financial difficulties,

www.jech.com
558 COMMENTARIES

J Epidemiol Community Health: first published as 10.1136/jech.57.8.558 on 25 July 2003. Downloaded from http://jech.bmj.com/ on October 25, 2019 by guest. Protected by copyright.
which are likely to be linked to rising health programmes, which emphasise causal association? J Epidemiol Community
Health 2003;57:594–600.
unemployment rates, are also associated costs containment and saving. Any effort 3 Dooley D, Catalano R, Wilson G. Depression
to an increased use of public funded will be in vain, however, if the clinicians and unemployment: panel findings from the
Epidemiological Catchment Area study. Am J
facilities. From 1988 to 1994, for exam- fail to use the most sensitive instrument Community Psychol 1994;22:745–65.
ple, the number of patients discharged they have: the ability to listen to patients 4 Whitley E, Gunnell D, Dorling D, et al.
from US hospitals with a diagnosis of a and their families’ complaints. Always Ecological study of social fragmentation,
poverty, and suicide. BMJ 1999;319:1034–7.
mental illness increased from 1.4 to 1.9 ask: how is your work going? 5 Kposowa AJ. Unemployment and suicide: a
millions over the whole period.10 In cohort analysis of social factors predicting
J Epidemiol Community Health suicide in the US National Longitudinal
particular, the rate of discharges with a 2003;57:557–558 Mortality Study. Psychol Med
diagnosis of a severe mental illness 2001;31:127–38.
6 Fergusson DM, Horwood LJ, Woodward LJ.
significantly increased from 196 to 314 ..................... Unemployment and psychosocial adjustment
per 100 000 of the general population. It Author’s affiliations in young adults: causation or selection? Soc
seems that the change in mental health Sci Med 2001;53:305–20.
A Preti, Department of Psychology, University of 7 Zimmerman SL. States’ spending for public
care provision that occurred in the USA Cagliari, and Genneruxi Medical Centre, welfare and their suicide rates, 1960 to
with the institution of the Medicaid pro- Cagliari, Italy 1995: what is the problem? J Nerv Ment Dis
2002;190:361–3.
gram diverted the most severe patients Correspondence to: Dr A Preti, Centro Medico 8 Proudfoot J, Guest D, Carson J, et al. Effect
to the public sector, so that public Genneruxi, via Costantinopoli 42, I-09129, of cognitive-behavioural training on
Cagliari, Italy; apreti@tin.it job-finding among long-term unemployed
programmes have increasingly replaced people. Lancet 1997;350:96–100.
private insurance as the most important 9 Brown GK, Beck AT, Steer RA, et al. Risk
source of payment in the USA. REFERENCES factors for suicide in psychiatric outpatients: a
20-year prospective study. J Consult Clin
Being creative in counteracting the 1 De Leo D. Why are we not getting any closer Psychol 2000;68:371–7.
most negative consequences of to preventing suicide? Br J Psychiatry 10 Mechanic D, McAlpine DD, Olfson M.
2002;181:372–4. Changing patterns of psychiatric inpatient
unemployment could therefore usefully 2 Blakely TA, Collings SCD, Atkinson J. care in the United States, 1988–1994. Arch
interlace with current active public Unemployment and suicide. Evidence for a Gen Psychiatry 1998;55:785–91.

Suicide not always result in worse mental


................................................................................... health.7 Indeed, a minority of people
show an increase in psychological well-

Unemployment and suicidal being once they become unemployed.


For which people does unemployment
have a deleterious impact (including an
behaviour increased risk of suicidal behaviour) and
for which people does it have a beneficial
D Lester, B Yang impact?
Blakely and his colleagues in their
................................................................................... article in this issue, using indirect meth-
ods, argue that about half of the in-
The role of economic policy creased risk of death from suicide is
attributable to the mediating role of the

I
n the mid-1980s, Stephen Platt pub- For future research, there are several increased level of mental illness. Eventu-
lished two reviews of the literature issues that need to be addressed. Firstly, ally, the issue of the role of mental illness
that indicated that unemployment the discrepancy between the results of in the association between unemploy-
was associated with an increased risk of studies at the individual level and at the ment and suicidal behaviour can be
completed suicide and an increased risk aggregate level needs to be addressed. resolved only by a study of people who
of attempted suicide (sometimes re- Why do time series studies of unemploy- receive adequate psychiatric evaluations
ferred to as parasuicide).1 2 As we have ment and suicide rates fail to find a while employed and subsequently when
pointed out, the association between consistent association, an inconsistency unemployed, together with appropriate
unemployment and suicidal behaviour found also in cross sectional studies over, control groups.
seems to be more reliable at the indi- for example, regions within a county? This The association between unemploy-
vidual level than at the aggregate level.3 discrepancy between the results of what ment and suicidal behaviour also raises
For example, in time series studies of 14 we have called macrosocionomic and another issue, one concerning public
nations with available data for the period microsocionomic research designs6 is com- policy decisions. At the present time,
1950–1985, Lester and Yang found a mon to many phenomena in the social and before construction projects are approved
positive association between unemploy- behavioural sciences and raises difficult by governments (local and national),
ment and completed suicide rates in only problems for sociological theories. environmental impact statements are
10 nations, and this association was sta- Secondly, the role of mental health in demanded and, if the environmental
tistically significant in only four the association between unemployment impact is considered to be too harmful,
nations.4 and suicidal behaviour needs to be the project may be delayed and even for-
The article by Tony Blakely and his col- explored further. Does unemployment bidden. Threatening the extinction of a
leagues in this issue of the journal increase the risk of serious psychiatric rare species or introducing toxic chemi-
provides excellent support for the associ- problems that in turn increase the risk of cals into the local environment are the
ation between unemployment and com- suicidal behaviour or, alternatively, are kinds of impacts that can thwart a
pleted suicide at the individual level.5 The those with psychiatric problems more project.
use of national records in a single country likely to become unemployed and also Economic decisions made by local and
for over 2 million 18–64 year olds provides more likely to engage in suicidal behav- national governments apparently have
a sample far greater than samples used in iour? an impact on people. In the present
previous research, and the inclusion of Ezzy has noted, in his review of the instance, unemployment seems to lead
control variables makes the conclusions of association between unemployment and to an increased mortality from suicide. It
the study more meaningful. mental health, that unemployment does is clear, therefore, that economic policy

www.jech.com
COMMENTARIES 559

J Epidemiol Community Health: first published as 10.1136/jech.57.8.558 on 25 July 2003. Downloaded from http://jech.bmj.com/ on October 25, 2019 by guest. Protected by copyright.
decisions made by governments (or by social scientists concerning the psycho- REFERENCES
their designated decision makers such as logical impact of unemployment in order 1 Platt SD. Unemployment and suicidal
the Federal Reserve Bank in the USA) to help them make appropriate behavior. Soc Sci Med 1984;19:93–115.
2 Platt SD. Parasuicide and unemployment. Br J
can have a tremendous impact on the decisions.9 This should become more Psychiatry 1986;149:401–5.
population. We have suggested that common, and it would provide an impor- 3 Lester D, Yang B. The economy and suicide.
those making such decisions should pre- tant role for social scientists in future Commack, NY: Nova Science, 1997.
4 Lester D, Yang B. Suicide and homicide in
pare formal “impact” statements in the public policy decisions. the 20th century. Commack, NY: Nova
same way that developers and construc- J Epidemiol Community Health Science, 1998.
tion companies are required to do.8 A 5 Blakely TA, Collings SCD, Atkinson J.
2003;57:558–559
Unemployment and suicide. Evidence for a
focus on solely economic issues may causal assoication? J Epidemiol Community
suggest particular actions for public Health 2003;57:594–600.
policy makers, while consideration of the ..................... 6 Lester D, Yang B. Microsocionomics versus
macrosocionomics as a model for explaining
psychological and social impact of those Authors’ affiliations
suicide. Psychol Rep 1991;69:735–8.
actions may change these decisions. D Lester, Psychology Program, The Richard 7 Ezzy D. Unemployment and mental health.
Indeed, many European nations have Stockton College of New Jersey, Pomona, New Soc Sci Med 1993;37:41–52.
Jersey 08240–0195, USA 8 Yang B, Lester D. New directions for
regulations in place preventing compa- B Yang, Department of Economics and economics. Journal of Socio-Economics
nies from laying off employees during International Business, Bennett S Lebow College 1995;24:433–46.
hard financial times for the companies, of Business, Drexel University, Philadelphia, 9 Lester D, Yang B. Memorandum. The
and the Employment Committee of the USA relationship between unemployment and
suicide. Submitted on request to the
House of Commons in the United King- Correspondence to: Dr D Lester; Employment Committee of the House of
dom has requested memoranda from lesterd@stockton.edu Commons, London, UK, 18 Jul 1994.

Suicide time previously unemployed people may


................................................................................... find work and thus vacate the “unem-
ployed” status. Secondly, it is unknown

Research on unemployment and in the analysis when job loss occurred.


All that is known is that at some point
before census night, cohort members
suicide became unemployed, but the timing of
unemployment is unknown. There is
A J Kposowa also no information on whether they had
experienced more than one episode of
...................................................................................
unemployment.
Problems and consequences Blakely et al seem to have linked the
mortality information to census data in
the three years after census night.1 While

M
any capitalist economies are significant associations were observed in this practice may have perhaps reduced
characterised by business cycles other age groups. In an effort to support the problem regarding the transitional
with concomitant increases in a causal argument, the authors have nature of employment status, it did not
joblessness during recessions and de- controlled for the usual socioeconomic eliminate it because of the long inter-
pressions, and reductions in unemploy- variables (education and income), and to census period. In the time lag between
ment in periods of economic expansion. convince readers that there is no con- the current census (the one linked to
In view of the potentially debilitating founding (selection bias) Blakely et al1 mortality data) and the prior census,
consequences of joblessness on health have also reported results of various sen- people may have still moved across the
and related outcomes, research on un- sitivity analyses using information from three categories of employment status.
employment and suicide continues to be other studies. In view of the above issues, it is
relevant in both epidemiology and soci- The analysis was competently done, imperative that researchers find ways to
ology. One controversy that continues is but the study is not without serious accommodate peculiarities associated
the issue of selection bias. The essential limitations. Firstly, the key independent with time varying covariates in cohort
question that remains unresolved is variable in the report, employment sta- and other longitudinal studies. A signifi-
whether the observed association be- tus is a time varying covariate, but it is cant part of the problem in this and most
tween unemployment and suicide re- not treated as such in the analysis. studies of this type is their dependence
ported in some studies reflects direct Failure to account for multiple occur- on official (government collected) data-
causation or whether there is some vari- rences even in a given calendar year can sets. Censuses are not taken primarily for
able that is causally prior to both unem- distort results by underestimating or epidemiological research. In many coun-
ployment and suicide. overestimating the consequences of job- tries enumeration occurs only once in
The report by Blakely et al1 presents an lessness. In short, imprecision and inac- ten or five years (depending on national
analysis of the New Zealand Census curacies are introduced into the analysis, mandates). As only one enumeration is
Mortality Study (NZCMS) that attempts and despite confidence intervals the done there is no provision for follow up
to shed some light on the above ques- validity of conclusions become suspect. data collection on the same people on a
tion. Using logistic regression models on Previous studies2 3 suggest that the effect weekly or monthly basis. Even if that
census mortality linked data on 1.65 of unemployment on suicide may be were possible, the logistics and accompa-
million men and women aged 18 to 64 more pronounced immediately after job nying financial costs would be prohibi-
years, they have observed that unem- loss. As time progresses the newly tive. One result of this dependence is that
ployment is strongly associated with sui- unemployed adjust to their novel status very often information is needed by the
cide among women and men in the age and they may be less inclined to commit researcher but it is unavailable in admin-
group 25–64. At the same time, no suicide. Furthermore, with passage of istrative (government) statistics. At

www.jech.com
560 COMMENTARIES

J Epidemiol Community Health: first published as 10.1136/jech.57.8.558 on 25 July 2003. Downloaded from http://jech.bmj.com/ on October 25, 2019 by guest. Protected by copyright.
other times (like in the present situa- labour market and have given up looking .....................
tion), data are available but not in the for work. Some US sociologists euphemis- Author’s affiliations
format appropriate for the selected re- tically refer to this group as “discouraged A J Kposowa, University of California at
search problem. workers”.5 Their number is never known, Riverside, USA
Ideally, one would have liked to see but in periods of severe and sustained
Correspondence to: Dr A J Kposowa,
controls for mental illness and general economic downturns, it is never negligi- Department of Sociology, University of
health status both of which could di- ble especially among racial/ethnic minori- California, 1214 Watkins Hall, 900 University
rectly affect unemployment and suicide. ties and other marginalised groups. Avenue, Riverside, CA 92521, USA;
For instance, mentally ill persons may be In view of the above, it is no surprise ajkposowa@att.net
at higher risk of becoming unemployed; that Blakely et al1 found a highly signifi-
they may also be at higher risk of cant association between non-active sta-
committing suicide.4 Blakely et al admit tus and suicide in two of their multivari- REFERENCES
that they lack such data. ate models (OR=2.63, CI=1.63 to 4.25
1 Blakely TA, Collings SCD, Atkinson J.
Related to the above, another impor- for women; OR=2.59. CI=1.89 to 3.55 Unemployment and suicide. Evidence for a
tant flaw in epidemiological and socio- for men). Although the non-active group causal association? J Epidemiol Community
logical research on unemployment and includes students, homemakers, the per- Health 2003;57:594–600.
suicide using census data is in the manently sick, and retired, as it is a 2 Kposowa AJ. Unemployment and suicide: a
residual category of persons not else- cohort analysis of social factors predicting
conceptualisation and measurement of suicide in the US National Longitudinal
employment status itself. In the New where classified, it most probably has a Mortality Study. Psychol Med
Zealand Census Mortality Study, the large number of persons that had given 2001;31:127–38.
employed are those already at work. The up looking for work before the census. 3 Preti A, Miotto P. Suicide and unemployment
The odds ratio for the non-active is in Italy, 1982–1994. J Epidemiol Community
unemployed are persons that are actively
Health 1999;53:694–701.
seeking work and available for work.1 greater in magnitude than that obtained
4 Lönnqvist JK. Psychiatric aspects of suicidal
Everyone else is placed into a residual for the unemployed in both multivariate behavior: depression. In Hawton K, Heeringen
category called the “non-active”. The models. This is noteworthy in view of the KE. The international handbook of suicide and
primary limitation in this conceptualisa- fact that the analyses were limited to attempted suicide. New York: Wiley,
persons in the age group 25–64 years. 2000:107–20.
tion of employment status is that it fails 5 Ritchey PN. Explanations of migration.
to take into account people who are job- J Epidemiol Community Health Annual Review of Sociology
less, but have become discouraged in the 2003;57:559–560 1976;2:363–404.

Suicide between the most socioeconomically


................................................................................... deprived 50% of small areas compared
with the least deprived 50%. This is

Unemployment and suicide further used as a vehicle to argue that


the association is likely to be underesti-
mated. However, ecological information
E Agerbo in a micro data study might introduce
...................................................................................
“hierarchical” measurement error, which
the authors acknowledge by suggesting
Is the link always causal? this as a target for future studies.
As in other studies,5 6 Blakely and
colleagues4 find that the suicide-

S
uicide is more frequent among founder of modern statistics) said: only unemployment association in part is
people who are unemployed.1 correlation and not causation can be mediated by mental illness, which they
The suicide-unemployment asso- estimated from observational data. This primarily conclude from their sensitivity
ciation has been debated since sociolo- viewpoint, however, has recently been analysis of biases. They further explain
gist Emil Durkheim’s classic study2 over relaxed by, for example, Jamie Robins that mentally ill people would to a larger
100 years ago concluded that unemploy- who introduced the concept of counter- degree be non-active on the labour mar-
ment increased social isolation, which factual and by econometricians who for ket rather than unemployed, which then
then raised the risk of suicide. He more than 50 years have been using suggest a sensitivity analysis for the
further concluded that the number of instrumental variables to pseudo- group of those who are non-active on the
suicides in a society did not have any randomise individuals to exposure. Pro- labour market. Although Blakely and
specific association with the occurrence fessor Judea Pearl’s new book is a colleagues4 use a set of external infor-
of mental disorders at the ecological brilliant introduction to these and other mation that differs from the information
level. techniques used to strengthen causal used in the reference by Sander Green-
Many studies have suggested that the reasoning.3 land, the sensitivity analysis is one of the
suicide-unemployment link is causal, or Blakely and colleagues say that the virtues of the paper, as it demonstrates
partially caused by a selection process suicide-unemployment association an approach to deal with missing con-
governed by the effect of common found in their paper is likely to be founder information. Their lowest esti-
unobserved factors, such as mental causal.4 They argue that the link is not mated relative risk of suicide among the
illness, leading both to unemployment mediated by financial stress (which by unemployed (1.35) is quite similar to the
and suicide, or that the link is reverse the way carried surprisingly little infor- rate found in a study where information
causal, so that a suicidal behaviour leads mation in the first place), as the inci- on mental illness was included.6 This
to unemployment, or more rarely ar- dence related to unemployment is com- study, on the other hand, includes only
gued, that there is no indication of paratively unchanged in the adjusted information from population based hos-
unemployment causing suicide. The regression, and because the odds of pital discharge records, and finds also
core problem is that, as Karl Pearson (a linking suicide were almost the same higher suicide rates among the mentally

www.jech.com
COMMENTARIES 561

J Epidemiol Community Health: first published as 10.1136/jech.57.8.558 on 25 July 2003. Downloaded from http://jech.bmj.com/ on October 25, 2019 by guest. Protected by copyright.
ill than the rate used by Blakely and in their data, and we should encourage 4 Blakely TA, Collings SCD, Atkinson J.
Unemployment and suicide. Evidence for a
colleagues.4 studies such as this.
causal association? J Epidemiol Community
It might be hypothesised that the J Epidemiol Community Health Health 2003;57:594–600.
suicide-unemployment association dif- 2003;57:560–561 5 Agerbo E, Nordentoft M, Mortensen PB.
fers among people who suffer from a Familial, psychiatric, and socioeconomic risk
..................... factors for suicide in young people: nested
mental disorder, as studies have sug- case-control study. BMJ 2002;325:74–7.
gested no association7 or even a non- Author’s affiliations 6 Mortensen PB, Agerbo E, Erikson T, et al.
significant 30% reduction in risk,8 and as E Agerbo, National Centre for Register-based Psychiatric illness and risk factors for suicide
Research, University of Aarhus, DK-8000 in Denmark. Lancet 2000;355:9–12.
individual longitudinal studies of delib- Aarhus C, Denmark 7 Appleby L, Dennehy JA, Thomas CS, et al.
erate self harm and unemployment do Aftercare and clinical characteristics of people
not present a coherent picture.1 One Correspondence to: Dr E Agerbo; ea@ncrr.dk with mental illness who commit suicide: a
study even finds that the suicide rates case-control study. Lancet 1999;353:1397–
increased with increasing income among REFERENCES 400.
1 Platt S, Hawton K. Suicidal behaviour and the 8 Powell J, Geddes J, Deeks J, et al. Suicide in
patients.9 This might be the effect of an labour market. In: Hawton K, van Heeringen psychiatric hospital in-patients. Risk factors
increased stigma10 or because employed K, eds. The international handbook of suicide and their predictive power. Br J Psychiatry
and attempted suicide. New York: Wiley, 2000;176:266–72.
patients are in a particularly stressful 9 Agerbo E, Mortensen PB, Eriksson T, Qin P,
2000:309–84.
situation. 2 Durkheim E. Sjalvmordet: En av sociologiens et al. Risk of suicide in relation to income level
As acknowledged by Blakely and stora klassisker (Translation of ‘Le Suicide’). in people admitted to hospital with mental
colleagues,4 their study does not provide Uppsala: Argos, 1968. illness: nested case-control study. BMJ
3 Pearl J. Causality: models, reasoning, and 2001;322:334–5.
strong evidence in favour of the hypoth- inference. New York: Cambridge University 10 Byrne P. Psychiatric stigma. Br J Psychiatry
esis, but they try to mend imperfections Press, 2000. 2001;178:281–4.

www.jech.com

S-ar putea să vă placă și