Soc Psychiatry Psychiatr Epidemiol (2008) 43:10041007

DOI 10.1007/s00127-008-0383-4

ORIGINAL PAPER

Stefan Priebe Marya Saidi John Kennedy Gyles Glover

How to select representative geographical areas in mental health service research

A method to combine different selection criteria

Received: 3 December 2007 / Accepted: 22 May 2008 / Published online: 23 June 2008

j Abstract Background Mental health service research can require the selection of representative geographical areas for data collection. This study designed and tested a new method of combining different relevant selection criteria within the context of a survey of housing services for people with mental health disorders in England. Methods Six criteria were considered relevant to select areas for the survey: deprivation, urban-ness, provision of community mental health care, residential care provision, total mental health care spend and pressure on housing generally. A measure was identied for each criterion and established for each of 166 local areas. Variables were converted to standardised scores and multidimensional scaling undertaken to produce a single axis representing all six variables. Study sites were chosen from this. Identifying the spread of the constituent variables among the nally selected areas we established how successfully the resulting selection represented each of the selection criteria. Reliability analyses were performed on the rank positions of each area. Results The measures were converted into one axis, and all areas were ranked according to the score on that specically developed new axis. The scores on the axis showed good reliability when single criteria were eliminated from the equation. The nally selected six areas demonstrated a reasonable spread

of scores of each of the constituent variables. Conclusion Converting several relevant criteria into one score is a feasible approach to ranking geographical areas to assist in identifying small samples that are arguably representative. The method may be used widely in similar research, but requires the availability of reliable data on relevant selection criteria. j Key words sampling multidimensional scaling housing services area selection mental health service research

Introduction
Epidemiological research is often concerned with selecting representative samples from larger populations. If the task is to select people from the general population there are well established methods for this such as random household sampling or geographical cluster sampling [7]. Yet, if the task is to assess services or patients in such services, the sampling strategy to identify a representative sample may have to be more specic and consider factors that are relevant for the specic service provision rather than the general population. If the research aim is to assess the service provision and use in one country or region, the ideal solution would be to assess every single service and possibly every single service user in the whole country or region [8]. This, however, is often not feasible for several reasons: (a) Complete surveys of all services can be very time consuming and too expensive given the available resources for such research; (b) identifying all relevant services and achieving reasonable response rates across a whole country can be unrealistic; and (c) the required data collection across many areas and services would increase problems of

S. Priebe (&) M. Saidi Unit for Social and Community Psychiatry Barts and the London School of Medicine Queen Mary University of London Newham Centre for Mental Health London E13 8SP, UK E-Mail: s.priebe@qmul.ac.uk J. Kennedy G. Glover North East Public Health Observatory University of Durham Queens Campus Thornaby Stockton on Tees, UK
SPPE 383

1005

quality management, e.g. inter-rater-reliability, for the study and may thus compromise the data accuracy. The methodological challenge is to select a small number of geographical areas that can be studied with the required detail and accuracy, but still may be seen as representative for a country as a whole. Convenience sampling of areas, usually based on the availability of data or accessibility of local partners for collaboration, is common, but does not provide representative ndings. Random sampling leaves the selection to chance without the consideration of criteria that may be relevant for the selection for the given purpose. If such criteria are considered and the number of areas is small, there is no established method to utilise several criteria at the same time. For example, stratication can at best reduce the inuence of one or two criteria, but rarely three or more if the overall number of areas to be selected is small. The aim of this study was to design and test a new method to combine several relevant selection criteria into one overall dimension along which each area can be placed. The context for the development of the new method is a survey on housing services for people with mental health problems in England, yet the method should have wider applicability. The study on housing services serves as an example illustrating the principles and potential usefulness of a new methodological approach. The survey required the research team to collect detailed data on the characteristics of housing services and residents. The objectives of the survey were to identify what types of services are provided, who exactly uses them, and what the costs of the services provision are. For this we decided rst to select six geographical areas and collect complete data of all services in the given area, and then select all or some of the services in each area and a randomly selected group of patients in each service for a more detailed data collection. The number of six areas was chosen pragmatically, as this was a realistic number to be studied in detail given the resources of the project. The rst step, therefore, was
Table 1 Selection criteria and corresponding measures for each local area Criterion Likely level of mental health care need in the population Degree of urbanisation Overall level of resources for mental health care locally Extent of community-based mental health services Provision of residential care placements for mentally ill people Pressure on local housing provision Measure

to select six areas. The areas were to be representative for England considering several criteria that were seen as relevant for the given purpose.

Methods
Our study took a set of 166 areas in England as its sampling frame. The areas were the mental health local implementation areas (LITS) which are mostly coterminous with administrative areas. Based on the literature [2] and using the combined expertise of our steering group, we considered the role of housing services within the domain of health and social care for people with mental disorders, and factors likely to inuence the extent of need and the difculty of providing them. We identied six criteria on which we considered it important that our sample from this set of areas was representative, and for which we had viable direct or indirect measures for all areas of the country. Table 1 shows the measures and the detail of the data sources used. Having determined these variables, we calculated a single numerical score which represented all of them. There are two ways to do this. Had the data items all been continuous variables, it would have been possible to calculate their rst principal component. However item 2, the urbanisation score, is a ranking scale with only three categories. The appropriate alternative approach in this situation is multidimensional scaling. Calculations were done using SPSS 14. All variables were converted into standardised scores. A proximity matrix was calculated using the PROXIMITIES function. From this multidimensional scaling was undertaken using the PROXCAL function. This produces a score indicating each areas position on a single axis calculated from all the variables included. LITS were then ranked on this axis. Since, for our purposes, a nationally representative sample (of six cases) was required, we divided the full list of LITS into sextiles on the ranking and selected, as a rst choice, the middle LIT in each. As a fallback position, if LITs were unwilling or unable to participate, we determined to select alternatives immediately above or below them in a random manner. This approach was purposefully intended to avoid the extremes of each parameter. Partly this was because extreme cases may be unusual, partly because in routinely collected data, these rank positions often signify data errors. We explored how sensitive the choice was to dropping each of the constituent scores in turn. Thus, the original PROXCAL analysis was repeated six times, whilst removing a different variable from the calculations each time. Areas were then ranked again according to their new corresponding scores, and the seven lists

The mental illness needs index [4] Department for the environment, food and rural affairs definition and local authority classification method [1]. Groupings were simplified to major urban, large or other urban and all others combined and scored 1, 2 and 3 respectively. Total spend on mental health care for working age adults per adult aged 1864. Source: Mental health finance mapping, 2005 [5] Whole time equivalent professional staff employed in all community based clinical teams per adult aged 1864. Source: Adult mental health service mapping, 2005. (For methodology see [3]) Total residential placements for working age adults with mental health problems per adult aged 1864. Source: Adult mental health service mapping, 2005. (For methodology see [3]) The proportion of households over-occupied by two rooms or more. Source: Census 2001 small area statistics [6].

1006 Table 2 Selected areas, their dimension score, and their rank positions for each of the selected criteria (urban/rural status has only three categories) Area number Dimension score Mental health need Urban/rural status Per capita mental health care spend Per capita community team staff Per capita residential care beds Severe household overcrowding 50 )0.78 122 3 163 156 108 158 31 )0.54 103 3 114 73 161 94 158 )0.24 108 2 121 28 153 84 65 0.04 41 2 41 78 61 106 2 0.31 59 1 27 22 41 38 90 1.06 20 1 119 29 106 7

(the six lists resulting from this analysis and the original one) were then compared with each other as well as tested for reliability using rank correlations. To explore how successful the approach had been, we devised a method to characterise how well each constituent variable was represented in the nal selection. Variables were characterised in terms of their range and their evenness of spread within it. Since our intention was to have less than the full range, we have expressed the observed range for each constituent (the difference between the top and bottom selected rank )RT and RB) as a fraction of our chosen optimum (in this case, 153 ) 13 = 140). We scored the evenness of spread of selected sites between their extremes for each parameter by ordering the ranks and calculating the sum of the squared differences between them. For any given range and number of cases, this is minimised when the gaps are even. Its possible range for a sample of size n is: Minimum RT d2 RT 2 d2 . . . :RT n d2 ; Maximum RT RT 1 2 RT 1 RT 2 2 . . . :RT n RT n 2 : Given this, a spread coefcient for each parameter can be calculated as: 1 Obs Value Min Value=Max Value Min Value: This will ideally be 1.

urban/rural status which has only three categories). The sample works reasonably well in most respects. The range of mental health care need is narrower than would be ideal, mainly because it lacks high scorers, while the spread of values for per capita team staff is rather low. Values on this parameter can be seen to be mainly in the 20 and 70s. Identifying the extent to which the categorical constituents are represented in the nal selection is simpler. In this case the six current urban categories condensed into three. The sample contained 2/52 rural, or signicantly rural sites, 3/50 large or other urban and 1/64 major urban.

Discussion
The purpose of this study has been to dene and test a method for selecting samples of areas, for health services research studies, which can at least be argued to have a rational and repeatable basis. This is a common task in health services research. Usually, as here, the number of sites that can be managed is limited by resources so that identifying a representative sample by multiple dichotomisation is not possible. In our case high and low groupings on each of six variables would have required 64 (26) sites! The likely success of the approach depends on the extent to which the important variables are correlated. Where a single axis can account for most of their dispersion this should perform satisfactorily. In our case this would also account for the fact that dropping any individual constituent variable from the scaling calculation had little effect on the nal result. Since the sample to be chosen was small, we had little choice but to condense the variables into a single dimension for sampling. Had we been able to select a
Table 3 Range and spread coefficients for numerical constituents for the optimal selection based on the unified dimension Range coefficient Mental health need Per capita mental health care spend Per capita community team staff Per capita residential care beds Severe household overcrowding 0.73 0.97 0.96 0.86 1.08 Spread coefficient 0.89 0.74 0.66 0.85 0.91

Results
A single unied axis was calculated as described. This was relatively successful, with a normalised raw stress score of 0.073 and accounting for 0.927 of the dispersion. A best choice set of six sites was chosen from it as described.

j Sensitivity analysis
Dropping each of the constituent scores did not alter the list substantially. Rank correlations of the seven lists yielded coefcients between 0.771 and 0.993.

j Representativeness of the sample chosen

The scores for the nal sample of six sites chosen can be characterised in terms of their scores on each of the variables we considered should be representatively covered. Table 2 shows the ranks. The possible range of these was 1166. Table 3 shows the values of our two coefcients for each of the ve numerical constituents (all other than

1007

larger number it could have been sensible to identify two dimension scores and specify representative positions by reference to both. This would have been desirable if the dispersion accounted for was less in a single axis model. The use of ranks rather than actual scores to evaluate the result was intended to allow for the fact that one selection criterion is ranked categorically and a number of constituent variables are not normally distributed. In this case the chosen approach should reect the pattern in the population distribution. The method is only intended to produce a sample which is representative on each constituent independently of the others. To produce a full set of combinations of the constituents would obviously require much larger samples and a different approach. Our approach to selecting from the single unied axis could be modied. One obvious example would have been to take the top and bottom and evenly spaced sites in between. This is a matter of choice. The important contribution we think this makes is the denition of a standard way to characterise how successfully a representative sample was actually achieved. The method produced a selection of six areas that may be regarded as representative considering selection criteria that were conceptually identied as relevant. The identied axis was stable, and the scores of the constituent variables can be seen as reasonably distributed in the nal selection. One might question whether we conceptually chose the most appropriate criteria for the purpose of the survey. This was a conceptual decision. Our aim in this paper however is methodological: to demonstrate how different criteria can be combined to select representative areas. This method may be applied using different criteria and for different purposes. Once areas have been identied, further sampling methods may be applied to select specic services in those areas or patients in those services or both for a detailed data collection. Thus, the presented sampling of areas may be the rst step of a survey that nally assesses characteristics of services and of individual patients using the services. The method can be replicated and has the strength to be transparent and based on a conceptual decision on the relevant selection criteria. The original conceptually driven selection of six criteria was not altered anymore during the analysis and later stages of the study. The method provided a relatively robust axis with a good level of stability with respect to the

inuence of individual constituent variables. Moreover, in case the data collection is not feasible in a selected area, the method provides an approach for replacing those areas with other areas with similar characteristics. However, the method also has limitations and can be applied only if (a) there is a sampling frame of a sufcient number of areas and their sizes are useful for the given research question, (b) a conceptual decision on relevant selection criteria can be taken, (c) reliable data on the chosen selection criteria is available for more or less all areas in the sampling frame, and (d) the selection criteria are sufciently correlated. One may conclude that the study has provided a new method for selecting representative areas for mental health service research. The data suggests that the selection represents a reasonable distribution of all the selection criteria that had been dened as relevant. It remains to be seen whether the approach will be useful in other contexts and for other research questions. Yet, in the absence of a more appropriate method to select representative areas, the approach reported here may be worth using in future research.

References
1. Bibby P, Shepherd J (2005) Developing a new classication of urban and rural areas for policy purposesthe methodology. F. a. R. A. Department for the Environment. Available from: http:// www.defra.gov.uk/rural/ruralstats/rural-defn/ Rural_Urban_Methodology_Report.pdf 2. Fakhoury WKH, Murray A, Shepherd G, Priebe S (2002) Research in supported housing. Soc Psychiatry Psychiatr Epidemiol 37:301315 3. Glover G, Barnes D, Wistow R, Bradley S (2005) Mental health service provision for working age adults in England 2003. University of Durham Centre for Public Mental Health, Durham, UK 4. Glover GR, Robin E, Emami J, Arabscheibani GR (1998) A needs index for mental health care. Soc Psychiatry Psychiatr Epidemiol 33:8996 5. Ingham A (2006) Mental health nance mapping for english local implementation teams. Autumn review 2005 6. Ofce for National Statistics, 2001 Census: standard area statistics (England and Wales). ESRC/JISC census programme, census dissemination unit, Mimas (University of Manchester) 7. Peen J, Dekker J, Schoevers RA, teb Have M, de Graaf R, Beekman AT (2007) Is the prevalence of psychiatric disorders associated with urbanization? Soc Psychiatry Psychiatr Epidemiol 42:984989 8. Rezvyy G, iesvold T, Parniakov A, Ponomarev O, Lazurko O, Olstad R (2007) The barents project in psychiatry: a systematic comparative mental health services study between Northern Norway and Archangelsk County. Soc Psychiatry Psychiatr Epidemiol 42:131139