Clinical Psychology Review 31 (2011) 11981208

Contents lists available at ScienceDirect

Clinical Psychology Review

Toward a denition of intolerance of uncertainty: A review of factor analytical studies of the Intolerance of Uncertainty Scale
Jane Birrell a,, Kevin Meares b, Andrew Wilkinson c, Mark Freeston a, b
a b c

Newcastle University, School of Psychology, 4th Floor Ridley Building, Framlington Place, Newcastle Upon Tyne, NE1 7RU, England, UK Northumberland Tyne and Wear NHS Foundation Trust, St Nicholas Hospital, Jubilee Road, Gosforth, Newcastle Upon Tyne, NE3 3XT, England, UK North Tyneside Primary Care Trust, North of Tyne NHS, Bevan House, 1 Esh Plaza, Sir Bobby Robson Way, Great Park, Newcastle Upon Tyne, NE13 9BA, England, UK

a r t i c l e

i n f o

a b s t r a c t
Since its emergence in the early 1990s, a narrow but concentrated body of research has developed examining the role of intolerance of uncertainty (IU) in worry, and yet we still know little about its phenomenology. In an attempt to clarify our understanding of this construct, this paper traces the way in which our understanding and denition of IU have evolved throughout the literature. This paper also aims to further our understanding of IU by exploring the latent variables measures by the Intolerance of Uncertainty Scale (IUS; Freeston, Rheaume, Letarte, Dugas & Ladouceur, 1994). A review of the literature surrounding IU conrmed that the current denitions are categorical and lack specicity. A critical review of existing factor analytic studies was carried out in order to determine the underlying factors measured by the IUS. Systematic searches yielded 9 papers for review. Two factors with 12 consistent items emerged throughout the exploratory studies, and the stability of models containing these two factors was demonstrated in subsequent conrmatory studies. It is proposed that these factors represent (i) desire for predictability and an active engagement in seeking certainty, and (ii) paralysis of cognition and action in the face of uncertainty. It is suggested that these factors may represent approach and avoidance responses to uncertainty. Further research is required to conrm the construct validity of these factors and to determine the stability of this structure within clinical samples. 2011 Elsevier Ltd. All rights reserved.

Article history: Received 6 February 2011 Received in revised form 18 July 2011 Accepted 20 July 2011 Available online 28 July 2011 Keywords: Generalized anxiety disorder Worry Intolerance of uncertainty Test validity Psychometric properties Factor analysis

Contents Part one . . . . . . . . . . . . . . . . . . 1.1. The evolution of IU and its denition . 2. Part two . . . . . . . . . . . . . . . . . . 2.1. Critical analysis of the factor analytical 2.2. Method . . . . . . . . . . . . . . . 2.3. Exploratory studies . . . . . . . . . 2.3.1. Evaluation criteria . . . . . . 2.3.2. Results . . . . . . . . . . . 2.3.3. Summary . . . . . . . . . . 2.4. Conrmatory analyses . . . . . . . . 2.4.1. Results . . . . . . . . . . . 2.4.2. Summary . . . . . . . . . . 2.4.3. Conclusions . . . . . . . . . 3. Discussion . . . . . . . . . . . . . . . . . Appendix A . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . 1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . studies of the Intolerance of Uncertainty Scale . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1199 1199 1201 1201 1201 1202 1202 1202 1203 1204 1204 1204 1205 1206 1207 1207

Corresponding author at: Parklands Hospital, Aldermaston Road, Basingstoke, Hampshire, RG24 9RH, England, UK. Tel.: + 44 1256 376 426. E-mail address: jane.birrell@nhs.net (J. Birrell). 0272-7358/$ see front matter 2011 Elsevier Ltd. All rights reserved. doi:10.1016/j.cpr.2011.07.009

Generalized anxiety disorder (GAD) is characterized by excessive and uncontrollable worry (APA, 1994). There have been a number of approaches to understanding this type of worry, resulting in several theoretical standpoints, including Borkovec's ideas about the utility of

J. Birrell et al. / Clinical Psychology Review 31 (2011) 11981208

1199

worry as an avoidance of emotional imagery and processing (Borkovec, Ray, & Stober, 1998), and Mennin, Heimburg, Turk, and Fresco's (2005), model of emotional dysregulation. Other researchers have focused more on cognitive processes, such as Wells (1997), who postulates that GAD involves both worry and meta-worry processes. These various streams of research led to the development and proposal of a number of treatment protocols, (e.g., Mennin, 2004; Wells, 1999) but, despite this evidence-based approach, GAD remains the most treatment resistant of all the anxiety disorders (Gould, Safran, O'Neill Washington, & Otto, 2004). However, in the 1990s, a team in Canada began to study worry and their research led to the proposal of a new cognitive model of GAD, consisting of four main constructs; intolerance of uncertainty (IU), poor problem-orientation, cognitive avoidance and positive beliefs about worry (Dugas, Gagnon, Ladouceur, & Freeston, 1998; Fig. 1). As Fig. 1 shows, IU was identied as a key construct involved in worry and, the strong relationship between IU and worry has been replicated in numerous studies (e.g., Buhr & Dugas, 2006; Dugas, Freeston, & Ladouceur, 1997). Most importantly, treatments based upon increasing tolerance of uncertainty have been found to reduce GAD symptoms in several RCTs. (e.g., Dugas & Ladouceur, 2000) and such treatments are now recommended within national guidelines (McIntosh et al., 2004). It seems, therefore, that developing our understanding of IU may provide important insight into worry processes and to develop more effective, evidence-based treatments for worry-related disorders. Through developing such an understanding, we may begin to explore the links between IU other important processes, identied in alternative models of GAD. For example, we may be able to determine the role of IU in meta-worry, or compare intolerance of uncertainty with intolerance of aversive imagery and emotional arousal. Despite the apparent insight that this could provide, relatively little is known about the specic phenomenology of IU. This paper aims to clarify our understanding of IU by reviewing this existing literature. The rst part of the paper addresses evolution of the construct and its denition since its emergence in the 1990s. The second section attempts to explore current understanding of the nature of IU by critically reviewing existing ndings surrounding the underlying factors measured by the IUS.

1. Part one 1.1. The evolution of IU and its denition IU research began some years prior to publication of Dugas et al.'s (1998) model of GAD. At this time there was an existing and extensive literature exploring a construct referred to as Intolerance of Ambiguity (IA), which was rst derived by Frenkel-Brunswick (1948). IA was proposed to represent an individual's tendency to interpret ambiguous situations as threatening and to respond to novel, complex and insoluble situations with discomfort and avoidance (Budner, 1962; Grenier, Barrette, & Ladouceur, 2005). Krohne (1993) specied that ambiguity of a situation is determined by its unpredictability, complexity, and insolubility. He went on to suggest that ambiguity in a situation leads to uncertainty, which he proposed to be the state that is aroused within the individual faced with the ambiguous stimulus. The literature surrounding IA mainly explored its applications to non-clinical elds of psychology, such as management and accounting. However, in studies exploring worry processes, Butler and Matthews (1987) found that worriers tend to interpret ambiguous events as threatening, and Metzger, Miller, Cohen, Sofka, and Borkovec (1990) found that worriers took longer to make decisions in ambiguous situations. This existing research, combined with their own clinical observations, suggested to the team at Laval University (including Ladouceur, Freeston, and Dugas) that it may be uncertainty that led to the perception of threat in ambiguous situations and that it was the notknowing that led to worry. On this basis they began to conceptualize intolerance of uncertainty as cognitive, emotional and behavioral reactions to uncertainty in everyday life situations (Freeston, Rheaume, Letarte, Dugas, & Ladouceur, 1994, p.792) and proposed that IU played an important role in worry. Ultimately, though, this early conceptualization may best be described as a clinical hunch (M. Freeston, personal communication, September 21, 2007). Despite this, the ensuing studies were promising. For example, Freeston et al. (1994) had developed a measure of IU, the Intolerance of Uncertainty Scale (IUS; Appendix A), which distinguished between non-clinical GAD participants and non-GAD controls. The measure also demonstrated convergent and discriminant validity and excellent internal consistency (Freeston et al., 1994). This was fundamental in the development of IU research, as the IUS became the standard measure of IU in all subsequent studies. The measure, however, was largely atheoretical and derived from clinical opinion (M. Freeston, personal communication, September 21, 2007), so it remained unclear exactly what the scale was measuring (see part two for a full discussion on this issue). Regardless of this lack of conceptual clarity, a number of early studies demonstrated that IU was strongly linked to worry. Freeston et al. (1994) showed that IU was related to trait worry in a nonclinical sample, even when mood variables were controlled for. This nding was replicated by Dugas et al. (1997), who found that IU made a unique contribution to worry over and above that of personal and mood variables. Ladouceur et al. (1995) demonstrated that GAD sufferers showed higher levels of IU than those with other anxiety disorders, such as panic or social phobia. IU was also shown to distinguish GAD patients and non-clinical GAD participants, from non-clinical moderate worriers (Ladouceur, 1993; Ladouceur, Blais, Freeston, & Dugas, 1998). Furthermore, Ladouceur, Talbot, and Dugas (1997) found that IU was higher in worriers than in non-worriers when both groups were faced with a moderately ambiguous task. As a result, Ladouceur et al. (1998) proposed a new denition of IU as, the way in which an individual perceives information in uncertain situations and responds to this information with a set of cognitive, emotional and behavioral reactions (p.141), but the nature of the perceptual bias and response patterns remained unclear and the denition was still broad.

Fig. 1. Cognitive model of GAD (Dugas et al., 1998).

1200

J. Birrell et al. / Clinical Psychology Review 31 (2011) 11981208

In light of this emerging evidence, the Laval team continued to pursue the relationship between IU and worry. In order to elucidate the nature of this relationship, the team attempted to manipulate IU and observe the effects this had on worry. Ladouceur, Gosselin, and Dugas (2000) developed a gambling task, designed to increase or decrease participants' level of IU. They found participants whose IU had been increased engaged in more worry. Moreover, in the original intervention study, in which the intervention targeted reductions in IU, such reductions were seen to precede reductions in worry during the treatment. This was interpreted as demonstration of the fact that IU may be a causal risk or vulnerability factor for clinical worry (Dugas & Ladouceur, 2000) and resulted in the revised denition of IU as, the excessive tendency of an individual to consider it unacceptable that a negative event may occur, however small the probability of its occurrence (Dugas, Gosselin, & Ladouceur, 2001 p.552). Having established that IU is a key process involved in worry, attempts were made to clarify its construct validity by examining the relationship between IU and other mood and personality variables. For example, Buhr and Dugas (2006) investigated the relationship between worry, IU, IA, perfectionism and perceived control. Once again, IU was found to be the most signicant predictor of worry when all the other variables were accounted for. The authors also report a moderate and signicant correlation between IU and IA, both of which correlated strongly with worry. They conclude, therefore, that this study not only provides evidence of construct validity, but it also poses the question as to whether IU and IA can be distinguished from each other. It is perhaps unsurprising that the two constructs correlate, given that early conceptualization of IU were partially derived from the IA literature, but exploration of the similarities of differences may help to provide a more focused denitions of both IU and IA. Grenier et al. (2005) completed a review of the literature surrounding the denition and measurement of IU and IA, which concluded that these were overlapping but distinct constructs, determined by an element of time; IU referred to potentially negative future events (future-oriented) and IA referred to the interpretation of current situations (present-oriented) (Grenier et al., 2005). However, these ndings have little empirical support and further research is required to clarify the distinction between these two constructs. Such research could result in greater specicity in our understanding of IU and IA, perhaps through conrming the future-oriented characteristic as a dening feature of IU, but this has yet to be carried out. There have been further studies comparing IU and other variables, independently of the Laval University research program and its subsequent continuation by Dugas at Concordia University. For example, Butzer and Kuiper (2006) examined the inuence of IU on the frequency of social comparisons made by undergraduate participants. They found that IU was the most signicant predictor of social comparisons, over and above anxiety, depression and self-concept clarity. In addition, de Bruin, Rassin, and Muris (2007) investigated the relationship between IU, meta-worry and neuroticism with both state and trait worry. They found that, despite making a signicant and unique contribution to trait worry, IU showed only a weak association to state worry, and concluded that IU may be relevant specically to clinical worry, namely GAD. The ndings in both studies that IU predicts and makes unique contributions to other phenomena further substantiates its construct validity, but do little to add to our understanding of the nature of IU. The Laval group claim to have replicated the nding that IU is indeed specic to GAD in several studies (Dugas et al., 2001, Ladouceur et al., 1999), despite the small literature base examining the role of IU in OCD (e.g., Steketee, Frost, & Cohen, 1998). In contradiction to the Laval group's ndings, Tolin, Abramowitz, Brigidi, and Foa (2003) found that IU, as measured by the IUS, was greater in checkers than non-checkers in a clinical OCD group and than in the control group. Moreover, Holaway, Heimberg, and Coles (2006) found that IU (also measured by the IUS) was linked to worry, GAD and OCD

symptoms and that the strength of this relationship did not differ signicantly between OCD and GAD symptoms. More recently, Seagrave (2009) found a positive correlation between IU and paranoid ideation. This poses interesting questions as to whether IU may be dened differently across different disorders, but research in this area is likely to be complicated by a lack of clear denition of IU in GAD. Attempts have also been made to dene IU in terms of specic cognitive processes. This approach has mainly focused on the information processing bias suggested to play a key role in anxiety disorders, in that anxiety leads to hypervigilance, selective processing of threat cues within the environment and the overestimation of the danger associated with these threats (Clark & Steer, 1996). In a study investigating the processes involved in indecisiveness, Rassin and Muris (2005) found a strong correlation between IU and indecisiveness, and in addition, found that indecisive individuals were more likely to interpret ambiguous situations as threatening. Similar ndings were reported by Dugas et al. (2005), who found that individuals with high IU were more likely to interpret ambiguous situations from the Ambiguous Situations Diary (AUSD; Davey, Hampton, Farrell, & Davidson, 1992) as threatening. Moreover, they found that this tendency was related more strongly to IU than to worry, anxiety or depression (Dugas et al., 2005). The denition of IU underwent further revision to incorporate this new understanding and in Koerner and Dugas, 2006 dened IU as, a dispositional characteristic that reects a set of negative beliefs about uncertainty and its implications (Koerner & Dugas, 2006 in Koerner & Dugas, 2008. p.620). Koerner and Dugas (2008) further developed this idea through investigating whether IU could be conceptualized as a cognitive vulnerability factor for clinical worry. They suggest that individuals who possess high levels of a cognitive vulnerability factor are more likely to demonstrate a specic cognitive bias, so IU may be the factor that leads to the interpretation of uncertain situations as threatening or negative. In order to assess the validity of this conceptualization, Koerner and Dugas suggest that IU must meet a number of criteria, including manipulability, temporal antecedence, stability and construct validity and their literature review provides a successful demonstration of these characteristics. The subsequent study reported in this paper utilized a vignette task to investigate the appraisals of individuals with high and low IU in ambiguous situations. This resulted in further replication of the nding that individuals with high levels of IU were more likely to make negative interpretations of ambiguous situations. The authors conclude, therefore, that, intolerance of uncertainty a dispositional characteristic that arises from a set of negative beliefs about uncertainty and its connotations and consequences is an important predictor of trait level worry and of the tendency to interpret ambiguous situations in a negative manner, and as such, is considered to be a cognitive disposition that might confer risk for GAD (Koerner & Dugas, p.631). Since emergence of IU in the 1990s as a key process in worry, there has been a growing interest in the variable within psychological research. A large proportion of this research, however, has focused on determining the construct validity of IU, without attempting to understand fully the nature of this construct. Further, much of the research has been performed by within an ongoing program at Laval and Concordia Universities. Some of the studies described above have provided new insights and the denition of IU has evolved to incorporate new ndings, including the link between IU and worry, the information processing bias found in individuals with IU, and the idea that IU may be cognitive vulnerability factor. However, these denitions are somewhat categorical and do not explain the phenomenology of IU. Many questions remain unanswered, such as why do individuals who have high levels of IU nd uncertain situations more

J. Birrell et al. / Clinical Psychology Review 31 (2011) 11981208

1201

threatening than certain ones? Developing a deeper understanding and a clearer denition of IU could move us closer to answering these questions. 2. Part two 2.1. Critical analysis of the factor analytical studies of the Intolerance of Uncertainty Scale The Intolerance of Uncertainty Scale (IUS; Freeston et al., 1994) was developed as part of the early research at Laval University (see Appendix A). As mentioned in part 1, the early denitions of IU were broad and understanding of the construct was limited. Despite the lack of research, the clinicians involved had a keen sense that IU was a common and signicant factor in individuals presenting with GAD. The IUS was developed on the basis of these clinical observations (M. Freeston, personal communication, September 21, 2007), but despite its largely atheoretical nature, the scale performed extremely well in psychometric testing. In the original tests, it demonstrated excellent internal consistency (0.91), good discriminant and convergent validity and its criterion-related validity was demonstrated by distinguishing those in the non-clinical GAD group from controls (Freeston et al., 1994). On the face of it, therefore, the IUS measures a face valid construct, albeit broadly dened, and this scale was subsequently adopted as the standard measure of IU and used distinguish between high and low IU groups in all the subsequent IU studies. Measurement of IU has, thus, remained stable over time, despite denitions of the construct having evolved (see part one). Koerner and Dugas (2008) highlight the need for data regarding the latent structure of IU and we propose that exploring this latent structure will help to enhance our understanding of this phenomenon. This part, therefore, aims to review all the factor analytical studies of the IUS in order to determine whether this helps to explain some of the unanswered questions surrounding the phenomenology of this construct. 2.2. Method An extensive literature search was carried out using the databases Embase and Medline (via Ovid); Psychinfo via APANet; Web of Science; Scopus and Pilots via CSA Illumina. As the search was highly specic in terms of the methodology used, a keyword search was performed using the search terms Intolerance of Uncertainty Scale or intolerance of uncertainty. A further search was
Table 1 Studies retained for review. Authors Year of publication Title

carried out using the terms factor or factor structure or factor analysis in any eld. These were then combined using the AND command to nd articles that referred to both intolerance of uncertainty or the IUS and factor structure (see Table 1 for results). In addition to this, searches were carried out for key authors including Dugas, Freeston, Carleton, Berenbaum, de Bruin, and Norton. These authors were also contacted directly to seek advice on any extensions or replications that were awaiting publication or remained unpublished. Searches were limited by date from 1994 to present due to the initial publication of the IUS in 1994. In order to make this search as inclusive as possible, very few exclusion criteria were used. This was particularly important as we wanted to include studies that had utilized any type of factor analysis (i.e., exploratory factor analysis, conrmatory factor analysis, structural equation modeling, etc.). Studies were included on the basis that they were carried out since 1994 (as this was when the IUS was rst published), were written in English and were cited in peer reviewed journals, books and textbooks and gray literature (such as conference proceedings and dissertations). Studies were included regardless of the gender and nationality of the participants. Those in which both clinical and/or analog samples were recruited were included in this analysis, as were those in which any type of factor analysis was carried out on the IUS. Any study which did not include some type of factor analysis of the IUS and those written in languages other than English were excluded from this analysis. Application of these criteria to results of the initial database searches yielded nine papers (see Table 1 for summary). One paper was excluded due to being published in German (Gerlach, Andor, & Patzelt, 2008) and one in Spanish (Rodriguez, Leon, Rovella, & Herrera, 2006). Samples utilized in the nal nine studies were then compared. The samples showed similar characteristics, with the mean ages ranging from 19 to 23.8. All of the samples were predominantly female (57 83%). Eight studies reported the use of under- or post-graduate students with mean IUS scores between 50.28 and 54.88 (standard deviations from 14.92 to 18.71). The de Bruin, Rassin, van der Heiden, and Muris (2006) sample consisted of 209 students (mean IUS score 65.89) and 23 outpatients (mean IUS score 89.87). The samples sizes ranged from 154 up to 1230 for individual analyses. Different methods of factor analysis were utilized within these papers, including a mixture of exploratory analyses (EFA) and conrmatory analyses (CFA), the former being used to search for potential underlying factors, and the latter to test out specic hypotheses regarding the factor structure. Best practice involves use of EFA to develop hypotheses about underlying factors, and CFA to then test these out (Osborne, Costello, & Kellow, 2008). In order to follow a systematic approach to reviewing these papers, therefore, the EFA

Source Available from Jane Birrell (jane.birrell@nhs.net) Psychological Assessment Journal of Anxiety Disorders Behaviour Research and Therapy Journal of Anxiety Disorders Netherlands Journal of Psychology Journal of Anxiety Disorders Behaviour and Research Therapy Personality and Individual Differences

Birrell, J., Cavanagh, K., Dudley, R., Meares, K., (2009) Wilkinson, A., & Freeston, M. H. Sexton, K. & Dugas, M. J. (2009) Berenbaum, H., Bredemeier, K., & Thompson, R. J. Carleton, R. N., Sharpe, D, Admundson, G. Carleton, R. N., Norton, M. A., & Admundson, G. J. G. de Bruin, G. O., Rassin, E., van der Heiden, C. & Muris, P. Norton, P. J. Buhr, K., & Dugas, M. J. Freeston, M. H. Rheaume, J. Letarte H., Dugas, M. J. & Ladouceur, R. (2008) (2007) (2007) (2006) (2005) (2002) (1994)

Exploring the factor structure of the Intolerance of Uncertainty Scale within a British sample. Dening Distinct Negative Beliefs about Uncertainty: Validating the Factor Structure of the Intolerance of Uncertainty Scale Intolerance of uncertainty: Exploring its dimensionality and associations with need for cognitive closure, psychopathology and personality. Anxiety sensitivity and intolerance of uncertainty: Requisites of the fundamental fears? Fearing the unknown: A short version of the Intolerance of Uncertainty Scale. Psychometric properties of a Dutch version of the Intolerance of Uncertainty Scale. A psychometric analysis of the Intolerance of Uncertainty Scale among four racial groups. The intolerance of uncertainty scale: psychometric properties of the English version. Why do people worry?

1202

J. Birrell et al. / Clinical Psychology Review 31 (2011) 11981208

studies are evaluated rst. The ndings from these then inform the subsequent evaluation of the CFA studies. Some of the identied studies used both EFA and CFA. In these cases the analyses are be evaluated separately. 2.3. Exploratory studies The EFA studies propose various factor structures with different numbers of factors (see Table 2). In order to determine the relative contribution of each study to our understanding of IU and, therefore, the structure most likely to demonstrate stability, the papers were subjected to evaluation against a number of criteria. 2.3.1. Evaluation criteria Criteria from previous reviews (Ellis, Ladany, Krengel, & Schult, 1996; Papworth & Milne, 2001) were considered, as well as published guidelines for critical analysis (e.g., CASP; NHS Public Health Resource Unit, 2008). These resources highlighted general characteristics, such as the clarity of the research question and the extent to which the ndings are applicable to research and clinical practice, to be considered in reviewing scientic literature. In addition to this, various statistical resources were consulted (e.g., Field, 2005; Osborne et al., 2008), in order to determine the most appropriate criteria by which to evaluate the methodological quality of the factor analyses. Several important issues arose from this. Firstly, it was apparent that factor analysis requires the researcher to make a number of assumptions about their data, such as whether the variables of interest are correlated or uncorrelated. As such, the researcher must make these assumptions explicit in order for the ndings to be interpretable to the reader. Secondly, as with most analyses, sample size is a major and controversial issue within factor analysis, with several different opinions as to the size of sample required. There are, however, tests that can be used to determine the adequacy of the sample size, such as the KaiserMeyerOlkin test. Given conicting advice surrounding sampling, it is recommended that researchers utilize such tests to give an indication of the adequacy of their sample and, therefore, the likelihood that that the factor structure found is stable. Another important consideration in evaluating factor analytical studies is that this method is largely dependent upon interpretation. For example, the researcher must use visual interpretation (of data,
Table 2 Proposed factor structures. Paper Freeston et al., 1994 Factor 1 2 3 4 5 1 2 3 4 1 2 1 2 3 4 1 2 3 4 1 2 1 2 Items

graphs, plots, etc.) to determine the factor structure extracted from his data. Decisions are made in order to decide the number of factors extracted and the variables that comprise each factor. Finally, the researcher interprets the meaning of the cluster of variables within each factor so that each factor can be given a label. Such reliance on interpretation means that it is essential that the researcher is explicit about the methods used and decisions taken in obtaining the results presented, so that the reader can judge the validity and likely stability of the proposed factor structure. Osborne et al. (2008) suggests that a factor analysis provides hypotheses to be tested. He suggests that the factor structure that emerges from a factor analysis should be further tested or replicated in order to determine its stability. There are several methods of testing the stability of a factor structure, such as carrying out a conrmatory factor analysis (as recommended by Osborne et al., 2008) or comparing current ndings in previous factor analytical studies (Rummel, 1970). Use of one (or both) of these methods to demonstrate the stability of a factor structure adds validity to the ndings of an exploratory factor analytical study. These issues were considered, in order to determine specic criteria appropriate to the evaluation of factor analytical studies. A grid was then developed comprising of criteria adapted from the CASP resources (NHS Public Health Resource Unit, 2008) and the specic factor analytical criteria (see Table 3 for summary). Each section of the grid comprises criteria that were considered essential features of a high quality factor analytical paper. Where appropriate, each criterion was broken down into a number of specic quality indicators, the presence of which was indicated that the criteria had been met. Presence or absence of each quality indicator was assessed by scoring 1 for presence of the indicator and 0 for its absence (quality score). Higher quality is, therefore, indicated by a higher score. Further, descriptors were utilized to indicate the extent to which the criteria had been met, from not met, partially met, or fully met (criterion descriptor). 2.3.2. Results Application of the evaluation grid shows that all six EFA studies met the general screening criteria and were close to fully meeting the criteria for the applicability of the ndings. This shows that all six papers were of sufcient standard for further evaluation. In order to

Name Uncertainty is unacceptable and should be avoided Being uncertain reects badly on a person Uncertainty frustration Uncertainty causes stress Uncertainty prevents action Uncertainty leads to the inability to act Uncertainty is stressful and upsetting Unexpected events are negative and should be avoided Being uncertain is unfair Prospective anxiety Inhibitory anxiety No labels

Buhr & Dugas, 2002

Carleton, Norton & Admunson, 2007 Norton (2005)

Berenbaum et al. (2008)

Sexton & Dugas, 2009

7, 8, 10, 11, 18, 19, 20, 21, 27 2, 3, 6, 9, 13, 14, 15, 17, 22 4, 16, 23, 26 5, 6, 24, 26 1, 12, 20 1, 9, 12, 13, 14, 15, 16, 20, 22, 25 2, 3, 4, 5, 6, 7, 8, 9, 15, 17, 24, 26 7, 8, 10, 11, 18, 19, 21 7, 16, 18, 23, 27 7, 8, 10, 11, 18, 19, 21 9, 12, 15, 20, 25 1, 6, 7, 9, 12, 13, 14, 15, 17, 20, 22, 24, 25, 26 5, 7, 8, 10, 11, 18, 19, 21 3, 4, 6, 16, 17, 23, 24, 25, 26 2, 3, 13, 22, 23 7, 8, 10, 11, 18, 19, 21 1, 12, 14, 15, 20, 22 16, 17, 24, 25, 26 2, 3, 4, 23 1, 2, 3, 9, 12, 13, 14, 15, 17, 18, 20, 22, 23, 24, 25 4, 5, 6, 7, 8, 10, 11, 18, 19, 21, 26, 27 1, 12, 13, 14, 15, 16, 17, 20, 22, 23, 24, 25 4, 5, 6, 7, 8, 10, 18, 19, 21

Birrell et al. (Unpublished report)

Desire for predictability Uncertainty paralysis Uncertainty distress Inexible uncertainty beliefs Uncertainty has negative behavioural and self-referent implications Uncertainty is unfair and spoils everything Desire for and active engagement in seeking predictability Cognitive and behavioural paralysis in the face of uncertainty

and * indicate factors with consistent item sets.

J. Birrell et al. / Clinical Psychology Review 31 (2011) 11981208 Table 3 Evaluation grid sections and criteria. Section Screening Selection Analysis Results Criteria Study addresses clearly focussed question Authors use appropriate methods to answer their question Sample was selected and recruited appropriately Methodological assumptions are explicit Sample size is adequate Factor structure has validity Factors clearly and appropriately labeled Stability of structure is demonstrated Valid conclusions are drawn from the ndings Clinical Implications of the ndings are discussed Research implications of the ndings are discussed

1203

Applicability of ndings

examine the likely factor structure of the IUS, the selection, analysis and results sections were considered most important. The studies demonstrated varied performances across these sections. Overall, evaluation revealed four studies that demonstrated some of the methodological quality indicators, but which were also weak in areas. For example, the Freeston et al. (1994) study reports development and original validation of the IUS. It was, thus, entirely exploratory and there were no previous ndings on which assumptions could be based to guide the analysis, or to which the results could be compared. Further, this study fails to perform better on the evaluation grid due to its lack of reporting of details, such as the extraction or rotation that were utilized. This may have been due to publication restraints and the relative importance of reporting other aspects of the study. Similarly, the de Bruin et al. (2006) paper was an initial exploration of the Dutch translation of the IUS and although the methodology is appropriate to this type of investigation, the lack of explicit rationales and assumptions meant this paper did not perform well. Although the authors did not statistically demonstrate the stability of their ndings, they do report the use of multiple extractions and rotations which yielded similar results, suggesting stability of their structure. The fact that they propose a uni-factorial structure, however, and do not describe the methods used to determine item loadings or provide output matrices, means that the ndings from this analysis provide little insight into understanding IU. The Norton (2005) study attempted to explore dimensions of the IUS across different cultural groups. Results reported were inconclusive and inconsistent across the groups. This may have reected instability of the factor structure, but is also likely to have been inuenced by the relatively small sample sizes of the separate groups (between 93 and 149). Combining the groups to give one larger data set was more likely to give an accurate, stable and generalizable structure. Indeed, analysis of the full data set, with all groups combined, (P. Norton, personal communication September 7, 2008) did yield an interpretable four factor structure, although this is somewhat messy, with a number of complex items. Finally, the score obtained by the Buhr and Dugas (2002) study reects limitations such as the inadequate description of the sample and recruitment. It did, however, demonstrate several methodological strengths, such as the use of principal factor analysis as the extraction method and an oblique rotation. Unfortunately, the interpretation of the ndings was not explained fully and so the structure proposed must be considered with caution. Re-interpretation of the results may be necessary when drawing conclusions from the data presented in this paper. The remaining two studies emerged as signicantly higher quality on the evaluation grid. The Berenbaum, Bredemeier, and Thompson (2008) study did, however, have some methodological limitations. For example, Osborne et al. (2008) argues that the method of extraction used (principle components) is more appropriate for data reduction rather than exploration of latent structure. The appropriateness of

using orthogonal rotation is also questionable given the likelihood that the factors will correlate in some way and the high inter-item correlations found in previous studies (Buhr & Dugas, 2002). Moreover, the method for deciding which items to retain for each factor is not described explicitly. Despite these limitations, examination of the factor loadings presented reveals a structure with relatively high loadings (one at 0.4 and the remainder between 0.51 and 0.75), and with no complex items. The distinguishing strength of this study is the inclusion of congruence analyses with ndings from earlier papers, which revealed consistency between their rst factor (desire for predictability), Buhr and Dugas (2002) third factor (unexpected events are negative and should be avoided) and Carleton, Norton, and Admundson's (2007) rst factor (prospective anxiety). Consistency was also found between their second factor (uncertainty paralysis), Buhr and Dugas (2002) rst factor (uncertainty leads to the inability to act) and Carleton, Norton, and Admundson's (2007) second factor (inhibitory anxiety). Their third and fourth factors did not map onto any of the factors from Buhr and Dugas (2002) proposed structure. The Sexton and Dugas (2009) study emerged as the highest quality and this reects the thorough methodology and explicit descriptions utilized. Of particular note was the large sample size (n = 1230), which would be considered as an excellent absolute sample size (Comfrey & Lee, 1992), and meets published criteria for satisfactory sample size based on participant to variable ratios (e.g., Hatcher, 1994). In addition, all aspects of the analysis were carried out in line with best practice guidelines (see Osborne et al., 2008). For example, the normality of the data was tested in order to determine the most appropriate extraction method and oblique rotation was used due to the assumption that the factors were likely to correlate. The number of factors was determined through the use of multiple methods including the minimum Eigen rule, scree plot, minimum average partial test (Velicer, 1976) and parallel analysis (Horn, 1965), which resulted in retention of 2 factors with no complex or hyperplane items. Lack of hyperplane items may be regarded as indicating stability, but could also be due to the cut-off used (0.4). This results in two broad factors that are difcult to dene and a more conservative cutoff (say 0.5) may have provided more focused factors, although Sexton (personal communication, January 15, 2009) argues that the premature exclusion of items may prevent us from investigating the construct validity of the factors further. The lack of complex items in such an inclusive structure suggests that the two factor structure is, indeed, likely to be stable and accurate. A limitation of all these studies is the exclusive use of North American samples. In order to address this, Birrell et al. (unpublished report) have carried out a further EFA involving a sample of 275 British undergraduates. This study is yet to be prepared for publication and so the evaluation criteria outlined in the grid regarding reporting of methodological decisions were not applicable, and this paper should be included in future reviews in order to determine its relative quality. It is worth noting, however, that analysis of these data in accordance with Osborne, Costello and Kellow's (2008) recommendations suggests a two factor structure of the IUS which is highly consistent with the structure proposed by Sexton and Dugas (2009), thus further substantiating this model. 2.3.3. Summary Synthesis of the results of the EFAs seems to suggest that there may be two factors that are consistent throughout the ndings. Examination of the factors found in each study (see Table 2) reveals similar ndings to those in the Berenbaum et al. (2008) study namely, that there are two sets of items that consistently group together. These sets (as shown in Table 4) are also subsumed within the factors found by Sexton and Dugas (2009) and Birrell et al. (unpublished report). It is, therefore, concluded that this two factor structure should be tested

1204 Table 4 Two consistent factors and their items. Factor 1 Freeston et al. (1994) Buhr and Dugas (2002) Norton (2007) Berenbaum et al. (2008) Sexton and Dugas (2009) Birrell et al. (2009) 7, 8, 10, 11, 18, 19, 20, 21, 27 7, 8, 10, 11, 18, 19, 21 5, 7, 8, 10, 11, 18, 19, 21 7, 8, 10, 11, 18, 19, 21 4, 5, 6, 7, 8, 10, 11, 18, 19, 21, 26, 27 4, 5, 6, 7, 8, 10, 18, 19, 21

J. Birrell et al. / Clinical Psychology Review 31 (2011) 11981208

Factor 2 NA 1, 9, 12, 13, 14, 15, 16, 20, 22, 25 1, 6, 7, 9, 12, 13, 14, 15, 17, 20, 22, 24, 25, 26 1, 12, 14, 15, 20, 22 1, 2, 3, 9, 12, 13, 14, 15, 17, 18, 20, 22, 23, 24, 25 1, 12, 13, 14, 15, 16, 17, 20, 22, 25

using CFA in order to establish the stability and accuracy of this 2factor model of the latent structure of the IUS. 2.4. Conrmatory analyses Several papers from the initial search reported CFAs carried out in order to conrm the ndings from the EFAs. In addition to this, Carleton has carried out a CFA on a new unpublished data set (N. Carleton, personal communication, January 16, 2009). In order to assess the relative contributions of the CFA ndings to this investigation, the results of the studies were compared to criteria set out by Hu and Bentler (1999), who proposed desired values for each criteria, which would provide evidence for the accuracy of the model tested (see Table 4). The desired value for the BentlerBonnet Normed Fit Index (NNFI) is recommended by Tabachnick and Fidell (2001). 2.4.1. Results The inconsistency in the number of factors proposed within the EFA studies resulted in a various structures being tested within the CFAs. Moreover, the CFA studies also varied in the item sets used in the analyses. This was due to the ndings of Carleton, Norton, and Admundson (2007), who tested the t of unitary-, four- and vefactor structures (based on the ndings of Freeston et al., 1994 and Buhr & Dugas, 2002) for the full 27-item IUS. None of these solutions demonstrated adequate t and the researchers identied a high level of item redundancy within the scale. Subsequent scrutiny of the ndings from the EFAs led them to hypothesize that there were, in fact, two factors consisting of 12 items that were likely to remain stable: unacceptability and avoidance of uncertainty (Freeston et al.) and uncertainty leading to the inability to act (Buhr & Dugas). Preliminary results of CFAs on this item set yielded promising results for the two factor structure (Carleton, Norton, & Admundson, 2007), although Table 5 shows that in the attempt to replicate the
Table 5 Summary of CFA results. Model (item set) N Chi2

ndings, there was one criterion (chi 2/df) that did not meet the desired value. These results led the authors to propose a shortened version of the IUS (IUS-12; Carleton, Norton & Admundson). In a separate study, Carleton, Sharpe, and Admundson (2007) used CFA to test a model including IUS-12 items and items from the Anxiety Sensitivity Index (Peterson & Reiss, 1992). More recently, a separate CFA was carried out on the IUS-12 data alone to test the t of a two factor solution (N. Carleton, personal communication, January 16, 2009) in which the model failed to meet several of the criteria for goodness of t. Carleton's most recent CFA (N. Carleton, personal communication, January 16, 2009) yielded more promising results for the two-factor solution of the IUS-12, with most of the criteria being met satisfactorily. These ndings suggest that the two-factor solution demonstrates sufcient stability to be considered accurate and valid. Furthermore, Carleton, Norton, and Admundson (2007) demonstrated that the IUS12 performed comparatively to the full 27-item IUS in tests of internal consistency, convergent and divergent validity, suggesting that the 15 extra items in the full IUS may well be redundant in measuring IU and may unhelpfully broaden the construct. In the Sexton and Dugas (2009) study described in the previous section, the model generated by their EFA was subsequently tested on a separate sample using CFA. The quality indicators for their EFA were also demonstrated in their execution of the CFA, particularly the large sample size (n = 1221), which substantiates the reliability of the ndings. Interestingly, this CFA also yielded signicant results for the two-factor structure tested, with all the criteria for goodness of t being met.

2.4.2. Summary When results of these CFAs are examined concurrently, they provide strong evidence for the two-factor structure of the IUS, both in its original 27-item form and in the shortened 12-item version. The most promising aspect of these results is that the items that constitute the rst factor in the Carleton, Norton, and Admundson studies (2007) map directly onto Sexton and Dugas (2009) second factor, and those that load onto Carleton et al.'s second factor map directly onto Sexton and Dugas' rst. It is interesting that the two factors are reversed in each study, with one factor explaining most of the variance in Carleton et al.'s studies and the other explaining most in the Sexton and Dugas study. This difference may be due to the number of extra items that load onto the factors in the latter, due to the inclusion of all 27 items across the two factors. We can, therefore, conclude that it is likely to be these two factors that constitute IU, as measured by the IUS. Despite the disagreement regarding the breadth of these factors, we can be condent that the items that load onto them consistently across the CFAs (i.e., the IUS-12

Chi2 df

CFI

SRMR

RMSEA (90% CI)

ECVI (90% CI)

NNFI

Desired values Sexton & Dugas, 2009 Carleton, 2009 (personal communication 2009) Reanalysis of Carleton, Sharpe, & Admundson, 2007 Carleton, Norton, & Admundson, 2007 sample1 Carleton, Norton, & Admundson, 2007 sample2

N 200 Signicant Larger = better 2-factor (27 items) 1221 Yes 2-factor (12 items) 307 2-factor (12 items) 293 2-factor (12 items) 254 2-factor (12 items) 818 Yes Yes

b2 Not reported 2.71 3.85

Close to 0.95 0.06 0.08 0.97 0.94 0.912 0.97 0.96 0.05 0.05 0.05 0.04 0.04 0.07 0.08 (0.060.09) 0.10 (0.080.11) 0.06 (0.040.08) 0.07 (0.060.07)

Smaller = better N 0.09 Not reported 0.63 (0.530.76) 0.87 (0.731.03) 0.58 (0.490.71) 0.35 (0.30.42) 0.96 Not reported Not reported Not reported Not reported

Not reported 1.9 Not reported 1.34

CFI = Comparative Fit Index; SRMR = Standardized Root Means Square Residual; RMSEA = Root Mean Square Error Approximation; ECVI = Expected Cross-Validation Index; NNFI = BentlerBonnet Normed Fit Index.

J. Birrell et al. / Clinical Psychology Review 31 (2011) 11981208 Table 6 Proposed factor structure of the IUS. Item no. Factor 1 7 8 10 11 18 19 21 9 12 15 20 25 Item Unforeseen events upset me greatly It frustrates me not having all the information I need One should always look ahead so as to avoid surprises A small unforeseen event can spoil everything, even with the best of planning I always want to know what the future has in store for me I hate being taken by surprise I should be able to organize everything in advance Uncertainty keeps me from living a full life When it's time to act, uncertainty paralyses me When I am uncertain I can't function very well The smallest doubt stops me from acting I must get away from uncertain situations

1205

Factor 2

items) may guide the denition of these two constituents (see Table 6). 2.4.3. Conclusions Results from the EFA studies suggest that at a minimum there are two latent variables measured by the IUS. Replication of this nding among these studies indicated the need for conrmatory analyses to test this hypothesis. Subsequent CFAs conrmed that a two-factor structure for the IUS demonstrated sufciently adequate t to conclude that the structure shown in Table 5 below is indeed stable and accurate. This conclusion, however, is not sufcient to further our understanding of the nature of IU. Tracey (1990) suggested that factor labels should be considered hypotheses to be subjected to further scrutiny, an approach which may allow some progression toward this aim. Tabachnick and Fidell (2001) state that interpretation of a factor, depends on the meaning of a particular combination of observed variables that correlate highly with each factor (p.583) and interpreting this meaning is as much an art as it is science. The challenge in understanding IU through the detailed analysis of the IUS, therefore, is to apply this art in determining the meaning of the two factors shown above. Table 7 shows the different labels assigned to these factors by the various authors. Rummel (1970) provides some guidelines to consider in the naming of factors, such as whether the label allows others to understand the nature of the factor, whether it stimulates interest and suggests hypotheses, and whether it is memorable and suitable for future use. These guidelines were incorporated into the grid used to evaluate the EFA studies. All of the studies scored only 2 out of 4 points of this section, except that of Berenbaum et al. (2008), who

obtained the full 4 points, suggesting that their labels may provide the most useful indicator of the nature of these factors. According to Berenbaum et al. (2008) the rst factor represents a desire for predictability, suggesting that the individual is focused on having enough information to make the future as certain as possible. This is consistent with Carleton's emphasis on the prospective nature of this factor, and with the idea that uncertainty is a negative experience (Buhr & Dugas, 2002; Freeston et al., 1994; Sexton & Dugas, 2009). These elements are evident within the items that make up this factor, with several references to looking ahead, the future and doing things in advance, as well as the negative connotations of words such as upset, hate, frustrate, and spoil. The desire for predictability label also implicitly suggests some active response to uncertainty in an attempt to make the situation more predictable, which perhaps is the authors' interpretation of the items such as, One should always look ahead so as to avoid surprises and I should be able to organize everything in advance. This idea is described by Meares and Freeston (2008), who suggest that individuals who are intolerant of uncertainty respond to situations in ways that reduce the level of uncertainty. This is often achieved through obtaining sufcient information for the individual to judge the situation as predictable (and, therefore, safe). This may also be consistent with the concept of need for cognitive closure, which has been dened as a desire for a rm answer to a question and an aversion toward ambiguity (Kruglanski & Webster, 1996, p.264), and indeed Berenbaum et al. (2008) demonstrated that desire for predictability correlated signicantly with the predictability, ambiguity and order subscales of the Need for Closure Scale (Webster & Kruglanski, 1994). The second factor is in some ways easier to interpret, partly due to the smaller number of items that map onto it (see Table 4). The majority of the labels assigned to this factor reect the sense of feeling stuck that is reported by many individuals when faced with uncertainty. Terms such as paralysis and inhibitory could be said to represent the meaning of the items accurately, as they refer to being unable to respond in uncertain situations (e.g., The smallest doubt stops me from acting and When it's time to act uncertainty paralyses me). Despite this apparent clarity, there are several ways in which this factor could be understood. For example, Dugas and Robichaud (2007) suggest that there are three possible processes occurring when individuals are faced with uncertainty; 1. Individuals who are intolerant of uncertainty interpret ambiguous situations as more threatening, therefore become anxious and worry about the threat. 2. Individuals who are intolerant of uncertainty require more information before making decisions in ambiguous situations, resulting in prolonged anxiety and worry. 3. Individuals who are intolerant of uncertainty have less condence in their decisions in ambiguous situations so they become anxious and worry about the implications of their decisions. It could, thus, be argued that uncertainty paralysis represents procrastination of action until sufcient information is obtained, or of decision-making due to a lack of condence, or that it is simply an anxious response. The question, therefore, seems to be whether uncertainty paralysis can be understood as an active cognitive avoidance process, or as the freeze part of the ght, ight or freeze response to threat. Of the items that load onto this factor (see Table 6), items 12, 15, and 20 certainly infer that uncertainty impairs the ability to function. Item 12, however, implies a behavioral sense of being unable to move (the freeze response), whereas item 20 strongly suggests the involvement of cognitive processes, such as doubt. Furthermore, item 25 refers to the avoidance of uncertainty altogether by getting away from uncertain situations.

Table 7 Proposed factor labels. Authors Freeston et al., 1994 Buhr & Dugas, 2002 Factor 1 Factor 2 NA Uncertainty leads to the inability to act No label Inhibitory anxiety

Uncertainty is unacceptable and should be avoided Unexpected events are negative and should be avoided Norton, 2005 No label Carleton, Norton, Prospective anxiety & Admundson 2007 Berenbaum Desire for predictability et al., 2008 Sexton & Uncertainty is unfair and Dugas, 2009 spoils everything Birrell et al. (2009) Desire for and active engagement in seeking predictability

Uncertainty paralysis Uncertainty has negative behavioral and self-referent implications Cognitive and behavioral paralysis in the face of uncertainty

1206

J. Birrell et al. / Clinical Psychology Review 31 (2011) 11981208

It is likely that a combination of the two explanations is most acceptable, as both seem equally plausible. We know that individuals who are faced with threat often nd themselves paralyzed and de Bruin et al. (2006) showed that this factor is related to measures of anxiety, such as the BAI (Beck & Steer, 1990) and the STAI-T (Spielberger, Gorsuch, & Lushene, 1970), suggesting that this factor may indeed be highly affect-laden (Sexton & Dugas, 2009). Surveying the items for this factor, however, suggests that uncertainty paralysis does indeed have some avoidant function, which is tting with the other components of the GAD model. For example, the model suggests that clinical worriers are likely to utilize cognitive avoidance strategies such as thought suppression (Dugas & Koerner, 2005). Borkovec et al. (1998) suggest that these strategies, and primarily worry itself, are a way of avoiding distressing mental images and the associated affect, thus preventing the processing of these images and feelings. It could, therefore, be hypothesized that individuals who are intolerant of uncertainty feel paralyzed or unable to function when faced with uncertainty as they are engaged in this form of avoidance. In addition to this, the GAD model (Dugas et al., 1998) includes a component of poor problem orientation which refers to a difculty in applying problem solving skills when in uncertain situations. This too may lead the individual to feel stuck or paralyzed in the uncertainty, as they are unable to solve the problem and move forward. In light of this, it is proposed that the uncertainty paralysis factor represents a sense of feeling unable to act when faced with uncertainty, due to active cognitive avoidance confounded by difculties in problem solving.

3. Discussion Intolerance of uncertainty has emerged as a key construct related to worry, and as a central component to understanding and treating GAD, but relatively little is known about its exact nature. Tracing the denition of the construct through the research reveals that, despite the developments over time, current understanding remains very broad and lacks specicity. Dugas has begun to conceptualize IU as a cognitive vulnerability factor for GAD (Koerner & Dugas, 2008; Sexton & Dugas, 2009), but this still leaves many questions unanswered. One method of developing our understanding of IU is to examine the latent variables measured by the IUS. A systematic search for such studies yielded nine papers, some of which utilized EFA and others used conrmatory analyses. A critical review of the EFA studies conrmed that the highest quality papers suggested that a two-factor structure of the IUS was most likely to demonstrate stability. The CFA studies did, in fact, test this hypothesis, among others, and two factors emerged as consistent across the studies with a set of items that loaded onto them across all the analyses. This item set was, therefore, interpreted in order to further our understanding of IU. The rst factor consisted of items that alluded to uncertainty being a negative experience and a preference for knowing what the future holds. This was labeled desire for predictability by Berenbaum et al. (2008), which is felt to accurately capture the meaning of the items within this factor. It is suggested that this factor represents an active approach to uncertainty in that it motivates the individual to seek sufcient information to increase predictability. The second factor, labeled uncertainty paralysis (Berenbaum et al., 2008), consisted of items that refer to the sense of feeling paralyzed or unable to function when faced with uncertainty. This is likely to reect a physiological symptom of anxiety (i.e., the freeze of ght, ight or freeze) but also to be the result of cognitive processing that occurs when faced with uncertainty, including cognitive avoidance and a maladaptive approach to cognitive problem-solving. It is, therefore, proposed that these two factors represent approach and avoidance strategies or processes that occur in individuals who are intolerant of uncertainty, when they are faced with ambiguous situations.

Findings presented here may have interesting clinical implications. For example, the use of subscale scores may reveal whether an individual responds to uncertainty with excessive approach strategies, or whether they are more likely to avoid. Treatment for the former client may focus more on response prevention, and for the latter a focus on exposure may be more effective. Similarly, an individual who scores higher on the uncertainty paralysis subscale, may benet from more detailed work on overcoming cognitive avoidance and developing skills to engage more effectively with problems in a step by step manner (see Meares & Freeston, 2008). Further CFA studies with large sample sizes are required to replicate the nding of a two-factor structure and to substantiate the stability of this solution. The studies examined within this review were all carried out on analog samples and further analyses on data from clinical samples is required to generalize the ndings and determine the clinical relevance of these factors. Moreover, two studies were omitted from this review as they were written in languages other than English. The analyses within these studies may provide further valuable information regarding the latent structure of different translations of the IUS and cultural differences in IU. Further research is required to rmly establish the construct validity of the factors in the Carleton, Norton, and Admundson (2007) and Sexton and Dugas (2009) studies and this may provide a clearer sense of whether the factors should be broadly dened or narrowed down. It is suggested that the ndings in this review may provide an important step forward in understanding the construct of IU, but also that they may be developed further in order to explore other clinically signicant questions. For example, if the two factors are to be further validated, and IU is seen as a vulnerability factor, then it may be fruitful to investigate whether there are specic features of the histories of individuals presenting with GAD which trigger this vulnerability. Further, it may be possible to explore whether there are features common to the early lives of those who are intolerant of uncertainty, such as particular attachment styles or the occurrence of early attachment traumas, and whether such features may be implicated in the development of a desire for predictability and uncertainty paralysis. Such ndings could be of signicant clinical importance and may be particularly relevant for the research currently exploring the application of the Laval model of GAD to children and adolescents (see Corner et al., 2009).

No. Item 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21. 22. 23. 24. 25. 26. 27. Uncertainty stops me from having a strong opinion. Being uncertain means that a person is disorganized. Uncertainty makes life intolerable. It's unfair having no guarantees in life. My mind can't be relaxed if I don't know what will happen tomorrow. Uncertainty makes me uneasy, anxious or stressed. Unforeseen events upset me greatly. It frustrates me not having all the information I need. Uncertainty keeps me from living a full life. One should always look ahead so as to avoid surprises. A small unforeseen event can spoil everything, even with the best of planning. When it's time to act, uncertainty paralyzes me. Being uncertain means that I am not rst rate. When I am uncertain, I can't go forward. When I am uncertain I can't function very well. Unlike me, others seem to know where they are going with their lives. Uncertainty makes me vulnerable, unhappy or sad. I always want to know what the future has in store for me. I can't stand being taken by surprise. The smallest doubt can stop me from acting. I should be able to organize everything in advance. Being uncertain means that I lack condence. I think it's unfair that other people seem to be sure about their future. Uncertainty stops me from sleeping soundly. I must get away from all uncertain situations. The ambiguities in life stress me. I can't stand being undecided about my future.

J. Birrell et al. / Clinical Psychology Review 31 (2011) 11981208

1207

According to Osborne et al. (2008), factor analysis shows some similarity to qualitative methods, due partly to the subjective interpretation on which it relies. This review forms the rst in this area but, due to the focus on a subjective process may be considered somewhat subjective itself. It is hoped that further such reviews, therefore, may bring us closer to a consensus as to the meaning of the factors underlying IU. Appendix A Intolerance of uncertainty scale items (Freeston et al., 1994). References
American Psychiatric Association (1994). Diagnostic and statistical manual of mental disorders (4th ed.). Washington DC: APA. Beck, A. T., & Steer, R. A. (1990). Beck anxiety inventory manual: Pearson (PsychCorp Ltd). *Berenbaum, H., Bredemeier, K., & Thompson, R. J. (2008). Intolerance of uncertainty: Exploring its dimensionality and associations with need for cognitive closure, psychopathology, and personality. Journal of Anxiety Disorders, 22, 117125. *Birrell, J. A., Cavanagh, K., Dudley, R., Meares, K., Wilkinson, A., & Freeston, M. H. (2009). Exploring the factor structure of the Intolerance of Uncertainty Scale within a British sample. Unpublished Report. Newcastle University. Borkovec, T. D., Ray, W. J., & Stober, J. (1998). Worry: A cognitive phenomenon intimately linked to affective, physiological and interpersonal behavioral processes. Cognitive Therapy and Research, 22(6), 561576. Budner, S. (1962). Intolerance of ambiguity as a personality variable. Journal of Personality, 30, 2950. *Buhr, K., & Dugas, M. J. (2002). The intolerance of uncertainty scale: Psychometric properties of the English version. Behaviour Research and Therapy, 40, 931945. Buhr, K., & Dugas, M. J. (2006). Investigating the construct validity of intolerance of uncertainty and its unique relationship with worry. Journal of Anxiety Disorders, 20, 222236. Butler, G., & Matthews, A. (1987). Anticipatory anxiety and risk perception. Cognitive Therapy and Research, 11, 551565. Butzer, B., & Kuiper, N. A. (2006). Relationships between the frequency of social comparisons and self-concept clarity, intolerance of uncertainty, anxiety and depression. Personality and Individual Differences, 41, 167176. *Carleton, N., Norton, P. J., & Admundson, J. G. (2007). Fearing the unknown: A short version of the intolerance of uncertainty scale. Journal of Anxiety Disorders, 21, 105117. *Carleton, N., Sharpe, D., & Admundson, J. G. (2007). Anxiety sensitivity and intolerance of uncertainty: Requisites of the fundamental fears? Behaviour Research and Therapy, 45, 23072316. Clark, D. A., & Steer, R. A. (1996). Empirical status of the cognitive model of anxiety and depression. In P. M. Salkovskis (Ed.), Frontiers of cognitive therapy (pp. 7596). New York: Guilford Press. Comfrey, A. L., & Lee, H. B. (1992). A rst course in factor analysis. Hillsdale: Lawrence Erlbaum. Corner, J. S., Roy, A. K., Furr, J. M., Gotimer, K., Beidas, R. S., Dugas, M. J., et al. (2009). The intolerance of uncertainty scale for children: A psychometric evaluation. Psychological Assessment, 21(3), 402411. Davey, G. C. L., Hampton, J., Farrell, J., & Davidson, S. (1992). Some characteristics of worrying: Evidence for worrying and anxiety as separate constructs. Personality and Individual Differences, 13, 133147. de Bruin, G. O., Rassin, E., & Muris, P. (2007). The prediction of worry in non-clinical individuals: The role of intolerance of uncertainty, meta-worry and neuroticism. Journal of Psychopathology and Behavioral Assessment, 29, 93100. *de Bruin, G. O., Rassin, E., van der Heiden, C., & Muris, P. (2006). Psychometric properties of a Dutch version of the intolerance of uncertainty scale. Netherlands Journal of Psychology, 62(2), 9197. Dugas, M. J., Freeston, M. H., & Ladouceur, R. (1997). Intolerance of uncertainty and problem orientation in worry. Cognitive Therapy and Research, 21(6), 596606. Dugas, M. J., Gagnon, F., Ladouceur, R., & Freeston, M. H. (1998). Generalized anxiety disorder: A preliminary test of a conceptual model. Behaviour Research and Therapy, 36, 215226. Dugas, M. J., Gosselin, P., & Ladouceur, R. (2001). Intolerance of uncertainty and worry: Investigating specicity in a nonclinical sample. Cognitive Therapy and Research, 2001, 25. Dugas, M. J., Hedayati, M., Karavidas, A., Buhr, K., Francis, K., & Philips, N. A. (2005). Intolerance of uncertainty and information processing: Evidence of biased recall and interpretations. Cognitive Therapy and Research, 29(1), 5770. Dugas, M. J., & Koerner, N. (2005). The cognitive behavioural treatment of generalized anxiety disorder: Current status and future directions. Journal of Cognitive Psychotherapy, 19, 6181. Dugas, M. J., & Ladouceur, R. (2000). Treatment of GAD: Targeting intolerance of uncertainty in two types of worry. Behavior Modication, 24(5), 635657. Dugas, M. J., & Robichaud, M. (2007). Cognitive-behavioral treatment for generalized anxiety disorder. New York: Taylor & Francis Group. Ellis, M. V., Ladany, N., Krengel, M., & Schult, D. (1996). Clinical supervision research from 1981 to 1993: A methodological critique. Journal of Counseling Psychology, 43(1), 3550.

Indicates studies included in critical review.

Field, A. (2005). Discovering statistics using SPSS (2nd ed.). London: SAGE Publications. *Freeston, M. H., Rheaume, J., Letarte, H., Dugas, M. J., & Ladouceur, R. (1994). Why do people worry? Personality and Individual Differences, 17(6), 791802. Frenkel-Brunswick, E. (1948). Intolerance of ambiguity as an emotional and perceptual personality variable. Journal of Personality, 18(108143). Gould, R. A., Safran, S. A., O'Neill Washington, D., & Otto, M. W. (2004). Cognitivebehavioral treatments: A meta-analytic review. In C. L. Turk, & D. S. Mennin (Eds.), Generalized anxiety disorder: Advances in research and practice (pp. 248264). New York: Guilford Press. Gerlach, A. L., Andor, T., & Patzelt, J. (2008). The signicance of intolerance of uncertainty in generalized anxiety disorder: Possible models and development of a German version of the intolerance of uncertainty scale. Zeitschrift fr Klinische Psychologie und Psychotherapie, 37(3), 190199. Grenier, S., Barrette, A. -M., & Ladouceur, R. (2005). Intolerance of uncertainty and intolerance of ambiguity: Similarities and differences. Personality and Individual Differences, 39, 593600. Hatcher, L. (1994). A step-by-step approach to using the SAS system for factor analysis and structural equation modelling. Cary: SAS Institute. Holaway, R. M., Heimberg, R. G., & Coles, M. E. (2006). A comparison of intolerance of uncertainty in analogue obsessivecompulsive disorder and generalized anxiety disorder. Anxiety Disorders, 20, 158174. Horn, J. L. (1965). A rationale and test for the number of factors in factor analysis. Psychometrika, 30, 179185. Hu, L. -T., & Bentler, P. M. (1999). Cutoff criteria for t indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modelling, 6, 155. Koerner, N., & Dugas, M. J. (2006). A cognitive model of generalized anxiety disorder: The role of intolerance of uncertainty. In G. C. L. Davey, & A. Wells (Eds.), Worry and its psychological disorders: Theory, assessment and treatment (pp. 201216). Chichester: John Wiley & Sons Ltd. Koerner, N., & Dugas, M. J. (2008). An investigation of appraisals in individuals vulnerable to excessive worry: The role of intolerance of uncertainty. Cognitive Therapy and Research, 32, 619638. Krohne, H. W. (1993). Vigilance and cognitive avoidance as concepts in coping research. In H. W. Krohne (Ed.), Attention and avoidance (pp. 1950). Toronto: Hogrefe and Huber. Kruglanski, A. W., & Webster, D. M. (1996). Motivated closing of the mind: Seizing and freezing. Psychological Review, 103, 263283. Ladouceur, R. (1993). Intolerance of uncertainty and reasons for worrying in generalized anxiety disorder. Paper presented at the Annual meeting of the Societe Quebeciose pour la Recherche en Psychologie. Ladouceur, R., Blais, F., Freeston, M. H., & Dugas, M. J. (1998). Problem solving and problem orientation in generalized anxiety disorder. Journal of Anxiety Disorders, 12(2), 139152. Ladouceur, R., Dugas, M. J., Freeston, M. H., Rheaume, J., Blais, F., Gagnon, F., et al. (1999). Specicity of generalized anxiety disorder symptoms and processes. Behavior Therapy, 30, 191207. Ladouceur, R., Freeston, M. H., Dugas, M. J., Rheaume, J., Gagnon, F., Thibodeau, N., et al. (1995). Specic association between generalized anxiety disorder and intolerance of uncertainty among anxiety disorder patients. Paper presented at the Annual meeting of the Association for Advancement of Behavior Therapy. Ladouceur, R., Gosselin, P., & Dugas, M. J. (2000). Experimental manipulation of intolerance of uncertainty: A study of a theoretical model of worry. Behaviour Research and Therapy, 38, 933941. Ladouceur, R., Talbot, F., & Dugas, M. J. (1997). Behavioral expressions of intolerance of uncertainty in worry. Behavior Modication, 21(3), 355371. McIntosh, A., Cohen, A., Turnbull, N., Esmonde, L., Dennis, P., Eatock, J., et al. (2004). Clinical guidelines and evidence review for panic disorder and generalised anxiety disorder. Shefeld: University of Shefeld/London, National Collaborating Centre for Primary Care. Meares, K., & Freeston, M. H. (2008). Overcoming worry: A self-help guide using cognitive behavioral techniques. London: Constable & Robinson Ltd. Mennin, D. S. (2004). Emotion regulation therapy for generalized anxiety disorder. Clinical Psychology & Psychotherapy, 11, 1729. Mennin, D. S., Heimburg, R. G., Turk, C. L., & Fresco, D. M. (2005). Preliminary evidence for an emotional dysregulation model of generalized anxiety disorder. Behaviour Research and Therapy, 43, 12811310. Metzger, R. L., Miller, M. L., Cohen, M., Sofka, M., & Borkovec, T. D. (1990). Worry changes decision making: The effect of negative thoughts on cognitive processing. Journal of Clinical Psychology, 46, 7888. *Norton, P. J. (2005). A psychometric analysis of the intolerance of uncertainty scale among four racial groups. Journal of Anxiety Disorders, 19, 699707. Osborne, J. W., Costello, A. B., & Kellow, J. T. (2008). Best practices in exploratory factor analysis. In J. W. Osborne (Ed.), Best practices in quantitative methods (pp. 8699). Thousand Oaks, USA: Sage Publications Inc. Papworth, M. A., & Milne, D. L. (2001). Qualitative systematic review: An example from primary prevention in adult mental health. Journal of Community and Applied Social Psychology, 11, 193210. Peterson, R. A., & Reiss, S. (1992). Anxiety sensitivity index manual (2nd ed.). Worthington: International Diagnostic Systems. Public Health Resource Unit (2008). Critical appraisal skills programme: Appraisal tools. Retrieved 27 March 2008 from. http://www.phru.nhs.uk/Pages/PHD/resources.htm Rassin, E., & Muris, P. (2005). Indecisiveness and the interpretation of ambiguous situations. Personality and Individual Differences, 39, 12851291. Rodriguez, M. G., Leon, R. C., Rovella, A. T., & Herrera, M. D. (2006). Adaptacion espanola de la escala de intolerancia hacia la incertidumbre: procesos cognitivos, ansiedad y depresion. Psicologia y Salud, 16(2), 219233.

1208

J. Birrell et al. / Clinical Psychology Review 31 (2011) 11981208 Tolin, D. F., Abramowitz, J. S., Brigidi, B. D., & Foa, E. B. (2003). Intolerance of uncertainty in obsessivecompulsive disorder. Journal of Anxiety Disorders, 17, 233242. Tracey, L. (1990). Treating factor interpretations as hypotheses. Social Behavior and Personality, 18, 309326. Velicer, W. F. (1976). Determining the number of components from the matrix of partial correlations. Psychometrika, 41, 321327. Webster, D. M., & Kruglanski, A. W. (1994). Individual differences in need for cognitive closure. Journal of Personality and Social Psychology, 67, 10491062. Wells, A. (1997). Cognitive therapy of anxiety disorders. A practice manual and conceptual guide. Chichester: John Wiley & Sons Ltd. Wells, A. (1999). A cognitive model of generalized anxiety disorder. Behavior Modication, 23, 526555.

Rummel, R. J. (1970). Applied factor analysis. Evanston: Northwestern University Press. Seagrave, V. (2009). Overcoming jumping to conclusions. Unpublished Doctoral Thesis. Newcastle University. *Sexton, K. A., & Dugas, M. J. (2009). Dening distinct negative beliefs about uncertainty: Validating the factor structure of the intolerance of uncertainty scale. Psychological Assessment, 21(2), 176186. Spielberger, C. D., Gorsuch, R. L., & Lushene, R. E. (1970). Manual for the state-trait anxiety inventory. Palo Alto: Consulting Psychologists Press. Steketee, G., Frost, R. O., & Cohen, I. (1998). Beliefs in obsessivecompulsive disorder. Journal of Anxiety Disorders, 12, 525537. Tabachnick, B. G., & Fidell, L. S. (2001). Using multivariate statistics (4th ed.). Needham Heights: Allyn & Bacon.