Mental Toughness: Is the Mental Toughness Test Tough Enough? S. Cory Middleton, Herbert W. Marsh, Andrew J.

Martin, Garry E. Richards, Jacqueline Savis, Clark Perry Jr, and Robert Brown Self-concept Enhancement and Learning Facilitation Research Centre University of Western Sydney, Australia

Paper presented at NZARE AARE, Auckland, New Zealand November 2003 MID03782

Mental Toughness: Is the Mental Toughness Test Tough Enough? S. Cory Middleton, Herbert W. Marsh, Andrew J. Martin, Garry E. Richards, Jacqueline Savis, Clark Perry Jr, and Robert Brown Self-concept Enhancement and Learning Facilitation Research Centre University of Western Sydney, Australia This represents the first stage of research in progress into the construct definition and validation of the mental toughness construct. The study evaluated the construct validity of responses to Loehrs (1986) mental toughness test, the Psychological Performance Inventory (PPI), by 263 student-athletes from an elite sports high school. As confirmatory factor analysis (CFA) yielded poor model fit and an improper solution for the a priori model, exploratory factor analysis was carried out using all the original PPI items. Item deletion and exploration of three through to ten alternative factor structures yielded a five-factor model that fitted the data well. Whereas the alternative structure yielded a much better model fit than the original PPI structure, further analyses showed that a variety of key correlates of mental toughness were more strongly correlated with the factors based on the original structure than factors based on the alternative structure. In conclusion, neither the original PPI nor the subset of PPI items in the better-fitting alternative model were sound measures of mental toughness, indicating that a good fit is a necessary but not sufficient condition for construct validation. Good instrumentation must be strong in terms conceptual/theoretical considerations, psychometric properties, and relationships to key correlates hypothesised to be meaningfully related to it. Examining definition and validation of the mental toughness construct is the crucial next stage of this research in progress. What separates athletes who thrive on elite competition from those who buckle under pressure? Why do some athletes succeed in the face of adversity whereas others do not? Why do some athletes experience unproductive negative affect in competition and adverse circumstances while others do not? Why do some athletes bounce back from personal failure whereas others are overwhelmed by it? Many suggest that the answer lies in mental toughness. In Search of Mental Toughness Sport psychologists (researchers and practitioners), coaches, sports commentators, sports fans, and athletes acknowledge the importance of mental toughness in sporting performance (see Goldberg, 1998; Hodge, 1994; Tunney, 1987; Williams, 1988). In early work on the issue, Loehr (1982, 1986) emphasised that athletes and coaches felt that at least fifty percent of success is due to psychological factors that reflect mental toughness. Similarly, Gould, Hodge, Perterson, and Petlichkoff (1987) emphasised that coaches feel that mental toughness is important in achieving success, while Norris (1999) has emphasised the importance of mental toughness in developing champion athletes. Despite widespread agreement on the importance and benefits of mental toughness and calls to identify psychological attributes that create champions, high quality research into mental toughness is limited. Most recently, Jones, Hanton and Connaughton (2002) conducted a qualitative study of elite athletes, aiming to define mental toughness and to determine the essential attributes required to be a mentally tough performer. The definition that emerged from their analysis concluded that:

Mental toughness is having the natural or developed psychological edge that enables you to: 1) Generally, cope better than your opponents with the many demands (competition, training, lifestyle) that sport places on a performer; and, 2) Specifically, be more consistent and better than your opponents in remaining determined, focused, confident, and in control under pressure (p. 209). They also identified twelve attributes as keys to mental toughness. These included attributes such as self-belief, an unshakeable focus, high levels of desire and determination (especially at times of distress), and overall consistency of effort and technique despite life and sport stresses In another qualitative study into mental toughness, Fourie and Potgieter (2001) analysed written responses from 131 expert coaches and 160 elite athletes. Their analysis identified twelve components of mental toughness including: motivation level, coping skills, confidence maintenance, cognitive skill, discipline and goal directedness, competitiveness, possession of prerequisite physical and mental requirements, team unity, preparation skills, psychological hardiness, and ethics. In contrast to Jones et al. (2002), the researchers did not propose a definition but instead suggested that further work was needed to finalise a working definition of mental toughness. A Measure of Mental Toughness Given the absence of an operational definition, there has been difficulty in developing a suitable measure of mental toughness. In professional practice, Loehrs Psychological Performance Inventory (PPI 1986) remains the most influential mental toughness instrument. Loehr published the PPI in his classic book on mental toughness (1986) and claimed that the instrument measured the seven most important psychological factors that reflect mental toughness: self-confidence, negative energy, attention control, visual and imagery control, motivation, positive energy, and attitude control. Although Loehr offered an intuitively and conceptually appealing discussion of the instrument, he presented no psychometric support for its use. Surprisingly, given the ongoing impact of Loehrs research and despite the fact that the PPI continues to be used in practice, there apparently have been no rigorous and formal evaluations of PPI responses. In fact, Murphy and Tammen (1998) point out that it should not be used because it lacked norms, validity, and reliability data and, more generally, a clear rationale in construction/selection of items. This is unfortunate because a rigorously tested mental toughness measure could assist researchers to unravel the critical role that mental toughness plays in sport, and thereby facilitate early talent identification and early intervention strategies. A Construct Validation Approach The focus of this research is on a construct validation approach. There now exists general agreement among sport and exercise psychology researchers for the need to develop sports psychology instruments that are relevant to sport settings (rather than to adopt general instruments from other areas of research) and to evaluate them within a construct validity framework (Gill, Dzewaltowski, & Deeter, 1988; Marsh, 1997; in press; Ostrow, 1996). A construct validation approach to the development of multidimensional instruments is based on theory, followed by item and reliability analysis, exploratory and confirmatory factor analysis, tests of convergent and divergent validity, validation in relation to external criteria, and application in research and practice. The availability of measures in the sport and exercise research that meet these ideals is limited.

Although there has been progress in the last decade (Marsh, in press), there is still a need for: (a) more carefully developed instruments, (b) better articulation of the links between instrument design, theory, and practice, and (c) improved application of methodological and statistical techniques. From a construct validation perspective, theory, measurement, empirical research, and practice are intertwined so that the neglect of one will undermine the others (see Marsh, 1997, for further discussion). Ideally, construct validation is an on-going process in which theory and practice are used to develop a measure, empirical research is used to test the theory and the measure, both the theory and the measure are revised in relation to research, new research is conducted to test these refinements, and theory and research are used to inform practice. Construct validity investigations can be classified as within-network or between-network studies. Within-network studies explore the internal structure of a construct. They begin with a logical analysis of internal consistency of the construct definition, measurement instruments, and generation of predictions. This is followed by studies that typically employ empirical techniques such as factor analysis. Between-network studies attempt to establish a logical, theoretically consistent pattern of relations between measures of a construct and other constructs. This research is often based on correlational procedures (e.g., structural equation models). The present investigation utilises both within-network and between-network approaches to examine the construct validity of PPI responses. Method Participants Respondents were 263 aspiring elite athletes (62% male, 38% female) who attend a specialised sports high school in Sydney, Australia. The school is one of the most prestigious sports high schools in Australia. Each year, student athletes from across the state compete for enrolment in major sports, including basketball, softball, rugby league, soccer, baseball, swimming, track and fields, dance aerobics, cricket and netball. The mean age of respondents was 13.8 years (SD=1.6 years). Respondents were drawn from Year 7 (19%), Year 8 (22%), Year 9 students (22%), Year 10 students (20%), Year 11 (10%), and Year 12 (7%). Materials Psychological Performance Inventory (PPI) The PPI is a 42-item self-report instrument designed to measure factors that reflect mental toughness. All questions in the PPI were answered using a 6-point Likert type scale, ranging from 1 (False) to 6 (True). Six items subsume each of the following seven factors: Self-confidence (e.g., I believe in myself as a player): Positive cognitions, feelings and images about what one can do and achieve. Negative energy (e.g., I get angry and frustrated during competition): The ability to control negative emotions such as fear, anger, frustration and resentment. Attention control (e.g., I can clear interfering emotions quickly and regain focus): The ability to sustain a continuous focus on the task at hand. The ability to tune in to whats important, and tune out to what is not. Visual and imagery control (e.g., Before competition, I picture myself performing perfectly): The ability to think in positive and supportive images and the ability to control the flow of mental images in a positive and constructive direction.

 Motivation level (e.g., I am highly motivated to play my best): The willingness to persevere with training schedules and to endure the pain, discomfort and self-sacrifice associated with forward progress. Positive energy (e.g., I can keep strong positive emotion flowing during competition): The ability to become energized through fun, joy, determination, positivity, and team spirit. Attitude control (e.g., I am a positive thinker during competition): Control over ones habits of thought reflecting the extent to which ones personal attitudes are consistent with those of successful high-level performances. Global Mental Toughness Measure (GMTM) The GMTM is a brief unidimensional scale designed by the authors to obtain a global measure of mental toughness. It contains six items as follows: a. b. c. d. e. f. I am mentally tough when training and in preparation I am mentally tough when I am in competition I am mentally tough when there is a chance that I might fail I am mentally tough when overcoming a setback I am mentally tough when I face particularly demanding challenges Overall I am mentally tough

All questions are answered on a 6-point Likert type scale, ranging from 1 (False) to 6 (True). Key Correlates Physical Self-Description Questionnaire (PSDQ) The PSDQ (Marsh, Richards, Johnson, Roche, & Tremayne, 1994; Marsh, 1997; in press) is a 70-item test that measures 11 components of physical self-concept. Research summarised by Marsh (1997; in press) demonstrated that the PSDQ responses are reliable (Median coefficient alpha = .92 across the 11 scales), are stable over the short term (median r = .83, 3 months) and long term (median r = .69, 14 months), have a well defined factor structure as shown by confirmatory factor analysis, and provide support for convergent and discriminant validity in relation to a variety of external criteria (Marsh, 1996; 1997). For the purposes of the current investigation, only three of the PSDQ scales were administered: Sports Competence, Global Physical, and Global Esteem (20 items in total). Each PSDQ item is a simple declarative statement to which participants respond using a 6-point true-false rating scale. Perceptions of Success Questionnaire (POS) The POS, developed by Roberts and Balague (1991; Roberts, Treasure & Balague, 1998; also see Marsh, 1994; 1997) is one of the major goal orientation instruments used in the sport area. Participants responded to the 14-item questionnaire that measures task and ego motivation orientations that are specific to sport. Research (Roberts & Balague,1991; Roberts, Treasure & Balague, 1998; also see Marsh, 1994; 1997) demonstrates that POS responses are reliable and have a well-defined factor structure as shown by confirmatory factor analysis. Elite Athlete Self-Description Questionnaire (EASDQ) The EASDQ is a 28-item self-report instrument designed by Marsh, Hey, Johnson and Perry (1997; also see Marsh, Hey, Roche, & Perry, 1997; Marsh, 1997; in press) to measure six

components of elite athletes self-concept: Skills, Body, Aerobic, Anaerobic, Mental, and Overall Performance. Participants responded to the items using a 6-point true-false response scale. Research (Marsh, Hey, Johnson & Perry, 1997; Marsh, Hey, Roche, & Perry, 1997; also see Marsh, 1997; in press) demonstrates that EASDQ responses are reliable and have a well-defined factor structure as shown by confirmatory factor analysis. Marsh (in press) reported that EASDQ responses by elite swimmers were strongly correlated with performances in 16 different events and contributed to prediction of subsequent performance in international championships beyond what could be explained in terms of previous performances. Flow Trait Scale (FLOW) The 36-item FLOW instrument is based on Csikszentmihalyis theory of flow as applied to a sport setting (see Jackson & Csikszentmihalyi, 1999). The instrument, developed by Jackson and colleagues (Jackson & Eklund, in press; Jackson & Marsh, 1996; Marsh & Jackson, 1999), measures nine flow experiences in sport: action-awareness merging, clear goals, unambiguous feedback, concentration on task, sense of control, time transformation, and autotelic (intrinsically rewarding) experience. Research (Jackson & Eklund, in press; Jackson & Marsh, 1996; Marsh & Jackson, 1999) demonstrates that the Flow responses are reliable (reliability coefficients varied between .80 and .92) and have a well-defined factor structure as shown by confirmatory factor analysis. Procedure Participants were informed that the researchers were interested in how they engage in their sport with the view to assisting coaches in promoting mental toughness. The researchers worked through the background/demographic questions on an accompanying instrument with the group. Following this, the rating scale was explained, and students then completed a few related example items. The administration of the questionnaire was by a read-aloud method, in which a researcher read out the items to the group of participants and they in turn provided their responses on their answer sheets. This administration procedure was based on the readaloud procedure employed with the Self Description Questionnaire II (Marsh, 1990) that is extensively administered to high school students. The intent of this procedure is to ensure that all students spend a similar amount of time on each item, to minimise any potential problems with reading, and to facilitate classroom management in the administration of the materials. Statistical Analyses Confirmatory factor analysis (CFA), performed with LISREL version 8.5 (Joreskog & Sorbom, 2001), was used to test the psychometric properties of the PPI. In CFA, the researcher posits an a priori structure and tests the ability of a solution based on this structure to fit the data by demonstrating that (a) the solution is well defined, (b) parameter estimates are consistent with theory and a priori predictions, and (c) the 2 and subjective indices of fit are reasonable (Marsh, Balla & McDonald, 1988; McDonald & Marsh, 1990). Maximum likelihood was the method of estimation used for the models. In evaluating goodness of fit of alternative models, we emphasize the root mean square error of approximation (RMSEA). Although the RMSEA is apparently the most widely endorsed criterion of fit, we also present the Tucker-Lewis index (TLI), the relative noncentrality index (RNI), the 2 test statistic, and an evaluation of parameter estimates. For RMSEAs, values of less than .05 and .08 are taken to reflect a close fit and a reasonable fit respectively (see Joreskog & Sorbom, 1993; Marsh, Balla & Hau, 1996; Schumacker &

Lomax, 1996). The TLI and RNI vary along a 0-to-1 continuum in which values greater than .90 and .95 are typically taken to reflect acceptable and excellent fits to the data respectively (McDonald & Marsh, 1990). The RNI contains no penalty for a lack of parsimony so that improved fit due to the introduction of additional parameters may reflect capitalization on chance, whereas the TLI and RMSEA contain penalties for a lack of parsimony. Whereas tests of statistical significance and indices of fit aid in the evaluation of the fit, there is ultimately a degree of subjectivity and professional judgment in the selection of a best model. Results Testing the Factor Structure of the PPI CFA was first conducted on the PPI. The first-order factor structure underlying the seven components of the PPI was not well defined, with the CFA yielding a 2 of 2297.44 (df=798), a TLI of .70, RNI of .73, and RMSEA of .085. The factor loadings, factor correlations, and reliabilities (Cronbachs alphas) are presented in Table 1 illustrating the highest loading on any factor as .73 and the lowest as .22. In addition to the poor model fit and even more problematic the solution was improper as indicated by factor correlations greater than 1.0. Using CFA, we then explored a series of post-hoc models in which the original seven factors were retained, but poor fitting items were eliminated. Whereas deletion of numerous items resulted in substantially better fit, all solutions remained improper in that a number of factor correlations approached or exceeded 1.0. Taken together, the poor model fit and the improper correlations suggest the PPI model as originally formulated was not strong. It was therefore deemed appropriate to explore alternative models, using exploratory factor analysis (principal components analysis) rather than CFA.

Table 1: Original and Alternative Confirmatory Factor Analysis Structures for the PPI
Items SC 1 SC 8 SC 15 SC 22 SC 29 SC 36 NE 2 NE 9 NE 16 NE 23 NE 30 NE 37 AtteC 3 AtteC 10 AtteC 17 AtteC 24 AtteC 31 AtteC 38 VIC 4 VIC 11 VIC 18 VIC 25 VIC 31 VIC 39 ML 5 ML 12 ML 19 ML 26 ML 33 ML 40 PE 6 PE 13 PE 20 PE 27 PE 34 PE 41 AttiC 7 AttiC 14 AttiC 21 AttiC 28 AttiC 35 AttiC 42 Factor 1 Factor 2 Factor 3 Factor 4 Factor 5 Factor 6 Factor 7 SC .33 .65 .62 .65 .73 .70 .60 .30 .65 .25 .31 .61 .58 .22 .59 .50 .43 .56 .47 .56 .61 .58 .69 .57 .65 .72 .56 .71 .47 .64 .64 .56 .39 .72 .53 .62 .72 .50 .59 .65 .59 .54 Factor Correlations .81 .82 .80 .84 .96 1.03 .77 1.00 .50 .61 .76 .80 .63 .20 .54 .44 .68 1.07 .73 .76 .83 .65 .80 .60 .72 -.01 .30 -.06 .65 NE Original PPI Factor Structure AtteC VIC ML PE Factor Loadings AttiC Alternative PPI Factor Structure CP CA IM EC V

.70 .70 .91

.52

.53

.51

.63 .56 .72 .59

.69 .79 .81

.87 .83

.50 .66 .82 .91 .65

.74 .80 .79 .75

1.03 .92 .77

.31 .53

.34

Factor Reliability

The original Psychological Performance Inventory (PPI) factors were: SC=Self-Confidence; NE=Negative Energy; AtteC=Attention Control; VIC=Visual and Imagery Control; ML=Motivation Level; PE=Positive Energy; AttiC=Attitude Control. The factors based on the alternative structure were: CP=Competition Positivity; CA=Competition Anxiety; IM=Internal Motivation; EC=Emotional Control; V=Visualisation. The PPI item numbers correspond to those in the original PPI instrument (Loehr, 1986).

Exploratory Factor Analysis of the PPI Principal components analysis with oblique rotation (as factors were expected to be correlated) was performed on the 42 PPI items. A variety of solutions were explored with a view to generating conceptually compelling factors that explained substantial variance, comprised high target loadings, and yielded few cross-loadings. This was accomplished through a process of deleting items that yielded substantial cross-loadings or did not load substantially on any factors and through examining a range of factor solutions (three- through to ten-factor solutions). A five-factor model based on responses to 16 items was deemed to be the most appealing with highest target loadings, fewest cross loadings, substantial explained variance, and factors that appeared to be conceptually meaningful. When we applied CFA to this five-factor model, the data fitted the model well (2=142.17 df=94, TLI=.94, RNI=.96, RMSEA=.044). Table 1 presents factor loadings derived from the CFA as well as correlations between factors and reliabilities. Factor 1 comprises four items and has been assigned the label competition positivity. Competition positivity describes the athletes ability to maintain positive thinking during competition and draws items from four different PPI factors (Attitude Control, Positive Energy, Motivational Level and Self-Confidence). Factor 2 comprises two items from two different PPI factors (Negative Energy and Attention Control) and has been labelled anxiety control. Anxiety control describes anxiety-related reactions to performance pressures. Factor 3 comprises three items (two from the Motivational Level scale and one from the Positive Energy scale) and has been labelled internal motivation. Internal motivation describes the athletes sense of fulfilment and satisfaction gained from playing or practising in their sport. Factor 4 comprises three items (two from the Negative Energy scale and one from the Attention Control scale) and has been labelled emotional control. Emotional control describes the athletes ability to control his or her emotional response to the pressures of competition. Factor 5 comprises four items (all from the Visual and Imagery Control scale) and has been labelled visualisation. Visualisation describes the athletes ability to use positive visualisation skills in training and competition. Construct Validation of the PPI and the Alternative Model The analysis detailed above suggests a psychometrically sound alternative factor structure that is, a structure with sound within-network properties. Construct validation requires the demonstration of appropriate relationships between the constructs being validated and other known constructs that is, sound between-network properties. Therefore, the next task is to examine the relationship between the PPI and our alternative structure and to also explore patterns of relationships between both models and other key correlates described above. Relationships between the PPI and the Alternative Factor Structure. Subscale scores for the seven PPI factors and the five factors in the alternative model were computed by generating the mean of component items. Pearson product moment correlations between the PPI factors and the five alternative factors were carried out and are presented in Table 2. Competition positivity correlated substantially with the PPIs self-confidence (.78), positive energy (.78), and attitude control (.81) factors despite having only one of the PPIs original scale items appearing in each of these alternative model factors. Internal motivation correlated substantially (.85) with the PPIs motivation level having two-thirds of its items from the original PPI scale. Emotional control shared substantial variance with the PPIs negative energy (.77), as did visualization with the PPIs visualization and imagery control at .94 not unexpectedly as the alternative factor comprised four of the PPIs visualization items. Taken together, these data suggest that constructs in the alternative model that are

conceptually akin to those in the PPI share greater variance than they do with constructs that are not so conceptually congruent. Indeed, anxiety the factor that departs most from Loehrs conceptualisation does not share substantial variance with any PPI factors (it shares 25% variance with the PPIs attention control). Table 2: Correlation matrix between PPI factors and the alternative five factors
Alternative Structure Competition PPI Factors Self-Confidence Negative Energy Attention Control Visualization & Imagery Control Motivation Level Positive Energy Attitude Control Positivity
.78 .41 .51 .60 .66 .78 .81

Competition Anxiety
.16 .50 .52 -.01 .04 .14 .16

Internal Motivation
.50 .28 .34 .47 .85 .68 .61

Emotional Control
.43 .77 .60 .17 .31 .40 .41

Visualization
.56 .23 .31 .94 .54 .55 .57

Relating Both Models to Key Correlates A battery of key correlates was also included in the study. These related to physical and athletic self-concept, goal orientation, flow, and a global mental toughness measure. These measures were selected on the basis of their conceptual relatedness to mental toughness either as correlates or outcomes of individuals mental toughness. Hence, they are deemed to be feasible constructs with which to validate our alternative structure and how this compares with the performance of the original PPI factors in relation to these measures. In addition to these key correlates, a mean scale score for all PPI items was computed. Analysis of the correlations was multifaceted. First, we examined the correlations between the original PPI factors with the key correlates and compared them to the correlations between the alternative factors and the key correlates. Second, we examined the correlations between the total PPI scale score and each of the key correlates. Third, we examined the correlations between the total PPI scale score and each of the key correlates relative to correlations between the specific (original PPI and alternative structure) subscale scores and the key correlates. Finally, we examined the correlations between our global mental toughness measure (GMTM) and each of the key correlates and compared these with the correlations between the total PPI scale score and each of the key correlates. Table 3 presents findings.

Table 3: Correlations between factors in both models and key correlates

Alternative Structure
CP CA IM EC V

PPI Factors SC
.74 .29 .25 .49 .49 .56 .61 .43 .47 .52 .64 .66 .78 .39 .62 .46 .60 .66 .48 .12 .63 .34 -.07 .30 .39 .50 .39 -.01 .37 .30 .43 .18 .28 .31 .44 .51 .42 .50 .55 .41 .31 .52 .17 .21 .35 .38 .47 .59 .70 .36 .67 .38 .58 .68 .40 .16 .66 .34 .41 .42 .60 .35 .47 .42 .46 .55 .61 .75 .35 .64 .40 .60 .72 .46 .12 .69 .28 .26 .38 .49 .42 .26 .27 .36 .49 .48 .49 .44 .62 .62 .71 .33 .65 .43 .62 .73 .49 .13 .64 .26 .21 .36 .40 .39 .40 .23 .29 .42 .49 .47 .47 .38 .42 .33 .41 .50 .44 .41 .50 .42 .43 .38 .62 .58 .67 .35 .55 .37 .62 .63 .47 .12 .61 .30 .32 .35 .43 .46 .44 .41 .28 .30 .36 .42 .47 .41 .45 -.03 .00 .29 .19 .12 .15 .14 .20 .36 .21 .32 .37 .38 .27 .38 .17 .48 .49 .54 .53 .44 .50 .49 .63 .65 .77 .38 .68 .45 .64 .75 .52 .13 .68 .40 .52 .55 .58 .68 .70 1.00 .74

Scale

Factor

NE

AtteC

VIC

ML

PE

AttiC

GMTM

PPI

.88 2 .33 .18 .41 .43 .45 .47 .37 .43 .41 .66 .62 .70 .32 .59 .43 .61 .65 .42 .17 .59 .02 .67 .29 .48 -.14 .15 -.04 .27 .08 .31 .23 .38 .10 .58 .29 .52 .04 .43 .22 .45 .02 .31 .12 .38 .04 .55 .23 .47 .08 .27 .13 .32 .07 .57 .27 .54 .01 .42 .25 .38 .19 .34 .23 .36 -.05 .39 .19 .35 .01 .37 .19 .36 -.03 .31 .16 .33 .01 .37 .14 .37 .09 .30 .34 .27 .02 .33 .23 .32 .05 .33 .20 .36 -.03 .14 -.11 .25 .07 .22 .21 .23 .12 .47 .29 .53

.7

GMTM

GMTM

POS

Mastery

.79

Competitiveness

.85

PSDQ

Sports Competence

.83

Global Physical

.94

Global Esteem

.80

EASDQ

Skill

.86

Body

.86

Aerobic

.90

Anaerobic

.87

Mental

.85

Overall Performance

.86

Flow

Challenge-skill

.83

Action-aware

.80

Clear goals

.77

Clear feedback

.84

Concentration

.71

Control

.84

Loss self-conscious

.79

Transformed time

.72

Autotelic Experience

.86

CP=Competition Positivity; CA=Competition Anxiety; IM=Internal Motivation; EC=Emotional Control; V=Visualisation

SC=Self-Confidence; NE=Negative Energy; AtteC=Attention Control; VIC=Visual and Imagery Control; ML=Motivation Level; PE=Positive Energy; AttiC=Attitude

Control

GMTM=Global Mental Toughness Measure; POS=Perceptions of Success; PSDQ=Physical Self Description Questionnaire; EASDQ=Elite Athlete Self Description

Questionnaire; PPI=Psychological Performance Inventory (mean score of all PPI items). All correlations greater than .11 are statistically significant (p < .05, two-tailed).

Our first observation was that in general the PPI factors were more strongly correlated with the key correlates than our alternative factors are with these same measures. Second, in all but two cases, the PPI total score correlates more strongly with the key correlates than do the subscales based on the alternative structure. Third, in all cases the PPI total score correlates more strongly with the key correlates than does the global mental toughness measure with the same correlates (PPI mean r=.52; GMTM mean r=.45). In terms of validation, then, the PPI performs better than the alternative structure and the global measure of mental toughness. Discussion The focus of this investigation was to analyse the psychometric (within-network) properties of the Psychological Performance Inventory (PPI). The PPI was developed to reflect the factors inherent in mental toughness. The factors have good face validity and are widely considered to be conceptually compelling. The PPI factor structure was examined using CFA, but the a priori model resulted in a poor fit to the data and an improper solution. Indeed, the associated correlation matrix for the instrument revealed numerous factor correlations approaching or exceeding unit value. From a within-network perspective, support for the PPI instrument is problematic. Exploratory factor analysis was then carried out and through an iterative process of item deletion and re-specification of factors, a five-factor solution was derived that explained most variance, demonstrated fewer cross-loadings and higher target loadings, and which was conceptually feasible. The fit of this alternative structure was good and so from a withinnetwork perspective, this model was considered to be superior to the PPI. However, when both models were related to a battery of key correlates, the PPI factors were consistently more strongly correlated with these measures and with the global mental toughness measure than were the factors from the alternative structure. From a betweennetwork perspective, then, the PPI is superior to the alternative structure and a unidimensional global measure of mental toughness. However, it is important to note when interpreting these correlations that reliabilities for the original PPI scores were higher than those in the alternative structure because PPI factors were based on more items. Mental Toughness Measurement What these data underscore is the importance of developing research measures that are strong in terms of conceptual (or theoretical) bases, within-network properties, and between network properties. This is not only important for the researcher collecting large-scale data but also the practitioner who wants to be confident that data collected on an individual athlete are meaningful and related to other processes or outcomes relevant to their given sport. Neither the PPI, nor the alternative structure qualified on all three counts. On this basis, we concluded that in the field of mental toughness there currently is not a sound measure of mental toughness. Therefore, it would be useful to identify the strengths of the measures considered here and to explore possible directions for future instrumentation. A particular strength of the PPI is its conceptual basis. The factors identified by Loehr (1986) have face validity, are intuitively appealing, and have support from recent qualitative research into mental toughness (Fourie & Potgieter, 2001; Jones et al., 2002). Five constructs in particular seem to consistently emerge in psychological skills research that incorporates mental toughness: self-confidence, control, motivation, imagery/mental practice, and concentration/attention (Murphy & Tammen 1998). All five are represented in the PPI.

Indeed, self-confidence emerged as the factor most strongly related to the key correlates. Hence, the problem with the PPI is not its conceptual basis or the factors it seeks to quantify, but rather, the problem seems to be the items that are used to assess the factors. A particular strength of the alternative structure is that it is composed of a set of factors that provided a good fit to the data. Through item deletion and re-assignment we were able to derive factors that were internally consistent and which yielded relatively fewer crossloadings than those in the PPI. However, it is our view that the alternative structure, although sound from a within-network perspective, may not be as conceptually appealing as the PPI and is certainly not as strong from a between-network perspective. One concern with the alternative structure is that it does not include a measure of self-confidence and nor does it comprise an intuitively appealing measure of control, two factors that are routinely identified in the mental toughness literature (Fourie & Potgieter, 2001; Jones et al., 2002; Loehr, 1986). Another concern is that it comprises factors that in many cases are not markedly related with key correlates. Moreover, it could be the case that stronger psychometric properties of the alternative structure might at least partly be due to capitalisation on chance through testing the same data repeatedly to generate a better fit. Drawing on the strengths identified through our analysis of each of these structures, we are able to provide some guidance for the development of future mental toughness instruments. There needs to be a strong conceptual/theoretical rationale for the measure, much along the lines of the PPI. Recent qualitative work (Fourie & Potgieter, 2001; Jones et al., 2002) and the pioneering work by Loehr (1986) provide a good basis for such conceptualizing. Also along the lines of the PPI, there needs to be adequate relatedness between the component factors and key correlates. In the first instance, this will emerge directly from the conceptual/theoretical validity of the instrument. However, it will also directly emanate from the item development. We suggest that it is at the item level, where the PPI falls short, and that this is the basis of the poor goodness of fit and improper solutions based on responses to the original PPI. Indeed, a better fitting structure based on an alternative structure was derived in this study. Through refinement of the item mix, we identified a five-factor model that yielded a good fit to the data. Hence, we suggest that development of items not only needs to be undertaken with a view to conceptual/theoretical and between-network validity but also with a view to sound factor structure (within-network validity). Limitations and Future Directions When interpreting these findings, it is important to recognise some limitations associated with the research and which may also provide directions for future research. The data were collected at only one school. Although this was an elite sports high school representing a very relevant population, it is important to extend data collection to other athletes. Also, given the size of the sample, the data were pooled and so invariance in findings as a function of gender, selected sport (particularly team compared with individual sports), and age (particularly the early adolescents compared with later adolescents) were not explored. This is important for future work. The key correlates were all self-report measures and therefore, broader measures might be included in future research derived through reports by significant others (e.g., coaches), performance outcomes, and observational techniques. A limitation of the alternative structure is that it required deletion of many PPI items and it is considered important to develop a larger (and high quality) item pool with which to test mental toughness. All the data collected were cross-sectional and so validation was based on responses from a single point in time. Future research should examine the predictive validity

of mental toughness by examining related processes and outcomes over time. Finally, the present data were based entirely on quantitative methods. There are numerous advantages to multi-method research encompassing both quantitative and qualitative approaches (Brewer & Hunter, 1989) and particularly in an area in which there is a need for strong conceptual and theoretical foundations, the use of both these complementary methods is recommended. Both qualitative and quantitative approaches will be used in the next stage of this research in progress that aims to provide definition and validation of the mental toughness construct. Conclusion The present study provided a number of insights into the status of the PPI, an interesting conundrum regarding the interface between conceptualisation and instrumentation, and guidance for future instrument development. Although the fit of the PPI was poor, its conceptualisation and validity were sound. Although an alternative model fit the data well, support for its conceptual underpinning and validity were not as strong. Taken together, these data suggest that there currently exists no comprehensively sound measure of mental toughness and that vital further work is required to develop a multifaceted mental toughness measure that is strong on conceptual, within-network, and between-network grounds. References Brewer, J., & Hunter, A. (1989). Multimethod research: A synthesis of styles. Newbury Park, CA: Sage. Fourie, S., & Potgieter, J. R. (2001). The nature of mental toughness in sport. South African Journal for Research in Sport, Physical Education and Recreation, 23, 63-72. Gage, N. (1989). The paradigm wars and their aftermath. Educational Researcher, 18, 4-10. Gill, D. L., Dzewaltowski, D. A., & Deeter, T. E. (1988). The relationship of competitiveness and achievement orientation to participation in sport and nonsport activities. Journal of Sport and Exercise Psychology, 10, 139-150. Goldberg, A. S. (1998). Sports slump busting: 10 steps to mental toughness and peak performance. Champaign, IL: Human Kinetics. Gould, D., Hodge, K., Peterson, K., & Petlichkoff, L. (1987). Psychological foundations of coaching: Similarities and differences among intercollegiate wrestling coaches. Sport Psychologist, 1, 293-308. Hodge, K. (1994). Mental toughness in sport: Lessons for life. The pursuit of personal excellence. Journal of Physical Education New Zealand, 27, 12-16. Jackson, S.A., & Csikszentmihalyi, M.C. (1999). Flow in Sports: The Keys to Optimal Experiences and Performances. Champaign, IL: Human Kinetics. Jackson, S.A., & Eklund, R.C. (in press). Assessing flow in physical activity: The Flow State Scale-2 and Dispositional Flow Scale-2. Journal of Sport and Exercise Psychology.

Jackson, S. A., & Marsh, H. W. (1996). Development and validation of a scale to measure optimal experience: The Flow State Scale. Journal of Sport and Exercise Psychology, 18, 1735. Jones, G., Hanton, S., & Connaughton, D. (2002). What is this thing called mental toughness? An investigation of elite sport performers. Journal of Applied Sport Psychology, 14, 205-218. Joreskog, K. G., & Sorbom, D. (1993). LISREL 8: Structural equation modeling with the SIMPLIS command language. Chicago: Scientific Software International. Joreskog, K., & Sorbom, D. (2001). LISREL 8.5. Scientific Software International. Loehr, J.E. (1982). Athletic excellence: Mental toughness training for sports. Forum Publishing Company. Loehr, J.E. (1986). Mental toughness training for sports: Achieving athletic excellence. Lexington, MA: Stephen Greene Press. Marsh, H. W. (1990). Self Description Questionnaire (SDQ) II: A theoretical and empirical basis for the measurement of multiple dimensions of adolescent self-concept: An interim test manual and a research monograph. San Antonio, TX: The Psychological Corporation (Republished in 1992, Publication Unit, SELF Research Centre, University of Western Sydney). Marsh, H. W. (1994). Sport motivation orientations: Beware of the jingle-jangle fallacies. Journal of Sport and Exercise Psychology, 16, 365-380. Marsh, H. W. (1996). Construct validity of Physical Self-Description Questionnaire responses: Relations to external criteria. Journal of Sport & Exercise Psychology, 18(2), 111131. Marsh, H. W. (1997). The measurement of physical self-concept: A construct validation approach. In K. Fox (Ed.), The physical self: From motivation to well-being (pp. 27-58). Champaign, IL: Human Kinetics. Marsh, H. W. (in press). A multidimensional physical self-concept: A construct validity approach to theory, measurement, and research. Psychology, The Journal of the Hellenic Psychological Society. Marsh, H. W., Balla, J. R., & Hau, K. T. (1996). An evaluation of incremental fit indices: A clarification of mathematical and empirical processes. In G. A. Marcoulides & R. E. Schumacker (Ed.), Advanced structural equation modeling techniques (pp. 315-353). Hillsdale, NJ: Erlbaum. Marsh, H. W., Balla, J. R., & McDonald, R. P. (1988). Goodness-of-fit indices in confirmatory factor analysis: The effect of sample size. Psychological Bulletin, 102, 391-410.

Marsh, H.W., Hey, J., Johnson, S., & Perry, C. (1997). Elite Athlete Self Description Questionnaire: Hierarchical confirmatory factor analysis of responses by two distinct groups of elite athletes, International Journal of Sport Psychology, 28, 237-258. Marsh, H. W., Hey, J, Roche, L. A., & Perry, C. (1997). The structure of physical selfconcept: Elite athletes and physical education students. Journal of Educational Psychology, 89, 369-380. Marsh, H.W., & Jackson, S.A. (1999). Flow Experience in sport: Construct validation of multidimensional, hierarchical state and trait responses. Structural Equation Modelling, 6, 343-371. Marsh, H.W., Richards, G.E., Johnson, S., Roche, L., & Tremayne, P. (1994). Physical SelfDescription Questionnaire: Psychometric properties and a multitrait-multimethod analysis of relations to existing instruments. Sport and Exercise Psychology, 16, 270-305. McDonald, R. P., & Marsh, H. W. (1990). Choosing a multivariate model: Noncentrality and goodness-of-fit. Psychological Bulletin, 107, 247-255. Murphy, S., & Tammen, V. (1998). In search of psychological skills. In J. Duda (Ed.), Advances in sport and exercise psychology measurement. (pp. 195-209). Morgantown, WV: Fitness Information Technology. Norris, E. K. (1999). Epistemologies of champions: A discoursive analysis of champions retrospective attributions; Looking back and looking within. Michigan: Michigan University Microfilms International. Ostrow, A. C. (1996). Directory of psychological tests in the sport and exercise sciences (2nd Ed). Morgantown, WV: Fitness Information Technology. Roberts, G. C., & Balague, G. (1991). The development and validation of the Perceptions of Success Questionnaire. Paper presented at the FEPSAC Congress, Cologne, Germany. Roberts, G. C., Treasure, D.C. & Balague, G. (1998). Achievement goals in sport: The development and validation of the Perception of Success Questionnaire. Journal of Sports Sciences, 16, 337-347. Schumacker, R. E., & Lomax, R. G. (1996). A beginner's guide to structural equation modeling. Mahwah, NJ: Erlbaum. Tunney, J. (1987). Thoughts on the line. Mental toughness: Biceps for the mind. Soccer Journal, 32, 49-50. Williams, R. M. (1988). The U.S. open character test: Good strokes help. But the most individualistic of sports is ultimately a mental game. Psychology Today, 22, 60-62.

The authors would like to thank Westfield Sports High School for its participation in this research and both the Australian Institute of Sport and the Australian Research Council for their support of this research. Thanks are also extended to James Loehr for his pioneering work in mental toughness that provided the inspiration for the present investigation. Requests for further information about this investigation can be made to Professor Herb Marsh, SELF Research Centre, University of Western Sydney, Penrith South DC, NSW 1797, AUSTRALIA. E-Mail: h.marsh@uws.edu.au. Cory Middleton and Clark Perry are located at the Australian Institute of Sport and Robert Brown is located at Westfield Sports High School.