Brain Injury, November 2008; 22(12): 905918

REVIEW

Assessment of neurobehavioural disability: A review of

existing measures and recommendations for a comprehensive
assessment tool

R. LL. WOOD1, N. ALDERMAN1,2, & C. WILLIAMS1

Swansea University, Singleton Park, Swansea, UK and 2Kemsley, National Centre for Brain Injury Rehabilitation, St
Andrews Healthcare, Northampton, UK
Brain Inj Downloaded from informahealthcare.com by Ceadac on 07/14/10
For personal use only.

(Received 2 April 2008; revised 27 June 2008; accepted 19 September 2008)

Abstract
Primary objective: Neurobehavioural disability (NBD) has a major impact on long-term psychosocial outcome: however,
measures used to identify characteristics of NBD have not been fully evaluated. This review therefore discusses issues
surrounding the concept and assessment of NBD following acquired brain injury (ABI) by examining the psychometric
properties and other qualities of some of the most well known instruments used to assess NBD. It is the authors intention to
use this review to encourage researchers to develop new NBD measurement tools.
Main outcomes: The review highlights a number of concerns with existing measures, including; the absence of a clear
theoretical framework, a failure to distinguish impairment, disability and handicap when measuring characteristics of NBD
and issues surrounding reliability and validity.
Conclusions: The authors propose, promote and encourage the development of further measures with improved reliability
and validity to help ensure that the unique characteristics of NBD are captured more effectively.
Keywords: Neurobehavioral, disability, handicap, reliability, validity, assessment,

Introduction
There are a variety of non-cognitive consequences of
acquired brain injury (ABI) which, through their
interaction with cognitive sequelae, can have
a devastating impact on social function. For example,
Oddy et al. [1] described how blunted emotions,
combined with inflexible and concrete thinking, can
result in difficulty understanding complex behaviour,
especially attempts at humour. These deficits in social
perception can act as a barrier to recognizing the
emotional states of others [24]. Patients may lose the
ability to interpret and express feelings in social
situations. As a result their behaviour can appear
inappropriate, self-focused and lacking in empathy,

contributing to problems of social cognition [57]

and difficulty maintaining social relationships.
Starkstein and Robinson [8] and Stewart and
Tannock [9] focused on disinhibited behaviour,
whilst McKinlay et al. [10] and Prigatano [11]
addressed problems associated with irritability,
which can affect up to 71% of people after ABI
[10]. Mateer [12] and Brewer et al. [13] comment on
the associated problem of impulsivity which, when
combined with irritability, can result in impulsive
aggression [1416]. Eames and Wood [17] drew
attention to episodic disorders of aggressive behaviour which have different characteristics to impulsive
aggression. The lack of inhibitory control associated
with these behaviours can actively interfere with

Correspondence: Professor Rodger Ll. Wood, Department of Psychology, School of Human Sciences, Swansea University, Singleton Park, Swansea SA2 8PP,
UK. Tel: (44) 1792 295778. Fax: (44) 1792 295679. E-mail: r.l.wood@swansea.ac.uk
ISSN 02699052 print/ISSN 1362301X online 2008 Informa UK Ltd.
DOI: 10.1080/02699050802491271

Brain Inj Downloaded from informahealthcare.com by Ceadac on 07/14/10

For personal use only.

906

R. Ll. Wood et al.

rehabilitation efforts and the general social acceptability of patients; consequently, these have been
conceived as constituting positive disorders of
behaviour [18].
Consistent with this conceptualization, Prigatano
[19] has drawn attention to negative disorders,
such as the psychosocial problems associated with
a lack of motivation. Such patients are often
viewed as lazy or depressed because they can
appear tired and disinterested in their environment. The lack of awareness exhibited by many
individuals, coupled with other forms of cognitive
impairment, mean that they may become confused
as to why people get upset with them, then
withdraw and experience a breakdown in personal
relationships, becoming more isolated [20].
Weddell et al. [21] also reported that head-injured
people spend more time at home, have fewer
friends and make or receive fewer visits than prior
to injury. In addition, they may experience low
self-esteem and feel more dependent on others.
Changes in cognition, behaviour and personality
therefore contribute to social handicap, because
many individuals become increasingly dependent
on family for social support [22].
One approach to understanding the complex
nature of behaviour change after ABI has been to
conceptualize such changes as neurobehavioural
disability (NBD) [2224]. This term has gained
considerable acceptance over the last two decades
and is now used to describe the long-term constellation of problems associated with ABI. Kreutzer et al.
[25] directly attributed poor outcome to the
presence of NBD, construed as adverse changes
in physical well-being, personality, mood, initiative,
social skills and cognitive abilities. Wood [22]
elaborated on the composition of NBD and argued
that it comprises elements of executive and attentional dysfunction, poor insight, awareness and
social judgement, labile mood and problems with
impulse control. Furthermore, when these interact
with specific cognitive problems (such as memory
impairment) and are further influenced by a persons
pre-morbid personality characteristics, they result in
social handicap that undermines the capacity for
independent social behaviour.
Attempts to categorize the diverse range of ABI
characteristics has culminated in recognition of the
need to develop measures of altered emotion, mood,
personality and behaviour [26, 27]. It has also been
advocated that difficulties with executive dysfunction should be included alongside assessment
of behavioural and emotional characteristics
[28]. However, it could be argued that any attempt
to categorize the complex pattern of changes
associated with ABI into clearly defined classes is
too simplistic, because of the complex interactions

between damaged neural systems and environmental

factors responsible for alterations in behaviour.
Consequently, it makes little sense to label them in
ways that do not reflect their multifarious origins.
However, compartmenting neurobehavioural legacies of ABI into broad categories is also problematic
because the categories lack specificity to
assist rehabilitation therapists in targeting treatment
goals.
Perhaps, therefore, a system of classification that
groups NBD characteristics according to ways in
which they affect a persons social functioning, in
terms of their ability to interact purposely with their
environment, has more to offer. For example,
Nelson et al. [29] identified changes associated
with ABI from the literature and then categorized
them in terms of their impact on emotional function.
They conceptualized five such factors: indifference
(denial or reduced awareness); mania (impulsivity,
irritability and euphoria); depression (including
apathy and withdrawal); inappropriateness (unusual
or bizarre behaviour); and pragnosia (impairment in
the pragmatics of communicative style). A system
like this works because it recognizes that behavioural
symptoms classed under a particular heading are
more than the product of purely psychological
factors. Instead, these behaviours may be symptomatic of interactions between few or many underlying variables, including the environment.
Furthermore, it does not preclude one variable
from influencing more than one category of difficulties. For example, attentional impairment may
contribute towards both indifference and mania.
The Nelson et al. [29] approach also recognizes that
the reason why characteristics of ABI fall within each
category can vary between people. This way of
conceptualizing ABI outcome, by seeking common
overlap between various symptoms, is certainly far
more useful to clinicians than that offered by broad
categories of function, such as cognitive vs. noncognitive. However, the Nelson et al. [29] classification system does not claim to capture the complete
spectrum of ABI changes that have been identified.
Whilst damage to neural systems can manifest itself
in various forms of abnormal behaviour, the range of
these behaviours is extensive and subject to further
modification by pre-morbid personality, as well as by
the social environment. Some NBD characteristics
are rarely seen in clinical practice (for example,
reduplicative misidentification syndromes, such as
Capgras Syndrome) whilst others are more common.
For example, Tateno et al. [30] found that over 33%
of their sample had engaged in significant aggressive
behaviour within the first 6 months after brain
injury. Similarly, Baguley et al. [31] demonstrated
that 25% of a sample of patients followed up
6 months, 2 and 5 years after injury met the

Assessment of neurobehavioural disability

studys criteria for significant aggressive behaviour;
consequently, these authors concluded aggression
was a frequent and long-term legacy of traumatic
brain injury (TBI). Very frequent and severe aggression also appears to be characteristic of many
patients referred to specialized neurobehavioural
rehabilitation services because of their challenging
behaviour [32, 33]. The large number of ABI
characteristics and the various ways they express
themselves is further confounded by how behavioural manifestations of NBD ebb and flow within
people over time.

Brain Inj Downloaded from informahealthcare.com by Ceadac on 07/14/10

For personal use only.

Measurement of neurobehavioural disability

Initial attempts to capture characteristics of NBD
relied on importing measurement instruments from
the psychiatric literature and applying them to
people with acquired neurological conditions.
However, these measures were neither designed
with the symptoms of NBD in mind, nor standardized for people with ABI [26]. Subsequent attempts
have been made to modify existing measures in an
attempt to increase their validity, including the
Minnesota Multiphasic Personality Inventory-2
(MMPI-2) [34, 35]. Perhaps the best known
adaptation is the Neurobehavioral Rating Scale
(NRS) [36], which is a modified version of the
Brief Psychiatric Rating Scale. The NRS incorporated modifications that aimed to increase content
validity regarding behaviours that are characteristic
of ABI. It consists of 27 items that are rated on a
7-point scale through a semi-structured interview
and a series of brief cognitive tests. However,
criticism about the validity and reliability of modified
measures such as the NRS have been expressed,
mainly regarding reduced accuracy of information
elicited from ABI patients at interview because of
reduced self-awareness and memory difficulties.
Other threats to integrity arise from the context
in which the interview takes place and the lack of
concordance between test performance and behaviour/function in the real world [29].
In response to the perceived shortcomings when
trying to adapt existing tools and in an effort to
adequately capture the characteristics of NBD,
a number of instruments have recently been developed specifically to measure behavioural and other
non-cognitive characteristics of ABI. Threats to
validity caused by using the person with ABI as the
only source of information are avoided by eliciting
knowledge from an informant who knows them well,
usually a relative. Other instruments also seek to
quantify the extent of change attributable to ABI by
obtaining and contrasting data about how the person
functioned before and after injury. However, in spite

907

of these efforts, Kolitz et al. [28] have argued that

there is still a need to develop a valid and
comprehensive instrument for the measurement of
NBD. In order to examine this claim and consider
what, if any, implications it has for the subsequent
development of NBD measures, this study proposes
to briefly review the psychometric and other qualities
of some of the most well known and widely used
instruments. Measures that have been specifically
developed to quantify behaviour change associated
with damage to frontal lobe structures will not be
considered as the presence of NBD is not confined
to injury to these anatomical areas [37]. We do not
claimed that the eight instruments considered here
comprise an exhaustive compendium of all NBD
measures. This study has focused on them because
their widespread use, both in clinical work and in
research on ABI outcome, render them worthy of
study to determine if they are fit for purpose or if
there is still a need to develop new tools in this area.
A brief description of each of these measures is
followed by a critical analysis of their construction
and statistical properties.
The Neurobehavioural Rating Scale (NRS)
The NRS, an extension of the Brief Psychiatric
Rating Scale (BPRS) [38] was developed by Levin
et al. [36] to quantify clinical observations of
behaviour displayed by individuals following TBI.
In the context of a semi-structured interview
a clinician rates the following 27-items: inattention,
somatic concern, disorientation, anxiety, expressive
deficit, emotional withdrawal, conceptual disorganization, disinhibition, guilt feelings, memory deficit,
agitation, inaccurate insight, depressive mood, hostility, decreased initiative, suspiciousness, fatigability,
hallucinatory behaviour, motor retardation, unusual
thought concern, blunted affect, excitement, poor
planning, lability of mood, tension, comprehensive
deficit and speech articulation deficit.
Levin et al. [36] administered the NRS to 101
patients who were heterogeneous with respect to
severity, chronicity and type of brain injury at
various stages of recovery. A principal component
analysis revealed four factors which were differentially related to injury severity and the presence of
a frontal lobe lesion. Factor I (cognition/energy)
consisted of items evaluating the coherence of
cognition and efficiency of memory, behavioural
slowing and emotional withdrawal. Factor II
(Metacognition) reflected inaccurate self-appraisal,
unrealistic planning and disinhibition. Unlike
Factors I and II, Factor III (somatic concern/
anxiety) included physical complaints, anxiety,
depression and irritability. Factor IV (language)

908

R. Ll. Wood et al.

consisted of the scales for rating expressive and

receptive language deficit.

Brain Inj Downloaded from informahealthcare.com by Ceadac on 07/14/10

For personal use only.

The Neurobehavioural Rating ScaleRevised

(NRS-R)
To enhance reliability and content validity of the
NRS, Levin et al. [39] developed the NRS-Revised
(NRS-R). Two new items were added (difficulties
in mental flexibility and irritability), whilst another
two were merged into a single item (tension and
anxiety became anxiety), whilst others were teased
into separate items (inattention divided into
reduced alertness and attention), resulting in
29-items altogether. Furthermore, Vanier et al.
[40] noted that the 7-point rating scale used in the
NRS resulted in variance between raters. As
a result it was replaced on the NRS-R with a
4-point scale.
Vanier et al. [40] obtained data on a sample of 286
TBI patients. Factor analysis disclosed a maximum
likelihood extraction of five correlated factors
(oblique rotation). Factor I (Intentional behaviour)
included items such as decreased initiative, difficulty
in planning, memory difficulties and difficulty in
mental flexibility. Factor II (Emotional state)
reflected mood encompassing anxiety, depression
and emotional withdrawal. The third factor was
labelled Survival oriented behaviour/emotional
state and included irritability, hostility, disinhibition
and lability of mood. Items loading on Factor IV
(Arousal state) reflect a general deficit in the
regulation of arousal which affects speed of information processing that, in turn, puts a high load on
attentional capacity. This requires greater effort to
maintain attention which, in turn, increases fatigability. The final Factor was labelled Language and
speech problems including items such as difficulties
in oral expression, oral comprehension and
articulation.
The Neurobehavioural Functioning Inventory (NFI)
The NFI is contained within the General Health and
History Questionnaire (GHHQ) [41] and elicits
information on pre-injury and post-injury demographics, work history, substance misuse and criminal history [29]. The original NFI was composed
of 105 items grouped into five categories: cognitive,
behavioural, social problems, somatic complaints
and communication, rating the frequency of problem occurrence on a 4-point scale. Kreutzer et al.
[25] administered the NFI to 520 TBI patients,
using three goodness-of-fit methods to assess the
optimal factor solution of the NFI. Six factors
were subsequently identified, namely Depression,
Somatic,
Memory/Attention,
Communication,
Aggression and Motor Response.

The Neurobehavioural Functioning Inventory66

(NFI-66)
Kreutzer and Devany [42] developed a 66-item
version of the NFI (NFI-66) through a factor
analysis of 733 patients responses to the full item
NFI. Sixty-six items were retained in a five factor
solution comprising thinking, somatization, depression, aggression and a miscellaneous factor.
The NFI has subsequently undergone several revisions with respect to the number of items included
and the underlying factor structure of the measure
[29, 42, 43].
The Neuropsychology Behaviour and Affect Profile
(NBAP)
The NBAP consists of five scales comprising 106
items designed to measure behavioural and affective
change in brain-impaired individuals [29]. The five
scales are those of: indifferencea tendency to
minimize a disability or current condition, an indifference to, or denial of, an illness; inappropriatenessbehaviour that is inappropriate to the context
in which it is occurring; depressiondysphoric
mood and/or loss of interest or pleasure in most
unusual activities; maniaelevated, expansive or
irritable mood, sustained high energy and high
levels of activity; and pragnosiaa deficiency in the
pragmatics of communicative style. In addition, a
Neutral Scale is included in an effort to control for
negative responding [26]. Response choices on the
NBAP are either yes or no and each statement
requires two responses relating to pre-morbid status
and current levels of functioning.
The Key Behaviours Change Inventory (KBCI)
The range of behaviours assessed by the KBCI [28]
was identified through informal interviews with
persons who had TBI, their family members and
caregivers discussions with TBI rehabilitation specialists and from relevant literature findings. Eight
domains, each comprising eight items, are rated on
a four-point scale, namely inattention, impulsivity,
apathy, interpersonal problems, communication
problems, unawareness of problems (anosognosia),
somatic difficulties and emotional adjustment. In
order to eliminate the possible influence of anosognosia on self-ratings, the KBCI was designed to be
completed by patients significant others, usually
a family member or caregiver.
The Head Injury Behaviour Scale (HIBS)
The HIBS is a 20-item questionnaire developed by
Godfrey et al. [44] comprising two versions, one
for the patient and a second for an informant,
usually a relative, to complete regarding the patient.

Brain Inj Downloaded from informahealthcare.com by Ceadac on 07/14/10

For personal use only.

Assessment of neurobehavioural disability

For each item on the HIBS, the respondent is
required to indicate whether the behaviour is a
problem (Yes or No) and, if so, how much
distress they experience as a consequence. Degrees
of distress are rated on a 4-point scale. The HIBS
provides an index of emotional and behavioural
disturbance and also a means of quantifying selfawareness. A total problem score and average rating
of distress score is calculated upon completion of the
measure.
A principal components analysis revealed two subscales that were labelled Emotional Regulation and
Behavioural Regulation. The Emotional Regulation
sub-scale was reported to reflect impaired emotional
control, occurring during interactions with caregivers. Conversely, the Behavioural Regulation subscale was thought to reflect problems typical of
Dysexecutive syndrome, carrying less emotional
significance for caregivers.
Ruff Neurobehavioural Inventory (RNBI)
Developed by Ruff and Hibbard [45], the RBI was
created in order to measure the well-being of people
whose lives have been changed by some catastrophic
event, including ABI. It is a self-report questionnaire
consisting of 243 items that generates information
about both pre-morbid and post-morbid functioning. Scales sensitive to abnormal response styles help
determine validity of patients responses. Factor
analysis revealed that items group around three
dimensions, namely: intrapersonal and interpersonal
functions, and quality of life.

Comparison of measures
The principal characteristics of these eight well
known instruments that have been used to describe
and measure NBD are shown in Table I. Half were
adopted from existing tools, which questions
whether measures not initially conceptualized as
specific to NBD can be sufficiently comprehensive
to reliably capture its characteristics. With regard to
clinical populations for whom various measures were
originally described, all but the NBAP were composed of people with acquired brain injury. This
particular tool was designed to capture psychological
changes in personality, affect and behaviour characteristic of neurological impairment, but was first
used to measure these consequences amongst people
with dementia rather than ABI. The RNBI was first
employed to study NBD across a wide range of
people and conditions, of which TBI was a subsample.
All the measures incorporate rating scales that
quantify the presence and/or extent of patients
difficulties. Ratings on the NRS and NRS-R are

909

generated by means of semi-structured interview and

tests; however, all other measures elicit ratings based
on the general perceptions or experience of the
respondent. On half of the measures reviewed
ratings are only obtained from the patient, which
raises questions regarding the validity of the responses. Informants may lack awareness and/or
insight or their responses may be influenced by
being involved in a claim for legal damages, which
are clearly a concern on three of the scales. The
RNBI is arguably most susceptible in this respect as
it only employs ratings from the patient. On both the
NRS and NRS-R a clinician rates items when
interviewing the patient. This may reduce threats
to validity, but it could still be argued that the
clinician may not have had the benefit of observing
the patient outside the context of the consulting
room, so cannot necessarily ratify what is reported in
that context as opposed to what relatives and others
experience in the real world. Table I shows that, on
four measures, ratings are obtained from both
patient and an informant; on one other measure
information is sought exclusively from the patient.
An approach to generating data that includes ratings
from somebody who knows the patient has the most
to offer because it allows a comparison with the
ratings made by patients themselves, theoretically
allowing any lack of insight or awareness to be
quantified. However, the informant is described as
most typically being a relative and threats to validity
may still therefore be an issue, especially in cases
involved in litigation. Other informants, most
notably caregivers and clinicians, may theoretically
be capable of the most objective reporting, but even
this will vary according to the raters level of knowledge about ABI and their experience of people who
exhibit NBD in different contexts (inside rehabilitation units or outside in the community). Variation
can also be expected on the basis of familiarity with
or training in the use of rating scales.
Sample sizes generally exceeded 100 participants;
the largest numbers were 520 (NFI) and 655 (NFI66). However the original report of the KCBI only
employed 25 participants. The number of items
comprising each measure also varies, ranging from as
few as 20 (HIBS) to as many as 243 (RNBI). With
some, the initial number of items before data
reduction methods were employed was considerably
higher. For example, the NFI and NFI-66 had 105,
the NBAP 106 and the KCBI 203. Table I shows
that data reduction methods were routinely
employed to cull redundant items and reduce the
size of measures and to identify the underlying factor
structure. In most cases, principal components
analysis (PCA) or exploratory factor analysis (EFA)
were used for these purposes, with between two and
eight sub-scales identified.

910

yes
yes
yes
no
no
yes
8
2
3
item analysis
PCA
EFA
64
20
243

66
66
no
yes

PCA principal components analysis; EFA exploratory factor analysis.

informant
self and informant
self
rating scale
rating scale
rating scale
no
no
no
KBCI
HIBS
RNBI

self and informant

self and informant
yes
no
NFI-66
NBAP

rating scale
rating scale

Head injury (655)

Progressive
neurological
disorder
TBI (25)
TBI (242)
numerous, including TBI

no
no
yes

yes
yes
yes
yes
4
5

yes
no
6

confirmatory
factor analysis
PCA
forced hand sort
70
TBI (520)
yes
NFI

self and informant

no

yes
yes
5
EFA
29
TBI (286)
yes
NRS-R

clinician

no

no
yes
4
PCA
27
TBI (101)
clinician
yes
NRS

interview, tests and

rating scale
interview, tests and
rating scale
rating scale

no

Validity
investigated
Number of factors
or sub-scales
Data reduction
method
Number
of items
Pre-morbid
function
Population
(sample size)
Respondent
Type
Adapted from
existing measure

Table I. Comparison of some of the characteristics described in the original reports of instruments used to measure NBD within ABI populations.

Brain Inj Downloaded from informahealthcare.com by Ceadac on 07/14/10

For personal use only.

Reliability
investigated

R. Ll. Wood et al.

The sample sizes, and numbers of items and
factors reported highlight further possible threats to
validity. Both low participant numbers and a high
subject-to-variable ratio may increase the chance of
introducing both Type 1 and 2 errors in identifying
reliable items and factor structure. Whilst there is no
definitive guide to what the minimum total sample
size should be when using either PCA or EFA, 50
would be seen as an absolute minimum [46], whilst
300 or more is desirable [47]. The number of
participants used to validate the KBCI falls well
below this minimum; of the remaining measures
only two, the NFI and NFI-66, had sample sizes in
excess of 300. The subject-to-variable ratio for those
studies that employed either PCA or EFA varies
from 4 : 1 to 32 : 1. As with sample size, there is no
one ratio that will work in all cases [47]. However,
estimates for a minimum ratio fall between 5 : 1 and
10 : 1, although it has been argued this should be
much higher (1530 : 1) when generalization is
critical [48]. This point has some relevance in that
the psychometric properties of NBD measures can
initially be established using low ratios, then applied
across much larger samples. With regard to the
scales considered here, only the NRS-R has a robust
enough ratio to warrant generalization. The HIBS
has a subject-to-variables ratio of 12 : 1: the other
measures have lower ratios that cannot reasonably
guarantee data reduction, which has resulted in
error.
Finally, Table I examines whether initial studies
that employed NBD measures contained accounts of
procedures used to determine the statistical properties of validity and reliability. Reliability was investigated in seven of these; however, an unexpected
finding was that data relating to reliability was absent
from three of the eight scales. It was surprising to
discover that only half the measures considered here
had made some attempt to present findings regarding both validity and reliability: given their purpose,
some comment on their psychometric properties
would seem essential. Validity and reliability will be
considered in more detail in the next section, both
with regard to the original study, as well as other
studies that have examined the eight NBD measures.

Validity and reliability of NBD measures

To further investigate claims regarding the qualities
of existing measures of NBD the psychometric
properties of the eight scales considered here will
be examined in more detail. This study proposes to
comment specifically on where each measure falls
regarding factorial, concurrent and predictive validity and internal, inter-rater and testre-test reliability. These properties were chosen as they are

correlation

ICC

Cronbachs alpha
Cronbachs alpha
Cronbachs alpha

Cronbachs alpha
Cronbachs alpha
Cronbachs alpha
4 factors
5 hypothetical factors

8 scales
2 factors
4 domains

Severity (GOS) Euroqol VAS

Differentiated dementia from controls
Differentiated stroke from controls
Differentiated CHI from controls
Differentiated CHI from MS and controls

Cronbachs alpha
5 factors

Cronbachs alpha
Neuropsychological measures MMPI
Severity (coma)

Severity (coma, GCS)

Severity (GOS) DRS neuropsychological measures

factors
factors
factors
factors
5
5
5
6

Severity (GOS) frontal lobe lesions

KPCI
HIBS
RNBI

NBAP

NFI-66

NFI

NRS-R

NRS

Levin et al. [36]

Corrigan et al. [49]
Vanier et al. [40]
McCauley et al. [50]
Kreutzer et al. [41]
Kreutzer et al. [25]
Seel et al. [51]
Kreutzer and Devanay
(unpublished) [42]
Weinfurt et al. [43]
Nelson et al. [29]
Nelson et al. [52]
Nelson et al. [26]
Kolitz et al. [28]
Godfrey et al. [44]
Ruff and Hibbard [45]

4 factors

911

% agreement

Pearsons r % agreement
Pearsons r % agreement
ICC
Cronbachs alpha
Cronbachs alpha

Inter-rater

Reliability

Internal
Concurrent/Predictive
Factorial
Measure

A cursory glance at Table II suggests that factorial,

concurrent and predictive validity appear to be well
understood for most measures. Information is
available concerning factorial validity for all eight
measures; we will return to this later. Knowledge
about concurrent and/or predictive validity is available too, except for the HIBS and RNBI. Four of the
measures relate predictably to severity of injury, as
defined by the Glasgow Coma Scale (GCS),
Glasgow Outcome Scale (GOS) or duration of
coma. Two correlate in the expected direction with
data from various neuropsychological measures.
McCauley et al. [50] found that the NRS-R
correlates with ratings made on a disability rating
scale (DRS), similarly the NFI-66 has a predictable
relationship with ratings of current health state, as
measured using the Euroqol visual analogue scale
(VAS) [43]. Kreutzer et al. [25] found associations
between the NFI and MMPI. Three of the measures
discriminated the presence of frontal lobe lesions
(NRS) and various forms of neurological impairment from controls (NBAP, KPCI). Furthermore,
the KPCI was found to discriminate between CHI
and MS patients.
Returning to the issue of factorial validity, Table II
shows that the number of factors or sub-scales range
from as few as two to as many as eight. Some of this
variance is undoubtedly due to differences in the
composition and size of the participant samples and
the number of items employed. However, it does
raise the issue of how comprehensively the various
measures capture NBD. Table II also shows that
when factorial validity has been independently
investigated, some discrepancy between the numbers
of factors appears. To investigate the content of their
component parts, Table III breaks each of the NBD
measures down into their constituent factors or subscales. This clearly demonstrates the variety of

Validity

Validity issues

Table II. Summary of main validity and reliability analyses pertaining to instruments used to measure NBD within ABI populations.

Brain Inj Downloaded from informahealthcare.com by Ceadac on 07/14/10

For personal use only.

fundamental in determining the quality of rating

scale measures. Table II attempts to summarize the
extent to which those aspects of validity and
reliability described above have been explored; it
also tries to show the extent of concordance that
exists between separate studies that have examined
the same measure. To achieve this one has utilised
both original descriptions and later studies, where
available, that report psychometric properties of the
various measures. Perhaps the most striking thing
about Table II is that none of the individual studies
consider all categories of validity and reliability;
furthermore, even for those measures that have been
the subject of two or more papers, information
concerning these psychometric properties is still not
known.

Testre-test

Assessment of neurobehavioural disability

912

R. Ll. Wood et al.

Table III. Factors and sub-scales comprising the eight NBD measures.

cognition/energy;
meta-cognition;
somatic concern/anxiety;
language
NRS-R
Vanier et al. [40]
intentional behaviour;
emotional state;
survival oriented behaviour;
arousal state;
language & speech problems
NFI
Kreutzer et al. [41]
cognitive; behavioural;
social problems;
somatic complaints;
communication
NFI-66
Kreutzer and
thinking; somatisation;
Devany (unpublished) [42] depression; aggression;
miscellaneous

Brain Inj Downloaded from informahealthcare.com by Ceadac on 07/14/10

For personal use only.

NRS

Levin et al. [36]

NBAP

Nelson et al. [29]

KPCI

Kolitz et al. [28]

HIBS

Godfrey et al. [44]

RNBI

Ruff and Hibbard [45]

McCauley et al. [50] executive cognition;

positive symptoms;
negative symptoms;
mood/affect;
oral/motor
Kreutzer et al. [25] depression; somatic;
memory/attention;
communication;
aggression; motor
Weinfurt et al. [43] cognitive deficits;
depression;
verbal & physical aggression;
somatization

indifference; inappropriateness;
depression; mania; pragnosia
inattention; unawareness of problems;
impulsivity; apathy; interpersonal problems;
communication problems;
somatic differences;
emotional adjustment
emotional regulation;
behavioural regulation
intrapersonal functions;
interpersonal functions;
quality of life

descriptive labels that have been employed to

characterize items that have been grouped together
and perhaps reflects the wider issue of how the
researchers have conceptualized what NBD actually
is. Some of these labels clearly reflect observable
behaviours and functions (such as aggression and
survival oriented behaviour), whereas others constitute operations whose existence is inferred rather
than observed (such as meta-cognition). Whilst
there are some factors and sub-scales whose labels
clearly infer commonality between some of the eight
measures, others are more idiosyncratic. Subsequent
investigations of the factor structure of three of these
(NRS-R, NFI and NFI-66) also demonstrate some
variation from those originally identified. This
variability in the number of factors, their descriptive
labels and the items comprising them conveys the
message that different researchers do not have
a homogenous view regarding what NBD actually is.
This lack of concordance may in part be attributable to how the individual items that are rated
within each scale were generated. The issues and
limitations regarding importation of measurement
instruments from the psychiatric literature and their
application to people with acquired neurological
conditions were discussed earlier. However, measures specifically designed for the purpose of
assessing behavioural and other changes associated

with ABI do not always make clear how items for

rating were generated. In some cases the outcome
literature is frequently cited, but whether a systematic or narrative search has been employed is not
known. The extent of the items to be rated will
reflect the extent of the sources of information that
were consulted. How NBD is conceptualized by the
researchers will also influence item generation. Most
of the measures employed either PCA or EFA to
arrive at a definitive set of items, but reliance on a
purely statistical approach to data reduction may
further limit how NBD is conceptualized. Perhaps
the alternative approach to both item generation and
retention employed by Kolitz et al. [28] has more to
offer? They used multiple methods, which included
interviewing people with TBI, their family members,
caregivers, rehabilitation professionals, as well as
a review of TBI outcome literature, to identify
individual items for rating and the categories in
which they were grouped. These were then reviewed
by a number of panels, again comprising people with
TBI, their relatives and clinicians, to arrive at the
final composition of the KBCI. Such an approach
certainly has much going for it by way of face
validity; however, whether it is superior to other
measures that have relied more heavily on application of multivariate statistical methods to determine
composition is debatable.

Assessment of neurobehavioural disability

Brain Inj Downloaded from informahealthcare.com by Ceadac on 07/14/10

For personal use only.

Reliability issues
In contrast to validity, a perfunctory examination of
Table II suggests that reliability properties have been
less extensively investigated amongst the eight
measures considered here. One property that was
determined in all the instruments reviewed was
internal consistency, defined as the extent to which
items thought to measure the same general construct
produce similar scores. Cronbachs alpha was
employed for this purpose in all cases and, using
Nunnallys [53] criteria, alpha coefficients for the
factors and scales that comprise these measures
generally exceed 0.70 and, as a consequence, have
acceptable reliability. However, there are exceptions.
For example, internal consistency on the NRS-R was
as low as 0.50 [40] and 0.49 on the NBAP [26]. In
contrast, knowledge concerning inter-rater and test
re-test reliability is lacking amongst the measures.
Where this has been investigated, Pearson correlations and percentage of agreement have predominantly been used to determine the extent of
agreement between different raters in a single
administration or its consistency amongst different
administrations. Use of these methods for both
purposes has been criticized: in the case of nominal
data, percentage agreement between raters does not
take into account the amount of agreement that
would be expected by chance; similarly for ordinal
data, Pearson correlations fail to take into account
differences between raters. Instead, calculation of
the Kappa statistic and Intraclass Correlation
Coefficient (ICC), respectively, are the appropriate
methods of determining reliability [54]. These data
are only known in the case of inter-rater reliability on
the NRS-R [40] and testre-test reliability on the
NBAP [26]. ICC values exceeded 0.92 on the
NBAP, suggesting it has good consistency over
time; values for the NRS-R were more variable
(0.560.85) which does raise some concerns regarding that particular instruments reliability between
raters.
Time since injury
Standardization of items and scoring do not appear
to have been influenced by time since injury,
although long-term follow-up studies suggest that
psychosocial outcome is influenced by length of time
post-injury [55, 56]. Those measures considered
here have utilised patient cohorts at various stages of
recovery, with most individuals being assessed 6
months or more after injury. Godfrey et al. [44] used
the HIBS to quantify change in patients over a
3-year period. It is not known whether factor
structures would change in the case of very longterm outcome, but, in the view of the authors, this is
unlikely. A core role of these measures and those that

913

are developed in the future is to help define how

NBD and social handicap evolve over time.
However, in the absence of these data and tools, it
is not clear whether scores should be weighted or
otherwise modified with regard to time since injury.
At this point, it would seem there are no obvious
conceptual or other reasons that would indicate that
this will be necessary when assessing patients at
a post-acute stage of recovery.

A conceptual framework for understanding and

measuring NBD
The points made about item selection above highlight a further issue regarding scales like these.
Although they have been modified or developed for
the purpose of identifying characteristics of NBD,
they do not appear to have been modelled on a
theoretical structure that can account for possible
relationships between causes and consequences of
NBD. Development of a measure that employs a
clear theoretical or conceptual model would clearly
facilitate a more informed view about NBD than can
be obtained simply by quantifying data about
behaviour.
The absence of a conceptual framework to
distinguished impairment and disability from handicap was identified some time ago [57], but has still
not been adequately addressed. As a consequence,
and as observed here, many measures currently in
use typically comprise a mixture of statements
associated with impairment, disability and handicap,
making it difficult to use the resulting score in a way
that is either clinically or socially meaningful as
a measure of progress in post-acute rehabilitation or
as a way of making any kind of reliable prognosis
about psychosocial outcome. One potential framework for the development of such scales is the
International Classification of Impairments Diseases
and Handicaps [58]. Greenwood [59] explored the
use of the revised International Classification of
Impairments Diseases and Handicaps (ICIDH-2) as
a method for classifying and understanding the
functional and social disadvantage associated with
brain injury. Wade [6062] and Wade and Halligan
[63] have also reported on the ICIDH-2 classification. Wade [62] provides an excellent summary of
this work, pointing out a number of levels and
contextual factors that need to be considered when
attempting to describe the impact of injury upon an
individual, which seems to offer a good framework
for understanding and classifying neurobehavioural
problems following brain injury.
The basic level of understanding relates to
pathology, something that is often absent in formulations for rehabilitation of NBD [64]. In cases of

Brain Inj Downloaded from informahealthcare.com by Ceadac on 07/14/10

For personal use only.

914

R. Ll. Wood et al.

acquired brain injury, for example, the ability to

understand the nature of pathology can, to a large
degree, predict the subsequent pattern of impairment and disability (for example, in a patient who
has suffered encephalitis, one can expect severe,
global impairment of memory, whilst a patient who
has suffered injury to the right frontal cortex will
exhibit few measurable cognitive weaknesses). Signs
and symptoms (comprising impairment) represent
the next descriptive level. Cognitive, motor, perceptual and communication problems fall into this
category. Wade points out how such impairments
are no more than conceptual constructs used to
explain observations of behaviour which may then be
confirmed by performance on a specialized test of
ability. The next level relates to the interaction
between a person and the environment and represents the stage at which neurobehavioural disability
can be characterized. The ICIDH-2 refers to this
level as Activities, focusing attention on alterations in
the way people carry out certain functional or social
activities that may reflect quantitative or qualitative
changes (or both). It is at this level that objective
observation is particularly important because the
level of agreement regarding some aspect of behaviour may determine the underlying cause of the
behaviour or potential for behaviour change (as
a result of rehabilitation). Observations at the level of
Activities determine the last descriptive level, that of
social handicap or, in the language of ICIDH-2,
Participation. In this context, this study interprets the
social meaning of observed behaviour and infers
changes in or relevance to social roles. These four
levels describe both the effect of illness or injury
upon the person, in terms of pathology and impairment and, through Activities & Participation, its
impact upon the persons sense of self and environment [64]. This is where context plays an important
part, because an individuals beliefs, expectations,
attitudes, etc., based on past experiences and/or
cultural factors, can determine how the person will
communicate the impact of their injury, in terms of
their behaviour.
Using this as a framework, one can see that scales
designed to measure NBD must be structured to
address observations associated with problems at the
level of Activities & Participation, because such scales
operate at the level of structured behavioural
observations (what the person does or does not
do), from which it may be possible to make
inferences about underlying cognitions or neurobiological processes. However, such scales should not
attempt to directly measure the neurological or
neuropsychological constructs that may or may not
be responsible for such behaviours.

Ideal characteristics of a measure for NBD

As a consequence of exploring some existing
measures of NBD, it seems reasonable to agree
with Kolitz et al. [28] that the ideal instrument
for this purpose has yet to be created. Indeed,
given the complexities of NBD it may be an
unreasonable expectation that any single tool
evolved for this purpose will ever be entirely
sufficient. However, that is not to say that a better,
more reliable and valid measure of NBD should not
be developed. However, if a new measure is to be
developed we need to consider what its characteristics might be. The authors would like to suggest the
following.
(1) It should have a clear theoretical or conceptual
basis. This will drive item generation beyond
the level of generalities of observable behaviour
and enable informed views on the origins of
these symptoms.
(2) The influence of the environment on behavioural symptoms of NBD should be quantified.
(3) An estimate of the impact of pre-morbid
personality on the current function and behaviour of the person with ABI should be determined. The most obvious means of achieving
this would be to ask the informant to retrospectively rate the patient on measures of NBD,
which could then be compared to ratings of
current behaviour, ability and function. The
informant would need to be a family member or
close friend of the patient, rather than
a clinician: it is acknowledged that there is
a threat of retrospective bias affecting the
validity of such a method.
(4) Expectations and the context of past, current
and future behaviour should be considered.
For example, if the person was not entirely
independent before injury, ratings of possible
NBD need to take this into account, especially
when planning rehabilitation and discharge
goals. However, the level of functional and
social handicap for a person who lived an
autonomous existence before ABI would be
correspondingly greater, despite similar ratings
of behaviour within a rehabilitation unit.
A system of weighting scores would convey
this difference.
In addition, this investigation has identified
a number of issues regarding the psychometric
properties such a measure should have.
(1) To be a valid measure of NBD the statistical
and other properties of the measure should be
determined using the target population for that
measure (for example, TBI).

Brain Inj Downloaded from informahealthcare.com by Ceadac on 07/14/10

For personal use only.

Assessment of neurobehavioural disability

(2) Likert-type rating scales seem most suited to
quantifying behaviours that are symptomatic of
NBD. However, these will be concerned with
capturing impressions or perceptions of behaviour, usually over a specified period of time, and
not specific incidents. Observational rating
scales, such as those developed for aggression
[65] and sexually inappropriate behaviour [66]
are most appropriate when this level of detail
is needed, but may generate information that is
too complex about single incidents to provide
a profile that leads to reasonable expectations
about psychosocial outcome.
(3) There are significant threats to validity inherent
in relying only on ratings made by the person
with ABI. Data should always be sought from
an informant, ideally a caregiver or rehabilitation worker.
(4) Sample sizes need to be sufficient to robustly
handle relevant statistical analyses. The ideal is
more is better. The literature suggests that
when data reduction methods such as PCA and
EFA are to be used, sample sizes that exceed
100 participants should be sought. In addition,
constraints should be put on the number of
factors such analyses should seek if the subjectto-variable ratio is low (minimum of 10 : 1 or
less).
(5) As well as factorial validity, information about
concurrent and predictive reliability should be
determined. Confidence in any NBD measure
will be increased if it can be demonstrated that it
relates in a predictable way to other indicators of
outcome and that it is able to reliably discriminate the target clinical population from neurologically healthy controls and other clinical
populations.
(6) In addition to internal consistency, other
properties of reliability need to be reported,
particularly those of inter-rater and testre-test
reliability. Extreme variance between raters
should not be present as this casts doubt that
the same construct is being measured. Similarly,
stability over time needs to be determined in
order to reliably attribute any change in scores
and ratings to, for example, rehabilitation
efforts. Appropriate statistical methods that are
less likely to result in errors also need to be
employed to determine these properties, for
example, Kappa and the intraclass correlation
coefficient, rather than percentage agreement
between rater and Pearson correlations.
(7) Finally, item selection and retention needs to be
comprehensive enough to reasonably capture
the extent and severity of NBD symptoms.
Multiple sources should be used, most notably
people with ABI, their families, caregivers and

915

rehabilitation workers. As argued earlier, working from a clearly defined theoretical or conceptual framework will further ensure the scope
of items is sufficient.
Should an awareness/insight coefficient constitute an
ideal characteristic of any measure of NBD?
Threats to the validity and reliability of information
that relies totally on the patient as informant have
been discussed. The tendency to under-report
difficulties also casts doubt on the validity of selfreport measures to inform treatment [6771].
However, there is still value in obtaining ratings
from patients, because these self-perceptions can
be compared with ratings obtained from relatives
or carers to help determine characteristics such
as self-awareness/insight or social judgement.
Comparisons of this kind may be regarded as
an ideal characteristic of future NBD measures.
For example, the authors of the NFI advocated that
any discrepancy between informant and patient
ratings could be used to help assess the degree of
post-injury adjustment. This may be related to
experience as a function of time since injury. For
example, Godfrey et al. [44] noted from HIBS
ratings that patients tended to under-report the
severity of behaviour problems within the first
6 months after closed head injury compared to
those injured 13 years earlier. Those who were at
a later stage post-injury evidenced better insight into
emotional and behaviour difficulties, but presented
higher levels of emotional dysfunction.
Cooper-Evans et al. [72] found a similar relationship between greater awareness of deficit and
increased psychological distress, but this was not
a function of time since injury. Their sample had
suffered ABI a mean of 10 years earlier. The
cognitive function of patients with the lowest levels
of insight was much worse than the group whose
insight was better preserved. However, such patients
paradoxically rated themselves as having higher selfesteem than the insightful patients, highlighting an
ethical dilemma for clinicians regarding the management of reduced awareness of deficit, because this
research suggests that self-esteem and psychological
distress may be adversely affected by therapy, whose
goal is to improve insight.
An index of insight and awareness embedded
within NBD measures would be useful, but clinically
it would need to be interpreted with caution, within
a broader set of outcome measures. It cannot be
reliably predicted that insight and awareness will
improve with time, so a measure of reduced insight
will not pinpoint any particular stage of neurobehavioural recovery. Therefore, whilst it might be useful
to provide such an index, it will not be possible to

916

R. Ll. Wood et al.

rely on a discrepancy score between a patient and

relative or carers rating as an objective and reliable
measure of insight. Similarly, flagging up such an
index should not automatically cue therapeutic
activity to improve awareness, as this could result
in greater levels of psychological distress for some
patients.

Brain Inj Downloaded from informahealthcare.com by Ceadac on 07/14/10

For personal use only.

Concluding remarks
ABI is one of the few clinical conditions for which
reliable long-term prognoses remain elusive. Current
assessment tools contain methodological problems of
definition, conceptualization and measurement of
NBD. Furthermore, many fail to use a conceptual
framework that distinguishes impairment and disability from handicap. As a consequence, scores from
assessment tools are difficult to use in a way that
is clinically or socially meaningful as an indication
of outcome. This study has proposed that ICIDH2
is a potential framework for the development of
new tools for quantifying NBD as it provides
a system that facilitates understanding and classifying neurobehavioural characteristics of ABI. Future
observational tools based on this framework could
overcome many of the issues with existing measures
discussed in this paper. Furthermore, maximizing
the reliability and validity of new tools will help
clinicians generate more robust understanding of the
nature and severity of neurobehavioural disability
and social handicap, whilst providing information
that is clinically and socially meaningful as an
indication of outcome. New and better measures
will improve formulations for rehabilitation of NBD,
planning and implementation of treatment goals,
discharge planning and knowledge about outcome.
Changes in scores over time should be capable
of reliable interpretation by clinicians as
reflecting meaningful change in levels of NBD
and social handicap, rather then perhaps being
a product of any poor statistical qualities of
a measurement tool.
This review has attempted to clarify some
issues that relate to the assessment of NBD which
need to be addressed in order to promote understanding of how its characteristics arise and how
these may subsequently impact on long-term psychosocial outcome. The authors hope that the
critical analysis presented in this paper will generate
further interest amongst researchers to develop
further measurement tools that will reliably inform
rehabilitation staff of those legacies of ABI that are
likely to have long-term psychosocial significance
and which need to be a focus of rehabilitation
interventions.

Acknowledgements
The information in this manuscript and the manuscript itself is new and original and has never been
published either electronically or in print.
Declaration of interest: The authors report no
conflicts of interest. The authors alone are responsible for the content and writing of the paper.

References
1. Oddy M, Coughlan T, Tyerman A, Jenkins D. Social
adjustment after closed head injury: A further follow-up
seven years after injury. Journal of Neurology, Neurosurgery,
and Psychiatry 1985;48:564568.
2. Cicerone KD, Tanenbaum LN. Disturbance of social
cognition after traumatic orbitofrontal brain injury. Archives
of Clinical Neuropsychology 1997;12:173188.
3. McDonald S, Flanagan SM, Rollins JB, Kinch JM. TASIT:
A new clinical tool for assessing social perception after
traumatic brain injury. Journal of Head Trauma
Rehabilitation 2003;18:219238.
4. Wood R Ll, Williams C. Neuropsychological correlates of
organic alexithymia. Journal of the International
Neuropsychological Society 2007;13:471479.
5. McDonald S, Flanagan S, Martin I, Saunders C. The
ecological validity of the TASIT: A test of social perception.
Neuropsychological Rehabilitation 2004;14:285302.
6. McDonald S, Saunders JC. Differential impairment in
recognition of emotion across different media in
people with severe traumatic brain injury. Journal of
the International Neuropsychological Society 2005;11:
392399.
7. Wood R Ll, Williams C. Inability to empathize following
traumatic brain injury. Journal of the International
Neuropsychological Society 2008;14:289296.
8. Starkstein SE, Robinson RG. Mechanisms of disinhibition
after brain lesions. Journal of Nervous and Mental Disease
1997;185:108114.
9. Stewart JL, Tannock R. Inhibitory control differences
following mild head injury. Brain and Cognition 1999;41:
411416.
10. McKinlay WW, Brooks DN, Bond MR, Martinage DP,
Marshall MM. The short-term outcome of severe blunt
head injury as reported by relatives of the injured person.
Journal of Neurology, Neurosurgery and Psychiatry 1981;44:
527533.
11. Prigatano GP. Personality disturbances associated with
traumatic brain injury. Journal of Consulting and Clinical
Psychology 1992;3:360368.
12. Mateer CA. The rehabilitation of executive disorders. In:
Stuss DT, Wincour G, Robertson IH, editors. Cognitive
neurorehabilitation. Cambridge: Cambridge University
Press; 1999. pp 314332.
13. Brewer TL, Metzger BL, Therrien B. Trajectories of
cognitive recovery following a minor brain injury. Research
in Nursing and Health 2002;25:269281.
14. Eslinger PJ, Grattan LM, Geder L. Impact of frontal lobe
lesions on rehabilitation and recovery from acute brain injury.
Neurorehabilitation 1995;5:161185.
15. Greve KW, Love J, Sherwin E, Stanford MS, Mathias C,
Houston R. Cognitive strategy usage in long term survivors
of severe traumatic brain injury with persisting impulsive
aggression. Personality and Individual Differences 2002;32:
639647.

Brain Inj Downloaded from informahealthcare.com by Ceadac on 07/14/10

For personal use only.

Assessment of neurobehavioural disability

16. Wood RL, Liossi L. Neuropsychological and neurobehavioural correlates of aggression following traumatic brain
injury.
Journal
of
Neuropsychiatry
and
Clinical
Neurosciences 2006;18:333341.
17. Eames P, Wood RLl. Episodic disorders of behaviour and
affect after acquired brain injury. Neuropsychological
Rehabilitation 2003;13:241258.
18. Wood RL. Behaviour disorders following severe brain injury:
Their presentation and psychological management. In:
Brooks N, editor. Closed head injury. Psychological, social,
and family consequences. Oxford: Oxford University Press;
1984. pp 195219.
19. Prigatano GP. Higher cerebral deficits; history of methods of
assessment and approaches to rehabilitation. BNI Quarterly
1986;3:1526.
20. Wood R Ll, Yurdakul LK. Change in relationship status
following traumatic brain injury. Brain Injury 1997;11:
491501.
21. Weddell R, Oddy M, Jenkins D. Social adjustment
after rehabilitation: A two year follow-up of patients with
severe head injury. Psychological Medicine 1980;10:
257263.
22. Wood RL. Understanding neurobehavioral disability. In:
Wood RL, McMillan T, editors. Neurobehavioral disability
and social handicap after head injury. Hove, UK: Psychology
Press, Taylor & Francis; 2001.
23. Wood RL. Brain injury rehabilitation: A neurobehavioral
approach. London: Croom Helm; 1987.
24. Wood RL. Neurobehavioral sequelae of traumatic brain
injury. New York: Taylor & Francis; 1990.
25. Kreutzer JS, Marwitz JH, Seel R, Devany C. Validation of
a neurobehavioural functioning inventory for adults with
traumatic brain injury. Archives of Physical Medicine and
Rehabilitation 1996;77:116124.
26. Nelson LD, Drebing C, Satz P, Uchiyama C. Personality
change in head trauma: A validity study of the
Neuropsychology Behaviour and Affect Profile. Archives of
Clinical Neuropsychology 1998;13:549560.
27. Satz P, Holsten SG, Uchiyama CL, Shimahara G,
Mitrushina MM, Forney DL, Zaucha K, et al.
Development and evaluation of validity scales for the
neuropsychology behaviour and affect profile: A dissembling
study. Psychological Assessment 1996;8:115124.
28. Kolitz BP, Vanderploeg RD, Curtiss G. Development of
the key behaviours change inventory: A traumatic
brain injury behavioural outcome assessment instrument.
Archives of Physical Medicine and Rehabilitation 2003;84:
277284.
29. Nelson LD, Satz P, Mitushina M, Van Gorp W, Cicchetti D,
Lewis R, Van Lacker D. Development and validation of the
neuropsychology behaviour and affect profile. Psychological
Assessment 1989;1:266272.
30. Tateno A, Jage RE, Robinson RG. Clinical correlates of
aggressive behaviour after traumatic brain injury. The Journal
of Neuropsychiatry and Clinical Neurosciences 2003;15:
155160.
31. Baguley IJ, Cooper J, Felmingham KL. Aggressive
behaviour following traumatic brain injury: How common is
common? Journal of Head Trauma Rehabilitation 2006;21:
4557.
32. Alderman N, Knight C, Henman C. Aggressive
behaviours observed within a neurobehavioural rehabilitation
service: Utility of the OAS-MNR in clinical audit and applied
research. Brain Injury 2002;16:469489.
33. Alderman N. Prevalence, characteristics and causes of
aggressive behaviour observed within a neurobehavioural
rehabilitation service: Predictors and implications for management. Brain Injury 2007;21:891911.

917

34. Alfano DP, Alan M, Finlayson J, Stearns GM, Neilson PM.

The MMPI and neurologic dysfunction: Profile configuration
and analysis. The Clinical Neuropsychologist 1990;4:6979.
35. Gass CS. MMPI-2 interpretation and closed-head-trauma:
A correction factor. Psychological Assessment 1991;3:2731.
36. Levin HS, High WM, Goethe KE, Sisson RA, Overall JE,
Rhoades HM, Eisenberg HM, Kalisky Z, Gary HE. The
neurobehavioural rating scale: Assessment of the behavioural
sequelae of head injury by the clinician. Journal of Neurology,
Neurosurgery, and Psychiatry 1987;50:183193.
37. Malloy P, Grace J. A review of rating scales for measuring
behaviour change due to frontal systems damage. Cognitive
and Behavioural Neurology 2005;18:1827.
38. Overall JE, Gorham DR. The brief psychiatric rating scale.
Psychological Reports 1962;10:799812.
39. Levin HS, Mazaux J-M, Vanier M, et al. Evaluation des
troubles neurophysiologiques et comportementaux des traumatises craniens per le clinicien: Proposition dune echelle
neurocomportementale et premiers resultas de sa version
francaise. Annales de Readaption et de Medicine Physique
1990;33:3540.
40. Vanier M, Mazaux J-M, Lambert J, Dassa C, Levin HS.
Assessment of neuropsychologic impairments after head
injury: Interrater reliability and factorial and criterion
validity of the Neurobehavioural Rating Scale-Revised.
Archives of Physical Medicine and Rehabilitation 2000;81:
796806.
41. Kreutzer JS, Leininger BE, Doherty K, Waaland PK. General
Health and History Questionnaire. Richmond, VA: Medical
College of Virginia, Rehabilitation Research and Training
Center on Severe Brain Injury; 1987.
42. Kreutzer JS, Devany CW. (Unpublished). Cited in: Weinfurt
KP, Willke R, Glick HA, Schulman KA. Towards a composite scoring solution for the neurobehavioural functioning
inventory. Quality of Life Research 1999;8:1724.
43. Weinfurt KP, Willke R, Glick HA, Schulman KA. Towards
a composite scoring solution for the neurobehavioural
functioning inventory. Quality of Life Research 1999;8:1724.
44. Godfrey HPD, Partridge FM, Knight RG. Course of insight
disorder and emotional dysfunction following closed head
injury: A controlled cross-sectional follow up study. Journal
of Clinical and Experimental Neuropsychology 1993;15:
503515.
45. Ruff RM, Hibbard KM. Ruff Neurobehavioural Inventory
(RNBI) professional manual. Lutz, FL: Psychological
Assessment Resources; 2003.
46. Barrett PT, Kline P. The observation to variable ratio in
factor analysis. Personality Study and Group Behavior
1981;1:2333.
47. MacCallum RC, Widaman KF, Zhang S, Hong S.
Sample size in factor analysis. Psychological Methods 1999;
4:8499.
48. Osborne, JW, Costello AB. Sample size and subject to item
ratio in principal components analysis. Practical Assessment,
Research and Evaluation, 9(11). Available online at: http://
PAREonline.net/getvn.asp?v9&n11, accessed 15 March
2008.
49. Corrigan JD, Dickerson J, Fisher E, Meyer P. The
Neurobehavioural Rating Scale: Replication in an
acute, inpatient rehabilitation setting. Brain Injury 1990;4:
215222.
50. McCauley SR, Levin HS, Vanier M, Mazaux J-M, Boake C,
Golfader PR, Rockers D, Butters M, Kareken DA, Lambert J
et al. The neurobehavioural rating scale-revised: Sensitivity
and validity in closed head injury assessment. Journal
of Neurology, Neurosurgery, and Psychiatry 2001;71:
643651.

Brain Inj Downloaded from informahealthcare.com by Ceadac on 07/14/10

For personal use only.

918

R. Ll. Wood et al.

51. Seel RT, Kreutzer JS, Sander AM. Concordance of patients

and family members ratings of neurobehavioural functioning
after traumatic brain injury. Archives of Physical Medicine
and Rehabilitation 1997;78:12541259.
52. Nelson LD, Mitushina M, Satz P, Sowa M, Cohen S. Crossvalidation of the neuropsychology behaviour and affect profile
in stroke patients. Psychological Assessment 1993;5:
374376.
53. Nunnally JC. Psychometric theory. 2nd ed. New York:
McGraw-Hill; 1978.
54. Howell DC. Statistical methods for psychology. 4th ed.
Belmont, CA: Duxbury Press; 1997.
55. Wood R Ll. Long-term outcome of serious traumatic brain
injury. European Journal of Anaesthesiology 2008;25:
115122.
56. Thomsen IV. Late outcome of very severe blunt head trauma:
A 1015 year second follow-up. Journal of Neurology,
Neurosurgery and Psychiatry 1984;47:260268.
57. Brooks DN, McKinlay W. Personality and behavioural
change after severe blunt head injurya relatives view.
Journal of Neurology, Neurosurgery and Psychiatry 1983;
46:336344.
58. World Health Organization. International classification of
impairments, disabilities and handicaps. Geneva: WHO;
1980.
59. Greenwood R. The consequences of brain injury:
Classification
and
assessment
of
outcome.
Neuropsychological Rehabilitation 1999;9:231240.
60. Wade DT. Social context as a focus for rehabilitation.
Clinical Rehabilitation 2001;15:459461.
61. Wade DT. Cognitive assessment and neurological rehabilitation. Clinical Rehabilitation 2002;16:117118.
62. Wade DT. Applying the WHO framework to the rehabilitation of patients with cognitive deficits. In: Halligan PW,
Wade DT, editors. Effectiveness of rehabilitation for cognitive deficits. Oxford: Oxford University Press; 2005.
63. Wade DT, Halligan P. New wine in old bottles; the WHO
ICF as an explanatory model of human behaviour. Clinical
Rehabilitation 2003;17:349354.

64. Wood RL, Worthington AD. Neurobehavioural rehabilitation in practice. In: Wood RL, McMillan T, editors.
Neurobehavioral disability and social handicap after head
injury. Hove, UK: Psychology Press, Taylor & Francis; 2001.
65. Alderman N, Knight C, Morgan C. Use of a modified version
of the Overt Aggression Scale in the measurement and
assessment of aggressive behaviours following brain injury.
Brain Injury 1997;11:503523.
66. Knight C, Alderman N, Johnson C, Green S, BirkettSwan L, Yorston G. The St Andrews Sexual Behaviour
assessment (SASBA): Development of a standardised recording instrument for the measurement and assessment of
challenging sexual behaviour in people with progressive and
acquired neurological impairment. Neuropsychological
Rehabilitation 2008;18:129159.
67. Ezrachi O, Ben-Yishay Y, Kay T, Diller L, Rattock J.
Predicting employment in traumatic brain injury following
neuropsychological rehabilitation. Journal of Head Trauma
Rehabilitation 1991;6:7184.
68. Hillier SL, Metzer J. Awareness and perceptions of
outcomes after traumatic brain injury. Brain Injury
1997;11:525536.
69. Port A, Willmott C, Charlton J. Self-awareness following
traumatic brain injury and implications for rehabilitation.
Brain Injury 2002;16:277289.
70. Prigatano GP, Fordyce DJ. Cognitive dysfunction and
psychosocial adjustment after brain injury. In: Prigatano
GP, Fordyce DJ, Zeiner HK, Roueche JR, Pepping M, Wood
BC, editors. Neuropsychological rehabilitation after brain
injury. Baltimore, MD: Johns Hopkins University Press;
1998. pp 117.
71. Sbordone RJ, Seyranian R, Ruff RM. Are the subjective
complaints of traumatically brain injured patients reliable?
Brain Injury 1998;11:505515.
72. Cooper-Evans S, Alderman N, Knight C, Oddy M. Selfesteem and self-concept as predictors of mood and behaviour
during rehabilitation following severe brain injury: An
exploratory study. Neuropsychological Rehabilitation
2008;18:607626.