THE AERONAUTICAL JOURNAL
The Cranfield aircraft handling qualities rating
scale: a multidimensional approach to the
assessment of aircraft handling qualities
D. Harris, J. Gautrey, K. Payne and R. Bailey
College of Aeronautics, Cranfield University
Cranfield, UK
ABSTRACT
The Cooper-Harper scale is currently the only well-established scale
for assessing aircraft handling qualities. However, as a result of having
a unidimensional format the Cooper-Harper scale lacks diagnostic
power and has also been criticised for exhibiting poor reliability. The
Cranfield Aircraft Handling Qualities, Rating Scale (CAHQRS) is a
new, multidimensional rating scale using concepts from two
established scales, the NASA-TLX workload scale and the Cooper-
Harper scale. This paper provides an overview of the development of
the CAHQRS and the results from a series of validation trials in an
engineering flight simulator. An in-flight refuelling task and an
approach and landing task using a range of control laws were used
in these evaluations. The results showed that the CAHQRS had high
positive correlations with the Cooper-Harper scale. However, the
CAHQRS also demonstrated greater validity, diagnostic power and
higher test/re-test reliability than the previous scale.
NOMENCLATURE
B raw regression weight
F F-ratio from analysis of variance
N
/s not significant
R regression coefficient
R2 regression coefficient squared (indicating amount of
variance accounted for in the criterion variable by the
predictor variables) in the sample.
Sig statistical significance (probability)
R2(adj) regression coefficient squared (indicating amount of
variance accounted for in the criterion variable by the
predictor variables) adjusted to reflect the likely variance
explained in the population.
r Pearson's correlation coefficient
t-value student f-ratio
p probability of making a type I decision error
P standardised regression weight
1.0 INTRODUCTION
The assessment of an aircraft's handling qualities has always been a
vital part in its design and development process. For over 50 years,
the US Department of Defense (DoD) has required military aircraft to
Paper No. 2461. Manuscript received 4 March 1999, revised version received 13 December 1999, accepted 14 February 2000.
Downloaded from http:/www.cambridge.org/core. Imperial College London Library, on 23 Nov 2016 at 11:31:15, subject to the Cambridge Core terms of use, available at
http:/www.cambridge.org/core/terms. http://dx.doi.org/10.1017/S0001924000028098
APRIL 2000 191
exhibit 'acceptable' handling qualities. Initially, this requirement
was simply defined in terms of stability and control. However,
subsequent versions of these specifications'1' have incorporated pilot
opinion into the assessment in addition to these classical engineering
parameters. Civil airworthiness requirements'2' 3' also require aircraft
to demonstrate satisfactory handling qualities as part of the certification
process. With the advent of fly-by-wire (FBW) flight control systems,
the control engineer now has many more options from which to
choose concerning the manner in which an aircraft will respond to a
pilot's control inputs and considerably more precise ways to refine
an aircraft's behaviour. This variety of options which were unavailable
30 years ago, has made the assessment of aircraft handling qualities
even more important than in the past. There are now many more
options from which to choose.
Despite the importance of providing an aircraft with acceptable
handling qualities, it still remains difficult to define precisely what is
meant by this term; it is even more difficult to operationalise the
term through measurement. Handling qualities have been defined as
'those qualities or characteristics of an aircraft that govern the ease
and precision with which a pilot is able to perform the tasks required
in support of the aircraft's role'4)'. Other authors'5' have suggested
that this definition should be expanded to include the ease with
which the pilot can compensate for any disturbing effects of the
environment. It should be noted, however, that both definitions
incorporate pilot opinion in them.
The incorporation of pilot opinion into the assessment of an
aircraft's handling qualities is essential. Simple physical measurement
alone of an aircraft's open-loop response to a disturbance input can
give a misleading impression concerning how easy an aircraft is to
fly. An aircraft may exhibit excellent open-loop stability, but in the
hands of a pilot trying to fly a high-gain, closed-loop task (for example
air-to-air refuelling), it may be almost uncontrollable'4'. An engineer
makes judgements about the aircraft's handling qualities from data
derived from only a part of the total pilot/vehicle system; only the
pilot can observe the complete dynamic behaviour of the aircraft.
Thus, handling qualities cannot be defined simply by engineering
descriptions of an aircraft's response to a pilot's control input. Any
description or assessment of handling qualities must include the pilot's
perceptions of the behaviour of the aircraft.
The only scale in widespread use for the pilot assessment of aircraft
handling is the Cooper-Harper Scale'6'. The Cooper-Harper scale is
itself an amalgam of two earlier handling qualities scales, the Cooper
scale'7' and the Cornell Aeronautical Laboratory (CAL) scale'8'. The
 192 THE AERONAUTICAL JOURNAL. APRIL 2000

Cooper-Harper Scale is a unidimensional, ten point scale, where the the pilot's qualitative comments, in practice these latter items are often
lower the final rating given, the more desirable are the aircraft's omitted from the analysis'"). They generate a great deal of data from
handling qualities. The scale uses a decision tree format that requires which it is difficult to assimilate and quantify patterns in handling
the user to make a series of up to three binary decisions about the deficiencies. The lack of diagnostic power in the scale is related to a
handling qualities of their aircraft. These decisions are concerned lack of content validity. For a scale to have content validity, its
with if the aircraft is controllable, or not; if it is controllable, is components must encompass all the pertinent aspects of the domain
adequate performance attainable with tolerable pilot workload; and to be assessed. This requires a systematic assessment of the domain
if adequate performance can be achieved, are the aircraft's handling at the outset of the scale's development"2'. It was also observed that
qualities satisfactory without improvement? With the exception of Cooper-Harper ratings reflected different aspects of an aircraft's
the first decision point (controllable or not) each binary decision handling qualities depending upon the nature of the manoeuvre being
leads to three further descriptions of aircraft performance which undertaken"3*. For example, during the climb, Cooper-Harper ratings
subsequently leads the pilot to a handling quality rating. An example were essentially an assessment of the aircraft's behaviour in pitch
of the Cooper-Harper scale is given in Fig. 1. The US DoD standard and roll; during the cruise the scale measured the effectiveness of the
for the flying qualities of piloted airplanes1" categorises flying qualities aircraft's trim systems; during turning manoeuvres it was measuring
into three levels. An aircraft with level one flying qualities is defined pitch, roll or yaw (depending upon if it was an assessment of entering
as being adequate for its mission and is satisfactory without the turn or maintaining the turn); and for the approach phase the
improvement. Level two flying qualities are adequate to accomplish handling qualities scale was evaluating the aircraft's behaviour in
the mission but at the expense of increased pilot workload or decreased pitch, roll and speed control. Furthermore, the handling qualities of
mission effectiveness. Level three flying qualities suggest that the an aircraft can only be defined with respect to a specific task. A
aircraft can be controlled safely but pilot workload is excessive global assessment of an aircraft's behaviour, irrespective of the
and/or mission effectiveness is inadequate. Level one flying qualities manoeuvre being performed, is meaningless. The Cooper-Harper
correspond to Cooper-Harper ratings of 3-5 and below; level two flying scale cannot directly quantify this inter-relationship between the
qualities correspond to Cooper-Harper ratings of between 3-5 and aircraft's handling qualities and the demands of the flight task in
6-5; level three aircraft have handling qualities ratings in excess of which they are being assessed.
6-5.
There have also been questions raised about the reliability of the
The Cooper-Harper scale does have a few shortcomings, however. Cooper-Harper scale" 4 ' 15 '. It has been found that the scale exhibits
The assessment of aircraft handling qualities is complex but this poor inter-pilot reliability and that often the ratings provided do not
complexity is not reflected in the scale. The scale lacks diagnostic tally with the pilot's opinions expressed on the comment cards for
power. A pilot makes a rating about the aircraft's behaviour but this the corresponding manoeuvre"4'. For example, these comments may
rating alone cannot describe those aspects of the aircraft's handling suggest that the evaluation pilot could not achieve the desired level
that may be unacceptable. Several authors have criticised the diagnostic of performance, although the Cooper-Harper ratings provided would
power of unidimensional scales (using the format found in the Cooper- suggest quite the contrary. In tests of new control laws for FBW
Harper scale) to assess multidimensional concepts. Although Cooper- aircraft"5' it was observed that different pilot's Cooper-Harper ratings
Harper scale ratings should never be interpreted without reference to for the same control configuration could range from (3): 'satisfactory
Adequacy for selected Aircraft Demands on the pilot in selected Pilot
task or operation Characteristics task or required operation. rating
Excellent Pilot compensation not a factor for desired
Highly Desirable performance
Good. Negligible Pilot compensation not a factor for desired
deficiencies performance
Fair. Some mildly Minimal pilot compensation required for
unpleasant deficiencies desired performance

Minor but annoying Desired performance requires moderate

Deficiencies deficiencies compensation
warrant Moderately objectionable Adequate performance requires extensive
improvement deficiencies pilot compensation
Very objectionable but Adequate performance requires
tolerable deficiencies considerable pilot compensation

Adequate performance not attainable with

Major deficiencies maximum tolerable pilot compensation.
Deficiencies Controllability not in question
warrant
Major deficiencies Intense pilot compensation is required to
improvement
retain control
Major deficiencies Considerable pilot compensation is
required for control

Improvement Control will be lost during some

mandatory Major deficiencies
proportion o f required operation

Pilot decisions

Figure 1. Cooper-Harper handling qualities rating scale (adapted from Cooper and Harper, 1969).

Downloaded from http:/www.cambridge.org/core. Imperial College London Library, on 23 Nov 2016 at 11:31:15, subject to the Cambridge Core terms of use, available at
http:/www.cambridge.org/core/terms. http://dx.doi.org/10.1017/S0001924000028098
 HARRIS ET AL T H E CRANFIELD AIRCRAFT HANDLING QUALITIES RATING SCALE: A MULTIDIMENSIONAL APPROACH...
without improvement but some minor deficiencies' to (7): 'deficiencies
require improvement adequate performance not attainable with
maximum pilot compensation.' In the same trial, it was not uncommon
for the same pilot, flying the same configuration, to vary his/her ratings
by up to three Cooper-Harper points on re-test. This is indicative of
poor within-rater reliability.
As a result of these criticisms of the Cooper-Harper scale, the
complexity of an aircraft's behaviour and the numerous options that
a control engineer can implement in a modern FBW system, clearly
what is required is a multi-dimensional approach to the assessment
of handling qualities. This should be able to describe all aspects of
an aircraft's behaviour and also take into account the interaction
between the aircraft's handling qualities and the flight task. It is
important that any such scale is simple to complete and is composed
of the minimum number of sub-scales to evaluate the whole handling
qualities domain comprehensively. Finally, the scale should be
applicable to all types of aircraft and it should relate directly to an
aircraft's behaviour in flight. These were the objectives when
developing the Cranfield Aircraft Handling Qualities Rating Scale
(CAHQRS). However, due to the complexity of rotary wing flight,
the prototype scale initially described only the handling qualities of
fixed wing aircraft.
2.0 THE CRANFIELD AIRCRAFT HANDLING
QUALITIES RATING SCALE
2.1 Derivation of the parameters of aircraft handling
qualities
The parameters of the CAHQRS were derived from data gathered
from 141 qualified test pilots from the register of the Society of
Experimental Test Pilots. These pilots were required to rate the
adequacy of their regular aircraft's handling using each of the 12
dimensions of handling qualities specified in the British Civil
Airworthiness Requirements, 1981 (section D2-FLIGHT)06>. These
were: longitudinal (pitch) control; longitudinal static stability; short
period pitch response; lateral (roll) control; static stability (lateral);
dynamic stability (spiral mode); directional (yaw) control; static
stability (directional); dynamic stability (Dutch roll); trimming (the
ease with which the aircraft could be trimmed swiftly and accurately);
trim rate; and speed control. Each of these dimensions was assessed
using a 1-10, Cooper-Harper format rating scale (see Fig. 1). The
respondents were required to assess their aircraft's handling in seven
common flight scenarios which were designed, in combination, to
encompass all of the 12 parameters described above. The flight
scenarios (including adequate and desired performance parameters)
were:
Take-off task: select the correct take-off pitch attitude (desired
1; adequate 3) whilst maintaining the heading (desired 1;
adequate 3).
Climb task: maintain desired climb speed (desired 2kt; adequate
5kt) whilst maintaining the heading (desired 2; adequate 5).
Cruise task: maintain desired altitude (desired 20ft; adequate
100ft) and desired heading, (desired 2; adequate 5), whilst
maintaining desired speed (desired 2kt; adequate 5kt).
Turning task 1: enter a coordinated turn to establish a 25 bank
angle, (desired 2; adequate 5) and maintain a 25 bank angle
whilst maintaining height (desired 50ft; adequate 100ft).
Turning task 2: roll out of a 25 banked turn onto a selected heading
(desired 2; adequate 5) whilst maintaining height (desired
50ft; adequate 100ft).
Approach task: select the correct attitude (desired 1; adequate
3) and desired rate of descent (desired 100ftmin-1; adequate
200ftmin-') whilst maintaining desired speed (desired 2kt;
adequate 5kt) and maintaining desired heading (desired 1;
adequate 3).
Downloaded from http:/www.cambridge.org/core. Imperial College London Library, on 23 Nov 2016 at 11:31:15, subject to the Cambridge Core terms of use, available at
http:/www.cambridge.org/core/terms. http://dx.doi.org/10.1017/S0001924000028098
193
Landing task: land on the required spot, (desired within 0-25 of
the aircraft's wingspan laterally; adequate within 0-5 of the aircraft's
wingspan; desired within two lengths of the aircraft from the
required spot longitudinally; adequate within four lengths of
the aircraft longitudinally) at the correct airspeed (desired 2kt;
adequate 5kt).
Handling qualities ratings were obtained for 75 different types of
aircraft, including commercial and military transport aircraft, large
turbofan powered passenger aircraft, business jets, commuter
turboprops, combat aircraft, general aviation aircraft, experimental
aircraft and historic aircraft.
The data obtained were subject to a series of confirmatory factor
analyses, (a statistical technique to derive a smaller number of
common, underlying dimensions from a larger data set*17' l8>). These
confirmatory factor analyses indicated that there were five stable
underlying dimensions to aircraft handling qualities that were common
to all flight scenarios. The statistical derivation of these underlying
dimensions are described elsewhere'13- "). These dimensions were
labelled; longitudinal (pitch) handling characteristics; lateral (roll)
handling characteristics; directional (yaw) handling characteristics;
trimming; and speed control. The relationship between the initial
variables and the higher order factors derived is described
diagrammatically in Fig. 2.
These five dimensions were subsequently used in the final derivation
of the Cranfield Aircraft Handling Qualities Rating Scale
(CAHQRS). A full description of the derivation of the CAHQRS
dimensions can be found elsewhere"3' l9>.
2.2 Format and administration of the CAHQRS
As previously discussed, the handling qualities of an aircraft cannot
be defined in isolation. They can only be defined with respect to a
specific task. For example, an aircraft may exhibit exemplary
behaviour in the landing flare but may have undesirable handling
characteristics when the pilot is engaged in a longer-duration, high-
gain task, such as air-to-air refuelling. There is a strong interaction
between aircraft handling qualities and the task at hand. Similarly,
different aspects of an aircraft's behaviour will be important in different
circumstances. For example, difficulty in trimming the aircraft
quickly and precisely may be of less importance on take-off or during
the landing flare than during cruising flight.
A directly analogous situation to the assessment of aircraft handling
qualities was faced by researchers into pilot workload in the 1980s.
Figure 2. Dimensions of aircraft handling qualities derived from the
confirmatory factor analyses.
 194 T H E AERONAUTICAL JOURNAL.
These researchers recognised that workload was a multidimensional
concept and that there was an interaction between the type of task
and the nature of the workload demands imposed upon the pilot.
This basic premise formed the basis of the NASA-TLX (National
Aeronautics and Space Administration Task Load Index)
multidimensional workload scale*20). The NASA-TLX requires the
pilot to rate the amount of workload experienced on each of six sub-
scales (e.g. mental demand and temporal demand) and also to assess
the importance of each of these sub-scales to the task that they are
undertaking. Sub-scale scores are a product of the workload rating
value and its associated importance rating. The diagnostic power of
the scale is enhanced by presenting workload profiles graphically by
plotting the amount of workload experienced for each sub-scale on
the y axis and the corresponding importance of that load on the x
axis. The sensitivity of the scale is further improved (compared to a
unidimensional workload scale) by 'gearing' the workload ratings
provided by their relative importance to the task. This approach used
in the NASA-TLX formed an excellent basis for the format of the
CAHQRS.
Pilots completing the CAHQRS are required to rate the aircraft's
behaviour when performing a specified manoeuvre on each of the
five dimensions identified (longitudinal handling characteristics; lateral
handling characteristics; directional handling characteristics; trimming;
and speed control) using a ten point rating scale. The scale point
descriptors used in the CAHQRS were derived from an early version
of the Cooper-Harper handling qualities scale'8). These scale points
are described in Table 1. The operational definitions of the five
dimensions of the CAHQRS are described in Table 2. These are
presented to pilots completing the scale as an aide memoir. However,
in addition to rating the aircraft's handling qualities on each of the
five dimensions, the pilot is also required to assess (using a further
Table 1
CAHQRS scale point descriptions (adapted from Cooper and
Harper, 1996)
Scale point description
Excellent, highly desirable
Good, pleasant, well-behaved
Fair; some mildly unpleasant characteristics. Good
enough for mission without improvement.
Some minor, but annoying, deficiencies. Improvement
is requested. Effect of performance is easily compensated
for by pilot.
Moderately objectionable deficiencies. Improvement is
needed. Reasonable performance requires considerable
pilot compensation.
Very objectionable deficiencies. Major improvements are
needed. Requires best available pilot compensation to
achieve acceptable performance.
Major deficiences which require mandatory improvement
for acceptance. Controllable. Performance inadequate for
mission, or pilot compensation required for minimum
acceptable performance in mission is too high.
Controllable with difficulty. Requires considerable pilot
skill and attention to retain control and continue mission.
Marginally controllable in mission. Requires maximum
available pilot skill and attention to retain control.
Uncontrollable in mission.
Downloaded from http:/www.cambridge.org/core. Imperial College London Library, on 23 Nov 2016 at 11:31:15, subject to the Cambridge Core terms of use, available at
http:/www.cambridge.org/core/terms. http://dx.doi.org/10.1017/S0001924000028098
APRIL 2000
five point scale) how important each of these aspects is for that
particular manoeuvre or task. This scale (called the 'criticality index')
is presented in Table 3. These criticality index scores should be derived
prior to undertaking the testing and should remain constant irrespective
of aircraft configuration. These scores represent characteristics of the
flight task, not the aircraft's handling qualities.
2.3 Scoring and presentation of CAHQRS data
After gaining CAHQRS data for a particular task the aircraft's handling
characteristics profile is best described by plotting criticality index
data for a particular dimension on the x-axis and the corresponding
Table 2
Parameter definitions of the five dimensions of the CAHQRS
Parameter Definition
I Longitudinal Is the pitch control adequate; is there
characteristics always enough pitch control the desired
manoueuvre, e.g. in the landing flare or
when selecting a rotate angle? Is the aircraft
highly stable longitudinally, or is it unstable;
does the aircraft tend to return to its
trimmed angle of attack?
Is the short period pitch response (SPPR)
quick or sluggish; is the damping low or
deadbeat; is there a danger of exciting
pilot induced oscillations (PIOs)?
II Lateral Is it possible to generate sufficient roll rate/
characteristics bank angle change, e.g. to counter gusts
during take off or landing? Is the aircraft
laterally stable or unstable, e.g. in a sideslip,
does the aircraft tend to roll in the natural
sense? Is the spiral mode stable or unstable,
Rating e.g. from a banked turn, does the aircraft
tend to hold a bank angle, roll out of the
1 turn or enter a spiral dive?
2 III Directional Is it always possible to co-ordinate the
characteristics aircraft; is there enough rudder to counter
3 cross-winds or engine failure? Is the aircraft
directionally stable, or not, e.g. if the rudder
is released in a sideslip, does the aircraft
4
yaw back into wind promptly, or not? Is
the Dutch roll instrusive, or is it easy to
compensate for?
5
IV Trim characteristics Is there always sufficient trim authority, or
is it sometimes necessary to hold a control
force because you have run out of trim? Is
the rate of trim adequate, or is it too fast or
6
too slow?
V Speed control Is it possible to quickly and accurately
change speed using the engine and/or
7
speed brakes?
Table 3
CAHQRS criticality index
8 Description Rating
Entirely unrelated to task performance 1
9 Moderately related to task performance 2
Related to task performance 3
Important for successful task performance 4
10 Critical for task performance 5
 HARRIS ET AL. THE CRANFIEJ D AIRCRAFT HANDLING QUALITIES RATING SCALE: A MULTIDIMENSIONAL APPROACH...
handling quality rating on the y-axis (cf. the NASA-TLX). By
presenting data this way a simple 'rule of thumb' for interpreting
CAHQRS data rapidly emerges: if the histogram box is wide (i.e. a
high score on the criticality index representing a component of the
aircraft's handling qualities vital for the task) the histogram bar
should be short (i.e. a good handling qualities rating). A hypothetical
CAHQRS profile is presented in Fig. 3.
When interpreting any multidimensional scale scores, emphasis
should be placed on interpreting the individual components, not on
the overall composite scale score. An overall handling qualities score
can be calculated, however. This technique is described later. For the
hypothetical task described in Fig. 3 it can be seen that the handling
qualities dimensions relating to the aircraft's longitudinal, lateral and
speed control were all rated to be 'critical for task performance.'
Directional control gained a moderate rating but trimming was
regarded as not being at all important for task performance. When
interpreting CAHQRS profiles, the critical aspect of the scale to bear
in mind is the interaction of the handling quality rating awarded with
the requirements of the task. The key to interpretation lies in
interpreting the areas under each aspect of the histogram, not the
height. From Fig. 3 it can be seen that the trimming of this hypothetical
aircraft was regarded as being extremely poor (it was awarded a rating
of seven 'major deficiencies which require mandatory improvement
for acceptance controllable') but trimming was not an important
aspect of that particular flight task. However, directional and speed
control characteristics were both regarded as being critical to the task
and were awarded handling qualities ratings of two and three,
respectively. Because of the relative importance of these components
of the aircraft's handling, it can be seen that a one-point reduction in
the handling qualities rating for either (or both) of these aspects
would have a much larger and more beneficial impact than a three or
four point reduction in the rating for trimming (but it needs to be
emphasised, only for that particular task).
Although the power of any multidimensional scale lies in the
interpretation of its individual components it is possible to calculate
an overall handling qualities score from the CAHQRS. This is done
by adding the cross products of each sub-scale's handling qualities
rating and criticality index scores and dividing this number by the
sum of the criticality index scores. In common with the Cooper-
Harper scale, low scores represent desirable handling qualities. In
the hypothetical example given in Fig. 3, the overall CAHQRS rating
is approximately 2-5. This number, however, should be treated with
some caution. An overall CAHQRS score of 2-5 in Cooper-Harper
terms would suggest that the aircraft was rated as being 'fair; with
HANDLING QUALITY
RATING
5 immediately after each trial, along with subjective comments on the
Longitudinal Lateral
Characteristics Charactenstics
CRITICALITY INDEX SCORE
Figure 3. Hypothetical handling qualities profile.
Downloaded from http:/www.cambridge.org/core. Imperial College London Library, on 23 Nov 2016 at 11:31:15, subject to the Cambridge Core terms of use, available at
http:/www.cambridge.org/core/terms. http://dx.doi.org/10.1017/S0001924000028098
195
some mildly unpleasant characteristics. Good enough for mission
without improvement.' However, close inspection of the sub-scales
would suggest that certain aspects of the aircraft's handling (its lateral
handling characteristics and trimming) were considerably worse than
this. Although trimming was rated as being worse, as alluded to earlier,
given the critical nature of lateral control characteristics to the
hypothetical task described it would be suggested that this is where
initial efforts to improve the aircraft's behaviour should be placed.
Furthermore, as for the Cooper-Harper scale, data from the
CAHQRS should only be interpreted with reference to the Test Pilot's
comments. The CAHQRS is not intended to be a replacement for the
Test Pilot's qualitative comments, it is merely an adjunct that allows
quantification of certain aspects of the aircraft's behaviour.
3.0 INITIAL TRIALS USING THE CAHQRS
For any scale to be useful it needs to be valid. In this context, the
CAHQRS should be able to distinguish between different control
laws that should manifest themselves as different aircraft behaviours
to the test pilot. Furthermore, the differences in the scale scores given
should reflect the engineering and physical properties of the flight
control system or aircraft.
A series of simulated flights was undertaken in an engineering
flight simulator to evaluate several fly-by-wire flight control laws'21-23'.
Two flight scenarios were utilised: an in-flight refuelling task and an
instrument landing system (ILS) approach and landing task in the
presence of a severe horizontal wind gradient. For both tasks,
longitudinal, lateral and directional dynamics were excited. Both
tasks required tight flight path control.
The in-flight refuelling task was purely a tracking task. None of
the physical effects of the tanker (e.g. its slipstream) was modelled.
A series of 11 lights which illuminated sequentially, each relating to
an in-trail distance, was used to determine when the evaluation aircraft
was in the desired position with respect to the tanker. The guidance
lights were designed to give the evaluation pilot both positional and
rate cues. It should be noted that as the slipstream of the tanker aircraft
was not modelled its stabilising effect which usually helps the receiver
aircraft maintain its lateral position, was absent. As a result, this task
required a great deal of control activity using all the primary flight
controls in order to maintain position.
For the windshear approach task, the pilot was briefed to fly a
standard ILS approach, which required tight flight path control along
a predetermined trajectory. This task required precise control in all
three axes. A 30kt headwind was used above 300ft, which reduced
linearly from 30kt to Okt between 300 and 50ft above ground level.
This horizontal wind gradient resulted in an almost instantaneous
loss of airspeed during the approach, which required the pilot to
compensate by increasing the power.
The aircraft simulated was a generic regional transport aircraft
with a low-wing, two under-wing engines and a seating capacity of
approximately 100. A number of different FBW control laws were
evaluated. These were representative of actual control laws used in
current transport aircraft, although most had slightly different
properties to those of a classical aircraft. An aircraft flight model
with conventional dynamics was used as a baseline. The classical
aircraft model was familiar to all of the evaluation pilots. The
evaluation pilots had also been exposed to the other FBW flight control
configurations tested. The pilots were also blind to the configuration
of the flight control system prior to its assessment.
CAHQRS criticality index ratings were obtained for both tasks
prior to commencing the trials. These remained constant. CAHQRS
and Cooper-Harper ratings were obtained from the evaluation pilot
aircraft's behaviour.
Data were obtained over 37 simulated sorties using 15 control law
variations. During the course of the trials six aircraft configurations
(in both scenarios) were repeated (unbeknown to the evaluation pilot)
 196 T H E AERONAUTICAL JOURNAL APRIL 2000

allowing some estimate to be made of the test/re-test reliability of
both the CAHQRS and the Cooper-Harper scale.
4.0 RESULTS
It is difficult to establish the criterion validity of any aircraft handling
qualities scale as it is almost impossible to establish a meaningful,
measurable engineering-based criterion variable that truly reflects an
aircraft's behaviour*4- 5>. However, it is established psychometric
practice in such circumstances to evaluate any new instrument
against an existing instrument that has demonstrated some degree of
validity. At a global level, the Cooper-Harper has demonstrated that
it can distinguish between aircraft with desirable and undesirable
handling characteristics, thus the CAHQRS should show a high
positive correlation with it. However, the two scales should not
show an extremely high correlation otherwise it could be concluded
that the new scale measures exactly the same components as the old
scale.
Over the 37 sorties, the overall CAHQRS score did show a
moderately high positive correlation with the corresponding Cooper-
Harper ratings (r = 0-58, p < 0001). While this figure would initially
appear to suggest that there was high agreement between the
CAHQRS and the Cooper-Harper scale it should be borne in mind
that a correlation of 0-58 represents only just over 33% of the variance
accounted for in Cooper-Harper ratings by the CAHQRS thereby
leaving almost 67% of the variance to be accounted for by other
factors. Thus, it can be concluded that this correlation between the
scales was not so high as to suggest that the new scale reflected exactly
the same measures of aircraft handling qualities as the previous scale
(q.v. Ref. 12).
What was of greater interest, however, was that when the ratings
(alone) from each of the five CAHQRS dimensions were entered
into a multiple regression as predictor variables against the criterion
of the Cooper-Harper rating, a highly significant solution was obtained
(R = 082; R2 = 0-67; R2{adj) = 0-62; p < 0001). The full multiple
regression solution is given in Table 4. On closer inspection of the
multiple regression results, though, it can be seen that only two of
the CAHQRS dimensions (longitudinal handling characteristics and
speed control) were significant predictors of the Cooper-Harper
scores obtained. Longitudinal control handling characteristics
showed a large, positive correlation (r = 0-78) with the Cooper-
Harper ratings, as did the speed control dimension of the CAHQRS
(r=0-69).
In effect, the Cooper-Harper ratings do not reflect the lateral,
directional or trim aspects of the aircraft's handling characteristics,
even though (as will be shown) all these dimensions obtained high
criticality ratings for both types of sortie flown. These results would
indicate that the Cooper-Harper scores only reflect a small part of
the domain of aircraft handling qualities demonstrating that the scale
lacks content validity.
Figures 4 and 5 illustrate example data from the experimental sorties
undertaken in the engineering flight simulator. In general, it was
observed that the aircraft flight control law that provided the most
desirable handling characteristics was that of a normal acceleration
law. This law inherently maintained flight path with no pilot input
and the response characteristics of the normal acceleration law used
for these evaluations were found to be very desirable, especially as
the pilot could use either a tight, closed loop control strategy or open
loop control strategy. Other control laws tested were found to be not
quite as suitable for the tasks described here primarily due to their
less desirable flight path response characteristics. The data for the
normal acceleration control law is illustrated against the baseline
aircraft, which exhibited the 'classical' handling characteristics
found in an unaugmented aircraft.
For the in-flight refuelling task illustrated in Fig. 4, the unaugmented
aircraft was awarded a Cooper-Harper rating of 4. Its 'fly-by-wire'
counterpart with a normal acceleration flight control law (with a
50ms delay and fast dynamics) was awarded a Cooper-Harper rating
of 2. The corresponding overall CAHQRS ratings were 4-18 and
3-35, respectively.
On inspection of the CAHQRS profiles it can be seen that the
augmented aircraft makes considerable gains over its classical
counterpart in the longitudinal and speed control dimensions (both

Table 4
Pearson correlation matrix and multiple regression solution with five CAHQRS dimension ratings as predictor variables and Cooper-Harper
scores as the criterion
Cooper-Harper Longitudinal Lateral Directional Trim
rating characteristics characteristics characteristics characteristics

Longitudinal 0-784**
characteristics
Lateral 0-232 0-264
characteristics
Directional 0109 0121 0-908**
characteristics
Trim 0118 0197 -0-348* -0-603*
characteristics
Speed 0-688** 0-646** 0-407** 0-261 -0-044
control

*p < 005 *p <001

Predictor B Standard error of B P (-value Sig

Longitudinal 0-46 0-11 0-58 0-78 409 < 0-001

N
Lateral -008 0-28 -0-10 0-23 -0-29 /s
Directional 0-34 0-28 005 011 012 N/S
Trim 002 0-23 001 012 008
Speed 0-32 0-14 0-34 069 2-27 <005
Constant 0-96 0-62 1-55

F = 12-86; p< 0001

Downloaded from http:/www.cambridge.org/core. Imperial College London Library, on 23 Nov 2016 at 11:31:15, subject to the Cambridge Core terms of use, available at
http:/www.cambridge.org/core/terms. http://dx.doi.org/10.1017/S0001924000028098
 HARRIS ET AL T H E CRANFIELD AIRCRAFT HANDLING QUALITIES RATING SCALE: A MULTIDIMENSIONAL APPROACH... 197

HANDLING QUALITY utilising a multidimensional scale to assess aircraft handling qualities.

RATING LEGEND
With reference to the CAHQRS profiles for the approach and
landing task shown in Fig. 5, it can be seen how different aspects of
Classical Aircraft an aircraft's handling qualities are important in different phases of
flight. Longitudinal and trimming aspects of the aircraft's handling
Fly-by-Wire (Normal Acceleration)
6 are now more important for the task in hand compared to the previous
task. Again, it can be seen that the improvements in the augmented
aircraft's handling are all associated with longitudinal, speed control
and trimming aspects of the flight task.
One of the major criticisms of the Cooper-Harper scale has been
4- its lack of reliability. During the course of the handling qualities trials
several configurations were repeated allowing an assessment of the
test/re-test reliability of the individual sub scales comprising the
CAHQRS. As the criticality ratings should remain constant for a given
task or manoeuvre, these were not subject to an assessment of their
test/re-test reliability. Using Pearson's correlations, the test/re-test
reliability for the longitudinal handling characteristics was 0-68; for
lateral handling characteristics it was 098; for directional
characteristics it was 0-63; the trimming sub-scale showed a test/re-test
correlation of 0-54; and finally the speed control dimension produced
a coefficient of 0-93. Overall, the test/re-test correlations observed
for the composite, overall CAHQRS scores were exceptionally high
Longitudinal Lateral Directional Trim Speed (r = 0-95). In comparison, the test/re-test correlation coefficient for
Characteristics Characteristics Characteristics Control the Cooper-Harper scale was only 0-23. All of these correlation
CRITICALITY INDEX SCORE coefficients should be treated with some caution, however, due to
the small test-re-test sample size. However, it can be suggested that
Figure 4. CAHQRS profiles for classical aircraft and normal acceleration all the CAHQRS sub-scale show much greater reliability than the
(fly-by-wire) flight control law for the in-flight refueling task. overall Cooper-Harper scores.
By using a simple, unidimensional 1-10 format, the Cooper-Harper
scale requires the rater to form mentally a composite score based on
HANDLING QUALITY
RATING
all the individual aspects of the aircraft's handling and then weight
LEGEND these components, relative to one another, with regard to the
Classical Aircraft requirements of the task being flown, before finally settling on a
Fly-by-Wire (Normal Acceleration)
handling qualities rating. The rater is required to make many implicit
6 judgements prior to making their final rating, hence there is scope
for variability in many aspects of the process that can contribute to
the lack of test/re-test reliability. The CAHQRS makes this process
more explicit. Weightings of each the components of aircraft handling,
in the form of the criticality index, are made explicitly prior to
performing the task. The pilot is then required to make explicit
evaluations of five well-defined aspects of the aircraft's behaviour,
irrespective of how important they are for the manoeuvre being
undertaken. Greater reliability is obtained as a result of making
more than a single rating of the aircraft's behaviour. The manner in
which the various aspects of an aircraft's handling behaviour are
1- combined to produce a rating is now through a mechanical and explicit
process. When using the Cooper-Harper scale the manner in which
the various components of the way the aircraft's behaviour was
4 4 3 4 combined was via subjective judgement and weighting to produce
Longitudinal Lateral Directional Trim Speed
Characteristics Characteristics Characteristics Control the final single score which reflected a composite of several parameters.
CRITICALITY INDEX SCORE

Figure 5. CAHQRS profiles for classical aircraft and normal acceleration
(fly-by-wire) flight control law for the approach and landing task.
of which were awarded criticality ratings of 4), and also on trimming
(although the later aspect was regarded as being entirely unrelated to
task performance). Further inspection of Fig. 4, though, indicates
that the augmented aircraft makes no gains over the baseline control
configuration in the areas of lateral handling characteristics and
actually exhibits worse directional handling qualities. It must be noted
that both these aspects of aircraft handling were also regarded as being
important for successful task performance, both being awarded a
criticality index rating of 4. Given the lack of diagnostic power inherent
in Cooper-Harper ratings, it is not too surprising that a superior score
was awarded for the augmented aircraft even though this latter aspect
of its handling was slightly worse. This illustrates the benefits of
Downloaded from http:/www.cambridge.org/core. Imperial College London Library, on 23 Nov 2016 at 11:31:15, subject to the Cambridge Core terms of use, available at
http:/www.cambridge.org/core/terms. http://dx.doi.org/10.1017/S0001924000028098
5.0 CONCLUSIONS
Given the control laws implemented in the augmented aircraft the
improvements observed in the longitudinal handling qualities, speed
control and trimming were all to be expected. However, this also
serves to illustrate that the CAHQRS is sensitive to changes in the
flight control systems in the way that would be expected, (i.e. it
shows predictive validity). Forming the sub-scales comprising the
CAHQRS from the dimensions of handling qualities found within
the airworthiness regulations also ensures that it exhibits content
validity. The dimensions derived were the product of the opinions of
141 fully qualified test pilots flying 75 different aircraft types which
ensures that the scale is generally applicable across all fixed-wing
aircraft types.
The CAHQRS was designed from the outset for ease of completion.
One of the common criticisms of multidimensional scales is that
 198 T H E AERONAUTICAL JOURNAL APRIL 2 0 0 0

they can be cumbersome to complete, especially if they need to be 21. GAUTREY, J.E. Flying Qualities and Flight Control System Design for a
completed in-flight. Although the CAHQRS requires five ratings to Fly-By-Wire Transport Aircraft, Engineering Doctorate Thesis, Cranfield
be made by the pilot after each manoeuvre, as opposed to just the University, UK, 1998.
single rating required by the Cooper-Harper scale, the problems of 22. GAUTREY J. Generic regional aircraft flying qualities for the windshear
and formation flying tasks, College of Aeronautics report no 9710,
using a multidimensional scale are partially obviated by determining
Cranfield University, UK, 1998.
the criticality ratings for each flight task prior to flight. The
23. GAUTREY J. Generic regional aircraft flying qualities for the approach
CAHQRS also builds on two established scales (the NASA-TLX and and landing task, College of Aeronautics report no 9701, Cranfield
the Cooper-Harper handling qualities scale), with which most test pilots University, UK, 1997.
will also be familiar.
Further validation work still needs to be undertaken involving a
greater range of manoeuvres using a wider range of aircraft types.
The development of a database of this kind is essential for further
validation work. Further work also needs to be undertaken to establish
the inter-rater and within-rater reliabilities for the scale. Nevertheless,
this initial validation study strongly suggests that the CAHQRS
shows great promise and has many benefits over unidimensional
scales of aircraft handling qualities. It seems to be both highly reliable
and valid with enhanced diagnostic properties when compared with
the Cooper-Harper scale.

REFERENCES
1. US DEPARTMENT OF DEFENSE Flying qualities of piloted airplanes (MIL-
F-8785C), Washington DC, 1980.
2. UK CIVIL AVIATION AUTHORITY JAR 25 Large aeroplanes, London,
1978.
3. US DEPARTMENT OF TRANSPORTATION Federal Aviation Regulations
Part 25 Airworthiness standards, Washington, DC, 1974.
4. HARPER, R.P. and COOPER, G.E. Handing qualities and pilot evaluation
JGuide, ContDyn, 1986, 9, pp 515-530.
5. GIBSON, J.C. The definition, understanding and design of aircraft handling
qualities, report LUT LR-756, Delft University of Technology, The
Netherlands, 1995.
6. COOPER, G.E. and HARPER, R.P. The use of pilot rating in the evaluation
of aircraft handling qualities, NASA TN-D-5153, 1969.
7. COOPER, G. Understanding and interpreting pilot opinion. Aeronaut Eng
Rev, 1957,16, p 47.
8. HARPER, R.P. and COOPER, G. A revised pilot rating scale for the
evaluation of handling qualities. Paper presented at AGARD specialists
meeting on stability and control, Cambridge, UK, 20-23 September,
1966.
9. MORAY, N., JOHANSEN, J., PEW, R.W., RASMUSSEN, J., SAUNDERS, A.F.
and WICKENS, CD. Mental Workload; Its Theory and Measurement,
Plenum; New York, 1979.
10. HILL, S.G., AVECCHIA, H.P., BYERS, J.C, BITTNER, A.C, ZAKLAD, R.P.
and CHRIST, R.E. Comparison of four subjective workload rating scales,
Hum Fact, 1992, 34, pp 429-439.
11. HARPER, R.P. The role of pilot rating in the development of handling
criteria, paper number CP-106, AGARD flight mechanics specialist
meeting, Ottawa, Canada, 28 September-1 October, 1971.
12. ANASTASI, A. Psychological Testing, 6th edition, MacMillan Publishing,
New York, 1990.
13. PAYNE, K.H. and HARRIS, D. Psychometric approach to the development
of a multidimensional scale to assess aircraft handling qualities,
manuscript submitted to the IntJ Aviat Psych, 1998.
14. WILSON, D.J. and RILEY, D.R. Cooper-Harper rating variability, AIAA
atmospheric flight mechanics conference paper A1AA-89 3358. AIAA,
1989.
15. FIELD, E.J. Flying Qualities of Transport Aircraft: Precognitive or
Compensatory? PhD thesis, College of Aeronautics, Cranfield University,
UK, 1995.
16. UK CIVIL AVIATION AUTHORITY British Civil Airworthiness Requirements
Section D large aeroplanes, London, 1981.
17. JORESKOG, K.G. and SORBOM, D. LISREL 7 user's reference guide.
Scientific Software, Mooresville, USA, 1988.
18. HAIR, J.F., ANDERSON, R.E., TATHAM, R.L. and BLACK, W.C. Multivariate
Data Analysis, 5th edition, Prentice Hall, New Jersey, 1998.
19. HARRIS, D., PAYNE, K and GAUTREY, J. A multidimensional scale to assess
aircraft handling qualities, Engineering Psychology and Cognitive
Ergonomics (Volume 3), Ashgate, Aldershot, UK, 1999, pp 277-285.
20. HART, S.G. and STAVELAND, L.E. Development of the NASA task load
index (TLX): results of empirical and theoretical research, Human Mental
Workload. North-Holland, Amsterdam, 1988, pp 139-183.
Downloaded from http:/www.cambridge.org/core. Imperial College London Library, on 23 Nov 2016 at 11:31:15, subject to the Cambridge Core terms of use, available at
http:/www.cambridge.org/core/terms. http://dx.doi.org/10.1017/S0001924000028098