19th ITS World Congress, Vienna, Austria, 22/26 October 2012 EU-00746

Qualitative and quantitative driver evaluation: a comparison.

B. Berghmans 1, A. Kerkhof 1,*
1. Key Driving Competences, Interleuvenlaan 62, 3001 Heverlee, Belgium.
* bram.kerkhof@keydriving.com, tel.: +32 (0) 16 39 78 30.

Abstract
A comparison is made between two methodologies for automated driver evaluation; quantitative and
qualitative scoring. It is shown that the quantitative method generates an inaccurate scoring due to the
sensitivity on external conditions. A well-constructed qualitative scoring algorithm can produce a
legitimate evaluation of the driver, regardless of the environment in which he has to perform his job.
Keywords: driver evaluation, qualitative scoring

Introduction

Several methods have been developed to evaluate a drivers performance in achieving optimal fuel
economy. Basically, the current methods of permanent evaluation can be categorized as either
quantitative or qualitative.

The quantitative method

The quantitative method is the most common method of evaluating the driving style. It consists of
identifying performance indicators based on readily available information. A common performance
indicator for eco-driving is the number of times the brake is applied per unit of distance travelled.

Even though a good driving style will generally show in these performance indicators, there is too much
external interference to make them useful for permanent monitoring. For instance, the number of times the
brake is applied is highly dependent on type of road (highway or inner city) and how busy the traffic is. A
driver should not receive a bad scoring for a justified action. In general, a quantitative score needs
interpretation in function of additional information about the circumstances in which the source data was
obtained.

Using such quantitative information directly to evaluate the driving performance is only valid when all the
external circumstances (route, traffic, weather ) are controlled, and reference scores for good driving
are known. Otherwise it is almost impossible to ascertain whether the obtained quantitative numbers
represent a good or bad driving style.

The qualitative method

A different approach to evaluate the driving style is trying to put into context the information that is
gathered. As in the example above, we can focus on the times that the driver applies the brake. However
instead of merely counting the number of brake events, we can evaluate each individual brake event.
Depending on the circumstances, the evaluation can be positive or negative. If the external conditions (for
instance more dense traffic) dictate that the driver has to brake more often, applying the brake will not
necessarily be negative for his score.

The scoring method that has been developed by Key Driving Competences (KDC) evaluates whether the
driver has made the correct decision for a number of actions he undertakes during a trip. A set of criteria for
evaluating a particular aspect of driving is referred to as a scoring rule. In order to obtain evaluation of the
driving style the output of several scoring rules are averaged.

Methodology
Qualitative and quantitative driver evaluation: a comparison.

Data acquisition

We have gathered data from 38 drivers using Transics TX-Max onboard computers that are connected to
the CAN bus of the vehicles. The computers are wirelessly connected to the networks of Transics and
KDC, and transmit data at the end of every trip. The drivers were selected by several criteria.

1. All drivers perform the same type of work. Specifically, this means that all the trucks have the
same average load, and the traffic conditions are comparable. This criterion facilitates the
evaluation of fuel consumption data. By constraining the external conditions as much as possible, a
change in fuel consumption from one situation to another is more closely correlated to the
performance of the driver.

2. Drivers with different skill levels are selected. By selecting experienced drivers who have been
trained in eco-driving together with inexperienced drivers, we aim to introduce a significant
amount of variation in driving style to show the correlation with fuel economy.

Quantitative scoring
For every trip made by these drivers, trip statistics were gathered and grouped on a weekly basis in post-
processing.. Subsequently we constructed quantitative performance indicators such as those commonly
found on commercial driving style monitoring systems.

Qualitative scoring
For the qualitative scoring we have used the eco-score that has been developed by KDC and is
commercially available on Transics TX-Max systems. The scores are calculated on a weekly basis for each
driver. The KDC/Transics score consists of a global score, and a number of partial scores that evaluate
certain specific aspects of driving. An overview is listed in Table 1.

Table 1: Qualitative scores on the KDC/Transics system

Global A global evaluation of the driving style.

Anticipation A measure of the anticipative behavior of the driver. Evaluates the
succession of acceleration/coasting/braking.
Gearing Scores how well the driver uses the gearbox.
Acceleration Gives a measure of the smoothness of the driving speed. A steady
speed or even acceleration and deceleration will result in a high
acceleration score.

All scores, quantitative and qualitative, are evaluated on their correlation with the average fuel
consumption.

Results

Partial results are shown in fig. 1-2. Four drivers were selected that portray different skill levels with
respect to ecological driving. The drivers are indicated by their identification number. For the quantitative
scores (fig. 1) two points should be remarked.

2
Qualitative and quantitative driver evaluation: a comparison.

Figure 1 Selection of quantitative performance indicators.

Figure 2 The Global score as a function of fuel consumption.

1. For the individual drivers there is no correlation between the fuel consumption and the quantitative
metrics. Trip data with a high percentage of coasting, or a low average RPM (this generally
considered to be good driving) are not strictly related to low levels of consumption.

3
Qualitative and quantitative driver evaluation: a comparison.

2. Although average fuel consumption exhibits a wide range, such range is not found in the
quantitative metrics.

For the qualitative score (fig. 2) there is a strong correlation with average fuel consumption, and it is clear
that the driver with the best fuel consumption is scored the highest. It should also be noted that the
individual drivers tend to hold the level of scoring every week, indicative of their eco-driving skills.

Conclusion

We have found that quantitative scoring is an ineffective method of evaluating how well a driver is
achieving the best fuel economy possible for the type work he has to perform.

Even when the external circumstances, such as average vehicle load or traffic conditions, are constrained
there is a very poor correlation between the quantitative scoring metrics and the average fuel consumption.
Also, the week-to-week variation of the quantitative scores portrays a large variation for the individual
drivers. If one would use only these metrics, it would be very difficult to identify the better drivers.

In general, quantitative scores measure the frequency of certain actions, but lack information on the
justifiability of these actions. One needs very detailed additional information about the trips in order to
evaluate whether a certain statistic relates to good or poor driving.

Qualitative scoring adds this information in the evaluation. It does not focus purely on the value of a certain
metric, but rather on the context in which that metric was obtained.