ARTICLE IN PRESS

Reliability Engineering and System Safety 93 (2008) 940–949

www.elsevier.com/locate/ress

Modelling the reliability of search and rescue

operations with Bayesian Belief Networks
Lisa Norrington, John Quigley, Ashley Russell, Robert Van der Meer
Department of Management Science, University of Strathclyde, Glasgow G1 1QE, Scotland
Available online 14 March 2007

Abstract

This paper uses a Bayesian Belief Networks (BBN) methodology to model the reliability of Search And Rescue (SAR) operations
within UK Coastguard (Maritime Rescue) coordination centres. This is an extension of earlier work, which investigated the rationale of
the government’s decision to close a number of coordination centres. The previous study made use of secondary data sources and
employed a binary logistic regression methodology to support the analysis. This study focused on the collection of primary data through
a structured elicitation process, which resulted in the construction of a BBN. The main findings of the study are that statistical analysis of
secondary data can be used to complement BBNs. The former provided a more objective assessment of associations between variables,
but was restricted in the level of detail that could be explicitly expressed within the model due to a lack of available data. The latter
method provided a much more detailed model, but the validity of the numeric assessments was more questionable. Each method can be
used to inform and defend the development of the other. The paper describes in detail the elicitation process employed to construct the
BBN and reflects on the potential for bias.
r 2007 Elsevier Ltd. All rights reserved.

Keywords: Bayesian Belief Networks; Statistical inference; Elicitation; Expert judgement; Reliability modelling; Risk assessment

1. Introduction study highlighted the importance of explanatory variables

The UK government is concerned with improving the
operational performance of the Maritime and Coastguard
Agency (MCA). Since 1994, there have been a number of
reviews of the effectiveness of Search And Rescue (SAR)
operations coordinated by the MCA, culminating in the
decision in 1999 to close a number of the Maritime Rescue
coordination centres. However, there appears to have been
a distinct lack of formal analysis to justify this decision.
The aim of this research is to conduct a formal analysis
of the Coastguard SAR service in order to identify the key
factors that influence its effectiveness and to provide a way
of measuring this influence. An earlier study had been
conducted with this aim, which made use of publicly
available secondary data (in the form of annual Incident
Statistics) and developed a binary logistic regression model
to support the assessment [1]. The findings of this previous
Corresponding author. Tel.: +44 141 548 3152; fax: +44 141 552 6686.
E-mail address: j.quigley@strath.ac.uk (J. Quigley).
such as the size or scale of incidents, staff workload and the
length of coastline monitored by each coordination centre.
Such variables may be interpreted as providing rather
crude composite indicators for the multitude of factors that
determine the effectiveness of this kind of complex
operational system. The contribution of the present study
is to support a more detailed explanation of the relation-
ship between the reliability of SAR operations and
identifiable causal factors.
In Section 2, we present a summary of background
research. First, we present a summary of recent events
relating to the UK Maritime and Coastguard Agency that
provided the motivation for this research; and second, we
present a summary of relevant research literature on
maritime risk. In Section 3, we present a summary of our
analysis of the available secondary data. This analysis is
conducted at a crude aggregate level, but highlights some
significant characteristics associated with better-perform-
ing centres. In Section 4, we describe the Bayesian Belief
Network (BBN) that we obtained and reflect on the process

0951-8320/$ - see front matter r 2007 Elsevier Ltd. All rights reserved.
doi:10.1016/j.ress.2007.03.006
 ARTICLE IN PRESS
L. Norrington et al. / Reliability Engineering and System Safety 93 (2008) 940–949
of eliciting subjective probabilities. In Section 5, we
conclude by reflecting on the synergies between statistical
modelling and BBNs, whereby the former supports the
validation of the latter and the latter provides a causal
description of the associations identified in the former.
2. Background
Within this section, we provide a brief overview of the
UK Maritime and Coastguard Agency as related to this
research project; and in Section 2.2 we provide a brief
summary of research conducted on maritime risk issues.
2.1. UK maritime rescue
The coordination centres operated by the UK Coast-
guard do not themselves undertake rescues. Instead, SAR
resources are strategically positioned around the UK to
meet specified response times; the personnel using the
equipment (lifeboats and coast rescue teams) tend to be
local volunteers. There are currently nearly 400 Coastguard
Rescue Teams (CRTs), 200 lifeboats and SAR helicopters
around the UK. However, the staff at the coordination
centres play the vital role of receiving alerts, communicat-
ing with the casualty, assessing the scale of the distress,
selecting the appropriate SAR response, and activating and
coordinating that response by alerting the relevant
personnel and directing them to the casualty. They
continue to fulfil this function until the emergency is over.
In the mid-1990s, there were 21 UK Coastguard districts,
each with its own coordination centre. From 1994
onwards, there were a number of reviews of, and reports
on, the Coastguard service concerning the organisation of
these districts. From 1994 until 1996, the MCA conducted
the ‘Focus for Change’ review. One of its key recommenda-
tions was to introduce a new grade of Coastguard Watch
Assistant to assist (and, in certain cases, to substitute for)
the Watch Keeping Officers working in the coordination
centres. This recommendation was implemented in
1996–1997.
In 1997, the MCA set out their view of the future in the
‘Five Year Strategy’ consultation document. This docu-
ment contained two key proposals: (1) to invest in a new
‘Integrated Coastguard Communications System’ (ICCS),
and (2) to reduce the number of co-ordination centres from
21 to 17. One of the main justifications given for the
proposed closure programme was to provide greater
exposure of Watch Keeping personnel to a wider spectrum
of incidents (in order to produce more experienced staff
through regular exposure to SAR activity and to provide
greater job satisfaction).
In 1998, a report from the Committee of Public
Accounts (of the UK Parliament) expressed concern that
without remedial action, ‘local knowledge’ would be lost if
closures of coordination centres went ahead. In response to
these serious concerns, the UK government asked Lord
Donaldson to conduct an overall ‘Review of the Five Year
941
Strategy for HM Coastguard’ [2]. His report was published
in August 1999. In this review, Lord Donaldson accepted
the case for closing three coordination centres but rejected
the case for the fourth. These recommendations were based
on striking a balance between staff workload and the local
knowledge required by staff to deliver an effective service.
Using the probability of a life being saved as the
dependent variable, Van der Meer et al. [1] developed a
binary logistic regression model to explain the differences
in performance between the 21 different coordination
centres. The average size of incidents, staff workload and
the length of coastline monitored (as a possible proxy for
the local knowledge required) were found to be significant
explanatory variables. This study found that the regression
model was more complex and realistic than any formal
analytical model likely to have been used by the UK
government, and also that the coordination centres selected
for closure were not necessarily the ones that were least
effective in their primary purpose (i.e., to save lives).
However, such regression studies, based on the use of
aggregate statistics, can only be of limited use, as—at
best—they provide rather crude composite indicators for
the multitude of factors that are actually involved.
This consideration provided the motivation to start
conducting in-depth interviews with Watch Keeping
personnel in order to construct a more detailed model of
the factors that influence the effectiveness of operations. A
BBN modelling approach was chosen as it would facilitate
the integration of information from various sources, such
as government reports and interviews with Watch staff.
2.2. Summary of literature on maritime risk research
There have been a number of high-profile maritime
incidents over the years, everything from the famous
Titanic disaster in 1912 through major oil spillages such as
the Sea Empress. The Empress, as reported by the BBC [3],
ran aground in February 1996 off the coast of Wales,
resulting in 72,500 tonnes of oil spilling into the sea and the
pollution of 200 miles of coastline. Such high-profile
incidents tend to raise public awareness of maritime safety
and have led to significant investigations into its improve-
ment.
Maritime risk assessment has become an essential area of
research and has been used to assess factors that lead to a
collision or grounding. An example is the Prince William
Sound Risk Assessment [4], which was undertaken follow-
ing the oil spillage from the Exxon Valdez in 1989. This
devastating environmental disaster led to a group of
relevant stakeholders (shipping companies, locals, envir-
onmentalists, etc.) forming a steering group to try and
assess the risks to oil transportation in Prince William
Sound, and to develop risk reduction methods that could
lead to a risk management plan. The assigned consultant
team utilised system simulation with probabilistic risk
assessment and expert judgment to answer these requests.
 ARTICLE IN PRESS
942 L. Norrington et al. / Reliability Engineering and System Safety 93 (2008) 940–949
Many other risk assessments have been undertaken
relating to tankers carrying cargo that poses a risk to the
environment. One such study was carried out as far back as
1979 by Ligthart [5] to assess the risks faced by the
Netherlands in introducing LNG carriers for the transpor-
tation of natural gas. As technology has advanced, more
sophisticated tools for carrying out analysis have been
developed. Software that is capable of assessing the risk of
collision, the level of damage and consequences has been
developed (Friis-Hansen and Simonsen [6]). This kind of
software provides an opportunity to mitigate risk in the
design phase of a ship. A similar example is the
development of SIMCOL, which is a simplified collision
model to assess the damage caused to a set of key variables
due to a collision [7]. A web-based solution to risk analysis
and routing in oil transportation is described in Iakovou
[8]. Through the development of an optimisation model to
minimise risks and the costs of transportation along key
shipping routes, a balance between the level of acceptable
risk and costs can be established by the end user, and a
series of ‘what if’ scenarios can be run. The user interface is
through the Interactive Oil Transportation Model (IOTS)
available over the Internet.
Once the risk of collision or grounding has been
identified, many researchers have tried to develop a means
to assess whether policies to mitigate risk will be effective.
For example, in an analysis of the effectiveness of the
vehicle traffic service (VTS) system in the Netherlands, ‘‘the
asp [accident sequence precursor] method models incidents
with the help of generic event trees. Each path consists of a
subset of precursors relevant to the determination of safety
systems’ effectiveness.’’ [9].
Also in the Netherlands, the Policy for Sea Shipping
Safety (POLSSS) was undertaken on behalf of the Dutch
Ministry for Transport in order to evaluate vessel traffic
management in Dutch waters. According to Van Urk and
de Vries [10], the study was intended to evaluate whether
the present level of risk was acceptable. This research
consisted of two surveys of stakeholders to assess the
perception of acceptability of the current risk from
shipping and the appropriateness of possible changes in
policy known as tactics. Full details of the surveys are
found in Pöyhönen [11]. In addition, a cost-effectiveness
analysis of possible policies was performed in order to
assess how effective they are from the perspective of
reducing the risk of an incident relative to the cost of
implementation. This cost-effectiveness analysis is de-
scribed by Walker [12]. A study into the use of both radar
and infrared technologies to develop a collision avoidance
system utilises the methodologies of collision risk with
improvements in technology in order to make the tracking
of possible dangerous situations more effective [13].
Research into the risk of collisions has influenced the
design of both ships and ports/harbours. Examples of
research relating to the design of ports and harbours can be
found in Trbojevic and Carr [14], where a qualitative
methodology of using ‘‘bow tie’’ diagrams is described,
with the left and right hand side of the diagrams
corresponding to fault and event trees. A risk matrix can
then be developed that can determine acceptable levels of
risk for different scenarios. Simulation techniques have
been widely applied to simulate the traffic flows in and out
of ports [15,16].
Much of the research outlined above has been influenced
by major maritime incidents and changes in governmental
policies. One such influence has been the adoption by the
International Maritime Organisation (IMO) of formal
safety assessment, which, according to Wang [17], ‘‘in-
volves using the techniques of risk and cost–benefit
assessment to assist in the decision making process’’.
To gain more insight into maritime risk assessment,
Guedes Soares and Teixeira [18] provide an overview of
the various different approaches to risk assessment over the
last 30 years.
From the above, it is clear that maritime risk research
has tended to focus on assessing the risk of collisions/
groundings and then investigating the methods of mitigat-
ing these risks, be it through ship design or policy. The
study by Van der Meer et al. [1] differs from this in that it
investigated the variables that contribute to a successful
maritime SAR operation, once a life-threatening situation
has actually developed.
3. Secondary data analysis
This section will present a summary of the analysis
conducted on the available secondary data. The original
study by Van der Meer et al. [1] was based on data for the
period 1995–1999; the present study uses data for the
period 1995–2004. We explore the data through contin-
gency tables, which can then form the basis for an
empirical BBN. In Section 3.2, we shall provide a
discussion of the shortcomings of the data and the
implications for inference.
3.1. Analysis of secondary data
This study is an investigation into the statistically
significant factors that influence the ratio of lives saved
to lives lost (relating to the SAR incidents in a given
Coastguard district in a given year), which we will simply
refer to as the ratio. We note that the ratio is a monotonic
transformation of the probability of a life being saved given
that the person involved is in a life-threatening situation
and, as such, the inference we derive is equally applicable
to both metrics.
The performance of districts with respect to the ratio
varies considerably from district to district. Fig. 1 is a
boxplot describing the distribution of the ratio between
1995 and 2004 for each UK Coastguard district. We
investigate the characteristics of these districts to determine
if there is a systematic difference due to some explanatory
factors, such as staff workload, average size of incidents or
length of coastline monitored.
 ARTICLE IN PRESS
L. Norrington et al. / Reliability Engineering and System Safety 93 (2008) 940–949 943

120.00

100.00

80.00
ratio

60.00 201
187
82
40
40.00 94 145

160

20.00
84

0.00
aberdeen
belfast
brixham
clyde
dover
falmouth
forth
hoyhead
humber
liverpoo
london
milford
oban
pentland
portland
shetland
solent
stornowa
swansea
thames
tyne tee
yarmouth
District

Fig. 1. Boxplot describing the distribution of the ratio of lives saved to lives lost for each district over the years 1995–2004.

We shall first establish the comparability of the data

before and after the closure of three of the coordination
centres. Subsequently, through the use of contingency
tables we shall identify the characteristics that are
associated with better-performing districts.
Fig. 2 is a boxplot comparing the distribution of the
ratio of lives saved to lives lost for the data for the ‘pre-
closure’ period 1995–1999 (Time code ¼ 0) and the ‘post-
closure’ period 2000–2004 (Time code ¼ 1). (While the
three centre closures all occurred near the beginning of the
second period, the following analysis is slightly flawed in so
far as the closures did not occur exactly on January 1, 2000,
but at three different dates some time afterwards). The
difference between these distributions is not statistically
significant. Both data sets are skewed distributions and
both are subject to outliers. For the pre-closure data, the
range of observations was (2.76, 106.25) with an arithmetic Fig. 2. Boxplot describing the distribution of the ratio of lives saved to
lives lost for periods 1995–1999 and 2000–2004.
mean of 27.08 and median of 21.55. For the post-closure
data the range of observations was (3.00, 85.25) with an
arithmetic mean of 23.88 and a median of 18.38. incident; (2) the staff workload, i.e., the ratio of the annual
While the closures of the three centers may have had an number of incidents to the number of Watch staff
impact on some specific districts, there appears to be no employed; and (3) the length of the coastline monitored
significant aggregate difference in the ratio of lives saved to by the coordination centre concerned. For this study we
lives lost. have discretised the data, for each variable respectively,
The earlier study by Van der Meer et al. [1] demonstrated into ‘low’ if the value was below the median; otherwise it
that the following explanatory variables tend to have a was classified as ‘high’. The discretised data were analysed
significant impact on the ratio of lives saved to lives lost: with contingency tables. We investigated whether staff
(1) the size of incidents, i.e., the average total number of workload had a significant impact on the ratio of lives
people at risk (calculated as lives saved plus lives lost) per saved to lives lost, conditioned on size of incident and
 ARTICLE IN PRESS
944 L. Norrington et al. / Reliability Engineering and System Safety 93 (2008) 940–949
Fig. 3. Boxplots of the ratio of lives saved to lives lost partitioned by
workload, size and length.
Table 1
Contingency table summarising association between workload and ratio
of lives saved to lives lost for high length of coastline and high size
Ratio Workload low Workload high
Low 16 (9.6) 2 (8.4)
High 14 (20.4) 24 (17.6)
Total 30 26
Chi-square statistic: 13.303, d.f.: 1.
length of coastline. Fig. 3 illustrates through boxplots the
conditional distributions of ratio of lives saved to lives lost
for high and low states of length (of coastline), size
(of incidents) and (staff) workload.
We partitioned the ratio of lives saved to lives lost into
high and low categories and conducted a contingency
analysis between workload and this ratio, conditioned on
the states of size and length. The only combination of
conditional states that made a statistically significant
difference when workload was increased was when both
length was high and size was high. For all other states
increasing workload made no significant difference. Table 1
is a summary of the contingency table results. NB: In each
cell is recorded the actual number of observations, with
within the brackets showing the expected number assuming
no association.
This approach to investigating the data also revealed
that size of incident has a positive impact on the ratio,
except under the conditions of low workload and low
length of coastline. Length of coastline is only marginally
significant in the condition where size is low and workload
is low, in which case length has a negative impact.
However, this marginal significance is dubious given the
multiple hypotheses being tested.
In summary, the secondary data analysis suggests that
higher-than-average workloads improve the reliability of
SAR operations when districts have to monitor a long
length of coastline and the average size of an incident
corresponds to many lives being at risk per incident.
Furthermore, those districts that deal with an above
average size of incidents typically have higher than average
ratios of lives saved to lives lost, except when lengths of
coastline are comparatively short and staff workload is
comparatively low in which case there is no significant
difference. Finally, that the length of coastline monitored
may have a negative impact on the reliability of a SAR
operation, when both the size of incident is comparatively
small and staff workload low. These results can be used to
construct an empirically based BBN, whereby the results
from the contingency tables can easily be expressed as
conditional probability tables (CPT). This will be discussed
in greater detail in Section 4.
3.2. Reflections on the secondary data analysis
There are a number of issues associated with the
secondary data that limit the validity of the results of the
inference described in Section 3.1. These criticisms mostly
relate to the fact that we are modelling a dynamical system
Total and that, as such, the relevance of historical data is
questionable.
18 First, there have been changes in the characteristics of
38
the Watch staff employed over the past years. The average
age of the workforce has decreased through retirements,
which clearly has implications for operational experience.
Moreover, the introduction of a new (Assistant) grade of
Watch staff has been having a further impact on the
experience level of the staff. These factors are not
addressed within the above analysis.
Secondly, SAR technology has changed over the past 10
years. While persons at risk may have better communica-
tion equipment (for example, mobile phones), the MCA
now has access to improved rescue resources e.g., lifeboats,
permitting faster response times. The Agency has also
introduced the Automatic Identification System (AIS),
which provides a continuous electronic picture of all
commercial ships, allowing easier identification and loca-
tion of distress calls from vessels fitted with this system.
Other issues concern the interpretation that we place on
the summary statistics. We measure workload rather
crudely by the ratio of SAR incidents to Watch staff
employed. With more precise data we could construct
measures that capture different types of SAR incidents and
different types of Watch staff. We measure the complexity
of the coastline crudely by its length; yet some districts may
be long and straight, while others may have complex
curvatures. The size of incidents is simply measured by the
average total number of lives at risk per incident; yet this
 ARTICLE IN PRESS
L. Norrington et al. / Reliability Engineering and System Safety 93 (2008) 940–949
does not discriminate between different degrees of incident
complexity that will affect the reliability of a SAR
operation.
Statistical analysis does not measure causation, only
correlation. We note that at the aggregate crude level at
which we have conducted the above analysis, we have
identified statistically significant associations. However, by
eliciting a BBN through expert judgement some of the
limitations found in the previous analysis can be addressed.
4. Elicitation of BBN
Within this section BBNs are briefly described, followed
by the description of the elicitation process undertaken to
develop the BBN for this situation. The final model is
shown along with a reflection on the process of elicitation.
4.1. The BBN
BBNs are a graphical method for reasoning under
uncertainty, sometimes known as a Directed Acyclic Graph
[19]. The random variables are denoted by nodes and the
directed arcs represent the influence. A node without any
arcs linking into it is known as a root node, a node with
arrows linking into it is known as a child node. A child
node without any arcs leading out is a leaf node.
It is usual to represent a node with discrete states.
However, it is important to try and limit the number of
states when the number of parents grows large, as
explained by Sigurdsson et al. [20]: ‘‘for a node with n
parents and each parent is a binary variable, we require 2n
probabilities to be elicited’’. Root nodes only require
marginal probability distributions to be described.
Non-root nodes require the elicitation of all combinations
Fig. 4. BBN showing the primary variables that contribute to an effective SAR operation.
945
in a CPT. Once all the CPTs have been described, the BBN
can be used to enter evidence and recalculate all the
probabilities of the variables. For a more complete
explanation, see [20,21].
4.2. Elicitation process discussed
The process to develop and quantify the BBN model was
undertaken at the MCA Headquarters in Southampton.
The various stages are outlined below:
4.2.1. Development of qualitative model
The qualitative aspect of the BBN was developed
through an interview process. This interview was con-
ducted with two members of MCA management. Both men
have worked as Watch Keeping Officers in the past, though
one more recently. They both have extensive experience in
both operational and management roles and were therefore
deemed to be classed as expert in their field for the
elicitation process.
It was explained to the interviewees that the aim was to
develop a graphical representation of all the key factors
that affect the success of a SAR operation. An example was
shown using the variable ‘weather’, to indicate the type of
graphical representation that was desired. Building the
model was conducted through interaction with both
experts. Using an A3 sheet of paper, the nodes and links
were added during the discussion. Throughout, reasons for
nodes and their position/links were justified verbally by
both of the experts. When disagreement about a variable or
location occurred, discussion ensued until a consensus was
reached.
The agreed model was built with Genie BBN software
[22] and can be seen in Fig. 4, with a full explanation of all
 ARTICLE IN PRESS
946 L. Norrington et al. / Reliability Engineering and System Safety 93 (2008) 940–949

Table 2
Table showing all variables identified during the elicitation process

Variables Definition States

Effective SAR operation Effective Search and Rescue operation in terms of success of outcome Yes No
Favourable weather Favourable weather conditions for SAR Yes No
Wind strength Strength of wind (Whether it is favourable for an effective SAR operation) Favourable Unfavourable
Favourable sea state Sea conditions favourable for an effective SAR operation Yes No
Visibility level The level of visibility Favourable Unfavourable
Daylight Whether the SAR operation is being undertaken in daylight Yes No
Air temperature Temperature in the air Non-critical Critical
Sea temperature Sea temperatures (N.B. sea conditions in UK are nearly always critical Non-critical Critical
80–90% time)
No. of concurrent incidents How busy a station is—Reflecting the effects of the season in terms of more Manya Fewb
complexity in rescue
Distance Distance of the incident from SAR units Reasonable Unreasonable
Acceptable response time Response time acceptable—people rescued in time Yes No
Local hazards known Local hazards known to aide in location e.g., overdue vessel understanding Yes No
risks on the route (where a vessel may become grounded)
Location known The position/location of the situation is known Yes No
Local knowledge Local knowledge sufficient to locate incident (in terms of place names at co- Sufficient Insufficient
ordination centres)
Sufficient resources The resources available are sufficient/capable for the operation at hand Yes No
Declared facilities Facilities known to the SAR teams Suitable Unsuitable
Passing ships Whether passing ships are available as an additional resource Yes No
Competency of SAR units The competency of the SAR units (personnel) Full Low
Severity of incident Severity of the incident in terms of the level of danger to the casualties High Low
Containment strategies Strategies to minimise the risk to casualties Yes No
Casualty situation The situation of the casualty in terms of equipment, state of mind and Good Bad
physical condition
Onboard a vessel Whether the casualty is onboard a vessel or in the sea Yes No
State of mind Casualty’s state of mind or willingness to live Good Bad
Training The level of maritime training that a casualty possesses High Low
Level of equipment Level of equipment that the casualty has in possession Sufficient Insufficient
Physical state The physical state of the casualty (vessel/person) Good Bad
Casualty injured Whether the casualty is injured Yes No
Level of competency MRCC The level of competency of the MRCC staff (paired-operational partnership) Full Low
staff
Level of experience Experience of staff High Low
Level of knowledge/skills Knowledge or skills of staff High Low
Attitude The attitude of staff Good Bad
Levels of staff training Levels of training activity High Low
Expectation of outcome Expectation of the outcome of the incident Good Bad
Morale Level of staff morale High Low
Effective communication Whether the communication is effective to the point of not adversely affecting Yes No
the outcome of the SAR operation
Effective alert Effective initial alert of situation Yes No
Effective SAR Effective SAR communication (communication between casualty and co- Yes No
Communication ordination centre and SRVs)
Continuity of communication Effective continuity of communication throughout the operation Yes No
Effective language Whether there is a common language either literal or technical so that the Yes No
communication is not impaired
Coordination of rescue unitsc Effective coordination of rescue units throughout the operation Yes No
a
Many ¼ Summer, Late spring, Early autumn co-ordination centres are more busy.
b
Few ¼ Winter, Early spring, Late autumn co-ordination centres are less busy.
c
This variable was later removed as it was felt to be captured by other variables relating to communication.

the variables and their possible states shown in Table 2.
The variables were discussed fully with one expert and
partially with the other for validation. The states of
the nodes have been kept binary, due to the large number
of variables elicited (40 in total). Having non-binary
states would make the task of quantification even more
arduous.
4.2.2. Quantification
As explained earlier, the usual quantification of BBNs
requires eliciting CPTs for all combinations of parent
nodes feeding into a child node. The elicitation process
needed to be completed within 3 days. Quantifying the
BBN was not possible given the short timeframe allotted
with the experts and also given the number of parent nodes
 ARTICLE IN PRESS
L. Norrington et al. / Reliability Engineering and System Safety 93 (2008) 940–949
feeding into the leaf node Effective SAR Operation. To
fully quantify this BBN would require eliciting almost 700
probabilities. As such we adapted a solution presented by
Celeux et al. [21] to the problem of too many parent nodes.
Firstly, we elicit probabilities for a successful SAR
operation conditioned on the state of the nodes that are
directly linked to the Effective SAR Operation node, of
which there are nine. This would require eliciting 512
conditional probabilities and a further nine to assess the
likelihood that each parent node was in a given state. The
reduction is justified through having the experts qualita-
tively take into account all factors within the BBN when
specifying a probability for nodes closer to the key node of
interest. This is similar to an elicitation approach described
in Hodge et al. [23], whereby, first, all relevant factors are
elicited with a qualitative description of their association
and then quantification is conducted—but not on all
variables, just on the subset with a direct influence on the
key variable of interest (i.e., effective SAR operation). The
purpose of eliciting a qualitative description of all the
variables with their relationships is to ensure the expert has
considered all relevant factors.
A further reduction in the number of probabilities to
elicit was considered by initially exploring for indepen-
dence through representing the BBN with a simple log-
linear model, where all the association terms of order
greater than two are constrained to be zero. If the
influences are adequately modelled and independent, then
we would require eliciting only 10 conditional probabilities
and an additional 14 to assess the likelihood of the state of
the parent nodes (NB: There are some dependencies among
the nine nodes as can been seen on Fig. 4 which accounts
for eliciting 14 rather than simply nine probabilities). We
explore the validation of this approach in the elicitation
process. The probability of a successful outcome, i.e., p,
can then be represented as in Eq. (1).
! in a favourable state.
X9
p ¼ exp b0 þ bi X i ,
i¼1
where
(
1 if factor i is favourable
Xi ¼ , ð1Þ
0 else:
This model formulation was decided upon due to its ease
of interpretation, as the influence that factors have on the
baseline probability is multiplicative. An equivalent form
of (1) is expressed in (2).
Y
9 location known.
p ¼ p0 ri
i¼1
where :
p0 ¼ expðb0 Þ
(
expðbi Þ; if X i ¼ 1
ri ¼ , ð2Þ
1; if X i ¼ 0:
947
Applying this model to the BBN of SAR operation
requires first eliciting information about the variables that
have a direct link into the central node Effective SAR
Operation in Fig. 4, and then eliciting the probability that
the variables would be in their various states which is
simplified through assuming independence. The relevant
variables for our model are:
1. Favourable weather
2. Competency of resources (SAR personnel)
3. Acceptability of response time
4. Number of concurrent incidents
5. Severity of incident
6. Location known
7. Effective communications
8. Competency of MRCC (Incident coordination) staff
9. Casualty situation
Having the experts first order the factors from most to
least influential and subsequently quantifying the relation-
ships, follows the ‘swing weight’ method to elicitation used
with multi-attribute decision analysis [24]. The following
six steps are required for the elicitation process.
1. If all nine variables were in a favourable state, what
would be the probability that there was a successful
outcome?
2. If all nine variables were in an unfavourable state, what
would be the probability that there was a successful
outcome?
3. Rank the factors from the most to least influential.
4. For each variable, starting with the most influential,
elicit a range and point estimate of impact on the
probability of a successful outcome from changing the
state of the variable, assuming all other variables remain
5. Repeat step 4 assuming all variables are set to
unfavourable, does that change the multiplicative
impact? If NO—then stop for that variable and assume
independent of the state of the other variables. If YES—
then identify the variables influencing the impact. For
the identified subset of variables elicit the probability for
all combinations of states.
6. Elicit the probabilities of the factors being in their
favourable or unfavourable state; those that are
exhibiting dependency must be conditioned. In the
present case, those to be considered together are:
favourable weather; acceptable response time and
A scale to translate from a verbal description to a
numeric probability and measure of impact were used to
aide the expert in eliciting numerical values for probabil-
ities and impact. These are illustrated in Fig. 5. The impact
scale was used to capture multiplicative effects on baseline
probabilities. Our experience within this elicitation was
 ARTICLE IN PRESS
948 L. Norrington et al. / Reliability Engineering and System Safety 93 (2008) 940–949
Fig. 5. Scales of impact—favourable and unfavourable and the scale of
probability.
that the experts felt more confident with the probabilities
than with the impact scale.
4.3. Reflections on the elicitation
Building a model interactively, as was undertaken
in this case, has the benefits of full discussions relating
to the links and positioning of nodes. It gives the
expert greater ownership of the model, perhaps aiding in
the ‘signing off’ on the model and ensuring that all
variables are fully understood by the facilitator. However,
it is difficult to undertake an interview and model
development concurrently without a high degree of
experience. It is essential for the facilitator to be aware of
this and to ensure a lack of experience is not detrimental to
the process.
Carrying out the process with two interviewees also has
benefits and drawbacks. The benefits are a more animated
justification of the model through discussions between the
two people. The model is also technically verified at each
stage, by the experts interacting together. However, there is
a potential problem of more confident experts dominating
the process, as contended by Walls and Quigley [25]: ‘‘to
elicit judgement from each individual expert and then
combine views [afterwards]ymight prove advantageous in
situations where the persuasive arguments of dominant or
over confident experts might prevail or less-confident
experts might be isolated’’. An additional difficulty with
undertaking this elicitation process is managing the two
experts. Keeping control of the situation and directing the
discussion becomes more difficult when not in a one-on-
one interview situation. The experts take more control of
the situation, unless considerable assertiveness is shown by
the facilitator.
It must also be acknowledged that there is a complicated
political environment associated with the work being
undertaken (as outlined in Section 2.1) and this may have
led to some caution on the part of the interviewees over
what was being put into the model. This implies that there
could be some inherent bias in the present model.
As with the process of building the qualitative model,
quantifying the model is riddled with difficulties. While a
BBN allows for the development of a complete picture of
all the major variables that impact on a SAR operation and
how they relate, actually quantifying variables such as Staff
Morale can become difficult. It then becomes necessary to
rely on expert judgment to provide subjective probabilities
to populate the model.
The use of scales helped to give each expert confidence in
what numerical information he was providing. Usually, the
experts spoke in terms of the verbal meanings and the
values were translated into a numerical assessment.
Identifying a useful medium of communication was
essential. There are many mathematical ways to represent
the problem: at first the multiplicative impact on prob-
abilities was considered; likewise we could have pursued
the odds of survival, but the experts felt more comfortable
assessing the probability of a successful SAR operation.
Scale was a key factor in this preference, where the experts
found probability an easier scale to interpret than an
impact factor such as in (2).
Dependability measures between variables prove the
greatest challenge to elicit. During step 5, it appeared easy
to elicit the impact when the other conditions were in a
favourable state, but when all conditional states are
changed to unfavourable, the elicitation exposed a lack of
credibility with the initial assessment. The expert felt that
all impacts would individually be reduced if all the other
factors remained constant in an unfavourable state.
Eventually, it became clear that only the Acceptable
Response Time variable could be classed as having a non-
associated impact to any other variable. It was the experts’
belief that this variable’s impact would not change,
irrespective of the states of the other variables. For all
other variables, it was felt that they were closely related and
that the impact was reduced. Time constraints did not
allow an elicitation of all combinations of impact variables
and the expert could not clarify a precise relationship
between various variables.
 ARTICLE IN PRESS
L. Norrington et al. / Reliability Engineering and System Safety 93 (2008) 940–949
5. Summary and future work
The BBN analysis concerning environmental factors
identified several other factors that the UK Coastguard
do not officially measure. However, captured within
the BBN are variables related to staff workload, local
knowledge and size and severity of incidents that
are consistent with the statistical analysis described in
Section 3.
The process of constructing the BBN resulted in a deeper
understanding of validity (and lack thereof) of the
surrogate measures used with the secondary data analysis.
However, the experts struggled to provide quantitative
data. This is not an uncommon experience as providing
defensible subjective probabilities places a heavy cognitive
burden on an expert, especially when considering depend-
ability measures between many variables. This research is
ongoing and we continue to explore a hybrid solution
between fully Bayesian and empirically based Bayesian
methods to provide some relief for the experts and acquire
more reliable inference.
The next stage of this research is concerned with
collecting and investigating more detailed incident statistics
in light of the BBN acquired. These data will be used to
populate the BBN with regard to some variables and to
inform a more supportive elicitation exercise for other
variables. Currently, we are adapting hybrid methodologies
such as those discussed in Quigley et al. [26] for risk
assessment on railways and Bedford et al. [27] for reliability
assessment of novel engineering systems, whereby expert
judgement is used to assess broad categories of association
and secondary data is used to provide more precise
measurements of association.
In summary, BBNs provide more explanatory detail to
support more effective operational improvements but
validity of the resulting model is an issue.
References
[1] Van der Meer R, Quigley J, Storbeck J. Using regression analysis to
model the performance of UK coastguard centres. J Operat Res Soc
2005;56:630–41.
[2] Donaldson Lord (JF). Review of the five year strategy for HM
coastguard. London: Department of the Environment, Transport and
the Regions; 1999.
[3] British Broadcasting Corporation. South West Wales Sea
Empress: 10 years on- what happened? An introduction. /http://
www.bbc.co.uk/wales/southwest/sites/seaempress/pages/intro.shtmlS.
949
[4] Merrick JRW, René van Dorp J, Mazzuchi T, Harrald JR, Spahn JE,
Grabowski M. The Prince William sound risk assessment. Interfaces
2002;32(6):25–40.
[5] Ligthart VHM. Determination of probability of marine accidents
with respect to gas carriers proceeding in Dutch Coastal and Inland
Waters. J Hazard Mater 1980;3:233–47.
[6] Friis-Hansen P, Simonsen BC. GRACAT: software for grounding
and collision risk analysis. Mar Struct 2002;15:383–401.
[7] Brown AJ. Collision scenarios and probabilistic collision damage.
Mar Struct 2002;15:335–64.
[8] Iakovou ET. An interactive multiobjective model for the strategic
maritime transportation of petroleum products: risk analysis and
routing. Saf Sci 2001;39:19–29.
[9] Goossens LHJ, Glansdorp CC. Operational benefits and risk
reduction of marine accidents. Safety and VTS 51(3):368–81.
[10] Van Urk W, de Vries WA. POLSSS: policy making for sea shipping
safety. Saf Sci 2000;35:139–50.
[11] Pöyhönen M. POLSSS: surveying stakeholders about acceptability of
risks and system changes. Saf Sci 2000;35:123–37.
[12] Walker WE. POLSSS: overview and cost-effectiveness analysis. Saf
Sci 2000;35:105–21.
[13] Sato Y, Ishii H. Study of a collision-avoidance system for ships.
Control Eng Pract 1998;6:1141–9.
[14] Trbojevic VM, Carr BJ. Risk based methodology for safety
improvements in ports. J Hazard Mater 2000;71:467–80.
[15] Hara K, Nakamura S. A comprehensive assessment system for the
maritime traffic environment. Saf Sci 1995;19:203–15.
[16] Bruzzone AG, Mosca R, Revetria R, Rapallo S. Risk analysis in
harbor environments using simulation. Saf Sci 2000;35:75–86.
[17] Wang J. The current status and future aspects in formal ship safety
assessment. Saf Sci 2001;38:19–30.
[18] Guedes Soares C, Teixeira AP. Risk assessment in maritime
transportation. Reliab Eng Syst Saf 2001;74:299–309.
[19] Lee CJ, Lee KJ. Application of Bayesian network to the probabilistic
risk assessment of nuclear waste disposal. Reliab Eng Syst Saf
2006;91:515–32.
[20] Sigurdsson JH, Walls L, Quigley J. Bayesian belief nets for managing
expert judgement and modelling reliability. Qual Reliab Int
2001;17:181–90.
[21] Celeux G, Corset F, Lannoy A, Ricard B. Designing a Bayesian
network for preventive maintenance from expert opinions in a rapid
and reliable way. Reliab Eng Syst Saf 2006;91:849–56.
[22] Genie BBN software available from /http://genie.sis.pitt.eduS.
[23] Hodge R, Evans M, Marshall J, Quigley J, Walls L. Eliciting
engineering knowledge about reliability during design—Lessons
learnt from implementation. Qual Reliab Eng Int 2001;17:169–79.
[24] Belton V, Steward T. Multiple criteria decision analysis: an integrated
approach. Dordrecht: Kluwer Academic Publishers; 2001.
[25] Walls L, Quigley J. Building prior distributions to support Bayesian
reliability growth modelling using expert judgement. Reliab Eng Syst
Saf 2001;74:117–28.
[26] Quigley J, Bedford T, Walls L. Estimating rate of occurrence of rare
events with empirical Bayes: a railway application. Reliab Eng Syst
Saf 2007;92:619–27.
[27] Bedford T, Quigley J, Walls L. Fault tree inference through
combining Bayes and empirical Bayes methods. In: Proceedings of
the ESREL conference, Estoril, Portugal; 2006.