Sunteți pe pagina 1din 12

Downloaded from http://qualitysafety.bmj.com/ on July 26, 2015 - Published by group.bmj.

com
BMJ Quality & Safety Online First, published on 21 July 2015 as 10.1136/bmjqs-2015-004021
SYSTEMATIC REVIEW

Barriers and facilitators related to


the implementation of surgical
safety checklists: a systematic
review of the qualitative evidence
Jochen Bergs,1 Frank Lambrechts,1 Pascale Simons,1 Annemie Vlayen,2
Wim Marneffe,1 Johan Hellings,2,3 Irina Cleemput,4 Dominique Vandijck2,5,6

▸ Additional material is ABSTRACT INTRODUCTION


published online only. To view
Objective The objective of this review is to Surgical safety checklists are intended to
please visit the journal online
(http://dx.doi.org/10.1136/bmjqs- obtain a better understanding of the user-related ensure consistency in patient perioperative
2015-004021). barriers against, and facilitators for, the safety and to introduce or maintain
implementation of surgical safety checklists. a culture that values achieving it.1 The use
For numbered affiliations see
end of article. Methods We searched MEDLINE for articles of these checklists is associated with
describing stakeholders’ perspectives regarding, increased patient safety awareness,
Correspondence to Jochen and experiences with, the implementation of improved communication, reduction of
Bergs, Faculty of Business
Economics, Hasselt University,
surgical safety checklists. The quality of the surgical claims and reduction in the
3500 Hasselt, Belgium; papers was assessed by means of the Qualitative number of postoperative complications
jochen.bergs@uhasselt.be Assessment and Review Instrument. Thematic including mortality.2–10 It cannot, however,
synthesis was used to integrate the emergent be assumed that the mere application of the
Received 30 January 2015
Revised 24 June 2015 descriptive themes into overall analytical themes. checklist will automatically lead to
Accepted 4 July 2015 Results The synthesis of 18 qualitative studies improved safety.11–13 Reported compliance
indicated that implementation requires change in with checklist items is assumed to be corre-
the workflow of healthcare professionals as well lated with the impact of surgical safety
as in their perception of the checklist and the checklists.10 14 Consequently, the clinical
perception of patient safety in general. The effectiveness of the checklist will vary with
factors impeding or advancing the required the implementation success.15 16
change concentrated around the checklist, the The implementation of new guidelines
implementation process and the local context. and safety interventions has been shown
We found that the required safety checks disrupt to be difficult in various healthcare sectors,
operating theatre staffs’ routines. Furthermore, which highlights the importance of the
conflicting priorities and different perspectives implementation process.17–19 Several
and motives of stakeholders complicate checklist studies have reported high levels of partici-
implementation. When approaching the checklist pation and checklist completion (ranging
as a simple technical intervention, the from 12% to 100%).20 However, the
expectation of cooperation between surgeons, implementation is more than merely
anaesthetists and nurses is often not addressed, ‘checking the box’. A discrepancy between
reducing the checklist to a tick-off exercise. ticking off checklist items and the perform-
Conclusions The complex reality in which the ance of the actions results in poor fidelity
checklist needs to be implemented requires an as regards the checklist’s intentions.21–29
approach that includes more than eliminating The implementation of a surgical safety
barriers and supporting facilitating factors. checklist is a complex social intervention.
Implementation leaders must facilitate team Factors influencing the dissemination and
learning to foster the mutual understanding of uptake of evidence-based interventions or
To cite: Bergs J, perspectives and motivations, and the realignment technological innovation may, therefore,
Lambrechts F, Simons P, et al. of routines. This paper provides a pragmatic not apply. In order to increase the under-
BMJ Qual Saf Published
Online First: [ please include
overview of the user-related barriers and facilitators standing of the user-related barriers to, and
Day Month Year] upon which theories, hypothesising potential facilitators of, the implementation of surgi-
doi:10.1136/bmjqs-2015- change strategies and interactions, can be cal safety checklists, we conducted a sys-
004021 developed and tested empirically. tematic review of the qualitative literature.

Bergs J, et al. BMJ Qual Saf 2015;0:1–11. doi:10.1136/bmjqs-2015-004021 1


Copyright Article author (or their employer) 2015. Produced by BMJ Publishing Group Ltd under licence.
Downloaded from http://qualitysafety.bmj.com/ on July 26, 2015 - Published by group.bmj.com

Systematic review

METHODS using the Qualitative Assessment and Review


Design Instrument (QARI).34 The 10 QARI criteria do not
A synthesis of the qualitative evidence synthesis was relate to validity or bias in the process-orientated
formulated by means of thematic synthesis.30 The methods as regards the reviews of effects, their
strength of this methodology is its potential to enable purpose being to establish the nature and appropriate-
conclusions to be drawn on the basis of common ness, the methodological approach, the specific
elements of otherwise heterogeneous studies. methods and the representation of the voices or
Conclusions drawn from thematic analysis fulfil an meanings of the study participants.35 However, given
important research aim of qualitative research by gen- that there is no accepted method for excluding quali-
erating hypotheses for which traditional systematic tative studies from the syntheses on the basis of their
reviews are poorly suited.30 quality, we did not exclude studies based on the QARI
scoring.36 37
Search strategy
The search was performed in MEDLINE (from incep- Extracting data from studies
tion to March 2015) using the following query: A data extraction form was developed cataloguing the
(“Surgical Procedures, Operative”[Mesh] OR “surgi- author, year of publication, journal, method of data
cal” OR “surgery” OR “operative)” AND collection, phenomena of interest, study setting,
(“checklist”[MeSH] OR “checklist” OR “time-out)” country of study, data analysis and main conclusions.
AND (“fidelity” OR “implementation” OR “adher- In syntheses of qualitative research, the ‘informants’
ence” OR “compliance” OR “barriers” OR “facilita- are the authors of the individual studies rather than
tors” OR “incentives”). Broad search terms were the participants in these studies. Therefore, the
applied without date restrictions in order to make the authors’ interpretations—presented, for example, by
search strategy as sensitive as possible. Methodological themes and categories—constitute our data. While the
filters for study design were not used because they authors’ interpretations were collected primarily from
reduce the sensitivity of searches.31 32 The reference the results sections, data found in the discussion
lists of all of the papers were scrutinised, and a cited sections were also extracted when relevant and well
reference search was made in the Web of Science for supported by data.
additional papers on the subject.
Thematic synthesis
Inclusion criteria Thomas and Harden described thematic synthesis as
Qualitative studies that explored the perspectives and “a tried and tested method that preserves an explicit
experiences of stakeholders with the implementation and transparent link between the conclusions and text
of surgical safety checklists were included. These sta- of the primary studies; as such it preserves principles
keholders were nurses, surgeons, anaesthesiologists, that have traditionally been important to systematic
residents, implementation leaders, administrators and reviewing”.30 Thematic analysis has three stages: (a)
any others directly involved in, or affected by, the line-by-line coding of the findings of primary studies
implementation. Following the guidance of the to extract the key concepts, (b) organisation of these
Cochrane Qualitative Research Methods Group, key concepts into related areas to construct ‘descrip-
which considers critical appraisal to be a technical and tive’ themes that formed the backbone of the structure
pragmatic exercise, we restricted the type of qualita- of the analysis and (c) development of ‘analytical’
tive studies included in this review.33 Only empirical themes based on the synthesis of the experiences and
studies with a description of the sampling strategy, the recommendations of authors of the original articles.
data collection procedures and the type of data ana- While the development of descriptive themes remains
lysis were included. These empirical studies had to close to the primary studies, the analytical themes rep-
report the methodology chosen and the methods or resent a stage of interpretation in which the reviewers
research techniques opted for since this facilitates the go beyond the primary studies and generate new inter-
systematic use of critical appraisal as well as a more pretive constructs, explanations or hypotheses.
pragmatic appraisal process. Therefore, descriptive QSR International’s NVivo 10 software was used to
papers, editorials and opinion papers that were not organise the codes into hierarchical structures.38 The
based on actual experiences related to the implemen- text of each included study (results and discussion
tation of surgical safety checklists were excluded. No section) was imported into the software verbatim.
language or country restrictions were applied. One of us ( JB) developed a set of descriptive codes
inductively by coding each line of the text of all of
Quality assessment the included studies. We looked for similarities and
The primary goal of our quality assessment was to differences between the codes in order to start group-
highlight the quality of the published literature on the ing them into a hierarchical tree structure, and new
subject. The full texts of the included articles were codes were created to catalogue the meaning of
reviewed by two independent authors ( JB and PS) groups of initial codes. This process resulted in a tree

2 Bergs J, et al. BMJ Qual Saf 2015;0:1–11. doi:10.1136/bmjqs-2015-004021


Downloaded from http://qualitysafety.bmj.com/ on July 26, 2015 - Published by group.bmj.com

Systematic review

structure with several layers in order to organise the potential influence on the study findings and vice
descriptive themes. The groupings were then further versa. Adequate representation of the participants,
refined by discussion and rechecking of the original and their opinions, was not provided in five studies,
studies ( JB, FL and PS). Successive drafts of a narra- and the congruity between the stated philosophical
tive that described the themes seen in the findings perspective and the research methodology was often
were then discussed by the wider study group ( JB, FL, difficult to evaluate. No additional exclusions were
PS, AV and DV) and further refined. All of the stages made after the technical appraisal.
of the process were checked by various experts (an
organisational psychologist, safety culture experts and
patient safety experts) to ensure accuracy and control Synthesis findings
of bias in the analysis. The thematic analysis of the primary articles revealed
five main themes with respective subthemes. Two of
these main themes—staffs’ perception of the checklist
RESULTS
and patient safety, and workflow adjustments—repre-
The search strategy yielded 535 papers. Screening for
sent user-related changes required to conduct the
eligible and inconclusive abstracts reduced this list
checklist as intended (dimension one). The other
to 45 papers. After full-text review, 27 studies21 22
27–29 39–60 three main themes—checklist, implementation process
were excluded from the final analysis
and local context—constitute factors affecting the
(see online supplementary appendix 1 for further
user-related changes (dimension two). Figure 2 shows
explanation). We included 18 studies involving >700
our data structure. It provides a graphical representa-
healthcare professionals (see figure 1).25 61–77 The
tion of how we progressed from subthemes (the result
studies involved 18 different countries. The data in
of grouping key concepts from the primary studies) to
these studies had been collected using interviews,
main themes describing the two dimensions.
focus groups, observations and open-ended surveys.
A detailed overview of the study characteristics is
provided in online supplementary appendix 2.
User-related changes required to conduct the checklist as
intended
Quality assessment In order to implement the checklist and assure it is
Overall, the selected studies scored well on the QARI used as intended, changes on the user level (ie,
(see online supplementary appendix 3 for an overview doctors and nurses) are needed. In what follows, we
of QARI scoring). Nevertheless, only 4 of the 18 describe the main themes and subthemes related to
papers stated the cultural and/or theoretical location this first dimension. Illustrative quotations for each of
of the researcher, so it was difficult to appraise his/her the themes are provided in table 1.

Figure 1 PRISMA diagram.

Bergs J, et al. BMJ Qual Saf 2015;0:1–11. doi:10.1136/bmjqs-2015-004021 3


Downloaded from http://qualitysafety.bmj.com/ on July 26, 2015 - Published by group.bmj.com

Systematic review

Figure 2 Data structure.

Staffs’ perception of the checklist and patient safety of the safety checks introduced by the checklist
The perception of the staff of the checklist and its involves changes in the workflow of the operating
items, and the perception of patient safety in general, theatre staff, which was often experienced as an
determines the individual willingness to use the check- increased workload. Besides individual workflow
list. Healthcare providers expressed concerns about adjustments, alignment between the workflows of sur-
legal implications, which hampered their willingness geons, anaesthesiologists and nurses is needed in
to use the checklist. Participants, especially nurses, order to execute the brief stops (ie, sign in, time out
expressed concerns about patient perception, which and sign out) required to complete the checklist.
led to the omission of items they perceived as causing Aligning the workflow of team members is difficult to
stress in patients (eg, expected blood loss) or perform- achieve. In addition, the checklist sometimes entails
ing the checks without verifying out loud. Most of the the repetition by nurses or doctors of one or more
studies reported concerns about time consumption safety checks—as some are already included in exist-
and efficiency. Participants felt, or presumed, that ing procedures—the redundant registration thus creat-
checklist execution consumed too much time and ing an administrative burden. As a result, doctors and
thereby hampered operating theatre efficiency. The nurses might experience the checklist as an additional,
perceived importance of the checklist items varied often unnecessary, task.
across professions and individuals, which led to
varying usage and support among surgeons, anaesthe-
Factors affecting the user-related changes
siologists and nurses. The perceived importance is
strongly related to understanding the intentions and Several factors act as a barrier or facilitator to the
aims of the checklist. In addition, risk perception user-related changes required to precede checklist
plays an important role in the conviction that risks usage. In what follows, we describe the main themes
assumed by the checklist exist in the immediate work and subthemes related to this second dimension.
environment. Finally, scepticism regarding the evidence Illustrative quotations for each of the themes are pro-
base was expressed. Surgeons and anaesthetists, in par- vided in table 2.
ticular, believed that the existing evidence was incon-
The checklist
clusive and did not support general implementation of
surgical safety checklists. A first theme emerged around design problems influ-
encing staff perception and workflow. Many health-
Workflow adjustments care providers found the content somehow irrelevant
Implementation of the checklist requires modification to their setting or suggested rewording certain items
of operating theatre staffs’ workflow. Proper execution to create a better fit with actual usage. Layout and

4 Bergs J, et al. BMJ Qual Saf 2015;0:1–11. doi:10.1136/bmjqs-2015-004021


Downloaded from http://qualitysafety.bmj.com/ on July 26, 2015 - Published by group.bmj.com

Systematic review

Table 1 Themes describing user-related changes required conducting the checklist as intended with exemplary citations from the study
findings
Main theme Subtheme Exemplary citations from the study findings
Staffs’ perception of the Concerns about legal Nurses were therefore concerned about the legal implications of signing the checklist
checklist and patient safety implications as they might be held accountable for errors.68
Concerns about patient For example, some stated that patients often did not understand why they had to
perception confirm their ID/procedure, etc., so many times during their surgical pathway, and
others felt that specific questions around blood loss and difficult airway (part of the
sign-in checks) would anxiety provoking for certain patients (this was a particular
concern if the patient was undergoing a local anaesthetic procedure and therefore
witnessed all of the checks).76
Concerns about time …respondents had significant concern about its perceived effect on OR efficiency.70
consumption and efficiency
Perceived importance Confirming the team members by name and role was the most missed check. The
explanation given for this was that a lack of staff turnover and degree of familiarity
with each other made this check appear less important.73
Scepticism regarding the Scepticism regarding the evidence base: The evidence base behind the checklist is
evidence base weak and/or not applicable to the current context.76
Workflow adjustments Individual workflow Two participants expressed concern about the interruption in workflow that was
adjustments sometimes required to complete the checklist discussion.61
Aligning the workflow of team These asynchronous workflows impacted on a healthcare professional’s ability to halt
members their work and collaboratively meet to communicate at a time-out procedure: Time out
was about to commence and the nurse initiating it asked the anaesthetist “Are you
joining us?” The anaesthetist replies, “No, we have things to do”.
[Obs_circnurs_125]25

form factor issues (eg, inconvenient format or experience psychological ownership, meaning that
complex computer applications) were also expressed. they must have the feeling that the intervention is
Second, respondents found that the execution process created, or at least tailored, to their needs. This sense
did not merge with existing processes. This created of ownership seems crucial in convincing doctors to
redundant safety checks and administration or even use the checklist. Even though the checklist is
conflicting workflows. Linked to beliefs and preju- supported by evidence and is endorsed by leading
dices, as previously mentioned, professionals need to organisations (eg, professional associations),

Table 2 Themes describing the factors affecting the user-related changes with exemplary citations from the study findings
Main theme Subtheme Exemplary citations from the study findings
The checklist Checklist content The binary (yes/no) response system was ambiguous and confusing.68
Execution process did not merge with … duplication with existing processes that already covered several of the items in the
existing processes surgical checklist.68
Psychological ownership … they [surgeons] did not necessarily agree with it, albeit this protocol was endorsed by
the College of Surgeons.62
staff should have been involved in adapting and implementing the SSC as a means of
fostering ownership.74
The implementation Education and training Many participants said that they did not receive information or training on how to use
process the SSC…. 74
Unclear guidelines Many participants said that staff were uncertain about how to use the SSC and who was
responsible for leading it.74
Surgeons commitment Physician’s support and motivation were crucial for implementing the checklist.63
The local context Executive leadership … lack of clarity and agreement with protocol specifics, and inadequate executive
leadership primarily resulted in reduced ownership and acceptance of the protocol by
physicians.68
Hospital leadership was not seen as involved in either promoting or actively
implementing the SSC.74
Organisational culture The same proportion of staff held the perception that the culture within their hospital
was that of a general resistance to the introduction of change, whatever form it takes,
particularly from more senior members of staff.76
Communication and teamwork We often talk about being one team, but it is in itself three teams. The surgeons don’t
see themselves as part of the team; they see the others forming the team, but they invite
in so to speak.62

Bergs J, et al. BMJ Qual Saf 2015;0:1–11. doi:10.1136/bmjqs-2015-004021 5


Downloaded from http://qualitysafety.bmj.com/ on July 26, 2015 - Published by group.bmj.com

Systematic review

individuals may have a different opinion on its useful- according to the practical, social and professional con-
ness and importance. Some parts or items of the ditions in their work environment.
checklist are perceived as relevant only to specific pro-
fessionals, which diluted the sense of shared Analytical themes
responsibility. Up to this point, we have produced a synthesis that
kept very close to the original findings of the studies.
The implementation process ‘Going beyond’ the content of the original studies has
The implementation process refers to the joint activity been identified by some as the defining characteristic
of implementing the checklist by actors who, in rela- of synthesis.30 With the development of analytical
tion to checklist implementation, engage in particular themes, we ‘went beyond’ the findings of the primary
ways of relating over time.78 This process must create studies and generated additional concepts, under-
a clear transition period supporting the acquisition of standings or hypotheses.
the new workflow. A large part of the implementation Disrupted routines and conflicting priorities
effort entails education and training. Participants from The introduction of a checklist in the operating theatre
the original studies found explaining why the checklist involves doctors and nurses changing existing rou-
is necessary, providing clear communication on organ- tines.25 Workflows on the individual, professional or
isational intentions, and addressing the concerns team level have to be altered and aligned in order to
expressed by staff to be of vital importance for creat- create a brief moment of reflection to review the safety
ing support and willingness prior to the actual imple- checks collectively.25 61–63 74 76 77 The workflow intro-
mentation. In addition, participants found it duced by the checklist often collides with existing rou-
important to show them how they must use the check- tines.61 63 This creates conflicts as existing workflows
list. Healthcare providers found that unclear guide- are established in function of different priorities (eg,
lines hampered checklist execution. Therefore, the efficiency or productivity).61 62 66 68 77 Hence, the pri-
introduction needs to be supported with clear guide- ority of patient safety, introduced by the checklist, has
lines and training on how, when and who will execute to compete with the already existing clinical and organ-
the checklist. During the implementation process, the isational priorities. Conflicts between priorities compel
commitment of all of the professionals is required to doctors and nurses to weigh using the checklist against
sustain checklist usage. Nurses found surgeon commit- other priorities. When a conflict in priorities emerges,
ment to be imperative. Given the hierarchical context the perceptions of operating theatre staff regarding
within the operating theatre, senior surgeons’ leader- patient safety drive their ultimate decision about
ship is of undeniable importance. whether or not to use the checklist.66

The local context Different perspectives and motives


The local context refers to the local historical- The motivation for implementing a surgical safety
relational context of checklist implementation that is checklist differs between healthcare providers and
always partly created in the joint activity that the hospital management.62 70 75 77 The use of a surgical
actors engage in.78 Participants expressed that, in add- safety checklist is often part of hospital-accreditation
ition to general leadership, executive leadership is requirements or other quality-improving programmes.
needed to communicate the importance of the check- Management feels, with the perspective of obtaining
list and patient safety in general. Executive leadership accreditation, that it is necessary to apply the checklist
needs to be exercised in order to create a context in very strictly. Healthcare providers, however, feel that
which doctors and nurses feel supported. Across some of the checklist items have little or no relevance
studies, respondents saw the organisational culture as in their specific setting. Without clear communication
a major barrier but also as a potential facilitating about the motives of hospital management, perspec-
element. Both the hospital-wide and the immediate tives drift apart and resistance towards the checklist
organisational culture play a mediating role. Although develops. The perspective and motivation of the
checklist usage is expected to change the safety doctors may also differ from those of the nurses.
culture, there is a very complex relationship between Because of organisational requirements, nurses feel it
the checklist, the procedures, the context, the culture necessary to use the checklist while doctors may not
and the behavioural changes.72 With the deepening of always concur with these requirements.66
the culture, respondents found that communication The checklist is implemented as a simple technical intervention
and teamwork issues hamper checklist execution. The implementation of checklists is more than requiring
These issues often stem from a hierarchical team that box be checked off: it is a complex social interven-
culture that obstructs the open culture and communi- tion with an expectation of interaction and cooperation
cation required to execute the checklist correctly. The between surgeons, anaesthetists and nurses.63 66 75
social interaction between team members has a great However, this important aspect is often poorly
impact on nurses’ decision to participate in checklist addressed during implementation. Implementation
usage. They seem to adjust their team involvement teams should, therefore, promote and support inter-

6 Bergs J, et al. BMJ Qual Saf 2015;0:1–11. doi:10.1136/bmjqs-2015-004021


Downloaded from http://qualitysafety.bmj.com/ on July 26, 2015 - Published by group.bmj.com

Systematic review

professional communication when introducing the safety has shown that senior staff may not always be
checklist. If not, the checklist will be used as a tick-off the best source of patient safety knowledge and
exercise.79 skills.82 This suggests that surgeons not only need to
be supported but also to be educated. A promising
DISCUSSION approach to deal with the difficulties associated with
This systematic review comprehensively investigates changing routines can be found in team learning.83
user barriers and facilitators to the implementation of Successful implementers used enrolment to motivate
surgical safety checklists. Although the themes have the team, designed preparatory practice sessions and
been presented separately, implementation problems early trials to create psychological safety (ie, a shared
are multifactorial, highly interdependent and affected belief that the team is safe for interpersonal risk
by considerable clinical complexity. Many themes taking) and encourage new behaviours, and thus
were common to papers regardless of interstudy promote shared meaning and process improvement
differences in geographical or clinical setting. through reflective practices.83 84
It was found that the implementation of a surgical The local context in which the checklist has to be
safety checklist requires change in perception of the implemented is important. The organisational culture
operating theatre staff regarding the checklist and its —that is, a social-organisational phenomena, in terms
items, and the perception of patient safety in general. of behaviour or attitudes, that emerges from a
In addition, doctors and nurses need to adjust their common way of sense making, based on shared
workflows. These changes are impeded or advanced values, beliefs, assumptions and norms85—influences
by characteristics of the checklist, the implementation the behaviour and perception of doctors and nurses.
process and the local context. People often view themselves as members of a com-
Based on the experience of high-reliability organisa- munity of practice with established norms and pro-
tions, characteristics of the checklist (eg, length, cesses that can change only when the entire group
layout and design, and content) are important.44 The changes. For example, a new workflow practice or
content of the checklist needs to be supported by sci- technology standard may be difficult to adopt unless
entific evidence and written in clear, understandable the entire group agrees at the same time to use the
words preferably embedded within existing processes. system. To improve the safety culture in the operating
The checklist must precisely mirror the intended oper- theatre, interventions should aim at minimising the
ation without creating ambiguity or confusion. The hierarchy and empowering nursing staff in addition to
checklist and its items must be relevant to the applied standardising and structuring the practicalities con-
setting. Exporting a checklist to situations in which it cerning the use of the checklist. Such initiatives need
was not meant to be used may impede further check- to be performed by the operating theatre manage-
list implementation.63 Obstacles stemming from the ment.66 Second, leadership (ie, the process of social
checklist apply not only to the content but also to psy- influence in which one person can enlist the aid and
chological ownership. Doctors and nurses need to feel support of others in the accomplishment of a common
as though the checklist has become a part or an exten- task86) from senior staff and the chief surgeon has been
sion of their selves. In other words, they have to feel frequently presented as a key to successful implementa-
that it is ‘mine’ or ‘ours’.80 Even better is collective tion.25 62–64 66 68 69 72 In addition, participants found
psychological ownership, with the entire operating executive leadership equally important. It is important
theatre staff feeling that the checklist is part of them to show that patient safety is regarded as a priority in
and their work.81 the hospital.62 64 66 69 72 Finally, communication and
The implementation process itself can act as a teamwork have a profound influence on checklist
barrier and so create aversion. As with any new pro- usage. The existence of a professional hierarchy in
cedure or guideline implementation, the checklist medicine and the differential status accorded to those
creates uncertainty and questions. Lack of consensus in different disciplines hampers teamwork and com-
guidelines will lead to personal interpretations and munication. Profession-derived status is associated
enhance confusion. Therefore, the introduction needs with psychological safety, so it is important to have a
to be supported by clear guidelines on how, when and sense of confidence that the team will not embarrass,
who will execute the checklist. These guidelines need reject or punish someone for speaking up when
to be formalised in a written procedure, and the exe- noticing a real or potential safety problem.
cution of the checklist also needs to be demonstrated. Psychological safety is a key antecedent of speaking up
Small-scale tests can familiarise doctors and nurses and learning behaviour in healthcare teams.87 It is sug-
with the checklist and experiment with workflow gested that leader inclusiveness moderates the relation-
adjustments. Reactions to these tests can be used to ship between status and psychological safety.87
alter or clarify checklist guidelines and procedures. We found that the workflow introduced by the
Surgeons, who play the central role during the pro- checklist often collides with existing routines.61 63
cedure, are often seen as leaders in the operating This creates conflicts as existing workflows are estab-
theatre. Research on the broader context of patient lished to achieve different priorities (eg, efficiency or

Bergs J, et al. BMJ Qual Saf 2015;0:1–11. doi:10.1136/bmjqs-2015-004021 7


Downloaded from http://qualitysafety.bmj.com/ on July 26, 2015 - Published by group.bmj.com

Systematic review

productivity). The effect of disrupted routines has electronic database (MEDLINE) was consulted. To
previously been recognised during the implementation mitigate the potential loss of relevant articles, a cited
of new technology in hospitals.83 Conflicts between and citing reference study in Web of Science was con-
priorities compel doctors and nurses to balance using ducted. Second, assessing the quality of the included
the checklist against other priorities. When a conflict studies remains controversial in qualitative reviews. The
in priorities emerges, the perceptions of the operating QARI was selected because it is referred to as the most
theatre staff regarding patient safety drives their ultim- coherent instrument for evaluating the validity of quali-
ate decision whether or not to use the checklist. We tative research.34 Last, this review specifies a list of
found that stakeholders’ perspectives and motives themes that are believed to influence checklist imple-
towards the checklist often differ. These different mentation but does not specify the interactions
views create tension and aversion. In order for a between them.
checklist to be considered a priority, staff ’s percep- The current research also has some notable
tions and attitudes regarding it and patient safety in strengths. First, to the best of our knowledge, this is
general must be supportive. Therefore, the safety the first overview of qualitative research on the bar-
checklist will be of little value if disruptive attitudes riers and facilitating factors regarding the implementa-
and behaviours are not addressed.88 This is in line tion of checklists. The results of this study present the
with the establishment of a climate of safety (ie, barriers and facilitators that play during the imple-
shared perceptions among the staff concerning the mentation of many patient safety initiatives in health-
policies, procedures, practices and kinds of behaviours care and, therefore, present a valuable learning
that will be rewarded and supported with regard to opportunity. Second, the use of thematic synthesis
safety89 90). This requires that a good safety climate enabled the synthesis of the results of otherwise het-
must be established prior to, and during, implementa- erogeneous studies.
tion. New routines require adjustment and training.
An important feature of checklists is the combination of
checks ensuring adherence to proven practices (eg, CONCLUSIONS
administration of antibiotics and use of pulse oximeters) Implementation of a checklist requires structural
and other non-technical items (eg, team introductions changes in workflow of the operating theatre staff work-
and confirmation of procedures). The principal purpose flow, as well as in their perceptions regarding the check-
of these non-technical items is to promote specific list and patient safety in general. The required changes
aspects of teamwork, communication and situational are impeded or advanced by three main factors: the
awareness.72 However, education and training during checklist, the implementation process and the local
checklist implementation often neglects these non- context. However, the complex reality in which the
technical items. The importance of team learning and checklist is implemented requires an approach that
training in order to modify teamwork and communica- includes more than getting rid of the barriers and sup-
tion has been implemented with positive effects.91 porting facilitating factors. Implementation leaders must
The checklist is in essence a complex social inter- facilitate team learning to foster mutual understanding
vention aimed to improve communication and team- of the perspectives and motivation and the adaptation
work in a strictly hierarchical context. Even when of existing routines. This paper provides a pragmatic
initial perceptions and attitudes regarding the checklist overview of the constructs upon which theories,
are positive,92 it does not guarantee long-term hypothesising potential change strategies and interac-
improvement.93 The existing hierarchies and the tribal tions, can be developed and tested empirically.
affiliations of professional groups must be altered in
order to create psychological safety. Each member has Author affiliations
1
Faculty of Business Economics, Hasselt University, Hasselt,
to be allowed to take interpersonal risk by speaking Belgium
2
up if any concern about safety arises without being Faculty of Medicine and Life Sciences, Hasselt University,
afraid of being embarrassed, rejected or punished.87 94 Hasselt, Belgium
3
Department of Management, General Hospital AZ Delta,
Implementation is much more complex than addres- Roeselare, Belgium
4
sing the barriers and enabling the facilitators found in Belgian Health Care Knowledge Center (KCE), Brussels,
this review. Implementation is an ever-changing Belgium
5
Faculty of Medicine and Health Sciences, Ghent University,
process for change in one aspect can generate a reac- Ghent, Belgium
6
tion in one or several other aspects and thereby create Department of General Internal Medicine, Ghent University
a wholly new environment. Hence, it is not enough to Hospital, Ghent, Belgium
have a list of barriers and facilitating factors: we also Contributors Study concept and design: JB and DV. Acquisition
of data: JB, PS and AV. Collection, analysis and interpretation of
need to deal with the interaction between them. data: JB, FL, PS, AV and DV. Draft of the manuscript: JB. All
authors contributed substantially in reviewing the manuscript
Strengths and limitations of this study and approved the final version.
The results of this thematic synthesis should be inter- Funding Limburg Sterk Merk.
preted within certain limitations. First, only one Competing interests None declared.

8 Bergs J, et al. BMJ Qual Saf 2015;0:1–11. doi:10.1136/bmjqs-2015-004021


Downloaded from http://qualitysafety.bmj.com/ on July 26, 2015 - Published by group.bmj.com

Systematic review
Provenance and peer review Not commissioned; externally 19 Blot K, Bergs J, Vogelaers D, et al. Prevention of central
peer reviewed. line-associated bloodstream infections through quality
improvement interventions: a systematic review and
meta-analysis. Clin Infect Dis 2014;59:96–105.
REFERENCES 20 Borchard A, Schwappach DL, Barbir A, et al. A Systematic
1 Bergs J, Hellings J, Simons PAM, et al. Surgical safety review of the effectiveness, compliance, and critical factors for
checklists: an update. Acta Chir Belg 2014;114:219–24. implementation of safety checklists in surgery. Ann Surg
2 Haynes AB, Weiser TG, Berry WR, et al. A surgical safety 2012;265:925–33.
checklist to reduce morbidity and mortality in a global 21 Saturno PJ, Soria-Aledo V, Da Silva Gama ZA, et al.
population. N Engl J Med 2009;360:491–9. Understanding WHO surgical checklist implementation: tricks
3 Weiser TG, Haynes AB, Dziekan G, et al. Effect of a 19-item and pitfalls. An observational study. World J Surg
surgical safety checklist during urgent operations in a global 2014;38:287–95.
patient population. Ann Surg 2010;251:976–80. 22 Hannam JA, Glass L, Kwon J, et al. A prospective,
4 Sewell M, Adebibe M, Jayakumar P, et al. Use of the WHO observational study of the effects of implementation strategy
surgical safety checklist in trauma and orthopaedic patients. on compliance with a surgical safety checklist. BMJ Qual Saf
Int Orthop 2011;35:897–901. 2013;22:940–7.
5 Askarian M, Kouchak F, Palenik CJ. Effect of surgical safety 23 Smythe WR. Surgical Safety Checklist compliance: a job done
checklists on postoperative morbidity and mortality rates, poorly! J Am Coll Surg 2013;217:867–73.e1–3.
Shiraz, Faghihy Hospital, a 1-year study. Qual Manag Health 24 Morgan PJ, Cunningham L, Mitra S, et al. Surgical safety
Care 2011;20:293–7. checklist: implementation in an ambulatory surgical facility.
6 Bliss LA, Ross-Richardson CB, Sanzari LJ, et al. Thirty-day Can J Anaesth 2013;60:528–38.
outcomes support implementation of a surgical safety checklist. 25 Braaf S, Manias E, Riley R. The “time-out” procedure: an
J Am Coll Surg 2012;215:766–76. institutional ethnography of how it is conducted in actual
7 Kwok AC, Funk LM, Baltaga R, et al. Implementation of the clinical practice. BMJ Qual Saf 2013;22:647–55.
World Health Organization Surgical Safety Checklist, Including 26 Poon SJ, Zuckerman SL, Mainthia R, et al. Methodology and
Introduction of Pulse Oximetry, in a Resource-Limited Setting. bias in assessing compliance with a surgical safety checklist.
Ann Surg 2013;257:633–9. Jt Comm J Qual Patient Saf 2013;39:77–82.
8 de Vries EN, Prins HA, Crolla RMPH, et al. Effect of a 27 Rydenfält C, Johansson G, Odenrick P, et al. Compliance with
comprehensive surgical safety system on patient outcomes. the WHO surgical safety checklist: deviations and possible
N Engl J Med 2010;363:1928–37. improvements. Int J Qual Health Care 2013;25:182–7.
9 de Vries EN, Eikens-Jansen MP, Hamersma AM, et al. 28 Levy SM, Senter CE, Hawkins RB, et al. Implementing a
Prevention of surgical malpractice claims by use of a surgical surgical checklist: more than checking a box. Surgery
safety checklist. Ann Surg 2011;253:624–8. 2012;152:331–6.
10 Bergs J, Vandijck D. Systematic review and meta-analysis of the 29 Vogts N, Hannam JA, Mitchell SJ. Compliance and quality in
effect of the World Health Organization surgical safety administration of a Surgical Safety Checklist in a tertiary New
checklist on postoperative complications. Br J Surg Zealand hospital. N Z Med J 2011;124:48–58.http://eutils.ncbi.
2014;101:150–8. nlm.nih.gov/entrez/eutils/elink.fcgi?dbfrom=pubmed&id=
11 Urbach DR, Govindarajan A, Saskin R, et al. Introduction of 21963925&retmode=ref&cmd=prlinks
surgical safety checklists in Ontario, Canada. N Engl J Med 30 Thomas J, Harden A. Methods for the thematic synthesis of
2014;370:1029–38. qualitative research in systematic reviews. BMC Med Res
12 Reames BN, Krell RW, Campbell DA, et al. A checklist-based Methodol 2008;8:45.
intervention to improve surgical outcomes in michigan: 31 Shaw RL, Booth A, Sutton AJ, et al. Finding qualitative
evaluation of the keystone surgery program. JAMA Surg research: an evaluation of search strategies. BMC Med
2015;150:208–15. ResMethodol 2004;4:5–5.
13 Reames BN, Scally CP, Thumma JR, et al. Evaluation of the 32 Pearson M, Moxham T, Ashton K. Effectiveness of search
effectiveness of a surgical checklist in medicare patients. Med strategies for qualitative research about barriers and
Care 2015;53:87–94. facilitators of program delivery. Eval Health Prof
14 van Klei WA, Hoff RG, Van Aarnhem EEHL, et al. Effects of 2011;34:297–308.
the introduction of the WHO “Surgical Safety Checklist” on 33 Hannes K. Chapter 4: critical appraisal of qualitative research.
in-hospital mortality: a cohort study. Ann Surg 2012;255:44–9. In: Noyes J, Booth A, Hannes K, et al., eds. Supplementary
15 Vats A, Vincent CA, Nagpal K, et al. Practical challenges of guidance for inclusion of qualitative research in Cochrane
introducing WHO surgical checklist: UK pilot experience. BMJ Systematic Reviews of Interventions. Version 1 (updated August
2010;340:b5433. 2011). Cochrane Collaboration Qualitative Methods Group,
16 Morgan L, Pickering SP, Hadi M, et al. A combined teamwork 2011:1–14.
training and work standardisation intervention in operating 34 Hannes KK, Lockwood CC, Pearson AA. A comparative
theatres: controlled interrupted time series study. BMJ Qual Saf analysis of three online appraisal instruments’ ability to assess
2015;24:111–19. validity in qualitative research. Qual Health Res
17 Bergs J, Verelst S, Gillet J-B, et al. Evaluating implementation 2010;20:1736–43.
of the emergency severity index in a belgian hospital. J Emerg 35 The Joanna Briggs Institute. Joanna Briggs Institute Reviewers’
Nurs 2014;40:592–7. Manual: 2014 edition. The Joanna Briggs Institute, 2014.
18 Simons PAM, Houben RMA, Backes HH, et al. Compliance to 36 Dixon-Woods M, Bonas S, Booth A, et al. How can systematic
technical guidelines for radiotherapy treatment in relation to reviews incorporate qualitative research? A critical perspective.
patient safety. Int J Qual Health Care 2010;22:187–93. Qual Res 2006;6:27–44.

Bergs J, et al. BMJ Qual Saf 2015;0:1–11. doi:10.1136/bmjqs-2015-004021 9


Downloaded from http://qualitysafety.bmj.com/ on July 26, 2015 - Published by group.bmj.com

Systematic review
37 Daly J, Willis K, Small R, et al. A hierarchy of evidence for 54 Pickering SP, Robertson ER, Griffin D, et al. Compliance and
assessing qualitative health research. J Clin Epidemiol use of the World Health Organization checklist in U.K.
2006;60:43–9. operating theatres. Brit J Surg 2013;100:1664–70.
38 QSR International Pty Ltd. NVivo qualitative data analysis 55 Cullati S, Licker M-J, Francis P, et al. Implementation of the
software. surgical safety checklist in Switzerland and perceptions of its
39 Verdaasdonk EGG, Stassen LPS, Widhiasmara PP, et al. benefits: cross-sectional survey. PLoS ONE 2013;9:e101915.
Requirements for the design and implementation of checklists 56 van Schoten SM, Kop V, de Blok C, et al. Compliance with a
for surgical processes. Surg Endosc 2009;23:715–26. time-out procedure intended to prevent wrong surgery in
40 Paull DE, Mazzia LM, Izu BS, et al. Predictors of successful hospitals: results of a national patient safety programme in the
implementation of preoperative briefings and postoperative Netherlands. BMJ Open 2014;4:e005075.
debriefings after medical team training. Am J Surg 57 Norton EK, Rangel SJ. Implementing a pediatric surgical safety
2009;198:675–8. checklist in the OR and beyond. AORN J 2010;92:61–71.
41 Sivathasan N, Rakowski KRM, Robertson BFM, et al. The 58 Putnam LR, Levy SM, Sajid M, et al. Multifaceted
World Health Organization’s ‘Surgical Safety Checklist’: should interventions improve adherence to the surgical checklist.
evidence-based initiatives be enforced in hospital policy? JRSM Surgery 2014;156:336–44.
Short Rep 2010;1:40–0. 59 Patel J, Ahmed K, Guru KA, et al. An overview of the use and
42 Robinson LD, Paull DE, Mazzia LM, et al. The role of the implementation of checklists in surgical specialities—a
operating room nurse manager in the successful systematic review. Int J Surg 2014;12:1317–23.
implementation of preoperative briefings and postoperative 60 Sendlhofer G, Mosbacher N, Karina L, et al. Implementation
debriefings in the VHA Medical Team Training Program. of a surgical safety checklist: interventions to optimize the
J Perianesth Nurs 2010;25:302–6. process and hints to increase compliance. PLoS ONE 2014;10:
43 Weiser TG, Lipsitz SR, Dziekan G, et al. Changes in safety e0116926.
attitude and relationship to decreased postoperative morbidity 61 Lingard L, Espin S, Rubin B, et al. Getting teams to talk:
and mortality following implementation of a checklist-based development and pilot implementation of a checklist to
surgical safety intervention. BMJ Qual Saf 2010;20:102–7. promote interprofessional communication in the OR. Qual Saf
44 Thomassen O, Espeland A, Søfteland E, et al. Implementation Health Care 2005;14:340–6.
of checklists in health care; learning from high-reliability 62 Gillespie BM, Chaboyer W, Wallis M, et al. Why isn’t ‘time
organisations. Scand J Trauma Resusc Emerg Med out’ being implemented? An exploratory study. Qual Saf
2011;19:53–3. Health Care 2010;19:103–6.
45 Lingard L, Regehr G, Orser B, et al. Evaluation of a 63 Thomassen O, Brattebø G, Heltne J-K, et al. Checklists in the
preoperative checklist and team briefing among surgeons, operating room: Help or hurdle? A qualitative study on
nurses, and anesthesiologists to reduce failures in health workers’ experiences. BMC Health Serv Res 2010;10:
communication. Arch Surg 2008;143:12–17–discussion 18. 342–2.
46 Ali M, Osborne A, Bethune R, et al. Preoperative surgical 64 Conley DM, Singer SJ, Edmondson L, et al. Effective surgical
briefings do not delay operating room start times and are popular safety checklist implementation. J Am Coll Surg
with surgical team members. J Patient Saf 2011;7:139–43. 2011;212:873–9.
47 Calland JF. Surgical safety checklists are an effective means for 65 Cunat C, Flatin V, Viale J-P. Implementation strategy of the
reducing surgical morbidity and mortality, but have we gone HAS French surgical check-list in a university hospital. Ann Fr
far enough in team-building and leadership to create high Anesth Reanim 2011;30:484–8.
reliability? Evid Based Med 2013;18:235–6. 66 Wæhle HV, Søfteland E, Hjälmhult E. Adjusting team
48 Styer KA, Ashley SW, Schmidt I, et al. Implementing the World involvement: a grounded theory study of challenges in utilizing
Health Organization surgical safety checklist: a model for a surgical safety checklist as experienced by nurses in the
future perioperative initiatives. AORN journal 2011;94:590–8. operating room. BMC Nurs 2012;11:16.
49 Delgado Hurtado JJ, Jiménez X, Peñalonzo MA, et al. 67 Healy JM. How hospital leaders implemented a safe surgery
Acceptance of the WHO Surgical Safety Checklist among protocol in Australian hospitals. Int J Qual Health Care
surgical personnel in hospitals in Guatemala city. BMC Health 2012;24:88–94.
Serv Res 2012;12:169. 68 Fourcade A, Blache J-L, Grenier C, et al. Barriers to staff
50 Bragg K, Schlenk EA, Wolf G, et al. Time out! Surveying adoption of a surgical safety checklist. BMJ Qual Saf
surgical barriers. Nurs Manage 2012;43:38–44. 2012;21:191–7.
51 Helmio P, Takala A, Aaltonen L-M, et al. First year with WHO 69 O’Connor P, Reddin C, O’Sullivan M, et al. Surgical checklists:
Surgical Safety Checklist in 7148 otorhinolaryngological the human factor. Patient Saf Surg 2013;7:14.
operations: use and user attitudes. Clin Otolaryngol 70 Papaconstantinou HT, Jo C, Reznik SI, et al. Implementation
2012;37:305–8. of a surgical safety checklist: impact on surgical team
52 Sheena Y, Fishman JM, Nortcliff C, et al. Achieving flying perspectives. Ochsner J 2012;13:299–309.
colours in surgical safety: audit of World Health Organization 71 Kaderli R, Seelandt JC, Umer M, et al. Reasons for the
“Surgical Safety Checklist” compliance. J Laryngol Otol persistence of adverse events in the era of safer surgery--a
2012;126:1049–55. qualitative approach. Swiss Med Wkly 2013;143:w13882.
53 Haugen AS, Murugesh S, Haaverstad R, et al. A survey of 72 Aveling EL, McCulloch P, Dixon-Woods M. A qualitative study
surgical team members’ perceptions of near misses and comparing experiences of the surgical safety checklist in
attitudes towards Time Out protocols. BMC Surg hospitals in high-income and low-income countries. BMJ Open
2013;13:46–6. 2013;3:e003039.

10 Bergs J, et al. BMJ Qual Saf 2015;0:1–11. doi:10.1136/bmjqs-2015-004021


Downloaded from http://qualitysafety.bmj.com/ on July 26, 2015 - Published by group.bmj.com

Systematic review
73 Bashford T, Reshamwalla S, McAuley J, et al. Implementation 83 Edmondson AC, Bohmer RM, Pisano GP. Disrupted routines:
of the WHO Surgical Safety Checklist in an Ethiopian Referral team learning and new technology implementation in
Hospital. Patient Saf Surg 2014;8:16. hospitals. Adm Sci Q 2001;46:685–716.
74 Gagliardi AR, Straus SE, Shojania KG, et al. Multiple 84 Edmondson AC, Dillon JR, Roloff KS. 6 Three perspectives on
interacting factors influence adherence, and outcomes team learning. Acad Manag Ann 2007;1:269–314.
associated with surgical safety checklists: a qualitative study. 85 Parmelli E, Flodgren G, Beyer F, et al. The effectiveness
PLoS ONE 2014;9:e108585. of strategies to change organisational culture to improve
75 Rönnberg L, Nilsson U. Swedish nurse anesthetists’ experiences healthcare performance: a systematic review. Implement Sci
of the WHO surgical safety checklist. J Perianesthesia Nurs 2011;6:33.
2014. In press. doi:10.1016/j.jopan.2014.01.011 86 Chemers M. An integrative theory of leadership. Psychology
76 Russ SJ, Sevdalis N, Moorthy K, et al. A qualitative evaluation Press, 1997.
of the barriers and facilitators toward implementation of the 87 Nembhard IM, Edmondson AC. Making it safe: the effects of
WHO surgical safety checklist across hospitals in England: leader inclusiveness and professional status on psychological
lessons from the “Surgical Checklist Implementation Project.” safety and improvement efforts in health care teams. J Organiz
Ann Surg 2015;261:81–91. Behav 2006;27:941–66.
77 Haugen AS, Høyland S, Thomassen O, et al. “It”s a State of 88 Bosk CL, Dixon-Woods M, Goeschel CA, et al. Reality check
Mind’: a qualitative study after two years’ experience with the for checklists. Lancet 2008;374:444–5.
World Health Organization’s surgical safety checklist. Cogn 89 Zohar D. Safety climate and beyond: a multi-level
Tech Work 2015;17:55–62. multi-climate framework. Saf Sci 2007;46:376–87.
78 Lambrechts F, Grieten S, Bouwen R, et al. Process Consultation 90 Zohar D. Thirty years of safety climate research: reflections
Revisited: Taking a Relational Practice Perspective. J Appl Behav and future directions. Accid Anal Prev 2010;42:6–6.
Sci 2009;45:39–58. 91 Edmondson AC. Framing for learning: lessons in
79 Rydenfält C, Ek Å, Larsson PA. Safety checklist compliance successful technology implementation. Calif Rev
and a false sense of safety: new directions for research. 2003;45:34–54.
BMJ Qual Saf 2014;23:183–6. 92 Bohmer AB, Wappler F, Tinschmann T, et al. The
80 Pierce JL, Kostova T, Dirks KT. Toward a theory of implementation of a perioperative checklist increases patients’
psychological ownership in organizations. Acad Manag Rev perioperative safety and staff satisfaction. Acta Anaesthesiol
2001;26:298–310. Scand 2012;56:332–8.
81 Pierce JL, Jussila I. Collective psychological ownership within 93 Bohmer AB, Kindermann P, Schwanke U, et al. Long-term
the work and organizational context: construct introduction effects of a perioperative safety checklist from the viewpoint of
and elaboration. J Organiz Behav 2009;31:810–34. personnel. Acta Anaesthesiol Scand 2013;57:150–7.
82 Walton M, Barraclough B. Clinical supervisors: are they the 94 Edmondson AC. Pychological safety and learning behaviour in
key to making care safer? BMJ Qual Saf 2013;22:609–12. work teams. Adm Sci Q 1999;44:350–83.

Bergs J, et al. BMJ Qual Saf 2015;0:1–11. doi:10.1136/bmjqs-2015-004021 11


Downloaded from http://qualitysafety.bmj.com/ on July 26, 2015 - Published by group.bmj.com

Barriers and facilitators related to the


implementation of surgical safety checklists:
a systematic review of the qualitative
evidence
Jochen Bergs, Frank Lambrechts, Pascale Simons, Annemie Vlayen,
Wim Marneffe, Johan Hellings, Irina Cleemput and Dominique Vandijck

BMJ Qual Saf published online July 21, 2015

Updated information and services can be found at:


http://qualitysafety.bmj.com/content/early/2015/07/21/bmjqs-2015-00
4021

These include:

Supplementary Supplementary material can be found at:


Material http://qualitysafety.bmj.com/content/suppl/2015/07/21/bmjqs-2015-00
4021.DC1.html
References This article cites 89 articles, 21 of which you can access for free at:
http://qualitysafety.bmj.com/content/early/2015/07/21/bmjqs-2015-00
4021#BIBL
Email alerting Receive free email alerts when new articles cite this article. Sign up in the
service box at the top right corner of the online article.

Notes

To request permissions go to:


http://group.bmj.com/group/rights-licensing/permissions

To order reprints go to:


http://journals.bmj.com/cgi/reprintform

To subscribe to BMJ go to:


http://group.bmj.com/subscribe/

S-ar putea să vă placă și