Case 3

After the crisis: a systematic and critical review

From an early age Faisal had a keen interest in current affairs, keeping up to date with
significant stories in the news. During the first semester of his MSc in Management,
he became particularly concerned about the frequency and impact of large industrial
accidents reported by the media. A meeting with his project tutor confirmed for Faisal
that crisis management was a topic worthy of a research project. His project tutor
suggested Faisal make a start on the research by undertaking a literature review on
crisis management. Faisal had read a book chapter by Denyer and Tranfield (2009)
and a book by Petticrew and Roberts (2006) on Systematic Review. Faisal was
particularly taken with the idea of Systematic Review as it provided him with a step-
by-step process for his review. He produced a ten-step protocol (or plan) for his
review and shared it with his project tutor. She was impressed. She felt that he was
technically competent and was confident in his ability to perform the review.
1 To help focus the review he sought advice and feedback from a community of
academics and practitioners via a discussion forum on LinkedIn. The group also
informed Faisal that much of the focus for research within this field had been on the
antecedents of disasters or managing incidents. However, fewer studies had focused
on organisational learning and change management after the event. Faisal decided that
this would be an interesting focus for his project.
2 He developed a well-formulated, answerable initial review question: What are the
enablers and barriers to organisational learning and change after major industrial
accidents?
3 He then searched an online database, Business Source Complete, to locate literature
that could be used to answer the review question: setting the start date for his search
as 1980, he used the following search strings:
a crisis* OR disaster* OR accident* OR catastrophe* (string 1)
b change* OR learning* (string 2)
Limiting the search to peer-reviewed academic journals this resulted in 9563 articles,
which was clearly too many. Faisal sought the advice of his discussion forum on
LinkedIn. One particularly insightful suggestion was to focus on industrial accidents
rather than all crisis events, as his search also included natural disasters, which was
making the scope of the review unmanageable in the time available. Another forum
member encouraged Faisal to reduce the scope further by focusing on the role of
investigations and Inquiry reports. Faisal thought that this was a good idea so added a
third search string:
C investigation* OR inquiry* OR inquest* (string 3)
Adding this third string produced 319 potentially relevant publications. He felt that
this was manageable. To ensure that he was thorough, Faisal repeated the search
process with a second online database – Web of Knowledge. While there were many
duplications, the second database revealed a further 16 potentially relevant
publications.
4 Given the new focus, Faisal amended his review question to: What is the role of
investigations and/or public inquiries in facilitating or inhibiting organisational
learning and change after major industrial accidents? He then read the abstracts of all
335 papers to see whether or not they addressed this question. He compiled a list of
criteria to explain and justify his decision to include or exclude a publication. Faisal
thought that 98 were relevant.
5 Through scanning the reference sections (bibliography) of the 98 relevant
publications, he identified a further 16 relevant papers.
6 The online group also suggested that Faisal needed to review the ‘grey literature’,
meaning reports, theses, conference proceedings, and other documents not published
commercially. The process of searching for the grey literature seemed to Faisal to be
less systematic and was painstaking as he was required to search a number of websites
suggested to him by the online group. However, he realised that it was important
because he found 23 relevant documents.
7 Faisal used a quality appraisal checklist downloaded from the Critical Appraisals
Skills Programme (CASP n d) to help make sense of and evaluate the quality of the
different types of research he encountered in his reading. He decided to discard 18
texts because they were particularly weak. He documented his reasons for exclusion.
8 Citation information for all 119 relevant texts was uploaded to Endnote – citation
management software that was supplied by his university’s library. Full-text PDFs
were retrieved for all the papers and also uploaded to Endnote.
9 Faisal extracted data from each individual study and created three tables related to
his review question: (i) the roles played by investigations and public inquiries; (ii) the
facilitators of learning and change after major industrial accidents; (ii) the inhibitors
of learning and change after major industrial accidents.
10 Faisal wrote a findings section, which summarised what was known and not
known about the topic. This had three sections that corresponded to his three tables
(see 9) and described each of the papers in turn.
With much excitement he sent a draft to his tutor. A week later he met with his tutor
who said that his review was ‘too descriptive’ and he hadn’t engaged critically with
the literature. She told him that while it is important to adopt a thorough and
systematic approach so that nothing of relevance is missed, it is equally important for
reviewers to develop their own ideas and express them as a ‘storyline’. He had more
work to do to achieve a high grade. The next day Faisal thought about his tutor’s
feedback. He didn’t really understand what being critical meant. Surely it was
disrespectful to be critical, particularly of papers published by ‘experts’ in the field.
Fortunately, a fellow student lent him a copy of a book by Wallace and Wray (2011)
on critical reading and writing.
Faisal soon realised that a critical approach was not just about identifying flaws,
weaknesses and inconsistencies in texts but also about offering alternative insights,
ideas and explanations. Applying the critical synopsis tool from Wallace and Wray’s
book to some of the papers he had read, he noticed that certain authors were coming
from different perspectives, or schools of thought, and that they often made a number
of assumptions. For example, some authors like Perrow (1984) assumed that
industrial accidents were inevitable whereas other authors like Roberts (1990) argued
that organisations could be managed in ways that result in exceptional levels of safety
and reliability. Over time he became more sensitive to his own position and personal
beliefs about the topic. He also realised that reading and writing required imagination
and creativity. Just like some of the really interesting papers that he had read, Faisal
tried to look at things in a new way, shed fresh light on texts he was reading and
reveal previously unnoticed connections between ideas.
Emanuel decided to keep his original findings section. However, he added a
discussion section that built an argument. Faisal also rewrote his introduction section
to ensure that he told the reader about his significant ideas early so that he would
capture their interest and motivate them to read the whole document. In the
conclusion section he made certain that he told the reader the most important things
that had emerged from the review and why they were important. Faisal’s tutor thought
that the revised document was exceptional.

References
CASP (n.d.) Critical Appraisal Skills Programme. Available at: http://www.casp-
uk.net/ [Accessed 10 June 2014]
Denyer, D. and Tranfield, D. (2009) ‘Producing a Systematic Review’, in D.
Buchanan, M. Petticrew and H. Roberts (eds) Systematic Reviews in the Social
Sciences. A Practical Guide. Oxford: Blackwell, pp. 671–89.
Perrow, C. (1984) Normal Accidents. New York: Basic Books.
Roberts, K. H. (1990) ‘Managing high reliability organizations’, California
Management Review, Vol. 32, pp. 101–13.
Wallace, M. and Wray, A. (2011) Critical Reading and Writing for Postgraduates
(2nd edn). London: Sage.

Questions
1 a Why is it important to have a clear focus and a well-formulated review question?
b What problems might you encounter if your focus is too broad or too narrow?
c Why was it important for Faisal to revise his review question?

2 a What are the advantages and disadvantages of being systematic in your approach?
b What are the advantages and disadvantages of being critical in your approach?
c How did Faisal balance being both systematic and critical?

3 Faisal has decided to produce a case study of the Deepwater Horizon accident for
his research project. How can he use the findings of his Literature Review to inform
his subsequent data collection?