REVIEW
Prognostic Accuracy of the Quick Sequential Organ Failure Assessment
for Mortality in Patients With Suspected Infection
A Systematic Review and Meta-analysis
Shannon M. Fernando, MD, MSc; Alexandre Tran, MD; Monica Taljaard, PhD; Wei Cheng, PhD; Bram Rochwerg, MD, MSc;
Andrew J.E. Seely, MD, PhD; and Jeffrey J. Perry, MD, MSc
Background: The quick Sequential Organ Failure Assessment
(qSOFA) has been proposed for prediction of mortality in pa-
tients with suspected infection.
Purpose: To summarize and compare the prognostic accuracy
of qSOFA and the systemic inflammatory response syndrome
(SIRS) criteria for prediction of mortality in adult patients with
suspected infection.
Data Sources: Four databases from inception through Novem-
ber 2017.
Study Selection: English-language studies using qSOFA for
prediction of mortality (in-hospital, 28-day, or 30-day) in adult
patients with suspected infection in the intensive care unit (ICU),
emergency department (ED), or hospital wards.
Data Extraction: Two investigators independently extracted
data and assessed study quality using standard criteria.
Data Synthesis: Thirty-eight studies were included (n =
385 333). qSOFA was associated with a pooled sensitivity of
60.8% (95% CI, 51.4% to 69.4%) and a pooled specificity of
D espite substantial advances in treatment, sepsis re-
mains a major cause of morbidity and mortality
worldwide (1, 2). In the emergency department (ED),
patients with infection and sepsis are an enormous bur-
den: An estimated 850 000 visits occur annually in the
United States (3). Similarly, patients admitted to hospi-
tal wards with infection are at risk for clinical deteriora-
tion, sepsis or septic shock, and even death (4). Rapid
identification of patients with sepsis and early initiation
of treatment with fluids and antibiotics have been asso-
ciated with improved outcomes (5, 6). Clinicians previ-
ously used the systemic inflammatory response syn-
drome (SIRS) criteria to screen patients with infection
for development of sepsis, which is diagnosed if 2 or
more criteria are fulfilled (7). However, some have sug-
gested that these criteria are too nonspecific, given that
they are present in the vast majority of patients admit-
ted to the intensive care unit (ICU) (8). In addition, re-
cent evidence from a large multicenter database of pa-
tients admitted to the ICU with severe sepsis showed
that approximately 1 out of every 8 patients was nega-
tive for SIRS (9).
See also:
Editorial comment . . . . . . . . . . . . . . . . . . . . . . . . . 293
Web-Only
Supplement
266 © 2018 American College of Physicians
Downloaded From: https://annals.org/ by a University of South Dakota User on 07/25/2018
Annals of Internal Medicine
72.0% (CI, 63.4% to 79.2%) for mortality. The SIRS criteria were
associated with a pooled sensitivity of 88.1% (CI, 82.3% to
92.1%) and a pooled specificity of 25.8% (CI, 17.1% to 36.9%).
The pooled sensitivity of qSOFA was higher in the ICU popula-
tion (87.2% [CI, 75.8% to 93.7%]) than the non-ICU population
(51.2% [CI, 43.6% to 58.7%]). The pooled specificity of qSOFA
was higher in the non-ICU population (79.6% [CI, 73.3% to
84.7%]) than the ICU population (33.3% [CI, 23.8% to 44.4%]).
Limitation: Potential risk of bias in included studies due to
qSOFA interpretation and patient selection.
Conclusion: qSOFA had poor sensitivity and moderate specific-
ity for short-term mortality. The SIRS criteria had sensitivity supe-
rior to that of qSOFA, supporting their use for screening of pa-
tients and as a prompt for treatment initiation.
Primary Funding Source: Canadian Association of Emergency
Physicians. (PROSPERO: CRD42017075964)
Ann Intern Med. 2018;168:266-275. doi:10.7326/M17-2820 Annals.org
For author affiliations, see end of text.
This article was published at Annals.org on 6 February 2018.
Given these perceived shortcomings, the recent
Third International Consensus Definitions for Sepsis
and Septic Shock (Sepsis-3) Task Force eliminated SIRS
from the definition of sepsis (10). To rapidly risk-stratify
patients with suspected infection, the Sepsis-3 investi-
gators proposed the new quick Sequential (Sepsis-
related) Organ Failure Assessment (qSOFA) for the
identification of patients at high risk for in-hospital
death (11). The qSOFA uses 3 criteria: low blood pres-
sure (systolic blood pressure ≤100 mm Hg), increased
respiration rate (≥22 breaths/min), and altered mental
status (Glasgow Coma Scale score ≤14). Fulfillment of 2
or more criteria indicated increased risk for in-hospital
death among both ICU and non-ICU patients in the der-
ivation cohort. Given that qSOFA is relatively new, de-
bate is ongoing about its potential benefits compared
with the SIRS criteria (12–14). qSOFA has particularly
important implications in the ED and in hospital wards
(in the pre-ICU setting), where it may help to identify
patients at risk for future deterioration and serve as an
important research tool for inclusion in sepsis-related
studies. Therefore, better understanding of its diagnos-
tic accuracy in patients with suspected infection is cru-
cial. We conducted a systematic review and meta-
analysis to assess the prognostic value of qSOFA for
prediction of mortality (in-hospital, 28-day, or 30-day)
in adult patients with suspected infection in the ED,
hospital wards, and the ICU.
 Prognostic Accuracy of qSOFA for Mortality
METHODS
The primary objective of this study was to obtain
summary estimates of diagnostic performance (includ-
ing sensitivity and specificity) across studies of qSOFA
for prediction of mortality (in-hospital, 28-day, or 30-
day) in patients with suspected infection. For studies
that also provided data on SIRS, the secondary objec-
tive was to obtain summary estimates of diagnostic per-
formance (including sensitivity and specificity) across
studies of the SIRS criteria for prediction of mortality
(in-hospital, 28-day, or 30-day) in patients with sus-
pected infection.
The investigators structured the systematic review
according to the PRISMA (Preferred Reporting Items for
Systematic reviews and Meta-Analyses) guidelines (15),
the Cochrane Handbook for Systematic Reviews of Di-
agnostic Test Accuracy (16), and existing guidelines for
reviews of diagnostic accuracy (17). The study protocol
was registered with PROSPERO (CRD42017075964).
Data Sources and Searches
We followed previously cited recommendations for
search strategies to identify diagnostic accuracy studies
(18). We searched MEDLINE, PubMed, EMBASE, and
the Cochrane Database of Systematic Reviews for arti-
cles and conference proceedings published before 19
November 2017. An experienced health sciences li-
brarian assisted with the development of the search
strategy. The search was conducted using the terms
qSOFA, quick Sepsis-related organ failure assessment,
quick sequential organ failure assessment, and sepsis-3
(Supplement Figure 1, available at Annals.org). We
used Science Citation Index to retrieve reports citing
the relevant articles identified by our search and then
entered them into PubMed. We conducted further sur-
veillance searches using the previously described “Re-
lated Articles” feature (19) to identify additional reports.
For our gray literature search, we manually searched
the bibliographies of all included articles. We also
conducted searches of abstracts from major confer-
ences (Society of Critical Care Medicine, American Tho-
racic Society, American College of Emergency Physi-
cians, Society for Academic Emergency Medicine, and
Canadian Association of Emergency Physicians) from
2016 and 2017. Abstract authors were not contacted
for further information on study findings.
Study Selection
We included all English-language abstracts and
full-text articles describing retrospective and prospec-
tive observational studies, as well as randomized and
quasi-randomized controlled trials. We included stud-
ies that involved adult patients (aged ≥16 years) with
suspected infection; were conducted in the ED, hospi-
tal wards, or the ICU; and applied qSOFA for prediction
of mortality (in-hospital, 28-day, or 30-day). We ex-
cluded studies that evaluated all-cause mortality over
longer or unspecified follow-up periods. We also ex-
cluded case reports, case series, animal studies, pedi-
atric studies, and studies that evaluated qSOFA for a
composite outcome (such as the combination of either
Annals.org
Downloaded From: https://annals.org/ by a University of South Dakota User on 07/25/2018
REVIEW
in-hospital mortality or ICU admission). For studies that
used the same database, we included only the study
with the most patients. We required each study in the
meta-analysis to have a 2 × 2 table of true-positive,
false-positive, true-negative, and false-negative counts,
either extracted from the original article or calculated
from other reported information, such as declared sen-
sitivity and specificity. We contacted authors for 2 × 2
table counts in instances where values could not be
obtained from the reported data. If the corresponding
author did not respond after 3 attempts at contact, the
study was excluded.
Study data were collected using Covidence. Titles
were imported into Covidence directly from MEDLINE
and EMBASE, and duplicates were removed. In the first
phase, 2 reviewers (S.M.F. and A.T.) independently
screened the titles and abstracts of all identified stud-
ies. Disagreements were resolved by consensus, and
no third-party adjudication was necessary. In the sec-
ond phase, 2 reviewers (S.M.F. and A.T.) independently
applied the eligibility criteria to the full text of the se-
lected articles from the first phase and reported rea-
sons for exclusion. Disagreements were resolved by
consensus.
Data Extraction and Quality Assessment
One investigator (S.M.F.) independently collected
the following variables from the included articles: au-
thor information, year of publication, study design, eli-
gibility criteria (including definition of “suspected infec-
tion” when available), number of patients included,
mean or median age, and mortality (in-hospital, 28-day,
or 30-day). A data extraction sheet (Supplement Table
1, available at Annals.org) was used to minimize risk for
transcription errors. Two investigators (S.M.F. and A.T.)
independently collected true-positive, false-positive,
true-negative, and false-negative counts; the total num-
ber of survivors and cases; and sensitivity and specific-
ity of qSOFA and the SIRS criteria (when available). Dis-
agreements were resolved by consensus.
Two reviewers (S.M.F. and A.T.) independently as-
sessed risk of bias of the included studies by using the
QUADAS-2 (Quality Assessment of Diagnostic Accu-
racy Studies 2) tool (20). Disagreements were resolved
by consensus. The QUADAS-2 tool is recommended
primarily for evaluation of quality in studies related to
accuracy of diagnostic tests. The tool assesses 4 poten-
tial areas for bias and applicability of the research ques-
tion: 1) patient selection (bias was considered more
evident if the study had nonconsecutive enrollment, a
case– control study design, or inappropriate exclu-
sions), 2) index test (qSOFA) (bias was considered more
evident if the qSOFA results were interpreted without
explicit blinding to the outcome), 3) reference standard
(mortality) (bias was considered more evident if the ref-
erence standard could misclassify the target condition),
and 4) flow and timing (bias was considered more evi-
dent if not all patients had qSOFA applied using the
same criteria, if qSOFA was applied at an inappropriate
time interval before death, or if not all patients were
included in the analysis).
Annals of Internal Medicine • Vol. 168 No. 4 • 20 February 2018 267
 REVIEW Prognostic Accuracy of qSOFA for Mortality

Data Synthesis and Analysis Estimates of test accuracy were plotted in the ROC
We presented results of individual studies graphi- space together with the summary ROC curve. The anal-
cally by plotting sensitivity and specificity estimates on yses were conducted using the MetaDAS (version 1.3
1-dimensional forest plots (ordered by sensitivity) and [22]) macro in SAS (SAS Institute), as recommended by
on the receiver-operating characteristic (ROC) space to the Cochrane Handbook for Systematic Reviews of Di-
visually assess for heterogeneity. To pool the results, agnostic Test Accuracy (16).
we applied the hierarchical summary ROC (HSROC) Subgroup analyses were planned to further inves-
model (21) and obtained summary point estimates of tigate heterogeneity of studies using only ICU patients,
sensitivity and specificity and their associated confi- only patients outside the ICU (that is, in the ED and
dence bounds, as well as diagnostic odds ratios and hospital wards), only ED patients, in-hospital mortality,
likelihood ratios. The HSROC model appropriately in- and 28- or 30-day mortality. Sensitivity analyses were
corporates both within-study and between-study vari- also conducted by repeating the analyses after exclud-
ability by defining separate models for each type of ing abstracts, studies applying qSOFA only to specific
variability. Within-study variability is described using a populations (such as patients with febrile neutropenia
binomial distribution for the number of positive test re- or septic shock), and studies with high risk of bias.
sults as a function of patients' true mortality status, and The overall rating of confidence in pooled prog-
between-study variability allows the cutoff point and di- nostic accuracy estimates was assessed using the
agnostic accuracy parameters to vary between studies. GRADE (Grading of Recommendations Assessment,
Development and Evaluation) approach (23). Assess-
ments were based on the following criteria: risk of bias
Table 1. Characteristics of the 38 Included Studies* of the included studies, precision, consistency, direct-
Description Frequency, ness of the evidence, and risk of publication bias. Con-
n (%)† fidence in prognostic accuracy estimates was rated as
Continent high, moderate, low, or very low. A GRADE evidence
North America 14 (36.8) profile was created using the guideline development
Europe 9 (23.7) tool (gradepro.org).
Asia 9 (23.7)
Australia/Oceania 4 (10.5) Role of the Funding Source
Africa 2 (5.3) This study was funded through a Junior Investiga-
tor Grant from the Canadian Association of Emergency
Year of publication
2016 8 (21.1) Physicians, which had no involvement in the analysis or
2017 30 (78.9) writing of this manuscript.

Publication type
Full-text article 36 (94.7) RESULTS
Conference abstract 2 (5.3)
Search Results
Study design After screening titles and abstracts and completing
Prospective cohort 14 (36.8) full-text assessments (Supplement Figure 2, available at
Retrospective cohort 24 (63.2) Annals.org), we included 39 cohorts from 38 articles
Population
(11, 24 – 60). The original derivation article by Seymour
ICU 8 (20.5) and colleagues (11) contained 2 validation cohorts (1
ED only 18 (46.2) from the ICU and 1 from outside the ICU), and both
ED and hospital ward 9 (23.1) cohorts were analyzed separately in our analysis. Eight
Hospital ward only 4 (10.3)
studies investigated only the ICU population (11, 25,
Median sample size (IQR), n 1071 (240–6024) 32, 44, 45, 52, 54, 56), 19 investigated only the ED
population (26, 28, 30, 33, 36, 38 – 40, 43, 46, 48, 49,
Definition of “suspected infection” 51, 53, 55, 57– 60), and 4 included only patients admitted
Attending physician diagnosis 13 (34.2) to hospital wards (24, 27, 37, 50). Two abstracts were in-
of infection
Initiation of body fluid cultures 10 (26.3) cluded in the analysis (44, 59). Seventeen studies investi-
and/or antibiotic treatment gated the utility of qSOFA in specific populations, includ-
Symptom criteria 7 (18.4) ing patients with pneumonia only (28, 47, 53), older or
Chart review by investigator 3 (7.9) elderly patients (31, 36, 58), SIRS-positive patients (35,
ICD-9 coding 3 (7.9)
Unknown 2 (5.3)
51), patients in low-resource settings (41, 42), patients
with severe sepsis or septic shock (43, 55, 59), patients
Outcome with cirrhosis (50), and febrile neutropenic patients (46).
In-hospital mortality 28 (73.7) Three studies required contact with the corresponding
28-d mortality 5 (13.2)
30-d mortality 5 (13.2)
author in order to obtain 2 × 2 table counts (30, 53, 54).
ED = emergency department; ICD-9 = International Classification of Study Characteristics
Diseases, Ninth Revision; ICU = intensive care unit; IQR = interquartile Table 1 describes the 38 included studies, and
range.
* Percentages may not sum to 100 due to rounding. Supplement Table 2 (available at Annals.org) provides
† Unless otherwise indicated. more details on characteristics of the individual studies.
268 Annals of Internal Medicine • Vol. 168 No. 4 • 20 February 2018 Annals.org

Downloaded From: https://annals.org/ by a University of South Dakota User on 07/25/2018

 Prognostic Accuracy of qSOFA for Mortality REVIEW

Figure 1. Forest plots of sensitivity and specificity for qSOFA (top) and the SIRS criteria (bottom) in the 38 included studies
(39 cohorts).

Study, Year (Reference) TPs, n FPs, n FNs, n TNs, n Sensitivity (95% CI) Specificity (95% CI) Sensitivity (95% CI) Specificity (95% CI)
qSOFA
Khwannimit et al, 2017 (45) 1027 1054 18 251 0.98 (0.97–0.99) 0.19 (0.17–0.21)
Seymour et al, 2016 (11) (ICU) 1186 5381 103 1262 0.92 (0.90–0.93) 0.19 (0.18–0.20)
Haydar et al, 2017 (39) 20 96 2 80 0.91 (0.71–0.99) 0.45 (0.38–0.53)
Umemura et al, 2017 (56) 99 225 10 53 0.91 (0.84–0.96) 0.19 (0.15–0.24)
April et al, 2017 (25) 35 127 4 48 0.90 (0.76–0.97) 0.27 (0.21–0.35)
Finkelsztein et al, 2017 (32) 26 71 3 52 0.90 (0.73–0.98) 0.42 (0.33–0.52)
Burnham and Kollef, 2017 (27) 60 205 7 238 0.90 (0.80–0.96) 0.54 (0.49–0.58)
Huson et al, 2017 (41) 13 78 2 236 0.87 (0.60–0.98) 0.75 (0.70–0.80)
Johnson et al, 2016 (44) 788 4137 299 3834 0.72 (0.70–0.75) 0.48 (0.47–0.49)
Huson et al, 2017 (42) 76 112 30 240 0.72 (0.62–0.80) 0.68 (0.63–0.73)
Freund et al, 2017 (33) 52 166 22 639 0.70 (0.59–0.80) 0.79 (0.76–0.82)
Siddiqui et al, 2017 (54) 7 19 3 29 0.70 (0.35–0.93) 0.60 (0.45–0.74)
Churpek et al, 2017 (29) 1133 10 595 516 18 433 0.69 (0.66–0.71) 0.64 (0.63–0.64)
Hwang et al, 2017 (43) 144 573 67 611 0.68 (0.62–0.74) 0.52 (0.49–0.54)
Forward et al, 2017 (34) 17 73 8 63 0.68 (0.46–0.85) 0.46 (0.38–0.55)
Raith et al, 2017 (52) 22 758 76 853 11 311 72 156 0.67 (0.66–0.67) 0.48 (0.48–0.49)
Quinten et al, 2017 (51) 5 29 3 156 0.63 (0.24–0.91) 0.84 (0.78–0.89)
Giamarellos-Bourboulis et al, 2017 (35) 528 755 340 1813 0.61 (0.57–0.64) 0.71 (0.69–0.72)
LeGuen et al, 2018 (37) 11 31 8 47 0.58 (0.33–0.80) 0.60 (0.49–0.71)
Seymour et al, 2016 (11) (outside ICU) 1037 10 342 849 54 294 0.55 (0.53–0.57) 0.84 (0.84–0.84)
Sterling et al, 2017 (55) 21 116 18 115 0.54 (0.37–0.70) 0.50 (0.43–0.56)
Chen et al, 2016 (28) 291 271 256 823 0.53 (0.49–0.57) 0.75 (0.73–0.78)
Park et al, 2017 (49) 84 136 75 714 0.53 (0.45–0.61) 0.84 (0.81–0.86)
Henning et al, 2017 (40) 172 1042 161 6259 0.52 (0.46–0.57) 0.86 (0.85–0.87)
Piano et al, 2017 (50) 23 31 22 164 0.51 (0.36–0.66) 0.84 (0.78–0.89)
Ranzani et al, 2017 (53) 189 1071 187 4577 0.50 (0.45–0.55) 0.81 (0.80–0.82)
Williams et al, 2017 (60) 164 741 163 7803 0.50 (0.45–0.56) 0.91 (0.91–0.92)
Wang et al, 2017 (58) 18 31 18 50 0.50 (0.33–0.67) 0.62 (0.50–0.72)
Wang et al, 2016 (57) 56 60 75 286 0.43 (0.34–0.52) 0.83 (0.78–0.86)
Donnelly et al, 2017 (31) 67 228 96 2070 0.41 (0.33–0.49) 0.90 (0.89–0.91)
Moskowitz et al, 2017 (48) 465 2986 728 19 985 0.39 (0.36–0.42) 0.87 (0.87–0.87)
Guirgis et al, 2017 (38) 125 412 207 2553 0.38 (0.32–0.43) 0.86 (0.85–0.87)
Weigle et al, 2016 (59) 17 112 29 66 0.37 (0.23–0.52) 0.37 (0.30–0.45)
de Groot et al, 2017 (30) 46 278 97 1859 0.32 (0.25–0.40) 0.87 (0.85–0.88)
González Del Castillo et al, 2017 (36) 20 63 52 936 0.28 (0.18–0.40) 0.94 (0.92–0.95)
Amland and Sutariya, 2017 (24) 55 594 170 5173 0.24 (0.19–0.31) 0.90 (0.89–0.90)
Kolditz et al, 2016 (47) 81 562 280 8404 0.22 (0.18–0.27) 0.94 (0.93–0.94)
Kim et al, 2017 (46) 4 18 16 577 0.20 (0.06–0.44) 0.97 (0.95–0.98)
Askim et al, 2017 (26) 8 59 60 1408 0.12 (0.05–0.22) 0.96 (0.95–0.97)
0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1

SIRS criteria
Khwannimit et al, 2017 (45) 1030 1236 15 69 0.99 (0.98–0.99) 0.05 (0.04–0.07)
April et al, 2017 (25) 38 171 1 4 0.97 (0.87–1.00) 0.02 (0.01–0.06)
Haydar et al, 2017 (39) 21 167 1 10 0.95 (0.77–1.00) 0.06 (0.03–0.10)
Churpek et al, 2017 (29) 1547 25 550 102 3478 0.94 (0.93–0.95) 0.12 (0.12–0.12)
Freund et al, 2017 (33) 69 584 5 221 0.93 (0.85–0.98) 0.27 (0.24–0.31)
Finkelsztein et al, 2017 (32) 27 108 2 15 0.93 (0.77–0.99) 0.12 (0.07–0.19)
Raith et al, 2017 (52) 31 648 127 062 2382 21 882 0.93 (0.93–0.93) 0.15 (0.15–0.15)
Seymour et al, 2016 (11) (ICU) 1173 5514 116 1129 0.91 (0.89–0.93) 0.17 (0.16–0.18)
Siddiqui et al, 2017 (54) 9 27 1 21 0.90 (0.55–1.00) 0.44 (0.29–0.59)
Johnson et al, 2016 (44) 974 5978 113 1993 0.90 (0.88–0.91) 0.25 (0.24–0.26)
Ranzani et al, 2017 (53) 334 4432 42 1216 0.89 (0.85–0.92) 0.22 (0.20–0.23)
Henning et al, 2017 (40) 229 3159 47 3315 0.83 (0.78–0.87) 0.51 (0.50–0.52)
Weigle et al, 2016 (59) 38 167 8 11 0.83 (0.69–0.92) 0.06 (0.03–0.11)
Moskowitz et al, 2017 (48) 978 12 864 215 10 107 0.82 (0.80–0.84) 0.44 (0.43–0.45)
Donnelly et al, 2017 (31) 128 1264 35 1166 0.79 (0.71–0.85) 0.48 (0.46–0.50)
Williams et al, 2017 (60) 253 3923 74 4621 0.77 (0.72–0.82) 0.54 (0.53–0.55)
Forward et al, 2017 (34) 18 108 7 28 0.72 (0.51–0.88) 0.21 (0.14–0.28)
González Del Castillo et al, 2017 (36) 47 508 25 491 0.65 (0.53–0.76) 0.49 (0.46–0.52)
Seymour et al, 2016 (11) (outside ICU) 1207 22 623 679 42 013 0.64 (0.62–0.66) 0.65 (0.65–0.65)
Askim et al, 2017 (26) 42 620 26 770 0.62 (0.49–0.73) 0.55 (0.53–0.58)
Piano et al, 2017 (50) 23 62 22 133 0.51 (0.36–0.66) 0.68 (0.61–0.75)
0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1

FN = false-negative; FP = false-positive; ICU = intensive care unit; qSOFA = quick Sequential Organ Failure Assessment; SIRS = systemic inflamma-
tory response syndrome; TN = true-negative; TP = true-positive.

Annals.org Annals of Internal Medicine • Vol. 168 No. 4 • 20 February 2018 269

Downloaded From: https://annals.org/ by a University of South Dakota User on 07/25/2018

 REVIEW
Of the included studies, 36.8% were conducted in
North America (all in the United States), 23.7% were
conducted in Europe, and 23.7% were conducted in
Asia. Fourteen of the studies (36.8%) were prospective
cohort studies, and the remaining 24 (63.2%) were ret-
rospective cohort studies. No case– control studies or
randomized controlled trials were included. The me-
dian sample size for all included studies was 1071 pa-
tients (interquartile range, 240 to 6024 patients). Our
inclusion criteria specified that trials had to recruit pa-
tients on the basis of “suspected infection,” but they
varied in how they defined this. Thirteen studies in-
cluded patients on the basis of diagnosis of infection by
the attending physician (33, 36, 37, 43, 45, 51, 52, 54 –
58, 60). Ten additional studies (11, 24, 25, 27, 29, 30,
32, 42, 48, 49) required initiation of body fluid cultures,
antibiotic treatment (intravenous or parenteral), or
both. Seven studies used symptom criteria for the diag-
nosis (26, 28, 41, 46, 47, 50, 53); 3 used retrospective
chart review by study investigators (34, 39, 40); and 3
retrospectively identified patients by using codes from
the International Classification of Diseases, Ninth Revi-
sion (31, 38, 44). Two trials did not explicitly state how
they defined suspected infection (35, 59). In terms of
outcome measures, 28 studies (73.7%) analyzed in-
hospital mortality (11, 24, 25, 29 –34, 37– 42, 44, 45, 48 –
56, 58, 59), 5 (13.2%) evaluated 28-day mortality (28,
35, 43, 46, 57), and 5 (13.2%) examined 30-day mortal-
ity (26, 27, 36, 47, 60).
Quality Assessment
Quality assessments using QUADAS-2 criteria are
summarized in Supplement Figure 3 (available at
Annals.org). Thirty-three studies (86.8%) had unclear
risk of bias in use of the index test (qSOFA) because
whether the qSOFA values were interpreted without
knowledge of the results of the reference standard
(mortality) was not explicitly stated. Twenty-one studies
had potential high risk of bias for applicability in patient
selection and were therefore excluded in a sensitivity
analysis. Sixteen of these studies focused on applica-
tion of qSOFA in specific populations, whereas the tool
was derived for application in all patients presenting
with suspected infection (11).
Results of Synthesis
Primary Analysis: Overall Accuracy
Figure 1 shows forest plots of the sensitivity and
specificity of qSOFA and the SIRS criteria reported in
the 38 included studies. Summary estimates of all diag-
nostic accuracy measures from the HSROC model are
shown in Table 2. The pooled sensitivity of qSOFA
across all included studies was 60.8% (95% CI, 51.4% to
69.4%), and the specificity was 72.0% (CI, 63.4% to
79.2%). The estimated diagnostic odds ratio for qSOFA
was 3.98 (CI, 3.22 to 4.92). The pooled estimates of
positive and negative likelihood ratios were 2.17 (CI,
1.82 to 2.58) and 0.55 (CI, 0.47 to 0.63), respectively.
The GRADE evidence profile for pooled prognostic ac-
curacy results of qSOFA is displayed in Supplement Ta-
ble 3 (available at Annals.org).
270 Annals of Internal Medicine • Vol. 168 No. 4 • 20 February 2018
Downloaded From: https://annals.org/ by a University of South Dakota User on 07/25/2018
Prognostic Accuracy of qSOFA for Mortality
The pooled sensitivity of the SIRS criteria across all
included studies was 88.1% (CI, 82.3% to 92.1%), and
the specificity was 25.8% (CI, 17.1% to 36.9%). The
HSROC model calculated a diagnostic odds ratio of
2.57 (CI, 2.12 to 3.11). The pooled positive and nega-
tive likelihood ratios were 1.19 (CI, 1.09 to 1.29) and
0.46 (CI, 0.40 to 0.54), respectively. The HSROC curves,
together with bivariate summary points of specificity
and sensitivity and their 95% confidence regions
for both qSOFA and the SIRS criteria, are shown in
Figure 2.
Subgroup and Sensitivity Analyses
The results of the subgroup and sensitivity analyses
to examine the prognostic accuracy of qSOFA and the
SIRS criteria in selected populations are presented in
Table 2. Forest plots and HSROC curves for the analy-
ses (ICU patients only, non-ICU patients only, ED pa-
tients only, studies reporting in-hospital mortality, stud-
ies reporting 28- or 30-day mortality, studies with high
risk of bias excluded, conference abstracts excluded,
and studies with qSOFA application only in specific
populations excluded) are presented in Supplement
Figures 4 to 11 (available at Annals.org). In the ICU
population, the pooled sensitivity was 87.2% (CI, 75.8%
to 93.7%) for qSOFA and 93.9% (CI, 88.5% to 96.8%)
for the SIRS criteria. The pooled specificity of qSOFA
was 33.3% (CI, 23.8% to 44.4%), compared with 13.0%
(CI, 6.6% to 23.8%) for the SIRS criteria. The pooled
positive and negative likelihood ratios for qSOFA in this
population were 1.31 (CI, 1.19 to 1.43) and 0.39 (CI,
0.25 to 0.61), respectively. The SIRS criteria had a
pooled positive likelihood ratio of 1.08 (CI, 1.02 to
1.14) and a pooled negative likelihood ratio of 0.47 (CI,
0.45 to 0.50).
The subgroup analysis of non-ICU patients in-
cluded 31 cohorts for qSOFA and 14 for the SIRS crite-
ria. The pooled sensitivity was 51.2% (CI, 43.6% to
58.7%) for qSOFA and 82.2% (CI, 74.5% to 87.9%) for
the SIRS criteria. The pooled specificity was 79.6% (CI,
73.3% to 84.7%) for qSOFA and 34.2% (CI, 22.6% to
48.0%) for the SIRS criteria. The pooled positive and
negative likelihood ratios for qSOFA were 2.50 (CI,
2.06 to 3.03) and 0.61 (CI, 0.55 to 0.69), respectively. In
contrast, the SIRS criteria had a positive likelihood ratio
of 1.25 (CI, 1.09 to 1.43) and a negative likelihood ratio
of 0.52 (CI, 0.42 to 0.65).
The subgroup analyses for ED patients found a
pooled sensitivity of 46.7% (CI, 38.3% to 55.2%) and
specificity of 81.3% (CI, 72.8% to 87.5%) for qSOFA. In
contrast, the SIRS criteria had a pooled sensitivity of
83.6% (CI, 75.9% to 89.1%) and a specificity of 30.6%
(CI, 17.7% to 47.5%). The sensitivity analyses excluding
studies with high risk of bias found an overall sensitivity
of 69.6% (CI, 54.8% to 81.2%) and specificity of 61.5%
(CI, 46.6% to 74.5%) for qSOFA and an overall sensitiv-
ity of 91.7% (CI, 85.3% to 95.4%) and specificity of
18.4% (CI, 9.6% to 32.5%) for the SIRS criteria.
Annals.org
 Prognostic Accuracy of qSOFA for Mortality REVIEW

Table 2. Overall, Subgroup, and Sensitivity Analyses of Summary Estimates for qSOFA and the SIRS Criteria

Variable qSOFA SIRS Criteria

Cohorts (patients), n (n)
Overall 39 (385 333) 21 (352 571)
ICU patients only 8 (203 229) 7 (202 738)
Patients outside ICU only 31 (182 104) 14 (149 833)
ED patients only 19 (61 894) 9 (49 640)
In-hospital mortality 29 (356 455) 18 (341 171)
28- or 30-d mortality 10 (28 878) 3 (11 400)
Abstracts excluded 37 (376 051) 19 (343 289)
Studies applying qSOFA to specific populations excluded 25 (369 881) 16 (342 419)
Studies with high risk of bias excluded 17 (339 042) 10 (324 101)

Sensitivity (95% CI), %

Overall 60.8 (51.4–69.4) 88.1 (82.3–92.1)
ICU patients only 87.2 (75.8–93.7) 93.9 (88.5–96.8)
Patients outside ICU only 51.2 (43.6–58.7) 82.2 (74.5–87.9)
ED patients only 46.7 (38.3–55.2) 83.6 (75.9–89.1)
In-hospital mortality 66.4 (56.6–75.1) 90.0 (84.9–93.5)
28- or 30-d mortality 43.2 (27.7–60.1) 70.0 (60.7–77.8)
Abstracts excluded 61.0 (51.2–70.0) 87.8 (81.3–92.3)
Studies applying qSOFA to specific populations excluded 64.2 (51.4–75.2) 90.1 (84.5–93.8)
Studies with high risk of bias excluded 69.6 (54.8–81.2) 91.7 (85.3–95.4)

Specificity (95% CI), %

Overall 72.0 (63.4–79.2) 25.8 (17.1–36.9)
ICU patients only 33.3 (23.8–44.4) 13.0 (6.6–23.8)
Patients outside ICU only 79.6 (73.3–84.7) 34.2 (22.6–48.0)
ED patients only 81.3 (72.8–87.5) 30.6 (17.7–47.5)
In-hospital mortality 65.5 (55.7–74.2) 22.2 (14.0–33.3)
28- or 30-d mortality 86.0 (74.8–92.7) 53.1 (50.4–55.7)
Abstracts excluded 73.3 (64.7–80.4) 27.2 (17.7–39.5)
Studies applying qSOFA to specific populations excluded 70.3 (58.4–80.0) 23.1 (14.2–35.2)
Studies with high risk of bias excluded 61.5 (46.6–74.5) 18.4 (9.6–32.5)

Diagnostic odds ratio (95% CI)

Overall 3.977 (3.216–4.919) 2.565 (2.115–3.110)
ICU patients only 3.391 (2.127–5.407) 2.279 (2.099–2.475)
Patients outside ICU only 4.073 (3.230–5.136) 2.395 (1.725–3.325)
ED patients only 3.795 (2.695–5.346) 2.248 (1.352–3.739)
In-hospital mortality 3.758 (2.872–4.918) 2.568 (2.150–3.068)
28- or 30-d mortality 4.658 (3.436–6.314) 2.632 (1.685–4.111)
Abstracts excluded 4.294 (3.582–5.147) 2.694 (2.279–3.185)
Studies applying qSOFA to specific populations excluded 4.251 (3.440–5.254) 2.727 (2.294–3.243)
Studies with high risk of bias excluded 3.663 (2.891–4.640) 2.483 (2.119–2.911)

Positive likelihood ratio (95% CI)

Overall 2.168 (1.820–2.582) 1.187 (1.090–1.293)
ICU patients only 1.307 (1.193–1.432) 1.079 (1.022–1.139)
Patients outside ICU only 2.501 (2.063–3.033) 1.248 (1.087–1.434)
ED patients only 2.491 (1.858–3.339) 1.205 (1.013–1.434)
In-hospital mortality 1.926 (1.607–2.308) 1.157 (1.067–1.254)
28- or 30-d mortality 3.080 (2.242–4.231) 1.490 (1.287–1.726)
Abstracts excluded 2.285 (1.927–2.709) 1.206 (1.098–1.326)
Studies applying qSOFA to specific populations excluded 2.164 (1.757–2.665) 1.171 (1.071–1.281)
Studies with high risk of bias excluded 1.809 (1.473–2.222) 1.124 (1.030–1.226)

Negative likelihood ratio (95% CI)

Overall 0.545 (0.470–0.633) 0.463 (0.398–0.537)
ICU patients only 0.385 (0.245–0.606) 0.473 (0.449–0.499)
Patients outside ICU only 0.614 (0.549–0.687) 0.521 (0.417–0.652)
ED patients only 0.656 (0.585–0.737) 0.536 (0.376–0.765)
In-hospital mortality 0.512 (0.425–0.618) 0.450 (0.396–0.513)
28- or 30-d mortality 0.661 (0.538–0.813) 0.566 (0.419–0.766)
Abstracts excluded 0.532 (0.456–0.620) 0.448 (0.392–0.511)
Studies applying qSOFA to specific populations excluded 0.509 (0.415–0.625) 0.430 (0.379–0.487)
Studies with high risk of bias excluded 0.494 (0.387–0.630) 0.453 (0.410–0.500)
ED = emergency department; ICU = intensive care unit; qSOFA = quick Sequential Organ Failure Assessment; SIRS = systemic inflammatory
response syndrome.

Annals.org Annals of Internal Medicine • Vol. 168 No. 4 • 20 February 2018 271

Downloaded From: https://annals.org/ by a University of South Dakota User on 07/25/2018

 REVIEW Prognostic Accuracy of qSOFA for Mortality

Figure 2. Hierarchical summary receiver-operating (25.8%). Of note, qSOFA had higher sensitivity in ICU
characteristic curves and bivariate summary points of
populations (87.2%) than non-ICU populations (51.2%)
but lower specificity in ICU populations (33.3% vs. 79.6%).
(specificity, sensitivity) and their 95% confidence regions
This extensive review of the literature provides the
for qSOFA (top) and the SIRS criteria (bottom).
best available assessment of the prognostic accuracy
of the novel qSOFA tool. Extensive searches using
qSOFA PubMed, EMBASE, MEDLINE, and the Cochrane Data-
1
base of Systematic Reviews (from inception to 19 No-
vember 2017) did not reveal any existing systematic
0.9
reviews or meta-analyses on the prognostic accuracy of
0.8
qSOFA. A search of ClinicalTrials.gov (from inception to
19 November 2017) found 2 ongoing prospective ob-
0.7 servational studies (from China and France) and 3 com-
pleted prospective observational trials (from France,
0.6 Belgium, and the Latin American Sepsis Institute) on
Sensitivity

the prognostic accuracy of qSOFA in patients with sus-

0.5 pected infection. These studies will provide further ev-
idence on the accuracy of this tool in various settings.
0.4
Identification of patients who are at risk for sepsis
and deterioration from infection is necessary for timely
0.3
treatment. Previous evidence on use of the SIRS criteria
0.2
for this purpose has shown that they can be sensitive in
detection while also identifying patients with greater
0.1 sepsis severity and higher risk for death (61– 63). The
SIRS criteria are commonly used in the pre-ICU setting
0 for initiation of sepsis treatment, and this has translated
0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0
into lower mortality among ED patients (5, 64). Since
Specificity
the publication of the Sepsis-3 definitions, clinicians
1
SIRS Criteria have been tempted to use qSOFA in a similar fashion.
When a screening tool for initiation of treatment is be-
0.9 ing chosen, high sensitivity is important (particularly in
outpatient settings) to avoid missing potential cases.
0.8 Our results show that the SIRS criteria are probably supe-
rior to qSOFA for screening patients with suspected infec-
0.7 tion for initiation of appropriate treatment. The sensitivity
of qSOFA for predicting mortality was poor, and use of
0.6 this tool for screening would likely miss many cases.
Sensitivity

As stated by the Sepsis-3 Task Force, qSOFA was

0.5
not created to replace the SIRS criteria as a screening
tool or as a prompt for initiation of treatment. Rather, it
0.4
was designed as an early warning risk stratification tool
0.3
for identification and escalation of care in patients at
high risk for death (12). A major limitation of the SIRS
0.2 criteria is that they are highly nonspecific (8, 65); how-
ever, our analysis of qSOFA showed that it had only
0.1 moderate specificity for prediction of mortality. This
was confirmed in subgroup analyses of patients outside
0
0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0
the ICU (in the ED or hospital wards) and patients in the
Specificity ED alone. These results were consistent with those of
another large cohort study comparing qSOFA with
qSOFA = quick Sequential Organ Failure Assessment; SIRS = systemic other commonly used early warning tools, which found
inflammatory response syndrome.
that qSOFA had the lowest accuracy in predicting mor-
tality (29). Finally, it is important to note that qSOFA
DISCUSSION involves criteria (particularly hypotension) that are in-
We performed a systematic review and meta- dicative of existing decompensation rather than future
analysis to evaluate the prognostic capability of qSOFA deterioration. Thus, its poor sensitivity limits its applica-
for predicting mortality (in-hospital, 28-day, or 30-day) bility compared with physician judgment for the dispo-
in adult patients with suspected infection. Overall, sition of initially stable patients, particularly in the ED
qSOFA was poorly sensitive (60.8%) and moderately (14). This is shown in Supplement Tables 4 and 5 (avail-
specific (72.0%) for prediction of mortality. The SIRS cri- able at Annals.org), which depict the influence of
teria had better sensitivity (88.1%) but worse specificity qSOFA and the SIRS criteria in deriving posttest prob-
272 Annals of Internal Medicine • Vol. 168 No. 4 • 20 February 2018 Annals.org

Downloaded From: https://annals.org/ by a University of South Dakota User on 07/25/2018

 Prognostic Accuracy of qSOFA for Mortality
ability of death from physician-determined pretest
probability in ICU and non-ICU patients, respectively.
For example, in ED patients with pretest probabilities of
death of 50% and 75%, the risks are 38% and 65%,
respectively, with a negative qSOFA result.
We also found that the performance of qSOFA var-
ied with the population in which it was applied. In par-
ticular, qSOFA was far more sensitive but far less spe-
cific in ICU patients than non-ICU patients. The reason
for this is unclear. There may be fundamental differ-
ences in test characteristics between patients outside
the ICU (with decompensation before resuscitation)
and those in the ICU (typically with persistent decom-
pensation after resuscitation). For example, systolic
blood pressure and respiratory rate in ICU patients may
be influenced by vasoactive medications and mechan-
ical ventilation, whereas mental status may be influ-
enced by sedative agents or persistent shock. Under-
standing how qSOFA performs in each setting is critical
when applying it to specific groups of patients.
This review has several strengths. It included a
comprehensive search of multiple databases, clear in-
clusion and exclusion criteria, and multiple subgroup
and sensitivity analyses. However, there are important
limitations, primarily related to the quality of the in-
cluded studies. First, many studies did not mention
whether the qSOFA score was calculated by persons
who were blinded to the outcome, which is a potential
source of bias. Second, many studies applied qSOFA
only to specific populations, such as febrile neutro-
penic patients or patients with pneumonia (27, 28, 31,
36, 37, 41– 43, 46, 50, 53, 58, 59), but the tool was
derived for application in all adult patients with sus-
pected infection (11). We performed a sensitivity anal-
ysis in which we removed these studies, but the conclu-
sions did not change. Third, the definition of suspected
infection varied among studies. Although this variation
has the potential to introduce bias, early evidence on
the predictive accuracy of qSOFA showed that it was
not significantly affected by different definitions of sus-
pected infection (66). Finally, there was variation in the
primary outcome examined. Most studies (including
the one in which the tool was derived [11]) used in-
hospital mortality, but some used 28- or 30-day mortality.
The sensitivity of qSOFA was substantially lower in these
studies, which decreased its overall pooled sensitivity.
Although we aimed to be inclusive in our search
strategy, the overall variability of the included studies
highlights the heterogeneous conditions under which
qSOFA has been applied in the literature. For this rea-
son, we performed multiple subgroup and sensitivity
analyses. Although summary estimates differed among
these analyses, the vast majority were in keeping with
the overall summary estimate for the accuracy of
qSOFA. Therefore, the prognostic performance of
qSOFA should be viewed not as a discrete point but as
a general concept with a range of performance be-
tween ideal and pragmatic settings.
In conclusion, our systematic review and meta-
analysis found that qSOFA was poorly sensitive and
moderately specific, whereas the SIRS criteria were
Annals.org
Downloaded From: https://annals.org/ by a University of South Dakota User on 07/25/2018
REVIEW
more sensitive but poorly specific. Our findings support
continued use of the SIRS criteria for screening of
patients with infection who are at risk for future deteri-
oration. qSOFA was more sensitive among ICU patients
than non-ICU patients, possibly due to differences in
test application among ICU patients.
From University of Ottawa, and Ottawa Hospital Research In-
stitute, Ottawa, and McMaster University, Hamilton, Ontario,
Canada.
Disclosures: Dr. Seely reports that he founded and holds eq-
uity in Therapeutic Monitoring Systems and has 2 patents is-
sued to the company. Authors not named here have disclosed
no conflicts of interest. Disclosures can also be viewed at www
.acponline.org/authors/icmje/ConflictOfInterestForms.do?ms
Num=M17-2820.
Grant Support: By a Junior Investigator Grant from the Cana-
dian Association of Emergency Physicians (Dr. Fernando).
Reproducible Research Statement: Study protocol: Registered
at PROSPERO (CRD42017075964). Statistical code: The Meta-
DAS SAS macro (recommended by the Cochrane Handbook
for Systematic Reviews of Diagnostic Test Accuracy [22]) is
available at http://methods.cochrane.org/sdt/software-meta
-analysis-dta-studies and is also available from Dr. Fernando
(e-mail, sfernando@qmed.ca). Data set: See the tables, fig-
ures, and Supplement.
Requests for Single Reprints: Shannon M. Fernando, MD,
MSc, Department of Emergency Medicine and Division of
Critical Care, Department of Medicine, University of Ottawa,
The Ottawa Hospital, Civic Campus, Room EM-206, 1053
Carling Avenue, Ottawa, Ontario K1Y 4E9, Canada; e-mail,
sfernando@qmed.ca.
Current author addresses and author contributions are avail-
able at Annals.org.
References
1. Liu V, Escobar GJ, Greene JD, Soule J, Whippy A, Angus DC, et al.
Hospital deaths in patients with sepsis from 2 independent cohorts.
JAMA. 2014;312:90-2. [PMID: 24838355]
2. Fleischmann C, Scherag A, Adhikari NK, Hartog CS, Tsaganos T,
Schlattmann P, et al; International Forum of Acute Care Trialists. As-
sessment of global incidence and mortality of hospital-treated sep-
sis. Current estimates and limitations. Am J Respir Crit Care Med.
2016;193:259-72. [PMID: 26414292] doi:10.1164/rccm.201504
-0781OC
3. Wang HE, Jones AR, Donnelly JP. Revised national estimates of
emergency department visits for sepsis in the United States. Crit
Care Med. 2017;45:1443-9. [PMID: 28817480] doi:10.1097/CCM
.0000000000002538
4. Bhattacharjee P, Edelson DP, Churpek MM. Identifying patients
with sepsis on the hospital wards. Chest. 2017;151:898-907. [PMID:
27374948] doi:10.1016/j.chest.2016.06.020
5. Seymour CW, Gesten F, Prescott HC, Friedrich ME, Iwashyna TJ,
Phillips GS, et al. Time to treatment and mortality during mandated
emergency care for sepsis. N Engl J Med. 2017;376:2235-44. [PMID:
28528569] doi:10.1056/NEJMoa1703058
6. Levy MM, Rhodes A, Phillips GS, Townsend SR, Schorr CA,
Beale R, et al. Surviving Sepsis Campaign: association between
performance metrics and outcomes in a 7.5-year study. Crit
Annals of Internal Medicine • Vol. 168 No. 4 • 20 February 2018 273
 REVIEW
Care Med. 2015;43:3-12. [PMID: 25275252] doi:10.1097/CCM
.0000000000000723
7. Bone RC, Balk RA, Cerra FB, Dellinger RP, Fein AM, Knaus WA,
et al. Definitions for sepsis and organ failure and guidelines for the
use of innovative therapies in sepsis. The ACCP/SCCM Consensus
Conference Committee. American College of Chest Physicians/Soci-
ety of Critical Care Medicine. Chest. 1992;101:1644-55. [PMID:
1303622]
8. Sprung CL, Sakr Y, Vincent JL, Le Gall JR, Reinhart K, Ranieri VM,
et al. An evaluation of systemic inflammatory response syndrome
signs in the Sepsis Occurrence In Acutely Ill Patients (SOAP) study.
Intensive Care Med. 2006;32:421-7. [PMID: 16479382]
9. Kaukonen KM, Bailey M, Pilcher D, Cooper DJ, Bellomo R. Sys-
temic inflammatory response syndrome criteria in defining severe
sepsis. N Engl J Med. 2015;372:1629-38. [PMID: 25776936] doi:10
.1056/NEJMoa1415236
10. Singer M, Deutschman CS, Seymour CW, Shankar-Hari M, An-
nane D, Bauer M, et al. The Third International Consensus Definitions
for Sepsis and Septic Shock (Sepsis-3). JAMA. 2016;315:801-10.
[PMID: 26903338] doi:10.1001/jama.2016.0287
11. Seymour CW, Liu VX, Iwashyna TJ, Brunkhorst FM, Rea TD, Sch-
erag A, et al. Assessment of clinical criteria for sepsis: for the Third
International Consensus Definitions for Sepsis and Septic Shock
(Sepsis-3). JAMA. 2016;315:762-74. [PMID: 26903335] doi:10.1001
/jama.2016.0288
12. Vincent JL, Martin GS, Levy MM. qSOFA does not replace SIRS in
the definition of sepsis. Crit Care. 2016;20:210. [PMID: 27423462]
doi:10.1186/s13054-016-1389-z
13. Marik PE, Taeb AM. SIRS, qSOFA and new sepsis definition [Ed-
itorial]. J Thorac Dis. 2017;9:943-5. [PMID: 28523143] doi:10.21037
/jtd.2017.03.125
14. Lamontagne F, Harrison DA, Rowan KM. qSOFA for identifying
sepsis among patients with infection [Editorial]. JAMA. 2017;317:
267-8. [PMID: 28114531] doi:10.1001/jama.2016.19684
15. Moher D, Liberati A, Tetzlaff J, Altman DG; PRISMA Group. Pre-
ferred reporting items for systematic reviews and meta-analyses: the
PRISMA statement. BMJ. 2009;339:b2535. [PMID: 19622551] doi:10
.1136/bmj.b2535
16. Deeks JJ, Bossuyt PM, Gatsonis C. Cochrane Handbook for Sys-
tematic Reviews of Diagnostic Test Accuracy. Version 1.0.0. 2013.
Accessed at http://srdta.cochrane.org on 19 December 2017.
17. Leeflang MM, Deeks JJ, Gatsonis C, Bossuyt PM; Cochrane Di-
agnostic Test Accuracy Working Group. Systematic reviews of diag-
nostic test accuracy. Ann Intern Med. 2008;149:889-97. [PMID:
19075208]
18. Beynon R, Leeflang MM, McDonald S, Eisinga A, Mitchell RL,
Whiting P, et al. Search strategies to identify diagnostic accuracy
studies in MEDLINE and EMBASE. Cochrane Database Syst Rev.
2013:MR000022. [PMID: 24022476] doi:10.1002/14651858.MR0
00022.pub3
19. Sampson M, Shojania KG, McGowan J, Daniel R, Rader T,
Iansavichene AE, et al. Surveillance search techniques identified the
need to update systematic reviews. J Clin Epidemiol. 2008;61:755-
62. [PMID: 18586179] doi:10.1016/j.jclinepi.2007.10.003
20. Whiting PF, Rutjes AW, Westwood ME, Mallett S, Deeks JJ, Re-
itsma JB, et al; QUADAS-2 Group. QUADAS-2: a revised tool for the
quality assessment of diagnostic accuracy studies. Ann Intern Med.
2011;155:529-36. [PMID: 22007046] doi:10.7326/0003-4819-155-8
-201110180-00009
21. Rutter CM, Gatsonis CA. A hierarchical regression approach to
meta-analysis of diagnostic test accuracy evaluations. Stat Med.
2001;20:2865-84. [PMID: 11568945]
22. Takwoingi Y, Deeks JJ. MetaDAS: a SAS macro for meta-analysis
of diagnostic accuracy studies. User Guide Version 1.3. 30 July 2010.
23. Balshem H, Helfand M, Schünemann HJ, Oxman AD, Kunz R,
Brozek J, et al. GRADE guidelines: 3. rating the quality of evidence.
J Clin Epidemiol. 2011;64:401-6. [PMID: 21208779] doi:10.1016/j
.jclinepi.2010.07.015
24. Amland RC, Sutariya BB. Quick Sequential [Sepsis-related] Or-
gan Failure Assessment (qSOFA) and St. John Sepsis Surveillance
274 Annals of Internal Medicine • Vol. 168 No. 4 • 20 February 2018
Downloaded From: https://annals.org/ by a University of South Dakota User on 07/25/2018
Prognostic Accuracy of qSOFA for Mortality
Agent to detect patients at risk of sepsis: an observational cohort
study. Am J Med Qual. 2017:1062860617692034. [PMID:
28693336] doi:10.1177/1062860617692034
25. April MD, Aguirre J, Tannenbaum LI, Moore T, Pingree A, Thax-
ton RE, et al. Sepsis clinical criteria in emergency department pa-
tients admitted to an intensive care unit: an external validation study
of quick Sequential Organ Failure Assessment. J Emerg Med. 2017;
52:622-31. [PMID: 27823893] doi:10.1016/j.jemermed.2016.10.012
26. Askim Å, Moser F, Gustad LT, Stene H, Gundersen M, Åsvold
BO, et al. Poor performance of quick-SOFA (qSOFA) score in pre-
dicting severe sepsis and mortality—a prospective study of patients
admitted with infection to the emergency department. Scand J
Trauma Resusc Emerg Med. 2017;25:56. [PMID: 28599661] doi:10
.1186/s13049-017-0399-4
27. Burnham JP, Kollef MH. qSOFA score: predictive validity in En-
terobacteriaceae bloodstream infections. J Crit Care. 2017;43:143-7.
[PMID: 28898743] doi:10.1016/j.jcrc.2017.09.011
28. Chen YX, Wang JY, Guo SB. Use of CRB-65 and quick Sepsis-
related Organ Failure Assessment to predict site of care and mortal-
ity in pneumonia patients in the emergency department: a retro-
spective study. Crit Care. 2016;20:167. [PMID: 27250351] doi:10
.1186/s13054-016-1351-0
29. Churpek MM, Snyder A, Han X, Sokol S, Pettit N, Howell MD,
et al. Quick Sepsis-related Organ Failure Assessment, systemic in-
flammatory response syndrome, and early warning scores for detect-
ing clinical deterioration in infected patients outside the intensive
care unit. Am J Respir Crit Care Med. 2017;195:906-11. [PMID:
27649072] doi:10.1164/rccm.201604-0854OC
30. de Groot B, Stolwijk F, Warmerdam M, Lucke JA, Singh GK,
Abbas M, et al. The most commonly used disease severity scores are
inappropriate for risk stratification of older emergency department
sepsis patients: an observational multi-centre study. Scand J Trauma
Resusc Emerg Med. 2017;25:91. [PMID: 28893325] doi:10.1186
/s13049-017-0436-3
31. Donnelly JP, Safford MM, Shapiro NI, Baddley JW, Wang HE.
Application of the Third International Consensus Definitions for Sep-
sis (Sepsis-3) classification: a retrospective population-based cohort
study. Lancet Infect Dis. 2017;17:661-70. [PMID: 28268067] doi:10
.1016/S1473-3099(17)30117-2
32. Finkelsztein EJ, Jones DS, Ma KC, Pabón MA, Delgado T, Naka-
hira K, et al. Comparison of qSOFA and SIRS for predicting adverse
outcomes of patients with suspicion of sepsis outside the intensive
care unit. Crit Care. 2017;21:73. [PMID: 28342442] doi:10.1186
/s13054-017-1658-5
33. Freund Y, Lemachatti N, Krastinova E, Van Laer M, Claessens YE,
Avondo A, et al; French Society of Emergency Medicine Collabora-
tors Group. Prognostic accuracy of Sepsis-3 criteria for in-hospital
mortality among patients with suspected infection presenting to the
emergency department. JAMA. 2017;317:301-8. [PMID: 28114554]
doi:10.1001/jama.2016.20329
34. Forward E, Konecny P, Burston J, Adhikari S, Doolan H, Jensen
T. Predictive validity of the qSOFA criteria for sepsis in non-ICU inpa-
tients [Letter]. Intensive Care Med. 2017;43:945-6. [PMID: 28357459]
doi:10.1007/s00134-017-4776-2
35. Giamarellos-Bourboulis EJ, Tsaganos T, Tsangaris I, Lada M,
Routsi C, Sinapidis D, et al; Hellenic Sepsis Study Group. Validation
of the new Sepsis-3 definitions: proposal for improvement in early
risk identification. Clin Microbiol Infect. 2017;23:104-9. [PMID:
27856268] doi:10.1016/j.cmi.2016.11.003
36. González Del Castillo J, Julian-Jiménez A, González-Martı́nez F,
Álvarez-Manzanares J, Piñera P, Navarro-Bustos C, et al; represent-
ing the Infectious Disease Group of the Spanish Emergency Medi-
cine Society (see addendum). Prognostic accuracy of SIRS criteria,
qSOFA score and GYM score for 30-day-mortality in older non-
severely dependent infected patients attended in the emergency
department. Eur J Clin Microbiol Infect Dis. 2017;36:2361-9. [PMID:
28755060] doi:10.1007/s10096-017-3068-7
37. LeGuen M, Ballueer Y, McKay R, Eastwood G, Bellomo R, Jones
D; Austin Health RRT qSOFA investigators. Frequency and signifi-
cance of qSOFA criteria during adult rapid response team reviews: a
Annals.org
 Prognostic Accuracy of qSOFA for Mortality
prospective cohort study. Resuscitation. 2018;122:13-8. [PMID:
29133019] doi:10.1016/j.resuscitation.2017.11.036
38. Guirgis FW, Puskarich MA, Smotherman C, Sterling SA, Gautam
S, Moore FA, et al. Development of a simple sequential organ failure
assessment score for risk assessment of emergency department pa-
tients with sepsis. J Intensive Care Med. 2017:885066617741284.
[PMID: 29141524] doi:10.1177/0885066617741284
39. Haydar S, Spanier M, Weems P, Wood S, Strout T. Comparison
of QSOFA score and SIRS criteria as screening mechanisms for
emergency department sepsis. Am J Emerg Med. 2017;35:1730-3.
[PMID: 28712645] doi:10.1016/j.ajem.2017.07.001
40. Henning DJ, Puskarich MA, Self WH, Howell MD, Donnino MW,
Yealy DM, et al. An emergency department validation of the SEP-3
sepsis and septic shock definitions and comparison with 1992 con-
sensus definitions. Ann Emerg Med. 2017;70:544-52. [PMID:
28262318] doi:10.1016/j.annemergmed.2017.01.008
41. Huson MA, Kalkman R, Grobusch MP, van der Poll T. Predictive
value of the qSOFA score in patients with suspected infection in a
resource limited setting in Gabon [Letter]. Travel Med Infect Dis.
2017;15:76-7. [PMID: 27826072] doi:10.1016/j.tmaid.2016.10.014
42. Huson MAM, Katete C, Chunda L, Ngoma J, Wallrauch C, Heller
T, et al. Application of the qSOFA score to predict mortality in pa-
tients with suspected infection in a resource-limited setting in
Malawi. Infection. 2017;45:893-6. [PMID: 28786004] doi:10.1007
/s15010-017-1057-5
43. Hwang SY, Jo IJ, Lee SU, Lee TR, Yoon H, Cha WC, et al. Low
accuracy of positive qSOFA criteria for predicting 28-day mortality in
critically ill septic patients during the early period after emergency
department presentation. Ann Emerg Med. 2017. [PMID: 28669551]
doi:10.1016/j.annemergmed.2017.05.022
44. Johnson A, Celi LA, Raffa J, Pollard T, Stone D. External valida-
tion of the Sepsis-3 guidelines. Crit Care Med. 2016;44(Suppl 1):91.
45. Khwannimit B, Bhurayanontachai R, Vattanavanit V. Comparison
of the performance of SOFA, qSOFA and SIRS for predicting mortal-
ity and organ failure among sepsis patients admitted to the intensive
care unit in a middle-income country. J Crit Care. 2017;44:156-60.
[PMID: 29127841] doi:10.1016/j.jcrc.2017.10.023
46. Kim M, Ahn S, Kim WY, Sohn CH, Seo DW, Lee YS, et al. Predic-
tive performance of the quick Sequential Organ Failure Assessment
score as a screening tool for sepsis, mortality, and intensive care
unit admission in patients with febrile neutropenia. Support Care
Cancer. 2017;25:1557-62. [PMID: 28062972] doi:10.1007/s00520
-016-3567-6
47. Kolditz M, Scherag A, Rohde G, Ewig S, Welte T, Pletz M;
CAPNETZ Study Group. Comparison of the qSOFA and CRB-65 for
risk prediction in patients with community-acquired pneumonia [Let-
ter]. Intensive Care Med. 2016;42:2108-10. [PMID: 27647332] doi:10
.1007/s00134-016-4517-y
48. Moskowitz A, Patel PV, Grossestreuer AV, Chase M, Shapiro NI,
Berg K, et al; Center for Resuscitation Science. Quick Sequential Or-
gan Failure Assessment and systemic inflammatory response syn-
drome criteria as predictors of critical care intervention among pa-
tients with suspected infection. Crit Care Med. 2017;45:1813-9.
[PMID: 28759474] doi:10.1097/CCM.0000000000002622
49. Park HK, Kim WY, Kim MC, Jung W, Ko BS. Quick Sequential
Organ Failure Assessment compared to systemic inflammatory re-
sponse syndrome for predicting sepsis in emergency department.
J Crit Care. 2017;42:12-7. [PMID: 28647650] doi:10.1016/j.jcrc.2017
.06.020
50. Piano S, Bartoletti M, Tonon M, Baldassarre M, Chies G, Romano
A, et al. Assessment of Sepsis-3 criteria and quick SOFA in patients
with cirrhosis and bacterial infections. Gut. 2017. [PMID: 28860348]
doi:10.1136/gutjnl-2017-314324
51. Quinten VM, van Meurs M, Wolffensperger AE, Ter Maaten JC,
Ligtenberg JJM. Sepsis patients in the emergency department: strat-
ification using the Clinical Impression Score, Predisposition, Infec-
tion, Response and Organ dysfunction score or quick Sequential Or-
Annals.org
Downloaded From: https://annals.org/ by a University of South Dakota User on 07/25/2018
REVIEW
gan Failure Assessment score? Eur J Emerg Med. 2017. [PMID:
28338533] doi:10.1097/MEJ.0000000000000460
52. Raith EP, Udy AA, Bailey M, McGloughlin S, MacIsaac C, Bellomo
R, et al; Australian and New Zealand Intensive Care Society
(ANZICS) Centre for Outcomes and Resource Evaluation (CORE).
Prognostic accuracy of the SOFA Score, SIRS criteria, and qSOFA
score for in-hospital mortality among adults with suspected infection
admitted to the intensive care unit. JAMA. 2017;317:290-300.
[PMID: 28114553] doi:10.1001/jama.2016.20328
53. Ranzani OT, Prina E, Menéndez R, Ceccato A, Cilloniz C, Méndez
R, et al. New sepsis definition (Sepsis-3) and community-acquired
pneumonia mortality. A validation and clinical decision-making
study. Am J Respir Crit Care Med. 2017;196:1287-97. [PMID:
28613918] doi:10.1164/rccm.201611-2262OC
54. Siddiqui S, Chua M, Kumaresh V, Choo R. A comparison of pre
ICU admission SIRS, EWS and q SOFA scores for predicting mortality
and length of stay in ICU. J Crit Care. 2017;41:191-3. [PMID:
28575814] doi:10.1016/j.jcrc.2017.05.017
55. Sterling SA, Puskarich MA, Glass AF, Guirgis F, Jones AE. The
impact of the Sepsis-3 septic shock definition on previously defined
septic shock patients. Crit Care Med. 2017;45:1436-42. [PMID:
28542029] doi:10.1097/CCM.0000000000002512
56. Umemura Y, Ogura H, Gando S, Kushimoto S, Saitoh D, Mayumi
T, et al; JAAM Sepsis Registry (JAAMSR) Study Group. Assessment
of mortality by qSOFA in patients with sepsis outside ICU: a post hoc
subgroup analysis by the Japanese Association for Acute Medicine
Sepsis Registry Study Group. J Infect Chemother. 2017;23:757-62.
[PMID: 28847586] doi:10.1016/j.jiac.2017.07.005
57. Wang JY, Chen YX, Guo SB, Mei X, Yang P. Predictive perfor-
mance of quick Sepsis-related Organ Failure Assessment for mortal-
ity and ICU admission in patients with infection at the ED. Am J
Emerg Med. 2016;34:1788-93. [PMID: 27321936] doi:10.1016/j
.ajem.2016.06.015
58. Wang AY, Ma HP, Kao WF, Tsai SH, Chang CK. Red blood cell
distribution width is associated with mortality in elderly patients with
sepsis. Am J Emerg Med. 2017. [PMID: 29133071] doi:10.1016/j
.ajem.2017.10.056
59. Weigle H, Trigonis R, Bunn D, Krakauer D, Long L, Perkins J.
Severe sepsis: what gets it right? A comparison of the quick Sequen-
tial Organ Failure Assessment versus systemic inflammatory re-
sponse syndrome criteria. Ann Emerg Med. 2016;68 (Suppl 1):
S132-3.
60. Williams JM, Greenslade JH, McKenzie JV, Chu K, Brown AFT,
Lipman J. Systemic inflammatory response syndrome, quick Sequen-
tial Organ Function Assessment, and organ dysfunction: insights
from a prospective database of ED patients with infection. Chest.
2017;151:586-96. [PMID: 27876592] doi:10.1016/j.chest.2016.10
.057
61. Knaus WA, Sun X, Nystrom O, Wagner DP. Evaluation of defini-
tions for sepsis. Chest. 1992;101:1656-62. [PMID: 1600787]
62. Rangel-Frausto MS, Pittet D, Costigan M, Hwang T, Davis CS,
Wenzel RP. The natural history of the systemic inflammatory re-
sponse syndrome (SIRS). A prospective study. JAMA. 1995;273:117-
23. [PMID: 7799491]
63. Brun-Buisson C. The epidemiology of the systemic inflammatory
response. Intensive Care Med. 2000;26 Suppl 1:S64-74. [PMID:
10786961]
64. Scott MC. Defining and diagnosing sepsis. Emerg Med Clin
North Am. 2017;35:1-9. [PMID: 27908326] doi:10.1016/j.emc.2016
.08.002
65. Vincent JL. Dear SIRS, I'm sorry to say that I don't like you. Crit
Care Med. 1997;25:372-4. [PMID: 9034279]
66. Churpek MM, Snyder A, Sokol S, Pettit NN, Edelson DP. Investi-
gating the impact of different suspicion of infection criteria on the
accuracy of quick Sepsis-related Organ Failure Assessment, systemic
inflammatory response syndrome, and early warning scores. Crit
Care Med. 2017;45:1805-12. [PMID: 28737573] doi:10.1097/CCM
.0000000000002648
Annals of Internal Medicine • Vol. 168 No. 4 • 20 February 2018 275
 Current Author Addresses: Dr. Fernando: Department of Author Contributions: Conception and design: S.M. Fer-
Emergency Medicine and Division of Critical Care, Depart- nando, A. Tran, M. Taljaard, A.J.E. Seely, J.J. Perry.
ment of Medicine, University of Ottawa, The Ottawa Hospital, Analysis and interpretation of the data: S.M. Fernando, A.
Civic Campus, Room EM-206, 1053 Carling Avenue, Ottawa, Tran, M. Taljaard, W. Cheng, B. Rochwerg, A.J.E. Seely, J.J.
Ontario K1Y 4E9, Canada. Perry.
Dr. Tran: Division of General Surgery, Department of Surgery, Drafting of the article: S.M. Fernando, A. Tran, M. Taljaard, W.
University of Ottawa, The Ottawa Hospital, General Campus, Cheng, B. Rochwerg, A.J.E. Seely, J.J. Perry.
501 Smyth Road, Ottawa, Ontario K1H 8L6, Canada. Critical revision of the article for important intellectual con-
Dr. Taljaard: Associate Professor, School of Epidemiology and tent: S.M. Fernando, A. Tran, M. Taljaard, W. Cheng, B.
Public Health, University of Ottawa, Clinical Epidemiology Rochwerg, A.J.E. Seely, J.J. Perry.
Program, Ottawa Hospital Research Institute, 1053 Carling Av- Final approval of the article: S.M. Fernando, A. Tran, M. Tal-
enue, Civic Box 693, Admin Services Building, Ottawa, On- jaard, W. Cheng, B. Rochwerg, A.J.E. Seely, J.J. Perry.
tario K1Y 4E9, Canada. Statistical expertise: M. Taljaard, W. Cheng, B. Rochwerg.
Dr. Cheng: Postdoctoral Research Fellow, Clinical Epidemiol- Administrative, technical, or logistic support: W. Cheng.
ogy Program, Ottawa Hospital Research Institute, 501 Smyth Collection and assembly of data: S.M. Fernando, A. Tran, W.
Road, Room Box 201B, Ottawa, Ontario K1H 8L6, Canada. Cheng.
Dr. Rochwerg: Assistant Professor, Department of Medicine,
Division of Critical Care, McMaster University, 1200 Main
Street West, Hamilton, Ontario L8N 3Z5, Canada.
Dr. Seely: Professor and Vice-Chair of Research, Division of
Thoracic Surgery, Department of Surgery, University of Ot-
tawa, The Ottawa Hospital, General Campus, 501 Smyth
Road, Ottawa, Ontario K1H 8L6, Canada.
Dr. Perry: Professor, Department of Emergency Medicine, Uni-
versity of Ottawa, Clinical Epidemiology Program, Ottawa
Hospital Research Institute, F6, The Ottawa Hospital, Civic
Campus, Ottawa, Ontario K1Y 4E9, Canada.

Annals.org Annals of Internal Medicine • Vol. 168 No. 4 • 20 February 2018

Downloaded From: https://annals.org/ by a University of South Dakota User on 07/25/2018