Clinical Nutrition 35 (2016) 158e162

Contents lists available at ScienceDirect

Clinical Nutrition
journal homepage: http://www.elsevier.com/locate/clnu

Original article

Identifying critically-ill patients who will benefit most from

nutritional therapy: Further validation of the “modified NUTRIC”
nutritional risk assessment tool
Adam Rahman a, b, Rana M. Hasan a, Ravi Agarwala c, Claudio Martin a, d, e,
Andrew G. Day f, Daren K. Heyland f, g, h, *
a
Department of Medicine, University of Western Ontario, London, Ontario, Canada
b
Gastroenterology, St. Joseph's Healthcare Centre/London Health Sciences Centre, Canada
c
Department of Anesthesia, Section of Critical Care Medicine, Wake Forest School of Medicine, Winston-Salem, NC, USA
d
Critical Care/Trauma Centre, London Health Sciences Centre, Victoria Campus, Canada
e
Lawson Health Research Institute, Canada
f
Clinical Evaluation Research Unit, Kingston General Hospital, Kingston, Ontario, Canada
g
Department of Community Health and Epidemiology, Queen's University, Kingston, Ontario, Canada
h
Department of Medicine, Queen's University, Kingston, Ontario, Canada

a r t i c l e i n f o s u m m a r y

Article history: Introduction: Better tools are needed to assist in the identification of critically ill patients most likely to
Received 5 June 2014 benefit from artificial nutrition therapy. Recently, the Nutrition Risk in Critically ill (NUTRIC) score has
Accepted 21 January 2015 been developed for such purpose. The objective of this study was to externally validate a modified
version of the NUTRIC score in a second database.
Keywords: Methods: We conducted a post hoc analysis of a database of a randomized control trial of intensive care
Nutrition
unit (ICU) patients with multi-organ failure. Data for all variables of the NUTRIC score with the exception
Nutritional risk
of IL-6 levels were collected. These included age, APACHE II score, SOFA score, number of co-morbidities,
Critically ill patients
Intensive care unit
days from hospital admission to ICU admission. The NUTRIC score was calculated using the exact same
thresholds and point system as developed previously except the IL-6 item was omitted. A logistic model
including the NUTRIC score, the nutritional adequacy and their interaction was estimated to assess if the
NUTRIC score modified the association between nutritional adequacy and 28-day mortality. We also
examined the association of elevated NUTRIC scores and 6-month month mortality and the interaction
between NUTRIC score and nutritional adequacy.
Results: A total of 1199 patients were analyzed. The mean total calories prescribed was 1817 cal (SD 312)
with total mean protein prescribed of 98.3 g (SD 23.6). The number of patients who received PN was 9.5%.
The overall 28-day mortality rate in this validation sample was 29% and the mean NUTRIC score was 5.5 (SD
1.6). Based on the logistic model, the odds of mortality at 28 days was multiplied by 1.4 (95% CI, 1.3e1.5) for
every point increase on the NUTRIC score. The mean (SD) nutritional adequacy was 50.2 (29.5) with an
interquartile range from 24.8 to 74.1. The test for interaction confirmed that the association between
nutritional adequacy and 28-day mortality is significantly modified by the NUTRIC score (test for inter-
action p ¼ 0.029). In particular, there is a strong positive association between nutritional adequacy and 28
day survival in patients with a high NUTRIC score but this association diminishes with decreasing NUTRIC
score. Higher NUTRIC scores are also significantly associated with higher 6-month mortality (p < 0.0001)
and again the positive association between nutritional adequacy and 6 month survival was significantly
stronger (and perhaps only present) in patients with higher NUTRIC score (test for interaction p ¼ 0.038).
Conclusion: The NUTRIC scoring system is externally validated and may be useful in identifying critically
ill patients most likely to benefit from optimal amounts of macronutrients when considering mortality as
an outcome.
© 2015 Elsevier Ltd and European Society for Clinical Nutrition and Metabolism. All rights reserved.

* Corresponding author. Queen's University, Kingston General Hospital, Angada 4 Room 5-416, 76 Stuart Street, Kingston, Ontario K7L 2V7, Canada. Fax: þ1 613 548 2428.
E-mail address: dkh2@queensu.ca (D.K. Heyland).

http://dx.doi.org/10.1016/j.clnu.2015.01.015
0261-5614/© 2015 Elsevier Ltd and European Society for Clinical Nutrition and Metabolism. All rights reserved.
 A. Rahman et al. / Clinical Nutrition 35 (2016) 158e162 159

1. Introduction Table 1
NUTRIC scoring system in original and second validation database.

Heyland et al. previously proposed a novel scoring tool, the Nutri- Variables in NUTRIC scoring Original Second
tion Risk in Critically ill (NUTRIC) score, which is the first nutritional NUTRIC score system development validation
risk assessment tool developed and validated specifically for intensive sample sample
Range Points
(n ¼ 598) (n ¼ 1, 199)
care unit (ICU) patients [1]. Many other risk scores and assessment
tools exist to quantify nutrition risk [2e7] but none have been spe- Age <50 0 130 (21.7) 199 (16.6)
50e<75 1 345 (57.7) 710 (59.2)
cifically designed for ICU patients [7]. Indeed, they generally consider
75 2 123 (20.6) 290 (24.2)
all critically ill patients to be at high nutritional risk [2,8]. However, the APACHE II <15 0 111 (18.6) 48 (4.0)
recognition that not all ICU patients will respond the same to nutri- 15e<20 1 135 (22.6) 157 (13.1)
tional interventions was the critical concept behind the NUTRIC score 20e28 2 226 (37.8) 508 (42.4)
28 3 126 (21.1) 486 (40.5)
[1,8,9]. The conceptual model incorporated candidate predictor
SOFA <6 0 220 (36.8) 157 (13.1)
markers of acute starvation, chronic starvation, acute inflammation 6e<10 1 247 (41.3) 624 (52.0)
and chronic inflammation [1,9]. All candidate predictors incorporated 10 2 131 (21.9) 418 (34.9)
into our final model predictors were significantly associated with 28- # Co-morbidities 0e1 0 160 (26.8) 392 (32.7)
day mortality [1]. Measure of under-nutrition, such as history or 2þ 1 438 (73.2) 807 (67.3)
Days from hospital 0e<1 0 375 (62.7) 757 (63.1)
reduced oral intake or recent weight loss, did not factor into the final
to ICU admit 1þ 1 223 (37.3) 442 (36.9)
model because of significant amounts of missing data. The final IL6* 0e<400 0 489 (81.8)
composite score accurately identified those patients who had higher 400þ 1 109 (18.2)
mortality rates or survivors with longer lengths of stay. In addition, Score range [IQR] 0e10 [3e6]. 0e9 [4e7].
there was an interaction between mortality, nutritional intake and Score mean ± SD 4.7 ± 2.2 5.5 ± 1.6
NUTRIC score discriminative performance
NUTRIC score suggesting that those with higher NUTRIC scores (6 or AUC 0.783 0.648
more) benefited the most from increasing nutritional intake. However, Gen R-Squared 0.169 0.055
the inferences about the validity of the NUTRIC score are limited Gen Max-rescaled 0.256 0.573
because they are derived and validated in the same database. R-Squared
Many methods of nutritional screening in hospitalized patients
are cumbersome and time-consuming and hence are not routinely during the first 28 ICU days while the patient remained ventilated
done [10]. The NUTRIC score is easy to calculate as it contains [12]. Only days prior to the date of death, ICU discharge or liberation
variables that are mostly easy to obtain in the critical care setting, from mechanical intubation were counted (evaluable) toward
with the exception of IL-6 levels which is not commonly measured. nutrition adequacy.
In practice, many units are using the NUTRIC score without the IL-6 Logistic regression was used to assess the strength of the asso-
level and the question remains as to the validity of the validity of ciation between the NUTRIC score and 28-day mortality. Measures
the NUTRIC score without IL-6 level (modified NUTRIC score). The of discrimination using the original data were compared to those
second stage in development of a clinical ICU prediction model is obtained from the original development database. Calibration (i.e.
external validation [11]. The aim of this study is to externally vali- goodness of fit) of the score was assessed by graphing observed
date [11] this modified NUTRIC score in a second, population of mortality rates at each score against the mortality predicted by a
critically ill patients. We hypothesize that the modified NUTRIC logistic model with NUTRIC score as a sole continuous predictor.
score will retain its validity in this new database by omitting the IL- The statistical significance of lack of fit was tested by the Hos-
6 levels, and we can increase the clinical utility of the tool. mereLemeshow goodness of fit test [13].
A logistic model including the NUTRIC score, the nutritional
2. Methods adequacy and their product (interaction) was performed to assess if
the NUTRIC score modified the association between nutritional
This study was a post hoc analysis of an existing database adequacy and 28-day mortality. This model was stratified by
derived from a randomized control trial conducted in 40 tertiary evaluable days since the length of stay could confound the rela-
ICU's in Europe and North America, after ethics approval was ob- tionship between nutritional adequacy and outcome due to the
tained. The purpose of the trial was to evaluate the effectiveness of ramping up of nutrition support over the first several ICU days. For
glutamine and antioxidant supplementation in critically ill patients ease of interpretation, figures were generated demonstrating the
[12]. All patients were attempted to be fed according to the Cana- association between nutrition adequacy and 28-day mortality
dian Critical Care Nutrition practice guidelines, independent of separately in patients with NUTRIC scores grouped as 0e5 and 6e9.
study supplements [12]. The trial randomized 1223 mechanically However, the test for interaction used NUTRIC score and nutrition
ventilated patients with multi-organ failure, with expected length adequacy as continuous variables. Finally, the logistic model was
of stay >5days with a primary outcome of 28-day mortality. run separately in patients who did and did not have enteral feeding
The NUTRIC score was calculated using the same thresholds and interrupted to assess if increasing NUTRIC score is associated with
point system as developed previously (Table 1) except the IL-6 item feeding intolerance.
was omitted (IL-6 levels were not collected in original study). Thus, Given capture of longer-term mortality rates in this database,
the NUTRIC score ranges from 0 to 9 rather than 0 to 10, as origi- we used a similar modeling approach with Cox proportional haz-
nally defined. The absence of IL-6 may makes assessment of the ards model [14] to estimate the overall association between NUTRIC
performance of the NUTRIC score more conservative, although it is score and 6-month survival and if the NUTRIC score significantly
not expected that the absence of this one item will have a strong modified the association between nutritional adequacy and 6-
impact on the score [1]. month survival.

2.1. Statistical analysis 3. Results

Nutrition adequacy was defined as the total proportion of the Five patients withdrew consent prior to treatment and were not
caloric prescription received (either enterally or parenterally) evaluable for 28-day mortality, and the amount of calories received
160 A. Rahman et al. / Clinical Nutrition 35 (2016) 158e162

was not known for an additional 19 patients. Thus, the current

analysis included 1199 patients.
The overall 28-day mortality rate in this validation sample is
29.0% compared to 23.1% in the NUTRIC development sample. The
distribution of the items included in the NUTRIC risk score are
presented for both this validation sample and the previous devel-
opment sample in Table 1. Fig. 1 compares the distribution of the
NUTRIC score between the current validation database and the
original developmental database. The NUTRIC score in the current
validation database has a higher mean and lower standard devia-
tion compared to the original developmental sample [mean (SD)
5.5 (1.6) versus 4.7 (SD 2.2)].
The ability of the NUTRIC score to predict 28-day mortality is
summarized in Fig. 2. The overall discriminate ability of the NUTRIC
score for predicting 28-day mortality is lower in this validation
sample than in the original development sample (0.648 vs. 0.783).
In particular, the mortality rate for patients with a maximum
NUTRIC score of 9 was only 53% (8/15) in the current validation
sample compared to 71% (12/17) in the developmental sample. No
Fig. 2. Predicted versus observed 28-day mortality.
patients with a NUTRIC score of 0 or 1 died in either sample (0/45
for original sample and 0/14 for validation sample). Fig. 2, shows
the observed (circles) and logistic model predicted (line) mortality less (95% CI 2.0%e9.1%, p ¼ 0.0023) of their caloric prescription than
rate by NUTRIC score. Although mortality generally increased with patients who did not have any interruptions.
NUTRIC score, the calibration of the model was statistically not Higher NUTRIC scores are significantly associated with higher 6-
ideal (HosmereLemeshow goodness of fit test p ¼ 0.013). However, month mortality (p < 0.0001) (Fig. 5). Patients who received <25%
although statistically significant, the lack of calibration did not of their caloric prescription had a higher mortality rate
appear to be clinically important (Fig. 2). Using these validation (p < 0.0001). Using Cox proportional hazards model, there is a
data, the logistic model estimated odds of mortality were multi- significant interaction between the NUTRIC score and the propor-
plied by 1.4 (95% CI, 1.3e1.5) for every point increase on the NUTRIC tion of prescription received (test for interaction p ¼ 0.038). In
score. particular, for patients with a NUTRIC score of 6e9 each 25% in-
The mean (SD) nutritional adequacy was 50.2 (29.5) with an crease in percent of caloric prescription received multiplied the
interquartile range from 24.8 to 74.1. Nutritional adequacy was not hazard rate (HR) of death by 0.82 (95% CI, 0.73e0.91), but there was
correlated with NUTRIC score. Fig. 3, demonstrates that increased no significant association identified between nutritional adequacy
nutritional adequacy appears to be associated with increased sur- and 6-month mortality in patients with lower NUTRIC scores.
vival in patients with higher NUTRIC scores. The test for interaction When examining the proportional hazards assumption we noted
confirmed the association between nutritional adequacy and that the strong association between low caloric intake and higher
mortality is significantly modified by the NUTRIC score (test for mortality appears to be limited to the first couple weeks with no
interaction p ¼ 0.013 and p ¼ 0.029 before and after controlling for clear difference in hazard rates after that.
evaluable days).
Fig. 4 demonstrates that the association between 28-day mor- 4. Discussion
tality, NUTRIC score and nutritional adequacy was similar among
patients who had enteral feeding interrupted due to intolerance We set out to provide a second validation of the NUTRIC score in a
(panel A) and those who did not (panel B). The 277 patients who second database and this time, without IL-6 levels. We report that a
had enteral feeding interrupted due to intolerance received 5.5% logistic model with NUTRIC score, excluding IL-6, as the sole

Fig. 3. Predicted probability of mortality versus nutrition received by nutrition risk

Fig. 1. Histogram of NUTRIC score. score.
 A. Rahman et al. / Clinical Nutrition 35 (2016) 158e162
Fig. 4. Predicted probability of mortality vs nutrition received by nutrition risk score.
Panel A: Among 277 patients who had at least one interruption of EN due to intoler-
ance. Panel B: Among 922 patients who never discontinued EN due to intolerance.
continuous independent variable predicted mortality with odds of
mortality multiplied by 1.4 (95% CI, 1.3e1.5) for every point increase
on the NUTRIC score. We demonstrate that increased nutritional
adequacy is associated with increased survival in patients with higher
NUTRIC scores (6) but not in patients with lower NUTRIC scores
(5). In practical terms the NUTRIC score without IL-6 levels may be
used to help determine which might benefit from enteral supple-
mentation, parenteral nutrition or strategies to enhance EN delivery.
One interpretation of the original NUTRIC validation study was
that sicker patients tolerate less nutrition rather than NUTRIC
identifying patients who benefit more from nutrition. The obser-
vation that nutritional intake is stable across all ranges of NUTRIC
and that the relationship between NUTRIC, intake and mortality is
consistent in patient with and without feeding intolerance argues
against this notion.
The NUTRIC score may have utility in the design and interpre-
tation of clinical trials of nutrition in the ICU setting. Based on our
observations, studies that include heterogeneous ICU patients, are
more likely to be negative than those that focus on high risk pa-
tients [1,15]. This may in part explain discordant results of recent
protein energy provision studies in ICU patients [15e17]. Future
clinical trials need to use the NUTRIC score or similar validated
measure in this population to describe nutritional risk.
We further analyzed effects of the modified NUTRIC score and
interaction with nutritional adequacy with 6-month mortality, not
performed in the original validation. Elevated NUTRIC score were
161
Fig. 5. NUTRIC scores versus 6-month mortality.
associated with increased 6-month mortality and more impor-
tantly there was significant interaction with nutritional adequacy
with higher NUTRIC score.
One of the limitations of our study was our inability to obtain IL-
6 levels. Even though our intent was to examine the model without
IL-6, we cannot demonstrate whether the difference in discrimi-
nation was due to exclusion of this variable. The calibration of this
model (slightly lower area under curve) is likely secondary to a
more homogenous population of sicker patients, compared to the
original development data. The lack of calibration across the range
of scores was statistically significant, and although this visually
appears to be an issue with low scores, we cannot explicitly make
that conclusion. We recognize that using mortality is also a limi-
tation as nutritional therapy may have other benefits for patients.
The major limitation of the NUTRIC score is uptake by clinicians
who may argue that there is little need for another risk score with
other risk score such APACHE II or SOFA score. These risks scores are
integrated within our NUTRIC scoring system but we recognize
calculation can be cumbersome limiting utility, and it has no value
for non-ICU patients. Others correctly point out that the NUTRIC
score does not contain traditional nutrition variables. Unfortu-
nately, in an ICU setting, these variables depend on history from
family members, which was inconsistent in our initial validation
sample. With these limitations, however, we observe that
increasing nutritional adequacy in patients with elevated NUTRIC
scores experience lower 28 day mortality. That is, increased
nutritional intake will decrease mortality in high-risk patients,
identified by the NUTRIC score, which elevates the NUTRIC score
beyond a mere statistical measure. Indeed, the NUTRIC score may a
valuable in identifying high-risk patients who can benefit from
increased nutritional provision in future studies.
5. Conclusion
We have demonstrated independent validation of the NUTRIC
score without IL-6 levels to help discriminate which ICU patients
will benefit more (or less) from early adapted protein-energy pro-
vision. This scoring tool represents the first nutritional risk
assessment tool developed and validated specifically for ICU pa-
tients. The NUTRIC score is a practical, easy-to-use tool based on
variables that are easy to obtain in the critical care setting. We
assert that not all ICU patients are the same, and there are some
that benefit more or less from artificial protein-energy provision in
the critical care setting. We recognize that using mortality is also a
limitation as nutritional therapy may other benefits for patients
162 A. Rahman et al. / Clinical Nutrition 35 (2016) 158e162
and that future clinical trials are required to answer discordant
results regarding energy provision in the ICU.
Conflict of interest
None declared.
References
[1] Heyland D, et al. Identifying critically ill patients who benefit the most from
nutrition therapy: the development and initial validation of a novel risk
assessment tool. Crit Care 2011 Nov 15;15(6):R268 [Epub].
[2] Detsky A, et al. What is subjective global assessment of nutritional status?
J Parenter Enteral Nutr 1987 JaneFeb;11(1):8e13.
[3] Group MA. A consistent and reliable tool for malnutrition screening. Nurs
Times 2003;99(46):26e7.
[4] Kruizenga H, et al. Development and validation of a hospital screening tool for
malnutrition: the short nutritional assessment questionnaire (SNAQ). Clin
Nutr 2005;24(1):75e82.
[5] Ferguson M, et al. Development of a valid and reliable malnutrition screening
tool for adult acute hospital patients. Nutrition 1999;15(6):458e64.
[6] Lim S, et al. Development and validation of 3-minute nutrition screening (3-
MinNS) tool for acute hospital patients in Singapore. Asia Pac J Clin Nutr
2009;18(3):395e403.
[7] Anthony P. Nutrition screening tools for hospitalized patients. Nutr Clin Pract
2008;23(4):373e82.
[8] Kondrup J, et al. ESPEN working group: nutritional risk screening (NRS 2002):
a new method based on an analysis of controlled clinical trials. Clin Nutr
2003;22(3):321e36.
[9] Jensen G, et al. International Consensus Guideline Committee: adult starvation
and disease-related malnutrition: a proposal for etiology-based diagnosis in
the clinical practice setting from the International Consensus Guideline
Committee Clinical Nutrition. J Parenter Enteral Nutr 2010;34(2):156e9.
[10] Murray M, et al. Nutritional assessment of intensive-care unit patients. Mayo
Clin Proc 1988;63(11):1106e15.
[11] Labare re J. How to derive and validate clinical prediction models for use in
intensive care medicine. Intensive Care Med 2014;40:513e27.
[12] Heyland D, et al. A randomized trial of glutamine and antioxidants in critically
ill patients. N Engl J Med 2013;368:1489e97.
[13] Hosmer D, Lemeshow S. Applied logistic regression. New York: Wiley; 2000,
ISBN 0-471-61553-6.
[14] Cox D, Oakes D. Analysis of survival data. New York: Chapman & Hall; 1884,
ISBN 041224490X.
[15] Heyland D, Cahill N, Day A. Optimal amount of calories for critically ill pa-
tients: depends on how you slice the cake! Crit Care Med 2011;12:2619e26.
[16] Rahman A, Corbett C. Nutrition in the acute phase of critical illness. N Engl J
Med 2014 Jun 19;370(25):2450.
[17] Rahman A, Martin C, Heyland D. Nutrition therapy for the critically ill surgical
patient with aortic aneurysmal rupture: defining and improving current
practice 2013 Aug 23;2013 [Epub ahead of print].