BRIEF METHODOLOGICAL REPORTS

Validation of the Triage Risk Stratication Tool to Identify Older Persons at Risk for Hospital Admission and Returning to the Emergency Department
Jacques S. Lee, MD, Graeme Schwindt, BSc,w Mara Langevin, MN, z Rola Moghabghab, MN, Shabbir M. H. Alibhai, MD, k # ww Alex Kiss, PhD, z z and Gary Naglie, MD k # ww

OBJECTIVES: To assess the predictive validity of the Triage Risk Stratication Tool (TRST) to identify return to the emergency department (ED) or hospitalization in a multicenter patient sample. DESIGN: Prospective, observational study with 1-year follow-up. SETTING: EDs of three hospitals in Toronto, Canada. PARTICIPANTS: Seven hundred eighty-eight subjects aged 65 to 101 (mean age 76.6, 58.5% female) who presented to the ED and were discharged home from the ED. MEASUREMENTS: Trained clinical assessors completed the TRST on patients aged 65 and older during a 4-week study period. Patients who subsequently returned to the ED or were admitted to the hospital were identied using hospital information systems and classied as experiencing the composite endpoint at 30, 120, and 365 days. RESULTS: The mean TRST score was 1.55 (range 05), and 147 (18.7%) patients experienced the composite endpoint of return to the ED or hospital admission by 30 days. The sensitivity of a TRST score of 2 or greater was 62%, (95% condence interval (CI) 5 5470%), specicity was 57% (95% CI 5 5361%), and likelihood ratio was 1.44 (95% CI 5 1.231.66). The area under the curve was 0.61 using a cutoff score of 2. CONCLUSION: The TRST demonstrated only moderate predictive ability, and ideally, a better prediction rule
From the Sunnybrook Research Institute wFaculty of Medicine, and Departments of #Medicine and Health Policy, Management, and Evaluation, University of Toronto, Toronto, Ontario, Canada; zUniversity Health Network, Toronto General Hospital, Toronto, Ontario, Canada; St. Michaels Hospital, St. Michaels Hospital, Toronto, Ontario, Canada; k Division of General Internal Medicine, University Health Network, Toronto, Ontario, Canada; zzClinical Epidemiology Unit, Sunnybrook Research Institute, Toronto, Ontario, Canada; and wwToronto Rehabilitation Institute, Toronto, Ontario, Canada. Presented at the American Geriatrics Society Annual Meeting, Orlando, Florida, May 1115, 2005. Address correspondence to Dr. Jacques S. Lee, Sunnybrook Health Sciences Center, C7-14, 2075 Bayview Avenue, Toronto, Ontario, Canada M4N 3M5. E-mail: jacques.lee@sunnybrook.ca DOI: 10.1111/j.1532-5415.2008.01959.x

should be sought. Future studies to develop better prediction rules should compare their performance with that of existing prediction rules, including the TRST and Identifying Seniors at Risk tool, and assess the effect of any new prediction rule on patient outcomes. J Am Geriatr Soc 56:21122117, 2008.

Key words: predictive value of tests; emergency medicine; geriatrics; adverse outcomes; clinical prediction rule

emographic trends suggest that the proportion of persons aged 65 and older will double in the next 25 years, fundamentally changing the composition of the North American population.1 There is concern that the rapid growth in the elderly population may overwhelm current healthcare models.2 These demographic trends may challenge emergency department (ED) care of older patients in particular because of limited geriatric training and resources to cope with the more-complex needs of older patients.3,4 Thus, new strategies to provide efcient emergency care to older persons will become increasingly important. Geriatric interventions have been shown to be more effective when targeted at high-risk groups.58 The Triage Risk Stratication Tool (TRST) was developed to identify older patients who are at risk for failed discharge home from the ED, dened as return to the ED, admission to the hospital, or admission to a nursing home within 30 to 120 days after discharge.9 The TRST is a ve-item clinical prediction rule designed for rapid administration by the triage nurse in an ED after minimal training.9 All items on the TRST demonstrated excellent interrater reliability (kappa 5 0.941.0) and 70% agreement with a criterion-standard comprehensive assessment by an advanced practice nurse (APN).10 Other clinical prediction rules to identify older persons at high risk include the Identifying Seniors At Risk (ISAR)11

JAGS 56:21122117, 2008 r 2008, Copyright the Authors Journal compilation r 2008, The American Geriatrics Society

0002-8614/08/$15.00

JAGS

NOVEMBER 2008VOL. 56, NO. 11

VALIDATION OF THE TRST

2113

and the Discharge of Elderly from Emergency Department (DEED)12 tools, but these tools require emergency staff to explain and distribute a self-assessment questionnaire to patients and to collect the completed form afterward, which is outside the usual work processes of emergency staff and may be a barrier to uptake of these tools. The TRST has been independently validated in a sample of 120 patients,13 achieving results that were similar to those of the original study that derived the TRST.9 Despite this, the validation study concluded that the TRST was not clinically usefulFin stark contrast to the conclusions made by the authors of the original study. The primary objective was to assess the ability of the TRST to identify older patients at high risk after discharge from the ED in a larger, multicenter patient sample at 30, 120, and 365 days. The secondary objectives were to examine the effect of using different TRST cutpoints to dene high risk and whether other combinations of variables better predicted patient outcomes than the TRST.

strated competence administering it. The ve TRST items were whether patients had a history or evidence of cognitive impairment (poor recall or not oriented); had difculty walking, transferring, or a history of recent falls; took ve or more medications; or had had an ED visit in the previous 30 days or a hospitalization in the previous 90 days and whether the health professional completing the TRST had any concerns about elder abuse or neglect, substance abuse, medication nonadherence, or difculty performing instrumental activities of daily living. Emergency staff were blinded to the results of the TRST scores. The composite endpoint was dened as return to the ED or admission to hospital within 30, 120, and 365 days after discharge from the ED.

METHODS Study Design This was a pragmatic observational cohort study.14 The research ethics boards of all participating institutions approved it. Study Setting and Population Subjects were aged 65 and older and presented to one of three EDs in Toronto, Canada. The three EDs included one suburban community hospital (Center A), one urban community hospital (Center B), and one urban teaching hospital (Center C). Each center enrolled patients during a 4-week period: October 2002 for Center A and March 2003 for Centers B and C. The three EDs have a combined annual census of more than 25,000 patients aged 65 and older. Data Collection An emergency nurse or a patient care coordinator (PCC) completed TRST forms on patients aged 65 and older who were being discharged from the ED. The PCCs routinely assess patients aged 65 and older to determine whether they require assistance after discharge (e.g., referrals to a falls prevention clinic). Registered nurses, social workers, or geriatric APNs perform this role. TRST forms were completed on a convenience sample of patients discharged during the 4-week study period at each site. PCCs attempted to obtain study measures by telephone the next day for subjects who were discharged after hours. The clinical workload of the emergency nurses and PCCs was the main determinant of whether TRST forms were completed. Outcomes data were identied using each hospitals information systems, and subjects were followed until July 2004. Measures Information on patient demographics, including sex, age, and primary language, was collected from the medical record. The TRST consists of ve dichotomous items and was designed to be completed in 2 to 5 minutes.9 A coinvestigator at each site trained emergency nurses and the PCCs in use of the TRST until they satisfactorily demon-

Data Analysis For the primary objective, the sensitivity, specicity, and likelihood ratios (LRs) of the TRST score in predicting the composite endpoint were calculated using a TRST cutoff score of 2, as chosen previously;6,9,10 95% condence intervals (CIs) were also calculated for these estimates. Sample-size calculations were based on the desired precision for the sensitivity of the TRST score.15 If the prevalence of the composite endpoint was 17.5% or greater, and the sensitivity of a TRST score of 2 or greater was 65% or greater, then a sample of 781 would yield a 95% CI for the estimate of sensitivity that was no wider than 8%. For the rst secondary objective, the performance of different cutoffs of the TRST was examined by computing the area under the receiver operating characteristic curve (AUC) using different cutpoints.16 For the second secondary objective, whether other combinations of variables predict adverse outcomes as well as or better than the TRST were explored using a logistic regression technique previously developed.17 Each of the ve items included in the TRST, as well as age (as a continuous variable), sex, living alone, and site of ED visit, were entered into a forward stepwise model selection. The sensitivity, specicity, and LRs were calculated at 30, 60, 90, and 365 days after the initial ED visit for the most parsimonious model selected using this approach. RESULTS There were 1,986 patients aged 65 and older who presented to the three EDs during the 4-week study period. Of these, 377 (19%) were admitted to the hospital and were ineligible. The convenience sample included TRST forms completed on 788 of 1,609 eligible subjects (49%). Participants baseline characteristics are presented in Table 1. The mean age for enrolled patients was 76.6 (range 65101), and 58.5% were female; the mean age of nonenrolled patients was 75.2, and 53.9% were female. Of the 788 subjects, the composite endpoint occurred in 147 (18.7%) by 30 days, 245 (31.1%) by 120 days, and 346 (43.9%) by 365 days. The sensitivity, specicity, and LR of various TRST cutscores for the composite endpoint at all time points are reported in Table 2. Reducing the threshold for the TRST from 2 or greater to 1 or greater increased the sensitivity from 62% to 82% but reduced specicity from 57% to 24%. Conversely, increasing the cutoff to 3 or greater increased the specicity to 80% but

2114

LEE ET AL.

NOVEMBER 2008VOL. 56, NO. 11

JAGS

Table 1. Participant Characteristics at Baseline

Characteristic Hospital A n 5 530 Hospital B n 5 179 Hospital C n 5 79 All Hospitals N 5 788

Age, mean (interquartile range) Female, n (%) English rst language, n (%) Lives alone, n (%) TRST items, n (%) Emergency department visit in previous 30 days or hospitalization in previous 90 days Cognition impaired Taking 44 medications Difculty walking Health professional concern TRST score, n (%) 0 1 2 3 4 5 TRST score, mean (95% condence interval)
TRST 5 Triage Risk Stratication Tool.

76.3 (7082) 305 (57.6) 316 (59.6) 106 (20.0) 235 (44.3) 48 (9.1) 259 (48.9) 212 (40.0) 130 (24.5) 117 (26.3) 145 (27.4) 129 (24.3) 83 (15.7) 49 (9.1) 8 (1.5) 1.67 (1.561.78)

76.7 (7084) 104 (59.4) 117 (72.2) 30 (16.8) 58 (32.4) 24 (13.4) 88 (49.2) 42 (23.5) 20 (11.2) 47 (26.3) 65 (36.3) 43 (24.0) 15 (8.4) 9 (5.0) 0 (0.0) 1.30 (1.11.5)

78.8 (7284) 50 (63.3) 41 (51.9) 19 (24.1) 21 (26.6) 11 (13.9) 40 (50.6) 27 (34.2) 9 (11.4) 18 (22.8) 30 (38.0) 18 (22.8) 10 (12.7) 3 (3.8) 0 (0.0) 1.37 (1.11.6)

76.6 (65101) 459 (58.6) 474 (60.1) 155 (19.7) 314 (39.8) 83 (10.5) 387 (49.1) 281 (35.7) 159 (20.2) 188 (23.1) 240 (30.5) 190 (24.1) 108 (13.7) 60 (7.6) 8 (1.0) 1.55 (1.471.64)

reduced the sensitivity to 34%. The sensitivity and specicity of the TRST at 30 days are compared with the results of previous studies6,9,10,13 in Table 3. The AUC for the TRST was maximized at a cutpoint of 2 for the composite endpoint at 30 days and was 0.61 (95% CI 5 0.560.66). The current study explored whether other combinations of variables predicted the composite endpoint better than the TRST. The following variables were independently associated with the composite endpoint in logistic regression: previous ED use in previous 30 days or hospital admission in previous 90 days at all time points, cognitive impairment at 30 days, and use of ve or more medications at 365 days (details not shown). The variable previous ED use in previous 30 days or hospital admission in previous 90 days was the strongest predictor of the composite endpoint at all time points, and addition of a second variable had a negligible effect on the predictive ability over the single-variable model (data not shown). At 30 days, this single-variable predictive model had a sensitivity of 57% (95% CI 5 4865%), a specicity of 64% (95% CI 5 6068%), and an LR of 1.57 (95% CI 5 1.311.85). At 120 days, this model had a sensitivity of 54% (95% CI 5 4760%), a specicity of 67% (95% CI 5 6270%), and an LR of 1.61 (95% CI 5 1.361.89). At 365 days, this model had a sensitivity of 50% (95% CI 5 4556%), a specicity of 68% (95% CI 5 6473%), and an LR of 1.59 (95% CI 5 1.341.89).

DISCUSSION This study validated the predictive performance of the TRST clinical prediction rule in a large multicenter study.18 It conrmed a moderate predictive performance of the TRST for identifying patients who will return to the ED or be hospitalized by 30 and 120 days. Follow-up was ex-

tended to 1 year, and it was found that the sensitivity and specicity of the TRST were comparable at 30, 120, and 365 days. The remarkable consistency of the results with those of previous studies10,13 gives us condence that the estimate reects the predictive ability of the TRST in actual clinical practice. Given this modest predictive performance, is the TRST a clinically useful prediction rule? This depends on two factors: what standard of predictive performance is considered clinically useful and whether an intervention exists that prevents adverse outcomes in the high-risk population identied using the rule. Despite the similarities in their ndings, one previous study9 encouraged the use of the TRST in the ED, whereas the other13 concluded that the TRST was not clinically useful. It is worth examining the standards for clinical usefulness used in these two previous publications to understand the basis of this disagreement. The original study concluded that the TRST performs signicantly better than chance at identifying high-risk patients but did not specify what threshold for predictive performance should be considered clinically signicant or useful.6,9,10 In contrast, the validation study found that an LR greater than 10 for the TRST should be used as the clinically useful diagnostic level.13 A test with such a high positive LR has the ability to make conclusive changes to pretest probability19 and can be used as a stand-alone test, the way a radiograph is used to diagnose a fracture. The TRST was designed to trigger a targeted geriatric assessment and not to be used as a stand-alone test. Thus, the standard for clinical usefulness suggested by that study13 may be overly rigorous. How does the predictive performance of the TRST compare with other ED-based geriatric clinical prediction rules? A clinical prediction rule was developed and validated to identify older ED patients at risk after ED

JAGS

NOVEMBER 2008VOL. 56, NO. 11

VALIDATION OF THE TRST

2115

discharge.11,21,22 The Identifying Seniors at Risk (ISAR) clinical prediction rule identies older ED patients at high risk for hospitalization,20 institutionalization,21 return visits to the ED within 30 days, and frequent ED revisits (42 visits in 120 days).22 An ISAR score of 2 or greater was 72% sensitive for hospitalization within 120 days of the ED visit, with a specicity of 58%, a LR of 1.7, and an AUC of 0.68.20 Thus, the ISAR may have slightly better sensitivity than the TRST, but studies that directly compare the TRST with the ISAR are required before denitive conclusions can be reached. Unfortunately, no standard criterion exists to dene what level of predictive performance should be considered clinically useful. The TRST may still be clinically useful, despite its modest predictive performance, if it performs better than clinicians unaided judgment alone.23,24 Geriatric syndromes are poorly recognized in the ED, and emergency physicians are less condent and make more errors when diagnosing older patients.2528 Older patients have higher rates of return to the ED and hospitalization after discharge than younger patients, suggesting that unaided clinical risk assessment may be suboptimal,2628 but further studies that directly compared unaided judgment with use of the TRST are warranted, because these data could improve the uptake of screening tools by clinicians who may not recognize the limitations of their unaided risk assessments. To be considered useful, a clinical prediction rule should be able to reduce adverse outcomes when linked to an intervention. Previous specialized geriatric service interventions are known to be more effective when targeted at high-risk groups, although none are specic to the ED setting.58 The TRST has been studied in a randomized clinical trial of a comprehensive geriatric assessment and subsequent referrals by an APN.6 Patients identied as high risk using the TRST who were randomized to this intervention had fewer hospital days and fewer nursing home admissions than with usual care. The intervention actually increased return visits to the ED in the rst 30 days without improving other outcomes in low-risk patients. The ISAR has also been used as a screening tool to identify high-risk patients who then received a standardized rapid geriatric assessment followed by referrals to address new problems identied. This two-step intervention reduced functional decline without increasing the cost of care, but healthcare use was unchanged.5 Thus, there is some evidence that interventions exist that may improve outcomes in patients identied as high risk using the TRST and other screening tools, but further studies are needed. It was found that the single-variable model ED use in previous 30 days or hospitalization in previous 90 days had a predictive performance similar to the full set of ve TRST variables and was a signicant predictor of the composite end point in the patient sample at all time points, but this predictive model should be validated in future research. Hospitalization in the previous 6 months was the predictive variable with the strongest association with adverse outcomes in the ISAR prediction rule as well.20 It could be that previous health service use is such a consistent predictor, because it indicates that a patient has a higher than average propensity in all three domains believed to determine ED use: patients predisposition to seek health care

CEP, Sensitivity, No CEP, Specicity, Likelihood CEP, Sensitivity, No CEP, Specicity, Likelihood CEP, Sensitivity, No CEP, Specicity, Likelihood % (95% CI) (n) % (95% CI) (n) Ratio (95% CI) % (95% CI) (n) % (95% CI) (n) Ratio (95% CI) % (95% CI) (n) % (95% CI) (n) Ratio (95% CI)

Table 2. Predictive Performance of the Triage Risk Stratication Tool (TRST) for the Composite End Point (CEP) at 30, 120, and 365 days

1.1 (1.01.2) 1.4 (1.21.7) 1.7 (1.32.2) 2.4 (1.53.8) 1.5 (0.36.2)
Any return to emergency department or admission to hospital. CI 5 Condence interval.

TRST Items

1 2 3 4 5

82 (7588) (121) 62 (5470) (91) 34 (2742) (50) 16 (1124) (24) 1 (05) (2)

24 (2128) (458) 57 (5361) (275) 80 (7783) (126) 93 (9195) (44) 99 (98100) (6)

30 Days

82 (7787) (201) 56 (5063) (138) 28 (2334) (69) 14 (1019) (34) 2 (14) (4)

25 (2229) (405) 58 (5462) (228) 80 (7784) 107) 94 (9196) (34) 98 (98100) 4)

12 Days

1.1 (1.01.2) 1.3 (1.21.6) 1.4 (1.11.9) 2.2 (1.43.5) 2.2 (0.68.0)

81 (7785) (281) 56 (5161) (194) 28 (2333) (97) 12 (916) (43) 1 (14) (5)

26 (2231) (325) 61 (5666) (172) 82 (7886) (79) 94 (9296) (25) 99 (98100) (3)

365 Days

1.1 (1.01.2) 1.4 (1.21.7) 1.6 (1.22.0) 2.2 (1.43.5) 2.1 (0.68.0)

2116

LEE ET AL.

NOVEMBER 2008VOL. 56, NO. 11

JAGS

Table 3. Sensitivity and Specicity of the Triage Risk Stratication Tool (TRST) in the Current Study, and Previous Studies, for the Composite End Point at 30 Days
Sensitivity, % (95% CI) TRST Items Current Study n 5 788 n 5 647 n 5 120 Specicity, % (95% CI) Current Study n 5 788 n 5 647 n 5 120

1 2 3 4 5

82 (7588) 62 (5470) 34 (2742) 16 (1124 1 (05)

85 65 (5772) 39 26 8

64 (4287)

24 57 (5361) 80 93 99

34 62 (5867) 83 94 97

55 (4466)

Results for TRST 2, which is the cutscore that a previous study recommended. CI 5 Condence interval.

from the ED as opposed to primary care, their lack of access to a family physician or other forms of primary care, and their greater need for health care. This conceptual model is a modication of the Andersen behavioral model, the dominant theory used to explain determinants of healthcare use.29 The degree to which previous ED use or hospitalization explains predisposing, enabling, and need factors should be investigated in future qualitative research.

LIMITATIONS TRST forms were completed on only 49% of eligible patients. This is similar to the proportion of patients enrolled in a previous study6,9,10 and probably reects the fact that both studies used pragmatic convenience sampling. Although the age and sex distributions of patients who were enrolled were similar to those of patients who were not enrolled, a selection bias cannot be excluded. It is possible that the emergency nurses and PCCs were more likely to complete forms on patients who seemed to be at highest risk; thus if universal screening of older ED patients using the TRST were implemented, the TRST might perform differently because of a potential spectrum effect, although it is likely that the predictive performance found in the current study reects how the TRST would perform in actual clinical practice. This study did not capture patients who were admitted to nursing homes after ED discharge, but nursing home admissions accounted for only 2.2% of outcomes in one previous study6,9,10 and none in the other.13 Given that the results for return to the ED and hospital admission were so similar to those in the two previous studies, it is unlikely that the rate of nursing home admissions would be high enough in the current population to materially alter the ndings. PCCs were not blinded to the results of the TRST score when determining whether postdischarge referrals were required; thus it is possible that this may have reduced the apparent predictive performance of the TRST by reducing the outcome rate. The similarity of the results to those of the two previous studies suggests that this was unlikely. Because only patients who returned to the three participating centers were captured, patients who experienced the study outcomes but returned to another institution may have been missed. The primary outcome rate in this study was 19% at 30 days and 31% at 120 days, which was lower than the 26% and 48% in one previous study6,9,10 but

similar to the 14% and 32% in the other.13 The effect of missing patients who presented to other centers is difcult to predict, but given the similarity of predictive performance in all three centers, these missed patients might have reduced the precision but not the validity of the ndings. Finally, it was not established whether return ED visits were planned at the time of discharge (e.g., returning for a dressing change). Including planned visits might have reduced the precision of the ndings as well, but this is unlikely to have substantially altered the results, because planned follow-up in the ED is discouraged because of overcrowding.

CONCLUSION The TRST demonstrated only moderate predictive ability in this independent patient sample. Ideally, a better prediction rule should be sought. Future studies are needed to develop prediction rules and to directly compare their predictive performance with existing prediction rules including the TRST and the ISAR. To demonstrate the clinical utility of any new prediction rules, studies will need to show that interventions for the high-risk individuals identied by these rules can reduce their adverse outcomes. ACKNOWLEDGMENTS Conict of Interest: The editor in chief has reviewed the conict of interest checklist provided by the authors and has determined that the authors have no nancial or any other kind of personal conicts with this manuscript. Dr. Gary Naglie was supported by the Mary Trimmer Chair in Geriatric Medicine Research, University of Toronto. Author Contributions: Jacques Lee: concept, design, analysis, interpretation, preparation. Graeme Schwindt and Rola Moghabghab: design, data collection, analysis, interpretation, preparation. Mara Langevin: design, data collection, analysis, preparation. Shabbir Alibhai: data collection, analysis, interpretation, preparation. Gary Naglie: analysis, interpretation, preparation. Sponsors Role: None. REFERENCES
1. U.S. Department of Health and Human Services, Centers for Disease Control and Prevention. Special Excerpt: Trend Tables on 65 and Older Population [on-line]. Available at: www.cdc.gov/nchs/data/misc/hus2003excerpt.pdf Accessed May 253, 2008.

JAGS

NOVEMBER 2008VOL. 56, NO. 11

VALIDATION OF THE TRST

2117

2. Rice DP, Fineman N. Economic implications of increased longevity in the United States. Annu Rev Public Health 2004;25:457473. 3. Jones JS, Rousseau EW, Schropp MA et al. Geriatric training in emergency medicine residency programs. Ann Emerg Med 1992;21:825829. 4. Hedges JR, Singal BM, Rousseau EW et al. Geriatric patient emergency visits. Part II: Perceptions of visits by geriatric and younger patients. Ann Emerg Med 1992;21:808813. 5. McCusker J, Jacobs P, Dendukuri N et al. Cost-effectiveness of a brief twostage emergency department intervention for high-risk elders: Results of a quasi-randomized controlled trial. Ann Emerg Med 2003;41:4556. 6. Mion LC, Palmer RM, Meldon SW et al. Case nding and referral model for emergency department elders: A randomized clinical trial. Ann Emerg Med 2003;41:5768. 7. Applegate WB, Miller ST, Graney MJ et al. A randomized, controlled trial of a geriatric assessment unit in a community rehabilitation hospital. N Engl J Med 1990;322:15721578. 8. Miller ST, Applegate WB, Elam JT et al. Inuence of diagnostic classication on outcomes and charges in geriatric assessment and rehabilitation. J Am Geriatr Soc 1994;42:1115. 9. Meldon SW, Mion LC, Palmer RM et al. A brief risk-stratication tool to predict repeat emergency department visits and hospitalizations in older patients discharged from the emergency department. Acad Emerg Med 2003;10: 224232. 10. Meldon SW, Mion LC, Palmer RM et al. Does a simple emergency department (ED) screening tool correlate with functional impairments among older adults? Acad Emerg Med 2004;11:448. 11. McCusker J, Bellavance F, Cardin S et al. Screening for geriatric problems in the emergency department: Reliability and validity. Identication of Seniors at Risk (ISAR) Steering Committee. Acad Emerg Med 1998;5: 883893. 12. Caplan GA, Brown A, Croker WD et al. Risk of admission within 4 weeks of discharge of elderly patients from the emergency departmentFthe DEED Study. Discharge of Elderly from Emergency Department. Age Ageing 1998;27:697702. 13. Fan J, Worster A, Fernandes CM. Predictive validity of the triage risk screening tool for elderly patients in a Canadian emergency department. Am J Emerg Med 2006;24:540544. 14. Armitage P. Attitudes in clinical trials. Stat Med 1998;17:26752683. 15. Buderer NM. Statistical methodology: I. Incorporating the prevalence of disease into the sample size calculation for sensitivity and specicity. Acad Emerg Med 1996;3:895900.

16. Hanley JA, McNeil J. The meaning and use of the area under a receiver operating characteristics (ROC) curve. Radiology 1982;143:2936. 17. Harrell F. Regression Modeling Strategies. New York: Springer-Verlag, 2002. 18. Lee J, Langevin M, Schwindt G et al. Validation of the Triage Risk Stratication Tool TRST Scale to identify elders at risk for returning to the emergency department. J Am Geriatr Soc 2005;53:S131. 19. Jaeschke R, Guyatt GH, Sackett DL. Users guides to the medical literature. III. How to use an article about a diagnostic test. B. What are the results and will they help me in caring for my patients? The Evidence-Based Medicine Working Group. JAMA 1994;271:703707. 20. McCusker J, Bellavance F, Cardin S et al. Prediction of hospital utilization among elderly patients during the 6 months after an emergency department visit. Ann Emerg Med 2000;36:438445. 21. McCusker J, Bellavance F, Cardin S et al. Detection of older people at increased risk of adverse health outcomes after an emergency visit: The ISAR screening tool. J Am Geriatr Soc 1999;47:12291237. 22. McCusker J, Cardin S, Bellavance F et al. Return to the emergency department among elders: Patterns and predictors. Acad Emerg Med 2000;7: 249259. 23. Bandiera G, Stiell IG, Wells GA et al. The Canadian C-spine rule performs better than unstructured physician judgment. Ann Emerg Med 2003;42: 395402. 24. Stiell IG, Clement CM, McKnight RD et al. The Canadian C-spine rule versus the NEXUS low-risk criteria in patients with trauma. N Engl J Med 2003;349:25102518. 25. McNamara RM, Rousseau E, Sanders AB. Geriatric emergency medicine: A survey of practicing emergency physicians. Ann Emerg Med 1992;21: 796801. 26. Singal BM, Hedges JR, Rousseau EW et al. Geriatric patient emergency visits. Part I: Comparison of visits by geriatric and younger patients. Ann Emerg Med 1992;21:802807. 27. Aminzadeh F, Dalziel WB. Older adults in the emergency department: A systematic review of patterns of use, adverse outcomes, and effectiveness of interventions. Ann Emerg Med 2002;39:238247. 28. Grief CL. Patterns of ED use and perceptions of the elderly regarding their emergency care: A synthesis of recent research. J Emerg Nurs 2003;29: 122126. 29. McCusker J, Karp I, Cardin S et al. Determinants of emergency department visits by older adults: A systematic review. Acad Emerg Med 2003;10: 13621370.