FormalPara Clinician’s capsule

What is known about the topic?

Little is known about the agreement or prognostic value among the InterRAI ED Screener, ER2, and PRISMA-7 in older ED patients.

What did this study ask?

Do these instruments agree in their classification of high-risk older ED patients and can they predict ED outcomes?

What did this study find?

All instruments could predict the need for hospitalization but performed poorly when predicting LOS. Instruments had fair agreement, and the ED Screener was most specific at predicting ED outcomes.

Why does this study matter to clinicians?

We provide data to support decision-making about which ED vulnerability screener to consider to best meet the needs of older patients and ED providers.

Introduction

Emergency departments (ED) are caring for an ever-increasing number of older adults presenting with complex medical and social needs [1]. Acuity-driven emergency management pathways habitually overlook the intricate and patient-specific needs of older ED patients [2, 3]. To improve the patient experience and health outcomes of older ED patients, geriatric emergency guidelines and experts have emphasized the importance of proactively targeting patients at risk for adverse health events, and thus in greatest need of a detailed geriatric assessment [4,5,6]. Comprehensive geriatric assessment is the gold standard approach when evaluating older adults. However, its administration may be difficult in the dynamic and fast-paced environment of the ED where staffing, resources, and medical acuity, are continuously changing [6,7,8].

Utilizing vulnerability screeners in the ED has been proposed as a pragmatic method to rapidly identify high-risk older patients for early intervention in the emergency management pathway [6]. Geriatric syndromes are commonly overlooked by ED clinicians and acuity-driven emergency models of care [9, 10], though they are more likely to be identified when using a vulnerability screener [11]. Thus, vulnerability screeners are used to identify and triage patients with high-risk geriatric syndromes (e.g., frailty, functional or cognitive impairment) that increase the risk of adverse health events [6, 12, 13].

Several ED vulnerability screeners have been derived and validated, with the interRAI ED Screener [14, 15], Emergency Room Evaluation and Recommendations (ER2) [16,17,18], and the Program of Research on Integration of Services for the Maintenance of Autonomy (PRISMA-7) [19,20,21] most commonly used in Canadian ED settings. These screeners have also been implemented and appraised in older ED populations internationally [15, 16, 20, 22,23,24,25,26,27]. The interRAI ED Screener, ER2, and PRISMA-7 prioritize and evaluate several shared geriatric syndromes, though little is known about the agreement between these instruments in classifying patients as 'high-risk.' Foreknowledge of geriatric complexity can support decision-making for clinicians and policymakers about which vulnerability screener to consider and which cluster of assessment items can best support targeted referrals to geriatric services from the ED, given the limited resources available.

In this study, we proposed to determine the level of agreement between the InterRAI ED Screener, ER2, and PRISMA-7. We also compared the discriminative accuracy of these instruments in predicting two patient-important outcomes, discharge to home and extended lengths of stay (LOS) in the ED.

Methods

Study design and setting

We conducted a nested sub-group study using data from a prospective multi-province cohort study evaluating frailty in older ED patients. Originally, study sites were spread between Ontario, Quebec, and Newfoundland, and all sites administered an ED vulnerability screener. For this study, we only evaluated data from Quebec hospitals, as they were the only sites to administer all three ED vulnerability screening instruments to eligible patients. Patients aged 65 years and older were consecutively screened for study eligibility by research staff during ED registration, based on a pre-defined study enrolment period of January 1, 2017–July 31, 2018. We obtained ethics approval from the research ethics boards of all participating hospital sites and the University of Waterloo. We used the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement to guide the reporting of this study [28].

Population

We recruited patients for the study during daytime hours (0700–1900) when older adults are more likely to present for emergency care [29]. Older adults were identified using ED rosters upon the start of the day shift (0700 h). Research staff completed the three instruments during the triage process or within a few hours of admission to the ED. We excluded patients determined to be medically unstable or in need of emergent care or resuscitation, leveraging the clinical decision-making of the triage nurse. We also excluded patients if they, or their caregivers, could not answer questions in either French or English. We obtained consent to participate from the patient or their substitute decision-maker.

Intervention: ED vulnerability screening

Research nurses assessed patients with the three vulnerability screeners after receiving standardized training on the instruments and supplementary software, where necessary. They also collected data on demographics and patient outcomes. Data from the vulnerability screeners were not made available to the clinical staff. Thus, risk scores from the instruments did not influence emergency management pathways or clinical decision-making. Nurses were neither trained to evaluate agreement statistics nor responsible for this task.

InterRAI ED Screener

The interRAI ED Screener is part of a multi-stage assessment system whereby ED screener findings inform the need for additional evaluation with a comprehensive geriatric assessment in-hospital or following discharge [6]. The interRAI ED Screener is designed to identify and triage older adults most likely to benefit from a secondary detailed geriatric assessment (e.g., interRAI ED Contact Assessment) and can be completed within minutes [14]. The interRAI ED Screener includes questions regarding activities of daily living (ADLs), cognitive skills for daily decision-making, self-rated mood and health, caregiver distress, unstable medical condition, and dyspnea [14]. The ED Screener is sequenced and adapted based on prior item responses, with most high-risk persons identified within a few screening questions [14], and scores range from one to six. The presence of any functional impairment automatically scores patients as moderate risk (≥ 4), with priority further driven by indicators of caregiver distress, low mood and poor self-hygiene [14]. The interRAI ED Screener is free, readily available (www.interRAI.org), and has proven to be predictive of patient-important outcomes, including mortality, lengthy hospital stays, and the need for more formal support following discharge [15, 22, 30].

PRISMA-7

The PRISMA-7 is designed to identify older adults with disabilities and can be completed within a few minutes [19, 20, 27]. The instrument has been reported to accurately classify frail ED patients compared to the results of a comprehensive geriatric assessment and is suitable for completion by emergency nurses and physicians [27, 31]. The PRISMA-7 consists of seven questions examining the patient's age, sex, limitations with activities of daily living or travel outside of the house, the need for regular support, the presence of a dependable support system, and the use of locomotive assistive devices (e.g., cane) [19]. The number of positive items flagged is summed to create a final score ranging from zero to seven. Scores of zero to two are classified as low-risk, and scores of three and greater as high-risk [19]. The PRISMA-7 has been widely accepted and utilized among many Quebec hospitals to evaluate high-risk older ED patients [32].

ER2

The ER2 is a brief questionnaire designed to identify older ED patients at high risk of short-term adverse events (e.g., ED length-of-stay, hospital admission). In prior evaluations, the instrument took less than three minutes to complete [16, 18]. A high-risk classification and a positive flag for temporal disorientation are closely linked with the presence of a major neurocognitive disorder [17]. The ER2 consists of six closed-ended questions about the patient's age, sex, the need for support in the home, polypharmacy, the use of a walking aid, and the ability to identify the current year and month correctly [16]. The latter two items carry five times the weight when positive, resulting in a scale ranging from zero to fourteen. The authors report that a score of zero to three is classified as low risk, four and five as moderate risk, and six or greater as high risk [16]. We elected to include the ER2, given its evaluation of well-established geriatric syndromes and the recent development and implementation of this instrument within Quebec hospital systems [17].

Outcome measures

The primary outcome of our study was a high-risk score from any of the three ED vulnerability screeners. Specifically, we were interested in the overall agreement between the three screeners in classifying high and low-risk patients. We calculated agreement in a post-hoc fashion, given the secondary nature of this study. We re-classified outputs from all instruments into 'high' and 'not high-risk' categories to account for the lack of a 'moderate risk' stratification group for the PRISMA-7. We defined high-risk using the cut-offs reported originally during instrument derivation [14, 16, 19]. Secondary outcomes for this study included discharge-to-home and an extended ED length-of-stay (> 24 h). We elected to use this cut-off as it conceptually represents a full day of care in the ED, is common among older ED patients, and better signals high-risk older ED patients than a 12-h cut-off [33]. ED registration time was used as the point of reference when determining ED length-of-stay, and discharge time as the end-point. Both discharge location and ED length-of-stay are a priority concern for patients [34, 35].

Data analysis

We reported descriptive statistics of all baseline characteristics using general frequency and central tendency measures. We evaluated instrument agreement using Cohen's Kappa. We utilized logistic regression to determine the discriminative ability of the three screener scores in predicting patient outcomes in their dichotomous, full, and statistically adjusted forms. Truncated models included vulnerability screeners using the cut-offs proposed by the original authors. Specifically, we used a cut-off of five or greater for the interRAI ED Screener, six or greater for the ER2, and three or greater for the PRISMA-7. We also conducted a sensitivity analysis testing agreement with the PRIMSA-7 using a cut-off of four or greater, mindful that many hospitals in Quebec use this cut-off to support decision-making and patient trajectories. Full-form models used the entire range of screener scores. Building on full-form models, adjusted models controlled for patient age, sex and academic status of the treating institution. Concordance statistics and the corresponding confidence intervals are reported for all models. We evaluated the statistical significance of differing concordance statistics using methods proposed by DeLong [36]. Cases with missing data were deleted within each analysis. We managed and analyzed data using the stats, psych, and pROC packages in R (version 4.0.0).

Results

A total of 1855 older ED patients were evaluated across the four hospital sites using all three vulnerability screeners. The mean age of the sample was 84 years, and most participants were female (58%). The mean LOS in the ED was 20.1 h, and many were classified as high-risk using the interRAI ED Screener (38.3%), PRISMA-7 (83.2%), and ER2 (59.1%). Over one-third of patients were discharged home from the ED (36%) or experienced an extended LOS in the ED (37%). Table 1 displays the patient profiles and screener scores assigned. The distribution of vulnerability screener scores can be seen in their full un-truncated form in the supplementary file. Missing data was only present for outcome data, though it was scant (< 1%). A flow diagram of all patients evaluated can be found in Fig. 1.

Table 1 Baseline characteristics for 1855 older ED patients evaluated in Quebec, Canada with interRAI ED Screener, ER2, and PRISMA-7
Full size table
Fig. 1
figure 1

Flow diagram of vulnerability screener use across study sites

Full size image

Agreement

Tables 2, 3 and 4 display the cross-tabulations of vulnerability screener scores and their agreement when classifying those at high-risk (i.e., Cohen's Kappa values). The overall agreement between the interRAI ED Screener and the ER2 was fair (K = 0.37; 95% CI 0.33–0.40). A comparable level of agreement between the PRISMA-7 and the ER2 was also noted (K = 0.39; 95% CI 0.36–0.43), though improved when using a cut-off of four or greater for the PRISMA-7 (K = 0.55; 95% CI 0.51–0.59). The agreement between interRAI ED Screener and the PRISMA-7 was poor using a cut-off of three or greater for the PRISMA-7 (K = 0.19; 95% CI 0.16–0.22). However, when using a cut-off of four or greater, the agreement between these two instruments improved (K = 0.32; 95% CI 0.29–0.36). Figure 2 displays the unique and shared attributes among vulnerability screeners used to classify high-risk patients.

Table 2 Crosstabulation of agreement between 1855 older ED patients assessed with the interRAI ED Screener and ER2
Full size table
Table 3 Crosstabulation of agreement between 1855 older ED patients assessed with the interRAI ED Screener and PRISMA-7
Full size table
Table 4 Crosstabulation of agreement between 1855 Older ED patients assessed with the PRISMA-7 and ER2
Full size table
Fig. 2
figure 2

Shared and unique attributes among the interRAI ED Screener, ER2, and PRISMA-7

Full size image

Discriminative ability of vulnerability screeners

Table 5 displays the discriminative accuracy of all models in their dichotomous, full, and adjusted forms.

Table 5 Discriminative accuracy of three ED vulnerability screeners in predicting discharge home and extended ED length of stay in 1841 older ED patients
Full size table

Discharge Home

The discriminative ability of the three ED vulnerability screeners were similar when evaluated in their intended form (c = 0.57–0.61), full form (c = 0.61–0.64), and adjusted form (c = 0.63–0.65). These models predicted hospital discharge with fair accuracy. There was no significant difference in model accuracy between the three instruments (p > 0.05). Similarly, assessing the models in their full and adjusted form did not significantly improve the discriminative accuracy of statistical models.

In adjusted models, the academic status of the treating institution significantly increased the odds of a discharge home in models evaluating the interRAI ED Screener (OR = 1.59; 95% CI 1.26–2.0) and the PRISMA-7 (OR = 1.46; 95% CI 1.1–1.75). Similar results were found for the ER2 (OR = 1.42; 95% CI 1.13–1.79). Male patients were also more likely to be discharged home when using the PRISMA-7 (OR = 1.46; 95% CI 1.17–1.8) and ER2 (OR = 1.28; 95% CI 1.02–1.6), the only screeners that evaluate sex.

Extended ED length-of-stay (> 24 h)

The discriminative accuracy of all three models was poor when predicting ED length-of-stay. All screeners had similar performance metrics, with concordance statistics ranging from 0.51 to 0.55. The adjusted model examining the discriminative accuracy of the interRAI ED Screener was the only multivariable model to note a significant association between academic status and a decreased risk for an extended LOS in the ED (OR = 0.78; 95% CI 0.62–0.97).

Discussion

Interpretation

When comparing ED vulnerability scores with one another, the interRAI ED Screener, ER2 and PRISMA-7 had fair agreement amongst themselves, despite sharing a number of similar assessment items. ED vulnerability scores from the three instruments were associated with patient-important and clinically relevant outcomes, including discharge home and length of stay in the ED. These tools have prognostic value, despite being designed to direct individuals for more detailed assessments or towards geriatric care pathways.

Previous studies

Few, if any, studies examined agreement between the three ED vulnerability screeners. However, the age and sex distributions of our study were similar to other studies evaluating the use of vulnerability screeners in the ED, with most studies having mean ages over 80 years and a greater proportion of female patients. Prior studies of agreement found the interRAI ED Screener demonstrates fair-to-good agreement in classifying high-risk older ED patients when compared with the Acutely Presenting Older Patient (APOP) screener and Identification of Seniors at Risk (ISAR) screener [37]. When using a comprehensive geriatric assessment as the reference standard to classify frail patients, the PRISMA-7 performed better but similar to the ISAR and the Clinical Frailty Scale when discriminating patients with frailty [31]. Our study supports prior work demonstrating a positive association between high-risk screener classifications, hospital admission and an extended ED LOS, when using the interRAI ED Screener [15, 30], PRISMA-7 [20], or ER2 [16, 21]. Our study differs in our use of a longer 24-h cut-off to define an extended stay. However, this likely promotes the external validity of our findings to other Canadian provinces where ED wait times are known to be lengthy [38, 39].

Strengths and limitations

A key strength of our study was the inclusion of multiple and diverse sites actively involved in recruiting and evaluating patients. In addition to academic and community sites with a variety of referral patterns in several health systems, we also included a wide variety of patients including older adults who presented with cognitive impairment, a cohort commonly excluded in clinical and emergency research.

We also note some limitations. We were pragmatic and cost constrained in our approach, so minimal clinical data were collected. Data on presenting complaints, diagnoses, and triage acuity would likely have enriched our analysis. Additionally, data on mortality were not collected, though prior work has demonstrated than less than 1% of older Canadian patients will die in the ED [40], and approximately 5% will die within one month of presentation [41, 42]. Finally, since we used ED discharge as the end-point for the patient journey rather than ED disposition decision, factors related to ED and hospital system care may confound our understanding of the association between vulnerability screeners and LOS.

Clinical implications

All three vulnerability screeners had similarities and key differences in their operating characteristics. When we examine disagreements, we note that the interRAI ED Screener classified 27% of individuals as low risk, while the ER2 classified them as high risk. When compared to the PRISMA-7 score, interRAI classified 45% of individuals who were considered high-risk based on PRISMA-7 scores. Thus, if ED care providers hope to identify the highest-risk individuals, then the interRAI ED Screener would be the best choice. If, on the other hand, ED care providers hope to use a more sensitive instrument, then using PRISMA-7 and to a lesser extent the ER2 would seem most appropriate. These differences may be a result of slight differences in questions, or the fact that the interRAI ED Screener is a response-adaptive algorithm.

Our study found all vulnerability screeners were able to classify those in need of hospital admission with fair accuracy even after adjusting for age, sex, and academic status of the treating institution. It is worth noting that only the ER2 was the only instrument specifically purposed to predict health service outcomes. Given the complexity of causal pathways for disposition and length of stay, it is noteworthy if any positive association is detected. These associations justify the need to identify high-risk individuals, propose detailed assessments, and consider care paths to ensure a timely and efficient response to complex geriatric syndromes and unmet social needs.

Research implications

From our work, we believe further work is required to understand how screening strategies may best be included in care pathways that make use of more detailed geriatric assessments or consultations from geriatric specialists. In addition, further work may examine whether screening strategies may be used to guide interventions designed to decrease admission rates or mitigate the consequences of ED visits.

Conclusion

Our study demonstrated an association between all included ED vulnerability screeners and patient-important outcomes. Thus, any of these three ED vulnerability screeners can support the triage and care of high-risk older adults in the ED. However, the interRAI ED Screener classified more individuals as low risk compared to the ER2 and PRISMA-7. Agreement among instruments improved using a PRIMSA-7 cut-off of four or greater. Our research supports the implementation of ED screening tools.