Introduction

Older adults are high users of hospital services, in both emergency departments (EDs) and inpatient care. Transitional care, comprising enhanced discharge planning in coordination with community-based home-care services, can be an effective strategy to reduce rates of readmission [1, 2]. Targeting enhanced transitional care interventions by identifying patients at increased risk of readmission early during the admission may be an efficient approach to delivering these services and to optimizing patient outcomes [3].

Various tools have been developed to predict hospital readmission. In this study, we used the Identification of Seniors At Risk (ISAR), a 6-question self-report screening tool originally developed for use in the ED. ISAR is one of the most widely used risk stratification tools among seniors in the ED [4]. Test–retest reliability, content validity, concurrent validity, and predictive validity for a range of adverse outcomes (functional decline, mortality, hospitalization, community service utilization) have been established [5,6,7,8,9]. ISAR is used typically as step 1 of a 2-step intervention, in which it is used to target patients for a step 2 clinical assessment of geriatric problems requiring tailored intervention. Interventions targeted in this way have demonstrated positive effects on functional decline and other outcomes, including cost-effective use of resources [10,11,12,13,14].

Based on a meta-analysis of 32 ISAR validation studies, Galvin concluded that at the recommended cut-point of 2 +, ISAR can be a valuable clinical decision-making adjunct, essentially as a “rule-out” tool for safely discharging patients from the ED [4]. Galvin argues that the two primary metrics for a risk stratification tool that aims to predict those with adverse outcomes (the criterion for predictive validity) are the sensitivity (the proportion of those with the criterion detected by the tool) and the negative predictive value (among those with negative test results, the proportion that do not experience the criterion). Recommendations to improve the tool included exploring differential weighting of items and raising the polypharmacy threshold. There were too few studies of patients admitted to hospital for separate analysis [4]. There is a need for more research on ISAR as a risk stratification tool in the hospitalized population.

The Revised ISAR (ISAR-R) is a revision of the original ISAR made by two of the authors (RW and JM) in a Canadian ED as part of the Elder Alert geriatric assessment and management program, through a process of repeated quality improvement cycles [15]. Modifications included: setting a higher threshold for the polypharmacy question (6 + versus 4 + on original ISAR to reduce the number of patients with a positive result) and rewording two other items to improve their clarity and ease of scoring (Additional file 1). Although the ISAR-R is widely used, its psychometric properties have not been evaluated. Based on our correspondence with ISAR users, the majority are using the ISAR-R (list of users available upon request). For example, the ISAR-R has been adopted for use in the U.S. Veterans Affairs hospitals, including the GERI-VET program [14]. Note that this latter program used the ISAR-R although the original ISAR was referenced instead (personal communication, Dr. Huded).

We had the opportunity to conduct a secondary analysis of the performance of the ISAR-R in the prediction of readmission among hospitalized patients in the context of an evaluation of an enhanced discharge planning intervention. Clinical staff were blinded to the ISAR-R score. Secondary objectives were: to examine the performance of the ISAR-R in sub-groups defined by age, informant (patient or caregiver), and language; and to examine the predictive performance of individual ISAR-R items.

Methods

Study design

This was a secondary analysis of data collected between October 2018 and December 2019 to evaluate the implementation of an enhanced discharge education intervention (McCusker J, Beauchamp S, Lambert SD, Yaffe MJ, Meguerditchian AN, John B-T, et al: Improving transitional care for seniors: results of a patient-centered quality improvement intervention, submitted). Older adults discharged home following admission to the medical (n = 4) or geriatric (n = 2) units of two Canadian general acute-care hospitals were enrolled before (PRE) and during (POST) implementation of the intervention. Length of stay, both on the unit and overall, and 30- and 90-day readmissions were extracted from administrative data kept by the two hospitals (insufficient resources were available to use provincial administrative data). In this study we combined the PRE and POST cohorts because there was no significant change in the readmission rate (23.4% PRE to 21.9% POST, OR 0.96, 95% CI 0.81, 1.13). The study protocol and consent forms were approved by the Research Ethics Committee responsible for both hospitals. All methods were carried out in accordance with relevant guidelines and regulations.

Recruitment

Patients were recruited as soon as possible after admission to the study units (including those transferred from other hospital units). Patients were enrolled at their first admission during the study period. Excluded were: patients admitted from a long-term care setting, those who were expected to be discharged to a long-term care setting, and those patients (or caregivers) who were unable to speak and read in English or French (See flowchart in Fig. 1).

Fig. 1
figure 1

Flowchart

Patient consent was sought for an in-hospital structured interview and a telephone follow-up after discharge to ask about discharge experiences (reported elsewhere). If clinical staff judged that the patient was not capable of informed consent (for physical or mental reasons) these patients were invited to provide assent for study staff to contact their family caregiver (unpaid family member or friend), who was invited to participate as a proxy respondent for the patient.

Data collection

After consent, we conducted a short (5–10 min) structured interview to collect the following information on the patient: level of education; language spoken at home; country of birth (Canada vs other); receipt of local authority (CLSC) home care services; and the ISAR-R. [15] (Additional file 1). ISAR-R was used as a covariate in the main study; results were not disclosed to hospital staff. If a patient responded positively to the ISAR-R question on hospitalization during the previous 6 months, we asked whether they had been admitted to a different hospital. The answers enabled us to estimate the likely under-reporting of readmissions.

Patient data were linked to the hospital discharge databases of their admission hospital using their medical record number. The databases were used to extract hospital admissions during the 90 days after discharge, as well as discharge diagnoses during the 12 months before admission, to compute the Charlson Comorbidity Index [16], a widely-used measure of multimorbidity. (As noted above, insufficient resources were available to use provincial administrative data.)

Sample derivation and statistical analysis

A total of 711 patients were identified by research staff as eligible and available to approach, and were invited to participate in the study. Of these, 485 (68.2%) consented and completed the in-hospital interview. Of these, 398 (82.1%) were discharged back home; 12 had one or more missing ISAR-R revised items, leaving 386 with complete ISAR-R data and formed the main analysis sample (see flowchart in Fig. 1).

To assess the representativeness of the main analysis sample (group A), we compared demographic and other characteristics with those of the following 3 groups: B—those with one or more missing ISAR-R responses (n = 12); C—those excluded because they had not been discharged home (n = 87); D—other patients aged 65 + discharged home from the same units during the study period (n = 2,878). The latter group was defined from hospital administrative data on all patients aged 65 + discharged home, excluding those in the structured interview sample; for those with more than one admission during the study period, we randomly selected one admission. It was not possible to identify patients in group D who would have been ineligible due to language or cognitive impairment with no proxy informant.

To compare the study sample with the excluded groups, we computed Chi-square tests for categorical variables and t-tests for continuous variables; the Kruskal–Wallis test was performed for skewed distributions [17]. The Bonferroni correction was applied at alpha 0.05 to account for multiple testing (level of significance after Bonferroni correction = 0.016) [18].

The performance of the ISAR-R on 30-day and 90-day readmission outcomes was assessed with sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV), computed for each outcome and ISAR-R cut-point. Area Under the ROC (receiver operating characteristic) Curve (AUC) and 95% confidence intervals were computed for each outcome to estimate the overall performance of the ISAR-R score [19, 20]. For each outcome, the AUC (95% confidence interval) of the ISAR-R score was computed by age group, informant (patient vs caregiver), language and previous hospital admission at different hospital; AUC values were compared between the subgroups [20].

The associations between the 6 ISAR-R items and each outcome (30-day and 90-day readmission) were analyzed with logistic regression. For each outcome, two multivariable models were fitted; the first model included all ISAR-R items; the second model also included the covariates (hospital of index admission, service, age, previous hospitalization at different hospital, language spoken at home, informant). Odds ratios and 95% confidence intervals were computed for all ISAR-R items, the C-statistic (concordance statistic) was computed from the logistic regression model [21]. The C-statistic is equivalent to the AUC, values closer to 1 indicate better performance of the model at correctly classifying outcomes.

The performance of two shorter versions of the ISAR-R, excluding items with lower predictive value for the outcomes, was analyzed following the same approach used for the full ISAR-R. All the analyses were conducted with Stata (version 15.1, Stata Corp, College Station, TX).

Results

In Table 1, we compare the characteristics of the study sample (group A) with those excluded or not participating in the primary study. No significant difference was observed between groups A and B (missing ISAR-R items). Those not discharged home (group C) were more often recruited at hospital 1, had longer overall hospital stays, ISAR-R was completed more often by the caregiver, and more frequently endorsed ISAR-R item 5 (memory problems). In comparison with other patients aged 65 + discharged home from the study units during the same time period (group D), the study sample was more likely to be discharged from hospital 2, to be less than 85 years old, and to have had a hospital admission in the previous 6 months.

Table 1 Comparison of study group with three other patient groups

In Table 2 we present the performance characteristics of all possible cut-points of ISAR-R for the 2 outcome variables, 30- and 90-day readmission (13.2 and 24.9%, respectively). (Note than the 90-day readmissions include the 30-day readmissions.) For 90-day readmission as the outcome, at the recommended cut-point of 2 + , 65% of participants have a positive score; sensitivity is 81%, specificity is 40%, positive predictive value is 31%, and negative predictive value is 87%; AUC is 0.63 (95% CI 0.57, 0.69). Using a higher cut-point of 3 +, the number testing positive decreases markedly (65% to 39%) as does the sensitivity (81% to 51% at 90 days); specificity increases to 65%. However, the negative predictive value remains high.

Table 2 Performance criteria for all ISAR cut-points and readmission at 30 and 90 days after discharge (n = 386)

Table 3 shows results of analyses of AUC across sub-groups defined by age, informant, language spoken, and previous hospitalization at a different hospital. AUC varied only by age-group for 30-day readmission only, being higher in age 65–74 [0.76 (0.63, 0.89)] than in those aged 75 and over, significantly so for age 85 and over.

Table 3 Area under the curve (AUC) and 95% confidence interval (CI) across subgroups (n = 386)

Table 4 shows the logistic regression models for each ISAR-R item and the two outcomes. Multivariable model 1 includes only the 6 ISAR-R items; model 2 also includes the covariates, hospital, unit, age, previous hospitalization at a different hospital, language, and informant. Previous hospitalization is the only significant predictor of 30-day readmission in the multivariable models 1 and 2. In model 1 for 90-day readmission, item 2 (more help needed than before admission) is also significant; in model 2, only item 4 (problem with vision) is significant. Notably, items 1 (need for help on a regular basis) and 6 (polypharmacy) show little association (OR less than 1.5) with either outcome, and item 5 (problem with memory) has an OR greater than 1.5 only in the univariate model for 90-day readmission.

Table 4 Logistic regression models for individual ISAR-R items and readmission outcomes (n = 386)

Additional file 2 shows the performance characteristics of two shorter versions of ISAR-R comprising items 2-4 and 2-5, respectively. Values for the AUC are somewhat higher than for the 6-item ISAR-R (0.64-0.66).

Discussion

The revised ISAR-R evaluated in this study is widely used, although only previously studied in a quality improvement context [15]. Thus, our results provide preliminary evidence of its predictive validity, as well as adding to the small number of studies of ISAR when used in hospitalized patients. Overall, the results are similar to those of previous studies of ISAR. The main advantages of the ISAR-R are the higher threshold for polypharmacy, reducing the number of patients who will screen positive, and the more intuitive phrasing and scoring of questions, facilitating administration and scoring [15].

The main changes made in the ISAR-R were in two items: problems with vision and polypharmacy (Additional file 1). The vision question was rephrased to avoid reverse scoring: the original question was “In general, do you see well?” with a “no” response scored as 1. In the revised ISAR-R, the question became “In general, do you have serious problems with your vision that cannot be corrected with glasses?”, with a “yes” response scored as 1. This question was one of the most important predictors of outcome in the current study, particularly for 30-day readmissions, with an unadjusted OR of 2.85 and OR adjusted for other ISAR-R items of 2.84. For 90-day readmission, the unadjusted and adjusted ORs were 1.75 and 1.64, respectively. For comparison, in a previous study the unadjusted and adjusted ORs for 6 month top decile of hospital days were 1.08 and 1.22 [7]. It appears that the revised vision question may improve performance as well as facilitating ISAR-R administration.

As regards polypharmacy (ISAR-R item 6), the threshold was increased from 4 + medications in the original ISAR to 6 + because the increase over time in the number of medications used by older people increased the number of patients with positive screens [15, 22, 23]. In our previous research, the polypharmacy question (4 + medications) had unadjusted and adjusted ORs of 1.43 and 1.57, respectively [7], compared to 1.25 and 1.08 in the current study using the 6 + threshold. It appears that the revised question may be less predictive of readmission than the original one. Furthermore, a high proportion of patients (61%) answered yes to this question. Possibly, use of a higher threshold may be more discriminating. Another item that performed poorly in our study is item 1 on needing help on a regular basis. In our original research on ISAR and prediction of 6 months hospitalization, unadjusted and adjusted ORs were 1.36 and 1.78 [7], respectively, versus 1.16 and 0.94 in current study. Interestingly, although items 1 and 6 did not contribute meaningfully to the prediction of readmission, the elimination of these items does not appreciably improve the performance of the tool in predicting readmission. Further research on prediction of functional decline and other outcomes by these items is warranted.

ISAR-R performed better in those aged 65–74 than in older age groups in predicting 30-day readmission, perhaps due to better cognitive functioning and more accurate reporting. The original ISAR performed similarly in different age groups in predicting various outcomes [6, 7]. As in previous studies of ISAR, ISAR-R performed similarly when using patient or proxy informants and in different language groups. These properties enhance the feasibility of using ISAR-R in diverse patient populations.

Only three previous studies, to our knowledge, have investigated the performance of the original ISAR in predicting readmission among hospitalized patients [24,25,26]. Our study found that ISAR-R had sensitivity, NPV, and specificity values within the ranges previously reported: high sensitivity (76–86%) and NPV (79–90%) but modest specificity (33–44%) [24,25,26]. The AUC was reported in only one of these studies as 0.60 (95% CI 0.55, 0.65) [26]; our estimate was somewhat higher (0.63, 95% CI 0.57, 0.69). These results indicate modest predictive performance of the tool, consistent with meta-analyses [4, 27].

While these results do not justify the use of either ISAR or ISAR-R as a stand-alone clinical prediction tool, these tools can be used as adjuncts in clinical decision-making. The high values of sensitivity indicate that the great majority of those at high risk of readmission will be detected; the high negative predictive value make the tool useful to rule out patients at risk of readmission, useful properties in a short, easily-administered tool. Nevertheless, the low specificity implies that there are substantial false-positives that can be identified at a second step clinical assessment of geriatric problems. For example, in a previous trial of a two-step ED intervention for older adults in the ED, using a standardized nursing assessment, 61.2% of those scoring ISAR 2 + were found to have one or more new or uncompensated geriatric problems [10]. Centers wishing to reduce the number of patients testing positive (and the false-positives) can increase the cut-point to 3 + . This higher cut-point reduces the sensitivity but maintains a high negative predictive value.

The search for better screening tools has been pursued, with results varying by patient population and tools compared. A meta-analysis of screening tools to predict functional decline in hospitalized older adults found 3 tools, including ISAR, worthy of further investigation [28]. In a subsequent head-to-head comparison, ISAR was found to perform similarly to or better than the other two tools and was judged to be the easiest to use in clinical practice [29]. Three other studies to our knowledge have compared ISAR directly to other screening tools in the same patient population when administered after admission to hospital [25, 30, 31]. Different outcomes were examined (functional decline, readmission). In general, all tools including ISAR had modest predictive performance. [Note that the 4-item so-called ISAR-HP was not derived from the ISAR and is based on different items [32]].

Limitations

Several study limitations should be highlighted. First, the study was a secondary analysis, limited by the data collected in the main study. Thus, we were not able to compare the performance of the ISAR-R to that of the original ISAR. Also, we were not able to follow patients discharged to LTC or alternative institutions (n = 87). Table 1 show that this group differs from the main sample in several respects (longer hospital stay, informant was more often the caregiver, more self-reported memory problems). A second limitation is that the study sample is not representative of all patients discharged home from the same study units during the same period: patients aged 85 + , those with a previous hospital admission, and those with longer hospital stays are over-represented (Table 1). Third, readmissions were defined from administrative data at the same hospital as the index admission. However, only a small minority of patients reported admissions to different hospitals; stratification for this variable did not significantly affect the AUC (Table 3).

Conclusions

This study provides preliminary evidence that the ISAR-R performs at least as well as the original ISAR in prediction the prediction of readmission among older patients following admission to hospital. This will be reassuring for the many institutions which have adopted the ISAR-R as a step 1 of a 2-step screening program, although further research is needed. It may be a valuable clinical tool to help stratify patients into lower and higher-risk groups to guide interventions that aim to reduce readmission. The advantages of the ISAR-R over the original ISAR are: 1) the more intuitive phrasing and scoring of the questions and 2) reduction of the number of patients that screen positive through revision of the polypharmacy threshold (as in the original QI study) [15]. Further research is recommended in three areas: first, the tool’s performance in predicting different outcomes (e.g., functional decline, mortality, and nursing home admission); second, optimal wording of ISAR questions for different populations; and third, how best to implement screening in different clinical contexts.