Background

Assisted living (AL) is typically defined as a setting that provides health and personal services within a secure residential environment for older adults not requiring the continuous monitoring and more intensive professional care of a long-term care facility [1, 2]. It is a growing and in many ways attractive housing option for older persons often described as frail. AL residents are prone to a number of adverse outcomes in the short term. Over a year approximately a sixth will die with a similar proportion moving to a higher level of care [3, 4]. Acute care hospitalizations also occur frequently among this group of older adults [4, 5].

There is consensus that the core feature of frailty is a heightened vulnerability to stressors (for a given chronological age and sex), which leads to an increased risk for multiple adverse health-related outcomes [6]. As a group, AL residents will have relatively higher levels of vulnerability than similarly aged community-dwelling individuals, but within the AL population there will be varying degrees of frailty. Gradients of frailty are found in long-term care populations with higher levels associated with an increased risk of mortality, cognitive decline, and new onset disability [7]. Identifying AL residents at a higher risk for adverse health outcomes would offer opportunities to both maintain remaining independence and enhance quality of life if this could be linked to effective interventions.

Few studies have directly compared the competing approaches to identifying frailty in more vulnerable older populations. Two frequently used measures are the frailty index, which is based on a count of accumulated deficits divided by the number of potential deficits [8], and the Cardiovascular Health Study (CHS) frailty criteria. With the CHS approach, the determination of frailty requires the presence of three or more characteristics felt to capture what has been termed the phenotype of frailty [9].

Data arising from the implementation of the interRAI family of assessment instruments have increasingly been used for aging-related research, including recent work on frailty [10–13]. For example, Armstrong and colleagues developed a 50-item frailty index based on items in the home care instrument [10]. Embedded within each interRAI instrument are various scales. One of them, the Changes in Health, End-stage disease and Signs and Symptoms (CHESS) scale, is a measure of health stability designed to identify individuals at high risk for clinically significant decline [11]. In nursing home populations, higher CHESS scores are predictive of mortality and hospitalization [11, 12]. Some have suggested that the CHESS scale is a frailty measure [11, 13].

The primary objective of this study was to compare the ability of two versions of a frailty index derived from interRAI assessment items and the CHS frailty criteria to accurately predict the occurrence of three outcomes relevant to an AL population (mortality, hospitalization, and transfer to a long-term care facility) over a year. This builds on work previously done by our group in trying to operationalize the assessment of frailty among older residents of AL facilities [14]. A secondary objective was to compare these frailty measures to the CHESS scale in their ability to predict these three outcomes.

Methods

Study design

This was a sub-study of the Alberta Continuing Care Epidemiological Studies (ACCES), which investigated the health status of residents and quality of care issues within AL and nursing home facilities across the Province of Alberta, Canada. The AL cohort of ACCES consisted of residents aged 65 years and older residing in designated (i.e., publicly-funded) assisted living and supportive housing facilities (DAL) in two urban and three rural health regions within Alberta. The details of this study have been reported elsewhere [3, 14]. A total of 1,089 AL participants were enrolled and assessed. This sub-study includes 1,066 residents who provided consent for linkage with Provincial administrative data and had a known outcome status during the one-year follow-up.

Ethics approval was obtained from the University of Calgary Conjoint Health Research Ethics Board, the University of Alberta Health Research Ethics Board and the University of Lethbridge Human Subject Research Committee. Administrative approvals from the health regions and/or facilities were also obtained. Informed consent was received from all participants or their legally appointed proxy decision-maker.

Trained research nurses administered the Resident Assessment Instrument for Assisted Living (interRAI-AL) and additional performance-based frailty measures at baseline (2006–2007) and at 1-year (when possible). The interRAI-AL tool is a comprehensive, standardized assessment of the sociodemographic characteristics, physical and cognitive status, health conditions, behavioural problems, and use of medications and services of residents [15].

Frailty measures and CHESS score

Three frailty measures were examined. Their specific composition is described in Appendix A. The first was a frailty index based on the report of Armstrong et al. [10] that we designated as the Armstrong Index (see Appendix A). We employed a modified version with 43 of the original 50 items, as not all items were available with the interRAI-AL tool. An alternative frailty index consisting of 83-items derived from the interRAI-AL was created using the procedure described by Searle et al. [16], which we designated as the Full Frailty Index (see Appendix A). At least 30–40 items, fulfilling recommended criteria, are suggested for inclusion in a frailty index. In general, the more items included, the more precise the estimates of frailty obtained [16]. Our intent with the Full Frailty Index was to make maximal use of the interRAI-AL tool to develop a precise, practical index that was also substantially different from the Armstrong Index (i.e., 44 of the 83 items were not contained in the Armstrong Index). Both indices are calculated as the proportion of accumulated to potential deficits present in a given individual. For most items included in both indices, there was less than 5 percent missing data (with ≥ 80% of items having no missing values) [17]. For three variables (bathing, walking, self-reported health), there were higher percentages with missing values (6.9%, 11.6%, 11.6%, respectively) because either the activity did not occur during the three-day assessment time frame (bathing, walking) or the resident could/would not respond (self-reported health). These variables were retained in the respective indices (bathing and self-reported health in both indices; walking in the Full Frailty Index as well) as each individual variable makes a small contribution to the indices and prior work indicated that doing this would not significantly affect our results [16]. Any variable where there was a missing value was removed from both the numerator and denominator of the ratio used to calculate the frailty index for an individual participant (i.e., if one of our research subjects had one variable with a missing value, when calculating their Full Frailty Index the denominator was 82). In accord with other studies [16, 18], a resident with a score of less than 0.2 was considered “robust” or “not frail” for both indices while scores between 0.2-0.3 (inclusive) indicated a pre-frail state and a score greater than 0.3 would lead to the resident being categorized as frail.

The determination of frailty using the CHS approach is based on the presence of five criteria: slow gait, muscle weakness, low physical activity, unintentional weight loss, and exhaustion. If none are present, the person is categorized as not frail [9]. The presence of three or more denotes the presence of frailty while one or two signifies pre-frail status. Two of these criteria are performance-based. Please see Appendix A. Nearly 40% of AL participants could not complete the CHS frailty assessment as originally intended. By using responses to observed items from the interRAI-AL assessment instrument we were able to reduce the proportion with missing data to 15% [14]. For all analyses using CHS criteria our sample size was reduced to 927.

The derivation of the CHESS scale followed standard interRAI procedures using items from the interRAI-AL (see Appendix B) [11]. Two symptoms (i.e., dehydration, decline in food/fluid intake) of the six included in the original CHESS scale were unavailable on the interRAI-AL form and not used in calculating the symptom component of the total score. Based on the distribution of the scores in our study population and our desire to keep the number of categories comparable to the frailty measures, we collapsed the typical five point scale and categorized the scores as follows: 0 indicating a low risk for serious decline, 1 as intermediate, and 2 or more denoting a high risk.

Outcome measures and baseline characteristics

Primary outcomes were mortality, one or more hospitalizations and transfer to a long-term care facility over the 12 months following the subject’s baseline assessment. For mortality and long-term care admission, information on event occurrence was determined from concurrent reviews of facility discharge records and family interviews (at the time of death or discharge) as well as data collected at the time of resident and family follow-up assessments. For hospitalization, residents’ data were linked with the Alberta Inpatient Discharge Abstract Database for 2002–03 to 2008–09. This administrative database captures essentially 100% of all hospital admissions in the Province of Alberta.

A baseline co-morbidity score was calculated based on the Charlson co-morbidity index [19] and a validated coding algorithm [20] using relevant diagnostic codes (any occurrence during 3 years prior to baseline) from the Alberta Inpatient Discharge Abstract database. Other baseline characteristics examined included an additional co-morbidity measure (based on the sum of recorded diagnoses on the interRAI-AL) and three validated scales derived from items on the tool: the Cognitive Performance Scale (CPS) [21], Depression Rating Scale (DRS) [22], and Activities of Daily Living (ADL) Self-Performance Hierarchy Scale [23].

Analysis

A weighted kappa statistic (with 95% confidence interval) was used to measure the agreement between the Armstrong Index, the Full Frailty Index, the CHS criteria, and the CHESS scale, using the 3-level risk categorization [24, 25]. Generalized linear models with a binomial distribution and log link were used to estimate risk ratios in analyses. Multivariable models adjusting for age (in years), sex, and co-morbidity were examined to assess the prognostic significance of each approach to defining frailty and the CHESS scale. The risks of one-year mortality, hospitalization (one or more), and transfer to long-term care for those categorized as frail or pre-frail by the Armstrong Index, the Full Frailty Index, and CHS criteria (compared with non-frail residents) were assessed in multivariable models. For the CHESS scale, high and intermediate risk levels were compared to the low risk group. The models considered potential confounding by age, sex and co-morbidity (using the Charlson co-morbidity index score). Predictive accuracy was assessed by comparing the area under the receiver operating characteristic curve (AUC) of a baseline model with only age and sex with that of models with age/sex/co-morbidity and age/sex/co-morbidity/frailty (or CHESS), respectively. To facilitate model comparisons, 95% confidence intervals for differences in AUC estimates (with associated p-values) were calculated [26]. Combined outcomes, death/hospitalization and death/move to long-term care, were also explored to investigate the possibility of competing risks. As there were no appreciable differences in our final conclusions, the results were not included.

We previously examined the effects of clustering (i.e., violation of the independence assumption due to nesting by AL-institution) on the outcomes and determined that the level of clustering was minimal [14]. We have therefore presented the results of our main analysis without adjustment for clustering to permit simpler statistical approaches without the loss of information. SAS Version 9.2 (SAS Institute, Inc., Cary, NC) was used for analyses.

Results

The enrolled cohort was predominantly female (818/1066 or 76.7%) with an average age of 84.9 (standard deviation 7.3) years (see Table 1). Multiple morbidities were common. Most subjects (n = 632, 59.3%) had at least mild cognitive impairment, nearly a fifth (n = 203, 19%) had significant depressive symptoms, and over a third (n = 426, 40%) had limited or greater impairment in the performance of activities of daily living.

Table 1 Baseline characteristics for ACCES – AL Cohort
Full size table

On the Armstrong Index, 9.3% (n = 99) were categorized as not frail, 38.5% (n = 410) pre-frail, and 52.3% (n = 557) as frail. The Full Frailty Index categorized 32.3% as not frail (n = 344), 39% pre-frail (n = 416), and 28.7% (n = 306) as frail. Using CHS criteria, 3.5% (n = 32) were categorized as not frail, 48.7% (n = 451) pre-frail and 47.9% (n = 444) as frail. On the CHESS scale, nearly half (46.5%, n = 496) were identified as being at a low risk for a serious decline, with 29.3% (n = 312) and 24.2% (n = 258) categorized as intermediate and high risk respectively.

A weighted kappa statistic showed the highest level of agreement between the Armstrong Index and the Full Frailty Index (kappa = 0.41, 95% CI 0.38-0.45) with just over half (51.7%) having identical categorizations as non-frail, pre-frail and frail on both criteria (see Table 2). The poorest agreement was between the CHS frailty measure and the CHESS scale (kappa = 0.11, 95% CI 0.08-0.14) with only 30.4% assigned to equivalent categories (non-frail/low risk, pre-frail/intermediate risk, and frail/high risk).

Table 2 Comparison and assessment of agreement among the frailty measures, based on 3-level risk categorization
Full size table

Over a year 15.9% (n = 170) of residents died, 39.8% (n = 424) were hospitalized at least once, and 19.1% (n = 204) moved to long-term care (see Table 3). Frail subjects defined by the three approaches as well as those at a high risk for decline on the CHESS scale showed a statistically significant increase in their risk of dying compared to those categorized as either not frail or at low risk for decline. The risk ratios in models adjusted for age, sex, and co-morbidity ranged from 1.74 (95% CI 1.07-2.81) with CHS frailty criteria to 2.35 (95% CI 1.56-3.54) for the Full Frailty Index. Only the Full Frailty Index was associated with a statistically significant increase in the risk of death for pre-frail subjects in our adjusted models (RR 2.00, 95% CI 1.33-3.00).

Table 3 One-year outcomes* associated with frailty measures, Risk Ratios (95% Confidence Intervals)
Full size table

Frail and pre-frail individuals as defined by the Armstrong Index had no statistically significant increase in their risk for hospitalization (please see Table 3). Those meeting CHS criteria for frailty were at greater risk for hospitalization (risk ratio in adjusted model was 1.45, 95% CI 1.15-1.83), but there was no statistically significant increase in the risk ratio for the pre-frail group. Risk ratios for hospitalization were significantly increased for both pre-frail and frail subjects on the Full Frailty Index, but no gradient was observed and confidence intervals for the risk ratios for pre-frail (1.37, 95% CI 1.13-1.66) and frail (1.28, 95% CI 1.04-1.57) subjects overlapped. A similar increased risk for hospitalization was observed for those categorized as intermediate or high risk for decline on the CHESS scale in adjusted analyses.

Residents categorized as frail on all approaches considered and the high risk group for decline on the CHESS scale showed a statistically significant greater risk of transfer to long-term care (please see Table 3). The adjusted risk ratio was strongest for the Armstrong Frailty Index (4.14, 95% CI 1.87-9.14) followed by the Full Frailty Index (3.30, 95% CI 2.29-4.76). Only the Full Frailty Index showed a statistically significant increased risk of long term care admission for pre-frail individuals (1.87, 95% CI 1.27-2.75) in adjusted analyses.

To compare the abilities of the three frailty approaches and the CHESS scale to predict our outcomes of interest, we examined areas under the ROC curve (AUC). As potential confounders we considered sex, age, and co-morbidity index scores (please see Table 4). The AUCs obtained for the frailty measures and the CHESS scale ranged from 0.683-0.701 for mortality, 0.609-0.629 for hospitalization, and 0.602-0.667 for admission to long-term care within a year. For death, the addition of frailty (however operationalized) or CHESS-defined risk for decline significantly improved on the AUC obtained with sex, age, and co-morbidity (p-values less than 0.03 for pair-wise comparisons), though the magnitude of improvement seen was relatively small (0.031-0.049). None of the AUCs with a frailty measure or the CHESS scale differed significantly from each other. As for hospitalization, only the CHS frailty criteria significantly improved on the model based on age, sex, and co-morbidity (AUC 0.629, 95% CI 0.592-0.665 versus AUC 0.598, 95% CI 0.561-0.635, p = 0.003 for the comparison). The magnitude of improvement was relatively small (0.031), and none of the AUCs with the addition of frailty or the CHESS scale differed significantly from each other. For transfer to long-term care, the addition of any of the frailty measures or the CHESS-defined risk of decline improved on the model with sex, age and co-morbidity (p-values less than 0.03 for pair-wise comparisons). The magnitude of improvement seen ranged from 0.048 to 0.113. The AUC for the Full Frailty Index (0.667) differed significantly from the AUCs for the CHESS scale (0.602, p = 0.016) and CHS frailty criteria (0.610, p = 0.003). The difference between the AUCs of the Full Frailty Index and the Armstrong Index (0.638) was of borderline significance (p = 0.087).

Table 4 Area under the ROC curve (95% Confidence Intervals) for models of one-year outcomes, comparing frailty measures
Full size table

As a sensitivity analysis, we looked also at two composite outcomes (hospitalizations + deaths and long-term care transfers + deaths) to see if censoring affected our results. As they were essentially unchanged, we have not reported the specific estimates for these composite outcomes.

Discussion

In this population of AL residents, both the presence of frailty defined by any of the three approaches examined and a higher CHESS score were associated with an increased risk of dying or being transferred to a long-term care facility. The magnitude of the ability of these measures to correctly classify subjects (relative to models including only age, sex and co-morbidity) was modest for mortality but better for long-term care placement. The AUCs obtained were in the 0.6 to 0.7 range. This generally indicates low accuracy in correctly differentiating risk [27], but our objective was not to develop a comprehensive prediction model for these outcomes in our AL population. The current study comparing these three frailty measures and the CHESS score builds directly on our prior work examining frailty in the AL setting [14].

The different approaches to detecting frailty were more similar than dissimilar with regard to predictive accuracy with a few exceptions. The Full Frailty Index performed significantly better in predicting a move to long-term care and was the only approach that showed higher mortality, hospitalization, and institutionalization risks among those categorized as pre-frail. Only the addition of CHS frailty criteria significantly improved on age, sex, and co-morbidity in predicting hospitalization. While our study did not directly address practicality, acceptability to practitioners, or clinical utility, a number of observations can be made. Both the Armstrong Index and CHS frailty criteria categorized relatively few AL residents as not frail and approximately half as frail. In comparison to the Full Frailty Index and the CHESS scale, they were less successful in creating a more circumscribed group of AL residents for potential interventions. This would be an important consideration when clinical resources are limited and have to be targeted. The Armstrong and Full Frailty Indices (and the CHESS scale) were constructed using data available on the interRAI-AL instrument, one of a suite of similar instruments being implemented across continuing care settings in Canada. While many items were included in the indices, their calculation could be automated. This could make the use of multi-item indices practical from a clinical standpoint. The CHS frailty criteria did not perform significantly better than the frailty indices (other than possibly in predicting hospitalization) and required the use of performance measures, which can be difficult to obtain on AL residents [14, 28]. The latter issue raises major feasibility concerns with this approach in an AL population. The ability to derive the CHESS scale and two frailty indices from previously collected data highlights the practical utility of these measures while the inclusion of a wider set of potentially relevant domains in operationalizing frailty represents a conceptual strength of the two frailty indices [6].

As shown in Table 2, the best agreement among the measures was between the Full Frailty and Armstrong indices. Although moderate agreement between these two indices was observed [29], it is surprising that this was not greater given the degree of commonality in items (see Appendix A) and their similar operational approaches. Interestingly, Rockwood et al. found that random combinations of items making up frailty indices led to little overlap across quartiles [30]. The lower than expected level of agreement between these two indices may also reflect the inclusion of several Instrumental ADLs (e.g., meal preparation, housework, managing medications) in the Armstrong (but not the Full Frailty) index. As these areas are generally managed or provided for residents by the AL facility (resulting in most residents being assessed as impaired in these areas) they would be considered poor criteria for a frailty measure [16]. This may also explain the low proportion of residents categorized as not frail by the Armstrong index. Fair agreement was observed between the Full Frailty Index and the CHESS and between the Armstrong Frailty Index and the CHS. The remaining kappa statistics indicated only slight agreement between the measures. Armstrong et al. had previously reported a low correlation between the CHESS and their frailty index (r = 0.35) [10].

Whether the CHESS scale is a frailty measure cannot be answered by our study. Compared with the other measures examined, the CHESS scale generally performed similarly in predicting our outcomes of interest, although associations were weaker for long term care placement. This was most evident for comparisons between CHESS and the two frailty indices. In settings where interRAI instruments are widely implemented, the CHESS scale offers the advantage of being relatively simple and easy to assess across care settings.

The reasons underlying the relatively weaker associations between frailty and the outcome of hospitalization are not entirely clear. When added to the model with age and sex, the Charlson co-morbidity index score improved the AUC for hospitalization (see Table 4) and has been previously shown to predict hospitalization (and mortality) in older institutionalized residents [31]. The further addition of frailty and/or health instability may offer relatively less predictive gain for this outcome after the addition of a co-morbidity index. Another possibility is competing risk though we found no evidence of this in the analyses performed for this study. In earlier work we failed to observe a statistically significant increase in the risk of hospitalization among frail men that we felt was due to their high mortality rate during follow-up [14]. Other considerations would include the inherent difficulty of predicting hospitalizations especially for catastrophic (e.g., fall with a hip fracture) changes in health, the modifying effects of factors such as advance planning and the resources available within AL facilities [32], and the obscuring influence of variability in local hospitalization rates [33].

There are some limitations in our study that would raise concerns about the generalizability of our findings to the entire AL population. First, over four hundred eligible residents did not enroll in the study [14]. While the age and sex distribution of those not enrolled was comparable to our enrolled cohort, we do not have other information on them. Second, we restricted eligibility to residents of publicly-subsidized AL facilities in Alberta. Our findings may not apply to residents in private AL or AL-type facilities in other jurisdictions that may have different admission criteria, staffing, and institutional policies. Hospitalizations were determined using provincial data, and we may have missed the rare event that occurred outside Alberta. Further research examining these and other relevant outcomes (e.g., functional decline) over longer periods of time is also warranted. Nearly 40% of AL participants could not complete the CHS frailty assessment as originally intended. By using responses to observed items from the interRAI-AL assessment instrument we were able to reduce the proportion with missing data to 15%. As noted previously this raises questions about the feasibility of this particular approach in an AL population.

Conclusions

Using a number of approaches, varying degrees of frailty can be detected within AL populations. Residents with more severe levels of frailty are more likely to die, be hospitalized, and require admission to long-term care. Whether the detection of frailty will turn out to be clinically useful will require further work, in particular intervention studies.

Appendix A

The following items were included in the cumulative deficit index meant to replicate that created by Armstrong et al. [10], (Table 5). Three items which were not exactly replicable using the interRAI-AL were, ”severe malnutrition”, “problem chewing” and “head trauma”. Also “Alzheimer” and “Dementia other than Alzheimer’s” were combined into one category. “Hip fracture”, “other fractures”, and “osteoporosis” were combined into one category. Also, “Feeling of sadness” and “Sad, pained worried facial expressions” were combined into one category. This reduced the index from 50 items to 43 items.

The presence of each condition added “1” to the person’s Index score (unless otherwise indicated).

The following index was created using the criteria based on Searle et al. [16] using information from the interRAI-AL, (Table 6). The presence of each condition added “1” to the person’s index score (unless otherwise indicated).

The following details the frailty criteria from the Cardiovascular Health Study (CHS) [9], (Table 7).

Table 5 Armstrong frailty index (43 items)
Full size table
Table 6 Full frailty index (83 items)
Full size table
Table 7 Cardiovascular Health Study Frailty Criteria1
Full size table

Appendix B

CHESS Scale

CHESS stands for Changes in Health, End-stage disease and Symptoms and Signs of medical problems [11]. It provides a measure of instability in health (which may be a consequence of frailty) and is believed to be a marker of imminent decline in health. The score is based on the following:

Symptoms: A summary count is first derived from the following symptoms (coded as 0 = no symptoms present; 1 = 1 symptom present; 2 = 2+ symptoms present).

  • vomiting

  • dehydration (insufficient fluid)*

  • decline in food/fluid intake*

  • weight loss

  • shortness of breath

  • edema

To this summary count variable, 1 point is added for “worsening of decision making over previous 90 days”, 1 point for “decline in activities of daily living over previous 90 days”, and 1 point for “end-stage disease”.

The range of values for the CHESS is 0 to 5, where 0 represents stability, and 5 represents highly unstable health.

*Note that two items from the original CHESS, were unavailable on the interRAI-AL form, and were not included in the calculation of symptoms:

  • dehydration

  • decline in food/fluid intake