Introduction

Identification of frailty among elderly has relevance for clinical decision-making and contributes to personalized care. Several frailty indices (FI) have been derived from aging databases. In previous research, a standard procedure for creating an FI was developed showing that frailty variables can be selected at random; the specific nature of the variables is not important, as long as they cover a range of systems [1, 2]. Recently, a short version of a questionnaire based on the ‘The Older Persons and Informal Caregivers Survey Minimum Data Set’ (TOPICS-MDS) [3, 4], the TOPICS-Short Form (TOPICS-SF) was developed and validated as a patient-reported outcome measure (PROM) in clinical settings [5]. We established if and how an FI based on this short form can be calculated. While medical health is covered in the PROM itself through an item on subjective health, information on the presence of chronic diseases is not. An accompanying case mix form is used to collect disease information from the medical record. The feasibility of the FI would increase if the score could be established from the self-reported information alone. The aim of this study was to assess the validity and reliability of this SF-FI with and without disease information.

Methods

This was a clinical, prospective study including geriatric in- and outpatients of 65 years and older at a university hospital in the Netherlands. We recruited a consecutive sample of patients and obtained the recommended sample size for this type of studies [6]. Exclusion criteria were: (1) not being able to answer on questionnaires due to severe cognitive impairments or a decreased level of consciousness, (2) requiring end-of-life care with a life expectancy of less than 2 weeks.

Complete TOPICS-MDS data were collected in an interview for the self-reported information, and by abstracting the present diseases and other case mix data from electronic medical records by a researcher. We then calculated an FI based on the 17 self-reported TOPICS-SF items alone (SF-FI). As a reference standard, we used the TOPICS FI based on the complete TOPICS-MDS (MDS-FI), which includes disease information on 18 chronic conditions and 7 extra items on aspects of well-being and health, amounting to a total of 42 items [4]. Data on relevant clinical and demographic variables (age, sex, and living situation) were collected, because they are previously found to be predictors of FI scores [4, 7]. The multi-morbidity was additionally operationalized as a Charlson co-morbidity index as established from the electronic health record by a researcher. The Clinical Frailty Scale (CFS) was registered by the geriatrician during comprehensive geriatric assessment on a 9-point scale (higher is more frail) [8].

Descriptive statistics were performed. To test criterion validity, a Pearson correlation analysis between the SF-FI and the MDS-FI was performed. To test the construct validity, SF-FI scores were examined according to predefined hypotheses (Table 1) for its correlation with patient characteristics such as age, sex, living situation and multimorbidity, using Spearman correlation analyses and independent sample t tests. We considered the SF-FI as having acceptable criterion and construct validity respectively, when the Pearson’s r for the correlation between the SF-FI and the MDS-FI was greater than 0.7 and at least 75% of our results were in correspondence with our hypotheses. This is in accordance with the COSMIN quality criteria for measurement properties [6, 9, 10]. To investigate the reliability of the SF-FI, the agreement between SF-FI scores and MDS-FI scores was visualized with a Bland Altman analysis. All statistical analyses were carried out using SPSS Statistics 25.

Table 1 Hypotheses to assess construct validity of the TOPICS-SF-FI
Full size table

Results

A total of 95 elderly patients were included in the study. The mean (± standard deviation) age was 81 (± 7) years, a slight majority (55%) were women. A strong correlation was found between SF-FI and MDS-FI scores (Pearson’s r = 0.92). Data supported four out of six hypotheses for the construct validity, which means that two third of our results complied with the a priori defined hypotheses. Investigating the reliability, the mean SF-FI score was remarkably higher than the mean MDS-FI score [0.44 (0.21), respectively 0.32 (0.14)]. The mean difference between the SF-FI and the MDS-FI was 0.12 (95% confidence interval 0.10 to 0.14). A Bland–Altman analysis showed that the interval of the levels of agreement between the SF-FI and the MDS-FI was − 0.08 to 0.32 (Fig. 1). Regression analysis showed that the difference between the two frailty indices increased significantly with increasing mean of the MDS-FI and SF-FI (R2 of 0.61, B = 0.47, p < 0.001).

Fig. 1
figure 1

Bland–Altman plot: levels of agreement between SF-FI and MDS-FI. Lines with blue shading: mean difference with 95% confidence interval. Lines with red shading: lower limit of agreement with 95% confidence interval. Lines with green shading: upper limit of agreement with 95% confidence interval

Full size image

Discussion

We showed that the construct validity between the SF-FI without morbidities and the MDS-FI is strong, but the reliability of the SF-FI was not satisfactory. Especially at higher frailty levels, the SF-FI scores overrated the frailty compared to the MDS-FI. This suggests that the excluded morbidity items make a considerable contribution to establish a reliable FI score and cannot be excluded. A possible explanation for the higher SF-FI scores lies in the nature of the excluded variables. The SF-FI contains mainly self-reported variables, whereas the MDS-FI also contains more objective disease information. Of the self-reported SF-items, 10 are (I)ADLs, and different (I)ADLs are more likely to co-occur than different morbidities. Additionally, it is likely that an FI based on a limited number of items (17 items) is more sensitive to a misbalance between the type of items included. While PROMs need to be short to be feasible and the ability to measure frailty based on self-reported PROM data continues to make sense, it is important to recognize that disease information (the casemix form accompanying the TOPICS-SF) cannot be skipped when calculating an FI. Further curtailment of a frailty index is not recommendable.

Conclusions

This study indicates that the reliability of the TOPICS-SF-FI is not sufficient when excluding information on morbidities. When measuring an FI it is important to carefully choose deficits as some items may be more correlated [(I)ADLs] than others (morbidities). This may over- or underestimate the true frailty, while conceptually, the type of deficits accumulated may not matter.