Background

To prevent functional decline, frail older persons probably benefit from integrated primary care services, including proactive and patient-centred care [1–4].

Still, efficient and accurate tools to map populations of older persons for frailty are not available in primary care. Performing a full CGA in every older person is considered to be the best way to identify frailty and to directly manage the risks for negative health outcomes [5, 6]. However, this would be very resource demanding and, therefore, not feasible in primary care [7–10]. An alternative might be to preselect frail older persons using simple triage tools [11]. However, implementation of these tools in daily practice has shown to be difficult. In addition to the moderate to fair predictive ability for most methods, there is the issue of low acceptability by the professional community [12].

Against this background, we developed the EASY-Care Two step Older persons Screening (EASY-Care TOS) procedure (Fig. 1) [13]. The first step is a simple triage tool with which the general practitioner (GP) makes a pre-selection of older people based on his prior knowledge and information he has already available of the patient. Further information is only gathered when the GP considers this necessary. To complete this first step, the GP reviews the patient record and answers 14 questions about the functioning of the patient in somatic, psychological, and social domains. The 14 questions are meant to trigger the GP into considering all relevant aspects. The GPs decide whether: (1) the patient is not frail; (2) the patient is frail; or (3) the available information is insufficient for a decision (unclear). Only persons in whom the GP judges the available information to be insufficient for a decision receive a structured assessment using the EASY-Care assessment instrument by a primary care nurse (step 2). This EASY-Care assessment takes place in the patient’s own home. Subsequently, the GP and primary care nurse make a frailty decision. The content of the subsequent steps of the EASY-Care TOS procedure can be found in Additional file 1: Appendix 1.

Fig. 1
figure 1

Schematic overview of the EASY-Care TOS. The first step, performed by the GP, is a professional appraisal based on prior knowledge about functioning, wellbeing, and the care context of the patient. The second step based on EASY-Care assessment by a primary care nurse is performed in the group that is initially ‘unclear’. EASY-Care TOS EASY-Care Two step Older people Screening, GP general practitioner

Full size image
Fig. 2
figure 2

Flowchart of the recruitment, inclusion and drop-out at follow-up of participants of the TOS-study. *GPs were reimbursed to perform the required assessments and could hire additional workforce, but depending on local situations were sometimes limited in the amount of workforce (themselves to perform step 1, and nurses to perform step 2) they could free up from other tasks to perform the assessments

Full size image

To facilitate the procedure, GPs and nurses received training on frailty and how to apply the EASY-Care TOS procedure. The GPs and nurses were trained to apply the following definition of a frail person: has decreased reserve capacity because of multiple health, mental, or social problems; this makes the person vulnerable to changes in the biopsychosocial context, especially when compensating factors are lacking [13].

A numerical cut off score for frailty (for example, a predefined number of positive responses out of the list of 14 items) is not part of the EASY-Care TOS procedure (neither after step 1 nor after step 2). Rather, the GPs use their clinical judgement in mapping a person’s frailty. This is based on the principle that professionals have the ability to examine and evaluate complex information from multiple sources: explicit information as well as intuition, competing observations, and missing information [13].

In addition to obtaining a judgement on a person’s frailty status, GPs also actively review their information on a person’s complete health and psychosocial status – including where information is missing – with the EASY-Care TOS procedure. Therefore, EASY-Care TOS offers an excellent starting point for the GP-nurse team for individualised care planning and treatment, especially because it incorporates the official EASY-Care instrument which is primarily intended for that aim [14], although the primary aim of the EASY-Care TOS procedure is to support GPs when mapping the older population they care for.

Compared to the application of other frailty measures in primary care, this procedure may be more efficient in its claim for resources and number of older persons who need to be assessed, as it makes optimal use of readily available information. Other triage tools often depend more heavily on new data collection [15–17], although approaches have been developed which derived a frailty index from routine data available in the GP electronic medical records [18]. In our previous work, we found that the EASY-Care TOS procedure fits in everyday clinical practice of primary care professionals. Construct validity and reliability as well as feasibility and acceptability of EASY-Care TOS by primary care professionals and patients have been demonstrated previously [19, 20]. GPs needed 3 to 15 min to complete step 1 and nurses needed 45 to 90 min to complete step 2 [20]. A head-to-head comparison against a consultant geriatrician led CGA was reported as well [21].

In this external validation, we studied the predictive validity of the EASY-Care TOS procedure for several measures of negative health outcome over one year follow up in a prospectively sampled, unselected cohort of GP registered persons 70-years-old and older. We compared the predictive accuracy of EASY-Care TOS with a more parsimonious alternative (using only baseline age and sex, number of morbidities, and number of medications status) and with a frailty judgement in all participants in hospital setting at baseline, after a geriatrician had performed a CGA independently from the GP.

Methods

Study population and data collection

In this cohort study, 12 GPs from four GP practices in and around Nijmegen (The Netherlands) assessed their patients of 70 years and older with the EASY-Care TOS procedure between February 2010 and August 2011. Their practices were situated in urban (n = 2), suburban (n = 1) and countryside (n = 1) areas. Of the 1,490 patients of 70 years and older registered in these practices at the end of 2009, the GPs randomly asked a total of 1,159 older patients to participate in the validation study (Fig. 2). The GPs excluded patients for participation in the validation study on the basis of the following exclusion criteria: 1) too ill or too weak to be assessed; and 2) under treatment of a geriatrician or underwent a comprehensive geriatric assessment in the past three months (as this information might influence the frailty judgement of the GP). A total of 587 older patients (51 % of the 1,159 approached) gave informed consent and were included in the study. The local ethics committee CMO Regio Arnhem-Nijmegen (http://www.cmoregio-a-n.nl/) approved the study (number of approval is 2009/223).

At baseline all participants were assessed with the complete EASY-Care TOS procedure, i.e. all subsequent assessment steps were finished irrespective of the outcome. This was done in spite of the logic of the procedure outlined in the background section, because this enabled us to study if the frailty evaluations of the GPs changed across the subsequent steps of the procedure. After the EASY-Care TOS procedure was completed, the participants also underwent a CGA at the geriatric outpatient clinic of the Radboud University Medical Centre in The Netherlands. After 12 months the participants received again a complete EASY-Care TOS procedure completing all subsequent steps once more to assess the prospective outcomes. The steps were performed by the GPs and nurses who also performed them at baseline, after GP practice employees had ascertained follow up status (death, institutionalisation, too ill to be further assessed) and scheduled a visit.

Comprehensive geriatric assessment

Within less than a month after completing the EASY-Care TOS procedure, the participants underwent a geriatric assessment at the geriatric outpatient clinic. This assessment consisted of an interview and complete medical examination by a geriatrician and an interview with the geriatric nurse and additional tests for cognition, mental wellbeing, physical functioning, functioning in (instrumental) activities of daily living [(I)ADL], and mobility. After this assessment the geriatrician and geriatric nurse judged the frailty status of the patient in the same way the GPs did with the EASY-Care TOS procedure after step 2. All participants in the hospital geriatric assessment were blinded for the results of EASY-Care TOS. No clinical intervention was delivered after the CGA.

EASY-Care TOS procedure frailty judgement

As explained in the background section, in this stepped procedure only persons for whom the GP cannot decide on the basis of the available information (step 1) whether the person is frail or not are eligible for the step 2 assessment to collect additional information. Therefore, in this validation study, we evaluated the validity of the frailty judgement based on the step 1 judgement for persons “frail” or “not frail” and based on the step 2 judgement for persons who could not be judged on the basis of step 1 alone.

Primary and secondary outcome measures

We were primarily interested in the discriminative ability of EASY-Care TOS for negative health outcome. This outcome was operationalised in our primary outcome measure “composite of negative health outcomes” as a composite of: 1) death; 2) institutionalisation to a home for the aged or a nursing home; 3) too ill to be assessed with EASY-Care TOS at follow up; or 4) an increase of at least 1 point on the Katz 6 ADL scale as derived from the EASY-Care TOS step 2 assessment [22], which means one extra ADL disability during 12 months of follow up. Death, institutionalisation and increase in ADL disabilities operationalised as described were evaluated separately as secondary outcome measures.

Statistics

Baseline participant characteristics were summarised using descriptive statistics. To evaluate the performance and added value of the EASY-Care TOS judgement for the outcomes of interest (composite of negative health outcomes, and mortality, ADL decline and institutionalisation separately), we used logistic regression to calculate diagnostic odds ratios with 95 % confidence intervals with and without adjusting for age, sex, GP practice and readily available information on the number of diseases [with categories: 0–1, 2, >2 diseases] and number of medications [with categories: <4, ≥4 medications]) [23].

Further, we established the relative increase (or decrease) in the Area Under the Receiver Operating Curve (AUC) for the EASY-Care TOS judgement in addition to information on age, sex, GP practice, and readily available information on the number of diseases and number of medications and compared the discriminatory performance of the EASY-Care TOS judgement with that of the frailty judgement by the geriatrician (CGA). For these calculations we used the ‘ROCCONTRAST’ statement in SAS 9.2. The AUC results were considered excellent for AUC values between 0.9-1, good for AUC values between 0.8-0.9, fair for AUC values between 0.7-0.8, poor for AUC values between 0.6-0.7 and failed for AUC values between 0.5-0.6 [24].

Results

Of the 587 patients included in the study, 495 (84 %) completed the follow-up assessment. Twenty-five additional participants satisfied our criterion of experiencing a negative health outcome because they died (n = 14), were institutionalised (n = 6), or were too ill to be assessed with EASY-Care TOS at follow up (n = 5) and, therefore, could not receive further follow up assessment. Thus, in our primary analyses evaluating the prediction of a composite of negative health outcomes, 520 participants (89 %) of the initial cohort were included. The baseline characteristics of the study population are displayed in Table 1.

Table 1 Baseline characteristics of the study population
Full size table

Sixty-seven participants were not assessed at follow-up because they were untraceable (n = 10), were not assessed within the time frame defined for the follow up assessment (n = 32) or withdrew their informed consent (n = 25). At baseline, the patients with missing follow-up were more frail according to EASY-Care TOS (36 of 67 (54 %) versus 195 of 520 (38 %), p = 0.01) and their mental wellbeing was worse (mean (± SD) Rand-36 mental wellbeing subscale of 11.6 ± 4.2 versus 10.4 ± 3.7, p = 0.01). We found no large differences on other baseline characteristics between these two groups, although on average participants who dropped out had less favourable scores.

Of the 520 participants for whom the primary outcome was available, 294 (56.5 %) were considered not frail in step 1, 171 (32.9 %) were considered frail, and in 55 (10.6 %) participants the GP judged the available information was insufficient to qualify a person’s frailty status and a step 2 assessment was considered necessary by the GP. After step 2, 31 (56.4 %) of these 55 persons were defined as not frail and 24 (43.6 %) as frail. In all, after completing the EASY-Care procedure in their patients the GPs rated 325 of 520 (62.5 %) as not frail and 195 (37.5 %) as frail. After 12 months of follow-up, 89 of 520 participants (17 %) had a negative health outcome using our primary outcome measure, the composite of negative health outcomes.

In the group classified as not frail by the GP at baseline (n = 325), 30 (9 %) showed a negative health outcome. Of the group classified as frail at baseline (n = 195), 59 (30 %) showed a negative health outcome. This resulted in an absolute difference of 21 % (with 95 % CI: 14–28 %) (Table 2). The absolute differences for our secondary outcomes were 14 % (7–20 %) for ADL decline between these groups, for institutionalisation 4 % (0.6–7 %), and for death 6 % (2–9 %).

Table 2 Adverse outcomes at 12 months follow-up
Full size table

At follow up, the odds ratio (95 % CI) of frailty according to EASY-Care TOS for a negative health outcome was 4.2 (2.5–6.8), and for the subdomains ADL decline 3.3 (1.9–5.8), institutionalisation 4.4 (1.3–14.3), and mortality 11.7 (2.5–53.3). After adjusting for age, sex, number of diseases and number of medications the odds ratios were 2.9 (1.6–5.1), 2.2 (1.1–4.2), 2.4 (0.6–10.2), and 11.9 (1.9–73.4) respectively (Table 3).

Table 3 Odds ratios (95 % confidence interval) for the occurrence of a composite of negative health outcomes, ADL decline, institutionalisation, or death
Full size table

Mapping the study population with only EASY-Care TOS frailty judgement as a predictor of the composite of negative health outcomes resulted in a sensitivity of 0.66 (95 % CI: 0.56–0.76) and a specificity of 0.68 (95 % CI: 0.64–0.73). This corresponds with an AUC of 0.67 (95 % CI: 0.62–0.73). When using only the CGA based frailty judgement by the geriatrician, the sensitivity was 0.80 (95 % CI: 0.71–0.88), while the specificity was 0.58 (95 % CI: 0.53–0.62), with an AUC of 0.69 (95 % CI: 0.64–0.74). For mortality the AUCs were 0.75 (0.65–0.85) and 0.76 (0.74–0.79), respectively.

Table 4 shows how additional information changed the AUC of models predicting the composite endpoint negative health outcome as well as ADL decline, institutionalisation, and death separately. The prediction of the composite of negative health outcomes on the basis of age, sex, and GP practice resulted in an AUC of 0.70. The addition of information about number of diseases and medications improved the point estimate for the AUC with 0.03 points to 0.73 (p = 0.07). Using EASY-Care TOS judgement instead, added 0.05 to the AUC (p = 0.02). The model including the frailty judgement on the basis of a CGA instead resulted in a AUC of 0.76 (increase +0.07, p = 0.005). For ADL decline, institutionalisation and death separately, the patterns were comparable, although the AUCs were generally lower for ADL decline and the increase in AUC which resulted from adding extra information was less prominent for the prediction of institutionalisation (see Table 4).

Table 4 Predictive accuracy for predicting composite of negative health outcomes, mortality, ADL decline, and institutionalisation after 12 months of follow-up
Full size table

Discussion

This two-step method to map a population of older persons for frailty in primary care makes use of information that is already available about patients. In the Dutch primary care context, which emphasises longitudinal continuity and a strong doctor-patient relationship, this meant that only about 10 % of the older population evaluated with EASY-Care TOS step 1 needed a home visit for the frailty appraisal. The frailty decision was made by the GP, taking into account contextual information about patients. This study proves that EASY-Care TOS frailty judgement predicted negative health outcomes in the general population in the course of 12 months with a discriminatory performance that was close to the performance of a frailty judgement based on a geriatrician-led CGA in all participants. Neither judgement produced AUCs which can be qualified as good or excellent discriminative accuracy [24] for the composite of negative health outcomes, perhaps reflecting the difficulty to predict especially ADL decline and institutionalisation. The AUCs for predicting mortality in the following 12 months were better.

The predictive accuracy of the EASY-Care TOS procedure is comparable to the predictive ability of other triage tools described in the literature [12, 16, 18, 25]. Moreover, its predictive accuracy is in the range of that of a Comprehensive Geriatric Assessment by geriatricians, which may be considered to be the best available reference standard at present. For the prediction of negative health outcomes with EASY-Care TOS we found an AUC of 0.67 and for mortality of 0.75. The AUCs for functional decline (defined as ADL decline or as ADL decline or institutionalisation depending on the study) in other studies ranged from 0.55 to 0.82 [26–28]. The AUCs for mortality are slightly higher with a maximum of 0.87, but often over shorter periods of follow up. Also, the CGA judgement did not result in a much higher AUC for mortality and functional decline than the EASY-Care TOS judgement in our study (Table 4). A recent study showed that adding different frailty markers to age, sex, and chronic diseases increased the predictive accuracy for disability about 3 % [29]. This is comparable with the increase we found with adding EASY-Care TOS judgement to age, sex, and diseases and medication.

The EASY-Care TOS procedure suited the working style of primary care professionals, with a recent study showing that GPs have indeed a shared understanding of the medical concept of vulnerability, which equated to frailty [30]. For any frailty mapping tool to result in impact, at some point the outcomes have to be appreciated – most often still – by a healthcare professional for the actions which should follow from it. With the EASY-Care TOS procedure, the GP can do this at once.

One of the major strengths of this study is that it is unique in determining the predictive ability of a frailty mapping instrument specifically developed in cooperation with and for primary care professionals. In our previous work we already established construct validity and feasibility for the use of the EASY-Care TOS procedure in a primary care setting. Primary care professionals mentioned that it fits their needs and was acceptable for use in daily practice [19, 20].

One of the limitations of this study is that it was done in a country with strong primary care. Dutch GPs have stable patient populations, and their knowledge of patients is enhanced by the length of the patient-physician relationship. The external validity of our results thus depends partly on the health care system in which Easy-Care TOS will be used. Another point of interest is that the results of the baseline assessment of EASY-Care TOS may have influenced our follow-up results. We urged the GPs at the start of the study to intervene as little as possible as a result of the baseline assessments. Nevertheless, the GPs did collect lots of information about their patients with EASY-Care TOS. This may have influenced GPs’ care for these patients over time, and thus may have influenced the outcomes. As possible care interventions will have focused on prevention of further decline, it may have led to underestimation of the predictive abilities of Easy-Care TOS. Another point is that we could not completely prevent selection bias in our study population. The patients who were lost to follow up were more frail, compared to the patients with follow up. However, as these differences were minimal, we think this will not have influenced our results essentially. Our study was too small to allow for subgroup analyses comparing the predictive validity of EASY-Care TOS for different age groups and other potentially relevant divisions of groups of older persons. This requires further study. Finally, despite showing predictive accuracy of the EASY-Care TOS method with this study, the impact on patient outcomes of identifying frail older persons through EASY-Care TOS, e.g. through a randomised comparison with regular care, remains to be shown. However, a randomised comparison of EASY-Care based intervention in frail older persons in primary care showed its effectiveness [14].

Conclusions

This external validation study showed that an instrument based on the use of multi-domain prior knowledge by the GP can predict adverse outcomes during early follow-up. For most patients a decision can be made in several minutes without further assessment, and only a minority needs a more time consuming visit by a health professional. The accuracy obtained in this way is comparable to a comprehensive geriatric assessment for the whole population. EASY-Care TOS meets the needs of primary care professionals, and has been shown to be feasible for use in primary care. Mapping frailty, based on contextual and prior knowledge by GPs, offers a good starting-point for more effective pro-active care for the frail older person.

Availability of data and materials

Data are available for researchers with a specific research question. Interested and potential collaborators are invited to contact the corresponding author, Dr. René Melis (Rene.Melis@radboudumc.nl).