FormalPara Key Points for Decision Makers

Dementia Care Mapping is not cost-effective versus usual care in preventing or supporting agitation in care-home residents with dementia.

The trial highlighted a number of challenges to the economic evaluation that future studies should consider at design stage including dealing with: low intervention compliance; high levels of missing data; uncertain validity of utility assessments in this population; and high cost outliers.

1 Introduction

It is estimated that over 35 million people have dementia worldwide and this figure is expected to increase [1,2,3]. While dementia disease course and outcomes vary across individuals, the condition usually has a significant impact on survival, functioning and quality of life. In addition, the societal costs of dementia, including costs to the healthcare system and the family costs of caring for those with the condition, are considerable. These are estimated to be over £26 billion per annum in the UK and projected to rise to £40 billion per annum by 2040 [3].

Currently there is no cure for dementia, and therefore interventions aim to slow cognitive decline [4] and improve quality of life [5]. A significant challenge in supporting people with dementia is the occurrence of behaviours that are termed ‘neuropsychiatric’ or ‘behavioural and psychological’. These behaviours include agitation, shouting, biting, aggression, depression, anxiety and delusions, and can lead to harm of the person and others around them, caregiver stress and additional healthcare resource use [6]. As many as 90% of people living with dementia experience one or more of these behaviours at some point [7]. Over a third of people with dementia live in care homes and close to 80% of residents experience such behaviours at some point [8], with agitation being the most common [8] and having a significant impact on resident quality of life [9] and care costs [4].

The occurrence of such behaviours is partly dependent on the ability of staff to provide person-centred care that meets residents’—often complex—individual needs [10]. Pharmacological treatments have limited efficacy and serious potential side effects including increased mortality, and thus psychosocial interventions are the recommended first line of treatment [11]. Psychosocial interventions can target the resident, caregiver or the care-home environment [12]. Dementia Care Mapping™ (DCM™) [13] is a care-home-level intervention that aims to support the embedding of person-centred care, improve the quality of care and, consequently, health outcomes for residents. It has been used widely in the UK and elsewhere [14, 15]. The DCM process involves provision of formal training to care-home staff to use the tool and a recurring five-phase implementation cycle of: briefing, observation, data analysis and reporting, feedback and action planning. Trained mappers conduct systematic observation of residents (behaviour, activities, mood, and quality of staff interactions and care approaches) and information is fed back to other staff involved in care to help with the implementation of person-centred care-action plans. This process is repeated every 4–6 months.

Despite widespread use of DCM, there is limited robust evidence of its effectiveness, and that provided by randomised and non-randomised studies is mixed [16]. Evidence on the value for money of DCM is scarcer still [17]. Only two studies report an economic evaluation of the intervention [4, 18], neither of which present cost-utility analyses.

Given that there are 16,000 care homes in England alone (mostly providing elderly care) [19] and growing numbers of people living with dementia, interventions in this context and population, even if relatively cheap or infrequently adopted, have the potential to generate substantial opportunity costs. Hence, such interventions should be evaluated thoroughly to establish value for money before full implementation.

The DCM EPIC trial was a pragmatic, cluster-randomised, controlled trial of usual care plus DCM (intervention) versus usual care (control) for reducing agitation in care-home residents with dementia [20]. The aim of the current research was to conduct a full economic evaluation of the DCM-EPIC trial from a health- and social-care-provider perspective and over a 16-month time horizon, incorporating comprehensive costing, cost-effectiveness and cost-utility analyses. Furthermore, we wished to highlight methodological issues we encountered in conducting an evaluation in this population and setting.

2 Methods

2.1 Trial

The DCM EPIC trial was conducted in England with care homes randomised 3:2 to intervention:control. Care homes and then individual residents were recruited into the study. Inclusion criteria for individuals included being a permanent care-home resident and having a formal diagnosis of dementia. The primary outcome was agitation as measured on the Cohen-Mansfield Agitation Inventory (CMAI), a 29-item measure assessing frequency of agitated or aggressive behaviour in the past 2 weeks [21]. The CMAI was completed by a staff member who knew the resident well. Data were gathered on health outcomes and dementia severity (Functional Assessment Staging Test (FAST) [22] and Clinical Dementia Rating (CDR) [23]). Usual care was defined as care routinely delivered within the setting, and was likely to include person-centred care but excluded DCM training. DCM adhered to procedures outlined in the DCM manual and guidance [13, 24]. In brief, two staff members from each home were trained to use DCM (mappers) and then aimed to implement three DCM cycles with an expert mapper supporting completion of the first cycle. DCM data were used to create action plans for improvements in the care home with progress on these actions monitored. Due to higher than expected resident loss to follow-up (approximately 50%), additional residents were recruited at 16 months to allow a cross-sectional analysis (open cohort).

Ethical approval was granted by NRES Committee Yorkshire & The Humber-Bradford Leeds (REC ref 13/YH/0016). The trial was registered with the International Standard Randomised Controlled Trial Register (ISRCTN reference 82288852). Further trial details are provided elsewhere [20, 25].

2.2 Economic Evaluation

The economic evaluation time horizon was 16 months. We did not model the intervention costs/consequences beyond the trial period, mainly due to lack of evidence for sustained effectiveness [4]. The evaluation followed the National Institute for Health and Care Excellence (NICE) reference case [26]. The primary end-point was cost per quality-adjusted life-year (QALY) gained at 16 months from the perspective of the health and personal social services provider. We also conducted a cost-effectiveness analysis presenting cost per unit change in CMAI.

2.3 Costs

DCM costs consisted of: (1) delivery and receipt of DCM training; and (2) DCM implementation. The DCM training required 4 days of staff time. Additional training costs included course fees, training materials, accommodation, meals, subsistence and travel. Implementation costs included mapper staff time and expert mapper involvement (cycle one). It was assumed (subsequently supported by the process evaluation) that additional time was not required for care-home staff to attend DCM briefing and feedback sessions, but that these were arranged at handover. We did not cost activities initiated as a result of the DCM process as it was assumed existing staff conducted these in existing sessions. The primary analysis assumed that the intervention was delivered as per protocol with all cycles implemented and costed. A sensitivity analysis only costed partially and fully implemented cycles.

Resident healthcare resource use in the previous 3 months was collected at 6 and 16 months by the researchers using care-home records. Simple linear extrapolation was used to handle coverage gaps in time due to the 3-month recall period (i.e. resource reported at 6 months was multiplied by 2). This was also collected for a proportion of residents at baseline. Additional data on hospital visits/stays were also collected from administrative data and used as the primary source of secondary healthcare use. Medication use was captured at the same time points. Unit costs for resources were obtained from the PSSRU [27], eMIT national database [28] and NHS reference cost database [29] (see Supplementary Table 1). All prices are 2017 British Pounds (£).

2.4 Utility

Utility was measured at baseline, and at 6 and 16 months using the EQ-5D-5L [30], completed by residents, and the EQ-5D-5L-Proxy, completed by staff and relatives. The UK general population tariff [31] was used with a (− 5L to − 3L) mapping algorithm used as a sensitivity analysis [32]. Utility values were also calculated using the DEMQOL-Proxy (DEMQOL-PROXY-U), completed by staff and relatives, and scored using the UK general population tariff [33].

The primary analysis was based on staff proxy EQ-5D-5L data as this represented the most complete response set. Further sensitivity analyses were based on the DEMQOL-Proxy-U. EQ-5D-5L utility data is also presented employing (in order of priority) resident, relative proxies and staff proxies using the latter when the former was not available at all time-points.

2.5 Analysis

Total QALYs were calculated based on EQ-5D-5L and DEMQoL-Proxy-U utilities. If residents died, their utility value was assumed to be zero with a linear transition from their previous health state. CMAI was calculated at each time point when residents were alive; those who died were excluded. Total costs combined intervention cost and resource use costs at 6 and 16 months. To capture the costs incurred prior to death, a daily cost was estimated based on previous resource consumption (at baseline or 6 months).

Incremental costs and QALYs (or CMAI) were estimated using a seemingly unrelated regression (SUR) approach accounting for the correlation between costs and outcomes. We implemented robust standard errors to account for the clustering at care-home level using the Stata nlsur package. The QALY model was adjusted for cluster size, baseline utility, age, gender and dementia severity (Clinical Dementia Rating). Costs were adjusted for the same factors excluding baseline utility. We estimated incremental cost-effectiveness ratios (ICERs) and used the NICE willingness-to-pay threshold of £20,000 (λ) to define cost-effectiveness [26]. We compared the CMAI ICERs to those estimated in previous research. A discount rate of 3.5% was used for costs and effects post 12 months.

Non-parametric bootstrapping was used to determine the level of sampling uncertainty around the ICER with results represented in a cost-effectiveness plane and cost-effectiveness acceptability curve (CEAC) [34]. Our base-case analyses took the intention-to-treat approach. Given the pragmatic nature of the trial and limited care-home compliance (only 25.8% completing more than the one expert mapper supported DCM cycle), we also explored the impact of intervention compliance on cost-effectiveness in sensitivity analyses.

2.6 Missing Data

We ran the resident-level analysis on complete cases (CCA) initially, requiring data on total QALYs and total costs. However, the primary analysis used data where missing values were imputed using multiple imputation (MI) based on an assumption that data were missing at random (MAR) [35]. The number of imputations (n = 48) reflected the ratio of missing:complete data. We used the predictive mean-matching approach and included baseline values (cost or utility), age and binary variables denoting mortality in the imputation model. We also included study site to account for clustering within care homes. Rubin’s [36] rules were used to combine parameter estimates on the imputed datasets. Since the base-case analysis used proxy response data, there was some protection against informative missing data; however, we also conducted a sensitivity analysis to account for a missing not-at-random (MNAR) scenario. This used a Pattern Mixture Modelling approach with costs and QALYs systematically increased or decreased and the effect on probability of cost-effectiveness plotted. We used the approaches outlined in [35]: increasing costs in year 1 by 10–50% and subsequent years by 10% in DCM; doing this for DCM and control; reducing QALYs by 10–50% in year 1 and 10% in subsequent years in DCM; doing this for DCM and control.

2.7 Open-Cohort Analysis

The open-cohort design meant that data from additional residents were available at 16 months. For the primary analysis, we only used data from residents consented into the trial at baseline (closed cohort; n = 726). An additional analysis was conducted that also incorporated data from the open cohort (n = 917). Where data were only available at 16 months, we imputed the total costs and QALYs for the whole trial period. This was implemented as a supplementary analysis to follow methods used in the primary statistical analysis. This approach necessarily assumes that survival is independent of the intervention and time spent in the care home and thus results should be considered illustrative only.

2.8 Sensitivity Analyses

Deterministic sensitivity analysis of the ICER tested the robustness of the results to changes in the analytical approach. We explored the impact on results of different approaches to costing, handling missing data and utility assessment.

3 Results

There were 389 residents in the complete case (where EQ-5D and resource use data are available at all time-points) and the sample described in Table 1. Supplementary Fig. 1 outlines the data available for the economic evaluation. Data for 46% (337/726) had to be multiply imputed. A greater proportion of missingness was due to missing resource use than EQ-5D data (21% and 8% missing at month 6; 21% and 16% missing at month 16 for resource use and EQ-5D, respectively). In the complete case data, there were 19 clusters in the control arm and 30 in the DCM arm. Mean (SD) number of residents per cluster were 14.11 (7.86) and 9.88 (4.29), respectively. The intra-cluster correlation coefficients for costs, EQ-5D-5L QALYs and DEMQoL-U QALYs were 0.02, 0.14 and 0.10, respectively. Supplementary Table 1 includes information about cluster balance and informative clustering. There was some evidence of cluster imbalance (standardised difference > 10%) and of significant correlation between costs and cluster size, hence we chose to adjust ICERs by the latter factor.

Table 1 Sample characteristics at baseline
Full size table

3.1 Costs

The costs of the DCM intervention and the assumptions behind this are described in Supplementary Table 2, and unit costs and descriptive statistics on resource use are included in Supplementary Tables 3 and 4, respectively. The total cost of the DCM intervention was estimated to be £421.07 per resident (£9290.30 per care home on average). Control arm intervention costs were assumed to be zero. Unadjusted resource use and total costs are presented in Table 2 (UK £ sterling 2017 prices). Total costs were £3539.00 and £2059.58 on average per resident in the intervention and control arms, respectively. Non-parametric Mann–Whitney U tests suggest these costs were significantly different for the imputed and complete case (both p < 0.05) samples.

Table 2 Intervention and resource use costs
Full size table

Primary-care costs were similar across arms while secondary-care costs were noticeably higher in the intervention arm. The intervention arm included six high-cost individuals whose costs exceeded the maximum in the control arm, due to long hospital stays or one-to-one care; these individuals were excluded along with seven other high-cost individuals (generated in the imputation) in a sensitivity analysis. Baseline costs were similar across arms although slightly higher (mean) in the control group. However, there was a higher maximum cost in the intervention arm. Given this, and because baseline costs were only available on a sub-sample, we only controlled for baseline costs as a sensitivity analysis.

3.2 Utility

Staff proxies provided the greatest proportion of utility data (n = 453; 62%), followed by relative proxies (n = 176; 24%) and resident self-report (n = 168; 23%). Figure 1 shows the utility values (with imputation) across different assessment strategies with the primary analysis based on data shown in Fig. 1a. There was baseline imbalance with the control arm having marginally higher baseline utility, which required adjustment. Mean EQ-5D scores declined during the trial over 16 months as may be expected given that deaths are coded as zero. The values from the 5L to 3L mapping were noticeably lower than the direct 5L valuations. While the utility differential between arms appeared relatively trivial, this was larger in the combined patient/carer/staff completion and DEMQoL-Proxy-U approaches.

Fig. 1
figure 1

Utility over time. Deaths coded as zero; c uses complete case data and a, b and d use imputed data; 3L indicates EQ-5D-3L

Full size image

3.3 Cost-effectiveness

The ICERs for the base-case and sensitivity analyses are given in Table 3A, B, respectively. In the base-case cost-utility analysis (closed cohort, staff proxy EQ-5D-5L with multiple imputation), intervention was more costly (by £1479) and more effective (by 0.024 QALYs) than control. This yielded an adjusted ICER of £64,380 per QALY, well above the £20,000 NICE threshold, indicating that DCM is not cost-effective versus control. Figures 2 and 3 are the cost-effectiveness plane and the CEAC, respectively, for the base-case analysis. A majority of the simulations lie above the willingness-to-pay threshold, suggesting that DCM is unlikely to be cost-effective versus control. The CEAC indicates that, where λ = £20,000, there is a very low probability (p = 0.13) that DCM will be cost-effective.

Table 3 (A) Base-case analysis and (B) sensitivity analyses
Full size table
Fig. 2
figure 2

Cost-effectiveness plane: intervention versus control. DCM + UC dementia care mapping plus usual care, UC usual care

Full size image
Fig. 3
figure 3

Cost-effectiveness acceptability curve. DCM dementia care mapping

Full size image

The complete-case analysis (full cost and utility data) yielded similar costs to the imputed sample but higher incremental QALYs for the intervention. With the exception of the analyses that excluded high-cost outliers (discussed below) and restricting the sample based on compliance (at least two cycles of DCM completed), the ICERs from sensitivity analyses also exceeded £20,000. The sensitivity analyses tended to show DCM having an incremental QALY benefit (ranging from 0.24 to 0.61) but with higher costs (ranging from £364 to £1774) than control. The open-cohort analysis yielded lower incremental costs and higher incremental benefits for the intervention than the base case but an ICER (£46,556) still above £20,000.

In sensitivity analyses excluding high-cost outliers in the intervention arm (n = 6), incremental costs were reduced dramatically and the ICER was reduced but was still above the cost-effectiveness threshold (£41,905/QALY) in the base case. However, the ICER fell below the threshold (£16,041/QALY) in the complete-case scenario. The ICER also decreased with greater intervention compliance (≥ 1 cycles received = £37,289; ≥ 2 cycles received = £13,081). Analysis adjusting for baseline costs yielded reduced ICERs but were based on a much smaller sample (n = 123) and cannot be considered robust. The analysis exploring the assumption of missing at random in the multiple imputation is shown in Supplementary Fig. 2. We can see that systematic increases in the cost or decreases in the QALYs of those with missing data do not dramatically change the probability that DCM would be cost-effective and it never exceeds 0.25.

Cost-effectiveness analyses based on improvement in CMAI indicated that while the intervention was more costly, it was also more effective in reducing agitation. Incremental cost-per-unit improvement in CMAI was £272 and £76 for intervention versus control for the imputed and complete case samples, respectively.

4 Discussion

We conducted one of the most comprehensive economic evaluations of a non-pharmacological intervention to prevent and support behavioural symptoms in dementia. The base-case analysis yielded an ICER of £64,380 per QALY, indicating that DCM in this context would not be an efficient use of health/social service resources.

There are no other cost-utility studies that permit a direct comparison. We estimated the cost of delivering DCM to be £421.07 per person over the trial period. This is higher than estimates presented elsewhere (£119.06 and £140.01 for two Australian studies [4] [both inflated from 2011 £GBP] and £2160 per care home for a Dutch study [37] [converted/inflated from 2013 $US]), although this may partially be explained by costing assumptions relating to the number of cycles and length of follow-up. These figures are also likely to be sensitive to the resident denominator (how many people the costs are spread across). However, reductions in the cost of DCM (based on spreading training costs over additional years or only costing implemented cycles) did not impact the decision. Our base-case estimate of the cost of CMAI unit improvement (£272) was higher than previous estimates (£26.77 [18] [converted/inflated from 2008 $Aus]; £6.61–£68.35 [4] [inflated from 2011 £GBP]), perhaps for the reasons given above. Costs of resource use were substantially higher in the intervention arm than the control. This was driven by higher secondary-care costs resulting from (n = 6) high-cost individuals in the intervention arm. A strength of this study was the range of utility assessment strategies used but, while these led to absolute QALY differences, the between-arm utility differences were small.

The sensitivity analyses were consistent in finding the intervention to be more costly and effective than control. Most analyses yielded ICERs well above the £20,000 threshold; a complete case analysis excluding high-cost outliers (£16,041/QALY), and analysis including only those care homes completing two or more DCM cycles (£13,081/QALY) being exceptions. Lower ICERs observed for analyses restricting the sample, e.g. requiring baseline costs for adjustment or including compliant care homes, were possibly due to the coincidental omission of cost outliers.

4.1 Methodological Challenges and Future Research

Economic evaluations in dementia and care-home settings share the same challenges as other primary research conducted in these contexts relating to data completeness and quality. Specific issues encountered here related to high loss to follow-up of residents and their proxies, low intervention compliance, the validity of quality-of-life data and accurate cost attribution in a group who incur higher than average resource use. Furthermore, choice of time horizon and perspective was not straightforward. A number of assumptions and approaches were adopted to deal with these issues.

We chose a narrow cost perspective (health and personal social services) in our analyses, as recommended by NICE. In theory, DCM could have institutional benefits that may result in cost savings (for example, via reduced staff turnover); however this was not borne out in our trial results and thus not considered. Future evaluations in this population could explore additional perspectives including that of the care home itself, especially where they commission interventions directly. However, this will be complicated by the fact that many homes have combined healthcare-funded and self-funded residents, still receive funding when residents are in secondary care and may receive quality improvement incentives from the healthcare system.

We limited our analysis to the trial period, electing not to model costs and benefits forward over time. There does appear to be a small QALY benefit for DCM over control; if this were extrapolated forward, assuming the intervention costs are sunk, then cost-effectiveness metrics for DCM may improve in line with the time horizon. This is especially true if benefits are extended to residents (present and future) not included in the trial. However, such an approach would be highly speculative and there is no evidence that any DCM effects would be sustained, either for individuals or institutionally. Furthermore, any future effect will be attenuated by the high staff turnover that often occurs in care homes, impacting on DCM delivery. Any modelling forward of benefits in this setting would have to justify the duration of effect both for the individual and the institution, justify the denominator population (for sunk costs) and potentially factor in additional costs to train new staff.

Poor intervention implementation was a significant challenge in the current trial. While our primary analysis was based on intention to treat, we explored the impact of compliance in the sensitivity analyses. The Complier Average Causal Effect (CACE) is an alternative approach to dealing with non-compliance that is beginning to see application in economic evaluations [38]; however, we did not believe it was relevant in our context as non-compliance was at the care-home level only and was observable.

Missing data (around 45% at 16 months) in the trial was substantial and handled in a standard way, relying on multiple imputation. Due to high losses to follow-up, the trial design was changed to an open cohort study, recruiting additional residents at final follow-up to maintain power in the statistical analysis for the primary end-point. However, since economic evaluations analyse the stream of costs and benefits across the trial period, this innovation led to a greater reliance on imputation and was consequently considered a supplementary analysis only. Additional research is needed to help understand whether open-cohort designs can have a role in economic evaluations. We used care-home records as a source for secondary-care use and would recommend this approach (or other administrative data capture such as NHS Digital) to minimise the impact of loss to follow-up and reduce missing data.

Previous research has highlighted the challenges in quality-of-life assessment in this group [39]. These challenges relate to the level of missing data, validity of either self- or proxy- reports, and the ability of available measures to capture relevant disease and intervention impacts. We used several approaches to capture utility to allow an assessment of which was most appropriate and practical. The utility capture strategy based on the DEMQoL-Proxy-U yielded the greatest incremental QALYs for DCM and around a £24,000 reduction in the ICER compared to the staff EQ-5D-5L method. It is not clear why this is the case: it may be evidence of greater sensitivity of the disease-specific measure but the absolute differences versus the EQ-5D are very small. Equally, it may relate to the sample reporting on this measure. Research suggests individuals with mild cognitive impairment can provide valid self-reports and for those that cannot, proxy reports may be used [39]. However, research is needed to identify the threshold for utility measure self-completion and to test methods of combining self-complete and proxy data, which may include adjustment of values or mapping.

The economic evaluation results were heavily influenced by high-cost outliers. When we examined the reason for hospital admission for the cost outliers, it was not possible to rule out attribution to the receipt of DCM. For example, it is conceivable (although, unlikely) that DCM was associated with the implementation of more activities and this may have led to adverse events such as falls. Given this, there was no reasonable justification for removing these individuals from the main analyses. While some consideration has been given to the impact of high-cost individuals [40], further research is needed on this topic. Additional information on the cause of secondary-care resource use may be useful since this often drives overall costs. However, a systematic and transparent method of judging whether that use was related to intervention receipt or not, for example, via an independent clinical assessment may also be needed. It is possible that such an assessment could sit within a quantitative process evaluation framework and may inform additional sensitivity analyses.

5 Conclusion

We did not find DCM, as delivered in this study, to be cost-effective versus control and conclude that future research should investigate the value for money of alternative strategies to help prevent and support behavioural symptoms in people living with dementia in care homes.

The DCM-EPIC trial setting and data posed a number of challenges for the economic evaluation. The use of administrative data and running several concurrent data capture strategies, followed by extensive sensitivity analyses using these data sources, may help mitigate the challenges and reduce uncertainty around the results. Additional research on innovative trial designs and statistical approaches to dealing with issues of compliance is required, as is research to optimise utility assessment in this population and to identify methods of dealing with cost outliers.