Background

Attrition is a major concern for accurate analysis of all trials and can influence the results of a study through potentially biasing the treatment effects and reducing the ability to detect differences [1,2,3]. Furthermore, conducting research with palliative care patients can be particularly challenging because of high levels of missing data and/or attrition due to high mortality rates and symptom burden [1, 4].

Authors report that the most important thing is to understand the reason for the missing data [5, 6]. Generally, missing data can be classified into three categories (Table 1): completely missing at random (CMAR), missing at random (MAR) and missing not at-random (MNAR), but in palliative care populations, missing data could mostly likely to be classified as MNAR because the patients being too unwell to complete a trial [1]. This is likely to be as a result of health deterioration, comorbidities and frailty [6], which are not random events [1, 7]. Recently, within the MORECare guidance, authors proposed three new categories to define the type of attrition in palliative care: attrition due to death (ADD), attrition due to illness (ADI) and attrition at random (AAR) [1]. Moreover, in 2013 the MORECare team developed guidance for conducting research with palliative care populations [8] and part of the checklist of conducting studies in palliative care, was how to deal with missing data and attrition [1, 8]. Consequently, the MORECare statements have been included in the EQUATOR Network website and database (http://www.equator-network.org/reporting-guidelines/morecare-statement/), to set clear standards on good practice in evaluating clinical studies in end of life care [9]. In fact, a barrier to the development of good practice in palliative care is the lack of quality research and evidence [8]. For this reason the MORECare guidance has been developed to identify research standards to aid future studies [8]. Furthermore, the inclusion of the MORECare guidance in the EQUATOR Network website could help to enhance the use of this guidance to improve the reliability of research in palliative care.

Table 1 Cathegories of missing data [1]
Full size table

A crucial aspect of clinical trials is the proportion of missing data and how researchers approach this to avoid serious biases [10]. The proportion of missing data is directly related to the quality of statistical inferences that can be made. Standard trial guidance suggests that the levels of missing data should be between 5 and 20%, meanwhile previous palliative care research has shown levels of missing data up to 80% of the total results [11]. Hussain’s (2016) review of the amount of missing data in clinical trials relating to palliative care populations found missing data levels of over 20% in half of the studies with an overall rate of 23.1%. In a detailed investigation of missing data in cancer trials involving palliative care populations Hui et al. (2013) found an attrition rate of 26% for the primary endpoint and 44% for participants reaching the end of the study [12]. Hui et al. also concluded that some investigators struggled to attribute the cause of the missing data.

Once the possible reasons for missing data are determined, the next step is to decide how to deal with them [6]. The method used to estimate the missing data needs to be reported, since different methods of estimating missing data, based on different assumptions, could lead to different conclusions [7]. For this reason, some literature suggests using more than one method for analysis and to discuss the potential bias of missing data [7]. This is particularly important for trials conducted in the field of palliative care, where most of the missing data are MNAR that could be estimated and minimised through the study design and taken account of in the final analysis. Moreover, there are no specific statistical methods recommended to analyise missing data not at random [13].

General guidance on the management of missing data in palliative care studies stated that missing data and attrition should be expected in a palliative care population and low levels of missing data or attrition could lead you to question whether the population was infact a palliative care population [1]. Another issue in trial reporting is describing or even deciding who the total palliative care population is. Only by defining this group can all potential participants be screened for eligiblility. In many hospitals a large number of patients could be deemed eligible for a study but they are spread out across different departments and include both out and inpatients. It is unclear what the usual practice is for identifying and screening palliative care populations and whether, for example, this means screening patients from particular outpatient clinics or carrying out database searches of diagnoses or symptoms [6].

Overall when reporting trials including palliative care populations three main areas were seen as requiring more scrutiny: classification of attrition, levels of attrition and their accompanying imputation methods and descriptions of trial populations.

In this study we wanted to investigate whether the MORECare classifications on reporting attrition in trials can be retrospectively applied to data retrieved from a systematic review on attrition in palliative care and thus help to better understand the reported results.

Methods

Primary Aims:

  • To describe whether the MORECare attrition classifications could be retrospectively applied to palliative care randomised controlled trials.

  • To describe whether there were any statistical differences between cancer and non-cancer patients and between the enrolment settings.

Secondary Aims:

  • To describe any methods used to handle missing data.

  • To describe if there was any correlation between the length of the time to primary outcome measure and the total attrition rate.

We conducted a systematic review to identify randomised controlled trials (RCTs) conducted in the last 5 years in a palliative care field. This review followed the methods of a Cochrane review [14] . This systematic review is part of a larger review looking at recruitment to RCTs in palliative care which covers the period from January 1990 to early October 2016 [15]. From this larger review, we selected randomised controlled trials involving palliative care populations from the last 5 years as reporting was likely to be of a better standard.

Identification and selection of studies

In the primary review [16] Embase, Medline, psychINFO and CINAHL databases were searched from the 1st January 1990 until the 8th October 2016 (see Table 2 and Fig. 1). Consequently, randomised controlled trials from 01.01.2010 to 08.10.2016, were extracted. The search included the terms palliat*, hospice* and ‘terminal care’ as they are seen as a robust and valid strategy to identify and retrieve palliative care literature [17,18,19,20]. The search terms used within Medline via EBSCO were palliat* or hospice* or terminal care or palliative care/ or palliative medicine/ or terminal care/ (not exploded) and randomi*ed. controlled trial* or randomised controlled trial/ (publication and topic). The search strategy was modified as necessary for the other databases searched (Table 2 for further details of the search terms used). The reference lists of the included studies were also hand searched to identify additional papers specifically focusing on recruitment to palliative care RCTs.

Table 2 Databases searched with search strategy
Full size table
Fig. 1
figure 1

PRISMA study flow diagram

Full size image

Study eligibility

Inclusion and exclusion criteria are listed in Table 3. AO and PS or LD screened the 3342 titles from the search. We used the WHO definition of palliative care which defines palliative care as an ‘approach which aims to improve the quality of life of patients and their families facing life threatening illness, through the prevention, assessment and treatment of pain and other problems, physical, psychosocial and spiritual’ [21] to identify palliative care populations.

Table 3 Inclusion and Exclusion Criteria
Full size table

Data extraction and analysis

Data were extracted by two independent reviewers (AO & PS or LD). If no agreement could be reached about inclusion of data extraction, an arbitrator (NP) was consulted. If there was insufficient information to make a decision about data, authors were contacted via email. If information were not forthcoming then the paper was excluded.

Data were collected to analyse the primary aim of the study, which was the retrospective application of the MORECare classifications on attributing the cause of attrition. We used the same criteria as Hussein (2016) to identify attrition, namely the number of participants lost by the time of the evaluation of the primary end point or final assessment if the primary endpoint was not made. The primary outcome was chosen because it was the most important outcome. Moreover it should have been defined at the time the study was designed to reduce bias [22] . In addition, we included attrition cases which occured between consent and randomization where available. Reasons for attrition were determined retrospectively to one of the MORECare classifications of ADD, ADI or AAR based upon decriptions within the paper. We used weighted means to describe the proportion of missing data.

To describe whether there were any statistical differences between cancer and non-cancer patients and between enrollment setting, we ran random effect models with and without moderators, using the Metafor package [23]. We aggregated double arcsine transformed values computing a weighted mean (with inverse-variance weight). To ease interpretation, we back transformed averages to estimated true proportions with corresponding 95% confidence intervals using Miller’s formula [24].

We used descriptive analysis to describe which imputation methods were used. The mean length of the time to primary outcome was calculated to assess whether length of stay was correlated with increased attrition rate using the Pearson correlation. Because the heterogeneity of data, it was not possible to calculate any correlation with patients’ overall survival, primary outcome and attrition.

Quality assessment

The quality of the trial was not assessed as the focus of the review was on attrition rates as recorded in the study.

Results

Study selection

Of the 3342 titles and abstracts screened, full text articles of 202 studies were assessed for eligibility of which 136 were included in the final analysis, which included 17,472 participants (Table 4).

Table 4 characteristics of included studies
Full size table

Study characteristics

We needed to decide which was the intervention and the control arm in 7 studies, because it was not specified by the authors. The median sample size was 75 (IQR 106). Among all the collected randomised controlled trials,few had a specific study design: 24 studies were feasibility/pilot studies, 3 were cluster trials, 2 were cross-over trials, one a fast-track trial. One study was designed to test the dose of a new drug. Four studies involved patients and their carers and one study patients and primary physicians. The median duration of studies to primary outcome measure was 7 weeks (IQR 11) with some studies having an intervention length of only a few hours or days. Thirteen studies did not mention the intervention duration.

In 5 studies participants were recruited from the hospice and in 28 from the hospital but it is unclear if these were inpatients and/or outpatients. The most common specific site mentioned in 47 studies was a ‘clinic’ which presumably meant outpatients. From the participants 25% were recruited from inpatient services, 30% from outpatient services and 16% recruited from both out patients and inpatients. For the remaining participants no indication was given as to whether they were inpatients or outpatients. Most patients had cancer (76%) with 20% having a non-cancer condition including heart failure, neurological conditions, respiratory, renal and liver disease or frail elderly populations. The remaining studies (4%) did not specify the patient’s condition.

Application of MORECare classifications

The attrition rate was not recorded in 17 trials leaving 119 trials with assessable total attrition data. We presented the data only with descriptive statistics because there were not sufficient data to calculate rates of attrition in the ADD, ADI and AAR groups (Table 5).

Table 5 Weighted means attrition using MORECare criteria (n = 91)
Full size table

We applied the MORECare classifications of attrition to 91 out of 119 papers that contained sufficient information on the cause of missing data. This reflects the difficulty in attributing the cause of missing data based upon the authors’ descriptions in the published papers. Some authors reported withdrawal as a cause of attrition, without specifying if this was related to a specific cause such as adverse events.

We found the main reason for attrition was attrition due to death (ADD) and accounted for a weighted mean of 31.6% (SD 27.4) of attrition cases. Attrition due to illness (ADI) was cited as the reason for 17.6% (SD 24.5) of participants. For 7% of total participants they left due to adverse events. In 50.8% (SD 26.5) of cases, the attrition was at random (AAR) with reasons such as patients being no longer contactable.

The weighted average attrition across all studies was 29% (95% CI 28–30%). The statistical analysis including participants’ diagnosis as a covariate (cancer vs non-cancer), was possibile in 113 studies. We did not observe significant differences between groups (non-cancer patients, 26%; 95% CI 18–34%; cancer patients, 24%; 95% CI 20–29%).

We were able to calculate whether including the study setting, inpatients and non-inpatients, as a covariate in 68 studies. We found significant differences between the two groups (p = 0.01), with a higher attrition rate for outpatients (29%; 95% CI 22–36%) than inpatients (16%; 95% CI 10–23%). These estimated proportions apperared to be all heterogeneous (ps < 0.0001). In some studies, authors did not distinguish the amount of inpatients for the amount of outpatients, thus it was not possible to conduct any statistical analysis.

Twenty trials reported data about attrition between enrolment and randomisation. These pre-randomisation data were too heterogeneous to be analysed. Although only 20 trials reported these missing data it may have been true for other studies too but not mentioned. Moreover, some authors commented upon the level of missing data in their papers, whilst in others no comment was made but attrition data was calculated from the CONSORT flow-chart. Because the data were heterogeneous, it was not possible to calculate any statistical difference between the studies that commented upon attrition and those studies that did not.

Use of imputation methods for primary endpoint

According to primary endpoint, 74 of 136 studies (54%) commented that they used an imputation method for missing data but only 36 (26%) recorded how they managed their missing data (Table 6).

Table 6 imputation methods
Full size table

As previously described, imputation methods should be reported, since different methods of estimating missing data, based on different assumption, could lead to different conclusions [7]. For this reason, part of literature suggests using more than one method for analysis and to discuss the potential bias of missing data [7]. Despite these recommendations, authors used different multiple imputation methods in only 15 studies. These methods were not uniform and different among each study.

Among the feasibility studies, one considered missing data as a random effect, five used a single imputation method (Area Under the Curve analysis, last observation carried forward, intention to treat analysis, conservative statistic). Only in one study, authors did not impute missing data because the main intention of the study was the feasibility of the intervention and also to explore the nature of missing data.

In few other studies [14], authors used different non statistical methods to deal with missing data, for example adapting their protocol to reduce the number of missing data (i.e. adapting the time of follow up or a specific questionnaire).

Length of intervention

In 108 out of the 136 studies, it was possible to describe the length of the intervention. In the remaining studies this was not possible because it was not clearly reported by the authors. The median time to primary outcome measure was 7 weeks. There was a significant correlation (r = 0.37, p < 0.01) between the length of time to primary outcome measure and the total attrition rate, meaning that the longer the time to primary outcome the increased chance of attrition.

Discussion

In this review we found that the MOREcare classifications could be applied retrospectively in about 67% of studies. In the remaining papers this was not possible due to insufficient details in reporting the reasons for missing data. We could not calculate any analysis in relation to the reason for attrition using the MORECare classifications due to insufficient data. Vague phrases such as withdrawn do not inform the reader as it is still unclear what the the reasons for the withdrawal were, for example was it due to progression of the illness or side effects of a drug or another reason? Dumville et al. (2006) recommend reporting the causes of attrition clearly to help understand the findings of a study [161] and applying the MORECare classifications gives an indication of not only what has happened in a trial but also the characteristics of the population involved.

Our review emphasises the need to identify primary outcome measures which should be measured sooner than later given the large amount of missing data in longer studies. Given the median time to primary endpoint was only 7 weeks, this shows that we are looking at end points potentially shorter than this but obviously this depends upon the focus of the study.

Palliative care populations are difficult to identify and these findings show a variation in where researchers looked for potential participants. Whilst we were able to make some comment upon where populations were identified from, this was difficult to extract as it was poorly recorded.

The level of missing data was higher than in other reviews (Hussain 2016; Hui 2013) which may reflect a broader definition of a palliative population. This is also reflected in the higher attrition rates noted in the non-cancer population and non-hospital populations. In the study by Hui (2013) the lower rates of attrition were in a cancer population based in one hospital. Modifications in trial design should be made for trials involving non-cancer, community based populations, as attrition rates were shown to be highest in these groups. Interestingly, we identified attrition even before randomisation. Perhaps this is something trial steering committees could monitor to assess the cause of attrition using the MORECare classifications, as it may help decide if attrition is due to the trial design or the population under study.

Only 26% of studies used any sort of imputation method for the primary outcome. All studies should comment upon missing data and notably report attrition following CONSORT guidelines not only for the primary outcome, but also for all the obtained results. Given all these studies were completed since 2010 you would expect this figure to be higher. With a rise in publishers asking for guidance in reporting of research to be followed hopefully this figure will increase. Moreover, according to the different type of missing data, different imputation methods can be used and it is recommended to use multiple imputation methods as a powerful tool for handling missing data with a sensitivity analysis [13].

One major concern about our review is that we relied upon our interpretation of descriptions of populations which we then decreed as palliative or not. Although the reviewers used the same definitions, their interpretation of the studies could have biased the reported results. Moreover, the causes of attrition have been interpreted according to the reasons given by the authors of the studies, which were not always clear. Hence some studies were excluded from this review because the causes of attrition were not clear. This may have changed the findings. As described, the high heterogeneity of collected data prevented further statistical analyses, such as calculation of the rates of attrition according to whether participants were in/outpatients, had cancer or not, or, according to type of attrition (ADD, ADI and AAR). The fact that most of the studies were about cancer patients limits the generalisability of our study in non-oncological settings. Moreover, most of the included studies were conducted in English-language nations.

This review included only randomised controlled trials, but more research is needed about the impact of missing data in other types of study design [162]. We assumed that from 2010, studies had a better standard of reporting and handling missing data. Further analysis about the correlation between the year of publication and the rate of missing data could have been assessed to analyse whether the reporting of missing data has improved over time.

Conclusion

The MORECare classifications provided a useful tool in highlighting attrition due to death in a readily accessible manner. In particular higher rates of attrition should be expected in longer trials, non-cancer andcommunity based palliative care populations. By applying the MORECare classifications we should be able to characterise trial populations more easily to enable a better understanding of the trials results. Moreover, the use of these classifications may help the readers to understand if authors described clearly the rate of missing data and if authors tried to take into account the rate of attrition in the interpretation of their results. The MoreCare guidelines could also help researchers to better design and conduct their studies in palliative care settings. In fact, the difficult we had in the collection of the data shows that more efforts should be done to report the results of the studies and to handle with missing information that could potentially bias the final results.