Background

Breathlessness is a disabling and distressing symptom of advanced non-malignant disease that severely reduces the quality of life of patients and their families [1, 2]. Defined as “a subjective experience of breathing discomfort that consists of qualitatively distinct sensations that vary in intensity” [3], it is the main symptom of advanced chronic obstructive pulmonary disease (COPD) but is also the common denominator in many other advanced non-malignant respiratory, cardiac and neuromuscular conditions. In approximately two-thirds of patients presenting with breathlessness the underlying cause is cardiopulmonary disease [4]. Uncontrolled breathlessness can be terrifying for patients and their families: achieving symptom control is therefore a priority [5]. In many instances addressing breathlessness from a symptomatic point of view is the only therapeutic option when it persists in spite of optimised medical management of the underlying condition. However, access to palliative care is known to be poor for patients with advanced non-malignant conditions compared to those with cancer [6–10], despite their well-established need [11, 12].

The Cambridge Breathlessness Intervention Service (BIS) is a multi-disciplinary complex intervention combining non-pharmacological and pharmacological interventions to support breathless patients with any advanced disease in managing their symptom, theoretically underpinned by a palliative care approach [13–15]. Developed and evaluated [16–21] using the Medical Research Council (MRC) framework for complex interventions [22], it has undergone a Phase III RCT with two sub-protocols: one for advanced cancer and one for advanced non-malignant disease (due to differing service models) [23]. The findings of the sub-protocol for patients with advanced cancer have been published previously [21]; this paper reports the findings of the contemporaneously conducted sub-protocol for patients with advanced non-malignant disease.

Methods

A detailed study protocol, including descriptions of both the intervention and standard care, has been published in an earlier issue of this journal [23]. Subsequent to this the intervention has been more fully described elsewhere [13]. Key aspects of study design, sampling, outcome measures, data collection and analysis for the sub-protocol for patients with advanced non-malignant disease are outlined below. A completed CONSORT 2010 Checklist is provided in Additional file 1.

Study design

We recruited patients with advanced non-malignant disease referred to BIS into a Phase III mixed method single-blind pragmatic fast-track RCT of BIS versus standard care (July 2008–June 2010). Participants were randomised to one of two groups using randomly permuted blocks of random size 2, 4 and 6, generated by the study statistician and concealed within sealed opaque envelopes until allocation notification by the intervention deliverer: the fast-track (intervention) group received BIS immediately whereas the waiting-list (control) group received BIS after four weeks. All participants continued receiving standard, including palliative, care.

Participants

Consecutive patients with non-malignant disease who were referred to BIS were invited to participate in the trial, by letter. Patients were eligible if they met BIS referral criteria (they had a diagnosed appropriately treated cause of breathlessness, were troubled by breathlessness in spite of optimisation of underlying illness, and might benefit from a self-management programme) and excluded (from the trial only) if they had received BIS previously. Recruited patients were asked to identify who gave them the most help and support at home (family member or friend); these informal carers were also invited to participate.

Ethics, consent and permissions

Ethical approval was given by Cambridgeshire2 NHS REC (Ref: 08/H0308/157). All participating patients and carers gave informed consent.

Participant-reported data and sample size

Patient ‘distress due to breathlessness’ (the primary outcome on which the trial was powered) was measured using a numeric rating scale (NRS). A sample size of 60 randomised patients (26 analysed per arm, with allowance for dropout) would provide 80 % power to detect a 2-point difference in mean distress between groups (SD = 2.5, alpha = 5 %), with increased precision anticipated from adjustment for baseline. Other key patient-reported measures included the Chronic Respiratory Questionnaire (CRQ) [24] and the Hospital Anxiety and Depression Scale (HADS) [25]. Key carer-reported outcome measures included an NRS for carer distress due to patient breathlessness and the HADS. The EQ-5D [26] and Client Service Receipt Inventory (CSRI) [27] were administered for the health economic analyses. Brief qualitative topic-guided interviews were also conducted with all patients and carers to explore their expectations and experiences of BIS. A full list of baseline measures and outcomes is reported in the published protocol [23].

Data collection

Participating patients completed a baseline interview (t1) prior to randomisation. This mixed method interview included both the quantitative patient-reported measures and the brief qualitative topic-guided interview described above. Carers were interviewed separately where possible, using a mixed method interview of quantitative carer-reported measures and a brief qualitative topic-guided interview. Similar mixed method follow-up interviews were conducted with both patients and carers at each subsequent follow-up interview (t2–t5) at fortnightly intervals. Interview two (t2; two weeks after baseline) was designed to represent the midway point in either receiving the BIS intervention for the intervention arm or the waiting-list period for the controls; interview three (t3) was designed to represent completion of BIS for the intervention arm, or the end of the waiting-list period prior to commencing BIS for the controls; interview four (t4) was designed to represent the midway point in receiving the BIS intervention for the controls (no t4 was conducted with patients and carers on the intervention arm); and the final interview (t5; eight weeks from baseline) was designed to represent four weeks after BIS for the intervention arm, and the completion of BIS for the controls. Data collection was designed to facilitate researcher blinding to group allocation for the collection of primary and key secondary outcomes at t3. That is, researcher blinding was explained to study participants on recruitment, they were reminded at the start of t2 not to let the researcher know their group allocation, and at the start of t3 they were asked not to let the researcher know their group allocation until the researcher came to open their group allocation envelope just prior to CSRI completion (after the collection of primary and key secondary outcomes at t3). Both clinical and administrative staff were also reminded of the importance of researcher blinding throughout the study.

Analysis

Intention-to-treat analyses, within completers, were conducted using a linear regression model; each outcome was adjusted for its baseline. Costs were calculated by combining service use data (collected for the two months prior to baseline and at four-week follow-up) with UK 2011/12 unit costs [28]. The cost of the intervention was calculated at £91 per contact based on specialist nursing input costs, with phone contacts costed at 25 % of this. Costs were combined with the primary outcome and EQ-5D-derived quality-adjusted life years (QALYs), with uncertainty explored using cost-effectiveness planes.

Qualitative interview data were transcribed and anonymised. As first described in our paper reporting the results of the sub-protocol for patients with cancer [21], two approaches were taken to the analysis of this unusually comprehensive qualitative dataset. First, transcripts of t3 interviews from all patients and carers were categorised into one of three intervention impact levels by three analysts working independently (Level 1: Significant impact - clearly stated BIS made a difference; Level 2: Some impact - no major change recognised, but valued specific aspects of BIS; Level 3: No impact – BIS made no difference at all). Categorisation commenced with a small number of interviews. The three analysts then met to compare their categorisations, discussing and resolving differences through clarifying both level definitions and data interpretation, before repeating this process for all remaining interviews with patients with non-malignant disease and their carers.

Second, as qualitative analysis of this size of dataset would be unmanageable, 20 patient (and associated carer) intervention arm transcripts were purposefully sampled against a four-cell matrix of t3 changes in the primary outcome, to achieve a maximum diversity sample [29]. The four cells represented: (Cell 1) patients who improved most on the primary outcome (who, predictably, had high baseline scores; the Biggest Improvers); (Cell 2) patients with high baseline scores (to match Cell 1) but who improved least (Limited Improvers); (Cell 3) patients who worsened (who transpired to have low-middling baseline scores; Worseners); and (Cell 4) patients with closest matches to Cell 3 baseline scores but who improved most (Moderate Improvers). Anonymised interview transcripts for this purposive sample were imported into NVivo software [30] to facilitate framework analysis [31]. This analysis explored patient and carer reports of the nature of the impact of BIS and which aspects of BIS were valued, and sought to identify possible mechanisms of impact.

Results

Recruitment and baseline characteristics

Figure 1 (CONSORT flow diagram) indicates that we randomised 87 patients and 79 (91 %) completed the trial to the key outcome measurement (t3). Eight patients withdrew from the trial prior to t3: three and five from the intervention and control arms respectively (including one death in each arm). Researchers remained blinded to group allocation for 67 % (53/79) of patients for collection of the t3 primary outcome.

Fig. 1
figure 1

CONSORT flow diagram

Full size image

Baseline characteristics were well matched across trial arms (Tables 1 and 2). Patients were predominantly elderly, male and living with others. COPD was the commonest non-malignant disease (predominantly GOLD stage 3 or 4; severe to very severe). Their mean HADS anxiety score was slightly higher than population norms and their mean HADS depression score notably higher (mean population norm for anxiety 6.14 (SD = 3.76) and depression 3.68 (SD = 3.07) [32]). Just over half (n = 46, 53 %) had anxiety scores that were clinically meaningful (clinically meaningful threshold of 7) and just under half (n = 39; 45 %) had clinically meaningful depression scores. Breathlessness, performance status and co-morbidity were as anticipated for these patients. Patient ‘distress due to breathlessness’ and CRQ domain scores were similar across trial arms.

Table 1 Baseline characteristics by arm, BIS Phase IIInm - patients with non-malignant conditions
Full size table
Table 2 Baseline characteristics by arm, BIS Phase IIInm – informal carers of patients with non-malignant conditions
Full size table

Carers were predominantly older women, and about half were in employment. As with the patients their mean HADS anxiety and depression scores were higher than population norms [32]. Anxiety scores were clinically meaningful for just over half of the carers (n = 25, 52 %) and just under a quarter (n = 10, 21 %) had clinically meaningful depression scores. Carer ‘distress due to patient breathlessness’ was lower than patient distress, but similar across trial arms.

Change in patient and carer distress due to breathlessness

Comparison of change in patient distress due to breathlessness (primary outcome measure; NRS range 0–10) from baseline (t1) to the key outcome measurement point (t3) showed that patients randomised to the intervention arm achieved a greater, 2.22-point, reduction in their distress due to breathlessness compared with a 1.56-point reduction for controls; however, this was not statistically significant: adjusted difference of –0.24 (95 % CI: –1.30 to 0.82), p = 0.65 (Table 3). Carers of patients randomised to the intervention arm achieved a greater, 1.03-point, reduction in their distress due to their patient’s breathlessness compared with a 0.2-point increase for controls, but again this was not statistically significant: adjusted difference of –0.42 (95 % CI: –1.86 to 1.02), p = 0.56 (Table 4).

Table 3 Changes in patient distress due to breathlessness (primary outcome), mastery, anxiety and depression, by trial arm, for patients with non-malignant conditions
Full size table
Table 4 Changes in carer distress due to their patient’s breathlessness and carer anxiety, by trial arm, for carers of patients with non-malignant conditions
Full size table

Change in patient mastery of breathlessness, and patient and carer anxiety and depression

Mean CRQ mastery scores improved slightly by t3 on both arms with greater improvement in the intervention arm; not statistically significant (Table 3). No significant differences were found between trial arms to t3 on other CRQ domains (dyspnoea, fatigue or emotional function). Mean patient anxiety scores decreased slightly for the intervention arm and increased slightly for the control arm by t3 and mean depression scores decreased slightly by t3 in the intervention arm and remained stable for controls; none of these modest changes was statistically significant (Table 3). Mean anxiety scores for carers achieved a greater, 1.65-point, reduction by t3 in the intervention arm compared with a 0.15-point reduction for controls, but again this was not statistically significant: adjusted difference of –1.22 (95 % CI: –2.84 to 0.40), p = 0.14 (Table 4). There was little change in other patient or carer secondary outcomes.

Reported benefit of BIS

Categorisation of qualitative interviews (n = 78; no qualitative interview for one patient) indicated that 56 % (n = 44) of patients, or patient-carer dyads, inferred that BIS had had a significant impact (Level 1). A further 36 % (n = 28) suggested BIS had had some impact (that is, no major change noted, but they valued specific aspects of BIS; Level 2) and 8 % (n = 6) reported no impact (Level 3) (Table 5).

Table 5 Reported benefit of BIS for patients with non-malignant conditions
Full size table

Table 6 shows the Impact Categorisation Levels for the purposively sampled qualitative interviews (sampled for maximum diversity of change on primary outcome to t3). As in the sub-protocol for patients with cancer, given the skewed distribution of the Impact Categorisation Levels it is unsurprising that most were Levels 1 and 2, even among ‘Worseners’ on the primary outcome (Cell 3).

Table 6 Purposively sampled t3 qualitative interviews with patients with non-malignant conditions (intervention arm) and their Impact Categorisation Levels
Full size table

Qualitative analysis of the purposively sampled interviews identified the nature of the impacts of BIS and which aspects of the BIS model were valued, and also sought to identify possible mechanisms of impact. The findings of these qualitative analyses for patients with non-malignant conditions and their carers reaffirmed the findings for patients with cancer and their carers reported previously [21]. Patients with non-malignant conditions and their carers described a range of impacts including reduced fear, anxiety, worry, and feelings of panic (“with the breathlessness I wasn’t scared […] I did a lot [of activity] yesterday and still I wasn’t panicked or distressed” [158t3p]), as well as feeling more confident about breathlessness. They valued the multi-disciplinary staff expertise (their knowledge and understanding of life with breathlessness), the characteristics of the BIS staff (their approachability and attentiveness) and their reassuring and positive approach, and the time BIS gave them to talk about breathlessness with an expert (“This was somebody who knew what we were talking about, talked the same language” [042t3p]). Like the patients with cancer [21], they again reported that being seen at home was especially helpful.

Patients with non-malignant conditions and their carers identified the same BIS-delivered interventions they found therapeutic as reported previously for patients with cancer and their carers [21]. These included: providing a handheld fan and teaching patients how to use it; encouragement of exercise and goal-setting; coaching in breathing techniques, positioning, pacing and relaxation; providing occupational therapy aids; information and education — again, as reported for patients with cancer [21], learning that “being breathless won’t kill me” was particularly important to some patients and carers. Again patients with non-malignant conditions reported the benefits of advice on daily strategies to ease breathlessness, and again many referred to these as “lots of little tips”, but patients with non-malignant conditions and their carers particularly valued how the service reviewed their established practices and strategies, praised effective approaches, and provided reassurance. As with the patients with cancer [21], our explanatory analysis suggests that it was not only the provision of these interventions that was important, but also that how they were delivered was key to their impact: delivery of interventions through the provision of knowledge (why and how interventions work or specific guidance on how and when to use a particular intervention) increased patients’ and carers’ confidence and legitimised strategies that at first appeared too simple to be effective or to have much impact. Thus, as for patients with cancer and their carers, the mechanism of impact of BIS appears to relate to the acknowledgement and validation of breathlessness, and improved knowledge about breathlessness, which enhances patients’ and carers’ understanding and their confidence in living with the symptom, reducing feelings of being alone with breathlessness. Table 7 provides illustrative quotes for the gaining of knowledge and confidence, and some participant-identified interventions for patients with non-malignant conditions and their carers.

Table 7 Illustrative quotes about mechanisms of impact and valued interventions from purposive sample of patients with non-malignant conditions
Full size table

As with patients with cancer [21], reviewing transcripts for the categorisation of impact levels exercise identified BIS contacts beyond the key measurement point for the primary outcome for a substantial proportion of patients with non-malignant conditions: 39 % of patients (30/78) described planned BIS contacts beyond t3. Thus, as was noted for with patients with cancer [21], there may have been further benefit beyond the primary outcome at t3.

In addition, this review of transcripts identified the potential therapeutic effect of the research interviews on patients with non-malignant conditions and their carers as well as the difficulty some had in separating their researcher from the intervention. Nearly a quarter of patients with non-malignant conditions and/or their carers explicitly stated that talking to their researcher about breathlessness was helpful: 23 % (18/78) compared with 18 % of patients with cancer and their carers (10/54). Just over a quarter had difficulty separating their researcher from the intervention: 26 % (20/78) compared with 15 % of patients with cancer and their carers (8/54). Table 8 provides illustrative quotes for the potential therapeutic effect of the research interviews on patients with non-malignant conditions and their carers.

Table 8 Illustrative quotes on the potential therapeutic effect of research interviews for patients with non-malignant conditions and their carers
Full size table

Costs and cost effectiveness

In the two months prior to baseline the waiting-list group was more likely to have had inpatient care, and this resulted in a difference in total costs of £1,678 (Table 9). Other service use at baseline was similar between the two groups. The difference in inpatient costs was in the opposite direction during the follow-up period and total costs were £712 higher for the fast-track group (Table 10). The difference after adjusting for baseline was £799, and this was not statistically significant (95 % CI, -£237 to £1904). Two of the six cases with inpatient stays in the fast-track group had stays substantially greater than for other admitted patients across both trial arms (n = 4 other admitted patients in each arm); further investigation of these two admissions defined them as unrelated to BIS. Excluding these two cases results in costs remaining higher in the fast-track group by £100.

Table 9 Service use and costs (2011/12 £s) in two-month period prior to baseline assessment for patients with non-malignant conditions
Full size table
Table 10 Service use and costs (2011/12 £s) in four-week follow-up period between baseline (t1) and t3 for patients with non-malignant conditions
Full size table

The EQ-5D utility scores for the fast-track group were 0.49, 0.58 and 0.59 at baseline, two-week follow-up and four-week follow-up respectively. The figures for the waiting-list group were 0.55, 0.58 and 0.54. The maximum QALY gain over the four-week follow-up period was 0.077. The QALY gain for the fast-track group was 0.0431, while for the waiting-list group it was 0.0430, indicating virtually no difference. This was mainly due to the lower baseline utility score (which is used in the QALY calculations) for the fast-track group. After controlling for baseline EQ-5D scores, it was shown that the fast-track group gained 0.003 extra QALYs (95 % CI, –0.001 to 0.007). The incremental cost-effectiveness ratio indicated that intervention resulted in a cost per QALY of £266,333, which is substantially above the threshold used by NICE (£20,000–£30,000). If the cost difference of £100 (excluding the intervention patients with extreme inpatient costs) is used, the cost per QALY is much less at £33,333 but still above the threshold. The cost-effectiveness plane revealed that there was an 86.5 % likelihood that the intervention had higher costs and resulted in more QALYs. There was only around a 7 % likelihood of lower costs and more QALYs. However, when the two outliers were removed, there was a 54.7 % likelihood of higher costs and more QALYs and a 33.9 % likelihood of lower costs and more QALYs.

Discussion

In this Phase III RCT of BIS for patients with non-malignant conditions there was the same positive direction of quantitative effect as reported previously for patients with cancer, but the effect was smaller and was not statistically significant; for those with cancer the service had a reasonably sized and statistically significant clinical effect [21]. The health economic findings showed that the intervention increased costs slightly; the costs per QALY were above the thresholds used by NICE. A remarkably similar proportion of patients with non-malignant disease and their carers reported qualitatively that BIS had made a positive difference: 92 % of patients with non-malignant disease and their carers compared with 94 % of those with cancer [21]. For patients with non-malignant disease and their carers, these impacts, the identifiable interventions they found helpful, and the mechanisms of impact were the same as those identified by patients with cancer and their carers. In addition, patients with non-malignant conditions and their carers particularly valued praise of their established strategies and the provision of reassurance, potentially reflecting the longevity of their experience of breathlessness. Thus the result of the analysis of qualitative data from the two sub-protocols was strikingly similar: BIS had made a positive impact for the majority of patients and carers living with breathlessness. Another service using similar complex, predominantly non-pharmacological, interventions has reported a clinically and cost-effective impact in this group [33].

We have considered three areas for discussion which may illuminate reasons for the differences in our main findings between the two disease groups: (1) baseline differences between those living with non-malignant conditions compared to those with cancer; (2) the notable trend of improvement over both trial arms for patients with non-malignant conditions to the key measurement point; and (3) exploration of the difference in magnitude of the quantitative and qualitative results in this mixed method trial.

Baseline differences between non-malignant conditions and cancer

There were noteworthy clinical differences at baseline between both patients and carers on the sub-protocol for non-malignant conditions (Tables 1 and 2) and the sub-protocol for cancer (equivalent Table 1 in Farquhar et al., 2014 [21]). Although the mean Charlson Co-Morbidity Index was worse (higher) for patients with cancer, the mean Australia-modified Karnofsky Performance Scale score was worse (lower) for patients with non-malignant disease. Similarly mean modified Borg scales and NRS scores for breathlessness, CRQ scores, and both patient and carer HADS scores for anxiety and depression were consistently worse (higher) for those living with non-malignant disease. Importantly, it is worth noting that initial ‘distress due to breathlessness’ (primary outcome measure) was also higher for patients with non-malignant disease and their carers.

Brief reports of work by Chowienczyk et al. [34] and Javadzadeh et al. [35], comparing patients with advanced COPD recruited on the protocol for non-malignant conditions to those recruited on the protocol for cancer, identified statistical and clinical differences between these two disease groups in terms of their Descriptors of Breathlessness and CRQ scores, suggesting that the groups were different in terms of both their experience of, and impact of, breathlessness. Different combinations of clusters of Descriptors of Breathlessness were associated with each diagnostic group; the cluster ‘chest tightness’ was associated with cancer patients [34]. Patients with advanced COPD scored lower across all four CRQ domains than patients with advanced cancer; this was statistically significant for the dyspnoea, mastery, and emotional functioning scores (p < 0.05), and clinically significant for the latter two, suggesting poorer respiratory health-related quality of life and a potential difference in referral threshold with a higher threshold for non-malignant disease [35].

Trend of improvement across trial arms

There was a notable trend of improvement over both trial arms for patients with non-malignant conditions to the key measurement point; that is, there was a clinically significant 1.56-reduction for the control arm by t3 (Table 3). This contrasts with the findings for patients with cancer, who showed very little change by t3 for the control arm on the primary outcome (0.23-reduction) (equivalent Table 2 in Farquhar et al., 2014 [21]). Three potential explanations for this improvement in patients with non-malignant conditions whilst they were in the control condition include regression to the mean, a referral effect, or a therapeutic effect from the research interviews. Regression to the mean is a well-established phenomenon, but less has been written about potential referral effects or therapeutic effects of research interviews.

Considering a potential referral effect, our pre-clinical findings had previously highlighted the isolation experienced by patients with breathlessness and the valued but sporadic nature of existing service contacts [16]. Similarly, Gysels and Higginson have described the “invisibility of breathlessness” in COPD and the impact of this on service access [12]. The fact that a referral to a specialist service such as BIS had occurred could have affected our measured outcomes if this diagnostic group of patients and carers had a sense of anticipation of expert help.

The potential therapeutic effect of research interviews is also worth considering. Similar in effect to measurement reactivity [36] or assessment reactivity [37] (in that this would be an unintended consequence of the data collection process), a therapeutic effect could arise from research interviews where qualitative interviewing techniques provide study participants with the opportunity and time to tell their story, and in response the interviewer actively listens and responds with empathy and understanding. There are neurophysiological explanations for why empathy is helpful in the management of breathlessness [38]. This therapeutic effect was a recurring theme in our qualitative data from patients with non-malignant conditions and their carers, some of whom explicitly stated that talking to their researcher was helpful. A component of the BIS model, and a palliative care approach, is actively listening to patients and carers. Indeed one of the many aspects of the intervention valued by patients was the opportunity to talk about breathlessness to someone who actively listened, and another was having their breathlessness validated or legitimised — our research interviews did both. It is also worth noting  that all five researchers across the two sub-protocols came from either a health or psychology background (three from nursing, one from psychology, and one from physiotherapy).

There is a small but growing literature within palliative and end of life care research of the non-harmful, and even positive, effects of research interviews on study participants (for example, [39–41]); however, this literature usually informs debates on the ethics of palliative and end of life care research and relates predominantly to qualitative observational studies. Here we suggest that, whilst qualitative interviews may indeed have a positive impact on study participants, positive impact may actually have a contaminating effect on quantitative research outcomes, diluting any measurable impact of interventions. This is problematic for the design of mixed method randomised studies of interventions with supportive or psychological components and has been noted by others [42]. The collection of baseline data through postal, rather than face-to-face or verbal methods, may have reduced this effect but at the cost of not obtaining qualitative baseline data and at the risk of greater loss to follow-up.

The more frequently verbalised therapeutic effect of the research interviews in the sub-protocol for patients with non-malignant conditions may relate to the fact that this sub-protocol included more researcher contacts than the sub-protocol for patients with cancer. It may also relate to a greater need for a therapeutic alliance [43, 44] in patients with non-malignant conditions and their carers, or the greater investment in psychological support for patients with cancer as part of supportive care, such as Macmillan Cancer Support and Maggie’s Centres. Patients experiencing breathlessness in non-malignant conditions have usually lived with their condition for much longer than those with cancer. Their trajectory of declining functioning and increasing breathlessness is less steep [45, 46], and so they may develop their own practical strategies to enable life with breathlessness. Thus it may be that BIS is still effective for patients with non-malignant conditions but in a different way than for those with cancer. In cancer, where the illness experience in the breathless patient is usually shorter in duration, the slope of decline in functioning and increasing breathlessness is potentially steeper, meaning they have had less time to self-develop strategies to enable life with breathlessness: thus BIS has to teach these. The reassuring role of BIS was certainly more predominant for patients with non-malignant conditions and their carers. As one of the BIS providers said: “the needs felt different, but the malignant patients had a lot of acute anxiety about what breathlessness was and what it meant, and how it would be in the future because the worsening of their condition was coming quite quickly to them and their condition was going to worsen […] relatively soon, whereas with the non-malignant patients […] didn’t have those acute anxieties, and it was something they’d lived with for a while, and […] they wanted to know more about perhaps exercising, keeping going and being able to do the things they loved and that slow change and the coming to terms with that slow change, whereas the malignant patients it was all about the very acute anxiety and distress of dealing with this condition that was changing quite rapidly” [BIS Provider 02].

Differences in the quantitative and qualitative data

There was a difference in magnitude between the quantitative and qualitative findings of this mixed method trial, with a more positive outcome from the qualitative data. This held true across both sub-protocols, but was more notable in the non-malignant disease sub-protocol.

Wagner et al. state that the “challenges inherent in reconciling apparently conflicting findings from qualitative and quantitative approaches […] has the potential to yield benefits that emerge only through the struggle to reconcile discrepant results and may provide a sum that is greater than the individual qualitative and quantitative parts” [47]. Reviewing the BIS RCT mixed method findings through the lens of Moffatt et al.’s six strategies for exploring “discrepant” or “conflicting” findings facilitated this “struggle” [48]:

  1. i)

    Treat the methods as fundamentally different – The BIS RCT quantitative and qualitative methods shared the ultimate purpose of assessing the effectiveness of BIS (although the qualitative methods sought to do more than this and explore mechanisms of action as well), but they did so in a fundamentally different way: through different, but related, questions and approaches. The methods, and their findings, should therefore be considered complementary rather than confirmatory.

  2. ii)

    Explore the methodological rigour of each component – The BIS RCT qualitative interviews were brief, but were focused and data-rich, and were conducted with the same sample (and across the entire sample); however, we have described above the potentially therapeutic effect of these interviews. Researchers were blinded to the key measurement point for the quantitative primary outcome, and study-specific training and monitoring (all data collection, quantitative and qualitative, was audio-recorded) facilitated quantitative data collection fidelity.

  3. iii)

    Explore dataset comparability – A strength of the mixed method BIS RCT was that the two data types were collected contemporaneously, by the same researcher (caseloads were held) and from the same sample.

  4. iv)

    Collect further data and make further comparisons – The progressive nature of patients’ conditions in the BIS RCT, trial design and funding limits prevented the collection of further data, plus this was felt unnecessary given our response to the first of the strategies of Moffat et al.

  5. v)

    Explore the process of the intervention – The possible referral effect and the delivery of the intervention beyond the BIS RCT protocol was such that the full effect of the service may not have been captured at the key measurement point, as discussed above; the primary outcome may have been collected too soon.

  6. vi)

    Explore whether the outcomes of the two components match – As noted above, the quantitative and qualitative components of the BIS RCT addressed different, but related, questions; thus we would not anticipate, or desire, a perfect match. There was some congruence between the qualitative data and the outcome measure for mastery (a CRQ domain), but the qualitative data revealed some specific concepts not measured by the primary or secondary outcomes, such as confidence and knowledge. Consideration should be given as to whether the ’right’ primary outcome and/or outcome measure was used for the trial, and for the non-malignant sub-protocol in particular, and this is discussed in more detail below.

Our use of the same primary outcome measure for both sub-protocols may have been flawed given the differences between the two groups outlined earlier; however, our paper reporting on the sub-protocol for cancer also questioned the appropriateness of this primary outcome given that some patients who deteriorated on the primary outcome measure also qualitatively reported benefits from BIS [21] (a finding repeated in the sub-protocol for non-malignant conditions). The findings of the unpowered Phase II pilot trial showed the same positive trend for the same primary outcome for patients with COPD, and thus did not suggest it was inappropriate [19].

Interestingly, Moffat et al. considered that, in retrospect, a more relevant outcome for their own study would have been ’ability to cope’ — this is an outcome that, with hindsight, might have been more relevant for BIS too. Our paper reporting on the sub-protocol for cancer noted that BIS may have increased coping capacity [21]. Not only are ’coping’ and ’distress’ different conceptually, but ’coping’ can have both positive and negative connotations, whereas ’distress’ is entirely negative; similarly, ’ability to live with/manage breathlessness’ or ’feeling equipped to live with/manage breathlessness’ can be positive or negative. Another alternative would be an outcome based on ’gains’, which is entirely positive (and is meaningful to the NHS).

Consideration should be given to using multiple primary outcomes. Complex interventions have multiple components which are likely to have multiple (positive and negative) effects; it may be misleading, or naïve, to have one primary outcome measure. To get the true impact of a complex intervention we need to move, as proposed by Carr-Hill nearly 25 years ago, beyond the primary outcome measure: “It has long been recognised that outcomes of treatment are multidimensional and complex, and that, to varying degrees, any single index measure of outcome will be inadequate to capture important differences” [49].

BIS is a palliative care service, and there may be different margins of benefit, and a greater spread of benefits, in palliative care complex interventions, with a cumulative effect from the addition of several smaller (quantitative) outcome benefits. It may simply be unrealistic to expect one primary outcome measure to capture it all. An editorial by Richards skillfully applies the “amalgamation of marginal gains” concept first described by Sir Dave Brailsford, team manager of Team GB’s highly successful cycling team [50], to health care [51]. Brailsford’s approach sought marginal gains from single components of the team’s training in order to improve overall performance; as applied by Richards, these gains relate to ’simple’ nursing interventions which individually make only a marginal difference, but in total reduce discomfort and anxiety. As described earlier, patients and carers reported that BIS helpfully provided ’lots of little tips’. There is also the inherent challenge in palliative and end of life care outcome measurement that we expect deterioration [52].

Consideration should also be given to using mixed method primary outcomes. Normand has noted that “faced with the complex and multidimensional objectives of palliative care, analysis of outcomes and cost-effectiveness needs to embrace the complexity, and […] draw on a range of evidence, complex measurement tools, and a good understanding of context” [53]. As Cawley et al. suggest, qualitatively capturing patients’ actual experience of palliative care is a more accurate measure of how and what patients judge as important in their healthcare, but note the challenge lies in “how we convert the very positive experience of individuals into a language of outcome measures” [54]. In the field of breathlessness, Rocker has asked how we can capture those aspects of breathlessness that really matter to patients and their families and asks whether a numeric rating scale change in response to a therapeutic intervention is sufficient, “or does it tell only part of the story?” [55]. Thus our qualitative data (which was rich, consistent and complete) may simply have encompassed more, been more global, capturing several outcomes resulting from several component interventions delivered by BIS, which may explain the difference in magnitude between the quantitative and qualitative datasets.

If both the quantitative and qualitative findings of the sub-protocol for patients with non-malignant conditions had indicated there was no impact from BIS, or a negative impact, then this might have suggested that: referrals to BIS for patients with cancer were more appropriate; that BIS delivered different interventions to the different disease groups (due to different needs) and that these were more or less effective in the different disease groups; that differences in the service model by disease group had an impact (such as the duration of BIS); or that BIS was simply more effective for patients with cancer. However, the difference in the findings between the two disease groups was only in the quantitative data, thus the data, and the arguments considered earlier in this discussion, suggest: a greater therapeutic effect of the research interviews on patients with non-malignant disease (due either to their previously unmet needs, or a dose response to their additional interview); that the primary outcome was more appropriate for patients with cancer either due to the longer term unrelenting nature of breathlessness in non-malignant disease such that distress remains high, or that BIS delivered different (but still important) interventions to patients with non-malignant disease (due to different needs) that didn’t impact on distress; or that our reliance on a single quantitative primary outcome measure is misguided and we should seek ways to compute multiple outcomes relating to components of complex interventions whilst also placing greater emphasis on qualitative outcomes. A combination of these explanations seems likely given accumulating knowledge suggesting the benefits of BIS-type models of care for breathlessness in advanced malignant and non-malignant disease [21, 33, 56].

Conclusions

In conclusion, BIS has a positive qualitatively identified impact on patients with non-malignant conditions and their carers, and there was the same positive direction of quantitative effect as reported previously for patients with cancer, but the effect was smaller and was not statistically significant when compared to standard care using the quantitative primary outcome of patient distress due to breathlessness. BIS resulted in slightly increased service costs for patients with non-malignant conditions; to see if BIS is cost effective would require a longer follow-up. There were important differences between participants on the sub-protocol for non-malignant conditions and that for cancer in terms of their baseline characteristics and the notable trend of improvement over both trial arms for patients with non-malignant conditions to the key measurement point; they may also have experienced a greater therapeutic effect from the research interviews which diluted the impact of the intervention. Trials of palliative care complex interventions should consider using multiple, mixed method, primary outcomes and ensure protocols limit potential contaminating therapeutic effects from the research design.