Background

Calls to ensure the ‘patient and family voice’ is encapsulated into the measurement of quality of care for the dying is well-established [1]. Hence, on a global basis, bereaved relatives’ evaluations (both using surveys and telephone interviews) form a key part of the evaluation of end-of-life care, especially within North America, Japan, and parts of Europe [2,3,4,5,6]. To ensure the highest level of care provision, it is important to be able to robustly evaluate the quality of current care [7].

This evaluation is especially pertinent within the United Kingdom (UK), as within recent years, care for dying patients has featured heavily within public and professional forums. Additionally, significant changes have occurred to national guidance underpinning the way that care should be provided [8, 9]. The Liverpool Care Pathway for the Dying Patient (LCP) [10], an integrated care pathway aiming to improve quality of care in the last days of life, came under intense media scrutiny. The LCP was a nationally endorsed document and used internationally to support the provision of care when it was recognised that an individual may be in the last days of life. A subsequent review of care for the dying advised that the LCP should be phased out by July 2014 [8]. Forty-four key recommendations were provided within the report, including the need for more individualised care and improving skills and competencies for clinical staff caring for dying patients [8]. Following this, the ‘One chance to get it right’ report identified five ‘priorities for care’ and highlighted that clear, sensitive, and timely communication is fundamental to ensure good quality of care is provided to dying patients and their families [9]. A recommendation for ‘individualised end-of-life care plans’ was made and the recent publication of the NICE guidelines for End-of-life care helped provide a framework for best clinical practice [11].

The most recent National UK ‘End of Life Care Audit – Dying in Hospital’ based on data collected in 2015, demonstrated improvements compared with the previous audit in 2013: a higher proportion of patients were recognised to be dying in a timely manner and for 95% of these, there was a documented discussion about this recognition with those identified as important to the patient [12]. There was, however, a reduction in the rates of anticipatory prescribing for symptoms commonly seen in the last days of life. While this could be perceived as positive and in keeping with NICE guidelines about individualised prescribing, a regional survey of first year doctors reported one of their main needs was for formal guidance with symptom control. These doctors reported difficulties with remembering doses of anticipatory medication [13]. Importantly, the 2015 National audit also highlighted variability in the results between individual hospitals and the continued limited availability of 24/7 palliative care services [12].

In view of these major changes in end-of-life healthcare policy, a quality assurance project was undertaken within a specific English region to assess current care provided to dying patients and their families in three healthcare settings: hospital, hospice, and community. In order to focus on care in the last days of life and immediate post-bereavement period, we used the ‘Care Of the Dying Evaluation’ (CODE™) questionnaire [14] as our post-bereavement tool. CODE™ seeks perceptions about quality of patient care and level of family support and contains sections on symptom control; nursing and medical care; communication; provision of fluids; place of death; and emotional and spiritual support. It is a shortened version of the original questionnaire, ‘Evaluating Care and Health Outcomes – for the Dying (ECHO-D), which was developed, validated and used within a hospice and hospital setting [15,16,17]. CODE™ has also been assessed for validity and reliability [14], has been used nationally within the National Care of the Dying Audit – Hospitals [18], and is currently the focus of an international project involving seven European and Latin American counties [19].

Undertaking robust research for those who are dying is challenging due to the sensitive and emotive area of enquiry. Ethical, moral and practical challenges exist, influencing recruitment, retention, difficulty in identifying suitable outcomes measures and the level of investment in research [20,21,22,23,24]. With this in mind and key for the context of this quality assurance project, it is important to identify a distinction between measurement for judgement and measurement for improvement. It has been argued that in order to facilitate the process of service improvement “we need just enough information to take a next step in learning” [25]. Benchmarking methodology offers a way in which to generate ‘just enough information’, through facilitating assessment, comparison and reflection of ‘relevant’ information on care delivery, to identify both gaps in performance and examples of best practice [26, 27]. This methodology promotes a collaborative rather than competitive approach to assessment focussing on sharing information, which is integral to continually improving the quality of care [27]. This ethos underpins the methodology of this reported quality assurance project.

Aims

Within a specific English region, the primary aim was to engage with clinical services across hospital, hospice and community settings to explore the current quality of care provided to dying patients and their families, from the perspective of bereaved relatives. In order to achieve this aim, we undertook a cross-sectional regional survey of bereaved relatives’ using the CODE™ questionnaire. The project was known as the Regional CODE™ survey and the conduct and results from this survey represents the main focus of this manuscript.

As a secondary aim, we wanted to explore views on project participation as perceived by representatives from individual organisations. To achieve this aim, one-to-one evaluation interviews with organisational representatives were undertaken. Within the context of this manuscript, we will focus on summarising key feedback about the levers and barriers to project participation.

Methods

Cross-sectional survey of bereaved relatives’ views

Participating organisations compiled a consecutive sample of the patients’ next-of-kin (NOK) according to eligibility criteria (Table 1) [28]. Organisations posted CODE™ information packs to the NOK three months following the patient’s death. The information pack included a:

  1. 1.

    covering letter

  2. 2.

    CODE™ questionnaire with unique identifier

  3. 3.

    freepost envelope for returning the questionnaire

  4. 4.

    information about accessing the web-based tool if on-line completion preferred.

Table 1 Eligibility criteria for Regional CODE™ survey
Full size table

To remain ethically sensitive and, as this was the first time a survey of this nature had been conducted regionally, no reminder letters were sent.

Evaluation interviews with named organisational representatives

A purposive sample of ‘named organisational representatives’ were approached to represent service leads from all three care settings (hospice, hospital and community). A letter of invitation and a Participant Information Sheet was posted to potential participants. Following written informed consent, those willing to participate, undertook a one-to-one semi-structured interview conducted by one of three experienced researchers (MG, HM, TM). Using an evaluation interview, where a ‘narrative’, conversational approach [29, 30] was employed, rather than a researcher developed evaluation form with a set of pre-determined questions and response options, was believed to promote a more participant led assessment. At the same time, it offered the opportunity for clarification and elaboration of important elements of feedback to aid understanding. These conversations were largely focused into: the current process for gaining and dealing with patient and relative feedback and complaints; reasons for participating in the project; general perspectives on taking part in the project (positives and negatives). Interviews were conducted prior to publication of the overall final report. However, an automated report of the CODE™ survey results for individual organisations was available to download via the data entry tool. This provided an opportunity for all participants to review the results for their individual organisation(s) and begin the process of action planning ahead of the final report.

An interview ‘topic guide’ was used to encourage the conversational ‘flow’ if this was not naturally occurring. Some specific areas covered were:

  • Role of the interviewees

  • Existing bereaved relative feedback processes within organisation (e.g. including the management and feedback of these user-views)

  • Perceived levers and barriers to project participation (e.g. team format, operational processes, anticipated barriers and overcoming these).

Depending on preferences, interviews were conducted either face-to-face or via the telephone.

Analysis

Bereaved relatives’ survey

CODE ™ data was analysed using descriptive statistics (number, percentage) for each individual question. Median (M) and Inter Quartile Ranges (IQR) were used where appropriate. All missing data is presented within the tables (but excluded from the descriptive analysis). In terms of project feedback, each participating organisation was given their own report detailing results from their bereaved relatives’ survey. Subsequently, they were also given a report detailing how their own results compared with other organisations within the region. Within this manuscript, we provide an example of how individual organisations’ results for the key outcomes could be compared with regional and national results.

Evaluation interviews

Each interview was audio-recorded, transcribed and analysed with a thematic approach using the “substance of the interview” [31] to formulate overarching themes and categories. One of the researchers (MG) read, and re-read the data from all 9 interviews, recorded initial impressions and developed thematic codes. Further discussion of codes was conducted with the wider team (TM, CM) to reach overall agreement on substantive categories representing the data as a whole. For the purposes of this manuscript, the codes relating to levers and barriers to project participation form the main focus.

Results

Response rate

From 30 eligible organisations, 19 initially agreed to participate (7 hospitals; 7 hospices; 5 community settings) although one community organisation subsequently was not able to participate due to resource issues. Of those who declined, the main reason was because they were already undertaking a bereaved relatives survey (n = 8); other reasons being ‘too few deaths’ within the inclusion period (n = 2) and one organisation simply reported that they were unable to participate on this occasion.

From 3402 deaths, just over half (52%, n = 1774) met the initial inclusion criteria. Approximately a quarter of these (n = 491, 28%) could not subsequently be included, with the main reason due to inaccurate NOK data (Fig. 1). From 1283 CODE™ questionnaire packs sent out, 354 returned completed questionnaires (28% response rate). The hospice setting had the largest response rate (82/225, 36%) compared with other settings (hospital 218/849, 26%; community 54/209, 26%).

Fig. 1
figure 1

Flow chart – Response rate within the Regional CODE™ survey

Full size image

Demographics (Table 2)

Overall, deceased patients were evenly split in terms of gender (n = 170 males, 52%) and, except for the hospice, tended to be from an older age group (n = 259, 70 years or older, 77%). More hospice (n = 74, 90%) and community (n = 44, 81%) patients had a cancer diagnosis compared with the hospital (n = 70, 32%).

Table 2 Demographic data for deceased patients and bereaved relatives within the Regional CODE™ surveya
Full size table

Participating bereaved relatives tended to be female (n = 225, 67%) and aged between 40 and 69 years (n = 196, 59%). Participants tended to be the spouse or partner to the patient (n = 160, 59%), with the exception of the hospital setting, where participants tended to be the ‘son / daughter’ (n = 87, 42%) completing the questionnaire. The majority of relatives and patients in this sample were ‘White British’ and of a ‘Christian’ religious affiliation.

CODE™ questionnaire results

Key outcomes (Table 3)

Overall, about three-quarters of respondents reported the patient was ‘always’ treated with dignity and respect by both doctors and nurses (doctors: n = 245, 72%; nurses: n = 257, 75%). A notable minority, primarily from the hospital setting, reported the patient was ‘never’ treated with dignity and respect by doctors (n = 12, 6%) and nurses (n = 9, 4%). The majority of bereaved relatives (n = 272, 82%) reported they were adequately supported in the last days of life.

Table 3 Key outcomes as perceived by bereaved relatives within the Regional CODE™ surveya
Full size table

Symptom control (Table 4)

The most commonly reported symptom was ‘restlessness’ with 225 respondents (65%) perceiving their family member appeared restless ‘some’ or ‘all of the time’. A small but notable minority perceived their family member had the following symptoms ‘all of the time’ in the last days of life (‘pain’ n = 37, 11%; ‘restless’ n = 51, 15%; ‘noisy rattle’ n = 57, 17%). Pain and restlessness being present ‘all of the time’ was most commonly reported by hospital respondents and ‘noisy rattle’ by hospice respondents.

Table 4 Symptom control and communication as reported by bereaved relatives within the Regional CODE™ surveya
Full size table

For those reporting that their family member had experienced pain, almost three quarters responded that enough had been done by the healthcare team to control this symptom (n = 214, 71%). This proportion reduced, however, to around half for restlessness and noisy rattle (n = 139, 53% and n = 90, 48% respectively). For all three symptoms, respondents perceived that symptom control was best optimised by the hospice healthcare team and least likely by the hospital healthcare team.

Communication (Table 4)

Overall, the majority of respondents perceived that nurses (n = 268, 78%) and doctors (n = 274, 81%) had time to listen and discuss the patient’s condition with them (answered ‘agree’ or ‘strongly agree’). Around one-fifth of hospital respondents either disagreed or strongly disagreed to the statement about nurses having time to listen (n = 43, 20%) and a similar proportion about doctors (n = 37, 18%).

In terms of detecting areas of unmet need, discussions about the appropriateness of giving clinically assisted hydration (CAH) were not routinely undertaken and occurred least frequently within the community setting. Overall, for those who hadn’t had a discussion about CAH (n = 245), 124 (51%) would have found these types of discussion helpful.

Around half of all respondents (n = 176, 51%) were told what to expect when their family member was dying. Of those who weren’t told (n = 187), just over three quarters (n = 144, 77%) perceived that these types of conversation would have been helpful.

Cross-comparison analysis of results between organisations

Each Clinical Commissioning Group (CCG) (the organisation responsible for commissioning health services for a particular area within the region) was provided with a report of the results for its responsible provider organisations. An example of the CODE™ results for the key outcome questions for two hospitals (within a single CCG) is shown in Fig. 2.

Fig. 2
figure 2

Comparison of hospital results (within one CCG) with National Care of the Dying Audit Hospital (NCDAH) and overall Regional CODETM results

Full size image

Responses were also compared with CODE™ results from the National Care of the Dying Audit – Hospitals (NCDAH) 2013/2014 and with collated results from all participating hospitals within this Regional CODE™ survey. Comparisons suggest that perceptions about quality of care within Hospital ‘A’ tended to be higher compared with Hospital ‘B’, with NCDAH data and with collated regional results. Each organisation was then tasked with using the CODE™ results to develop action plans to help further develop clinical services.

Views of representatives from participating organisations

Nine individual interviews were conducted representing views across 11 services (4 hospices, 4 hospitals and 2 community settings) as one representative came from an integrated service (combined hospice / hospital /community). The interviewees’ roles were: Consultant in Palliative Medicine (n = 2); Clinical Nurse Specialist & Team Leader (n = 2); Clinical Services Manager (n = 2); Outreach Services Manager (n = 1), and Senior Manager (for End-of-life care / evaluation and quality) (n = 2). Two interviews were conducted face to face with the remainder by telephone. Interviews lasted between 35 and 60 min.

The main themes identified as ‘levers’ to present and future project participation were: ‘lack of existing feedback processes’ to obtain user-representation views about the service; ‘clear operational processes’, including a user-friendly electronic tool, written guidance and a telephone helpline; ‘significant user-feedback and opportunity to use results in a meaningful way’ which would help inform future care provision. The main theme relating to ‘barriers’ was the ‘fear of causing distress to bereaved relatives’ (which was subsequently not found to be the case in this project). Additional themes were: current ‘organisational systems not set up to capture information’; lack of organisational ‘buy in’ and lack of administrative support (Table 5).

Table 5 Summary of interviewees feedback about ‘Regional CODE’ – levers and barriers for participation
Full size table

Discussion

Overall, this was the first bereaved relatives’ survey spanning across three healthcare settings undertaken within this specific English region. The ability to engage with 18 (60%) of all organisations, and the fact that the main reason for non-participation was due to current participation in bereaved relatives’ surveys, was testament to the success of this quality assurance initiative. This model of providing quality assurance is one which could be replicated within other regions and across countries as a whole. Providing opportunities for user feedback towards services is a key health care priority both nationally and internationally. Within the UK, this commitment is outlined within the NHS constitution [32] and NHS Mandate [33]. Improving support to and engagement with bereaved relatives is also a key objective within the Next Steps On The NHS Five Year Forward View [34]. Results from this project confirm that the majority of these regional organisations did not have formal mechanisms to capture bereaved relatives’ views. CODE™ is a valid and reliable tool which could bridge this clinical need and support organisations to obtain feedback of this nature. Interviewees reported that the data generated was perceived to provide valuable local information to better understand experiences of care in the last days of life. Additionally, the opportunity to subsequently benchmark their results with others regionally was seen as a positive outcome.

Generally, the majority of bereaved relatives reported good perceptions of care, although a small but significant minority reported poor experiences. Those who experienced care within the acute hospital settings were more likely to report issues in terms of the care received or the level of family support provided. Motivations for organisations’ participation could have been influenced by the expectation that bereaved relatives’ perceptions were likely to be favourable. Responses, however, demonstrated variability in the care provided, implying that the motivation was to genuinely seek user-feedback about their service. Although concerns were raised by interviewees about potential distress to bereaved relatives, in reality there were few complaints from bereaved relatives who participated in this project. This finding would reinforce previous research suggesting that bereaved relatives are keen to be given the opportunity to engage in research due to a number of different motivations [35, 36].

The project was not without limitations. There was a high proportion of missing or inaccurate NOK details and, as it was imperative the CODE™ questionnaire was sent to the most appropriate individual, this limited the total number of potential participants. Additionally, this may have led to selection bias as, within some organisations, those who were engaged with Specialist Palliative Care services were more likely to have their details documented accurately. The wider implications of inaccurate details include the potential impact on providing bereavement support if NOK can’t be accurately contacted. By being involved with the quality assurance project, however, this helped highlight to individual organisations some of the limitations of their internal reporting mechanisms and facilitated the potential development of solutions in these organisations.

Overall the response rate of returned and completed CODE ™ questionnaires (27%) was lower than expected (estimated to be 35–40%). It was also lower compared with previous studies of this nature including those which have used CODE™ [14, 18]. One reason for the lower response rate was the decision at the project outset not to use reminder letters, a method recognised to increase response rates [23]. Ongoing developments for the CODE™ questionnaire include further Public and Patient engagement to ascertain the best methods to approach and recruit potential participants for these types of surveys [19].

We did not obtain the demographic details of non-responders. This would have enabled a comparison between respondents and non-responders and provided information about the representativeness of our sample compared with the population as a whole. This is especially pertinent due to the fact that most participants were of white ethnic origin and stated they had a ‘Christian’ religious affiliation. The key element of the project, however, was to assess whether it was possible to engage with clinical services and hence a balance needed to be met between making it sufficiently feasible for individual organisations to participate versus the ideal methodology for conducting a survey of this nature.

Overall, perceptions about hospital care were poorer compared with hospice and community settings especially in terms of symptom control and communication. These areas are recognised themes from patients and family carer feedback [37] although for pain control, findings from previous studies have varied. One recent study demonstrated that those who died in hospital were more likely to have experienced pain compared with those who died at home [38] but others have not supported this finding [39]. It is noteworthy, however, that the deceased patient populations differed between care settings, with hospitals having a greater prevalence of non-cancer patients. In the most recent UK National VOICES survey, findings also suggested that pain control was potentially better met for those with a cancer diagnosis compared with those dying from non-malignant disease [5].

The greatest proportion of discussions about fluids at the end-of-life was reported by the hospital participants and the lowest within the community setting. This may relate to the fact that it is more challenging to provide clinically assisted hydration (CAH) in home or care home facilities. And in turn, healthcare professionals may not have broached this issue due to these types of concerns. The greatest degree of unmet information need about CAH was in the hospital setting, although across all settings, participants would have appreciated discussions. This may be in part due to the generally positive views of CAH by family members [40, 41] and the symbolic meaning that it brings [42]. It is important to understand that making ‘blanket’ assumptions about the appropriateness of engaging in such discussions cannot be supported, and sensitive communication, using open screening questions is likely to be required to promote the provision of individualised and responsive care [43].

CODE™ specifically focuses on care of patients in the last days of life and family support, and this is the first time it has been used to assess the quality of care in a number of different healthcare settings (hospice, hospital and community). Due to the methods of questionnaire administration, the survey allows each organisation to have individual and personalised feedback about bereaved relatives’ perceptions of care. Previous national surveys of this nature have tended to feedback at a regional or CCG area level [5]. Additionally, the project subsequently enabled individual organisations to ‘benchmark’ themselves with findings from other similar organisations which offers the opportunity to facilitate cross-site learning. Although not conducted within the scope of this project, future follow-up interviews with the representatives from participating organisations would be beneficial to establish the key actions undertaken to help improve their clinical services. Additionally, future developments may include the opportunity to repeat the post-bereavement survey to assess the impact of changes over time. By assessing the feasibility that an evaluation such as this was possible within this region, a future post-bereavement evaluation would offer the opportunity to undertake more robust survey methodology to help further address any confounding factors.

Some of the real strengths of this project include the ability for individual organisations to:

  • demonstrate active engagement with bereaved relatives and have user-views about their service; this is particularly pertinent to provide evidence for the regulator organisations such as the ‘Care Quality Commission’ within the UK

  • have a quick ‘at a glance’ report to highlight the strengths of the organisation and areas that require further development

  • compare their own findings with other similar organisations as a whole and with others within their locality; additionally, for hospitals being able to ‘benchmark’ their data with previous national findings

  • utilise the CODE™ findings to create action plans to enact service improvement initiatives as well as facilitate learning from other organisations [26, 27].

Conclusions

With the national recommendations to use individualised care plans for dying patients, the optimal way to evaluate the impact of these and provide assurance that care is being delivered in a timely and sensitive manner should be determined. Post-bereavement questionnaires are recognised to be a useful way to do this and CODE™ represents a valid and reliable outcome measure to use. The successful engagement with the project is testament to the commitment of staff and organisations with the clear desire to seek user-feedback to improve care. The model of evaluation and the use of a ‘benchmarking’ approach, could be used at a local, national and international context to help drive an ongoing continuous quality improvement programme to improve care for dying patients.