Background

With an ageing population and increasing numbers of people with life-limiting illness, there is a growing demand for palliative care day services. Consequently, there is a need to measure the quality and value of these services, but there is little agreement on which aspects of care should be used to do this. Previous studies have indicated that day services may be associated with high patient satisfaction, but there is currently limited evidence demonstrating that services lead to significant improvements in symptom control or quality of life [1, 2]. Palliative day services have also been shown to differ substantially across settings, with wide variations in staffing structures, core components of care and models of delivery [3, 4]. Evaluating quality of palliative day services is essential for assessing care across diverse settings and for monitoring quality improvement approaches. Development and implementation of quality indicators for palliative day services is therefore a key priority.

Recent studies have used established methods to develop quality indicators in general palliative care [5, 6], but there is evidence to suggest that separate indicators may be required in different contexts, such as non-residential settings including day hospice, community or primary care settings [7]. Since quality of care is a multidimensional concept, its assessment can require different measures. Indicators are not a direct measure of quality but are centred on evidence-based standards of care. In this manner, indicators can be used as quantitative measures to assess care processes, structures or outcomes and to monitor, evaluate and improve the quality of important clinical and support functions, as well as governance and management factors that can influence patient outcomes [8]. Consequently, they allow for comparisons to be made over time and across settings and to set priorities and support patient choice. It is important to recognise that process and structural indicators can only be considered as valid when it can be demonstrated through high-quality evidence that they result in improved outcomes. These associations can derive from primary research, as well as reviews of existing evidence and expert consensus methods.

This planned review will inform the initial phases of the UK Consensus Project on Quality in Palliative Care Day Services, a project which aims to develop an evidence-based quality indicator set for day services using the multiphase RAND/UCLA appropriateness method [9].

Objectives

The aim of this scoping review will be to map the extent, range and nature of the existing evidence around models of delivery, care domains and quality indicators used to evaluate palliative day services. In doing so, the review will also seek to detect gaps in this evidence and identify future directions for research in the area. Specific objectives will be (1) to identify existing quality indicators relevant to palliative day services and to evaluate the quality of their development and (2) to categorise existing quality indicators by care domain and model of delivery.

Methods

A scoping review methodology will be employed as these approaches take account of a broader range of study designs and methods than systematic reviews, which typically address research questions around the effectiveness of a specific intervention. Established methodological frameworks and recommendations will be used to guide the review process [10,11,12]. The review will include the following components or phases: (1) formulation of research questions, (2) identification and selection of relevant evidence, (3) extraction of potential quality indicators from existing evidence, (4) charting and synthesis of results including development of a conceptual model and (5) engagement and consultation with key stakeholders to ensure relevance of the conceptual model to current practice and to facilitate knowledge exchange processes. In addition to the use of this methodological framework, the Preferred Reporting Items for Systematic reviews and Meta-Analyses for Protocols (PRISMA-P) checklist [13] will be followed to ensure preparation of a robust protocol (see Additional file 1).

Search strategy

A systematic search strategy will be carried out by combining key Medical Subject Heading (MeSH) terms including “Quality indicator”, “Patient-centered care”, “Quality criteria” and “Quality improvement”. The proposed MEDLINE search strategy is shown in Table 1. Key search terms will be identified though discussion between project team members and with other experts and stakeholders in the area. The fully developed systematic search strategy will be peer reviewed using the Peer Review of Electronic Search Strategies (PRESS) tool [14]. Five electronic databases (Ovid MEDLINE via Ovid, EMBASE via Ovid, CINAHL via EBSCO, PsycINFO and the Cochrane Central Register of Controlled Trials) will be searched from January 2000 to the present day. Only evidence published since 2000 will be included to ensure relevance to current palliative care settings and day service delivery models. No language restrictions will be applied during searches. Hand searches of the reference lists from included articles and relevant systematic reviews will also be carried out. The search strategy will also use previous recommendations, which are based on a feasible and robust method of applying a systematic search to identify grey literature [15]. Grey literature searches will be conducted using http://www.opengrey.eu/ and Google Scholar to identify projects in progress, practice guidelines and policy documents or reports not indexed in the electronic databases. Google Trends (https://www.google.com/trends/), a publically accessible resource providing data on geospatial and temporal patterns for user-specified terms, will be used to identify closely matching terms commonly used to locate information around quality of care. Given the broad scope of the review, only the first 50 returns from the website searches will be extracted. The websites of ten key palliative care and general healthcare organisations will also be searched.

Table 1 Proposed MEDLINE search strategy developed using Medical Subject Headings (MeSH)
Full size table

Study screening and inclusion

Two researchers (SOC, MD) will independently conduct the database searches and import all potential evidence sources into an EndNote referencing software database (EndNote X7.3, Thomson Reuters, Ontario, Canada). The same researchers will then screen the title and abstract of records identified during the searches to assess relevance of the content in comparison with the above eligibility criteria. Non-relevant titles will be removed at this stage. Full-text copies of potentially relevant articles will be obtained and reviewed to determine inclusion using the same criteria. Agreement between reviewers at this stage will be determined using the κ statistic with an accepted cut-off point of above 0.60 used to indicate adequate agreement [16]. A third reviewer will be used to resolve any disagreement or uncertainty around final inclusion (NMcC). All evidence excluded during screening will be recorded, along with the reason for exclusion. Reviewers will not be masked to author name or publication type during the screening process.

Study eligibility criteria

To broaden the scope of the review and reflect the diverse nature of day services, the review will include all evidence identified which involved the development or evaluation of quality indicators to assess palliative care within any non-residential setting, including day hospice and community or primary care settings. The review will focus on quality indicators for adults receiving care for any life-limiting conditions including cancer or uncontrolled symptoms due to cancer treatment, heart failure, pulmonary disease, dementia, end stage liver or renal disease and neurologic conditions including multiple sclerosis, motor neurone disease or Parkinson’s disease. We will not include indicators developed to assess care in patients undergoing surgical procedures, as measures of quality and care pathways are distinct in this group of patients. The following types of evidence will be considered: consensus development methods; randomised or quasi-randomised controlled trials; mixed methods; prospective, longitudinal or retrospective case-control studies. Clinical practice guidelines, consensus statements, systematic reviews and relevant reports or policy documents will also be included. Where indicators are identified from these latter sources, the review will attempt to ascertain the original development research. All included evidence will be required to identify or propose at least one potential quality indicator related to structures, processes or outcomes at the individual patient or service level of care. Quality indicators will be defined as any measure, able to be expressed as a fraction, which compares actual care against an ideal criterion [17, 18]. The review will therefore include not only evidence directly identifying candidate indicators but also evidence that discusses the outcome or event of interest without proposing the event as a quality indicator.

Data extraction

Two researchers (SOC, MD) will independently use a standard data abstraction form to chart the following information from the included evidence: author and date, country, document type (research paper, guideline, policy document etc.), target population/setting, indicator sets (domains), methods used in development, proposed methods of measurement and evidence of testing or implementation. Specific information on numerator and denominator descriptions, definitions and type of quality indicators will also be extracted and classified as structural, process or outcome indicators using the Donabedian Framework [19]. According to this model, structural indicators include factors that affect the context in which care is delivered (i.e. organisational characteristics such as availability of equipment and services or staffing). Process indicators relate to the delivery of care (i.e. assessment and interventions), while outcome indicators are concerned with the effectiveness of care in terms of change in patient important outcomes (i.e. health status, satisfaction with care and overall quality of life). Indicators removed at each phase will also be verified by all authors.

The data abstraction form will be piloted using a sample of ten non-relevant quality indicator articles in order to standardise the abstraction process and make any required modifications. The quality of each indicator will be assessed by all reviewers using the Appraisal of Indicators through Research and Evaluation (AIRE) tool, which uses a combined score based on 20 items across four domains: purpose, relevance and organisational context; stakeholder involvement; scientific evidence, and additional evidence, formulation and usage [20]. Each item will be rated on a 4-point Likert scale ranging from “1” (strongly disagree: confident that the criterion has not been fulfilled or no information was available) to “4” (strongly agree: confident that the criterion has been fulfilled). The mean score, reported as a percentage, will be determined for each domain, as well as for the overall score assigned. Indicators will be considered to be of good quality if they score 50% or more in all four domains. The quality of overall evidence will be assessed initially by two reviewers (SOC, MD) using the Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) system [21]. This system appraises and summarises the quality and strength of recommendations across important outcomes. Overall evidence for each outcome will be rated as very low, low, moderate or high quality, based on study design and limitations, consistency of effect and the directness or generalisability of evidence [22]. Final decisions on methodological quality will be reached through consensus between review authors. While appraisal of methodological quality is not considered as essential during conduct of a scoping review [23], the assessments planned as part of this review will provide valuable information for the evidence summary tables that will be produced for the initial phases of the RAND/UCLA appropriateness method [9].

All quality indicators extracted from the evidence sources, including their original descriptions, will be combined in a single excel file (Microsoft, Washington, USA). This list will then be scrutinised and non-relevant or duplicate indicators will be removed. The original and revised file will be reviewed and cross checked by a second reviewer, with any disagreement or uncertainty resolved via discussion with a third reviewer. The final list of quality indicators will be agreed by consensus among the reviewers. The indicators will then be grouped according to their proposed quality domain. These domains will be determined based on those used in previous quality indicator development projects identified during the review and following discussion with the review team (see Tables 2 and 3).

Table 2 Proposed quality domains for the project
Full size table
Table 3 Quality domains used in key clinical guidelines, quality indicator development projects and systematic reviews
Full size table

Data synthesis

Due to the predicted breadth and heterogeneity of the identified evidence, meta-analyses and statistical methods of synthesis are not planned as part of the review. Quantitative results will be reported using numerical counts, including year of publication, country, evidence type and the number of indicators in each care domain. A qualitative approach will be used to describe indicator characteristics in terms of their method of reporting, evidence of reliability testing and the methodological quality of their development. All data processing, including calculation of the κ statistic to determine reliability of the search strategy, will be handled using SPSS (Version 21.0). Extracted information will be used to develop plain language evidence summary tables, with quantitative data displayed in graphical form. These will be used to guide the expert rating phase using the RAND/UCLA appropriateness method [9]. To further aid the expert rating process, a concise and standardised list of indicator definitions will be developed by using a priori criteria to standardise the wording of the indicators in the evidence summary tables. For example, where indicator definitions include any IF and THEN statements, or reference to a specified condition, these will be removed. Any mention of exact timing (e.g. how many days before an intervention is started after referral) will also be removed as it is expected that this will differ for day services in comparison to other care settings. Since it is anticipated that some indicators will be generally comparable but may be measured in a number of ways or used in different care settings, the extracted information will be also used to develop a conceptual model, based on the Donabedian framework [18], and a modified version of the framework used in the OECD Health Care Quality Indicators Project [24]. The model will summarise the impact of different aspects of quality in a palliative day care context and will help to identify areas of discussion among the expert panel during the consensus meeting phase of the RAND/UCLA appropriateness process [9]. This model will also be used to identify gaps in the evidence or areas where indicators are not currently available to evaluate existing services. Particular focus will be placed on the context in which certain indicators have been developed or implemented to ensure relevance to current practice (see Additional file 2).

Discussion

Identifying, developing and implementing evidence-based quality indicators is critical to the evaluation and continued improvement of palliative care. The planned scoping review will form part of a larger research process aimed at developing capacity in the evaluation and implementation of quality indicators in palliative day services. The review is intended to reflect and effectively summarise the diverse nature of palliative care delivered in day hospice, community or primary care settings. It is anticipated that a substantial number of indicators, distributed across different areas and models of care will be identified. Key stakeholders involved in the overall project (including hospice and day service leads, medical and nursing staff, healthcare professionals (physiotherapists, occupational therapists, complementary therapists, psychologists), volunteer staff and service users) will also be included in the summarising and reporting phase of the research process. Feedback provided by relevant stakeholders at this stage will be used to ensure relevance to current practice and to promote knowledge exchange and translation processes. The review findings will be used to identify gaps in the current evidence and to inform a planned systematic review. Findings will also be used to assist clinicians and policymakers ascertain whether existing indicators are appropriate for evaluating palliative day services at the patient and service level and to categorise how indicator measurement and quality improvement approaches fit within existing models of care.