Introduction

Palliative care (PC) is defined by the World Health Organization (WHO) as an approach that improves the quality of life of patients and their families facing the problems associated with life-threatening illnesses [1]. Traditionally, palliative care was labeled only for patients at the terminal stages [2] although it became evident that those patients may encounter physical psychological, and spiritual problems early in the disease course. These problems become clearer with the aging of the population, so more patients with incurable life-threatening illnesses and limited survival are in critical need of palliative care [3].

Recently, emerging evidence has shown significant benefits from the early introduction of palliative care in the management of this group of patients. These benefits included significantly better quality of life, less symptom burden [4], clear discharge and follow-up plan, active communication with the patients and families about diagnosis and prognosis [5,6,7], and a higher degree of satisfaction [8, 9] as well as the less aggressive end of life interventions [10, 11].

Since 2012, the National Comprehensive Cancer Care Network’s palliative care guidelines recommend screening of all patients for palliative care issues during initial oncology consultation and at clinically relevant times [12]. Thus WHO [1], the American Society of Clinical Oncology (ASCO), and the European Society for Medical Oncology (ESMO) have recently recommended early integration of oncology and palliative care [13,14,15] in inpatient and outpatient care together with anticancer treatment.

Recently, in Kuwait, the introduction of u (SPC) service was done through the standalone Palliative Care Center (PCC) in 2011but it is used to care for patients at the end of life. In September 2016, the palliative medicine department in PCC started to look after the symptoms of cancer patients in medical oncology wards at Kuwait Cancer Control Center (KCCC). Criteria for palliative care referral were settled and updated regularly (Appendix 1). Since then, the oncologists in KCCC have referred their patients early to the specialized palliative care (SPC) team for symptom control either as an inpatient or outpatient consultation. So, the PCO integration between SPC as an embedded service in KCCC and the oncology department was started. Measurement of the impact of this implementation on structure, process, different available models per center, and efficient function of this implementation was needed. Till now, to the best of our knowledge, no available tool to assess and measure these outcomes for further improvement of the PCO service.

Objectives

  1. A.

    Primary objectives.

  1. 1.

    To create a reliable and validated tool to measure the impact of early integration of palliative care and oncology (PCO).

  2. 2.

    To apply this validated and reliable tool on the oncology staff for further validation.

  1. B.

    Secondary Objectives.

  1. 1.

    To view the opinion of medical oncologists and nurses about the structure of clinical practice and the process of care of the current integrated PCO program.

  2. 2.

    To view and compare the opinions of medical oncologists and nurses about the impact of early PCO on symptom management, communication with the patient and family, and end-of-life care before and after early integration.

  3. 3.

    To determine the impact of early referral to implementation of PCO on work burden, attitude, and satisfaction of medical oncologists and nurses.

Patients and methods

Study design and setting

A novel semi-structured questionnaire was created through a multi-stage process of survey development to measure the opinion of medical oncologists and nurses about the impact of the integration of PCO. Measurement of the implementation of PCO integration regarding the structure, process, different available models, and efficient function aimed to ensure better symptom management, good communication with patient and family, end-of-life care, and how it would help to decrease the work burden, improved attitude and increases the satisfaction of the medical oncologists and nurses.

After the development and validation of the questionnaire, it was distributed in KCCC. All medical oncologists and nurses in the medical oncology department who had at least two years of experience working at the KCCC were invited to participate. From them at least one year before and one year after the start of the integration to ensure that all participants attended and witnessed the differences.

Pilot study: development and evaluation of the preliminary questionnaire

Instrument design

Based on the current literature, unfortunately, there are limited validated tools to measure the actual clinical impact of early integration of specialist palliative care (SPC) in a tertiary cancer center. Determination of content domains was obtained from the literature review [8, 15,16,17,18,19,20,21,22,23]. Interview with the respondents and focus groups (Target population) and expert panel opinion. This aimed to give a clear image of the boundaries of the dimension components. Qualitative data collection from the above resources was done and then returned to the research question to ensure items were relevant to the question [24].

The focus group had four oncologists and nine nurses (target population) and the expert panel in the academic and clinical field of palliative care and oncology co-coordinators guided throughout the entire development process of the questionnaire. The membership of the expert panel included five consultants from palliative care, two medical oncology consultants, the head nurses of palliative medicine and oncology departments, and two professionals in psychometric analysis. The main clinical aspects needed to assess the impact of palliative care on oncology (PCO) were divided into seven main domains. The first main section was about the current structure of clinical practice and the process of care while other sections were about clinical domains such as a comparison of differences in symptom management, communication, and end-of-life care before and after at least one year of early integration of PCO and in turn how the improvement of these aspects affect work burden, the attitude, and satisfaction of oncology staff together with one section about the current structure of clinical practice and the process of care.

Judgment

After a series of meetings and discussions with an expert panel and focus group, the items were refined to a pool of approximately 291 items covering all dimensions and organized in a suitable format and sequence in a usable form. Most of the items were identified and measured through a points Likert scale [25, 26]. In many studies, five points scale is comprehensible and helps to quantify subjective preferential opinion, attitude, thinking, and feeling accurately and in a scientifically accepted, validated, and reliable manner [26, 27]. Participants were asked to show their level of agreement [from strongly disagree (1) to strongly agree (5)] with the given statement (items) on a metric scale. So, all the statements in combination in each section should be interlinked with each other [26,27,28,29,30]. While some items were added in ‘yes or no’ form to be more decisive e.g., awareness about the availability of different PC services before and after PCO integration such as inpatient consultation and outpatient clinics, and one open-ended question about aspects of services that may need improvement.

Several questions were added about demographics, and palliative care education to characterize the respondents such as sex, age, any formal palliative care training…etc. Now, the preliminary instrument was then ready for piloting and included 124 items.

The questionnaire was distributed to palliative medicine physicians and oncology staff followed by qualitative and quantitative analysis.

Evaluation of validity and reliability of the final questionnaire

The final questionnaire was distributed to oncology staff in KCCC (medical oncologists and nurses). Again, qualitative and quantitative analyses of the items were done. To measure the test-retest reliability [31, 32] of the final version, the questionnaire was administered on two separate occasions, with an interval of two weeks between them. Two weeks were expected to be long enough for participants to have forgotten their original responses, but not sufficiently long for much real change in their opinions about the impact of early integration of palliative care. To match the two sets of questionnaires, birth dates were used. The first set of questionnaires was used for construct validity and internal consistency.

The final validated tool is composed of 9 sections:

The first and second sections of the questionnaire included oncologists’ and nurses’ socio-demographic characteristics such as age, gender, qualifications (nurses and physicians), years of work experience, occupation, and unit/ ward (physicians and nurses), and prior palliative care education.

The third section inquired about the structure, models, and inpatient and outpatient consultation services of palliative care that should be existing for cancer patients and what is currently available compared to before PCO integration.

The fourth to sixth sections were evaluating the impact of PCO integration on the quality of care, including symptom management, communication with patients and their families, and end-of-life care. Participants were asked to rate their level of agreement with items both before and after the introduction of the PCO.

The seventh to ninth sections measured the impact of the PCO on the work burden, attitude, and satisfaction of the oncology staff. These all were measured using a 5-point Likert scale to measure the degree of agreement/satisfaction with different items.

Finally, one item enquired if participants found the PCO service could be improved (Yes/ No) and one open-ended question allowed space for participants to suggest any improvement. Development, Pilot study, and validation of the questionnaire took around one and a half years to finalize the survey.

Ethical consideration

The approval of the ethical committee of the Ministry of Health was obtained (2017/62). The aim of the study and expected outcomes were discussed with all participants including the expert panel and focus group guaranteeing the privacy of the data and written informed consent was obtained from them. Permission from the KCCC Director and heads of oncology and nursing departments in KCCC was also taken.

Statistical analysis

Revision of the raw data, coding of the data, and master tables preparation were done. Data entry, manipulation, and analysis were done on SPSS (Statistical Package for Social Science), Version 20.

Development and evaluation of the preliminary questionnaire

Content validity was confirmed by the expert panel and the focus group (target population). Moreover, they tested and checked for relevancy, clarity, comprehensiveness, and representation. Content validity was assessed qualitatively for grammar, proper wording, order, and scoring and quantitively content validity Ratio (CVR) for each item. Item content validity Index (I-CVI) and scale content validity index (S-CVI) were also calculated then the instrument items were checked for comprehensiveness [29, 30, 33,34,35]. These repeated three rounds to ensure the highest content validity.

To ensure high face validity and the representation of a reasonably valid sample of items from the substantive areas of interest, items were then reviewed by all members of the expert panel and the focus group to select the best [29, 30, 35].

The results of the pilot study were analyzed for item discrimination and internal consistency and qualitatively. For item discrimination, Pearson correlation was used to compare each item with its subtotal score. Most literature chooses < 0.2 as a cut point for an item-to-total-score correlation to be rejected due to the inability to discriminate different answers related to this section during testing [27, 28], and to make it more relevant we retained only the items that met the correlation of > 0.3 or higher. Internal consistency using Cronbach’s alpha was measured separately for the different sections and 0.6 or more are accepted for research purposes [28, 36]. While the qualitative analysis was done by careful revision of each comment made by respondents. Repeated factor analyses (exploratory and confirmatory) were done throughout the study to help detect the number of the components’ domains and the correlations between items in each domain and between different domains.

Validation of the final questionnaire “Impact of early integration of palliative care and oncology’’(IEI PCO survey)

The results of the final survey were tested again for construct validity, internal consistency, and Test-retest reliability. Unfortunately, testing of construct validity [31, 32] of the final version needed to be administered into two groups known to differ in their level of impact of PCO integration, and this PCO integration is the first one held in Kuwait either with cancer or any other specialty. So, we decided to compare the two responses for the same participant as before and after integration nevertheless, this was only done for four sections.

Testing for item normality by Shapiro-Wilk and Kolmogorov-Smirnov tests was done. For comparing two repeated measurements for the same group Wilcoxon Signed Ranks test for non-parametric items while the Paired t-test for parametric items was used when appropriate. The chi-square test or Fisher’s Exact test was used when appropriate to compare quantitative and qualitative variables respectively. Cronbach’s alpha was used to test internal consistency as mentioned above. Test-retest reliability was done to verify that the results produced were consistent over time. The intraclass correlation coefficient (ICC) test was used to compare the response of the same group before and after two weeks. More than 0.8 was considered an accepted cut-off point for reliability and consistency over time.

Qualitative analysis of the open-ended question at the end of the survey was done through inductive thematic analysis for quantitative data to identify the main themes through analysis of the responses of the participants.

Likert scale items scored as continuous/interval and the score of each construct was summed together and divided by the number of 5-point Likert scale items then the sum of each construct was collected. The questionnaire is divided into two main parts; structural (section III) and clinical part (sections IV to IX). Scoring of structural domains (section III) (21 items) scored from 2 to 21 points. Q1 and Q6 were divided by the number of the items (5 points Likert scale), and Q2 (yes = 1 after integration) scored from 0 to 8 points according to awareness of the availability of PCO integration service at the respondent’s center. Q3, Q4, and Q5 scored then divided by 3 so the maximum points were 3. So, the difference (19 points) was divided into three unequal parts based on weighted scoring during factor analysis. Poor structural PCO integration scored < 7, incomplete structural PCO integration ≥ 7 to < 13, and Accepted/Good structural integration ≥ 13 to 21 points.

Scoring of clinical domains (section IV to IX: 42 items): the score ranged from 6 to 30, so, the difference (24 points) was divided into three equal parts for scoring. Poor clinical PCO integration scored < 14, incomplete clinical PCO integration ≥ 14 to < 22, Accepted/Good clinical integration ≥ 22 to 30 points. The total score (min 8 to max 51) was divided as follows; poor PCO integration scored < 21, incomplete PCO integration ≥ 21 to < 35, and Accepted/Good integration ≥ 35 to 51 points.

Results

Development and evaluation of the preliminary questionnaire: 124 items

Qualitative content analysis was done as mentioned before. The expert panel requested thrice to judge content validity, CVR, and CVI. Moreover, they assessed for instrument comprehensiveness based on the construct of the underlying study [29, 33,34,35,36]. CVR of more than 0.59 based on the Lawshe Table [37] according to expert numbers were considered valid. Item content validity Index (I-CVI) > 79% considered valid, 70–79% needs revision and < 70% should be eliminated scale content validity index (S-CVI) was also calculated and then checked for items comprehensiveness [29, 30, 33,34,35].

First round of judgment, 104 items were removed because they had CVR < 0.59. The remaining items based on content experts were modified, and overlapped items were removed. The remaining items were 172 items. (Table 1S, 2S) Second round, CVR, I-CVI, and S-CVI for judgment of relevancy and clarity were calculated. 14 items were removed for low CVR (< 0.59) 27 items were removed for low CVI (< 70%) and 16 items were modified for intermediate CVI. After modification, 125 items remained. They discussed with the expert panel member for the third time for relevancy, clarity, and comprehensiveness for each dimension of the construct underlying the study.

After their judgment, face validity was tested by a sample of the respondents together with the members of the expert panel to select the best in terms of its importance, clarity, apparent attractiveness and relevance of the questions to the research objectives, and interpretability [24, 38, 39]. This process reduced the number of items to 109 as a preliminary questionnaire ready for piloting.

Most of the expert panel members participated in all meetings till the end of the pilot stage of the study. This stage took around one year and expert panel meetings were held bimonthly till the pilot stage finished.

Of one hundred and eight questionnaires distributed, the response rate was 75.93% and 82 questionnaires were returned and completed. Males were 39% (n = 32), their mean age was 37.13 (7.41), oncologists were 31.7% (n = 26) and nurses were 68.3% (n = 56). Item discrimination was tested for each section separately from section III to section IX (Table 1).

Table 1 Item Analysis for preliminary “Impact of Early Integration of Palliative Care and Oncology Survey’’(IEI PCO survey):
Full size table

For item discrimination, 30 items were excluded due to poor discrimination (r < 0.3). Other item-to-total-score correlations (r) were ranging from 0.307 up to 0.947. Details are mentioned in Table 1. Furthermore, all items had a statistically significant relationship with the subtotal score (p-value < 0.05). Internal consistency was measured separately for the different sections using Cronbach’s alpha. It was acceptable ranging from 0.603 to 0.8. The main comment was the length of the questionnaire so some of the less relevant items in sections I and II (Demographic data and PC education) were removed to reduce the length of the questionnaire. Some changes to the wording were made in response to comments written on the questionnaires, to reduce uncertainty and maximize the clarity of the questions. Items with missed responses by most of the participants were also removed after missing data analysis testing in SPSS and then tested for the possibility of imputation process by using missing completed at random test (MCAR). This process of analysis leads to the dropping of nearly one-third (n = 48) of the original items. (Table 1)

Validation of the final questionnaire “Impact of early integration of palliative care and oncology’’(IEI PCO survey): 76 items

The compliance was acceptable (66.41%). Distributed questionnaires were 256, and the completed ones were 170 so the response rate was 66.41%. The total number of oncologists who completed the survey was 39 (22.9%) and nurses were 131(77.1%). (Supplementary file: Fig. 1S)

Based on the analysis described above, the number of items was reduced to 76. The test to retest using intraclass correlation coefficient for testing reliability for the item was very high, ranging from 0.823 to 0.986 and the overall reliability was 0.91. Item analysis and internal consistency of the final survey were similar to the preliminary one. The internal consistency reliability of each section was established using Cronbach’s alpha (0.673 to 0.978). Item-to-sub-score correlations were very good for most of the items > 0.50 to 0.940. (Table 2)

Table 2 Item Analysis for “Impact of Early Integration of Palliative Care and Oncology Survey’’(IEI PCO survey)
Full size table

In the final factor analysis, the Kaiser-Meyer-Olkin Measure of Sampling Adequacy was > 0.7 for each component’s items and 0.702 for all items in all components together. Bartlett’s Test of Sphericity was significant (p value < 0.01) for each component’s items and all items in all components together.

The mean total score of the structural domains was 16.02 ± 2.13 points, clinical domains were 24 ± 1.78, and total PCO integration was 40.1 ± 2.9 points. This was denoting good structural and clinical PCO integration in KCCC.

The differences in demographic characteristics, palliative care education, and PCO models between oncologists and nurses are shown in Table 3. After PCO integration, all oncologists and nurses agreed that all or most of the patients were seen on the same day of inpatient consultation while in outpatient consultations, 66.6% (n = 26) of oncologists and 72.5% (n = 95) of nurses agreed that most the patients were seen on the same day.

Table 3 Differences in Demographic characteristics and Palliative care education between oncologists and nurses
Full size table

There were statistically significant differences between oncologists and nurses separately in overall sub-scores of symptom control, communication, and end-of-life care before and after PCO integration (p-value for all < 0.001) (Supplementary File: Table 3S, Fig. 2S). Moreover, there were also statistically significant differences in the response of each item before and after PCO integration in the models of delivery, symptom control, communication, and in the end of life care (p-value for all < 0.001) This met the criterion of construct validity. (Table 4)

Table 4 The differences between models for the delivery of PCO, symptomatic management, communication with patients and family and end-of-life care before and after PCO integration
Full size table

After PCO integration, more awareness about the already available PCC services such as standalone PCC with 24-hour service and outpatient clinic (Z score − 7.071, -9.377, P-value < 0.001 for both) and the new PCO services such as inpatients consultation services, on-demand joint oncology-palliative care outpatientclinic, weekends and holidays, SPC inpatient ward rounds, and 24/7 phone calls for continuity of care (Z score − 10.934, -10.788, -12.329, -9.849, p-value < 0.001 for all).

Significant improvements in symptoms were appreciated by oncologists and nurses regarding the management of total pain, dyspnea, nausea and vomiting, constipation and other GIT symptoms, psychological issues, delirium, and opioids use (Z scores: -9.681, -9.069, -8.159, -7.401, -9.349, -8.432,-9.683, p-value for all < 0.001). Oncologists and nurses valued repeated honest communication, discussion of the goals of care, dealing more effectively with ending active treatment, and higher acceptance of patients and families of palliative care policy of transfer (Z score: -8.699, -8.171, -8.525, -7.593, p-value for all < 0.001). Significant progress in the care of end-of-life symptoms, clear compassionate communication about the patient’s condition and prognosis with the family, limitation of the role of life-sustaining treatment, and managing patients and family wishes including the place of death were esteemed by oncologists and nurses ( Z score: -9.419, -8.473, -8.582, -10.591, 10.721,-7.810, p-value for all < 0.001) (Table 4).

Most of the oncology staff agreed to the current structure and the process of clinical care of PCO, appreciated the discharge plan and continuity of care, admitted less work burden, a better attitude, and higher satisfaction about the role of palliative care in most items. (Table 5)

Table 5 Differences in the awareness about the structure and clinical process of PCO, continuity of care, work burden and satisfaction after PCO between oncologists and nurses
Full size table

Qualitative analysis of 84 participants responded to the open-ended question at the end of the survey. This was done through inductive thematic analysis. Participants replied positively craving for improving PCO integration. The thematic analysis yielded 3 predominant themes: the need for a regular Specialized Palliative Care (SPC) clinic in KCCC, a palliative care unit embedded in KCCC, and a regular palliative care education program annually or biannually.

Discussion

No one can deny the role of early palliative care in the trajectory of life-threatening illness, especially with a new era of a rapidly aging population worldwide that is associated with an increased incidence of cancer [40, 41]. Given advancing medicine and more and more disease-modifying therapies worldwide, it becomes difficult for a single oncologist to manage all aspects of cancer care such as diagnosis, disease-modifying therapies, symptom control, and end-of-life care [41]. More clinical support is needed through PCO integration during the entire disease journey, not only for alleviating symptom burden but helping the oncologists achieve a high standard of cancer care [31].

To our knowledge, no available tools to describe the impact of PCO integration on different aspects of patient care together such as symptom control, end-of-life care, and communication, and how these aspects affect oncology staff regarding work burden, attitude, and satisfaction. A few questionnaires were used to measure a few aspects of the impact of early PCO integration such as those used in Salins et al. [8] and Zagonel et al. [15] studies. In Salins et study, the authors used a survey questionnaire that was not validated to know the role of early specialized palliative care (SPC) on symptom control, communication, and health-related communication only [8] while in Zagonel et al. study, authors created and validated a questionnaire that focuses mainly on different aspects of communication, training, research, and organizational models [15]. The new IEI PCO questionnaire focused on nearly all clinical aspects of patient care. Validation of the questionnaire was done through both a pilot study and testing of the final questionnaire. The final Survey results are similar to the preliminary one in terms of item analysis, internal consistency, item to subs-score correlation.

In our study, oncologists and nurses perceived PCO integration of palliative care as a positive experience regarding all aspects of cancer care such as symptom control, end-of-life care, and communication when comparing these impacts of PCO before and after integration. While; after PCO integration; they admitted less work burden, a better attitude, and higher degrees of satisfaction toward palliative care. Moreover, oncologists and nurses found that the specialized palliative care (SPC) team smoothed planned discharge, clear follow-up plans, and overall better continuity of care.

Many studies were conducted on the co-management model of early involvement of specialized palliative care (SPC) in oncology care helped the patients in many aspects. Better symptom control outcomes in the group under palliative care were proved in two randomized clinical trials [32, 42] and other studies [8, 43].

Another aspect of improved care is better end-of-life care and communication. Clinical studies showed that early referral provides room for the therapeutic relationship between the specialized palliative care (SPC) team and patients and families that allows discussion of sensitive issues such as cessation of disease-directed treatment, changing goals of care, diagnosis and prognosis, and better management of end of life issues [4, 8, 44, 45]. As the oncologists often hesitate to discuss these sensitive issues this may make the patients and their families depressed and lose hope.

According to the literature, the timing of simultaneous specialized palliative care (SPC) initiation is unclear [46]. Many of these “red flags” concern severe or ongoing symptoms, or other emergency admissions as reasons for SPC initiation [4]. Also, the cancer specialist is considered the gatekeeper of the initiation of concurrent SPC [25, 47, 48]. To date, still many oncologists would be uncomfortable initiating SPC “gold standard”, although the triggers recommended by ASCO [15] give valuable guidance. Finally; as outlined by the Lancet Oncology Commission; [40] the timing of PCO integration has to be tailored according to each cancer center’s needs after thorough interdisciplinary discussion.

Currently, the criteria of referral “triggers” to our specialized palliative care (SPC) in Kuwait is outlined in Appendix (1) but moderate to severe symptoms and frequent admission are at the top of the list for referral to SPC.

The availability of different models of PCO integration and prompt response of specialized palliative care (SPC) helps to deliver high-quality service for cancer patients in Kuwait. Currently, still, community-based palliative care (home care) is newly started by non-governmental organizations but is not well established. Few patients are covered by that service.

The last report published about the mapping of palliative care worldwide described the situation in 2017 and labeled Kuwait at the 3a level as an isolated palliative care provision [49] but currently; since integration in 2016, the situation much improved. Moreover, the Kuwait Cancer Control Center (KCCC) was categorized as ESMO-DC (Designated center) for PCO in 2017.

Limitations of this study

This newly developed Likert-based questionnaire assesses the opinions of oncologists and nurses about the impact of PCO integration on different structural and clinical aspects of cancer care hence improving these services for better care. It is the first of its kind to measure the impact of PCO integration worldwide. This questionnaire was developed, validated, and tested for reliability through a pilot study and the application of the final questionnaire. However, the questionnaire is missing an assessment of integrated research and education programs and this is mainly due to the length of the questionnaire so we preferred to focus on the impact on clinical care as the first step before education and research.

Although a small sample size, this was considered representative because it included all available medical oncologists and nurses involved in the care of cancer patients in Kuwait.

Conclusion

The IEI PCO questionnaire demonstrates the psychometric criteria for content, face, and construct validity and reliability. It provides a valuable tool to assess the impact of PCO integration. The opinion of medical oncologists and nurses was significantly positive toward the early integration of PCO in Kuwait in most aspects of care. This integration led to improved symptom control, end-of-life care, communication, and planned discharge and follow-up plans. Moreover, decreases the work burden, improves attitude, higher satisfaction of the oncology staff, and continuity of care.