Introduction

Patient-reported outcome measures (PROMs) are standardized questionnaires completed by patients. Their purpose is to assess patients’ perception of a variety of health and well-being indicators that provide important information to healthcare professionals (HCPs) for patient care. PROMs have been advocated for use in routine cancer care for some time now [1] and there is evidence that they may improve symptom control, patient well-being, cost effectiveness, patient engagement and survival [2,3,4,5,6,7,8]. Routine use of PROMs is also considered a major indicator of integration between oncology and palliative care [9].

Reviews of the literature have shown that HCPs perceive PROMs as facilitating the identification and assessment of symptoms and of psychological, social and spiritual distress [10,11,12]. Also patients perceive PROMs as relevant, easy to use and useful for describing their health-related conditions [13, 14]. However, systematic PROM collection is not widely implemented in routine oncology practice for individual patient care [15,16,17]. Difficulty in changing established work practices, a lack of time, and fear of a negative impact on the patient-clinician relationship are the main causes of their limited use [13, 18, 19]. Electronic assessment of PROMs (ePROMs) has been identified as a potential means to overcome barriers to routine use [20, 21] and an opportunity to maintain and further develop a patient-centered approach to care in the era of big data [22].

Although the findings on these topics are progressively increasing, there is a lack of studies evaluating the knowledge and use of different PROMs, and there are still few reports on HCPs’ attitudes towards ePROMs. Within a wider project called Patient Voices [23] aimed at promoting the use of PROMs in routine cancer care at an Italian comprehensive cancer center, the present survey has the purpose of assessing the knowledge, use and attitudes of HCPs towards PROMs and ePROMs in clinical practice and research and of exploring respondent-related factors associated with the above dimensions.

Methods

Participants

All HCPs (physicians, nurses, psychologists, physiotherapists, radiotherapy and radiology technicians) employed in clinical work at the Fondazione IRCCS Istituto Nazionale dei Tumori of Milan (INT) in January 2019 (N = 992) were eligible for the study. INT is a comprehensive cancer center associated to the Organization of European Cancer Institutes, pursuing the prevention, early diagnosis and treatment of cancer. Eligible HCPs were invited to participate in an anonymous web survey sent through the SurveyMonkey® system. Survey reminders were sent to non-responders until a response rate of at least 50% was reached [24].

Assessment

An ad hoc questionnaire was developed to investigate two main areas: 1) knowledge and use of PROMs; 2) attitudes of HCPs towards PROM and ePROM use in clinical practice. Data on sex, professional role, hospital department of affiliation, and seniority (years since degree/diploma) were also collected.

Knowledge and use of PROMs

A list of 17 tools for PROM collection commonly used in oncology to assess multiple dimensions of quality of life, psychological distress and physical symptoms was presented to the survey participants. The full list is reported in supplementary Table 1. For each tool respondents were asked to choose one of four possible options (1 = I don’t know the tool; 2 = I know the tool but have never used it; 3 = I have used the tool occasionally; 4 = I have used the tool frequently). Three further yes/no questions investigated the use of other tools not included in the list, the use of ad hoc developed (not validated) tools, and the use of self-translated (not validated) tools. One item investigated whether the respondent had used PROMs mainly in research, clinical practice or equally in both settings. Finally, two multiple choice questions asked which professional role among a list of five (physician, nurse, psychologist, social worker, volunteer) should present and propose the completion of a PROM to the patient and which of them should assist the patient in completing it.

Attitudes and opinions about PROMs

The survey continued with a list of 17 statements on PROM strengths (e.g. “PROMs can be useful for documenting the quality of care we offer to patients”) and weaknesses (e.g. “During the visit there is no time for adequate administration of PROMs”). One item investigated participants’ preferred collection mode (electronic vs paper & pencil) and eight further items addressed the strengths and weaknesses of electronic PROM collection (e.g. “ePROMs allow a graphic display of the symptom and quality of life trend of scores over time” or “ePROMs are difficult to implement due to the lack of familiarity with electronic devices in some groups of patients”). Responses were collected through five-step Likert scales, from 1 = totally disagree to 5 = totally agree.

Statistical analysis

When planning the survey we calculated that a sample size of 450 respondents would allow to estimate a two-sided 95% confidence interval (CI) for the mean scores with a precision (half CI width) of 9.2% of its standard deviation [25]. In case of a dichotomous answer to questionnaire items (yes/no) and in the conservative hypothesis of 50% positive answers, the precision would be 4.6% [25].

A data quality check was performed to identify – and eliminate from the analysis – careless or inattentive responding, as this is a source of measurement error that can obscure meaningful results [26]. The following indicators were calculated for each respondent:

  • survey completion time less than 4 min

  • identical answers to all items on PROM tool knowledge and use

  • identical answers to all items about the strengths and weaknesses of PROMs

  • identical answers to all items about the strengths and weaknesses of ePROMs

  • a diagonal response pattern in at least one block of items (e.g. 12,341,234)

Respondents scoring positive on at least four of the above five indicators were removed from the analysis. Respondent versus non-responder characteristics were compared using the χ2 test for independence on aggregate data from the list of invited HCPs.

Average scores and their 95% CIs were calculated for single strength and weakness items of both PROMs and ePROMs. In order to reduce the dimensionality of the set of items regarding knowledge, use and attitudes towards PROMs and ePROMs we calculated the following summary scores with the corresponding Cronbach alpha (α), when relevant: number of PROMs known, number of PROMs used frequently, overall average score of PROM strength items (α = 0.93), overall average score of PROM weakness items (α = 0.85), overall average score of ePROM strength items (α = 0.87), and overall average score of ePROM weakness items (α = 0.63). Multivariate linear regression models were applied to determine if sex, profession (physician, nurse, other), seniority (years since degree/diploma) and hospital department (surgery, medical oncology & hematology, critical & supportive care, other) (independent variables) were associated with each of the six indicators mentioned above (dependent variables). Two separate multivariate logistic regression models were applied to assess the association of the same independent variables as above with use of PROMs in clinical practice and with preference for either a paper or electronic format. Due to subgroup sample size constraints, only first-order interactions among independent variables were examined in the regression models.

In evaluating regression models, statistical significance was deemed for p < 0.01 and 99% confidence intervals. Statistical analyses were performed using the Stata 16 software (StataCorp. 2019. Stata Statistical Software: release 16. College Station, TX: StataCorp LLC.).

The Patient Voices project, including the present survey, was approved by INT’s institutional review board (INT 167/18).

Results

Of the 992 HCPs invited to the survey, 552 opened the link and viewed the questionnaire; 523 filled out the first two screens of the survey and were defined as respondents (response rate 52.3%). Data of 12 respondents were eliminated during the quality check described in the Methods section, leaving a final set of 511 participants. Among them, 475 completed the survey till the end (completion rate 47.9%). Respondents and non-responders were comparable with regard to profession (p = 0.28) and department of affiliation (p = 0.44), while respondents were slightly more frequently female (68% vs 62%, p = 0.048).

Table 1 shows the characteristics of the 511 participants: the majority were women (68.1%), the professions were mainly nurses (45.6%) and physicians (42.3%), working either in surgery (28.8%) or medical oncology and hematology (24.7%)departments; a small majority (52.5%) had > 20 years’ seniority.

Table 1 Characteristics of respondents (N = 511)
Full size table

Knowledge and use

Figure 1 shows data on the knowledge and use of 17 common PROM tools sorted by purpose (psychological aspects and patient experience measures, physical symptoms, quality of life). Unidimensional pain scales were the most frequently known (80.9%) and used tools (61.3%), followed by multidimensional pain scales, known by 69.9% of respondents and frequently used by 16.2%. On average, respondents reported knowing 5.8 tools (range 0–17) but having used at least occasionally only 2.6 (range 0–15), frequent use being even more limited (1.4 tools on average). A number of self-report questionnaires including PedsQL, STAI (or SDS), MOS SF-36 (or SF-12) and EQ-5D were reported to be little known by HCPs (“Do not know” > 75%). In general, physical symptom scales were more frequently known and used than quality of life, psychological aspects and patient experience measures.

Fig. 1
figure 1

Percentage of known and used PROMs. Note: The full list of references and acronyms for the PROMs in Fig. 1 is included in Supplementary Table 1

Full size image

Sixty-five respondents (13%) reported having used other PROMs not included in the list. PROMs reported at least twice were DN4, Breast-Q, IPSS, MMPI and Morse S (see Supplementary Table 1 for acronyms and references).

The multiple linear regression results indicated that profession and department were significantly associated with the number of PROMs known (Table 2, first column): nurses and “other professions” reported knowing 2.9 and 2.7 fewer tools, respectively, than physicians, while HCPs working in medical oncology and hematology departments reported knowing 1.9 more tools than those working in surgical departments. The number of PROMs used frequently (Table 2, second column) was the only regression model in which one interaction proved significant (between seniority and department of affiliation p), while none of the factors not included in the interaction emerged as significant. To facilitate model interpretation, we report in Fig. 2 the number of PROMs frequently used by various combinations of seniority and department as estimated by the model. While HCPs in medical oncology and hematology departments consistently reported a greater use of tools than other HCPs, the difference with other departments was much more pronounced for younger HCPs.

Table 2 Results of multiple linear and logistic regression models to explore respondent characteristics associated with knowledge, use and attitudes towards PROMs and ePROMs
Full size table
Fig. 2
figure 2

Regression model estimates of mean number of frequently used PROMs by hospital department and seniority

Full size image

Only 16 respondents (3.2%) declared having used PROMs developed ad hoc without any validation, and 15 (3%) reported having translated into Italian and used a questionnaire without any validation. Respondents opined that PROMs should be presented to patients preferably by nurses (64.4%) or physicians (61.4%) (data not reported in the table; more than one response was possible). Instead, it was mainly the nurse (63.6%) who was indicated as being responsible for assisting patients in completing the questionnaire. The figure in Supplementary Fig. 1 shows the detailed percentage distribution.

Two hundred and two respondents (41.5%) declared never having used any PROM tool, 111 (22.8%) having used them mainly in research and 106 (21.7%) in clinical practice, while 68 (14.0%) had used PROMs about equally in both settings. Figure 3 reports the PROM use setting by department, showing that the largest number of HCPs who had never used any PROM tool were in the department category “Other”. It also shows that PROMs were used mainly for research purposes by 30.1 and 25.6% of HCPs working in medical oncology/hematology and surgery departments, respectively, while the reported use for clinical practice purposes was around 19% for both; this ratio was reversed for HCPs working in critical & supportive care department, where PROM use for clinical practice was higher than that for research purposes (33.6% vs 17.6%). Overall, 174 respondents (35.7, 95% CI 31–40%) declared having used PROMs in clinical practice (mainly in clinical practice or equally in clinical practice and research). The multivariate logistic regression model results (Table 2, third column) indicated that use in clinical practice was not associated with any of the independent variables examined except for department, with HCPs in critical & supportive care department being more likely to use PROMs in clinical practice (OR = 2.0, 99% CI 1.0–4.1).

Fig. 3
figure 3

PROM use setting by hospital department

Full size image

Attitudes and opinions

Respondents showed a generally positive attitude towards the use of PROMs, with an overall average score of strengths outweighing weaknesses (3.6 vs 2.9, range 1–5) (Table 3). As regards weaknesses, items concerning the difficulty of administering PROMs due to a lack of resources (time, personnel) obtained higher average scores than items concerning a lack of PROM utility or reliability (Table 3). The multiple linear regression results indicated that none of the HCP characteristics examined were associated with the overall average score of PROM strengths (Table 2, fourth column), whereas seniority and profession were associated with weakness scores (Table 2, fifth column), albeit with quite low effect sizes. In particular, nurses had a less negative attitude than physicians (average weakness score 0.2 lower than physicians), while older HCPs expressed a more negative attitude than younger ones, with a 0.3-point higher average weakness score.

Table 3 Mean and 95% CI for PROM strengths and weaknesses: single-item and overall scores
Full size table

Three hundred eighteen respondents (66.9, 95% CI 62.5–71.2) expressed a preference for the administration of PROMs in an electronic format. The multivariate logistic regression results (Table 2, sixth column) indicated that only profession was significantly associated with the preferred format: nurses were less likely to prefer ePROMs than physicians (OR = 0.5, 99% CI 0.3–0.98).

All items indicating ePROM strengths (Table 4) gained average scores close to 4 (range 1–5); in regard to weaknesses, respondents were mainly concerned with the potential lack of familiarity with electronic devices of some patients (average 3.4). Looking at the overall average scores, we found that ePROM strength scores outweighed weakness scores. The multiple linear regression results showed that nurses were slightly less positive than physicians about ePROMs (Table 2, seventh column), men were slightly less negative than women, and senior HCPs were slightly more critical than younger ones (Table 2, eighth column).

Table 4 Mean and 95% CI for ePROM strengths and weaknesses: single-item and overall scores (N = 475)
Full size table

Discussion

The results of this survey indicate that HCPs in a comprehensive cancer center, especially physicians, were familiar with a fair number of PROMs and expressed a generally positive attitude towards them. As reported in previous studies [10, 11, 16], the strengths of PROMs were considered to outweigh their weaknesses; however, frequent use was not common, which is also consistent with available evidence [10, 17, 27]. In particular, all tools but pain scales were reported to be used frequently only by a small percentage of respondents. It is worth noting that in Italy pain scales were introduced in hospital medical charts by a national law [28] in 2010 and their completion according to specific operating procedures has since been mandatory. This suggests that cultural and scientific developments may not be sufficient for successful implementation and that organizational and resource allocation interventions may be just as important [29].

In general, the fact that physical-symptom tools tend to be more frequently known and used than PROMs focused on quality-of-life or psychological aspects may be explained by the multiple barriers faced by oncologists in the assessment and management of mental health distress in patients with cancer [30].

Interestingly, the survey indicated that junior HCPs working in oncology/hematology departments reported using more tools than others, and that PROM use in research was more widespread in oncology/hematology and surgery departments than other areas. Likewise, senior HCPs identified more PROM weaknesses than younger colleagues. Such results are in agreement with the fact that junior HCPs are actively involved in data collection for research projects and also that they are taught about the value and use of PROMs in oncology care during their training. Consistent with these results is the more common use of PROMs in clinical practice in the critical & supportive care department, which includes palliative care, supportive care, clinical psychology as well as anesthesia and postoperative pain services. These are settings where patient care is mainly focused on physical and psychosocial symptoms, the assessment of which is strongly grounded on self-reporting.

Consistent with findings in the literature [10, 13, 19, 27], the perceived weaknesses of PROMs mainly concern the lack of resources and the additional workload for their administration, while concerns about their utility are less strong. Conversely, there is published evidence that routine PROM assessment allows more effective use of time and resources [11, 31]. The aspects related to concerns about the reliability of PROMs that have emerged in other studies [13, 19] were less common in the present survey results. PROM tools were originally developed for use in research, and perhaps their administration is still identified as a “research task” potentially interfering with clinical practice rather than an effective means of clinician-patient communication.

Around two-thirds of HCPs reported preferring the use of PROMs in an electronic rather than paper-and-pencil format, with nurses reporting lower ePROM strength scores. Since electronic PROM assessment was not a reality at INT when the survey was carried out, the expressed preferences are merely theoretical; however, respondents showed an overall positive attitude towards ePROMs, acknowledging their potential in making the process of data collection and use more efficient. Boyce et al. [10] reported that there may be concerns that technology diminishes the “human touch” in the care process and this may be explain why nurses were found to be slightly more critical towards ePROMs.

The present study has some limitations. Firstly, the survey was carried out in a single institution, a comprehensive cancer center with a tradition in health-related quality of life research. This may limit the generalization of the results, as they may not fully reflect oncology practice in other, less specialized and research-oriented settings. Secondly, the survey questionnaire was developed ad hoc; however, with the exception of the overall score of weaknesses of ePROMs, where Cronbach’s alpha value was at the limit of acceptability, all overall scores assessing attitudes had alpha values above 0.85.

Conclusions

Although routine use of PROMs in clinical practice has shown a positive impact on patient care, management and even prognosis, their use in this comprehensive cancer center was still found to be less than optimal, confirming previous evidence. It is therefore important to plan specific efforts and appropriate strategies to achieve higher levels of implementation, which should be based on wide stakeholder involvement and application of national and international standardized procedures. Given the limited implementation of PROMs despite the relatively positive attitude of HCPs, the Patient Voices [23] project aims to fill this gap not only from a technological standpoint (through the development of an ePROM system integrated with the electronic medical record) but also from a cultural perspective, engaging both HCPs and patients in the choice and routine use of these tools for more effective patient care.