Introduction

Assessment of patient experiences can improve the quality, experience, and outcomes of healthcare, and provide useful information for decisions about patient management and health service delivery [1,2,3]. Patient-reported experience measures (PREMs)—defined as ‘a measure of a patient’s perception of their personal experience of the healthcare they have received’ [2, 4, 5]—aim to identify where improvements in patient experience are required, judge the success of efforts to improve health services, and meaningfully capture what happens during the patient’s course of illness and treatment [3, 6].

Several generic and condition-specific PREMs have been developed and are currently used in clinical settings such as hospitals [3, 7]. The primary uses of PREMs include: (a) enabling patients to reflect comprehensively on interpersonal aspects of their healthcare experience; (b) providing reliable metrics for public reporting, benchmarking of institutions/centers and healthcare plans; and (c) generating patient-level information for driving service quality improvement strategies [1].

Despite these potential benefits, evidence is mixed regarding whether routine PREM assessment in clinical practice improves healthcare services [8,9,10,11,12]. Inconsistent findings may be due to lack of clear guidance on how to use PREM data to meaningfully inform improvements to the quality of care [13, 14]. Other possible challenges in their use include: (1) difficulty in measuring change at all levels of the healthcare system or institution [5, 10]; (2) time and resources available for staff to collect and analyze data [15]; (3) lack of consistency in measurement of patients’ experience across services and institutions [11]; (4) variation in how these data are used [11]; and (5) lack of skills and expertise among staff to effectively use and interpret patient experience data [16]. With limited resources, healthcare providers find it challenging to rigorously assess whether they do indeed provide cost effective and high-quality healthcare services.

Given the critical role healthcare professionals and patients play in ensuring PREMs data is used to successfully improve healthcare services, a better understanding of their experiences of using PREMs in routine clinical practice is needed. Perceptions of these key stakeholders may yield unique insight into how PREM programs could be implemented more effectively in clinical practice to guide meaningful improvements in care. This is important to ensure that the time and money invested in collecting PREM data is not wasted, and the benefits fully realized. To explore this issue in-depth, we conducted a systematic review of the qualitative literature to examine the perspectives of patients and healthcare providers towards using PREMS in routine clinical care.

Methods

Our systematic review of qualitative studies was conducted according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidance [17]. We limited our review to qualitative studies as these types of designs allow for in-depth enquiry into patient and healthcare provider perceptions of implementing PREMs in routine practice.

Searches

Our search strategy comprised a comprehensive set of key terms for ‘PREMs’ and ‘qualitative research’ (Additional file 1: appendix 1). Searches were performed in electronic databases—MEDLINE, EMBASE, and PsycINFO from inception to May 2021. Electronic searches were supplemented by searches of reference lists of included studies and identified related papers.

Eligibility criteria

Studies that met the following criteria were included:

  • Qualitative study design (e.g. individual interviews or focus groups); mixed method studies were considered if a qualitative component was included;

  • Sample was any patient and/or healthcare professional

  • Focus of the study was on exploring perceptions of implementing patient-reported measures (PRMs) in routine clinical practice and the PRM focus was on experiences of healthcare

Studies were excluded if they were not in English; did not have adequate information (e.g. conference abstracts); or focused on PREM development, validation, or selection for use in a particular clinical setting.

Study selection

One reviewer (AJ or RC) screened retrieved titles and abstracts for eligibility and 25% selected at random were cross-checked [18] by a second reviewer (CR). Where abstracts met eligibility or relevance was ambiguous, papers were obtained and reviewed in full. Full texts were independently reviewed by two reviewers (CS and AJ or RC). Disagreements were resolved through team discussion.

Data extraction

A data extraction form was developed including study title, identifying information (author names, country), methods, aims, location and setting, study design, participants (population group, eligibility criteria, sample size, population characteristics), recruitment method, PREMs used, and study findings. All information from included studies were extracted by one reviewer (AJ) and cross checked by a second reviewer (CS) for accuracy against the original article.

Quality assessment

Included studies were assessed for quality against the 32-item Consolidated Criteria for Reporting Qualitative research (COREQ) checklist [19] by one reviewer (AJ). A second reviewer (CS) assessed study quality for 25% of included studies. The COREQ checklist assesses reporting in qualitative literature in three domains: (1) research team and reflexivity (includes personal characteristics, relationship with participants); (2) study design (includes theoretical framework, participant selection, setting, data collection); and (3) data analysis and findings (includes data analysis, reporting). It includes 32 items and is scored as 0 = not reported, 1 = partially reported, and 2 = fully reported, with each article receiving a total quality score out of 64, converted into a percentage. Thus, higher scores indicate higher quality reporting.

Synthesis of results

We used an interpretive descriptive approach using thematic analysis [20] to analyze the textual data extracted from the results sections of studies included in this review. This methodology was selected as it offers good transparency and is an adaptation of secondary data synthesis of ‘thematic analyses’. The process involved three stages: (1) becoming familiar with the data extracted from studies and generating initial codes, (2) identifying similarities between codes and searching for themes, (3) reviewing, and defining themes both within and across studies. Two reviewers (AJ and CS; both post-doctoral researchers with expertise in PRM methodology) independently coded from each study. Descriptive themes developed as findings from studies representing similar phenomena were grouped. These were then collated into higher level analytical themes according to our research questions. A third reviewer (CR; experienced PRM methodologist) reviewed the themes and codes to ensure that they accurately reflected the data reported in included studies.Hence, there was minimal potential for bias on data generation and analysis. Study characteristics were considered to explain any differences in findings across studies [20].

Results

Study selection

Electronic searches retrieved 2683 abstracts, of which 52 were potentially relevant and 20 articles met eligibility criteria (Fig. 1). Of these, nine used semi-structured interviews, two used focus groups, and nine used mixed methods (i.e., combination of semi-structured interviews, focus groups, patient surveys and/or clinician surveys) (Table 1).

Fig. 1
figure 1

Prisma flow diagram describing study inclusion for the review

Full size image
Table 1 Summary of included studies reporting healthcare professionals’ and patients’ perspectives on using PREMs (N = 20)
Full size table

Study characteristics

Table 1 provides a summary of the 20 included studies. Five studies (25%) were conducted in primary care settings [4, 21,22,23,24], nine (45%) in hospitals [8, 25,26,27,28,29,30,31,32], and the rest either in psychological therapy services [33], a disability care organization [34] or other healthcare settings such as Veterans Health Administration [35] or National Health Service (NHS) [36]. Studies were conducted in the UK (N = 9; 45%), the USA (N = 8; 40%), Ireland (N = 1; 5%), Brazil (N = 1; 5%) and the Netherlands (N = 1; 5%). Sixteen (80%) included clinicians only, one (5%) included patients only, and three (15%) included both patients and clinicians. Total sample size across studies was 150 patients (range across studies = 3 to 127) and 553 clinicians (range across studies = 8 to 60). Patient participants were diverse, including chronic wounds, chronic conditions, adult cardiology, geriatric, orthopedic and stroke patients. Clinician participants included medical and executive directors, CEOs, surgeons, general practitioners, anesthetists, secondary care doctors, psychologists, counsellors, nurses, patient satisfaction coordinators, other allied health professionals, integrated care providers and administrative staff.

The main aims of included studies were quality improvement, exploration of benefits and challenges of using PREMs in clinical practice, assessing factors affecting use of PREM data in quality improvement, clinicians’ perceptions of the usefulness of PREM data, and investigating ways to effectively implement PREM data in routine clinical practice.

Quality appraisal

Study quality ranged from 20.3 to 57.8% [4, 21,22,23,24,25,26,27,28,29, 31,32,33,34,35,36,37,38]. No study met all quality criteria; however, some quality items were adequately addressed by all studies (Fig. 2). All studies clearly reported their major themes, 19 of 20 studies reported on the consistency between the data presented and study findings, and 18 of 20 studies reported on derivation of themes. However, most studies (84%) did not describe the research team and reflexivity in detail including interviewer characteristics, nature of the relationship with interviewees, and participants’ feedback on findings (Fig. 2).

Fig. 2
figure 2

Quality of reporting across included articles (n = 20) per COREQ item

Full size image

Synthesis of results

The findings across studies were synthesized into six themes: facilitators to PREM implementation; barriers to PREM implementation; healthcare providers’ perspectives towards using PREMs; patients’ perspectives towards using PREMs; advantages of using PREMs in clinical practice; limitations and practical considerations to reduce resistance of PREM usage.

Facilitators to PREM implementation

Studies reported PREMs were best used in a cyclical manner to give staff time to review findings and implement changes [25, 30, 33]. Organizational facilitators included a working culture supportive of improvement, change and recognizing patient views [30, 35]. Support from management and staff encouragement were seen as key in facilitating staff engagement with improvement related activities such as using and actioning PREM data [28, 31, 34, 35]. Staff reported that management allowing dedicated time to discuss PREM data, initiate improvements to health services and keeping staff informed about the use of patient feedback were key factors for successful PREM implementation [28, 29, 31, 36]. Furthermore, patient involvement in identifying areas for improvement [33, 37] could benefit implementation. Successful quality improvement programs require: (1) training and educational campaigns aimed at promoting quality improvement programs, (2) desire to deliver high quality patient-centered care by senior management and promoting a patient-centered culture within the hospital, (3) staff prompting and reminding each other to distribute PREMs and maintain data collection rates, (4) repeating surveys at regular intervals for longitudinal comparisons, (5) using a team-based approach to collect PREMs, and (6) making PREMs data visible and easily accessible to staff [4, 22, 28, 30, 31, 33, 35]. Staff reported that publicly releasing individual hospital PREM data for comparison across hospitals helped raise awareness of quality of care within institutions and its staff [28, 29].

Barriers to PREM implementation

Despite support for PREM usage in hospitals for health service evaluation and quality improvement, six studies reported several barriers related to hospital resources and environment [28, 30, 33, 34]. The most common barrier was resource limitations (e.g., availability of staff to collect and action PREM data) [28, 34, 35]. PREM completion is an additional responsibility for staff and time and resource constraints may adversely affect timely administration, consequently reducing the number of PREMs implemented for quality improvement purposes[28, 34]. Insufficient funding for PREM collection infrastructure prevented some hospitals from collecting PREM data or being able to prioritize and implement changes needed to achieve significant improvements to care [28, 30]. Reluctance among staff to overburden patients with paperwork by administering surveys during discharge [26, 28, 29, 33] may also influence implementation. The perception that patients would be overburdened with paperwork could limit the opportunity for patients to provide feedback on their experiences [26, 28, 29, 35]. Additional barriers noted by staff included: (1) lack of commitment to quality improvement, (2) worry about being held accountable for poor service outcomes, and (3) uncertainty about how to use PREMs data [30, 31, 35]. Staff also reported the collection of PREMs led to considerably increased workload due to the high volume of patients seen at healthcare services [30, 31, 35]. Although hospitals valued patient’s perspectives and established clear objectives about the importance of quality improvement, these barriers prevented staff from spending the time necessary to engage in quality improvement-related activities and initiatives. Additional staff-related barriers included skepticism about the usefulness of PREM data, lack of awareness about the importance of quality improvement programs, lack of interest in collecting PREM data, and unwillingness to change, improve or participate in quality improvement initiatives [31, 35]. Some staff also questioned whether they had the skills necessary to improve patient-centered care [4, 35]. Other staff challenges included concerns about the technology or automated data systems used for PREM collection as well as difficulties analyzing, interpreting and translating findings into actionable information [33, 35].

Healthcare providers’ perspectives towards PREMs

Studies that explored staff perspectives to using PREMs routinely in practice reported contradictory views [22, 23, 25, 34]. Some studies reported a positive outlook to PREMs, reporting the information gathered was invaluable, positively impacted their work and provided additional motivation to improve their skills, while others found staff felt less certain about the usefulness of PREMs [22, 25]. The main concern was patients’ ability to provide accurate and relevant feedback; mainly due to positive bias (patients tendency to provide positive feedback and reluctance to criticize their healthcare provider) and halo effects (patients’ tendency to attribute a previous negative experience to future healthcare experiences) [4, 22, 31]. Other factors reported to impact patients’ ability to provide accurate feedback included failure to understand experience measures, inconsistency between different patients despite similar experiences, and inability to evaluate clinical competence [4, 22, 31].

Overall, healthcare providers’ reported PREMs data could be used to prevent or minimize harm to patients by reducing risk of injury [29, 33] as well as providing information important for an institution’s safety practices and process [21, 26, 35]. PREM data was also useful for improving staff awareness of the quality of healthcare services provided and identifying areas needing improvement that staff were not aware of at both an individual and organizational level [21, 29, 33, 37]. PREM feedback, particularly when positive, served as an important opportunity to commend staff and reinforce good practice [4, 35, 37]. However, some staff stated that compliments were not treated in the same way as complaints. It was reported that complaints about staff were given high importance, prompting immediate action but compliments were not celebrated. Some staff considered quantitative data to be of limited value in understanding patient experiences and favored qualitative data (in the form of patient narratives and quotes), which were considered to facilitate more in depth understanding of patient concerns and experiences [22, 29, 31].

Patients’ perspectives towards PREMs

Three studies exploring patients’ perspectives on using PREMs routinely in practice reported the biggest challenge patients faced in providing feedback was the fear of being questioned about their treatment process and asked for an opinion about it while they were receiving treatment [24, 31, 32]. Patients stated that PREMs data have potential to improve overall care coordination and provided a unique opportunity to produce timely feedback on measurable processes and outcomes so that healthcare teams could use PREM data to improve their overall performance [24, 32]. Patients also reported an implementation challenge to be addressed by hospitals was to identify competent staff to help sustain the quality improvement program [24, 32].

Advantages of PREMs

PREMs assessing quality improvement focused on factors such as physician and nursing care, staff courtesy, cleanliness, comfort, waiting times, education about follow-up after discharge as well as satisfaction with information provided, the hospital system, services (e.g. food) and treatment outcomes such as pain management [21, 22, 26, 28, 30]. Some health services combined staff (clinicians, allied health, nurses) opinions with PREM data, recognizing the link between personal, professional and team development [21, 39]. Notable advantages of using PREMs in hospitals/clinics included: (1) gaining insight into patient perspectives, (2) longitudinal data to guide the development and implementation of quality improvement activities, monitoring changes and sustainability of improvement efforts, (3) early identification of new areas for quality improvement, (4) helping management prioritize action plans necessary to improve the quality of patient-centred care, and (5) offering management and staff the opportunity to evaluate their efforts to improve health services [8, 21,22,23, 26, 28, 30, 35, 37, 39]. Additionally, PREMs enabled comparison across clinicians, practices, and health services, and may motivate individual hospitals towards quality improvement [8, 21,22,23, 26, 28, 30, 35, 37, 39]. For patients, completing PREMs at regular intervals may help track their experiences in health services. PREMs that included a section for free text comment helped patients describe their expectations from their clinician and note any questions they wanted to ask clinicians during their consultations [8, 28, 31, 34].

Limitations and practical considerations to reduce resistance to PREM usage

Staff reported PREM data from a single patient may not detect key areas for service level quality improvement. Rather, PREM data should be aggregated at sufficient intervals (e.g., three monthly) to enable key areas for improvement to be detected [27, 28, 35, 37]. However, some clinicians felt the cost of collecting PREMs in a large sample was not worth the information generated. Others worried that collecting and reporting PREM data might be perceived as a threat to health service providers [22, 37]. Communication skills training for clinical and nonclinical staff to facilitate PREM-related conversations with patients might encourage staff to administer PREMs [23, 27, 28, 34, 35]. Staff also reported PREMs could be time consuming for patients to complete and raised expectations that could not be met by hospitals or individual staff [34]. Some patients worried about confidentiality and anonymity of their PREM data and how it would be used [4, 34].

Studies reported that in order to reduce staff resistance to using PREMs routinely in clinical practice, it was important to provide: training on the value of PREMs, clear guidance on data collection to avoid involuntary errors, assistance to choose the right measures for the context and intended purpose (e.g., appropriate for the particular medical condition/diagnosis), and discussion of results relevant to improving health services [31, 35, 37, 39, 40].

Discussion

This research systematically reviewed the qualitative literature on patients’ and healthcare providers' perspectives towards using PREMs in routine clinical care. We identified six main themes reflecting barriers and facilitators to PREM implementation, patients’ and healthcare providers’ perspectives towards using PREMs, advantages of using PREMs in clinical practice, and limitations and practical considerations to reduce resistance to PREM usage.

Key facilitators to PREM implementation were organizational culture, support from management, dedicating time to PREM activities in hospitals, quality improvement programs, and providing comparisons of PREM data across hospitals. Key barriers to PREM implementation included resource constraints such as limited staff availability, lack of time, insufficient funding, reluctance and resistance among staff, lack of clarity on how to use PREM data, and concerns about technology used to process PREM data.

Barriers identified were broadly similar to previous literature: lack of resources [28, 34, 35], time, and expertise in analysis of data and quality improvement [35, 40]. Another barrier commonly reported was lack of training for staff, suggesting that staff were interested in better understanding of how to use and interpret PREM data [10, 26, 31, 35, 36]. In order to successfully shift towards a more patient-centered healthcare service, it is crucial to engage relevant staff (both clinical and nonclinical) in structured training across healthcare settings and create awareness about the importance and value of collecting PREM data [10, 23, 27, 28, 31, 34,35,36].

Implementing PREM data collection was challenging for some staff regardless of setting [28, 34, 35]. Data collection planning, persistence, and commitment from the management and staff were required [10, 11] to succeed in implementation. Clinical staff (doctors, nurses and allied health professionals) reported the importance of having a designated person to take ownership of data collection [29, 34] including identifying eligible patients, monitoring their enrollment and appointments, and contacting patients who did not return completed PREMs. These tasks demand additional weekly resources and were more challenging at clinics or hospitals that used paper-based medical records.

Clinical staff also noted that some patients’ dissatisfaction may be beyond their control. This includes patients assuming newer devices and interventions had better care standards and were disappointed to find older technology used in hospitals. Similarly, patients with multiple comorbid conditions such as diabetes, hypertension, and substance abuse may experience worsening symptoms during their treatment or procedures, and this can be unsatisfactory for patients; individuals have varying tolerance for pain and this may impact their overall care experience; or insurer limitations may result in patients’ reporting unsatisfactory experiences related to their healthcare service [29, 34].

Healthcare providers’ perspectives towards using PREMs were mixed. Some reported PREM data improved services and awareness among staff about the services they provided. Collecting and analyzing PREM data can improve patients’ experiences and transform practices and medical institutions [41]. However, other healthcare providers reported additional concerns about accuracy of patient feedback because several other factors such as positive bias, halo effect, timing of survey administration, and differing expectations may influence patients’ perceptions of the services they receive. These results are further supported by previous studies [42,43,44] which report patient perceptions about the quality of care received correlated with lower complication rates and clinical quality of the medical institution (as measured by performance indicators). Further, Stein et al. [44] reported that some healthcare providers felt patient satisfaction was a poor indicator of quality of care.

Patient experiences and outcomes may also be confounded by characteristics such as age, disease stage and phase, in the context of chronic or acute illness [45]. Therefore, to ensure patient feedback and data add value to organizations, they may be integrated with indicators from other sources (such as their treatment and related side effects, insurance experience, comorbid conditions, wait times, logistics, and any other inconveniences). Taking a multi-dimensional performance evaluation approach would facilitate implementation of PREMs in routine care, and staff may not always be held responsible for patient dissatisfaction. Furthermore, integrated data may help organizations understand the underlying relationship between effectiveness, safety, and experience at the patient level.

Consistent with others [46], we found that patients may be burdened with paperwork during discharge, thus limiting the opportunity for them to provide accurate feedback on their experiences. To reduce this burden, web based PREMs can be introduced, allowing patients to complete them at home post-discharge. This methodology has been proven to be inexpensive for organizations and timesaving for staff, when compared to postal surveys or telephone surveys [47] and does not cause bias in the results [48].

This review supports the positive impact of care coordination on patients’ experience and found that patients considered PREMs a critical feedback tool to assess and improve the quality of care across various clinical settings and between staff within a healthcare team [24, 32]. Arguably, for some health conditions such as chronic wounds, disease specific PREMs may be useful to assess condition-specific aspects of care coordination, interdisciplinary communication, and shared decision-making [24, 32].

Longitudinal data collected using PREMs may also enable patient participation in articulating and mapping their own experiences of health services. Despite challenges and limitations, service providers who persevered with data collection, put in efforts to strengthen patient relationships, and engaged patients in the assessment and planning process capitalized on the benefits of PREMs data [28, 34].

Findings from this review highlight clinicians’ concerns about collecting PREMs, particularly their worries about disruption of patient care [8, 28, 35, 38]. Frequent feedback was perceived as more useful than only annual data collection for quality improvement initiatives. However, some were skeptical about the validity of results [26, 28, 34]. Further, patients appreciated prompt recognition of their complaints and feedback [29, 34] and were more likely to complete PREMs if they perceived their feedback led to improvements in their health service experience.

Hence, to ensure quality improvement programs are successful, hospitals should take patients’ feedback into immediate consideration and make efforts to improve their experience [29, 34]. For patients to complete PREMs successfully, it is essential to train staff on why and how to use PREMs within the context of their specific clinical environment. For example, inviting patients to complete PREMs on a busy clinic day may result in lower completion rates compared to collection of PREMs on a quiet day [8, 26, 29, 33]. Including real-time feedback systems into web based PREMs to instantly capture patients’ feedback and to provide quick response to any issues reported could promote a patient-driven quality improvement culture. This may be well received by patients when done regularly. Web based PREMs would facilitate PREM data collection, but requires investment in set-up of software systems for data collection and processing.

To our knowledge this is the first review to synthesize the qualitative evidence on healthcare professional and patient perspectives and experiences of using PREMs in clinical practice. This helps guide future quality improvement initiatives. However, limitations of this review should be considered when interpreting the findings. Overall, quality of study design and conduct for some studies was unclear due to the poor quality of reporting. Quality improvement initiatives using PREMs may occur within hospitals and clinics regularly, but findings may be held locally and not publicly available. Further, this review only included articles published in English so the experiences of using PREMs in countries where English is not the first language were not represented. Nevertheless, a range of common themes were identified across studies, providing useful information for future PREMs related research.

Some notable gaps were identified in this review. Only four studies explored patients’ experiences and perspectives of PREMs. We did not extract PREM development and psychometric properties as this was beyond the scope of this paper. Consequently, we excluded studies focused on reporting PREM development, validation, or selection for use in a particular clinical setting. Future research should explore this and how PREMs data are currently being used in everyday clinical practice and describe any structured plans developed by hospitals or healthcare organizations on how to use patient feedback for quality improvement. Understanding patient perceptions of using PREMS in routine clinical care is critical to optimize patient engagement and may potentially alleviate staff concerns about overburdening patients. In the context of healthcare improvements, further investigation is needed into how PREM data is used to inform changes to the delivery of health services and provider behavior. Future PREMs initiatives should also include follow-up assessments to examine changes in patient experiences because of improvements made by hospitals based on feedback received.

Conclusions

This study highlights the advantages of collecting and using PREM data from the healthcare professionals’ and patients’ perspectives and identifies barriers and facilitators to implementing PREMs into routine clinical care. The primary use of PREMs has been to initiate health service quality improvements. Key factors facilitating and hindering the collection and use of PREMs include organizational-, staff- and patient-related factors. Sufficient resources, support from organization leadership, formal staff training in using and interpreting PREM data, choosing the right measure, and patient engagement in the assessment and planning are crucial for the benefits of PREM data to be realized. Future studies may focus on addressing barriers to PREM implementation and evaluate the effects of PREMs on health service quality improvements.