Measuring Patient Experience: Concepts and Methods
- 6.9k Downloads
Providing a good patient experience is a key part of providing high-quality medical care. This paper explains why patient experience is important in its own right, and its relationship to other domains of quality. We describe methods of measuring patient experience, including issues relating to validity, reliability and response bias. Differences in reported patient experience may sometimes reflect differences in expectations of different population groups and we describe the arguments for and against adjusting patient experience data for population characteristics. As with other quality improvement strategies, feeding back patient experience data on its own is unlikely to improve quality: sustained and multiple interventions are usually required to deliver sustained improvements in care.
KeywordsNational Health Service Patient Experience Communication Skill Training Quality Improvement Strategy Mystery Shopper
Key Points for Decision Makers
Quality is a multi-dimensional concept, and a single indicator does not (and should not) reflect quality in other domains. Patient experience is important in its own right.
Patient experience is consistently and positively associated with other quality outcomes including patient safety and clinical effectiveness across a wide range of studies, and healthcare facilities providing high-quality clinical care tend to have better experiences reported by patients. However, these associations are frequently modest in size. Clinical quality and patient experience should be considered as distinct but inter-related aspects of quality.
Differences in patient experience scores between population (e.g. ethnic) groups may in part reflect differences in expectations.
Adjusting patient experience scores for population characteristics (e.g. ethnicity, deprivation) increases the acceptability of the results to healthcare providers. Case-mix adjustment in general makes only small differences to scores, though the greatest positive effect is on providers serving vulnerable populations.
Significant quality improvement in general requires multiple strategies which are sustained over time. The same is probably true when using patient experience measures as a quality improvement tool. Simple feedback is unlikely to produce significant improvements in care.
Patient experience is an important outcome of medical care [1, 2] and a key component of quality of care [2, 3, 4, 5]. Policy makers worldwide are increasingly interested in gathering patient experience data to assess providers against a range of performance indicators and to stimulate quality improvement. In parallel, new technologies are driving innovative and cost-effective approaches to measuring patient experience. In this paper we consider what patient experience is, how it is measured, how patient experience relates to other aspects of the quality of care, and how data on patient experience can be used and interpreted.
2 What is ‘Patient Experience’?
The terms patient experience, patient perspective, patient reports, patient perception and patient satisfaction are often used interchangeably, with the potential to cause confusion and misunderstanding.
Patient experience, one of the most commonly used terms, can be conceptualised both as patients’ experiences of care and as feedback received from patients about those experiences. For example, the English National Health Service (NHS) outlines eight domains defined as critical to a ‘good’ patient experience, including respect, information and communication, physical comfort, emotional support, and access to care . This aspiration for patient experience does not, however, make an explicit link to the mechanisms by which such experience may be fed back to providers. In contrast, some healthcare analysts, e.g. Dr Foster Intelligence, define patient experience as “feedback from patients on ‘what actually happened’ in the course of receiving care or treatment, both the objective facts and their subjective views of it” . This places the focus for patient experience firmly both on what happens to patients, and how they report that experience.
This also highlights the blurred edges between the concepts of ‘patient experience’ and ‘patient satisfaction’. Often used interchangeably, the concepts are distinct, although the nature and direction of their relationship continues to be debated. Satisfaction is a multi-dimensional but often poorly defined concept, centred on the subjective experiences of patients . It is influenced by patients’ expectations of and preferences for care, by variations in response tendencies amongst different patient groups, as well as by the quality of care received [9, 10]. Satisfaction has been conceptualised and investigated as one determinant of patient experience , yet other contradictory formulations place patient experience as an element of satisfaction instead . This can obscure the concept and sometimes raises unfounded criticism of measures of patient experience in general. The appropriate conceptual view may depend on the level at which feedback from patients is being studied. Most commonly, when investigating the perceived quality of service providers such as hospitals or general practitioners (GPs), questions to patients about their satisfaction with the care provided form one subsidiary aspect of a wider set of items about their overall patient experience, which combines both views on and descriptions of care. In contrast, investigations of global ratings of an entire healthcare system (e.g. ‘the National Health Service’) lend themselves more naturally to placing satisfaction as the overarching concept, and subsequently exploring how experiences of distinct episodes of care drive overall ratings.
patient reports: patients’ accounts of what happened (e.g. did you receive information about your treatment?); and
patient evaluation: patients’ ratings of their experience (e.g. how would you rate the information you received about your treatment?).
Occasionally, these two approaches are combined, e.g. when patients are asked how long they waited, and also whether the waiting time was too long. This enables an assessment to be made of the acceptability of some aspects of care; in this case, waiting time . Critics of the evaluation approach argue that patient reports are more actionable and less likely to get overly positive responses . For this reason, recent patient experience questionnaires draw heavily on report approaches: for example, the Picker Institute questionnaires developed for use in the English NHS predominantly include report items, with one final item asking patients to rate the overall care received . While these criticisms are valid for general satisfaction questions, it is less clear that accusations of limited utility are valid when evaluation questions ask about very specific aspects of care—for example, “How good was the GP at involving you in decisions about your care?” (with responses: very good, good …. to … very poor) from the national GP Patient Survey. Furthermore, the distinction between report and evaluation questions is not always as clear-cut as sometimes suggested, as report items often contain an evaluative component—for example, by including the phrase “as much as you wanted” in the following report item from an NHS inpatient survey: “Were you involved as much as you wanted to be in decisions about your care and treatment?” (with responses: yes definitely; yes to some extent; no).
More recently, research has considered how well items on patient experience surveys can distinguish between differing expectations of care and different experiences. For example, a US study explored whether different patient groups, on this occasion differentiated on the basis of ethnic background, reported differing patient experience when presented with apparently identical healthcare. Through administering questions on expectations of care, a series of written vignettes, a video-depicted doctor–patient interaction, and modified communication (patient experience) items from a national survey (Consumer Assessment of Healthcare Providers and Systems surveys), it found that specific patient experience questions did indeed reflect difference in care, rather than differences in expectation .
3 How Can ‘Patient Experience’ be Measured?
Patient experience data may be gathered in a wide variety of ways. Most frequently in routine clinical practice, patients’ views are assessed using questionnaire surveys. Until recently, these were most often administered on paper, either handed out in clinic or posted to patients following attendance at a healthcare provider. New developments in technology, however, mean that surveys can be conducted through SMS messages to patients’ phones, through online surveys, or through the use of handheld devices or kiosks to get real-time feedback, e.g. daily assessments by patients of their care on a ward. Patient experience can also be assessed using interviews or focus groups, methods more commonly used in research because of the costs of routinely collecting and analysing such data. Other sources of feedback on patient experience include complaints and compliments received by medical staff, and feedback from patient groups. Different methods of feedback may draw different pictures: for example, data collected through interviews more frequently result in reports of negative experiences of care when compared with the results of surveys, which have been criticised for producing bland positive responses [16, 17]. Nevertheless, general summary measures of patient experience are popular with policy makers because of their simplicity, e.g. the ‘friends and family test’ now widely used in the UK NHS in which patients are asked whether they would recommend a facility (e.g. hospital or GP practice) to their friends and family .
As noted already, patient experience is one component of quality of care, and the question arises as to whether patients can provide valid and credible information to measure quality. As mentioned earlier, a range of domains of experience can be measured using both report and evaluation items, including accounts of waiting time, doctor–patient communication, staff responsiveness, availability of patients’ information and cleanliness. Some of these aspects can be assessed and confirmed objectively as well. For example, doctor–patient communication could be evaluated by external raters viewing videotapes of consultations to score the quality of the interaction, and access to primary care can be measured by using ‘mystery shopper’ approaches to measure the proportion of times a request for a particular appointment time could be met [19, 20]. These measures are not without their limitations: for example, raters assessing consultation videos are well-known to range from ‘hawks’ to ‘doves’, with subsequent variations in the scores they assign. It is therefore important to address these issues by, for example, using multiple raters to assess the same consultation. For instance, a recently developed instrument to assess the effectiveness of communication across an entire doctor–patient consultation (the Global Consultation Rating Scale, based on the Calgary-Cambridge Guide to Medical Interview), found that among simulated patient consultations, presenting consultations in a random order and using three or more raters addressed order effects in scoring consultations, and achieved acceptable reliability (>0.8) in assessing the quality of communication in a consultation . Whilst these alternative methods of examining experience are useful and can also give us an idea of the ‘objective facts’, they do not assess other relevant aspects of patient experience, namely how patients feel about their care.
Reliability is another important consideration when using surveys to measure quality of care. There need to be sufficient responses in order to distinguish genuine differences between providers (e.g. GP, hospital ward) from random variation due to chance. There are well-described methods for determining how many questionnaire surveys need to be returned in order to provide a reliable measure of care . As well as surveys based on low numbers (which may indicate low reliability), providers also express concern when the response rate is low. Their concern here is that patients who respond may be a biased sample, e.g. with systematically positive or negative experiences. However, response rates are only weakly associated with non-response bias in surveys that adhere to high standards of survey methodology [23, 24], and surveys with response rates typical of those in public sector surveys (e.g. 35–40 %) are often regarded as acceptable for the purpose of routine healthcare monitoring.
4 How does Patient Experience Relate to Other Measures of Quality?
Patient experience is multi-dimensional, and a single indicator does not (and does not need to) reflect quality in other domains . A number of studies have looked at the associations between patient experience and other measures of quality of care, and in particular between measures of patient experience and clinical quality. There are two reasons to consider these relationships. Firstly, to determine whether patient experience could act as a proxy measure of clinical quality, and secondly, to address the concern that an excessive focus on improving patient experience could be at the expense of clinical quality.
A recent systematic review found a positive association between patient experience and clinical effectiveness and patient safety . The authors concluded that patient experience should be regarded as “one of the central pillars of quality in health care”, with dimensions of quality being examined together rather than in isolation. Other studies have focussed on the impact of particular dimensions of patient experience on clinical outcomes. For example, effective doctor–patient communication can have an impact on health outcomes, including emotional health, symptom resolution, functional and physiological measures, and pain control . Recent analysis of English GP Patient Survey and Quality and Outcomes Framework (QOF) data found statistically significant positive associations between patient-reported doctor communication skills and the technical quality of care provided by a practice . Communication is also correlated with better patient adherence (the degree to which patients follow the recommendations of their health professionals), and a recent meta-analysis found that the odds of patient adherence were 1.62 times higher when a doctor had received communication skills training . Key moderators of the impact of training included the severity of the disease (training had less effect the more serious a patient’s disease was), the components of training (inclusion of communication skills around adherence led to a greater effect), and the doctor’s specialty (training had more effect if the doctors under study were paediatricians).
Positive associations between patient experience and outcomes may not always be seen: one trial of communication skills training among primary care staff found that the intervention improved communication with patients at the expense of worse diabetic control . This is, however, an outlier in the evidence we have to date, and a more detailed examination would be required to ascertain whether the design of the intervention may have impacted on assessments of communication. Overall, the evidence suggests that, on average, providing a good-quality patient experience does not compromise technical quality or patient outcomes, although the positive associations between these variables are often modest in size [26, 28, 29]. Nonetheless, modest associations can still have practical significance—for example, in a meta-analysis study, the odds of patient adherence to medication were found to be 2.16 times higher if a physician had good communication skills, despite the overall weak association between the two quality domains (r = 0.15) . In general, good-quality clinical care is associated with good patient experience.
5 How Should Patient Experience Data be Interpreted?
Although reporting of patient experience data has the potential to improve quality, it is also important to understand the complex nature of the data collected. For example, in English primary care, patients report more negative experiences in practices with a higher proportion of younger patients, ethnic minorities, and patients living in more socio-economically deprived areas . This could be because they receive worse care, or because the expectations of some population groups are different, or because they interpret the questions in survey instruments in different ways. There is evidence to suggest that patient experience questionnaires do allow patients to reflect on experiences independent of expectations . Nonetheless, it is rarely possible to determine the reason for such systematic differences, and in the past the results have been inconsistent and contradictory .
There is some evidence that different population groups may have different expectations of care. For example, Asian patients reported more negative evaluations of waiting times compared with White patients, even when the actual times that the groups reported waiting were similar . A further difficulty in interpreting patient experience data is that different population groups may place greater emphasis on some aspects of care than others—for example, less educated patients may place greater relative value on continuity of care than more educated patients . An example of different ways in which different population groups might interpret the actual response options presented on a questionnaire was reported by Elliott et al. , who found that more educated patients were less likely to use extreme response options at both ends of a scale. If these differences were pronounced, it would make it more difficult to draw comparisons between population groups across a range of quality domains.
These issues can be used to make an argument for case-mix adjustment of patient experience data. In practice, adjusting for patient characteristics in either primary care or hospitals has a relatively small impact on unadjusted values, though the effects tend to be greatest for providers serving vulnerable populations [31, 36]. In addition to ethnicity and deprivation, other variables to consider in the context of case-mix include age, sex, self-reported health status, and presence of longstanding psychological or emotional conditions [31, 36]. The impact of some of these variables (such as age) is likely to be relevant to patient responses in all surveys; others, such as whether a patient is admitted to hospital as an elective or an emergency, are obviously survey specific. What case-mix adjustment does do is to allow comparisons between different sites to be made without the immediate complaint from care providers that the patient populations are not comparable. For example, practices gaining most from case-mix adjustment in English primary care were on average smaller and treated patients who tended to be younger, ethnic minorities, living in more socio-economically deprived areas, and in worse health compared with patients from other practices . Adjusted data therefore have the potential to allow fairer comparison between health practices and/or providers. The argument against case-mix adjustment is that it may be a way of institutionalising sub-standard care by masking poor care provided to some patient subgroups, and therefore some argue that both adjusted and unadjusted data on patient experience should be reported [31, 36].
6 How Can Patient Experience Data be Used to Improve Quality?
Patient experience data are now widely used to measure the quality of health services. The NHS in England was among the first to mandate a national patient survey in 2001, with the USA following with its own national Hospital Consumer Assessment of Healthcare Provider and Systems survey (CAHPS). Australia, Canada, New Zealand and many European countries now use measures of patient experience and have made systematic arrangements for measuring and monitoring patients’ views at a national level . A survey of quality improvement strategies in 389 European hospitals found that patient experience data were increasingly used to assess quality of care alongside measures of clinical quality and safety . In England, several years’ data are now available publically on patients’ experience in acute, mental health and primary care, including specific surveys of maternity services, cancer services and emergency departments, and an expansion to the use of patient-reported outcome measures (PROMS) to evaluate the effectiveness of elective surgical care [39, 40, 41, 42, 43, 44, 45].
Despite data on patient experience being increasingly collected worldwide, there are still questions in regards to its uses and value. There is clear evidence that although health professionals in principle have positive attitudes towards patient feedback, they raise objections to its use when presented with results that appear critical of their own care [46, 47]. Improving quality requires a strategy of implementing multiple interventions, sustained over time. For example, the UK achieved major improvements in clinical quality in primary care with the introduction of a number of strategies, including national guidelines, nationally directed programmes of audit, public release of information on quality of care and pay-for-performance . However, the impact of patient experience at a local level remains to be explored, particularly as the impact of effective local initiatives may be lost in national aggregated data . Surveys of patient experience carried out on their own are unlikely to have much effect in changing practice . Nevertheless, collecting patient experience data is the initial but essential step to understanding challenges and opportunities in improving the quality of healthcare.
Public reporting of patient experience in combination with an array of interventions that take in to consideration the context of a healthcare system may have greater potential to stimulate providers to improve quality [51, 52, 53]. Although public reporting may encourage improvements through increased competition between health providers and professionals, a review exploring the impact of public reporting of patient experience, in combination with patient outcomes (such as mortality rates in hospital wards), found that some surgeons became more reluctant to care for high-risk patients (e.g. ethnic minorities) . As with any form of incentive, it is essential that the benefits and unintended consequences are weighed and addressed to ensure equitable provision of care.
This paper summarises some of the key issues surrounding patient experience, which is an important aspect of quality of care. There continues to be much work needed to refine our approaches to defining and measuring patient experience. Critically, we still have only limited insight into how the capturing of patient experience data can feed back into the drive for further improvements in patient experience. What is encouraging is the sustained recognition of policy makers and providers of the importance of listening to and acting upon the views and feelings of patients within healthcare. As Donabedian said (p. 251) : “It is when we help consumers help us that they can make their greatest contribution to enhancing the quality of care, even as we make ours”.
The authors declare that they have no competing interests. F.A. carried out the initial literature review and drafted the paper. M.R. and J.B. assisted with drafting and critically revised the manuscript. All authors read and approved the final manuscript. M.R. is the guarantor of this work.
- 1.Department of Health. The NHS outcomes framework 2013/14. London: Department of Health; 2012.Google Scholar
- 2.Institute of Medicine. Crossing the quality chasm: a new health system for the 21st century. Washington, DC: National Academy Press; 2001.Google Scholar
- 5.Council of Europe. Recommendation on development and implementation of quality improvement systems (QIS) in health care and explanatory memorandum (41st meeting, 24–26 June). Strasbourg: Council of Europe; 1997.Google Scholar
- 6.Department of Health. NHS patient experience framework. Department of Health. 2011. https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/215159/dh_132788.pdf. Accessed 5 Mar 2014.
- 7.Dr Foster. Intelligent board: patient experience. London; 2010. http://drfosterintelligence.co.uk/wpcontent/uploads/2011/06/Intelligent-Board-2010.pdf. Accessed 6 Mar 2014.
- 15.Weinick RM, Elliott MN, Volandes AE, Lopez L, Burkhart Q, Schlesinger M. Using standardized encounters to understand reported racial/ethnic disparities in patient experiences with care. Health Serv Res. 2011;46(2):491–509. doi: 10.1111/j.1475-6773.2010.01214.x.PubMedCentralPubMedCrossRefGoogle Scholar
- 18.NHS Choices. The NHS friends and family test. 2014. http://www.nhs.uk/NHSEngland/AboutNHSservices/Pages/nhs-friends-and-family-test.aspx. Accessed 28 Feb 2014.
- 19.NHS Wakefield Clinical Commissioning Group. Patient and public engagement annual report: April 2012–March 2013. NHS Wakefield Clinical Commissioning Group, Wakefield. 2013. http://www.wakefieldccg.nhs.uk/wp-content/uploads/2013/02/Patient-and-Public-Engagement-Annual-Report-April-2012-March-2013-FINAL.pdf. Accessed 12 Mar 2014.
- 20.NIHR Clinical Research Network. Mystery shopper report. Leeds: National Institute for Health Research Clinical Research Network; 2013.Google Scholar
- 21.Burt J, Abel G, Elmore N, Campbell J, Roland M, Benson J, et al. Assessing communication quality of consultations in primary care: initial reliability of the Global Consultation Rating Scale, based on the Calgary-Cambridge Guide to the Medical Interview. BMJ Open. 2014;. doi: 10.1136/bmjopen-2013-004339.PubMedCentralPubMedGoogle Scholar
- 26.Doyle C, Lennox L, Bell D. A systematic review of evidence on the links between patient experience and clinical safety and effectiveness. BMJ Open. 2013;3(1). doi: 10.1136/bmjopen-2012-001570.
- 27.Stewart MA. Effective physician-patients communication and health outcomes—a review. Can Med Assoc J. 1995;152(9):1423–33.Google Scholar
- 28.Llanwarne NR, Abel GA, Elliott MN, Paddison CAM, Lyratzopoulos G, Campbell JL, et al. Relationship between clinical quality and patient experience: analysis of data from the English quality and outcomes framework and the National GP Patient Survey. Ann Fam Med. 2013;11(5):467–72. doi: 10.1370/afm.1514.PubMedCentralPubMedCrossRefGoogle Scholar
- 35.Elliott MN, Haviland AM, Kanouse DE, Hambarsoomian K, Hays RD. Adjusting for subgroup differences in extreme response tendency in ratings of health care impact on disparity estimates. Health Serv Res. 2009;44(2p1):542–61. doi: 10.1111/j.1475-6773.2008.00922.x.PubMedCentralPubMedCrossRefGoogle Scholar
- 39.NHS Choices. Data downloads. 2014. http://www.nhs.uk/aboutnhschoices/contactus/pages/freedom-of-information.aspx. Accessed 28 Feb 2014.
- 40.Picker Institute Europe. NHS patient surveys. 2014. http://www.nhssurveys.org/. Accessed 28 Feb 2014.
- 41.Ipsos MORI. NHS England GP patient survey. 2014. http://www.gp-patient.co.uk/. Accessed 28 Feb 2014.
- 42.NHS Choices. NHS choices datasets. 2014. http://www.nhs.uk/aboutNHSChoices/professionals/developments/Pages/NHSChoicesDatasets.aspx. Accessed 28 Feb 2014.
- 43.National Cancer Intelligence Network. Cancer patient experience. 2014. http://www.ncin.org.uk/cancer_information_tools/cancer_patient_experience. Accessed 28 Feb 2014.
- 44.Health and Social Care Information Centre. Patient reported outcome measures. 2014. http://www.hscic.gov.uk/proms. Accessed 6 Mar 2014.
- 45.NHS England. MyhealthLondon website. 2014. http://www.myhealth.london.nhs.uk/. Accessed 6 Mar 2014.