INTRODUCTION

Public sharing of patient experience scores to advance patient centeredness has gained traction but remains controversial due to concerns that this may inadvertently contribute to inappropriate care.1, 2 Starting in October 2015, at Virginia Mason Medical Center, provider-level patient satisfaction ratings and comments were displayed prominently on the institutional public website. The objectives of this study were to determine whether measures of potentially inappropriate care changed following public sharing of patient satisfaction data, and secondarily to examine the relationship between inappropriate care and provider ratings.

METHODS

We performed a single-center retrospective cohort study from January 2015 to December 2017 of primary care providers who had at least 500 adult patient visits and 20 adult patient surveys both before and after the October 2015 initiation of public reporting. The intervention was public reporting of patient ratings of providers based on the 5-point Likert scale question “likelihood of your recommending care provider” (Clinician and Group Consumer Assessment of Healthcare Providers and Systems, Press Ganey Associates, South Bend, Indiana). Because higher patient satisfaction is associated with provider continuity,3 surveys were excluded if patients reported they were not visiting with their usual provider. For each provider, we also determined the rates of three a priori markers of potentially inappropriate care from the institutional electronic health record: antibiotics for acute respiratory illness (ARI),4 MRI for uncomplicated low back pain (using previously reported sets of ICD9/10 visit codes),5 and narcotic prescribing (any central nervous system depressant drug with abuse potential, including opiates, benzodiazepines, and barbiturates) for non-cancer patients.6

For the primary analysis of inappropriate care markers before and after the intervention, we conducted multilevel logistic regression models on patient visit-level data, clustering for provider. In separate models, the dependent variables were antibiotic order, lumbar MRI order, and narcotic order. Odds ratios were estimated adjusting for potential patient confounders (sex, age, new patient visit) and provider confounders (age, specialty, sex). Additional provider potential confounders considered but not included in the final model due to collinearity or lack of association with the outcomes were professional degree, years in practice, and years at our institution. Patient-level covariates considered but not included were race/ethnicity, education, self-rated health, and survey response method. Analysis of patient ratings of providers versus inappropriate care markers was performed using linear regression on provider-level data with adjustment for provider age. All analysis was performed using STATA MP 12 (Stata Corp, College Station, TX). This work was determined by the institutional review board to be quality improvement and exempt from formal review.

RESULTS

The study included 475,154 patient primary care visits, with 10,888 completed patient surveys for 74 providers (Table 1). The mean provider age was 47 years, with 19 years since medical school graduation, and 13 years at our institution. Most (96%) were Medical Doctors and practiced Internal Medicine (80%). During the study time frame, use of online rather than paper surveys increased (online 11% (311/2960) to 66%, (5232/7928), p < 0.001).

Table 1 Patient Surveys and Patient Primary Care Visits Before and After Public Reporting of Quality Ratings
Full size table

Public sharing of provider ratings was not associated with statistically significant increases in lumbar imaging rate (odds ratio (OR) 0.90, 95% CI 0.62, 1.30), antibiotic rate (OR 1.02, 95% CI 0.95, 1.10), or narcotic rate (OR 1.01, 95% CI 1.00, 1.03) (Table 2).

Table 2 Association Between Potentially Inappropriate Care Markers and Public Reporting of Quality Ratings, by Visit
Full size table

After adjustment for provider age, the potentially inappropriate care markers of lumbar imaging (attributable risk (AR) − 0.014, 95% CI − 0.321, 0.293), antibiotics for ARI (AR − 0.096, 95% CI − 0.318, 0.127), and narcotic order (AR 0.545, 95% CI − 0.021, 1.112) were not associated with higher provider ratings. The mean provider rating both before and after public reporting was 4.81 (p = 0.66).

DISCUSSION

Public reporting of provider ratings was not associated with increase in the inappropriate care markers of antibiotics for ARI, imaging for low back pain, or prescribing narcotics. Further, there was no significant association between patient provider rating and these markers of potentially inappropriate care at baseline.

This study reflects the experience at a single institution only. Limitations include a potential ceiling effect, survey response bias, residual confounding from changing patient and survey characteristics, and changes in scores with time independent of public sharing of survey results.

We conclude that concerns that public reporting of provider rating scores may drive providers toward inappropriate overutilization may be unwarranted.