A Comparison of Online Physician Ratings and Internal Patient-Submitted Ratings from a Large Healthcare System
Physician online ratings are ubiquitous and influential, but they also have their detractors. Given the lack of scientific survey methodology used in online ratings, some health systems have begun to publish their own internal patient-submitted ratings of physicians.
The purpose of this study was to compare online physician ratings with internal ratings from a large healthcare system.
Retrospective cohort study comparing online ratings with internal ratings from a large healthcare system.
Kaiser Permanente, a large integrated healthcare delivery system.
Physicians in the Southern California region of Kaiser Permanente, including all specialties with ambulatory clinic visits.
The primary outcome measure was correlation between online physician ratings and internal ratings from the integrated healthcare delivery system.
Of 5438 physicians who met inclusion and exclusion criteria, 4191 (77.1%) were rated both online and internally. The online ratings were based on a mean of 3.5 patient reviews, while the internal ratings were based on a mean of 119 survey returns. The overall correlation between the online and internal ratings was weak (Spearman’s rho .23), but increased with the number of reviews used to formulate each online rating.
Physician online ratings did not correlate well with internal ratings from a large integrated healthcare delivery system, although the correlation increased with the number of reviews used to formulate each online rating. Given that many consumers are not aware of the statistical issues associated with small sample sizes, we would recommend that online rating websites refrain from displaying a physician’s rating until the sample size is sufficiently large (for example, at least 15 patient reviews). However, hospitals and health systems may be able to provide better information for patients by publishing the internal ratings of their physicians.
KEY WORDSphysician online ratings physician online reviews patient satisfaction
The authors would like to thank Mimi Hugh MS MPH and the Kaiser Permanente Southern California Member Appraisal of Physician/Provider Services (MAPPS) Committee for their assistance with this study, as well as all the physicians of the Southern California Permanente Medical Group.
Compliance with Ethical Standards
All elements of the study were approved by the Kaiser Permanente Institutional Review Board (IRB).
Conflict of Interest
The authors declare that they do not have a conflict of interest.
- 2.Bakhsh W, Mesfin A. Online ratings of orthopedic surgeons: analysis of 2185 reviews. Am J Orthop (Belle Mead NJ). 2014;43(8):359–63.Google Scholar
- 3.Black EW, Thompson LA, Saliba H, Dawson K, Black NM. An analysis of healthcare providers’ online ratings. Inform Prim Care. 2009;17(4):249–53.Google Scholar
- 11.Merrell JG, Levy BH, 3rd, Johnson DA. Patient assessments and online ratings of quality care: a “wake-up call” for providers. Am J Gastroenterol. 2013;108(11):1676–85.Google Scholar
- 12.Sabin JE. Physician-rating websites. Virtual Mentor. 2013;15(11):932–6.Google Scholar
- 21.Randhawa S, Viqar A, Strother J, et al. How Do Patients Rate Their Radiation Oncologists in the Modern Era: An Analysis of Vitals.com. Cureus. 2018;10(9):e3312.Google Scholar
- 22.Press Ganey Associates Inc. “Patient Experience - Patient-Centered Care.” Available online at: http://www.pressganey.com/solutions/patient-experience. Last accessed 26 June 2019.
- 23.Agency for Healthcare Research and Quality. “CAHPS Clinician & Group Survey.” Available online at: https://www.ahrq.gov/cahps/surveys-guidance/cg/index.html. Last accessed 26 June 2019.
- 28.Smith A, Anderson M. “Online Shopping and E-Commerce.” Pew Research Center, 2016 December. Available online at: http://assets.pewresearch.org/wp-content/uploads/sites/14/2016/12/16113209/PI_2016.12.19_Online-Shopping_FINAL.pdf . Last accessed 26 June 2019.
- 29.McKinstry B, Yang SY. Do patients care about the age of their general practitioner? A questionnaire survey in five practices. Br J Gen Pract. 1994;44(385):349–51.Google Scholar
- 30.Kaiser Permanente. “Southern California Fast Facts.” Kaiser Permanente. 2019 March. Available online at: https://about.kaiserpermanente.org/who-we-are/fast-facts/southern-california-fast-facts. Last accessed 26 June 2019.