, Volume 102, Issue 1, pp 63–71 | Cite as

Does experience matter? A meta-analysis of physician rating websites of Orthopaedic Surgeons

  • R. A. JackII
  • M. B. Burn
  • P. C. McCulloch
  • S. R. Liberman
  • K. E. Varner
  • J. D. Harris
Original Article



To perform a systematic review evaluating online ratings of Orthopaedic Surgeons to determine: (1) the number of reviews per surgeon by website, (2) whether the number of reviews and rate of review acquisition correlated with years in practice, and (3) whether the use of ratings websites varied based on the surgeons’ geographic region of practice.


The USA was divided into nine geographic regions, and the most populous city in each region was selected. HealthGrades and the American Board of Orthopaedic Surgery (ABOS) database were used to identify and screen (respectively) all Orthopaedic Surgeons within each of these nine cities. These surgeons were divided into three “age” groups by years since board certification (0–10, 10–20, and 20–30 years were assigned as Groups 1, 2, and 3, respectively). An equal number of surgeons were randomly selected from each region for final analysis. The online profiles for each surgeon were reviewed on four online physician rating websites (PRW, i.e. HealthGrades, Vitals, RateMDs, Yelp) for the number of available patient reviews. Descriptive statistics, analysis of variance (ANOVA), and Pearson correlations were used.


Using HealthGrades, 2802 “Orthopaedic Surgeons” were identified in nine cities. However, 1271 (45%) of these were not found in the ABOS board certification database. After randomization, a total of 351 surgeons were included in the final analysis. For these 351 surgeons, the mean number of reviews per surgeon found on all four websites was 9.0 ± 14.8 (range 0–184). The mean number of reviews did not differ between the three age groups (p > 0.05) with 8.7 ± 14.4, (2) 10.3 ± 18.3, and (3) 8.0 ± 10.8 for Groups 1, 2, and 3, respectively. However, the rate that reviews were obtained (i.e. reviews per surgeon per year) was significantly higher (p < 0.001) in Group 1 (2.6 ± 7.7 reviews per year) compared to Group 2 (1.4 ± 2.4) and Group 3 (1.1 ± 1.4). There was no correlation between the number of reviews and years in practice (R < 0.001), and there was a poor correlation between number of reviews and regional population (R = 0.199).


The number of reviews per surgeon did not differ significantly between the three defined age groups based on years in practice. However, surgeons with less than 10 years in practice were accumulating reviews at a significantly higher rate. Interestingly nearly half of “Orthopaedic Surgeons” listed were not found to be ABOS-certified Orthopaedic Surgeons.


Orthopaedic Surgeon Online Physician rating website Years in practice Geographic region 


Compliance with ethical standards

Conflict of interest

RAJ, MBB, and SRL have no relevant conflicts of interest. KEV declares IP royalties with Solana, is a paid consultant with Solana, and has stock/stock options with Wright Medical. JDH declares association with Editorial Board for Arthroscopy: The Journal of Arthroscopic and Related Surgery, and Frontiers in Surgery, publication royalties from SLACK, Inc., research support from Depuy Synthes and Smith & Nephew, is a paid consultant from Smith & Nephew and NIA Magellan, and is on committees with AANA Research, AOSSM Self-Assessment, AAOS Osteoarthritis Pain and Function Workgroup. PCM is on the speakers' bureau for Genzyme, receives research support from Depuy, A Johnson & Johnson Company, Arthrex, and Zimmer, and is on the medical publication editorial board for the Journal of Knee Surgery and


  1. 1.
    Frost C, Mesfin A (2015) Online reviews of orthopedic surgeons: an emerging trend. Orthopedics 38(4):e257–e262CrossRefPubMedGoogle Scholar
  2. 2.
    Mehta SJ (2015) Patient satisfaction reporting and its implications for patient care. Accessed July 2016
  3. 3.
  4. 4.
    Emmert M, Sander U, Pisch F (2013) Eight questions about physician-rating websites: a systematic review. J Med Internet Res 15(2):e24CrossRefPubMedPubMedCentralGoogle Scholar
  5. 5.
    Lagu T, Hannon NS, Rothberg MB, Lindenauer PK (2010) Patients’ evaluations of health care providers in the era of social networking: an analysis of physician-rating websites. J Gen Intern Med 25(9):942–946CrossRefPubMedPubMedCentralGoogle Scholar
  6. 6.
    Gao GG, McCullough JS, Agarwal R, Jha AK (2012) A changing landscape of physician quality reporting: analysis of patients’ online ratings of their physicians over a 5-year period. J Med Internet Res 14(1):e38CrossRefPubMedPubMedCentralGoogle Scholar
  7. 7.
    Ellimoottil C, Hart A, Greco K, Quek ML, Farooq A (2013) Online reviews of 500 urologists. J Urol 189(6):2269–2273CrossRefPubMedGoogle Scholar
  8. 8.
    Keckley P (2011) 2011 Survey of health care consumers in the United States: key findings, strategic implications. Deloitte Center for Health Solutions, Washington, DCGoogle Scholar
  9. 9.
    Kadry B, Chu LF, Kadry B, Gammas D, Macario A (2011) Analysis of 4999 online physician ratings indicates that most patients give physicians a favorable rating. J Med Internet Res 13(4):e95CrossRefPubMedPubMedCentralGoogle Scholar
  10. 10.
    Mostaghimi A, Crotty BH, Landon BE (2010) The availability and nature of physician information on the internet. J Gen Intern Med 25(11):1152–1156CrossRefPubMedPubMedCentralGoogle Scholar
  11. 11.
    Ellimoottil C, Leichtle S, Wright C et al (2013) Online physician reviews: the good, the bad, and the ugly. Accessed Jan 2016
  12. 12.
    Trehan SK, Daluiski A (2016) Online patient ratings: why they matter and what they mean. J Hand Surg Am 41(2):316–319CrossRefPubMedGoogle Scholar
  13. 13.
    Greaves F, Pape UJ, King D et al (2012) Associations between Internet-based patient ratings and conventional surveys of patient experience in the English NHS: an observational study. BMJ Qual Saf 21(7):600–605CrossRefPubMedGoogle Scholar
  14. 14.
    Reimann S, Strech D (2010) The representation of patient experience and satisfaction in physician rating sites. A criteria-based analysis of English- and German-language sites. BMC Health Serv Res 10:1–14CrossRefGoogle Scholar
  15. 15.
    Lopez A, Detz A, Ratanawongsa N, Sarkar U (2012) What patients say about their doctors online: a qualitative content analysis. J Gen Intern Med 27(6):685–692CrossRefPubMedPubMedCentralGoogle Scholar
  16. 16.
    Shute N (2013) Online grades for doctors get an incomplete. Accessed 1 Jan 2016
  17. 17.
    Falkenberg K (2013) Why rating your doctor Is bad for your health. Accessed 1 Jan 2016
  18. 18.
    Moher D, Liberati A, Tetzlaff J, Altman DG (2010) Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Int J Surg 8(5):336–341CrossRefPubMedGoogle Scholar
  19. 19.
    Peckham C (2015) Medscape orthopedist compensation report 2015, p 7. Accessed Mar 2016
  20. 20.
    (2015) Annual estimates of the resident population for incorporated places of 50,000 or more, ranked by July 1, 2015 population: April 1, 2010 to July 1, 2015. Accessed Mar 2016
  21. 21.
    The American Board of Orthopaedic Surgery. Verify certification. Accessed Mar 2016
  22. 22.
  23. 23.
    C.S. Mott Children’s Hospital. National poll on children’s health: Many parents wary of online ratings for doctors 2016; Accessed 1 Apr 2016
  24. 24.
    Friedberg M, Pronovost P, Shahian D et al (2015) A methodological critique of the ProPublica Surgeon Scorecard. Accessed Jan 2016
  25. 25.
    Rothenfluh F, Germeni E, Schulz PJ (2016) Consumer decision-making based on review websites: are there differences between choosing a hotel and choosing a physician? J Med Internet Res 18(6):e129CrossRefPubMedPubMedCentralGoogle Scholar

Copyright information

© Istituto Ortopedico Rizzoli 2017

Authors and Affiliations

  • R. A. JackII
    • 1
  • M. B. Burn
    • 1
  • P. C. McCulloch
    • 1
  • S. R. Liberman
    • 1
  • K. E. Varner
    • 1
  • J. D. Harris
    • 1
  1. 1.Department of Orthopedics and Sports MedicineHouston Methodist HospitalHoustonUSA

Personalised recommendations