Improving Patients’ Choice of Clinician by Including Roll-up Measures in Public Healthcare Quality Reports: an Online Experiment
Abstract
Background
Public reports on healthcare quality typically include complex data. To lower the cognitive burden of interpreting these data, some report designers create summary, or roll-up, measures combining multiple indicators of quality into one score. Little is known about how the availability of roll-ups affects clinician choice.
Objective
To determine how presenting quality scores at different levels of aggregation affects patients’ clinician choices.
Design
We conducted a simulated clinician-choice experiment, randomizing participants to three versions of a public reporting website and comparing their clinician choices. One version aggregated all clinician-level quality measures into roll-ups, the second provided disaggregated (drill-down) scores only, and the third offered both roll-ups and drill-downs.
Participants
Five hundred fifty panelists drawn from a probability-based Internet panel.
Main Measures
We assessed the amount of effort participants exerted by tracking the length of time spent on the website and the number of concrete actions taken on the website (e.g., clicking items). We evaluated decision quality by measuring whether participants selected a clinician who performed more poorly than others and incongruence between participants’ stated preferences for dimensions of quality and their chosen clinician’s performance on those dimensions.
Key Results
Participants seeing drill-downs alone (mean = 14.9) or with roll-ups (mean = 19.2) took more actions than those who saw roll-ups alone (mean = 10.5) (ps < 0.05). However, participants seeing only drill-downs made poorer choices than those who saw roll-ups alone or with drill-downs. More participants seeing drill-downs chose a clinician who was outperformed (36.3% versus 23.4% [roll-up] and 25.6% [drill-down + roll-up], ps < 0.05) and made choices incongruent with stated preferences (51.2% versus 45.6% [roll-up] and 47.5% [drill-down + roll-up], ps < 0.05). The distinction between roll-up and drill-down was somewhat stronger for sicker participants.
Conclusions
Our results suggest that roll-ups in healthcare quality reports, alone or as a complement to drill-downs, can help patients make better decisions for themselves.
KEY WORDS
healthcare quality patient experience patient satisfaction composite measure summary scoreNotes
Funding
This paper was supported by two cooperative agreements (2U18HS016980 and 1U18HS016978) from AHRQ to RAND Corporation and Yale University, respectively.
Compliance with Ethical Standards
Conflict of Interest
The authors declare that they do not have a conflict of interest.
Prior Presentations
This research has not been presented previously.
Supplementary material
References
- 1.Findlay SD. Consumers’ interest in provider ratings grows, and improved report cards and other steps could accelerate their use. Health Aff. 2016;35(4):688–96.CrossRefGoogle Scholar
- 2.Martino SC, Kanouse DE, Miranda DJ, Elliott MN. Can a more user-friendly Medicare Plan Finder improve consumers’ selection of Medicare plans?. Health Serv Res. 2017;52(5):1749–71.CrossRefGoogle Scholar
- 3.Sinaiko AD, Eastman D, Rosenthal MB. How report cards on physicians, physician groups, and hospitals can have greater impact on consumer choices. Health Aff. 2012;31(3):602–11.CrossRefGoogle Scholar
- 4.Hibbard JH, Peters E. Supporting informed consumer health care decisions: data presentation approaches that facilitate the use of information in choice. Annu Rev Public Health. 2003;24:413–33.CrossRefGoogle Scholar
- 5.Kurtzman ET, Greene J. Effective presentation of health care performance information for consumer decision making: a systematic review. Patient Educ Couns. 2016;99(1):36–43.CrossRefGoogle Scholar
- 6.Schlesinger M, Kanouse DE, Martino SC, Shaller D, Rybowski L. Complexity, public reporting, and choice of doctors: a look inside the blackest box of consumer behavior. Med Care Res Rev. 2014;71(5 Suppl):38s–64s.CrossRefGoogle Scholar
- 7.Schlesinger M, Kanouse DE, Rybowski L, Martino SC, Shaller D. Consumer response to patient experience measures in complex information environments. Med Care. 2012;50(Suppl):s56–64.CrossRefGoogle Scholar
- 8.Gebele C, Tscheulin DK, Lindenmeier J, Drevs F, Seemann AK. Applying the concept of consumer confusion to healthcare: development and validation of a patient confusion model. Health Services Management Research. 2014;27(1–2):10–21.CrossRefGoogle Scholar
- 9.Shi Y, Scanlon DP, Bhandari N, Christianson JB. Is anyone paying attention to physician report cards? The impact of increased availability on consumers’ awareness and use of physician quality information. Health Serv Res. 2017;52(4):1570–89.CrossRefGoogle Scholar
- 10.Centers for Medicare and Medicaid Services. Hospital Compare star ratings fact sheet. Available at: https://www.cms.gov/newsroom/fact-sheets/hospital-compare-star-ratings-fact-sheet. Accessed 10 September 2018.
- 11.Cerully JL, Martino SC, Rybowski L, Finucane ML, Grob R, Parker AM, Schlesinger M, Shaller D, Martsolf GR. Using “roll-up” measures in health care quality reports: perspectives of report sponsors and national alliances. American Journal of Managed Care. 2017;23(6):e202–207.Google Scholar
- 12.Zaslavsky AM, Shaul JA, Zaborski LB, Cioffi MJ, Cleary PD. Combining health plan performance indicators into simpler composite measures. Health Care Financing Review. 2002;23(4):101–15.Google Scholar
- 13.Schwartz B. The Paradox of Choice: Why More is Less. New York, NY, US: Harper Collins Publishers; 2004.Google Scholar
- 14.Martsolf GR, Scanlon DP, Christianson JB. Multistakeholder perspectives on composite measures of ambulatory care quality: a qualitative descriptive study. Med Care Res Rev. 2013;70(4):434–48.CrossRefGoogle Scholar
- 15.Association of American Medical Colleges. Guiding principles for public reporting of provider performance. Available at: https://www.aamc.org/download/370236/data/guidingprinciplesforpublicreporting.pdf. Accessed 10 September 2018.
- 16.Thompson A. RE: Enhancements of the Overall Hospital Quality Star Rating, August 2017. Available at: https://www.aha.org/system/files/advocacy-issues/letter/2017/170925-let-thompson-cms-star-ratings.pdf. Accessed 10 September 2018.
- 17.Romano P, Hussey P, Ritley D. Selecting quality and resource use measures: a decision guide for community quality collaboratives. Available at: https://www.ahrq.gov/professionals/quality-patient-safety/quality-resources/tools/perfmeasguide/index.html. Accessed 10 September 2018.
- 18.Boyce T, Dixon A, Fasolo B, Reutskaja E. Choosing a high-quality hospital: the role of nudges, scorecard design and information. Available at: https://www.kingsfund.org.uk/sites/default/files/field/field_publication_file/Choosing-high-quality-hospital-role-report-Tammy-Boyce-Anna-Dixon-November2010.pdf. Accessed 10 September 2018.
- 19.Orlowski JM. Re: AAMC comments on the measure selection for Hospital Compare star ratings TEP report. Available at: https://www.aamc.org/download/425936/data/aamccommentletteroncmsstarratingstep.pdf. Accessed 10 September 2018.
- 20.Nickels TP. RE: CMS-1677-P, Medicare Program; Hospital Inpatient Prospective Payment Systems (PPS) for Acute Care Hospitals and the Long-Term Care Hospital PPS and Proposed Policy Changes and FY 2018 Rates; Quality Reporting Requirements for Specific Providers; Medicare and Medicaid Electronic Health Record (EHR) Incentive Program Requirements for Eligible Hospitals, Critical Access Hospitals, and Eligible Professionals; Provider-Based Status of Indian Health Service and Tribal Facilities and Organizations; Costs Reporting and Provider Requirements; Agreement Termination Notices; Proposed Rule (Vol. 82, No. 81) April 28, 2017. Available at: https://www.aha.org/system/files/advocacy-issues/letter/2017/170613-let-nickels-verma-cms-flexibility-efficiencies.pdf. Accessed 10 September 2018.
- 21.Cerully JL, Parker AM, Schlesinger M, Martino SC, Shaller D, Rybowski L, Grob R, Finucane ML. The Select MD 2.0 Provider Choice Experiment: Methodological Overview. Santa Monica, CA: RAND Corporation; 2016. Available at: https://www.rand.org/pubs/working_papers/WR1152.html. Accessed 10 September 2018.
- 22.GfK. Knowledge Panel® design summary. Available at: http://www.knowledgenetworks.com/knpanel/docs/KnowledgePanel(R)-Design-Summary-Description.pdf. Accessed 10 September 2018.
- 23.Grob R, Schlesinger M, Parker AM, Shaller D, Barre LR, Martino SC, Finucane ML, Rybowski L, Cerully JL. Breaking narrative ground: innovative methods for rigorously eliciting and assessing patient narratives. Health Serv Res. 2016;51(Suppl 2):1248–72.CrossRefGoogle Scholar
- 24.Montgomery H. Decision rules and the search for a dominance structure: Towards a process model of decision making. Advances in Psychology. 1983;14:343–69.CrossRefGoogle Scholar
- 25.Fowler FJ Jr., Levin CA, Sepucha KR. Informing and involving patients to improve the quality of medical decisions. Health Aff. 2011;30(4):699–706.CrossRefGoogle Scholar
- 26.United States Census Bureau. Statistical abstract of the United States: 2012. Available at: https://www.census.gov/library/publications/2011/compendia/statab/131ed.html. Accessed 10 September 2018.
- 27.Ward BW, Schiller JS, Goodman RA. Multiple chronic conditions among US adults: a 2012 update. Preventing Chronic Disease. 2014;11.Google Scholar
- 28.Blackwell DL, Lucas JW, Clarke TC. Summary health statistics for U.S. adults: national health interview survey, 2012. Vital and Health Statistics, Series 10, Number 260. 2014.Google Scholar
- 29.Associated Press – NORC Center for Public Affairs. Finding quality doctors: how Americans evaluate provider quality in the United States. Available at: http://www.apnorc.org/projects/Pages/finding-quality-doctors-how-americans-evaluate-provider-quality-in-the-united-states.aspx. Accessed 10 September 2018.
- 30.Darby C, Crofton C, Clancy CM. Consumer Assessment of Health Providers and Systems (CAHPS): evolving to meet stakeholder needs. Am J Med Qual. 2006;21(2):144–7.CrossRefGoogle Scholar
- 31.Agency for Healthcare Research and Quality. Research on improving patient experience. Available at: https://www.ahrq.gov/cahps/quality-improvement/reports-and-case-studies/index.html. Accessed 10 September 2018.
- 32.Afendulis CC, Sinaiko AD, Frank RG. Dominated choices and Medicare Advantage enrollment. J Health Econ. 2015;119:72–83.Google Scholar
- 33.Sinaiko AD, Hirth RA. Consumers, health insurance and dominated choices. J Health Econ. 2011;30:450–457.CrossRefGoogle Scholar