ABSTRACT
BACKGROUND
Although benefits of performance measurement (PM) systems have been well documented, there is little research on negative unintended consequences of performance measurement systems in primary care. To optimize PM systems, a better understanding is needed of the types of negative unintended consequences that occur and of their causal antecedents.
OBJECTIVES
(1) Identify unintended negative consequences of PM systems for patients. (2) Develop a conceptual framework of hypothesized relationships between PM systems, facility-level variables (local implementation strategies, primary care staff attitudes and behaviors), and unintended negative effects on patients.
DESIGN, PARTICIPANTS, APPROACH
Qualitative study design using dissimilar cases sampling. A series of 59 in-person individual semi-structured interviews at four Veterans Health Administration (VHA) facilities was conducted between February and July 2009. Participants included members of primary care staff and facility leaders. Sites were selected to assure variability in the number of veterans served and facility scores on national VHA performance measures. Interviews were recorded, transcribed and content coded to identify thematic categories and relationships.
RESULTS
Participants noted both positive effects and negative unintended consequences of PM. We report three negative unintended consequences for patients. Performance measurement can (1) lead to inappropriate clinical care, (2) decrease provider focus on patient concerns and patient service, and (3) compromise patient education and autonomy. We also illustrate examples of negative consequences on primary care team dynamics. In many instances these problems originate from local implementation strategies developed in response to national PM definitions and policies.
CONCLUSIONS
Facility-level strategies undertaken to implement national PM systems may result in inappropriate clinical care, can distract providers from patient concerns, and may have a negative effect on patient education and autonomy. Further research is needed to ascertain how features of centralized PM systems influence whether measures are translated locally by facilities into more or less patient-centered policies and processes.
Similar content being viewed by others
REFERENCES
Asch SM, McGlynn EA, Hogan MM, Hayward RA, Shekelle P, Rubenstein L, et al. Comparison of quality of care for patients in the Veterans Health Administration and patients in a national sample. Ann Intern Med. 2004;141(12):938–945.
Campbell SM, Reeves D, Kontopantelis E, Sibbald B, Roland M. Effects of pay for performance on the quality of primary care in England. N Engl J Med. 2009;361(4):368–378.
Lindenauer PK, Remus D, Roman S, Rothberg MB, Benjamin EM, Ma A, et al. Public reporting and pay for performance in hospital quality improvement. N Engl J Med. 2007;356(5):486–496.
Rosenthal MB, Frank RG, Li Z, Epstein AM. Early experience with pay-for-performance: from concept to practice. JAMA. 2005;294(14):1788–1793.
Davies M, Spears W, Pugh J. What VA providers really think about clinical practice guidelines. Federal Practitioner. 2004;15.
Hayward RA. Performance measurement in search of a path. N Engl J Med. 2007;356(9):951–953.
Jha AK, Perlin JB, Kizer KW, Dudley RA. Effect of the transformation of the Veterans Affairs Health Care System on the quality of care. N Engl J Med. 2003;348(22):2218–2227.
Casalino LP. The unintended consequences of measuring quality on the quality of medical care. N Engl J Med. 1999;341(15):1147–1150.
Sheldon T. Promoting health care quality: what role performance indicators? Qual Health Care. 1998;7(Suppl):S45–S50.
Wachter RM, Flanders SA, Fee C, Pronovost PJ. Public reporting of antibiotic timing in patients with pneumonia: lessons from a flawed performance measure. Ann Intern Med. 2008;149(1):29–32.
Snyder L, Neubauer RL. Pay-for-performance principles that promote patient-centered care: an ethics manifesto. Ann Intern Med. 2007;147(11):792–794.
Eddy DM. Performance measurement: problems and solutions. Health Aff (Millwood). 1998;17(4):7–25.
Boyd CM, Darer J, Boult C, Fried LP, Boult L, Wu AW. Clinical practice guidelines and quality of care for older patients with multiple comorbid diseases: implications for pay for performance. JAMA. 2005;294(6):716–724.
McDonald R, White J, Marmor TR. Paying for performance in primary medical care: learning about and learning from "success" and "failure" in England and California. J Health Polit Policy Law. 2009;34(5):747–776.
Blumenthal D. The vital role of professionalism in health care reform. Health Aff (Millwood). 1994;13(1):252–256.
Bokhour BG, Burgess JF Jr, Hook JM, White B, Berlowitz D, Guldin MR, et al. Incentive implementation in physician practices: A qualitative study of practice executive perspectives on pay for performance. Med Care Res Rev. 2006;63(1 Suppl):73S–95S.
Fisher ES. Paying for performance–risks and recommendations. N Engl J Med. 2006;355(18):1845–1847.
Sittig DF, Krall MA, Dykstra RH, Russell A, Chin HL. A survey of factors affecting clinician acceptance of clinical decision support. BMC Med Inform Decis Mak. 2006;6:6.
Bindman AB. Can physician profiles be trusted? JAMA. 1999;281(22):2142–2143.
Chassin MR, Loeb JM, Schmaltz SP, Wachter RM. Accountability measures–using measurement to promote quality improvement. N Engl J Med. 2010;363(7):683–688.
Shen Y. Selection incentives in a performance-based contracting system. Health Serv Res. 2003;38(2):535–552.
Werner RM, Asch DA, Polsky D. Racial profiling: the unintended consequences of coronary artery bypass graft report cards. Circulation. 2005;111(10):1257–1263.
Patton MQ. Qualitative Research and Evaluation Methods. 3rd ed. Thousand Oaks, CA: Sage; 2002.
Bradley EH, Curry LA, Devers KJ. Qualitative data analysis for health services research: developing taxonomy, themes, and theory. Health Serv Res. 2007;42(4):1758–72.
Lipton HL, Bero LA, Bird JA, McPhee SJ. The impact of clinical pharmacists' consultations on physicians' geriatric drug prescribing. A randomized controlled trial. Med Care. 1992;30(7):646–658.
Jyrkka J, Enlund H, Korhonen MJ, Sulkava R, Hartikainen S. Polypharmacy status as an indicator of mortality in an elderly population. Drugs Aging. 2009;26(12):1039–1048.
Flaherty JH, Perry HM III, Lynchard GS, Morley JE. Polypharmacy and hospitalization among older home care patients. J Gerontol A Biol Sci Med Sci. 2000;55(10):M554–M559.
Espino DV, Bazaldua OV, Palmer RF, Mouton CP, Parchman ML, Miles TP, et al. Suboptimal medication use and mortality in an older adult community-based cohort: results from the Hispanic EPESE Study. J Gerontol A Biol Sci Med Sci. 2006;61(2):170–175.
Iwata M, Kuzuya M, Kitagawa Y, Suzuki Y, Iguchi A. Underappreciated predictors for postdischarge mortality in acute hospitalized oldest-old patients. Gerontology. 2006;52(2):92–98.
Thomas WI, Thomas DS. The child in America: Behavior problems and programs. New York: Knopf; 1928:571–2.
Wilkinson EK, McColl A, Exworthy M, Roderick P, Smith H, Moore M, et al. Reactions to the use of evidence-based performance indicators in primary care: a qualitative study. Qual Health Care. 2000;9(3):166–174.
Kerr EA, Krein SL, Vijan S, Hofer TP, Hayward RA. Avoiding pitfalls in chronic disease quality measurement: a case for the next generation of technical quality measures. Am J Manag Care. 2001;7(11):1033–1043.
Kerr EA, Smith DM, Hogan MM, Hofer TP, Krein SL, Bermann M, et al. Building a better quality measure: are some patients with 'poor quality' actually getting good care? Med Care. 2003;41(10):1173–1182.
Epstein AM. Paying for performance in the United States and abroad. N Engl J Med. 2006;355(4):406–408.
Doran T, Fullwood C, Gravelle H, Reeves D, Kontopantelis E, Hiroeh U, et al. Pay-for-performance programs in family practices in the United Kingdom. N Engl J Med. 2006;355(4):375–384.
National Committee for Quality Assurance (NCQA) Web site. Available at: http://www.ncqa.org/tabid/187/Default.aspx. Accessed February 21, 2011.
Acknowledgements
We gratefully acknowledge Joe Francis (Veterans Health Affairs Chief Quality and Performance Officer) for providing comments on a previous draft of this manuscript.
Conflict of Interest
None disclosed.
Funding/Support
This work was supported by a Veterans Administration Health Services research grant (IIR-07-140).
Role of the Sponsors
The funding organizations had no role in the design and conduct of the study, in the collection, analysis, and interpretation of the data, or in the preparation, review, or approval of the manuscript.
Author information
Authors and Affiliations
Corresponding author
Electronic supplementary material
Below is the link to the electronic supplementary material.
ESM 1
(PDF 222 KB)
Rights and permissions
About this article
Cite this article
Powell, A.A., White, K.M., Partin, M.R. et al. Unintended Consequences of Implementing a National Performance Measurement System into Local Practice. J GEN INTERN MED 27, 405–412 (2012). https://doi.org/10.1007/s11606-011-1906-3
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11606-011-1906-3