Skip to main content

Advertisement

Log in

Unintended Consequences of Implementing a National Performance Measurement System into Local Practice

  • Original Research
  • Published:
Journal of General Internal Medicine Aims and scope Submit manuscript

ABSTRACT

BACKGROUND

Although benefits of performance measurement (PM) systems have been well documented, there is little research on negative unintended consequences of performance measurement systems in primary care. To optimize PM systems, a better understanding is needed of the types of negative unintended consequences that occur and of their causal antecedents.

OBJECTIVES

(1) Identify unintended negative consequences of PM systems for patients. (2) Develop a conceptual framework of hypothesized relationships between PM systems, facility-level variables (local implementation strategies, primary care staff attitudes and behaviors), and unintended negative effects on patients.

DESIGN, PARTICIPANTS, APPROACH

Qualitative study design using dissimilar cases sampling. A series of 59 in-person individual semi-structured interviews at four Veterans Health Administration (VHA) facilities was conducted between February and July 2009. Participants included members of primary care staff and facility leaders. Sites were selected to assure variability in the number of veterans served and facility scores on national VHA performance measures. Interviews were recorded, transcribed and content coded to identify thematic categories and relationships.

RESULTS

Participants noted both positive effects and negative unintended consequences of PM. We report three negative unintended consequences for patients. Performance measurement can (1) lead to inappropriate clinical care, (2) decrease provider focus on patient concerns and patient service, and (3) compromise patient education and autonomy. We also illustrate examples of negative consequences on primary care team dynamics. In many instances these problems originate from local implementation strategies developed in response to national PM definitions and policies.

CONCLUSIONS

Facility-level strategies undertaken to implement national PM systems may result in inappropriate clinical care, can distract providers from patient concerns, and may have a negative effect on patient education and autonomy. Further research is needed to ascertain how features of centralized PM systems influence whether measures are translated locally by facilities into more or less patient-centered policies and processes.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Figure 1

Similar content being viewed by others

REFERENCES

  1. Asch SM, McGlynn EA, Hogan MM, Hayward RA, Shekelle P, Rubenstein L, et al. Comparison of quality of care for patients in the Veterans Health Administration and patients in a national sample. Ann Intern Med. 2004;141(12):938–945.

    PubMed  Google Scholar 

  2. Campbell SM, Reeves D, Kontopantelis E, Sibbald B, Roland M. Effects of pay for performance on the quality of primary care in England. N Engl J Med. 2009;361(4):368–378.

    Article  PubMed  CAS  Google Scholar 

  3. Lindenauer PK, Remus D, Roman S, Rothberg MB, Benjamin EM, Ma A, et al. Public reporting and pay for performance in hospital quality improvement. N Engl J Med. 2007;356(5):486–496.

    Article  PubMed  CAS  Google Scholar 

  4. Rosenthal MB, Frank RG, Li Z, Epstein AM. Early experience with pay-for-performance: from concept to practice. JAMA. 2005;294(14):1788–1793.

    Article  PubMed  CAS  Google Scholar 

  5. Davies M, Spears W, Pugh J. What VA providers really think about clinical practice guidelines. Federal Practitioner. 2004;15.

  6. Hayward RA. Performance measurement in search of a path. N Engl J Med. 2007;356(9):951–953.

    Article  PubMed  CAS  Google Scholar 

  7. Jha AK, Perlin JB, Kizer KW, Dudley RA. Effect of the transformation of the Veterans Affairs Health Care System on the quality of care. N Engl J Med. 2003;348(22):2218–2227.

    Article  PubMed  Google Scholar 

  8. Casalino LP. The unintended consequences of measuring quality on the quality of medical care. N Engl J Med. 1999;341(15):1147–1150.

    Article  PubMed  CAS  Google Scholar 

  9. Sheldon T. Promoting health care quality: what role performance indicators? Qual Health Care. 1998;7(Suppl):S45–S50.

    PubMed  Google Scholar 

  10. Wachter RM, Flanders SA, Fee C, Pronovost PJ. Public reporting of antibiotic timing in patients with pneumonia: lessons from a flawed performance measure. Ann Intern Med. 2008;149(1):29–32.

    PubMed  Google Scholar 

  11. Snyder L, Neubauer RL. Pay-for-performance principles that promote patient-centered care: an ethics manifesto. Ann Intern Med. 2007;147(11):792–794.

    PubMed  Google Scholar 

  12. Eddy DM. Performance measurement: problems and solutions. Health Aff (Millwood). 1998;17(4):7–25.

    Article  CAS  Google Scholar 

  13. Boyd CM, Darer J, Boult C, Fried LP, Boult L, Wu AW. Clinical practice guidelines and quality of care for older patients with multiple comorbid diseases: implications for pay for performance. JAMA. 2005;294(6):716–724.

    Article  PubMed  CAS  Google Scholar 

  14. McDonald R, White J, Marmor TR. Paying for performance in primary medical care: learning about and learning from "success" and "failure" in England and California. J Health Polit Policy Law. 2009;34(5):747–776.

    Article  PubMed  Google Scholar 

  15. Blumenthal D. The vital role of professionalism in health care reform. Health Aff (Millwood). 1994;13(1):252–256.

    Article  CAS  Google Scholar 

  16. Bokhour BG, Burgess JF Jr, Hook JM, White B, Berlowitz D, Guldin MR, et al. Incentive implementation in physician practices: A qualitative study of practice executive perspectives on pay for performance. Med Care Res Rev. 2006;63(1 Suppl):73S–95S.

    Article  PubMed  Google Scholar 

  17. Fisher ES. Paying for performance–risks and recommendations. N Engl J Med. 2006;355(18):1845–1847.

    Article  PubMed  CAS  Google Scholar 

  18. Sittig DF, Krall MA, Dykstra RH, Russell A, Chin HL. A survey of factors affecting clinician acceptance of clinical decision support. BMC Med Inform Decis Mak. 2006;6:6.

    Article  PubMed  Google Scholar 

  19. Bindman AB. Can physician profiles be trusted? JAMA. 1999;281(22):2142–2143.

    Article  PubMed  CAS  Google Scholar 

  20. Chassin MR, Loeb JM, Schmaltz SP, Wachter RM. Accountability measures–using measurement to promote quality improvement. N Engl J Med. 2010;363(7):683–688.

    Article  PubMed  CAS  Google Scholar 

  21. Shen Y. Selection incentives in a performance-based contracting system. Health Serv Res. 2003;38(2):535–552.

    Article  PubMed  Google Scholar 

  22. Werner RM, Asch DA, Polsky D. Racial profiling: the unintended consequences of coronary artery bypass graft report cards. Circulation. 2005;111(10):1257–1263.

    Article  PubMed  Google Scholar 

  23. Patton MQ. Qualitative Research and Evaluation Methods. 3rd ed. Thousand Oaks, CA: Sage; 2002.

    Google Scholar 

  24. Bradley EH, Curry LA, Devers KJ. Qualitative data analysis for health services research: developing taxonomy, themes, and theory. Health Serv Res. 2007;42(4):1758–72.

    Article  PubMed  Google Scholar 

  25. Lipton HL, Bero LA, Bird JA, McPhee SJ. The impact of clinical pharmacists' consultations on physicians' geriatric drug prescribing. A randomized controlled trial. Med Care. 1992;30(7):646–658.

    Article  PubMed  CAS  Google Scholar 

  26. Jyrkka J, Enlund H, Korhonen MJ, Sulkava R, Hartikainen S. Polypharmacy status as an indicator of mortality in an elderly population. Drugs Aging. 2009;26(12):1039–1048.

    Article  PubMed  Google Scholar 

  27. Flaherty JH, Perry HM III, Lynchard GS, Morley JE. Polypharmacy and hospitalization among older home care patients. J Gerontol A Biol Sci Med Sci. 2000;55(10):M554–M559.

    Article  PubMed  CAS  Google Scholar 

  28. Espino DV, Bazaldua OV, Palmer RF, Mouton CP, Parchman ML, Miles TP, et al. Suboptimal medication use and mortality in an older adult community-based cohort: results from the Hispanic EPESE Study. J Gerontol A Biol Sci Med Sci. 2006;61(2):170–175.

    Article  PubMed  Google Scholar 

  29. Iwata M, Kuzuya M, Kitagawa Y, Suzuki Y, Iguchi A. Underappreciated predictors for postdischarge mortality in acute hospitalized oldest-old patients. Gerontology. 2006;52(2):92–98.

    Article  PubMed  Google Scholar 

  30. Thomas WI, Thomas DS. The child in America: Behavior problems and programs. New York: Knopf; 1928:571–2.

    Google Scholar 

  31. Wilkinson EK, McColl A, Exworthy M, Roderick P, Smith H, Moore M, et al. Reactions to the use of evidence-based performance indicators in primary care: a qualitative study. Qual Health Care. 2000;9(3):166–174.

    Article  PubMed  CAS  Google Scholar 

  32. Kerr EA, Krein SL, Vijan S, Hofer TP, Hayward RA. Avoiding pitfalls in chronic disease quality measurement: a case for the next generation of technical quality measures. Am J Manag Care. 2001;7(11):1033–1043.

    PubMed  CAS  Google Scholar 

  33. Kerr EA, Smith DM, Hogan MM, Hofer TP, Krein SL, Bermann M, et al. Building a better quality measure: are some patients with 'poor quality' actually getting good care? Med Care. 2003;41(10):1173–1182.

    Article  PubMed  Google Scholar 

  34. Epstein AM. Paying for performance in the United States and abroad. N Engl J Med. 2006;355(4):406–408.

    Article  PubMed  CAS  Google Scholar 

  35. Doran T, Fullwood C, Gravelle H, Reeves D, Kontopantelis E, Hiroeh U, et al. Pay-for-performance programs in family practices in the United Kingdom. N Engl J Med. 2006;355(4):375–384.

    Article  PubMed  CAS  Google Scholar 

  36. National Committee for Quality Assurance (NCQA) Web site. Available at: http://www.ncqa.org/tabid/187/Default.aspx. Accessed February 21, 2011.

Download references

Acknowledgements

We gratefully acknowledge Joe Francis (Veterans Health Affairs Chief Quality and Performance Officer) for providing comments on a previous draft of this manuscript.

Conflict of Interest

None disclosed.

Funding/Support

This work was supported by a Veterans Administration Health Services research grant (IIR-07-140).

Role of the Sponsors

The funding organizations had no role in the design and conduct of the study, in the collection, analysis, and interpretation of the data, or in the preparation, review, or approval of the manuscript.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Adam A. Powell PhD.

Electronic supplementary material

Below is the link to the electronic supplementary material.

ESM 1

(PDF 222 KB)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Powell, A.A., White, K.M., Partin, M.R. et al. Unintended Consequences of Implementing a National Performance Measurement System into Local Practice. J GEN INTERN MED 27, 405–412 (2012). https://doi.org/10.1007/s11606-011-1906-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11606-011-1906-3

KEY WORDS

Navigation