Abstract
Given the rising burden of healthcare costs, both patients and healthcare purchasers are interested in discerning which physicians deliver quality care. We proposed a methodology to assess physician clinical performance in preventive cardiology care, and determined a benchmark for minimally acceptable performance. We used data on eight evidence-based clinical measures from 811 physicians that completed the American Board of Internal Medicine’s Preventive Cardiology Practice Improvement ModuleSM to form an overall composite score for preventive cardiology care. An expert panel of nine internists/cardiologists skilled in preventive care for cardiovascular disease used an adaptation of the Angoff standard-setting method and the Dunn-Rankin method to create the composite and establish a standard. Physician characteristics were used to examine the validity of the inferences made from the composite scores. The mean composite score was 73.88 % (SD = 11.88 %). Reliability of the composite was high at 0.87. Specialized cardiologists had significantly lower composite scores (P = 0.04), while physicians who reported spending more time in primary, longitudinal, and preventive consultative care had significantly higher scores (P = 0.01), providing some evidence of score validity. The panel established a standard of 47.38 % on the composite measure with high classification accuracy (0.98). Only 2.7 % of the physicians performed below the standard for minimally acceptable preventive cardiovascular disease care. Of those, 64 % (N = 14) were not general cardiologists. Our study presents a psychometrically defensible methodology for assessing physician performance in preventive cardiology while also providing relative feedback with the hope of heightening physician awareness about deficits and improving patient care.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Angoff, W. H. (1971). Scales, norms, and equivalent scores. In R. L. Thorndike (Ed.), Educational measurement (pp. 514–515). Washington, DC: American Council on Education.
Brennan, R. L. (2006). Educational measurement (4th ed.). Westport, CT: Praeger.
Centers for Disease Control & Prevention. (2013a). Million Hearts™. Retrieved January 1, 2013 from http://millionhearts.hhs.gov/index.html.
Centers for Disease Control & Prevention. (2013b). Strategies for Increasing Adult Vaccination Rates. Retrieved January 1, 2013 from http://www.cdc.gov/vaccines/recs/rate-strategies/adultstrat.htm.
Centers for Medicare & Medicaid Services. (2013). Physician compare initiative. Retrieved January 1, 2013 from www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/physician-compare-initiative/index.html.
Donabedian, A. (1988). The quality of care: How can it be assessed? Journal of the American Medical Association, 260, 1743–1748.
Duffy, F. D., Lynn, L. A., Didura, H., Hess, B., Caverzagie, K., Grosso, L., et al. (2008). Self assessment of practice performance: Development of the ABIM practice improvement module (PIM). Journal of Continuing Education in the Health Professions, 28, 39–46.
Dunn–Rankin, P. (1983). Scaling methods. Hillsdale, NY: Erlbaum.
Henderson, J. T., & Weisman, C. S. (2001). Physician gender effects on preventive screening and counseling: An analysis of male and female patients’ health care experiences. Medical Care, 39, 1281–1292.
Hess, B. J., Weng, W., Holmboe, E. S., & Lipner, R. S. (2012). The association between physicians’ cognitive skills and quality of diabetes care. Academic Medicine, 87, 157–163.
Hess, B. J., Weng, W., Lynn, L. A., Holmboe, E. S., & Lipner, R. S. (2011). Setting a fair performance standard for physicians’ quality of patient care. Journal of General Internal Medicine, 26, 467–473.
Holmboe, E. S., Arnold, G. K., Weng, W., & Lipner, R. (2010). Current yardsticks may be inadequate for measuring quality improvements from the medical home. Health Affairs, 29, 859–866.
Holmboe, E. S., Meehan, T. P., Lynn, L., Doyle, P., Sherwin, T., & Duffy, F. D. (2006). Promoting physicians’ self-assessment and quality improvement: The ABIM diabetes practice improvement module. Journal of Continuing Education in the Health Professions, 26, 109–119.
Kaplan, S. H., Griffith, J. L., Price, L. L., Pawlson, L. G., & Greenfield, S. (2009). Improving the reliability of physician performance assessment: Identifying the “physician effect” on quality and creating composite measures. Medical Care, 47, 378–387.
Landon, B. E., Normand, S. L., Blumenthal, D., & Daley, J. (2003). Physician clinical performance assessment: Prospects and barriers. Journal of the American Medical Association, 290, 1183–1189.
Lee, W.-C., Brennan, R. L., & Wan, L. (2009). Classification consistency and accuracy for complex assessments under the compound multinomial model. Applied Psychological Measurement, 33, 374–390.
Lipner, R. S., Weng, W., Arnold, G. K., Duffy, F. D., Lynn, L. A., & Holmboe, E. S. (2007). A three-part model for measuring diabetes care in physician practice. Academic Medicine, 82(Suppl 10), S48–S52.
Mosier, C. (1943). On the reliability of a weighted composite. Psychometrika, 8, 161–168.
National Committee for Quality Assurance. (2012). Focus on Obesity and on Medicare Plan Improvement: The State of Healthcare Quality 2012. Retrieved January 1, 2013 from http://www.ncqa.org/Portals/0/State%20of%20Health%20Care/2012/SOHC%20Report%20Web.pdf.
Ornstein, S., Jenkins, R. G., Nietert, P. J., Feifer, C., Roylance, L. F., Nemeth, L., et al. (2004). A multi method quality improvement intervention to improve preventive cardiovascular care. Annals of Internal Medicine, 141, 523–532.
Reeves, D., Campbell, S. M., Adams, J., Shekelle, P. G., Kontopantelis, E., & Roland, M. O. (2007). Combining multiple indicators of clinical quality: An evaluation of different analytic approaches. Medical Care, 45, 489–496.
Scholle, S. H., Pawlson, L. G., Solberg, L. I., Shih, S. C., Asche, S. E., Chou, A. F., et al. (2008). Measuring practice systems for chronic illness care: Accuracy of self-reports from clinical personnel. Joint Commission Journal of Quality &Patient Safety, 34, 407–416.
Weng, W., Hess, B. J., Lynn, L. A., Holmboe, E. S., & Lipner, R. S. (2010). Measuring physicians’ performance in clinical practice: Reliability, classification accuracy, and validity. Evaluation and the Health Professions, 33, 302–320.
Acknowledgments
We thank the standard-setting panel and Halyna Didura for her assistance in preparing the data.
Conflict of interest
Drs. Hess, Weng, and Lipner are employees of the ABIM and co-inventors of a business method invention describing the application of the standard-setting method to practicing physicians. The invention is patent pending. Dr. Caverzagie has a small portion of his salary reimbursed by the ABIM for efforts related to PIM development and maintenance.
Author information
Authors and Affiliations
Corresponding author
Appendix
Rights and permissions
About this article
Cite this article
Lipner, R.S., Weng, W., Caverzagie, K.J. et al. Physician performance assessment: prevention of cardiovascular disease. Adv in Health Sci Educ 18, 1029–1045 (2013). https://doi.org/10.1007/s10459-013-9447-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10459-013-9447-7