Abstract
The objective of this study was to determine whether there is enough variation in utilization of medical services which can be reliably attributed to primary care providers to justify practice profiling. A total of 221 primary care providers caring for 81,775 patients with a variety of medical conditions in eight Veterans Affairs medical centers and their associated clinics during calendar year 1999 were used for the study. After controlling for case-mix variation, the physician-level variation in utilization among providers who had at least 75 patients was evaluated. Adjusted Clinical Groups (ACGs) were used for case-mix adjustment. Physician effect was measured by the intraclass correlation coefficient (ICC). Patient-level utilization outcomes were total hospital bed days of care, laboratory costs, radiology costs, and pharmacy costs for each patient during calendar year 1999. The physician effect (ICC) ranged from .04 for bed days to .12 for laboratory costs. The reliabilities of the physician effect estimates computed using the Spearman-Brown prediction formula ranged from .75 for bed days to .91 for laboratory costs for providers with 75 or more patients. The profiling methodology reliably detected practice differences among providers in all utilization outcomes studied. However, the value of profiling as a means of improving quality of care or efficiency is yet untested.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Aguilar, O. and West, M. “Analysis of hospital quality monitors using hierarchical time series models,” in Case studies in Bayesian Statistics (C. Gatsonis, R.E. Kass, and C. Carlin, eds.), Springer, New York, 1999.
Bindman, A. “Can physician's profiles be trusted (editorial)?” JAMA 281(22): 2142-2143, 1999.
Borowski, S.J. and Cowper, D.C. “Dual use of VA and non-VA primary care,” J. Gen. Intern. Med. 14, 274-280, 1999.
Bravo, C. and Potvin, L. “Estimating the reliability of continuous measures with Cronbach's alpha or the intraclass correlation coefficient: Toward the integration of two traditions,” J. Clinical Epidemiology 44, 381-390, 1991.
Christiansen, C.L. and Morris, C.N. “Fitting and checking a two-level Poisson model: Modeling patient mortality rates in heart transplant patients,” in Bayesian biostatistics (D.A. Berry and D.K. Stangl, eds.), Marcel Dekker, New York, 1996.
Christiansen, C.L. and Morris, C.N. “Improving the statistical approach to health care provider profiling,” Ann. Int. Med. 127, 764-768, 1997.
Crocker, L. and Algina, J. Introduction to classical and modern test theory, Harcourt Brace Janovitch College Publishers, Fort Worth, TX, 1986.
Daniels, M.J. and Gatsonis, C. “Hierarchical polytomous regression models with applications to health services research,” Stat. in Med. 16, 2311-2326, 1997.
Draper, N.R. and Smith, H. Applied regression analysis, 2nd ed., Wiley, New York, 1981.
Gatsonis, C.A., Epstein, A.M., Newhouse, J.P., Normand, S.-L., and McNeil, B.J. “Variations in the utilization of coronary angiography for elderly patients with an acute myocardial infarction:Ananalysis using hierarchical logistic regression,” Med. Care 33, 625-642, 1995.
Goldstein, H. Multilevel statistical models, 2nd ed., Edward Arnold, London, 1995.
Goldstein, H. and Spiegelhalter, D. “League tables and their limitations: Statistical issues in comparisons of institutional performance,” J. Roy. Stat. Soc., Series A. 159, 385-444, 1996.
Hofer, T.P., Hayward, R.A., Greenfield, S., Wagner, E.H. Kaplan, S.H., and Manning, W.G. “The unreliability of individual physician 'report cards' for assessing the costs and quality of care of a chronic disease,” JAMA 281(22), 2098-2105, 1999.
The Johns Hopkins University ACG Case Mix Adjustment System, Version 4.1, Johns Hopkins University, Baltimore, MD, 1998.
Iezzoni, L.I., Daley, J., Schwartz, M., Hughes, J.S., and Ash, A.S. Risk adjustment for measuring healthcare outcomes, 2nd ed. Health Administration Press, Ann Arbor, MI. 1997.
Kackar, R.N. and Harville, D.A. “Approximations for standard errors of estimators of fixed and random effects in mixed linear models,” J. Am. Stat. Assn. 79(388), 853-862, 1984.
Kass, R.A. and Steffey, D. “Approximate Bayesian inference in conditionally independent hierarchical models (parametric empirical Bayes model),” J. Am. Stat. Assn. 84(407), 717-726, 1989.
Kassirer, J.P. “The use and abuse of medical profiles,” N. Engl. J. Med. 330, 634-636, 1994.
Khuri, S.F., Daley, J., Henderson, W., Hur, K., Gibbs, J.O., Barbour, G., Demakis, J., Irvin, G., Stremple, J.F., Grover, F., McDonald, G., Passaro, E., Fabri, P.J., Spencer, J., Hammermeister, K., and Aust, J.B. “Risk adjustment of the postoperative mortality rate for the comparative assessment of the quality of surgical care: Results of the national Veterans Affairs surgical risk study,” J. Am. Coll. Surg. 185(4), 315-327, 1997.
Kiefe, C.I., Weissman, N.W., Allison, J.J., Farmer, F., Weaver, M., and Williams, O.D. “Identifying achievable benchmarks of care: Concepts and methodology,” Int. J. Qual. Health Care 10(5), 443-447, 2000.
McCulloch, C.E. and Searle, S.R. Generalized, linear, and mixed models, Wiley, New York, 2001.
Normand, S.-L., Glickman, M.E., and Gatsonis, C. “Statistical models for profiling providers of medical care: Issues and applications,” J. Am. Stat. Assn. 92, 803-814, 1997.
Normand, S.-L., Glickman, M.E., and Ryan, T. “Modeling mortality rates for elderly heart attack patients: Pro-filing hospitals in the Cooperative Cardiovascular Project,” in Case studies in bayesian statistics (C. Gatsonis, J. Hodger, R. Kass, and N. Singpurwalla, eds.), Springer, pp. 155-236, 1997.
Nunnaly, J.C. Psychometric theory, McGraw-Hill, New York, 1978.
Oxman, A.D., Thomson, M.A., Davis, D.A., and Haynes, R.B. “No magic bullets: A systematic review of 102 trials of interventions to improve professional practice,” JAMA 153, 1423-1431, 1995.
Parente, S.T., Weiner, J.P., Garnick, D.W., Fowles, J., Lawthers, G., and Palmer, R.H. “Profiling resource use by primary-care practices: Managed Medicare implications,” Health Care Financing Review 17(4), 23-42, 1996.
Prasad, N.G.N. and Rao, J.N.K. “The estimation of the mean squared error of small-area estimators,” J. Am. Stat. Assn. 85(409), 163-171, 1990.
Randall, M., Kilpatrick, K.E., Pendergast, J.F., Jones, K.R., and Vogel, W.B. “Differences in patient characteristics between Veterans Administration and community hospitals,” Med. Care 25(11), 1099-1110, 1987.
Salem-Schatz, S., Moore, G., Rucker, M., and Pearson, S.D. “The case for case-mix adjustment in practice profiling. When good apples look bad,” JAMA 272(11), 871-874, 1994.
SAS User's Guide, Version 8, SAS Institute, Cary, NC, 2000.
Starfield, B., Weiner, J., Mumford, L., and Steinwachs, D. “Ambulatory Care Groups: A categorization of diagnoses for research and management,” Health Care Financing Review 26(1), 53-74, 1991.
Verbeke, G. and Molenberghs, G. eds. Linear mixed models in practice, 2nd ed. Springer, New York, 1997.
Weiner, J., Dobson, A., Maxwell, S., Coleman, K., Starfield, B., and Anderson, J. “Risk-adjusted Medicare capitation rates using ambulatory and inpatient diagnoses,” Health Care Financing Review 17(3) 77-99, 1996.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Pietz, K., O'Malley, K.J., Byrne, M. et al. Physician-Level Variation in Practice Patterns in the VA Healthcare System. Health Services & Outcomes Research Methodology 3, 95–106 (2002). https://doi.org/10.1023/A:1024257107013
Issue Date:
DOI: https://doi.org/10.1023/A:1024257107013