Optimal Bayesian probability levels for hospital report cards



There is a growing trend towards the production of “hospital report-cards” in which hospitals with higher than acceptable mortality rates are identified. Several commentators have advocated for the use of Bayesian hierarchical models in provider profiling. These methods are frequently based upon the posterior probability that a hospital’s mortality rate exceeds a specific benchmark. However, the minimum probability level required for classifying a hospital as having higher than acceptable mortality has never been formally justified. We developed Bayes Rules for determining optimal probability levels so as to minimize mean posterior costs associated with false classifications under specific loss functions. Using Monte Carlo simulation methods we then determined the ability of posterior tail probabilities of unacceptable performance to accurately identify hospitals with higher than acceptable mortality.


Provider profiling Hospital mortality Outcome assessment Bayesian models Posterior inference Quality of care Hospital report cards Decision analysis Bayes Rules 


  1. Alter, D.A., Naylor, C.D., Austin, P.C., Tu, J.V.: Long-term MI outcomes at hospitals with or without on-site revascularization. J. Am. Med. Assoc. 285, 2101–2108 (2001).CrossRefGoogle Scholar
  2. Austin, P.C.: A comparison of Bayesian methods for profiling hospital performance. Med. Decis. Making 22, 163–172 (2002).PubMedCrossRefGoogle Scholar
  3. Austin, P.C.: The reliability and validity of Bayesian methods for hospital profiling: a Monte Carlo assessment. J. Stat. Plan. Infer. 128, 109–122 (2005).CrossRefGoogle Scholar
  4. Austin, P.C., Anderson, G.M.: Optimal statistical decisions for hospital report cards. Med. Decis. Making 25, 11–19 (2005).PubMedCrossRefGoogle Scholar
  5. Austin, P.C., Alter, D.A., Tu, J.V.: The accuracy of fixed and random effects models in calculating risk-adjusted mortality rates: a Monte Carlo assessment. Med. Decis. Making 23, 526–539 (2003).PubMedCrossRefGoogle Scholar
  6. Berger, J.O.: Statistical Decision Theory and Bayesian Analysis. Springer-Verlag, New York, NY (1980).Google Scholar
  7. Burgess, J.F. Jr., Christiansen, C.L., Michalak, S.E., Morris, C.N.: Medical profiling: improving standards and risk adjustments using hierarchical models. J. Health Econ. 19, 291–309 (2000).PubMedCrossRefGoogle Scholar
  8. Christiansen, C.L., Morris, C.N.: Improving the statistical approach to health care provider profiling. Ann. Intern. Med. 127, 764–768 (1997).PubMedGoogle Scholar
  9. DeGroot, M.H.: Optimal Statistical Decisions. McGraw-Hill, New York (1970).Google Scholar
  10. Geweke, J.: Evaluating the accuracy of sampling-based approaches to calculating posterior moments. In: Bernardo, J.M., Berger, J.O., Dawid, A.P., Smith, A.F. (eds.) Bayesian Statistics 4, pp. 169–193. Clarendon Press, Oxford (1994).Google Scholar
  11. Gilks, W.R., Thomas, A., Spiegelhalter, D.J.: A language and program for complex Bayesian modelling. Statistician 43, 169–178 (1994).CrossRefGoogle Scholar
  12. Gilks, W.R., Richardson, S., Spiegelhalter, D.J.: Introducing Markov chain Monte Carlo. In: Gilks, W.R., Richardson, S., Spiegelhalter, D.J. (eds.) Markov Chain Monte Carlo in Practice, pp. 1–19. Chapman & Hall, London (1996).Google Scholar
  13. Goldstein, H.: Statistical information and the measurement of educational outcomes. J. R. Stat. Soc. A 155, 313–315 (1992).Google Scholar
  14. Goldstein, H., Spiegelhalter, D.J.: League tables and their limitations: statistical issues in comparisons of institutional performance. J. R. Stat. Soc. A 159, 385–443 (1996).CrossRefGoogle Scholar
  15. Goldstein, H., Rasbash, J., Yang, M., Woodhouse, G., Pan, H., Nuttal, D., Thomas, S.: A multilevel analysis of school examination results. Oxford Rev. Educ. 19, 425–433 (1993).CrossRefGoogle Scholar
  16. Hofer, T.P., Hayward, R.A.: Identifying poor-quality hospitals. Can hospital mortality rates detect quality problems for medical diagnoses? Med. Care 34, 737–753 (1996).PubMedCrossRefGoogle Scholar
  17. Jacobs, F.M.: Cardiac Surgery in New Jersey in 2002: A Consumer Report. New Jersey Department of Health and Senior Services, Trenton (2005).Google Scholar
  18. Localio, A.R., Hamory, B.H., Cocks Fisher, A., TenHave, T.R.: The public release of hospital and physician mortality data in Pennsylvania. A Case Study. Med. Care 35, 272–286 (1997).PubMedCrossRefGoogle Scholar
  19. Luft, H.S., Romano, P.S., Remy, L.L., Rainwater, J.: Annual Report of the California Hospital Outcomes Project. California Office of Statewide Health Planning and Development, Sacramento (1993).Google Scholar
  20. Massachusetts Data Analysis Center: Adult Coronary Artery Bypass Graft Surgery in the Commonwealth of Massachusetts, January 1–December 31, 2002. Department of Health Care Policy, Harvard Medical School, Boston (2004).Google Scholar
  21. McGlynn, E.A.: Introduction and overview of the conceptual framework for a national quality measurement and reporting system. Med. Care 41, I-1–I-7 (2003).Google Scholar
  22. MRC Biostatistics Unit: BUGS Version 0.510. Cambridge, United Kingdom (1995).Google Scholar
  23. Naylor, C.D., Rothwell, D.M., Tu, J.V., Austin, P.C., The Cardiac Care Network Steering Committee: Outcomes of coronary artery bypass graft surgery in Ontario. In: Naylor, C.D., Slaugher, P.M. (eds.) Cardiovascular Health Services in Ontario: An ICES Atlas, pp. 189–197. Institute for Clinical Evaluative Sciences, Toronto, Canada (1999).Google Scholar
  24. New York State Department of Health: Coronary Artery Bypass Graft Surgery in New York State 1989–1991. New York State Department of Health, Albany, New York (1992).Google Scholar
  25. Normand, S.L., Glickman, M.E., Gatsonis, C.A.: Statistical methods for profiling providers of medical care: issues and applications. J. Am. Stat. Assoc. 92, 803–814 (1997).CrossRefGoogle Scholar
  26. Pennsylvania Health Care Cost Containment Council: A Consumer Guide to Coronary Artery Bypass Graft Surgery, vol. 4. Pennsylvania Health Care Cost Containment Council, Harrisburg (1995).Google Scholar
  27. Pennsylvania Health Care Cost Containment Council: Focus on Heart Attack in Pennsylvania. Research Methods and Results. Pennsylvania Health Care Cost Containment Council, Harrisburg (1996).Google Scholar
  28. Raudenbush, S.W., Bryk, A.S.: Hierarchical Linear Models: Applications and Data Analysis Methods, 2nd ed. Sage Publications, Thousand Oaks (2002).Google Scholar
  29. Scottish Office: Clinical Outcome Indicators, 1994. Clinical Resource and Audit Group, Edinburgh (1995).Google Scholar
  30. Spiegelhalter, D.J., Aylin, P., Best, N.G., Evans, S.J.W., Murray, G.D.: Commissioned analysis of surgical performance using routine data: lessons from the Bristol inquiry. J. R. Stat. Soc. A 165, 191–231 (2002).CrossRefGoogle Scholar
  31. Thomas, J.W., Hofer, T.P.: Accuracy of risk-adjusted mortality rates as a measure of hospital quality of care. Med. Care 37, 83–92 (1999).PubMedCrossRefGoogle Scholar
  32. Thomas, N., Longford, N.T., Rolph, J.E.: Empirical Bayes methods for estimating hospital-specific mortality rates. Stat. Med. 13, 889–903 (1994).PubMedCrossRefGoogle Scholar
  33. Tu, J.V., Pashos, C.L., Naylor, C.D., Chen, E., Normand, S.L., Newhouse, J.P., McNeil, B.J.: Use of cardiac procedures and outcomes in elderly patients with myocardial infarction in the United States and Canada. New Engl. J. Med. 336, 1500–1505 (1997).PubMedCrossRefGoogle Scholar
  34. Tu, J.V., Austin, P., Naylor, C.D., Iron, K., Zhang, H.: Acute myocardial infarction outcomes in Ontario. In: Naylor, C.D., Slaugher, P.M. (eds.) Cardiovascular Health Services in Ontario: An ICES Atlas, pp. 83–110. Institute for Clinical Evaluative Sciences, Toronto (1999a).Google Scholar
  35. Tu, J.V., Naylor, C.D., Austin, P.: Temporal changes in the outcomes of acute myocardial infarction in Ontario, 1992–96. Can. Med. Assoc. J. 161, 1257–1261 (1999b).Google Scholar
  36. Tu, J.V., Austin, P.C., Walld, R., Roos, L., Agras, J., McDonald, K.M.: Development and validation of the ontario acute myocardial mortality prediction rules. J. Am. College Cardiol. 37, 992–997 (2001a).CrossRefGoogle Scholar
  37. Tu, J.V., Austin, P.C., Chan, B.T.B.: Relationship between annual volume of patients treated by admitting physician and mortality after acute myocardial infarction. J. Am. Med. Assoc. 285, 3116–3122 (2001b).CrossRefGoogle Scholar
  38. Wennberg, D.E., Birkmeyer, J.D. (eds).: The Dartmouth Atlas of Cardiovascular Health Care. Centre for Evaluative Clinical Sciences, Dartmouth (1999).Google Scholar
  39. Zhou, X.-H., Obuchowski, N.A., McClish, D.K.: Statistical Methods in Diagnostic Medicine. John Wiley & Sons, New York (2002).Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2007

Authors and Affiliations

  1. 1.Institute for Clinical Evaluative SciencesTorontoCanada
  2. 2.Department of Public Health SciencesUniversity of TorontoTorontoCanada
  3. 3.Department of Health Policy, Management, and EvaluationUniversity of TorontoTorontoCanada
  4. 4.Department of StatisticsUniversity of TorontoTorontoCanada

Personalised recommendations