Skip to main content

Advertisement

SpringerLink
Log in

Detecting and visualizing outliers in provider profiling via funnel plots and mixed effect models

  • Published:
Health Care Management Science Aims and scope Submit manuscript

Abstract

In this work we propose the use of a graphical diagnostic tool (the funnel plot) to detect outliers among hospitals that treat patients affected by Acute Myocardial Infarction (AMI). We consider an application to data on AMI hospitalizations recorded in the administrative databases of our regional district. The outcome of interest is the in-hospital mortality, a variable indicating if the patient has been discharged dead or alive. We then compare the results obtained by graphical diagnostic tools with those arising from fitting parametric mixed effects models to the same data.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Notes

  1. This is a risk index developed by 3M Health Information Systems and it combines the ICD-9-CM codes of Primary diagnosis, Secondary diagnoses and Procedures. Once the clinical model for severity of illness and risk of mortality was developed for each base APR-DRG, it was evaluated with historical data, extensively reviewed and regularly updated.

References

  1. AHRQ QualityIndicators (2007) Guide to inpatient quality indicators: quality of care in hospitals - volume, mortality, and utilization. Department of Health and Human Services Agency for Healthcare Research and Quality, Version 3.1

  2. Austin PC, Naylor CD, Tu JV (2001) A comparison of a Bayesian vs. a frequentist method profiling hospital performance. J Eval Clin Pract 7:35–45

    Article  Google Scholar 

  3. Austin PC, Alter DA, Tu JV (2003) The use of fixed- and random-effects models for classifying hospitals as mortality outliers: a Monte Carlo assessment. Med Dec Making 23:526–539

    Article  Google Scholar 

  4. Barbieri P, Grieco N, Ieva F, Paganoni AM, Secchi P (2010) Exploitation, integration and statistical analysis of Public Health Database and STEMI archive in Lombardia Region. Complex data modeling and computationally intensive statistical methods Series “Contribution to Statistics”. Springer, pp 41–56

  5. Campbell MJ, Jacques RM, Fotheringham J, Maheswaran R, Nicholl J (2012) Developing a summary hospital mortality index: retrospective analysis in English hospitals over five years. Br Med J344:e1001

    Article  Google Scholar 

  6. De Marco MF, Lorenzoni L, Addari P, Nante N (2002) Evaluation of the capacity of the APR-DRG classification system to predict hospital mortality. Epidemiol Prev 26(4):183–190

    Google Scholar 

  7. DerSimonian R, Laird N (1986) Meta-analysis in clinical trials. Control Clin Trials 7:177–188

    Article  Google Scholar 

  8. Egger M, Smith GD, Schneider M, Minder C (1997) Bias in meta-analysis detected by a simple, graphical test. Br Med J 315(7109):629–624

    Article  Google Scholar 

  9. Farrel PJ, Groshen S, MacGibbon B, Tomberlin TJ (2010) Outlier detection for a hierarchical Bayes model in a study of hospital variation in surgical procedures. Stat Methods Med Res 19:601–619

    Article  Google Scholar 

  10. Garbe S, Primer E (2007) Administrative health databases in observational studies of drug effects-advantages and disadvantages. Nat Clin Pract Rheumatol 3:725–732 [review]

    Article  Google Scholar 

  11. Grieco N, Ieva F, Paganoni AM (2012) Performance assessment using mixed effects models: a case study on coronary patient care. IMA J Manag Math 23(2):117–131

    Article  Google Scholar 

  12. Guglielmi A, Ieva F, Paganoni AM, Ruggeri F, Soriano J (2013) Semiparametric Bayesian modeling for the classification of patients with high observed survival probabilities. J R Stat Soc- Ser C. (in press)

  13. Jones HE, Spiegelhalter DJ (2011) The identification of “unusual” health-care providers from a hierarchical model. Am Stat 65(3):154–163

    Article  Google Scholar 

  14. Light RJ, Pillemer DB (1984) Summing up: the science of reviewing research. Harvard University Press, Cambridge

    Google Scholar 

  15. Mohammed MA, Deeks JJ (2008) In the context of performance monitoring, the caterpillar plot should be mothballed in favor of the funnel plot. Ann Thorac Surg 86:348

    Article  Google Scholar 

  16. Normand SLT, Glickman ME, Gatsonis CA (1997) Statistical methods for profiling providers of medical care: issues and applications. J Am Stat Assoc 92:803–814

    Article  Google Scholar 

  17. R Development Core Team (2009) R: a language and environment for statistical computing. R Foundation for Statistical Computing, Vienna [Online]. http://www.R-project.org

    Google Scholar 

  18. Racz J, Sedransk J (2010) Bayesian and frequentist methods for provider profiling using risk-adjusted assessments of medical outcomes. J Am Stat Assoc 105(489):48–58

    Article  Google Scholar 

  19. Romani PS, Chan BK (2000) Risk-adjusting acute myocardial infarction mortality: are APR-DRGs the right tool?Health Serv Res 34(7):1469–1489

    Google Scholar 

  20. Ministero della Sanità (1991) Decreto 28 Dicembre, [online] http://www.trovanorme.salute.gov.it/dettaglioAtto?id=12967&articolo=3

  21. Spiegelhalter DJ (2002) Funnel plots for institutional comparisons (letter). Qual Saf Health Care 11:390–391

    Article  Google Scholar 

  22. Spiegelhalter DJ (2005) Funnel plots for comparing institutional performance. Stat Med 24:1185–1202

    Article  Google Scholar 

  23. Spiegelhalter DJ, Sherlw-Johnson C, Bardsley M, Blunt I, Wood C, Grigg O (2012) Statistical methods for healthcare regulation: rating, screening and surveillance. J R Stat Soc - Ser A 175:1–47

    Article  Google Scholar 

  24. Wirehn AB, Karlsson HM, Cartensen JM et al (2007) Estimating Disease Prevalence using a population-based administrative healthcare database. Scand J Public Health 35:424–431

    Article  Google Scholar 

  25. Van Walraven C, Austin P (2012) Administrative database research has unique characteristics that can risk biased results. J Clin Epidemiol 65:126–131

    Article  Google Scholar 

Download references

Acknowledgments

This work is within the Strategic Program “Exploitation, integration and study of current and future health databases in Lombardia for Acute Myocardial Infarction”. The authors wish to thank Regione Lombardia - Healthcare division for having funded and sustained the project, Lombardia Informatica S.p.A. for having provided data and all the physicians who collaborated to STEMI Archive planning and data collection.

Author information

Authors and Affiliations

  1. MOX - Modellistica e Calcolo Scientifico, Dipartimento di Matematica, Politecnico di Milano, via Bonardi 9, 20133, Milan, Italy

    Francesca Ieva & Anna Maria Paganoni

Authors
  1. Francesca Ieva
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Anna Maria Paganoni
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Francesca Ieva.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Ieva, F., Paganoni, A.M. Detecting and visualizing outliers in provider profiling via funnel plots and mixed effect models. Health Care Manag Sci 18, 166–172 (2015). https://doi.org/10.1007/s10729-013-9264-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10729-013-9264-9

Keywords

Mathematics Subject Classifications (2010)

Search

Navigation