Skip to main content

Advertisement

SpringerLink
Log in

Classifying Hospitals as Mortality Outliers: Logistic Versus Hierarchical Logistic Models

  • Systems-Level Quality Improvement
  • Published:
Journal of Medical Systems Aims and scope Submit manuscript

Abstract

The use of hierarchical logistic regression for provider profiling has been recommended due to the clustering of patients within hospitals, but has some associated difficulties. We assess changes in hospital outlier status based on standard logistic versus hierarchical logistic modelling of mortality. The study population consisted of all patients admitted to acute, non-specialist hospitals in England between 2007 and 2011 with a primary diagnosis of acute myocardial infarction, acute cerebrovascular disease or fracture of neck of femur or a primary procedure of coronary artery bypass graft or repair of abdominal aortic aneurysm. We compared standardised mortality ratios (SMRs) from non-hierarchical models with SMRs from hierarchical models, without and with shrinkage estimates of the predicted probabilities (Model 1 and Model 2). The SMRs from standard logistic and hierarchical models were highly statistically significantly correlated (r > 0.91, p = 0.01). More outliers were recorded in the standard logistic regression than hierarchical modelling only when using shrinkage estimates (Model 2): 21 hospitals (out of a cumulative number of 565 pairs of hospitals under study) changed from a low outlier and 8 hospitals changed from a high outlier based on the logistic regression to a not-an-outlier based on shrinkage estimates. Both standard logistic and hierarchical modelling have identified nearly the same hospitals as mortality outliers. The choice of methodological approach should, however, also consider whether the modelling aim is judgment or improvement, as shrinkage may be more appropriate for the former than the latter.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Jarman, B., Gault, S., Alves, B., et al., Explaining differences in English hospital death rates using routinely collected data. BMJ 318:1515–1520, 1999.

    Article  Google Scholar 

  2. Aylin, P., Bottle, A., Jen, M. H., et al., HSMR mortality indicators. http://www1.imperial.ac.uk/resources/3321CA24-A5BC-4A91-9CC9-12C74AA72FDC Accessed 10 May 2013.

  3. Bottle, A., and Aylin, P., Intelligent information: A national system for monitoring clinical performance. Health Serv. Res. 43:10–31, 2008.

    Article  Google Scholar 

  4. Alharbi, O., Rabeneck, L., Paszat, L. F., et al., A population-based analysis of outpatient colonoscopy in adults assisted by an anaesthesiologist. Anaesthesiology 111:734–740, 2009.

    Article  Google Scholar 

  5. Krumholz, H. M., Brindis, R. G., Brush, J. E., et al., Standards for statistical models used for public reporting of health outcomes. Circulation 113:456–462, 2006.

    Article  Google Scholar 

  6. Cohen, L. H., Cardiac surgery in the adult. 3rd edition. http://cardiacsurgery.ctsnetbooks.org/ Accessed 20 March 2013.

  7. DeLong, E., Hierarchical modeling: Its time has come. Am. Heart J. 145:16–18, 2003.

    Article  Google Scholar 

  8. Diez-Roux, A. V., Multilevel analysis in public health research. Annu. Rev. Public Health 21:171–192, 2000.

    Article  Google Scholar 

  9. Steele, F., Module 5: Introduction to multilevel modelling concepts. http://www.cmm.bristol.ac.uk/learning-training/ Accessed 1 August 2013.

  10. Hox, J. J., Applied multilevel analysis. TT Publikaties Amsterdam, 1995. http://www.eric.ed.gov/PDFS/ED384636.pdf Accessed 1 August 2013.

  11. Alexandrescu, R., Jen, M. H., Bottle, A., et al., Logistic vs. Hierarchical modeling: An analysis of a statewide inpatient sample. J. Am. Coll. Surg. 213:392–401, 2011.

    Article  Google Scholar 

  12. The Information centre. Hospital episode statistics. 2011. www.hesonline.nhs.uk/ Accessed 11 March 2013.

  13. NHS Connecting for Health. Secondary users service. 2011. http://www.connectingforhealth.nhs.uk/systemsandservices/sus Accessed 11 March 2013.

  14. HCUP CCS: Healthcare Cost and Utilization Project (HCUP). Clinical Classifications Software (CCS) for ICD-10. Agency for Healthcare and Quality, Rockville, MD 2009. http://www.hcup-us.ahrq.gov/toolssoftware/icd_10/ccs_icd_10.jsp Accessed 11 March 2013.

  15. Sundararajan, V., Henderson, T., Perry, C., et al., New ICD-10 version of the Charlson comorbidity index predicted in-hospital mortality. J. Clin. Epidemiol. 57:1288–1294, 2004.

    Article  Google Scholar 

  16. Gridlink. Office for National Statistics. 2011. http://www.ons.gov.uk/ons/guide-method/geography/geographic-policy/gridlink-/index.html.

  17. Wong, S., Flowers, J., and Lyratzopoulos, Y., Hospital admissions for myocardial infarction in the East of England, 1997/98–2007/08. Available from: http://www.erpho.org.uk/Download/Public/18587/1/Briefing%2020%20STEMI%204%20sides%20A4%20FINAL.pdf Accessed 1 August 2013.

  18. Seccareccia, F., Perucci, C. A., D’Errigo, P., et al., The Italian CABG Outcome Study: Short-term outcomes in patients with coronary artery bypass graft surgery. Eur. J. Cardiothorac. Surg. 29:56–62, 2006.

    Article  Google Scholar 

  19. Grant, S. W., Grayson, A. D., Purkayastha, D., et al., Logistic risk model for mortality following elective abdominal aortic aneurysm repair. Br. J. Surg. 98:652–658, 2011.

    Article  Google Scholar 

  20. SAS Institute Inc, SAS/STAT® 9.2 user’s guide. The GLIMMIX procedure. SAS Institute Inc., Cary, NC, 2008.

    Google Scholar 

  21. Cohen, M. E., Dimick, J. B., Bilimoria, K. Y., et al., Risk adjustment in the American College of Surgeons National Surgical Quality Improvement Program: A comparison of logistic versus hierarchical modeling. J. Am. Coll. Surg. 209:687–693, 2009.

    Article  Google Scholar 

  22. Cohen, M. E., Ko, C. Y., Bilimoria, K. Y., et al., Optimizing ACS NSQIP modeling for evaluation of surgical quality and risk: Patient risk adjustment, procedure mix adjustment, shrinkage adjustment, and surgical focus. J. Am. Coll. Surg. 217:336–346, 2013.

    Article  Google Scholar 

  23. Glance, L. G., Dick, A., Osler, T. M., et al., Impact of changing the statistical methodology on hospital and surgeon ranking: The case of the New York State cardiac surgery report card. Med. Care 44:311–319, 2006.

    Article  Google Scholar 

  24. The Centre for Medicare and Medicaid Services. Statistical issues in assessing hospital performance. http://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/HospitalQualityInits/Downloads/Statistical-Issues-in-Assessing-Hospital-Performance.pdf Accessed 1 August 2013.

  25. Spiegelhalter, D. J., Funnel plots for comparing institutional performance. Stat. Med. 24:1185–1202, 2005.

    Article  MathSciNet  Google Scholar 

  26. Cook, N. R., Use and misuse of the receiver operating characteristic curve in risk prediction. Circulation 115:928–935, 2007.

    Article  Google Scholar 

  27. Snijders, T. A. B., and Bosker, R. J., Multilevel analysis. An introduction to basic and advanced multilevel modeling. SAGE Publications, London, 1999.

    MATH  Google Scholar 

  28. D’Errigo, P., Tosti, M. E., Fusco, D., et al., Use of hierarchical models to evaluate performance of cardiac surgery centres in the Italian CABG outcome study. BMC Med. Res. Methodol. 7:29, 2007.

    Article  Google Scholar 

  29. Hannan, E. L., Wu, C., DeLong, E. R., and Raudenbush, S. W., Predicting risk-adjusted mortality for CABG surgery: Logistic versus hierarchical logistic models. Med. Care 43:726–735, 2005.

    Article  Google Scholar 

  30. Austin, P. C., Alter, D. A., and Tu, J. V., The use of fixed- and random-effects models for classifying hospitals as mortality outliers: A Monte Carlo assessment. Med. Decis. Making 23:526–539, 2003.

    Article  Google Scholar 

  31. Austin, P. C., Tu, J. V., and Alter, D. A., Comparing hierarchical modeling with traditional logistic regression analysis among patients hospitalized with acute myocardial infarction: Should we be analyzing cardiovascular outcomes data differently? Am. Heart J. 145:27–35, 2003.

    Article  Google Scholar 

  32. Clark, D. E., Hannan, E. L., and Wu, C., Predicting risk adjusted mortality for trauma patients: Logistic versus multilevel logistic models. J. Am. Coll. Surg. 211:224–231, 2010.

    Article  Google Scholar 

  33. Tan, A., Freeman, J. L., and Freeman, D. H., Jr., Evaluating health care performance: Strengths and limitations of multilevel analysis. Biom. J. 49:707–718, 2007.

    Article  MathSciNet  Google Scholar 

  34. Mohammed, M. A., Manktelow, B. N., and Hofer, T. P., Comparison of four methods for deriving hospital standardised mortality ratios from a single hierarchical logistic regression model. Stat. Methods Med. Res. Nov 6, 2012.

  35. Bland, J. M., and Altman, D. G., Measuring agreement in method comparison studies. Stat. Methods Med. Res. 8:135–160, 1999.

    Article  Google Scholar 

Download references

Acknowledgments

The Dr Foster Unit is largely funded by a research grant from Dr Foster Intelligence, an independent healthcare information company and joint venture with the UK Department of Health. The Dr Foster Unit at Imperial is affiliated with the Centre for Patient Safety and Service Quality at Imperial College Healthcare NHS Trust and funded by the National Institute of Health Research. We are grateful for support from the NIHR Biomedical Research Centre funding scheme.

Conflict of interest

The authors declare that they have no conflict of interest.

Ethics

We have permission from the NIGB under Section 251 of the NHS Act 2006 (formerly Section 60 approval from the Patient Information Advisory Group) to hold confidential data and analyse them for research purposes. We have approval to use them for research and measuring quality of delivery of healthcare, from the South East Ethics Research Committee.

Author information

Authors and Affiliations

  1. Dr. Foster Unit at Imperial College, Department of Primary Care and Public Health, Imperial College London, London, EC4Y 8EN, UK

    Roxana Alexandrescu, Alex Bottle, Brian Jarman & Paul Aylin

  2. Department of Palliative Care, Policy and Rehabilitation, King’s College London, London, SE5 9PJ, UK

    Roxana Alexandrescu

Authors
  1. Roxana Alexandrescu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alex Bottle
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Brian Jarman
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Paul Aylin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Roxana Alexandrescu.

Additional information

This article is part of the Topical Collection on Systems-Level Quality Improvement

Rights and permissions

Reprints and permissions

About this article

Cite this article

Alexandrescu, R., Bottle, A., Jarman, B. et al. Classifying Hospitals as Mortality Outliers: Logistic Versus Hierarchical Logistic Models. J Med Syst 38, 29 (2014). https://doi.org/10.1007/s10916-014-0029-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10916-014-0029-x

Keywords

Search

Navigation