Classifying Hospitals as Mortality Outliers: Logistic Versus Hierarchical Logistic Models
The use of hierarchical logistic regression for provider profiling has been recommended due to the clustering of patients within hospitals, but has some associated difficulties. We assess changes in hospital outlier status based on standard logistic versus hierarchical logistic modelling of mortality. The study population consisted of all patients admitted to acute, non-specialist hospitals in England between 2007 and 2011 with a primary diagnosis of acute myocardial infarction, acute cerebrovascular disease or fracture of neck of femur or a primary procedure of coronary artery bypass graft or repair of abdominal aortic aneurysm. We compared standardised mortality ratios (SMRs) from non-hierarchical models with SMRs from hierarchical models, without and with shrinkage estimates of the predicted probabilities (Model 1 and Model 2). The SMRs from standard logistic and hierarchical models were highly statistically significantly correlated (r > 0.91, p = 0.01). More outliers were recorded in the standard logistic regression than hierarchical modelling only when using shrinkage estimates (Model 2): 21 hospitals (out of a cumulative number of 565 pairs of hospitals under study) changed from a low outlier and 8 hospitals changed from a high outlier based on the logistic regression to a not-an-outlier based on shrinkage estimates. Both standard logistic and hierarchical modelling have identified nearly the same hospitals as mortality outliers. The choice of methodological approach should, however, also consider whether the modelling aim is judgment or improvement, as shrinkage may be more appropriate for the former than the latter.
KeywordsStandardised mortality ratios Logistic regression Hierarchical logistic regression
The Dr Foster Unit is largely funded by a research grant from Dr Foster Intelligence, an independent healthcare information company and joint venture with the UK Department of Health. The Dr Foster Unit at Imperial is affiliated with the Centre for Patient Safety and Service Quality at Imperial College Healthcare NHS Trust and funded by the National Institute of Health Research. We are grateful for support from the NIHR Biomedical Research Centre funding scheme.
Conflict of interest
The authors declare that they have no conflict of interest.
We have permission from the NIGB under Section 251 of the NHS Act 2006 (formerly Section 60 approval from the Patient Information Advisory Group) to hold confidential data and analyse them for research purposes. We have approval to use them for research and measuring quality of delivery of healthcare, from the South East Ethics Research Committee.
- 2.Aylin, P., Bottle, A., Jen, M. H., et al., HSMR mortality indicators. http://www1.imperial.ac.uk/resources/3321CA24-A5BC-4A91-9CC9-12C74AA72FDC Accessed 10 May 2013.
- 6.Cohen, L. H., Cardiac surgery in the adult. 3rd edition. http://cardiacsurgery.ctsnetbooks.org/ Accessed 20 March 2013.
- 9.Steele, F., Module 5: Introduction to multilevel modelling concepts. http://www.cmm.bristol.ac.uk/learning-training/ Accessed 1 August 2013.
- 10.Hox, J. J., Applied multilevel analysis. TT Publikaties Amsterdam, 1995. http://www.eric.ed.gov/PDFS/ED384636.pdf Accessed 1 August 2013.
- 12.The Information centre. Hospital episode statistics. 2011. www.hesonline.nhs.uk/ Accessed 11 March 2013.
- 13.NHS Connecting for Health. Secondary users service. 2011. http://www.connectingforhealth.nhs.uk/systemsandservices/sus Accessed 11 March 2013.
- 14.HCUP CCS: Healthcare Cost and Utilization Project (HCUP). Clinical Classifications Software (CCS) for ICD-10. Agency for Healthcare and Quality, Rockville, MD 2009. http://www.hcup-us.ahrq.gov/toolssoftware/icd_10/ccs_icd_10.jsp Accessed 11 March 2013.
- 16.Gridlink. Office for National Statistics. 2011. http://www.ons.gov.uk/ons/guide-method/geography/geographic-policy/gridlink-/index.html.
- 17.Wong, S., Flowers, J., and Lyratzopoulos, Y., Hospital admissions for myocardial infarction in the East of England, 1997/98–2007/08. Available from: http://www.erpho.org.uk/Download/Public/18587/1/Briefing%2020%20STEMI%204%20sides%20A4%20FINAL.pdf Accessed 1 August 2013.
- 20.SAS Institute Inc, SAS/STAT® 9.2 user’s guide. The GLIMMIX procedure. SAS Institute Inc., Cary, NC, 2008.Google Scholar
- 24.The Centre for Medicare and Medicaid Services. Statistical issues in assessing hospital performance. http://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/HospitalQualityInits/Downloads/Statistical-Issues-in-Assessing-Hospital-Performance.pdf Accessed 1 August 2013.
- 31.Austin, P. C., Tu, J. V., and Alter, D. A., Comparing hierarchical modeling with traditional logistic regression analysis among patients hospitalized with acute myocardial infarction: Should we be analyzing cardiovascular outcomes data differently? Am. Heart J. 145:27–35, 2003.CrossRefGoogle Scholar
- 34.Mohammed, M. A., Manktelow, B. N., and Hofer, T. P., Comparison of four methods for deriving hospital standardised mortality ratios from a single hierarchical logistic regression model. Stat. Methods Med. Res. Nov 6, 2012.Google Scholar