Background

On June 2016, Québec’s health and welfare commissioner published a report stating that the province had experienced years of the worst emergency department (ED) wait times in Canada and among high-income countries [1]. Two years later, Québec data on ED length of stay (LOS) are still concerning and in fact all Canadian provinces lag behind many other jurisdictions in their attempts to grapple with this important public health issue [1, 2]. A compelling body of evidence associates overcrowding and prolonged ED LOS with adverse effects such as increased time to thrombolysis [3]; delays in antibiotics administration [3,4,5] and pain management [3, 6, 7]; patient dissatisfaction [3]; and an increased in-hospital and out of hospital mortality [3, 8,9,10]. However, metrics of ED operations only capture part of the quality picture as they focus only on time-based quality measures and do not include patient outcomes that remain the foremost validation of the performance of health systems and the ultimate targets for quality improvement initiatives [11].

Assessing health outcomes in the ED setting represent a significant challenge as most ED-relevant outcomes are usually observed in other care settings (e.g. ICU, hospital ward or home care) where patients have been transferred after initial ED management. Notwithstanding the complexity of linking outcomes to ED care, most expert guidelines still recommend the inclusion of outcome measures, such as mortality, to a comprehensive ED performance assessment framework [12,13,14].

The hospital standardized mortality ratio for emergency department sensitive conditions (ED-HSMR) represents such a metric [15,16,17]. Adapted from a methodology used by the Canadian Institute for Health Information (CIHI) for tracking overall in-hospital mortality [18], the ED-HSMR specifically measures risk-adjusted mortality for patients admitted to hospital with emergency sensitive conditions, which are conditions where ED management may potentially improve outcomes. Using a consensus methodology [17] and a national survey of ED care providers [15], 92 potential emergency-sensitive conditions (e.g. ectopic pregnancy) were identified and evaluated. Among these conditions, 37 mortality-related emergency sensitive diagnosis groups (e.g., A41 sepsis) from the 10th Canadian version of the International Classification of Disease (ICD-10-CA) were used to develop the emergency care sensitive HSMR variant. Easily retrieved from administrative databases, the ED-HSMR has been shown to be valid and reliable [16]. It allows institutions or jurisdictions to follow their ED patient mortality over time and trigger internal performance reviews if trends are worrisome. It also provides opportunities for interprovincial comparisons of health outcomes observed among patients with emergency-sensitive conditions.

Consequently, to gain a broader understanding of ED performance and of the quality of care provided to emergency-sensitive conditions in Canada, this study aimed to describe in-hospital mortality across provinces using the ED-HSMR.

Methods

Study design and setting

We conducted a retrospective cohort study on national hospital discharge databases. We reviewed discharges from acute inpatient care institutions between April 1st 2009 and March 31st 2012 for nine provinces and three territories, and between April 1st 2009 and March 31st 2011 for the province of Québec. CIHI provided all administrative and patient-level data. Under the Canada Health Act, provinces and territories must provide universal health coverage to their citizens. All Canadian acute care facilities are publicly funded and owned.

Hospital selection

We included all Canadian acute care facilities with an emergency department and hospital-based acute inpatient care. We excluded cancer centres, children’s hospitals and heart institutes because they treat specific populations with very different case-mixes. For risk-adjustment purpose, hospitals were classified into one of four peer-groups (teaching, large community, medium community and small community) based on academic designation, patient complexity and volume (see Additional file 1). This classification was adopted from the validated methodology used by the Canadian Institute for Health Information to calculate an all-cases HSMR [18].

Case selection

We included all patients discharged dead or alive from hospital during the study period if they met the following criteria: 1) Admission to hospital through the ED; 2) Discharge from hospital with one of the 37 mortality-related emergency-sensitive diagnosis groups captured in the ED-HSMR (see Additional file 2); 3) Age between 29 days and 120 years at hospital admission; 4) Hospital length of stay equal to or less than 365 days; 5) Canadian resident. Patients were excluded if: 1) they were deceased at ED arrival; and 2) they were discharged against medical advice. Inclusion and exclusion criteria were directly derived from the methodology employed by CIHI to calculate the overall Canadian HSMR [18].

Statistical analysis

Characteristics of the institutions and patients

Where it appropriately applied, medians and interquartile ranges, means or proportions with 95% confidence intervals were used to describe the characteristics of the institutions and of the cases included. Unadjusted mortality rates for each one of the 37 emergency-sensitive diagnosis groups included in the ED-HSMR were calculated per hospital and per year of the study period.

ED-HSMR calculation

ED-HSMRs were calculated for the fiscal years (April to March) 2010–2011 and 2011–2012 with the following equation:

$$ \frac{\mathrm{Actual}\ \mathrm{number}\ \mathrm{of}\ \mathrm{deaths}\ \mathrm{among}\ \mathrm{patients}\ \mathrm{with}\ \mathrm{emergency}\hbox{-} \mathrm{sensitive}\ \mathrm{diagnosis}\ \mathrm{groups}\ \mathrm{in}\ 2010\hbox{-} 11\ \mathrm{or}\ 2011\hbox{-} 12}{\mathrm{Expected}\ \mathrm{number}\ \mathrm{of}\ \mathrm{deaths}\ \mathrm{among}\ \mathrm{same}\ \mathrm{patients}\ \mathrm{based}\ \mathrm{on}\ \mathrm{mortality}\ \mathrm{probabilities}\ \mathrm{in}\ \mathrm{the}\ \mathrm{reference}\ \mathrm{year}\left(2009\hbox{-} 10\right)}\times 100 $$

A 95% confidence interval was calculated for each HSMR value using the Byar’s approximation [18]. A HSMR value above 100 means that more deaths than expected occurred in an acute care facility. Conversely, a HSMR value below 100 means fewer deaths than expected occurred in an acute care facility. In other words, an ED-HSMR of 114 would indicate that each admission to a specific hospital for an emergency-sensitive condition has a probability of death 14% higher than what is expected from the Canadian average.

Expected deaths

We estimated the expected number of deaths in 2010–11 or 2011–12 using fixed effects logistic regression models derived from the reference year (2009–10) for each hospital-peer group. After modelling mortality with different independent variables, the following covariates were retained in the final ED-HSMR predictive models: diagnosis groups, age (continuous), gender (dichotomous), in-hospital length of stay (6 groups: 1, 2, 3–9, 10–15, 16–21 and 22–365 days) and comorbidities (3 groups based on Charlson index score: Group 0 = score 0 (outside Québec) or scores 0 and 1 (Québec); Group 1 = scores 1 and 2 (outside Québec) or scores 2, 3 and 4 (Québec); Group 2 = scores 3 and more (outside Québec) or scores 5 and more (Québec); see Additional file 3). For managing missing data, we used a single imputation method, assigning most frequent values for categorical variables and medians, for continuous variables.

Probability of death at patient-level was calculated using the appropriate hospital-peer group specific model (teaching, large community, medium community and small community) from the reference year. After conversion from the log odds of death (pdeath = elog odds of death / [1 + (elog odds of death)]), all individual patient probabilities were summed to get the expected number of deaths in a specific hospital, in 2010–11 or 2011–12. An ED-HSMR for a specific acute care facility was only calculated if more than 20 deaths were expected within the study year at the institution, as fewer deaths yield unreliable and volatile HSMR measures [19].

The discriminatory power and calibration of the risk-adjustment models used to estimate the expected number of deaths for each hospital were reported in a previous publication [16]. Areas under receiver operating characteristic curves of the predictive models used in the ED-HSMR were 0.80, 0.80, 0.80 and 0.81 for the teaching, large-community, medium-community and small-community peer-group hospitals, respectively.

Hospital-level and aggregated provincial ED-HSMRs

We report hospital-level ED-HSMRs through tables and caterpillar plots. Stratifications by peer-groups and provinces are graphically represented. Aggregated provincial ED-HSMR values were calculated by dividing the sum all observed (O) deaths with the sum of all expected (E) number of deaths of all institutions of a province or a territory (O/E × 100). All patients were included in aggregated measures, even those from hospitals with less than 20 expected deaths where no site-specific HSMR could be estimated. Analyses were performed using Stata version MP 11.2 (StataCorp, TX, USA).

Results

Characteristics of cases and hospitals

During the 3-year study period, 1,335,379 patients were admitted to 629 hospitals across 11 provinces and territories from the ED with one of the 37 mortality-related emergency sensitive diagnosis groups captured in the ED-HSMR as the most responsible diagnosis. Table 1 describes their characteristics. Half of the cohort was composed of females older than 73 years and hospitalized for 5 days or less. Our study population presented a low comorbidity burden as more than 80% of patients had a Charlson index score of 0 or 1. Small hospitals represented 58% of all hospitals included in our study, but treated only 10.8% of all patients admitted. Hospital distribution was markedly different in the province of Québec where 42.5% of all hospitals were either teaching or large-community institutions. Provinces and Territories in the rest of Canada had a higher proportion of small-community hospitals (Table 2).

Table 1 Characteristics of patients* (n = 1,335,379)a
Full size table
Table 2 Number of hospitals (N = 629) per peer-group and province*
Full size table

Overall and diagnosis group specific mortality rates

For the 3 years of the study period (including the reference year), overall mortality in our cohort was 8.9%, with chronic obstructive pulmonary disease, pneumonia, heart failure, acute myocardial infarction, stroke/cerebral infarction, and sepsis accounting for 55.5% of all deaths (Table 3). Diagnosis groups with the highest mortality rates were cardiac arrest (65.1%) and shock not elsewhere classified (50.0%).

Table 3 Study population (N = 1,335,379) distribution and mortality rates by Diagnosis Group
Full size table

Hospital-level ED-HSMRs

The ED-HSM was estimated for 46.7 and 33.9% (Québec data not available) of all Canadian hospitals in 2010 and 2011, respectively (see Additional files 4 and 5). Among hospitals with sufficient number of expected deaths to calculate the ED-HSMR, 22.8% in 2010 and 31.5% in 2011 had a HSMR 95% confidence interval falling below the 100 threshold (fewer deaths than expected), and 10.5% in 2010 and 7.0% in 2011 had a HSMR 95% confidence interval falling above (more deaths than expected). Results are stratified by province and hospital-peer group in Table 4.

Table 4 Number of Hospitals with an ED-HSMR 95% confidence interval falling under or over 100 by peer-group and province in 2010 (N = 294) and 2011 (N = 213)*
Full size table

Provincial aggregated ED-HSMRs

Figure 1 illustrates the aggregated ED-HSMR estimates by province for 2010–11 and 2011–12. In both years, Nova Scotia and Newfoundland-Labrador had provincial ED-HSMRs higher than 100, while the 95% confidence intervals of the aggregated measures for Prince Edward Island and Manitoba crossed the 100 threshold. All other provinces and territories experienced fewer deaths than what was expected from mortality trends recorded in Canada during the reference year (2009–10).

Fig. 1
figure 1

ED-HSMR (95% confidence interval) by province and territories for years 2010–11 and 2011–12. Green: < 100 = Fewer deaths than expected. Yellow: =100 = Observed deaths are equal to expected deaths. Red: > 100 = More deaths than expected

Full size image

Discussion

We analyzed adjusted mortality rates in Canada using the ED-HSMR. Our results show that there is significant interhospital and interprovincial variation in mortality trends for patients admitted with emergency-sensitive conditions and that this variation appears to be consistent over the 2 years studied. The ED-HSMR could be used to measure the performance of Canadian hospital systems in emergency care and guide quality improvement initiatives.

To the best of our knowledge, we are the first to report on variations in mortality across Canadian provinces for a comprehensive list of emergency sensitive conditions. Two recent studies published in 2017 have reported mortality trends in Canada for two domains overlapping with emergency medicine: trauma and acute myocardial infarction (AMI) care. Both studies pointed out interprovincial disparities in the management of these acutely ill populations. Tran et al. examined Canadian and provincial 30-day in-hospital mortality rates after percutaneous coronary interventions (PCI) and coronary artery bypass grafting (CABG) between 2004 and 2013. Ten-year adjusted odds (2004–2013) of mortality were higher after PCI in Saskatchewan and after CABG in Newfoundland and Labrador [20]. Similarly, Moore et al. reviewed data on 78,707 trauma patients in 7 provinces from 2006 to 2012. Authors reported that Nova Scotia, Newfoundland and Labrador, and Manitoba were the provinces with the highest risk-adjusted mortality rates in 2011 and 2012 [21]..

Our analyses yielded similar results with Newfoundland/Labrador, Nova Scotia, Manitoba and Prince Edward Island having the highest ED-HSMR estimates. Although our composite mortality indicator captures some trauma-related diagnoses (e.g. intracranial injury) and AMI, it is meant to employ a broad approach for identifying mortality trends involving 37 emergency sensitive conditions (e.g. sepsis). As a result, trauma and AMI care may explain a part of, but not all observed disparities reported in our own study. Moreover, mortality for emergency sensitive conditions appears to be higher than expected in provinces where the proportion of the population living in rural communities is highest [22]. This finding echoes previous reports that highlighted significant disparities in access to high-quality emergency care in Canadian rural communities [23, 24]. In the same way it was done for trauma care across the country, Canadian decision- and policy-makers could use the ED-HSMR to trigger in-depth performance assessment to help identify opportunities for improving emergency care structures and processes to improve patient outcomes.

Since 2005, CIHI calculates annually for all Canadian hospitals a HSMR based on 72 diagnosis groups (e.g. Alzheimer’s disease) explaining 80% of all Canadian in-hospital deaths. Although using the same methodology, our HSMR variant differs significantly from CIHI’s by capturing only those conditions for which ED care may potentially influence mortality (e.g. pulmonary embolism). Many reports have advocated caution in using HSMRs or any other mortality indicators [19, 25]. Incomplete risk adjustment, low rate of preventable deaths and inconstant concordance between mortality and other quality indicators are some of the limitations impeding inference we could draw from mortality metrics to potential quality of care breaches. Moreover, in-hospital mortality is usually remote from the ED and occurs on hospital wards, making it virtually impossible to completely isolate ED influence on patient outcomes. However, as imperfect as it may be, other reports have shown that tracking mortality rates can drive system-level changes for improving patient care and healthcare organizations [26,27,28,29,30]..

Similar problems arise with ED wait time metrics. Like mortality rates, they are reflections of a care system rather than processes of care specific to the ED. Blocked access to in-hospital beds, limited primary care resources for management of patients with multimorbidities and a lack of patient education on appropriate use of EDs are some of the factors that can impact ED flow, but are not caused per se by poor ED care. As for ED wait time metrics, the ED-HSMR could serve as a strong incentive for hospital departments and community health care resources to move from siloed approached to quality improvement and work together to improve outcomes of patients with emergency sensitive conditions.

The Québec’s health and welfare commissioner report published in 2016 raises important questions. As worrisome as they are, wait times henceforth do not appear to be the only important risk factor impacting outcomes of ED patients with emergency sensitive conditions. Indeed, Québec ED-HSMR in 2010–11 reveals fewer deaths than expected. This complementary information highlights the importance of synergistically using structure, process and outcome metrics when evaluating health care quality. We believe that all jurisdictions should adopt a comprehensive quality framework that would include among other metrics the ED-HSMR.

Limitations

This study has inherent limitations of all studies using administrative database sources. Previous reports have shown that coding of administrative data is usually accurate and inaccuracies had modest effect on an HSMR [31,32,33]. The ED-HSMR does not adjust for all potential confounders, such as severity of disease, smoking habits or socio-economic status. Although severity of disease and smoking habits are not easily extractable from existing hospital databases, socio-economic status can be assessed using deprivation index derived from patient postal codes. Future iteration of the ED-HSMR could test socio-economic status as a potential explanatory variable. Furthermore the ED-HSMR is not a performance measure suitable for small hospitals because of the low numbers of observed and expected in-hospital deaths. Small hospitals represent 58% of all Canadian hospitals. Further research is needed to identify outcome measures that could be used to assess the quality of care provided in low-volume institutions. Finally, we acknowledge that our data are several years old. However, we believe that our results demonstrate the feasibility of using an ED sensitive condition hospital standardized mortality ratio to measure ED performance as a way to supplement existing wait times and access-to-care indicators.

Conclusion

In conclusion, we analyzed Canadian in-hospital mortality rates using the ED-HSMR. Our study revealed important variation in risk-adjusted mortality for patients admitted to hospital with emergency-sensitive conditions among Canadian provinces. These results warrant in-depth evaluations to understand the root causes of the observed regional variation. The ED-HSMR may be a useful outcome indicator to complement existing process indicators in measuring ED performance.