Our study population included all singleton births in British Columbia, Canada, between April 1st, 2004 and December 31st, 2013. Obstetrical, demographic, medical, and neonatal data on these deliveries were obtained from the BC Perinatal Data Registry (BCPDR), a quality-controlled population-based database that contains abstracted medical records from over 99 % of live births and stillbirth ≥20 weeks or ≥500 g in the province (including home births) [8]. The BCPDR is maintained by Perinatal Services BC, a public provincial agency with a mandate to enhance the delivery and quality of perinatal health care services in the province. Maternal and neonatal medical records are abstracted by Health Information Management professionals, who are trained through a minimum 2-year education program followed by a national certification. Data quality is maintained by use of provincially-standardized medical chart forms, use of a specialized data entry tool containing logic checks, and ongoing quality-assurance reviews at the hospital and provincial levels. The BCPDR has been used extensively for surveillance and research purposes [9–14] Pregnancy terminations were excluded. Ethics approval was obtained from the Research Ethics Board of the University of British Columbia/BC Children’s and Women’s Hospital (#H14-00990).
The AOI was calculated as the percent of deliveries in which an adverse labour and delivery outcome occurred to either mother or infant. An adverse outcome was defined as the occurrence of any of: maternal death during delivery admission, intrapartum stillbirth or in-hospital newborn death ≥2500 g at or beyond 37 weeks’ gestation with no congenital anomalies or fetal hydrops, uterine rupture, maternal admission to ICU (based on use of maternal intensive care procedures such as intubation or ventilation as a proxy), birth injury, unanticipated operative procedure, neonatal care unit admission (NICU) ≥2 days or transfer within 24 h of birth to a facility with a NICU for a non-anomalous infant ≥2500 g at or beyond 37 weeks with no fetal hydrops, 5-min Apgar score <7 for a non-anomalous infant ≥2500 g at or beyond 37 weeks with no fetal hydrops, blood transfusion, or 3rd or 4th degree perineal tear. The NICU admission time required to qualify as an adverse outcome was increased to ≥2 days from the original definition of 24 hours as this was believed to better reflect clinical practice patterns in British Columbia (i.e. to ensure that newborns admitted to the NICU for observation only were not classified as having an adverse outcome). The specific BCPDR variable, International Classification of Diseases Tenth Revision, Canada, and Canadian Classification of Health Interventions codes used to define each outcome are detailed in the Appendix.
The WAOS was calculated by first assigning points to each adverse outcome (i.e. weighting the outcomes according to severity), summing the total number of points in all deliveries with an adverse outcome, then dividing the total number of points by the total number of deliveries. As established in the original WAOS development process [6], a score of 750 was assigned for maternal death, 400 for intrapartum or in-hospital newborn death ≥2500 g, 100 for uterine rupture, 65 for maternal ICU admission, 60 for birth injury, 40 for unanticipated operative procedure, 35 for NICU admission ≥2 days or transfer within 24 h of birth to a facility with a NICU for an infant ≥2500 g, 25 for a 5-min Apgar score <7, 20 for blood transfusion, and 5 for 3rd or 4th degree perineal tear. The SI was calculated as the total number of points assigned for each adverse event divided by the total number of deliveries with an adverse event (i.e. the average severity of adverse events among those deliveries experiencing an adverse event). Thus, the WAOS reflects a combination of the frequency and severity of events, while the SI examines how severe the adverse events were (within the pregnancies that experienced an adverse event). The SI and WAOS can most easily be interpreted when used for comparisons: monitoring changes in the measures over time (i.e., flagging increases or decreases in more severe events) and contrasting severity of events between hospitals, regions, or care providers.
We calculated the risk of AOI, WAOS, and SI for each year and for each hospital with an average obstetrical volume of >10 deliveries per year with pointwise 95 % confidence intervals. Hospitals with 10 or fewer deliveries were excluded to avoid inclusion of hospitals that did not have planned obstetrical services (but may perform a small number of emergency deliveries). British Columbia has a universal health care system, and all hospitals are publicly funded. Risks between years or between hospitals were considered to be significantly different if the 95 % confidence intervals were mutually exclusive. Confidence intervals for WAOS and SI were calculated using bootstrapping with 1000 iterations. We ranked each hospital according to risk of AOI, WAOS, and SI, and examined the extent to which an institution’s risk of AOI agreed with the institution’s WAOS or SI by calculating a Pearson’s correlation coefficient.
To account for differences in patient characteristics between hospitals that might influence the apparent AOI risk, we used the obstetrical risk-adjustment approach developed by Bailit et al. [15]. In this approach, a logistic regression model was built to estimate each woman’s AOI risk based on her individual characteristics. This model included all patient factors that were associated with a composite adverse neonatal outcome, severe perineal laceration, or post-partum haemorrhage in the models of Bailit et al., but excluded insurance status (as this is not applicable in the single-payer Canadian health care system) and cocaine or methamphetamine use (due to lack of reliable data). These variables were: maternal age, pre-pregnancy body mass index, smoking during pregnancy, obstetrical history (parity combined with previous cesarean), premature rupture of membranes, multiple births, diabetes in pregnancy, hypertensive disorders of pregnancy, placenta previa, placenta accreta, placental abruption, and birthweight for gestational age percentile. The decision on which variables to include in the model was made a priori, and we did not exclude any variables based on lack of statistical significance. A robust cluster variance estimator was used to account for clustering of women within hospitals. As previously described, [15] we then averaged the predicted risks for all women within the same hospital to obtain the expected adverse outcome rate for each hospital (i.e., the hospital’s expected rate of adverse outcomes, given the characteristics of the women delivering there). The actual AOI risk was compared with the expected risk by calculating an observed to expected ratio. A risk-adjusted rate was then obtained for each hospital by multiplying its observed-to-expected ratios by the population AOI rate with bootstrapped 95 % confidence intervals. We were unable to apply a risk adjustment methodology to the hospital-specific WAOS and SI as these measures have non-standard statistical distributions (e.g., WAOS is highly skewed with a majority of 0 values) that are not amenable to multivariable regression modelling (i.e., by estimating a mean, median, or odds).
We conducted a number of sensitivity analyses to assess the robustness of the measures to alternative definitions. First, we included dehiscence or hysterectomy in the uterine rupture component. Second, we restricted the blood transfusion component to cases in which a post-partum haemorrhage was also coded. Third, we added visceral injury to the list of unanticipated operative procedures. Fourth, we included stillbirths with ‘unknown’ timing to the intrapartum stillbirths, and lastly, we excluded 3rd and 4th degree tears from the Index. We additionally examined the potential impact of under-capture of maternal deaths and ICU admissions by tripling the incidence of each, and recalculating the AOI, WAOS, and SI.