Although most pregnancies result in a good outcome, complications are not uncommon and can be associated with serious implications for mothers and babies. Predictive modeling has the potential to improve outcomes through a better understanding of risk factors, heightened surveillance for high-risk patients, and more timely and appropriate interventions, thereby helping obstetricians deliver better care. We identify and study the most important risk factors for four types of pregnancy complications: (i) severe maternal morbidity, (ii) shoulder dystocia, (iii) preterm preeclampsia, and (iv) antepartum stillbirth. We use an Explainable Boosting Machine (EBM), a high-accuracy glass-box learning method, for the prediction and identification of important risk factors. We undertake external validation and perform an extensive robustness analysis of the EBM models. EBMs match the accuracy of other black-box ML methods, such as deep neural networks and random forests, and outperform logistic regression, while being more interpretable. EBMs prove to be robust. The interpretability of the EBM models reveal surprising insights into the features contributing to risk (e.g., maternal height is the second most important feature for shoulder dystocia) and may have potential for clinical application in the prediction and prevention of serious complications in pregnancy.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Price excludes VAT (USA)
Tax calculation will be finalised during checkout.
In our work, SMM is a composite term for 6 adverse diagnoses: (i) hysterectomy, (ii) blood transfusion, (iii) disseminated intravascular coagulation, (iv) amniotic fluid embolism, (v) thromboembolism, and (vi) eclampsia.
Note that predicting preterm preeclampsia can only be done in practice \(< 37\) weeks into the pregnancy per definition of preterm.
Labor type refers to whether labor was spontaneous or induced.
Osterman MJ, Hamilton BE, Martin JA, Driscoll AK, Valenzuela CP (2022) Births: final data for 2020
Declercq E, Zephyrin L (2021) Severe Maternal Morbidity in the United States: A Primer. Commonwealth Fund
Creanga AA, Bateman BT, Kuklina EV, Callaghan WM (2014) Racial and ethnic disparities in severe maternal morbidity: a multistate analysis, 2008–2010. Am J Obstet Gynecol 210(5):435–1
Gao C, Osmundson S, Yan X, Edwards DV, Malin BA, Chen Y (2019) Learning to identify severe maternal morbidity from electronic health records. Stud Health Technol Inform 264:143
Lengerich BJ, Caruana R, Weeks WB, Painter I, Spencer S, Sitcov K, Daly C, Souter V (2021) 1017 insights into severe maternal morbidity in the NTSV population. Am J Obstet Gynecol 224(2):629–630
Cartus AR, Naimi AI, Himes KP, Jarlenski M, Parisi SM, Bodnar LM (2021) Can ensemble machine learning improve the accuracy of severe maternal morbidity screening in a perinatal database? Epidemiology 33(1):95–104
Callaghan WM, MacKay AP, Berg CJ (2008) Identification of severe maternal morbidity during delivery hospitalizations, united states, 1991–2003. Am J Obstet Gynecol 199(2):133–1
Lengerich BJ, Caruana R, Weeks WB, Painter I, Spencer S, Sitcov K, Daly C, Souter V (2021) 46 length of labor and severe maternal morbidity in the NTSV population. Am J Obstet Gynecol 224(2):33
Bennett R, Mulla ZD, Parikh P, Hauspurg A, Razzaghi T (2022) An imbalance-aware deep neural network for early prediction of preeclampsia. Plos One 17(4):0266042
Moreira MW, Rodrigues JJ, Oliveira AM, Saleem K, Neto AJV (2017) Predicting hypertensive disorders in high-risk pregnancy using the random forest approach. In: 2017 IEEE International conference on communications (ICC), pp 1–5. IEEE
Bosschieter TM, Xu Z, Lan H, Lengerich B, Nori H, Sitcov K, Painter I, Caruana R, Souter V (2023) Preterm preeclampsia prediction using intelligible machine learning. Am J Obstet Gynecol 228(1):409
Jhee JH, Lee S, Park Y, Lee SE, Kim YA, Kang S-W, Kwon J-Y, Park JT (2019) Prediction model development of late-onset preeclampsia using machine learning-based methods. PLoS One 14(8):0221202
Tsur A, Batsry L, Toussia-Cohen S, Rosenstein M, Barak O, Brezinov Y, Yoeli-Ullman R, Sivan E, Sirota M, Druzin M et al (2020) Development and validation of a machine-learning model for prediction of shoulder dystocia. Ultrasound Obstet Gynecol 56(4):588–596
Bartal MF, Tsur A, Sibai BM, Aran D, Bicocca MJ, Chauhan SP (2021) 651 clinical efficacy of a machine learning model for prediction of shoulder dystocia. Am J Obstet Gynecol 224(2):409
Bosschieter TM, Lan H, Xu Z, Lengerich B, Nori H, Sitcov K, Painter I, Caruana R, Souter V (2023) Unique insights into risk factors for antepartum stillbirth using explainable AI. Am J Obstet Gynecol 228(1):403–404
Allotey J, Whittle R, Snell K, Smuk M, Townsend R, von Dadelszen P, Heazell A, Magee L, Smith G, Sandall J et al (2022) External validation of prognostic models to predict stillbirth using international prediction of pregnancy complications (IPPIC) network database: individual participant data meta-analysis. Ultrasound Obstet Gynecol 59(2):209–219
Malacova E, Tippaya S, Bailey HD, Chai K, Farrant BM, Gebremedhin AT, Leonard H, Marinovich ML, Nassar N, Phatak A et al (2020) Stillbirth risk prediction using machine learning for a large cohort of births from Western Australia, 1980–2015. Sci. Rep. 10(1):1–8
Steyerberg EW, Harrell FE (2016) Prediction models need appropriate internal, internal-external, and external validation. J Clin Epidemiol 69:245–247
Wynants L, Collins GS, Van Calster B (2017) Key steps and common pitfalls in developing and validating risk models. BJOG Int Obstet Gynaecol 124(3):423–432
Kleinrouweler CE, Cheong-See FM, Collins GS, Kwee A, Thangaratinam S, Khan KS, Mol BWJ, Pajkrt E, Moons KG, Schuit E (2016) Prognostic models in obstetrics: available, but far from applicable. Am J Obstet Gynecol 214(1):79–90
Squires DA (2011) The US health system in perspective: a comparison of twelve industrialized nations. Issue Brief (Commonwealth Fund) 16:1–14
Yamada G, Hayakawa K, Asai Y, Matsunaga N, Ohtsu H, Hojo M, Hashimoto M, Kobayashi K, Sasaki R, Okamoto T et al (2021) External validation and update of prediction models for unfavorable outcomes in hospitalized patients with COVID-19 in Japan. J Infect Chemother 27(7):1043–1050
Chen T, Guestrin C (2016) XGBoost: a scalable tree boosting system. In: Proceedings of the 22nd Acm Sigkdd International Conference on Knowledge Discovery and Data Mining, pp 785–794
Breiman L (2001) Random forests. Mach Learn 45(1):5–32
Gardner MW, Dorling S (1998) Artificial neural networks (the multilayer perceptron)–a review of applications in the atmospheric sciences. Atmos Environ 32(14–15):2627–2636
Kleinbaum DG, Dietz K, Gail M, Klein M, Klein M (2002) Logistic regression, pp 536. Springer
Qayyum A, Qadir J, Bilal M, Al-Fuqaha A (2020) Secure and robust machine learning for healthcare: a survey. IEEE Rev Biomed Eng 14:156–180
Hastie T, Tibshirani R (1986) Generalized additive models. Stat Sci 1(3):297–310. https://doi.org/10.1214/ss/1177013604
Lou Y, Caruana R, Gehrke J (2012) Intelligible models for classification and regression. In: Proceedings of the 18th ACM SIGKDD international conference on knowledge discovery and data mining, pp 150–158
Nori H, Jenkins S, Koch P, Caruana R (2019) InterpretML: a unified framework for machine learning interpretability. arXiv preprint arXiv:1909.09223
Lou Y, Caruana R, Gehrke J, Hooker G (2013) Accurate intelligible models with pairwise interactions. In: Proceedings of the 19th ACM SIGKDD international conference on knowledge discovery and data mining, pp 623–631
Zhou J, Tse G, Lee S, Liu T, Wu WK, Cao Z, Zeng DD, Wong ICK, Zhang Q, Cheung BMY (2020) Identifying main and interaction effects of risk factors to predict intensive care admission in patients hospitalized with COVID-19: a retrospective cohort study in Hong Kong. medRxiv
Decroos T, Davis J (2019) Interpretable prediction of goals in soccer. In: Proceedings of the AAAI-20 workshop on artificial intelligence in team sports
Xenopoulos P, Freeman WR, Silva C (2022) Analyzing the differences between professional and amateur esports through win probability. Proceedings of the ACM Web conference 2022:3418–3427
Maxwell AE, Sharma M, Donaldson KA (2021) Explainable boosting machines for slope failure spatial predictive modeling. Remote Sens 13(24):4991
Cannarozzo C (2021) The merger-driven evolution of early-type galaxies and the connection with their dark matter halos
Alt H, Godau M (1995) Computing the Fréchet distance between two polygonal curves. Int J Comput Geom Appl 5(01n02):75–91
(2023) Foundation for health care quality: obstetrical care outcome assessment program (OBCOAP). https://www.qualityhealth.org/obcoap
Economic Innovation Group (2023) Introduction to the Distressed Communities Index (DCI). https://eig.org/distressed-communities/ Accessed 28 Jan 2023
Xu Z, Bosschieter TM, Lan H, Lengerich B, Nori H, Sitcov K, Painter I, Souter V, Caruana R (2023) Predicting severe maternal morbidity at admission for delivery using intelligible machine learning. Am J Obstet Gynecol 228(1):404–405
Lan H, Bosschieter TM, Xu Z, Lengerich B, Nori H, Sitcov K, Painter I, Souter V, Caruana R (2023) Understanding risk factors for shoulder dystocia using interpretable machine learning. Am J Obstet Gynecol 228(1):753
Economic Research Service USDOA (2020) Rural-Urban Commuting Area Codes. https://www.ers.usda.gov/data-products/rural-urban-commuting-area-codes/ Accessed Jan. 28, 2023
Jones CP (2001) Invited commentary: “race,’’ racism, and the practice of epidemiology. Am J Epidemiol 154(4):299–304
Bedoya AD, Economou-Zavlanos NJ, Goldstein BA, Young A, Jelovsek JE, O’Brien C, Parrish AB, Elengold S, Lytle K, Balu S et al (2022) A framework for the oversight and local deployment of safe and high-quality prediction models. J Am Med Inform Assoc 29(9):1631–1636
Paulus JK, Kent DM (2020) Predictably unequal: understanding and addressing concerns that algorithmic clinical prediction may increase health disparities. NPJ Digit Med 3(1):99
Chang C-H, Tan S, Lengerich B, Goldenberg A, Caruana R (2021) How interpretable and trustworthy are GAMs? In: Proceedings of the 27th ACM SIGKDD conference on knowledge discovery & data mining, pp 95–105
We are indebted to Kristin Sitcov, Executive Director Clinical Programs at the Foundation for Health Care Quality (FHCQ), for her support for this project.
Author VS is a medical director at Natera. The other authors declare no competing interests.
Appendix A: Lists of features
Appendix A: Lists of features
For each of the four outcomes, we provide a full list of features used. The features are ranked according to their feature importance from highest (first feature) to lowest (last feature) as computed by the EBM. Feature importance is measured as the mean absolute contribution to the log odds of the risk prediction over all samples.
1.1 A.1 Severe maternal morbidity
(1) Preeclampsia/gestational hypertension, (2) labor type (spontaneous labor or induction of labor), (3) nulliparity, (4) initial cervical dilation, (5) race, (6) Distressed Communities Index quintile (Economic Innovation Group), (7) scheduled cesarean (planned cesarean birth), (8) initial maternal BMI (either pre-pregnancy or at first prenatal visit), (9) labor allowed (vaginal birth attempted), (10) hour of admission, (11) Hispanic/Latina ethnicity, (12) gravidity (the total number of pregnancies the pregnant person has had including the current pregnancy), (13) number of previous stillbirths, (14) gestational diabetes, (15) history of uterine surgery, (16) maternal age, (17) month of admission, (18) IVF, (19) final maternal BMI, (20) cervical ripening, (21) maternal height, (22) history of classical incision, (23) insurance type, (24) absence minimal prenatal care, (25) number of previous cesareans, (26) induced labor indication, (27) history of low vertical incision, (28) parity, (29) membrane status (membranes ruptured or intact at the time of admission to labor and delivery), (30) presentation at delivery, (31) placental abruption, (32) history of uterine rupture, (33) day of week at admission, (34) sex of baby, (35) illicit substance use during pregnancy, (36) pre-pregnancy diagnosis of mental illness, (37) Rural–Urban Commuting Area (RUCA) code class, (38) pre-pregnancy diagnosis of diabetes, (39) ASA recommended for use during pregnancy (documentatoin of low dose aspirin recommendation in the medical record), (40) pre-pregnancy use of nicotine, (41) pre-pregnancy use of marijuana, (42) use of nicotine during pregnancy, (43) pre-pregnancy illicit substance use, (44) chronic hypertension, (45) cholestasis of pregnancy, (46) use of marijuana during pregnancy, (47) pre-pregnancy use of alcohol, (48) use of alcohol during pregnancy, (49) number of previous preterm births.
1.2 A.2 Shoulder dystocia
(1) Birthweight, (2) maternal height, (3) time from admission to complete cervical dilation, (4) final maternal BMI, (5) oxytocin use, (6) race, (7) Distressed Communities Index quintile, (8) Insurance type, (9) initial cervical dilation, (10) use of regional anesthesia, (11) time from complete dilation till delivery, (12) maternal age, (13) nulliparity, (14) number of weeks on delivery, (15) gravidity, (16) initial maternal BMI, (17) indication for operative vaginal delivery, (18) IVF, (19) cervical effacement, (20) month of admission, (21) membrane status, (22) vacuum use, (23) cervical ripening, (24) hour of admission, (25) pre-pregnancy diagnosis of diabetes, (26) pre-pregnancy diagnosis of mental illness, (27) gestational diabetes, (28) pre-pregnancy illicit substance use, (29) Hispanic/Latina Ethnicity, (30) Forceps, (31) day of week at admission, (32) Rural–Urban Commuting Area (RUCA) code class, (33) indication for induced labor, (34) pre-pregnancy use of alcohol, (35) history of uterine rupture, (36) nicotine use during pregnancy, (37) sex of baby, (38) pre-pregnancy use of marijuana, (39) alcohol use during pregnancy, (40) labor type, (41) parity (number of previous births of \(\ge \) 20 weeks’ gestation), (42) cholestasis of pregnancy, (43) preeclampsia/gestational hypertension, (44) use of marijuana during pregnancy, (45) use of illicit substance during pregnancy, (46) absent of minimal prenatal care, (47) ASA recommended for use during pregnancy, (48) pre-pregnancy use of nicotine, (49) number of previous preterm births, (50) number of previous stillbirths, (51) history of uterine surgical history, (52) history of classical incision, (53) pre-pregnancy diagnosis of hypertension, (54) history of low vertical incision, (55) number of previous cesarean births, (56) placental abruption.
1.3 A.3 Preterm preeclampsia
(1) BMI (2) nulliparity, (3) chronic hypertension, (4) maternal age, (5) pre-pregnancy diagnosis of mental illness, (6) number of previous stillbirths, (7) gravidity, (8) race, (9) Distressed Communities Index quintile, (10) number of previous preterm births, (11) Rural–Urban Commuting Area (RUCA) code class, (12) pre-pregnancy diagnosis of diabetes, (13) maternal height, (14) Hispanic/Latina ethnicity, (15) illicit substance use during pregnancy, (16) previous low vertical incision, (17) (other) uterine surgical history, (18) number of previous cesarean births, (19) history of uterine rupture, (20) marijuana use during pregnancy (21) pre-pregnancy use of marijuana, (22) pre-pregnancy illicit substance use, (23) IVF, (24) sex of baby, (25) pre-pregnancy use of alcohol, (26) previous classical cesarean, (27) alcohol use during pregnancy, (28) nicotine use during pregnancy, (29) pre-pregnancy use of nicotine.
1.4 A.4 Antepartum stillbirth
(1) BMI, (2) race, (3) Distressed Communities Index quintile, (4) maternal age, (5) maternal height, (6) Rural–Urban Commuting Area (RUCA) code class, (7) gravidity, (8) pre-pregnancy diagnosis of diabetes, (9) number of previous cesarean births, (10) number of previous stillbirths, (11) pre-pregnancy use of alcohol, (12) pre-pregnancy use of nicotine, (13) IVF, (14) baby gender, (15) nulliparity, (16) history of classical incision, (17) history of uterine surgical history, (18) previous low vertical incision, (19) pre-pregnancy use of marijuana, (20) chronic hypertension, (21) nicotine use in pregnancy, (22) pre-pregnancy diagnosis of mental illness, (23) marijuana use during pregnancy, (24) illicit substance use during pregnancy, (25) Hispanic/Latina ethnicity, (26) number of previous preterm births, (27) pre-pregnancy illicit substance use, (28) alcohol use during pregnancy, (29) history of uterine rupture.
About this article
Cite this article
Bosschieter, T.M., Xu, Z., Lan, H. et al. Interpretable Predictive Models to Understand Risk Factors for Maternal and Fetal Outcomes. J Healthc Inform Res (2023). https://doi.org/10.1007/s41666-023-00151-4