Skip to main content
SpringerLink
Log in

Interpretable Predictive Models to Understand Risk Factors for Maternal and Fetal Outcomes

Journal of Healthcare Informatics Research (2023)Cite this article

Abstract

Although most pregnancies result in a good outcome, complications are not uncommon and can be associated with serious implications for mothers and babies. Predictive modeling has the potential to improve outcomes through a better understanding of risk factors, heightened surveillance for high-risk patients, and more timely and appropriate interventions, thereby helping obstetricians deliver better care. We identify and study the most important risk factors for four types of pregnancy complications: (i) severe maternal morbidity, (ii) shoulder dystocia, (iii) preterm preeclampsia, and (iv) antepartum stillbirth. We use an Explainable Boosting Machine (EBM), a high-accuracy glass-box learning method, for the prediction and identification of important risk factors. We undertake external validation and perform an extensive robustness analysis of the EBM models. EBMs match the accuracy of other black-box ML methods, such as deep neural networks and random forests, and outperform logistic regression, while being more interpretable. EBMs prove to be robust. The interpretability of the EBM models reveal surprising insights into the features contributing to risk (e.g., maternal height is the second most important feature for shoulder dystocia) and may have potential for clinical application in the prediction and prevention of serious complications in pregnancy.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Notes

  1. In our work, SMM is a composite term for 6 adverse diagnoses: (i) hysterectomy, (ii) blood transfusion, (iii) disseminated intravascular coagulation, (iv) amniotic fluid embolism, (v) thromboembolism, and (vi) eclampsia.

  2. Note that predicting preterm preeclampsia can only be done in practice \(< 37\) weeks into the pregnancy per definition of preterm.

  3. Labor type refers to whether labor was spontaneous or induced.

References

  1. Osterman MJ, Hamilton BE, Martin JA, Driscoll AK, Valenzuela CP (2022) Births: final data for 2020

  2. Declercq E, Zephyrin L (2021) Severe Maternal Morbidity in the United States: A Primer. Commonwealth Fund

  3. Creanga AA, Bateman BT, Kuklina EV, Callaghan WM (2014) Racial and ethnic disparities in severe maternal morbidity: a multistate analysis, 2008–2010. Am J Obstet Gynecol 210(5):435–1

    Article  Google Scholar 

  4. Gao C, Osmundson S, Yan X, Edwards DV, Malin BA, Chen Y (2019) Learning to identify severe maternal morbidity from electronic health records. Stud Health Technol Inform 264:143

    Google Scholar 

  5. Lengerich BJ, Caruana R, Weeks WB, Painter I, Spencer S, Sitcov K, Daly C, Souter V (2021) 1017 insights into severe maternal morbidity in the NTSV population. Am J Obstet Gynecol 224(2):629–630

    Article  Google Scholar 

  6. Cartus AR, Naimi AI, Himes KP, Jarlenski M, Parisi SM, Bodnar LM (2021) Can ensemble machine learning improve the accuracy of severe maternal morbidity screening in a perinatal database? Epidemiology 33(1):95–104

    Article  Google Scholar 

  7. Callaghan WM, MacKay AP, Berg CJ (2008) Identification of severe maternal morbidity during delivery hospitalizations, united states, 1991–2003. Am J Obstet Gynecol 199(2):133–1

    Article  Google Scholar 

  8. Lengerich BJ, Caruana R, Weeks WB, Painter I, Spencer S, Sitcov K, Daly C, Souter V (2021) 46 length of labor and severe maternal morbidity in the NTSV population. Am J Obstet Gynecol 224(2):33

    Article  Google Scholar 

  9. Bennett R, Mulla ZD, Parikh P, Hauspurg A, Razzaghi T (2022) An imbalance-aware deep neural network for early prediction of preeclampsia. Plos One 17(4):0266042

    Article  Google Scholar 

  10. Moreira MW, Rodrigues JJ, Oliveira AM, Saleem K, Neto AJV (2017) Predicting hypertensive disorders in high-risk pregnancy using the random forest approach. In: 2017 IEEE International conference on communications (ICC), pp 1–5. IEEE

  11. Bosschieter TM, Xu Z, Lan H, Lengerich B, Nori H, Sitcov K, Painter I, Caruana R, Souter V (2023) Preterm preeclampsia prediction using intelligible machine learning. Am J Obstet Gynecol 228(1):409

    Article  Google Scholar 

  12. Jhee JH, Lee S, Park Y, Lee SE, Kim YA, Kang S-W, Kwon J-Y, Park JT (2019) Prediction model development of late-onset preeclampsia using machine learning-based methods. PLoS One 14(8):0221202

    Article  Google Scholar 

  13. Tsur A, Batsry L, Toussia-Cohen S, Rosenstein M, Barak O, Brezinov Y, Yoeli-Ullman R, Sivan E, Sirota M, Druzin M et al (2020) Development and validation of a machine-learning model for prediction of shoulder dystocia. Ultrasound Obstet Gynecol 56(4):588–596

    Article  Google Scholar 

  14. Bartal MF, Tsur A, Sibai BM, Aran D, Bicocca MJ, Chauhan SP (2021) 651 clinical efficacy of a machine learning model for prediction of shoulder dystocia. Am J Obstet Gynecol 224(2):409

    Article  Google Scholar 

  15. Bosschieter TM, Lan H, Xu Z, Lengerich B, Nori H, Sitcov K, Painter I, Caruana R, Souter V (2023) Unique insights into risk factors for antepartum stillbirth using explainable AI. Am J Obstet Gynecol 228(1):403–404

    Article  Google Scholar 

  16. Allotey J, Whittle R, Snell K, Smuk M, Townsend R, von Dadelszen P, Heazell A, Magee L, Smith G, Sandall J et al (2022) External validation of prognostic models to predict stillbirth using international prediction of pregnancy complications (IPPIC) network database: individual participant data meta-analysis. Ultrasound Obstet Gynecol 59(2):209–219

    Article  Google Scholar 

  17. Malacova E, Tippaya S, Bailey HD, Chai K, Farrant BM, Gebremedhin AT, Leonard H, Marinovich ML, Nassar N, Phatak A et al (2020) Stillbirth risk prediction using machine learning for a large cohort of births from Western Australia, 1980–2015. Sci. Rep. 10(1):1–8

    Article  Google Scholar 

  18. Steyerberg EW, Harrell FE (2016) Prediction models need appropriate internal, internal-external, and external validation. J Clin Epidemiol 69:245–247

    Article  Google Scholar 

  19. Wynants L, Collins GS, Van Calster B (2017) Key steps and common pitfalls in developing and validating risk models. BJOG Int Obstet Gynaecol 124(3):423–432

    Article  Google Scholar 

  20. Kleinrouweler CE, Cheong-See FM, Collins GS, Kwee A, Thangaratinam S, Khan KS, Mol BWJ, Pajkrt E, Moons KG, Schuit E (2016) Prognostic models in obstetrics: available, but far from applicable. Am J Obstet Gynecol 214(1):79–90

    Article  Google Scholar 

  21. Squires DA (2011) The US health system in perspective: a comparison of twelve industrialized nations. Issue Brief (Commonwealth Fund) 16:1–14

    Google Scholar 

  22. Yamada G, Hayakawa K, Asai Y, Matsunaga N, Ohtsu H, Hojo M, Hashimoto M, Kobayashi K, Sasaki R, Okamoto T et al (2021) External validation and update of prediction models for unfavorable outcomes in hospitalized patients with COVID-19 in Japan. J Infect Chemother 27(7):1043–1050

    Article  Google Scholar 

  23. Chen T, Guestrin C (2016) XGBoost: a scalable tree boosting system. In: Proceedings of the 22nd Acm Sigkdd International Conference on Knowledge Discovery and Data Mining, pp 785–794

  24. Breiman L (2001) Random forests. Mach Learn 45(1):5–32

    Article  MATH  Google Scholar 

  25. Gardner MW, Dorling S (1998) Artificial neural networks (the multilayer perceptron)–a review of applications in the atmospheric sciences. Atmos Environ 32(14–15):2627–2636

    Article  Google Scholar 

  26. Kleinbaum DG, Dietz K, Gail M, Klein M, Klein M (2002) Logistic regression, pp 536. Springer

  27. Qayyum A, Qadir J, Bilal M, Al-Fuqaha A (2020) Secure and robust machine learning for healthcare: a survey. IEEE Rev Biomed Eng 14:156–180

    Article  Google Scholar 

  28. Hastie T, Tibshirani R (1986) Generalized additive models. Stat Sci 1(3):297–310. https://doi.org/10.1214/ss/1177013604

    Article  MathSciNet  MATH  Google Scholar 

  29. Lou Y, Caruana R, Gehrke J (2012) Intelligible models for classification and regression. In: Proceedings of the 18th ACM SIGKDD international conference on knowledge discovery and data mining, pp 150–158

  30. Nori H, Jenkins S, Koch P, Caruana R (2019) InterpretML: a unified framework for machine learning interpretability. arXiv preprint arXiv:1909.09223

  31. Lou Y, Caruana R, Gehrke J, Hooker G (2013) Accurate intelligible models with pairwise interactions. In: Proceedings of the 19th ACM SIGKDD international conference on knowledge discovery and data mining, pp 623–631

  32. Zhou J, Tse G, Lee S, Liu T, Wu WK, Cao Z, Zeng DD, Wong ICK, Zhang Q, Cheung BMY (2020) Identifying main and interaction effects of risk factors to predict intensive care admission in patients hospitalized with COVID-19: a retrospective cohort study in Hong Kong. medRxiv

  33. Decroos T, Davis J (2019) Interpretable prediction of goals in soccer. In: Proceedings of the AAAI-20 workshop on artificial intelligence in team sports

  34. Xenopoulos P, Freeman WR, Silva C (2022) Analyzing the differences between professional and amateur esports through win probability. Proceedings of the ACM Web conference 2022:3418–3427

    Google Scholar 

  35. Maxwell AE, Sharma M, Donaldson KA (2021) Explainable boosting machines for slope failure spatial predictive modeling. Remote Sens 13(24):4991

    Article  Google Scholar 

  36. Cannarozzo C (2021) The merger-driven evolution of early-type galaxies and the connection with their dark matter halos

  37. Alt H, Godau M (1995) Computing the Fréchet distance between two polygonal curves. Int J Comput Geom Appl 5(01n02):75–91

    Article  MATH  Google Scholar 

  38. (2023) Foundation for health care quality: obstetrical care outcome assessment program (OBCOAP). https://www.qualityhealth.org/obcoap

  39. Economic Innovation Group (2023) Introduction to the Distressed Communities Index (DCI). https://eig.org/distressed-communities/ Accessed 28 Jan 2023

  40. Xu Z, Bosschieter TM, Lan H, Lengerich B, Nori H, Sitcov K, Painter I, Souter V, Caruana R (2023) Predicting severe maternal morbidity at admission for delivery using intelligible machine learning. Am J Obstet Gynecol 228(1):404–405

    Article  Google Scholar 

  41. Lan H, Bosschieter TM, Xu Z, Lengerich B, Nori H, Sitcov K, Painter I, Souter V, Caruana R (2023) Understanding risk factors for shoulder dystocia using interpretable machine learning. Am J Obstet Gynecol 228(1):753

    Article  Google Scholar 

  42. Economic Research Service USDOA (2020) Rural-Urban Commuting Area Codes. https://www.ers.usda.gov/data-products/rural-urban-commuting-area-codes/ Accessed Jan. 28, 2023

  43. Jones CP (2001) Invited commentary: “race,’’ racism, and the practice of epidemiology. Am J Epidemiol 154(4):299–304

    Article  Google Scholar 

  44. Bedoya AD, Economou-Zavlanos NJ, Goldstein BA, Young A, Jelovsek JE, O’Brien C, Parrish AB, Elengold S, Lytle K, Balu S et al (2022) A framework for the oversight and local deployment of safe and high-quality prediction models. J Am Med Inform Assoc 29(9):1631–1636

    Article  Google Scholar 

  45. Paulus JK, Kent DM (2020) Predictably unequal: understanding and addressing concerns that algorithmic clinical prediction may increase health disparities. NPJ Digit Med 3(1):99

    Article  Google Scholar 

  46. Chang C-H, Tan S, Lengerich B, Goldenberg A, Caruana R (2021) How interpretable and trustworthy are GAMs? In: Proceedings of the 27th ACM SIGKDD conference on knowledge discovery & data mining, pp 95–105

Download references

Acknowledgements

We are indebted to Kristin Sitcov, Executive Director Clinical Programs at the Foundation for Health Care Quality (FHCQ), for her support for this project.

Author information

Authors and Affiliations

  1. Stanford University, Stanford, CA, USA

    Tomas M. Bosschieter, Zifei Xu & Hui Lan

  2. Massachusetts Institute of Technology, Cambridge, MA, USA

    Benjamin J. Lengerich

  3. Microsoft Research, Redmond, WA, USA

    Harsha Nori & Rich Caruana

  4. Foundation for Healthcare Quality, Seattle, WA, USA

    Ian Painter & Vivienne Souter

Authors
  1. Tomas M. Bosschieter
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zifei Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hui Lan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Benjamin J. Lengerich
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Harsha Nori
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Ian Painter
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Vivienne Souter
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Rich Caruana
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

TMB, VS, and RC developed and planned the experiments. TMB, ZH, and HL carried out the experiments. BJL and HN helped design the framework underlying the experiments. IP and VS prepared and enabled access to the data. VS provided domain expertise. RC developed and supervised the project. All authors took part in writing the manuscript.

Corresponding authors

Correspondence to Tomas M. Bosschieter, Vivienne Souter or Rich Caruana.

Ethics declarations

Competing interests

Author VS is a medical director at Natera. The other authors declare no competing interests.

Appendix A: Lists of features

Appendix A: Lists of features

For each of the four outcomes, we provide a full list of features used. The features are ranked according to their feature importance from highest (first feature) to lowest (last feature) as computed by the EBM. Feature importance is measured as the mean absolute contribution to the log odds of the risk prediction over all samples.

1.1 A.1 Severe maternal morbidity

(1) Preeclampsia/gestational hypertension, (2) labor type (spontaneous labor or induction of labor), (3) nulliparity, (4) initial cervical dilation, (5) race, (6) Distressed Communities Index quintile (Economic Innovation Group), (7) scheduled cesarean (planned cesarean birth), (8) initial maternal BMI (either pre-pregnancy or at first prenatal visit), (9) labor allowed (vaginal birth attempted), (10) hour of admission, (11) Hispanic/Latina ethnicity, (12) gravidity (the total number of pregnancies the pregnant person has had including the current pregnancy), (13) number of previous stillbirths, (14) gestational diabetes, (15) history of uterine surgery, (16) maternal age, (17) month of admission, (18) IVF, (19) final maternal BMI, (20) cervical ripening, (21) maternal height, (22) history of classical incision, (23) insurance type, (24) absence minimal prenatal care, (25) number of previous cesareans, (26) induced labor indication, (27) history of low vertical incision, (28) parity, (29) membrane status (membranes ruptured or intact at the time of admission to labor and delivery), (30) presentation at delivery, (31) placental abruption, (32) history of uterine rupture, (33) day of week at admission, (34) sex of baby, (35) illicit substance use during pregnancy, (36) pre-pregnancy diagnosis of mental illness, (37) Rural–Urban Commuting Area (RUCA) code class, (38) pre-pregnancy diagnosis of diabetes, (39) ASA recommended for use during pregnancy (documentatoin of low dose aspirin recommendation in the medical record), (40) pre-pregnancy use of nicotine, (41) pre-pregnancy use of marijuana, (42) use of nicotine during pregnancy, (43) pre-pregnancy illicit substance use, (44) chronic hypertension, (45) cholestasis of pregnancy, (46) use of marijuana during pregnancy, (47) pre-pregnancy use of alcohol, (48) use of alcohol during pregnancy, (49) number of previous preterm births.

1.2 A.2 Shoulder dystocia

(1) Birthweight, (2) maternal height, (3) time from admission to complete cervical dilation, (4) final maternal BMI, (5) oxytocin use, (6) race, (7) Distressed Communities Index quintile, (8) Insurance type, (9) initial cervical dilation, (10) use of regional anesthesia, (11) time from complete dilation till delivery, (12) maternal age, (13) nulliparity, (14) number of weeks on delivery, (15) gravidity, (16) initial maternal BMI, (17) indication for operative vaginal delivery, (18) IVF, (19) cervical effacement, (20) month of admission, (21) membrane status, (22) vacuum use, (23) cervical ripening, (24) hour of admission, (25) pre-pregnancy diagnosis of diabetes, (26) pre-pregnancy diagnosis of mental illness, (27) gestational diabetes, (28) pre-pregnancy illicit substance use, (29) Hispanic/Latina Ethnicity, (30) Forceps, (31) day of week at admission, (32) Rural–Urban Commuting Area (RUCA) code class, (33) indication for induced labor, (34) pre-pregnancy use of alcohol, (35) history of uterine rupture, (36) nicotine use during pregnancy, (37) sex of baby, (38) pre-pregnancy use of marijuana, (39) alcohol use during pregnancy, (40) labor type, (41) parity (number of previous births of \(\ge \) 20 weeks’ gestation), (42) cholestasis of pregnancy, (43) preeclampsia/gestational hypertension, (44) use of marijuana during pregnancy, (45) use of illicit substance during pregnancy, (46) absent of minimal prenatal care, (47) ASA recommended for use during pregnancy, (48) pre-pregnancy use of nicotine, (49) number of previous preterm births, (50) number of previous stillbirths, (51) history of uterine surgical history, (52) history of classical incision, (53) pre-pregnancy diagnosis of hypertension, (54) history of low vertical incision, (55) number of previous cesarean births, (56) placental abruption.

1.3 A.3 Preterm preeclampsia

(1) BMI (2) nulliparity, (3) chronic hypertension, (4) maternal age, (5) pre-pregnancy diagnosis of mental illness, (6) number of previous stillbirths, (7) gravidity, (8) race, (9) Distressed Communities Index quintile, (10) number of previous preterm births, (11) Rural–Urban Commuting Area (RUCA) code class, (12) pre-pregnancy diagnosis of diabetes, (13) maternal height, (14) Hispanic/Latina ethnicity, (15) illicit substance use during pregnancy, (16) previous low vertical incision, (17) (other) uterine surgical history, (18) number of previous cesarean births, (19) history of uterine rupture, (20) marijuana use during pregnancy (21) pre-pregnancy use of marijuana, (22) pre-pregnancy illicit substance use, (23) IVF, (24) sex of baby, (25) pre-pregnancy use of alcohol, (26) previous classical cesarean, (27) alcohol use during pregnancy, (28) nicotine use during pregnancy, (29) pre-pregnancy use of nicotine.

1.4 A.4 Antepartum stillbirth

(1) BMI, (2) race, (3) Distressed Communities Index quintile, (4) maternal age, (5) maternal height, (6) Rural–Urban Commuting Area (RUCA) code class, (7) gravidity, (8) pre-pregnancy diagnosis of diabetes, (9) number of previous cesarean births, (10) number of previous stillbirths, (11) pre-pregnancy use of alcohol, (12) pre-pregnancy use of nicotine, (13) IVF, (14) baby gender, (15) nulliparity, (16) history of classical incision, (17) history of uterine surgical history, (18) previous low vertical incision, (19) pre-pregnancy use of marijuana, (20) chronic hypertension, (21) nicotine use in pregnancy, (22) pre-pregnancy diagnosis of mental illness, (23) marijuana use during pregnancy, (24) illicit substance use during pregnancy, (25) Hispanic/Latina ethnicity, (26) number of previous preterm births, (27) pre-pregnancy illicit substance use, (28) alcohol use during pregnancy, (29) history of uterine rupture.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and Permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bosschieter, T.M., Xu, Z., Lan, H. et al. Interpretable Predictive Models to Understand Risk Factors for Maternal and Fetal Outcomes. J Healthc Inform Res (2023). https://doi.org/10.1007/s41666-023-00151-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s41666-023-00151-4

Keywords

search

Navigation