Abstract
A major challenge in clinical In-Vitro Fertilization (IVF) is selecting the highest quality embryo to transfer to the patient in the hopes of achieving a pregnancy. Time-lapse microscopy provides clinicians with a wealth of information for selecting embryos. However, the resulting movies of embryos are currently analyzed manually, which is time consuming and subjective. Here, we automate feature extraction of time-lapse microscopy of human embryos with a machine-learning pipeline of five convolutional neural networks (CNNs). Our pipeline consists of (1) semantic segmentation of the regions of the embryo, (2) regression predictions of fragment severity, (3) classification of the developmental stage, and object instance segmentation of (4) cells and (5) pronuclei. Our approach greatly speeds up the measurement of quantitative, biologically relevant features that may aid in embryo selection.
B. D. Leahy, W.-D. Jang, H. Y. Yang—These authors contributed equally to this work.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Alikani, M., Cohen, J., Tomkin, G., Garrisi, G.J., Mack, C., Scott, R.T.: Human embryo fragmentation in vitro and its implications for pregnancy and implantation. Fertil. Steril. 71(5), 836–842 (1999)
Amir, H., et al.: Time-lapse imaging reveals delayed development of embryos carrying unbalanced chromosomal translocations. J. Assist. Reprod. Genet. 36(2), 315–324 (2018)
Armstrong, S., Bhide, P., Jordan, V., Pacey, A., Marjoribanks, J., Farquhar, C.: Time-lapse systems for embryo incubation and assessment in assisted reproduction. Cochrane Database Syst. Rev. (5), Art. No. CD011320 (2019)
Bellman, R.: Dynamic programming. Science 153(3731), 34–37 (1966)
Broughton, D.E., Moley, K.H.: Obesity and female infertility: potential mediators of obesity’s impact. Fertil. Steril. 107(4), 840–847 (2017)
Cohen, J., Alikani, M., Trowbridge, J., Rosenwaks, Z.: Implantation enhancement by selective assisted hatching using zona drilling of human embryos with poor prognosis. Hum. Reprod. 7(5), 685–691 (1992)
Cui, W.: Mother or nothing: the agony of infertility. Bull. World Health Organ. 88, 881 (2010)
Dai, J., He, K., Sun, J.: Instance-aware semantic segmentation via multi-task network cascades. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3150–3158 (2016)
Dolinko, A.V., Farland, L., Kaser, D., Missmer, S., Racowsky, C.: National survey on use of time-lapse imaging systems in IVF laboratories. J. Assist. Reprod. Genet. 34(9), 1167–1172 (2017)
Elder, K., Dale, B.: In-Vitro Fertilization. Cambridge University Press, Cambridge (2020)
Filho, E.S., Noble, J., Poli, M., Griffiths, T., Emerson, G., Wells, D.: A method for semi-automatic grading of human blastocyst microscope images. Hum. Reprod. 27(9), 2641–2648 (2012)
Franasiak, J.M., et al.: The nature of aneuploidy with increasing age of the female partner: a review of 15,169 consecutive trophectoderm biopsies evaluated with comprehensive chromosomal screening. Fertil. Steril. 101(3), 656–663 (2014)
Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. MIT Press, Cambridge (2016)
He, K., Gkioxari, G., Dollár, P., Girshick, R.: Mask R-CNN. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2961–2969 (2017)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Hoffman, R., Gross, L.: Modulation contrast microscope. Appl. Opt. 14(5), 1169–1176 (1975)
Khan, A., Gould, S., Salzmann, M.: Deep convolutional neural networks for human embryonic cell counting. In: Hua, G., Jégou, H. (eds.) ECCV 2016. LNCS, vol. 9913, pp. 339–348. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46604-0_25
Kheradmand, S., Singh, A., Saeedi, P., Au, J., Havelock, J.: Inner cell mass segmentation in human HMC embryo images using fully convolutional network. In: 2017 IEEE International Conference on Image Processing (ICIP), pp. 1752–1756. IEEE (2017)
Khosravi, P., et al.: Deep learning enables robust assessment and selection of human blastocysts after in vitro fertilization. NPJ Digit. Med. 2(1), 1–9 (2019)
Kragh, M.F., Rimestad, J., Berntsen, J., Karstoft, H.: Automatic grading of human blastocysts from time-lapse imaging. Comput. Biol. Med. 115, 103494 (2019)
Lau, T., Ng, N., Gingold, J., Desai, N., McAuley, J., Lipton, Z.C.: Embryo staging with weakly-supervised region selection and dynamically-decoded predictions. arXiv preprint arXiv:1904.04419 (2019)
Lin, T.-Y., et al.: Microsoft COCO: common objects in context. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8693, pp. 740–755. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10602-1_48
Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3431–3440 (2015)
Nickkho-Amiry, M., Horne, G., Akhtar, M., Mathur, R., Brison, D.R.: Hydatidiform molar pregnancy following assisted reproduction. J. Assist. Reprod. Genet. 36(4), 667–671 (2019). https://doi.org/10.1007/s10815-018-1389-9
Norwitz, E.R., Edusa, V., Park, J.S.: Maternal physiology and complications of multiple pregnancy. Semin. Perinatol. 29(5), 338–348 (2005)
Petersen, B.M., Boel, M., Montag, M., Gardner, D.K.: Development of a generally applicable morphokinetic algorithm capable of predicting the implantation potential of embryos transferred on day 3. Hum. Reprod. 31(10), 2231–2244 (2016)
Racowsky, C., Stern, J.E., Gibbons, W.E., Behr, B., Pomeroy, K.O., Biggers, J.D.: National collection of embryo morphology data into society for assisted reproductive technology clinic outcomes reporting system: associations among day 3 cell number, fragmentation and blastomere asymmetry, and live birth rate. Fertil. Steril. 95(6), 1985–1989 (2011)
Rad, R.M., Saeedi, P., Au, J., Havelock, J.: A hybrid approach for multiple blastomeres identification in early human embryo images. Comput. Biol. Med. 101, 100–111 (2018)
Practice Committee of the American Society for Reproductive Medicine: Guidance on the limits to the number of embryos to transfer a committee opinion. Fertil. Steril. 107(4), 901 (2017)
Rubio, I., et al.: Clinical validation of embryo culture and selection by morphokinetic analysis: a randomized, controlled trial of the embryoscope. Fertil. Steril. 102(5), 1287–1294 (2014)
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2818–2826 (2016)
Tran, D., Cooke, S., Illingworth, P., Gardner, D.: Deep learning as a predictive tool for fetal heart pregnancy following time-lapse incubation and blastocyst transfer. Hum. Reprod. 34(6), 1011–1018 (2019)
Xie, S., Girshick, R., Dollár, P., Tu, Z., He, K.: Aggregated residual transformations for deep neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1492–1500 (2017)
Acknowledgements
We acknowledge M. Venturas and P. Maeder-York for help validating labels and approaches. This work was funded in part by NIH grant 5U54CA225088 and NSF Grant NCS-FO 1835231, by the NSF-Simons Center for Mathematical and Statistical Analysis of Biology at Harvard (award number 1764269), and by the Harvard Quantitative Biology Initiative. DJN and DBY also acknowledge generous support from the Perelson family, which made this work possible.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Supplementary material 1 (mp4 1601 KB)
Supplementary material 2 (mp4 2362 KB)
Supplementary material 3 (mp4 2090 KB)
Supplementary material 4 (mp4 2146 KB)
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Leahy, B.D. et al. (2020). Automated Measurements of Key Morphological Features of Human Embryos for IVF. In: Martel, A.L., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2020. MICCAI 2020. Lecture Notes in Computer Science(), vol 12265. Springer, Cham. https://doi.org/10.1007/978-3-030-59722-1_3
Download citation
DOI: https://doi.org/10.1007/978-3-030-59722-1_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-59721-4
Online ISBN: 978-3-030-59722-1
eBook Packages: Computer ScienceComputer Science (R0)
