Placental Vessel Extraction with Shearlets, Laplacian Eigenmaps, and a Conditional Generative Adversarial Network
The placenta is the key organ of maternal–fetal interactions, where nutrient, oxygen, and waste transfer take place. Differences in the morphology of the placental chorionic surface vascular network (PCSVN) have been associated with developmental disorders such as autism, hinting that the PCSVN could potentially serve as a biomarker for early diagnosis and treatment of autism. Studying PCSVN features in large cohorts requires a reliable and automated mechanism to extract the vascular networks. This paper presents two distinct methods for PCSVN enhancement and extraction. Our first algorithm, which builds upon a directional multiscale mathematical framework based on a combination of shearlets and Laplacian eigenmaps, is able to intensify the appearance of vessels with high success in high-contrast images such as those produced in CT scans. Our second algorithm, which applies a conditional generative adversarial neural network (cGAN), was trained to simulate a human-traced PCSVN given a digital photograph of the placental chorionic surface. This method surpasses any existing automated PCSVN extraction methods reported on digital photographs of placentas. We hypothesize that a suitable combination of the two methods could further improve PCSVN extraction results and should be studied in the future.
KeywordsPlacenta Autism Vascular networks Shearlets Wavelets Laplacian eigenmaps Neural networks Deep learning cGAN Generative models
The project was part of the MBI Women Advancing Mathematical Biology: Understanding Complex Biological Systems with Mathematics 2017 Workshop organized by the Association for Women in Mathematics. Funding for the workshop was provided by MBI, NSF ADVANCE “Career Advancement for Women Through Research-Focused Networks” (NSF-HRD 1500481), Society for Mathematical Biology, and Microsoft Research.
Over the course of the project, we received biology expertise and support from Drs. Ruchit Shah, George Merz, and Richard K. Miller.
The authors also wish to thank the following people who contributed to the collection of the placentas in the National Children’s Study Placenta Consortium: C.J. Stodgell, L. Salamone, L.I. Ruffolo, A. Penmetsa, P. Weidenborner (University of Rochester), J. Culhane, S. Wadlinger, M. Pacholski, M.A. Kent, L. Green (University of Pennsylvania), R. Wapner, C. Torres, J. Perou (Columbia University), P. Landrigan, J. Chen, L. Lambertini, L. Littman, P. Sheffield, A. Golden, J. Gilbert, C. Lendor, S. Allen, K. Mantilla, Y. Ma (Ichan School of Medicine), S. Leuthner, S. Szabo (Medical College of Wisconsin), J.L. Dalton, D. Misra (Placental Analytics), N. Thiex, K.Gutzman, A. Martin, B. Specker (South Dakota University), J. Swanson, C. Holliday, J. Butler (University of California at Irvine), A. Li, R.M.A.P.S. Dassanayake, J. Nanes, Y. Xia (University of Illinois at Chicago), J.C. Murray, T.D. Busch, J. Rigdon (University of Iowa), Kjersti Aagaard, A. Harris (Baylor College of Medicine), T.H. Darrah, E. Campbell (Boston University), N. Dole, J. Thorp, B. Eucker, C. Bell (University of North Carolina at Chapel Hill), E.B. Clark, M.W. Varner, E. Taggart, J. Billy, S. Stradling, J. Leavitt, W. Bell, S. Waterfall (University of Utah), B. O’Brien, M. Layton, D. Todd, K. Wilson, M.S. Durkin, M.-N. Sandoval (Westat, Inc).
Most importantly, we thank the study participants who donated their placentas.
- 1.N. Almoussa, B. Dutra, B. Lampe, P. Getreuer, T. Wittman, C. Salafia, L. Vese, Automated vasculature extraction from placenta images, in Proceedings of SPIE Medical Imaging Conference, vol. 7962, 2011Google Scholar
- 5.J.-M. Chang, N. Huynh, M. Vasquez, C. Salafia, Vessel enhancement with multi-scale and curvilinear filer matching for placenta images, in Proceeding of the 2013 20th International Conference on Systems, Signals and Image Processing (IWSSIP), 2013, pp. 125–128Google Scholar
- 6.J.-M. Chang, H. Zeng, Y.-M. Chang, R. Shah, C. Salafia, C. Newschaffer, R. Miller, P. Katzman, M.-F. Moye, C. Walker, L. Croen, Autism risk classification using placental chorionic surface vascular network features. BMC Med. Inform. Decis. Mak. 17(1), 162 (2017)Google Scholar
- 8.R. Coifman, Y. Meyer, V. Wickerhauser, Adapted wave form analysis, wavelet-packets and applications, in Proceedings of the second International Conference on Industrial and Applied Mathematics, ICIAM 91 (Society for Industrial and Applied Mathematics, Philadelphia, 1992), pp. 41–50Google Scholar
- 9.W. Czaja, M. Ehler, Schrödinger Eigenmaps for the analysis of bio-medical data. CoRR, abs/1102.4086, 2011Google Scholar
- 11.M. Do, M. Vetterli, Contourlets: a directional multiresolution image representation, in ICIP (1), 2002, pp. 357–360Google Scholar
- 12.M. Egbor, T. Ansari, N. Morris, C.J. Green, P.D. Sibbons, Morphometric placental villous and vascular abnormalities in early- and late-onset pre-eclampsia with and without fetal growth restriction. J. Obstet. Gynecol. 113(5), 580–589 (2006)Google Scholar
- 15.I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, Y. Bengio, Generative adversarial nets, in Advances in Neural Information Processing Systems, vol. 27, ed. by Z. Ghahramani, M. Welling, C. Cortes, N.D. Lawrence, K.Q. Weinberger (Curran Associates, Inc., Red Hook, 2014), pp. 2672–2680Google Scholar
- 19.J. Ham, D. Lee, S. Mika, B. Schölkopf, A kernel view of the dimensionality reduction of manifolds, in Proceedings of the Twenty-first International Conference on Machine Learning, ICML ’04 (ACM, New York, 2004), pp. 47–55Google Scholar
- 20.S. Ioffe, C. Szegedy, Batch normalization: accelerating deep network training by reducing internal covariate shift, in International Conference on Machine Learning, 2015, pp. 448–456Google Scholar
- 21.P. Isola, J.-Y. Zhu, T. Zhou, A.A. Efros, Image-to-image translation with conditional adversarial networks, in CVPR, 2017Google Scholar
- 22.P. Jarmuzek, M. Wielgos, D.A. Bomba-Opon, Placental pathologic changes in gestational diabetes mellitus. Neuroendocrinol. Lett. 36(2), 101–105 (2015)Google Scholar
- 23.D. Labate, W.-Q. Lim, G. Kutyniok, G. Weiss, Sparse multidimensional representation using shearlets. Opt. Photon. 2005, 59140U (2005)Google Scholar
- 29.C.J. Newschaffer, L.A. Croen, M.D. Fallin, I. Hertz-Picciotto, D.V. Nguyen, N.L. Lee, C.A. Berry, H. Farzadegan, H.N. Hess, R.J. Landa, S.E. Levy, M.L. Massolo, S.C. Meyerer, S.M. Mohammed, M.C. Oliver, S. Ozonoff, J. Pandey, A. Schroeder, K.M. Shedd-Wise, Infant siblings and the investigation of autism risk factors. J. Neurodev. Disord. 4, 1–16 (2012)CrossRefGoogle Scholar
- 31.O. Ronneberger, P. Fischer, T. Brox, U-net: convolutional networks for biomedical image segmentation, in International Conference on Medical Image Computing and Computer-Assisted Intervention (Springer, Berlin, 2015), pp. 234–241Google Scholar
- 32.C.M. Salafia, C. Platt, T. Girardi, R. Shah, G. Merz, D.P. Misra, Placental structure in ASD: does the placenta mirror the diagnosis? in 2014 International Meeting for Autism Research, page Abstract No. 17578, May 14–17 2014Google Scholar
- 35.K. Yacoubou Djima, L. Simonelli, D. Cunnigham, W. Czaja, Detection of anomaly in human retina using Laplacian Eigenmaps and vectorized matched filtering, in Proceedings of SPIE, vol. 9413, Mar 2015, pp. 94132F-1–94132F-11Google Scholar
- 36.S.K. Zhou, H. Greenspan, D. Shen, Deep Learning for Medical Image Analysis (Elsevier Science, Amsterdam, 2017)Google Scholar