Better Feature Matching for Placental Panorama Construction
Twin-to-twin transfusion syndrome is a potentially fatal placental vascular disease of twin pregnancies. The only definitive treatment is surgical cauterization of problematic vascular formations with a fetal endoscope. This surgery is made difficult by the poor visibility conditions of the intrauterine environment and the limited field of view of the endoscope. There have been efforts to address the limited field of view of fetal endoscopes with algorithms that use visual correspondences between successive fetoscopic video frames to stitch those frames together into a composite map of the placental surface. The existing work, however, has been evaluated primarily on ex vivo images of placentas, which tend to have more visual features and fewer visual distractors than the in vivo images that would be encountered in actual surgical procedures. This work shows that guiding feature matching with deep learned segmentations of placental vessels and grid-based motion statistics can make feature-based registration tractable even in in vivo images that have few distinctive visual features.
KeywordsFeature matching Fetoscopy Grid-based motion statistics Mosaic construction Twin-to-twin transfusion syndrome
This work was supported by the National Institutes of Health grant number T35DK104689 (NIDDK Medical Student Research Fellowship). The authors would like to thank Andreas Lauritzen for his assistance with data collection.
- 2.Bian, J., Lin, W.Y., Matsushita, Y., Yeung, S.K., Nguyen, T.D., Cheng, M.M.: GMS: grid-based motion statistics for fast, ultra-robust feature correspondence. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2828–2837, July 2017Google Scholar
- 3.Daga, P., et al.: Real-time Mosaicing of Fetoscopic Videos Using SIFT, vol. 9786, pp. 9786–9786-7 (2016)Google Scholar
- 6.Gaisser, F., Peeters, S.H.P., Lenseigne, B., Jonker, P.P., Oepkes, D.: Fetoscopic panorama reconstruction: moving from ex-vivo to in-vivo. In: Valdés Hernández, M., González-Castro, V. (eds.) Medical Image Understanding and Analysis, pp. 581–593 (2017)Google Scholar
- 8.Luks, F.: Schematic Illustration of Endoscopic Fetal Surgery for Twin-to-Twin Transfusion Syndrome, December 2009Google Scholar
- 9.Mair, E., Hager, G.D., Burschka, D., Suppa, M., Hirzinger, G.: Adaptive and generic corner detection based on the accelerated segment test. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6312, pp. 183–196. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-15552-9_14CrossRefGoogle Scholar
- 12.Tella-Amo, M., et al.: A combined EM and visual tracking probabilistic model for robust mosaicking: application to fetoscopy. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, June 2016Google Scholar