More Knowledge Is Better: Cross-Modality Volume Completion and 3D+2D Segmentation for Intracardiac Echocardiography Contouring

  • Haofu LiaoEmail author
  • Yucheng Tang
  • Gareth Funka-Lea
  • Jiebo Luo
  • Shaohua Kevin Zhou
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11071)


Using catheter ablation to treat atrial fibrillation increasingly relies on intracardiac echocardiography (ICE) for an anatomical delineation of the left atrium and the pulmonary veins that enter the atrium. However, it is a challenge to build an automatic contouring algorithm because ICE is noisy and provides only a limited 2D view of the 3D anatomy. This work provides the first automatic solution to segment the left atrium and the pulmonary veins from ICE. In this solution, we demonstrate the benefit of building a cross-modality framework that can leverage a database of diagnostic images to supplement the less available interventional images. To this end, we develop a novel deep neural network approach that uses the (i) 3D geometrical information provided by a position sensor embedded in the ICE catheter and the (ii) 3D image appearance information from a set of computed tomography cardiac volumes. We evaluate the proposed approach over 11,000 ICE images collected from 150 clinical patients. Experimental results show that our model is significantly better than a direct 2D image-to-image deep neural network segmentation, especially for less-observed structures.


  1. 1.
    Allan, G., et al.: Simultaneous analysis of 2D echo views for left atrial segmentation and disease detection. IEEE TMI 36(1), 40–50 (2017)MathSciNetGoogle Scholar
  2. 2.
    Bartel, T., Müller, S., Biviano, A., Hahn, R.T.: Why is intracardiac echocardiography helpful? Benefits, costs, and how to learn. Eur. Hear. J. 35(2), 69–76 (2013)CrossRefGoogle Scholar
  3. 3.
    Caruana, R.: Multitask learning. In: Thrun, S., Pratt, L. (eds.) Learning to Learn, pp. 95–133. Springer, Boston (1998). Scholar
  4. 4.
    Isola, P., Zhu, J.Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: Proceedings of the CVPR, pp. 1125–1134 (2017)Google Scholar
  5. 5.
    Lin, N., Yu, W., Duncan, J.S.: Combinative multi-scale level set framework for echo image segmentation. Med. Image Anal. 7(4), 529–537 (2003)CrossRefGoogle Scholar
  6. 6.
    Pathak, D., Krahenbuhl, P., Donahue, J., Darrell, T., Efros, A.A.: Context encoders: feature learning by inpainting. In: Proceedings of tne CVPR, pp. 2536–2544 (2016)Google Scholar
  7. 7.
    Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). Scholar
  8. 8.
    Sánchez-Quintana, D., López-Mínguez, J.R., Macías, Y., Cabrera, J.A., Saremi, F.: Left atrial anatomy relevant to catheter ablation. Cardiology Research and Practice (2014)Google Scholar
  9. 9.
    Sarti, A., Corsi, C., Mazzini, E., Lamberti, C.: Maximum likelihood segmentation of ultrasound images with rayleigh distribution. IEEE Trans. Ultrason. Ferroelectr. Freq. Control. 52(6), 947–960 (2005)CrossRefGoogle Scholar
  10. 10.
    Schönemann, P.H.: A generalized solution of the orthogonal procrustes problem. Psychometrika 31(1), 1–10 (1966)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Zhou, S.K.: Shape regression machine and efficient segmentation of left ventricle endocardium from 2D B-mode echocardiogram. Med. Image Anal. 14(4), 563–581 (2010)CrossRefGoogle Scholar
  12. 12.
    Zhou, S., Shen, D., Greenspan, H. (eds.): Deep Learning for Medical Image Analysis. Academic Press, Cambridge (2017)Google Scholar
  13. 13.
    Zoni-Berisso, M., Lercari, F., Carazza, T., Domenicucci, S.: Epidemiology of atrial fibrillation: european perspective. Clinical Epidemiol. 6, 213–20 (2014)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Haofu Liao
    • 1
    Email author
  • Yucheng Tang
    • 3
  • Gareth Funka-Lea
    • 2
  • Jiebo Luo
    • 1
  • Shaohua Kevin Zhou
    • 4
  1. 1.Department of Computer ScienceUniversity of RochesterRochesterUSA
  2. 2.Medical Imaging Technologies, Siemens HealthineersPrincetonUSA
  3. 3.Department of Electrical and Computer EngineeringNew York UniversityNew YorkUSA
  4. 4.Institute of Computing Technology, Chinese Academy of SciencesBeijingChina

Personalised recommendations