Automatic Whole Heart Segmentation Using Deep Learning and Shape Context

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10663)


To assist 3D cardiac image analysis, we propose an automatic whole heart segmentation using a deep learning framework combined with shape context information that is encoded in volumetric shape models. The proposed processing pipeline consists of three major steps: scout segmentation with orthogonal 2D U-nets, shape context estimation and refining segmentation with U-net and shape context. The proposed method was evaluated using the MMWHS challenge data. Two sets of networks were trained separately for contrast-enhanced CT and MRI. On the 20 training datasets, using 5-fold cross-validation, the average Dice coefficients for the left ventricle, the right ventricle, the left atrium, the right atrium and the myocardium of the left ventricle were 0.895, 0.795, 0.847, 0.821, 0.807 for MRI and 0.935, 0.825, 0.908, 0.881, 0.879 for CT, respectively. Further improvement may be possible given more training data or advanced data augmentation strategy.


Deep learning Fully convolutional network Heart segmentation Shape context Statistic shape model 



This research has been partially funded by the Swedish Research Council (VR), grant no. 2014-6153, and the Swedish Heart-Lung Foundation (HLF), grant no. 2016-0609.


  1. 1.
    Lloyd-Jones, D.M., Larson, M.G., Beiser, A., Levy, D.: Lifetime risk of developing coronary heart disease. Lancet 353(9147), 89–92 (1999)CrossRefGoogle Scholar
  2. 2.
    Zhang, X., Cowan, B.R., Bluemke, D.A., Finn, J.P., Fonseca, C.G., Kadish, A.H., Lee, D.C., Lima, J.A.C., Suinesiaputra, A., Young, A.A., et al.: Atlas-based quantification of cardiac remodeling due to myocardial infarction. PLoS ONE 9(10), e110243 (2014)CrossRefGoogle Scholar
  3. 3.
    Bergström, G., Berglund, G., Anders Blomberg, J., Brandberg, G.E., Jan Engvall, M., Eriksson, U.F., Flinck, A., Hansson, M.G., et al.: The swedish cardiopulmonary bioimage study: objectives and design. J. Intern. Med. 278(6), 645–659 (2015)CrossRefGoogle Scholar
  4. 4.
    Zheng, Y., Barbu, A., Georgescu, B., Scheuering, M., Comaniciu, D.: Four-chamber heart modeling and automatic segmentation for 3-D cardiac CT volumes using marginal space learning and steerable features. IEEE Trans. Med. Imaging 27(11), 1668–1681 (2008)CrossRefGoogle Scholar
  5. 5.
    Zhuang, X., Shen, J.: Multi-scale patch and multi-modality atlases for whole heart segmentation of MRI. Med. Image Anal. 31, 77–87 (2016)CrossRefGoogle Scholar
  6. 6.
    Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3431–3440 (2015)Google Scholar
  7. 7.
    Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). CrossRefGoogle Scholar
  8. 8.
    Leventon, M.E., Grimson, W.E.L., Faugeras, O.: Statistical shape influence in geodesic active contours. In: 2000 Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 316–323. IEEE (2000)Google Scholar
  9. 9.
    Wang, C., Frimmel, H., Smedby, Ö.: Fast level-set based image segmentation using coherent propagation. Med. Phys. 41(7), 073501 (2014)CrossRefGoogle Scholar
  10. 10.
    Wang, C., Smedby, Ö.: Automatic multi-organ segmentation in non-enhanced CT datasets using hierarchical shape priors. In: Proceedings of the 22nd International Conference on Pattern Recognition (ICPR). IEEE (2014)Google Scholar
  11. 11.
    Wang, C., Wang, Q., Smedby, Ö.: Automatic heart and vessel segmentation using random forests and a local phase guided level set method. In: Zuluaga, M.A., Bhatia, K., Kainz, B., Moghari, M.H., Pace, D.F. (eds.) RAMBO/HVSMR -2016. LNCS, vol. 10129, pp. 159–164. Springer, Cham (2017). CrossRefGoogle Scholar
  12. 12.
    Tu, Z., Bai, X.: Auto-context and its application to high-level vision tasks and 3D brain image segmentation. IEEE Trans. Patt. Anal. Mach. Intell. 32(10), 1744–1757 (2010)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.School for Technology and Health (STH)KTH Royal Institute of TechnologyHuddingeSweden

Personalised recommendations