Class-Balanced Deep Neural Network for Automatic Ventricular Structure Segmentation

  • Xin Yang
  • Cheng Bian
  • Lequan Yu
  • Dong Ni
  • Pheng-Ann Heng
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10663)


Segmenting ventricular structures from cardiovascular MR scan is important for quantitative evaluation of heart. Manual delineation is time-consuming and tedious and lack of reproductivity. Considering MR image quality, heart variance, spatial inconsistency and motion artifacts during scanning, it is still a non-trivial task for automatic segmentation methods. In this paper, we propose a general and fully automatic solution to concurrently segment three important ventricular structures. Rooting in the deep learning trend, our method starts from 3D Fully Convolutional Network (3D FCN). We then enhance the 3D FCN with two well-verified blocks: (1) we conduct transfer learning between a pre-trained C3D model and our 3D FCN to get good initialization and thus suppress overfitting. (2) since boosting the gradient flow in network is beneficial to promote segmentation performance, we attach several auxiliary loss functions so as to expose early layers to better supervision. Because the volume size imbalance among different ventricular structures often biases the training of our 3D FCN, to this end, we investigate the capacity of different loss functions and propose a Multi-class Dice Similarity Coefficient (mDSC) based loss function to re-weight the training for all classes. We verified our method, especially the significance of mDSC, on the Automated Cardiac Diagnosis Challenge 2017 datasets for MR image segmentation. Extensive experimental results demonstrate the promising performance of our method.



The work in this paper was supported by the grant from National Natural Science Foundation of China under Grant 61571304, a grant from Hong Kong Research Grants Council (Project no. GRF 14203115), a grant from the National Natural Science Foundation of China (Project No. 61233012) and a grant from Shenzhen Science and Technology Program (JCYJ20170413162256793).


  1. 1.
    Chen, H., Ni, D., Qin, J., et al.: Standard plane localization in fetal ultrasound via domain transferred deep neural networks. IEEE JBHI 19(5), 1627–1636 (2015)Google Scholar
  2. 2.
    Çiçek, Ö., Abdulkadir, A., et al.: 3D U-net: learning dense volumetric segmentation from sparse annotation. arXiv preprint arXiv:1606.06650 (2016)
  3. 3.
    Dou, Q., Yu, L., et al.: 3D deeply supervised network for automated segmentation of volumetric medical images. Med. Image Anal. 41, 40–54 (2017)CrossRefGoogle Scholar
  4. 4.
    Kaus, M.R., von Berg, J., Weese, J., et al.: Automated segmentation of the left ventricle in cardiac MRI. Med. Image Anal. 8(3), 245–254 (2004)CrossRefGoogle Scholar
  5. 5.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: NIPS, pp. 1097–1105 (2012)Google Scholar
  6. 6.
    Lee, C.Y., Xie, S., Gallagher, P., Zhang, Z., Tu, Z.: Deeply-supervised nets (2015)Google Scholar
  7. 7.
    Liu, Y., Captur, G., Moon, J.C., Guo, S., Yang, X., Zhang, S., Li, C.: Distance regularized two level sets for segmentation of left and right ventricles from cine-MRI. Magn. Reson. Imaging 34(5), 699–706 (2016)CrossRefGoogle Scholar
  8. 8.
    Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: CVPR, pp. 3431–3440 (2015)Google Scholar
  9. 9.
    Milletari, F., Navab, N., et al.: V-net: fully convolutional neural networks for volumetric medical image segmentation. In: 2016 Fourth International Conference on 3D Vision (3DV), pp. 565–571. IEEE (2016)Google Scholar
  10. 10.
    Peng, P., Lekadir, K., Gooya, A., Shao, L., Petersen, S.E., Frangi, A.F.: A review of heart chamber segmentation for structural and functional analysis using cardiac magnetic resonance imaging. Magma 29, 155 (2016)CrossRefGoogle Scholar
  11. 11.
    Peters, J., Ecabert, O., Meyer, C., Schramm, H., Kneser, R., Groth, A., Weese, J.: Automatic whole heart segmentation in static magnetic resonance image volumes. In: Ayache, N., Ourselin, S., Maeder, A. (eds.) MICCAI 2007. LNCS, vol. 4792, pp. 402–410. Springer, Heidelberg (2007). CrossRefGoogle Scholar
  12. 12.
    Pizer, S.M., Amburn, E.P., et al.: Adaptive histogram equalization and its variations. Comput. Vis. Graph. Image Process. 39(3), 355–368 (1987)CrossRefGoogle Scholar
  13. 13.
    Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). CrossRefGoogle Scholar
  14. 14.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
  15. 15.
    Tran, D., Bourdev, L., Fergus, R., Torresani, L., Paluri, M.: Learning spatiotemporal features with 3D convolutional networks. In: ICCV, pp. 4489–4497 (2015)Google Scholar
  16. 16.
    Tran, P.V.: A fully convolutional neural network for cardiac segmentation in short-axis MRI. arXiv preprint arXiv:1604.00494 (2016)
  17. 17.
    Xie, S., Tu, Z.: Holistically-nested edge detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1395–1403 (2015)Google Scholar
  18. 18.
    Yu, L., Yang, X., Qin, J., Heng, P.-A.: 3D FractalNet: dense volumetric segmentation for cardiovascular MRI volumes. In: Zuluaga, M.A., Bhatia, K., Kainz, B., Moghari, M.H., Pace, D.F. (eds.) RAMBO/HVSMR -2016. LNCS, vol. 10129, pp. 103–110. Springer, Cham (2017). CrossRefGoogle Scholar
  19. 19.
    Zhuang, X.: Challenges and methodologies of fully automatic whole heart segmentation: a review. J. Healthc. Eng. 4(3), 371–408 (2013)CrossRefGoogle Scholar
  20. 20.
    Zhuang, X., Rhode, K.S., Razavi, R.S., Hawkes, D.J., Ourselin, S.: A registration-based propagation framework for automatic whole heart segmentation of cardiac MRI. IEEE Trans. Med. Imaging 29(9), 1612–1625 (2010)CrossRefGoogle Scholar
  21. 21.
    Zotti, C., Luo, Z., et al.: Novel deep convolution neural network applied to MRI cardiac segmentation. arXiv preprint arXiv:1705.08943 (2017)

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Department of Computer Science and EngineeringThe Chinese University of Hong KongSha TinHong Kong
  2. 2.National-Regional Key Technology Engineering Laboratory for Medical Ultrasound, School of Biomedical Engineering, Health Science CenterShenzhen UniversityShenzhenChina
  3. 3.Guangdong Provincial Key Laboratory of Computer Vision and Virtual Reality Technology, Shenzhen Institutes of Advanced TechnologyChinese Academy of SciencesShenzhenChina

Personalised recommendations