Incorporating prior shape knowledge via data-driven loss model to improve 3D liver segmentation in deep CNNs

  • Saeed Mohagheghi
  • Amir Hossein ForuzanEmail author
Original Article



Convolutional neural networks (CNNs) have obtained enormous success in liver segmentation. However, there are several challenges, including low-contrast images, and large variations in the shape, and appearance of the liver. Incorporating prior knowledge in deep CNN models improves their performance and generalization.


A convolutional denoising auto-encoder is utilized to learn global information about 3D liver shapes in a low-dimensional latent space. Then, the deep data-driven knowledge is used to define a loss function and combine it with the Dice loss in the main segmentation model. The resultant hybrid model would be forced to learn the global shape information as prior knowledge, while it tries to produce accurate results and increase the Dice score.


The proposed training strategy improved the performance of the 3D U-Net model and reached the Dice score of 97.62% on the Sliver07-I liver dataset, which is competitive to the state-of-the-art automatic segmentation methods. The proposed algorithm enhanced the generalization and robustness of the hybrid model and outperformed the 3D U-Net model in the prediction of unseen images.


The results indicate that the incorporation of prior shape knowledge enhances liver segmentation tasks in deep CNN models. The proposed method improves the generalization and robustness of the hybrid model due to the abstract features provided by the data-driven loss model.


Deep learning Convolutional neural network 3D liver segmentation Prior knowledge 


Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical standard

We further confirm that any aspect of the work covered in this manuscript that has involved either experimental animals or human patients has been conducted with the ethical approval of all relevant bodies and that such approvals are acknowledged within the manuscript.


  1. 1.
    Campadelli P, Casiraghi E, Esposito A (2009) Liver segmentation from computed tomography scans: a survey and a new algorithm. Artif Intell Med 45:185–196CrossRefGoogle Scholar
  2. 2.
    Heimann T, Van Ginneken B, Styner MA, Arzhaeva Y, Aurich V, Bauer C, Beck A, Becker C, Beichel R, Bekes G (2009) Comparison and evaluation of methods for liver segmentation from CT datasets. IEEE Trans Med Imaging 28:1251–1265CrossRefGoogle Scholar
  3. 3.
    Chen H, Dou Q, Yu L, Chen H, Jin Y, Yang X, Qin J, Heng P (2016) 3D deeply supervised network for automatic liver segmentation from CT volumes 3D deeply supervised network for automated segmentation of volumetric medical images. In: International conference on medical image computing and computer-assisted intervention. Springer, Berlin, pp 149–157Google Scholar
  4. 4.
    Lu F, Wu F, Hu P, Peng Z, Kong D (2017) Automatic 3D liver location and segmentation via convolutional neural network and graph cut. Int J Comput Assist Radiol Surg 12:171–182CrossRefGoogle Scholar
  5. 5.
    Hu P, Wu F, Peng J, Liang P, Kong D (2016) Automatic 3D liver segmentation based on deep learning and globally optimized surface evolution. Phys Med Biol 61:8676CrossRefGoogle Scholar
  6. 6.
    Taha AA, Hanbury A (2015) Metrics for evaluating 3D medical image segmentation: analysis, selection, and tool. BMC Med Imaging 15:29CrossRefGoogle Scholar
  7. 7.
    Masci J, Meier U, Ciresan D, Schmidhuber J (2011) Stacked convolutional AEs for hierarchical feature extraction. In: Icann. Springer, Berlin, pp 52–59Google Scholar
  8. 8.
    Kamnitsas K, Ledig C, Newcombe VFJ, Simpson JP, Kane AD, Menon DK, Rueckert D, Glocker B (2017) Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation. Med Image Anal 36:61–78CrossRefGoogle Scholar
  9. 9.
    Ravishankar H, Thiruvenkadam S, Venkataramani R (2017) Joint deep learning of foreground, background. In: International conference on information processing in medical imaging. Springer, Berlin, pp 622–632Google Scholar
  10. 10.
    Ronneberger O, Fischer P, Brox T (2015) U-Net: convolutional networks for biomedical image segmentation. In: International conference on medical image computing and computer-assisted intervention. Springer, Berlin, pp 234–241Google Scholar
  11. 11.
    Çiçek Ö, Abdulkadir A, Lienkamp SS, Brox T, Ronneberger O (2016) 3D U-Net: Learning dense volumetric segmentation from sparse annotation BT—medical image computing and computer-assisted intervention—MICCAI 2016. In: International conference on medical image computing and computer-assisted intervention. Springer, Berlin, pp 424–432Google Scholar
  12. 12.
    BenTaieb A, Hamarneh G (2016) Topology aware fully convolutional networks for histology gland segmentation. In: International conference on medical image computing and computer-assisted intervention. Springer, Berlin, pp 460–468Google Scholar
  13. 13.
    Chen H, Qi X, Yu L, Dou Q, Qin J, Heng P-A (2017) DCAN: deep contour-aware networks for object instance segmentation from histology images. Med Image Anal 36:135–146CrossRefGoogle Scholar
  14. 14.
    Ledig C, Theis L, Huszár F, Caballero J, Cunningham A, Acosta A, Aitken A, Tejani A, Totz J, Wang Z, Shi W (2017) Anatomically constrained neural networks (ACNN): application to cardiac image enhancement and segmentation-supplementary. In: Proceedings of the 30th IEEE conference on comput vis pattern recognition, CVPR 2017, pp 105–114.
  15. 15.
    Hinton G, Vinyals O, Dean J (2015) Distilling the knowledge in a neural network. arXiv Prepr arXiv:150302531
  16. 16.
    Wu Z, Song S, Khosla A, Yu F, Zhang L, Tang X, Xiao J (2015) 3d shapenets: a deep representation for volumetric shapes BT. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1912–1920Google Scholar
  17. 17.
    Sharma A, Grau O, Fritz M (2016) Vconv-dae: deep volumetric shape learning without object labels. In: European conference on computer vision. Springer, Berlin, pp 236–250Google Scholar
  18. 18.
    Xue Y, Xu T, Zhang H, Long LR, Huang X (2018) Segan: adversarial network with multi-scale l1 loss for medical image segmentation. Neuroinformatics 16:383–392CrossRefGoogle Scholar
  19. 19.
    Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3431–3440Google Scholar
  20. 20.
    Dalca AV, Guttag J, Sabuncu MR (2018) Anatomical priors in convolutional networks for unsupervised biomedical segmentation. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, pp 9290–9299Google Scholar
  21. 21.
    Ravishankar H, Venkataramani R, Thiruvenkadam S, Sudhakar P, Vaidya V (2017) Learning and incorporating shape models for semantic segmentation. In: Lecture notes in computer science (including subseries lecture notes in artificial intelligence and lecture notes in bioinformatics)Google Scholar
  22. 22.
    Clevert D-A, Unterthiner T, Hochreiter S (2015) Fast and accurate deep network learning by exponential linear units (ELUS). arXiv Prepr arXiv:151107289
  23. 23.
    Pedamonti D (2018) Comparison of non-linear activation functions for deep neural networks on MNIST classification task. arXiv Prepr arXiv:180402763
  24. 24.
    Van Ginneken B, Heimann T, Styner M (2007) MICCAI workshop on 3D segmentation in the clinic: a grand challenge. In: 3D segmentation in the Clinic: a grand challenge, pp 3–4Google Scholar
  25. 25.
    Soler L, Hostettler A, Agnus V, Charnoz A, Fasquel J, Moreau J, Osswald A, Bouhadjar M, Marescaux J (2010) 3D image reconstruction for comparison of algorithm database: a patient-specific anatomical and medical image database. IRCAD, Strasbourg, Fr. Tech. RepGoogle Scholar
  26. 26.
    Xu Y, Lin L, Hu H, Wang D, Zhu W, Wang J, Han XH, Chen YW (2018) Texture-specific bag of visual words model and spatial cone matching-based method for the retrieval of focal liver lesions using multiphase contrast-enhanced CT images. Int J Comput Assist Radiol Surg 13:151–164. CrossRefPubMedGoogle Scholar
  27. 27.
    Wang J, Li J, Han XH, Lin L, Hu H, Xu Y, Chen Q, Iwamoto Y, Chen YW (2019) Tensor-based sparse representations of multi-phase medical images for classification of focal liver lesions. Pattern Recognit Lett. CrossRefGoogle Scholar
  28. 28.
    Abadi M, Barham P, Chen J, Chen Z, Davis A, Dean J, Devin M, Ghemawat S, Irving G, Isard M, Kudlur M, Levenberg J, Monga R, Moore S, Murray DG, Steiner B, Tucker P, Vasudevan V, Warden P, Wicke M, Yu Y, Zheng X (2016) TensorFlow: a system for large-scale machine learning. In: OSDI, pp 265–283Google Scholar
  29. 29.
    Kingma D, Ba J (2014) Adam: a method for stochastic optimization. arXiv Prepr arXiv:14126980
  30. 30.
    Gunasundari S, Suganya Ananthi M (2012) Comparison and evaluation of methods for liver tumor classification from CT datasets. Int J Comput Appl 39:46–51. CrossRefGoogle Scholar
  31. 31.
    Al-Shaikhli SDS, Yang MY, Rosenhahn B (2015) Automatic 3D liver segmentation using sparse representation of global and local image information via level set formulation. arXiv Prepr arXiv:150801521
  32. 32.
    Dong C, Chen Y, Foruzan AH, Lin L, Han X, Tateyama T, Wu X, Xu G, Jiang H (2015) Segmentation of liver and spleen based on computational anatomy models. Comput Biol Med 67:146–160CrossRefGoogle Scholar
  33. 33.
    Zheng Y, Ai D, Mu J, Cong W, Wang X, Zhao H, Yang J (2017) Automatic liver segmentation based on appearance and context information. Biomed Eng Online. CrossRefPubMedPubMedCentralGoogle Scholar
  34. 34.
    Lu X, Xie Q, Zha Y, Wang D (2018) Fully automatic liver segmentation combining multi-dimensional graph cut with shape information in 3D CT images. Sci Rep. CrossRefPubMedPubMedCentralGoogle Scholar

Copyright information

© CARS 2019

Authors and Affiliations

  1. 1.Department of Biomedical Engineering, Engineering FacultyShahed UniversityTehranIran

Personalised recommendations