Neural Computing and Applications

, Volume 29, Issue 5, pp 1257–1265 | Cite as

Medical image semantic segmentation based on deep learning

  • Feng Jiang
  • Aleksei Grigorev
  • Seungmin Rho
  • Zhihong Tian
  • YunSheng Fu
  • Worku Jifara
  • Khan Adil
  • Shaohui Liu
Neural Computing in Next Generation Virtual Reality Technology


The image semantic segmentation has been extensively studying. The modern methods rely on the deep convolutional neural networks, which can be trained to address this problem. A few years ago networks require the huge dataset to be trained. However, the recent advances in deep learning allow training networks on the small datasets, which is a critical issue for medical images, since the hospitals and research organizations usually do not provide the huge amount of data. In this paper, we address medical image semantic segmentation problem by applying the modern CNN model. Moreover, the recent achievements in deep learning allow processing the whole image per time by applying concepts of the fully convolutional neural network. Our qualitative and quantitate experiment results demonstrated that modern CNN can successfully tackle the medical image semantic segmentation problem.


Medical image Semantic segmentation Neural network X-Ray 



This work is partially funded by the MOE—Microsoft Key Laboratory of Natural Language Processing and Speech, Harbin Institute of Technology, the Major State Basic Research Development Program of China (973 Program 2015CB351804), and the National Natural Science Foundation of China under Grant Nos. 61572155, 61672188, and 61572153. We would also like to acknowledge NVIDIA Corporation who kindly provided two sets of GPU.

Compliance with ethical standards

Conflict of interest

The authors declared that they have no conflicts of interest to this work. We declare that we do not have any commercial or associative interest that represents a conflict of interest in connection with the work submitted.


  1. 1.
    Shotton J, Winn J, Rother C Crimininsi A (2006) TextonBoost: joint appearance, shape and context modeling for multi-class object recognition and segmentation. In Proceedings of European conference on computer vision, vol 3951, Chapter 1, pp. 1–15Google Scholar
  2. 2.
    Jiang J, Trundle P, Ren J (2010) Medical image analysis with artificial neural networks. Comput Med Imaging Graph 34(8):617–631CrossRefGoogle Scholar
  3. 3.
    Zheng S, Jayasumana S, Romera-Paredes B, Vineet V, Su Z, Du D, Huang C, Torr P (2015) Conditional random fields as recurrent neural networks. In: Proceedings of the ICCV, pp 1529–1537Google Scholar
  4. 4.
    Long J, Shelhamer E, Darrell T (2015) [Slices] fully convolutional networks for semantic segmentation. In: Cvpr 2015Google Scholar
  5. 5.
    Krizhevsky A, Sutskever I, Hinton GE (2012) ImageNet classification with deep convolutional neural networks. In: Proceedings of the NIPS, pp 1–9Google Scholar
  6. 6.
    Girshick R, Donahue J, Darrell T (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. In: 2014 IEEE conference on computer vision pattern recognition, pp 580–587Google Scholar
  7. 7.
    Yan Z, Zhang H, Jia Y, Breuel T, Yu Y (2016) Combining the best of convolutional layers and recurrent layers: a hybrid network for semantic segmentation. arXiv:1603.04871
  8. 8.
    Visin F, Ciccone M, Romero A, Kastner K, Kyunghyun C, Bengio Y, Matteucci M, Courville A (2016) ReSeg: a recurrent neural network-based model for semantic segmentation. In IEEE conference on computer vision pattern recognition workshopsGoogle Scholar
  9. 9.
    Pinheiro PHO, Collobert R (2014) Recurrent convolutional neural networks for scene Labeling. In: Proceedings of the 31st international conference on Machine Learning, pp 82–90Google Scholar
  10. 10.
    Chen B-W, Wang J-C, Wang J-F (2009) A novel video summarization based on mining the story-structure and semantic relations among concept entities. IEEE Trans Multimedia 11(2):295–312CrossRefGoogle Scholar
  11. 11.
    Chen B-W, Chen C-Y, Wang J-F (2013) Smart homecare surveillance system: behavior identification based on state transition support vector machines and sound directivity pattern analysis. IEEE Trans Syst Man Cybern Syst 43(6):1279–1289CrossRefGoogle Scholar
  12. 12.
    Chen B-W, Tsai A-C, Wang J-F (2009) Structuralized context-aware content and scalable resolution support for wireless VoD services. IEEE Trans Consum Electron 55(2):713–720CrossRefGoogle Scholar
  13. 13.
    Chen L-C, Barron JT Papandreou G Murphy K Yuille AL (2015) Semantic image segmentation with task-specific edge detection using CNNs and a discriminatively trained domain transform. p 12Google Scholar
  14. 14.
    Gastal ESL, Oliveira MM (2011) Domain transform for edge-aware image and video processing. ACM Trans Graph 30(4):1CrossRefGoogle Scholar
  15. 15.
    Chen L-C, Papandreou G, Kokkinos I, Murphy K, Yuille AL (2014) Semantic image segmentation with deep convolutional nets and fully connected CRFs. In: Iclr, pp 1–14Google Scholar
  16. 16.
    Ngo TA, Carneiro G (2015) Lung segmentation in chest radiographs using distance regularized level set and deep-structured learning and inference. In: IEEE international conference on image processing (ICIP), pp 2140–2143Google Scholar
  17. 17.
    Wolf I, Böttger T, Grunewald K, Schöbinger M, Fink C, Risse F, Kauczor HU, Meinzer HP (2007) Implementation and evaluation of a new workflow for registration and segmentation of pulmonary MRI data for regional lung perfusion assessment. Phys Med Biol 52(5):1261–1275CrossRefGoogle Scholar
  18. 18.
    Candemir S, Jaeger S, Palaniappan K, Musco JP, Singh RK, Xue Z, Karargyris A, Antani S, Thoma G, McDonald CJ (2014) Lung segmentation in chest radiographs using anatomical atlases with nonrigid registration. IEEE Trans Med Imaging 33(2):577–590CrossRefGoogle Scholar
  19. 19.
    Chae S-H, Lee J, Won C, Pan SB (2014) Lung segmentation using prediction-based segmentation improvement for chest tomosynthesis. Int J Biosci Biotechnol 6(3):81–90Google Scholar
  20. 20.
    Li C, Xu C, Gui C, Fox MD (2010) Distance regularized level set evolution and its application to image segmentation. IEEE Trans Image Process 19(12):3243–3254MathSciNetCrossRefMATHGoogle Scholar
  21. 21.
    Zhang W, Zeng S, Wang D, Xue X (2015) Weakly supervised semantic segmentation for social images. In: Proceedings of the IEEE computer society conference on computer vision pattern recognition, vol 07, 12-June, pp. 2718–2726Google Scholar
  22. 22.
    Papandreou G, Chen L-C, Murphy KP, Yuille AL (2015) Weakly-and semi-supervised learning of a deep convolutional network for semantic image segmentation. In Proceedings of the ICCV, pp 1742–1750Google Scholar
  23. 23.
    Vezhnevets A, Buhmann JM (2010) Towards weakly supervised semantic segmentation by means of multiple instance and multitask learning. In: Proceedings of the IEEE computer society conference on computer vision pattern recognition, pp 3249–3256Google Scholar
  24. 24.
    Xu J, Schwing AG, Urtasun R (2014) Tell me what you see and i will show you where it is. In: 2014 IEEE conference on computer vision pattern recognition (CVPR), pp 3190–3197Google Scholar
  25. 25.
    Rajchl M, Lee MCH, Oktay O, Kamnitsas K, Passerat-palmbach J, Bai W, Kainz B, Rueckert D (2017) DeepCut: object segmentation from bounding box annotations using convolutional neural networks. IEEE Trans Med Imaging 36(2):674–683Google Scholar
  26. 26.
    Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. ImageNet Chall, pp 1–10. arXiv:1409.1556
  27. 27.
    Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. Proceedings of the IEEE computer society conference computer vision pattern recognition vol 07, 12-June, pp 1–9Google Scholar
  28. 28.
    Zaremba W, Sutskever I, Vinylas O (2014) Recurrent neural network regularization. Arxiv preprint arXiv:1409.2329
  29. 29.
    Jia Y, Shelhamer E, Dohanue J et al. (2014) Caffe: convolutional architecture fpr fast feature embedding. arXiv:1408.5093
  30. 30.
    Sutskever I, Martens J, Dahl GE (2013) On the importance of initialization and momentum in deep learning. In Jwml W&Cp, vol 28, issue 2010, pp 1139–1147Google Scholar
  31. 31.
    Bottou L (2012) Stochastic gradient descent tricks. In: Montavon G, Orr GB, Müller KR (eds) Neural networks: tricks of the trade. Lecture notes in computer science, vol 7700. Springer, Berlin, HeidelbergGoogle Scholar

Copyright information

© The Natural Computing Applications Forum 2017

Authors and Affiliations

  • Feng Jiang
    • 1
  • Aleksei Grigorev
    • 1
  • Seungmin Rho
    • 2
  • Zhihong Tian
    • 3
    • 1
  • YunSheng Fu
    • 3
  • Worku Jifara
    • 1
  • Khan Adil
    • 1
  • Shaohui Liu
    • 1
  1. 1.Department Computer Science and TechnologyHarbin Institute of TechnologyHarbinChina
  2. 2.Department of Media SoftwareSungkyul UniversityAnyangKorea
  3. 3.Institute of Computer ApplicationChinese Academy of Engineering PhysicsMianyangChina

Personalised recommendations