Fully convolutional networks (FCNs)-based segmentation method for colorectal tumors on T2-weighted magnetic resonance images

  • Junming Jian
  • Fei Xiong
  • Wei Xia
  • Rui Zhang
  • Jinhui Gu
  • Xiaodong Wu
  • Xiaochun MengEmail author
  • Xin GaoEmail author
Scientific Paper


Segmentation of colorectal tumors is the basis of preoperative prediction, staging, and therapeutic response evaluation. Due to the blurred boundary between lesions and normal colorectal tissue, it is hard to realize accurate segmentation. Routinely manual or semi-manual segmentation methods are extremely tedious, time-consuming, and highly operator-dependent. In the framework of FCNs, a segmentation method for colorectal tumor was presented. Normalization was applied to reduce the differences among images. Borrowing from transfer learning, VGG-16 was employed to extract features from normalized images. We conducted five side-output blocks from the last convolutional layer of each block of VGG-16 along the network, these side-output blocks can deep dive multiscale features, and produced corresponding predictions. Finally, all of the predictions from side-output blocks were fused to determine the final boundaries of the tumors. A quantitative comparison of 2772 colorectal tumor manual segmentation results from T2-weighted magnetic resonance images shows that the average Dice similarity coefficient, positive predictive value, specificity, sensitivity, Hammoude distance, and Hausdorff distance were 83.56, 82.67, 96.75, 87.85%, 0.2694, and 8.20, respectively. The proposed method is superior to U-net in colorectal tumor segmentation (P < 0.05). There is no difference between cross-entropy loss and Dice-based loss in colorectal tumor segmentation (P > 0.05). The results indicate that the introduction of FCNs contributed to accurate segmentation of colorectal tumors. This method has the potential to replace the present time-consuming and nonreproducible manual segmentation method.


Colorectal tumor Fully convolutional network Segmentation MRI 



This work was supported by the National Natural Science Foundation of China (81571772).

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed consent

Informed consent was obtained from all individual participants included in the study.


  1. 1.
    Stewart B, Wild CP (2017) World cancer report 2014. World Health Organisation, GenevaGoogle Scholar
  2. 2.
    Siegel RL, Miller KD, Fedewa SA, Ahnen DJ, Meester RG, Barzi A, Jemal A (2017) Colorectal cancer statistics, 2017. CA: Cancer J Clin 67(3):177–193Google Scholar
  3. 3.
    Kekelidze M, D’Errico L, Pansini M, Tyndall A, Hohmann J (2013) Colorectal cancer: current imaging methods and future perspectives for the diagnosis, staging and therapeutic response evaluation. World J Gastroenterol 19(46):8502CrossRefPubMedPubMedCentralGoogle Scholar
  4. 4.
    Huang Y-q, Liang C-h, He L, Tian J, Liang C-s, Chen X, Ma Z-l, Liu Z-y (2016) Development and validation of a radiomics nomogram for preoperative prediction of lymph node metastasis in colorectal cancer. J Clin Oncol 34(18):2157–2164CrossRefPubMedGoogle Scholar
  5. 5.
    Christ PF, Elshaer MEA, Ettlinger F, Tatavarty S, Bickel M, Bilic P, Rempfler M, Armbruster M, Hofmann F, D’Anastasi M (2016) Automatic liver and lesion segmentation in CT using cascaded fully convolutional neural networks and 3D conditional random fields. In: International conference on medical image computing and computer-assisted intervention. Springer, New York, pp 415–423Google Scholar
  6. 6.
    Nie K, Shi L, Chen Q, Hu X, Jabbour SK, Yue N, Niu T, Sun X (2016) Rectal cancer: assessment of neoadjuvant chemoradiation outcome based on radiomics of multiparametric MRI. Clin Cancer Res 22(21):5256–5264CrossRefPubMedGoogle Scholar
  7. 7.
    Zhao X, Wu Y, Song G, Li Z, Fan Y, Zhang Y (2016) Brain tumor segmentation using a fully convolutional neural network with conditional random fields. In: Crimi A, Menze B, Maier O, Reyes M, Winzeck S, Handels H (eds) Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries: Second International Workshop, BrainLes 2016, with the Challenges on BRATS, ISLES and mTOP 2016, Held in Conjunction with MICCAI 2016, Athens, Greece, October 17, 2016, Revised Selected Papers. Springer, Cham, pp 75–87.
  8. 8.
    Pereira S, Pinto A, Alves V, Silva CA (2016) Brain tumor segmentation using convolutional neural networks in MRI images. IEEE Trans Med Imag 35(5):1240–1251CrossRefGoogle Scholar
  9. 9.
    Day E, Betler J, Parda D, Reitz B, Kirichenko A, Mohammadi S, Miften M (2009) A region growing method for tumor volume segmentation on PET images for rectal and anal cancer patients. Med Phys. PubMedCrossRefGoogle Scholar
  10. 10.
    Irving B, Cifor A, Papież BW, Franklin J, Anderson EM, Brady SM, Schnabel JA (2014) Automated colorectal tumour segmentation in DCE-MRI using supervoxel neighbourhood contrast characteristics. In: Medical image computing and computer-assisted intervention—MICCAI 2014. Springer, Cham, pp 609–616Google Scholar
  11. 11.
    Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp 1097–1105Google Scholar
  12. 12.
    Zeiler MD, Fergus R (2014) Visualizing and understanding convolutional networks. In: European conference on computer vision. Springer, New York, pp 818–833Google Scholar
  13. 13.
    Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. arXiv preprint arXiv:1409.4842, pp 1–9Google Scholar
  14. 14.
    Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z (2016) Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2818–2826Google Scholar
  15. 15.
    He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. arXiv preprint arXiv:1512.03385, pp 770–778Google Scholar
  16. 16.
    Ciresan D, Giusti A, Gambardella LM, Schmidhuber J (2012) Deep neural networks segment neuronal membranes in electron microscopy images. In: Advances in neural information processing systems. Morgan Kaufmann, San Mateo, pp 2843–2851Google Scholar
  17. 17.
    Trebeschi S, van Griethuysen JJM, Lambregts DMJ, Lahaye MJ, Parmer C, Bakers FCH, Peters NHGM., Beets-Tan RGH, Aerts HJWL. (2017) Deep learning for fully-automated localization and segmentation of rectal cancer on multiparametric MR. Sci Rep 7:5301. CrossRefPubMedPubMedCentralGoogle Scholar
  18. 18.
    Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. arXiv preprint arXiv:1605.06211, pp 3431–3440Google Scholar
  19. 19.
    Chang PDF (2016) Convolutional deep residual neural networks for brain tumor segmentation. In: International Workshop on Brainlesion: glioma, multiple sclerosis, stroke and traumatic brain injuries. Springer, New York, pp 108–118CrossRefGoogle Scholar
  20. 20.
    Luo Y, Cheng H, Yang L (2016) Size-invariant fully convolutional neural network for vessel segmentation of digital retinal images. In: Signal and information processing association annual summit and conference (APSIPA), 2016 Asia-Pacific. IEEE, pp 1–7Google Scholar
  21. 21.
    Fu H, Xu Y, Wong DWK, Liu J (2016) Retinal vessel segmentation via deep learning network and fully-connected conditional random fields. In: 2016 IEEE 13th international symposium on biomedical imaging (ISBI). IEEE, pp 698–701Google Scholar
  22. 22.
    Huang L, Xia W, Zhang B, Qiu B, Gao X (2017) MSFCN-multiple supervised fully convolutional networks for the osteosarcoma segmentation of CT images. Comput Methods Programs Biomed 143:67–74CrossRefPubMedGoogle Scholar
  23. 23.
    Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. In: International conference on medical image computing and computer-assisted intervention. Springer, New York, pp 234–241Google Scholar
  24. 24.
    Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:14091556Google Scholar
  25. 25.
    Nyúl LG, Udupa JK, Zhang X (2000) New variants of a method of MRI scale standardization. IEEE Trans Med Imag 19(2):143–150CrossRefGoogle Scholar
  26. 26.
    Nyul LG, Udupa JK (1999) On standardizing the MR image intensity scale. Magnet Reson Med 42(6):1072–1081CrossRefGoogle Scholar
  27. 27.
    Milletari F, Navab N, Ahmadi S-A (2016) V-net: fully convolutional neural networks for volumetric medical image segmentation. In: 2016 Fourth International Conference on 3D vision (3DV). IEEE, pp 565–571Google Scholar
  28. 28.
    He K, Zhang X, Ren S, Sun J (2015) Delving deep into rectifiers: surpassing human-level performance on imagenet classification. In: Proceedings of the IEEE international conference on computer vision, pp 1026–1034Google Scholar
  29. 29.
    Zeiler MD, Krishnan D, Taylor GW, Fergus R (2010) Deconvolutional networks. In: 2010 IEEE Computer Society Conference on computer vision and pattern recognition, 13–18 June 2010. pp 2528–2535.
  30. 30.
    Kingma D, Ba J (2014) Adam: A method for stochastic optimization. arXiv preprint arXiv:14126980Google Scholar
  31. 31.
    Crum WR, Camara O, Hill DLG (2006) Generalized overlap measures for evaluation and validation in medical image analysis. IEEE Trans Med Imag 25(11):1451–1461CrossRefGoogle Scholar
  32. 32.
    Silveira M, Marques JS (2008) Level set segmentation of dermoscopy images. In: 2008 5th IEEE International Symposium on biomedical imaging: from nano to macro. IEEE, pp 173–176Google Scholar
  33. 33.
    Taha AA, Hanbury A (2015) An efficient algorithm for calculating the exact Hausdorff distance. IEEE Trans Pattern Anal Mach Intell 37(11):2153–2163CrossRefPubMedGoogle Scholar
  34. 34.
    Chollet F (2017) Keras (2015).
  35. 35.
    Tajbakhsh N, Shin JY, Gurudu SR, Hurst RT, Kendall CB, Gotway MB, Liang J (2016) Convolutional neural networks for medical image analysis: full training or fine tuning? IEEE Trans Med Imag 35(5):1299–1312CrossRefGoogle Scholar

Copyright information

© Australasian College of Physical Scientists and Engineers in Medicine 2018

Authors and Affiliations

  1. 1.University of Science and Technology of ChinaHefeiChina
  2. 2.Suzhou Institute of Biomedical Engineering and TechnologyChinese Academy of SciencesSuzhouChina
  3. 3.Department of RadiologyThe Sixth Affiliated Hospital of Sun Yat-sen UniversityGuangzhouChina
  4. 4.Department of AnorectalThe people’s hospital of Suzhou New DistrictSuzhouChina

Personalised recommendations