Multimodal Brain Tumor Segmentation Using Encoder-Decoder with Hierarchical Separable Convolution

  • Zhongdao Jia
  • Zhimin Yuan
  • Jialin PengEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11846)


To address automatic segmentation of brain tumor from multi-modal MRI volumes, a light-weight encoder-decoder network is presented. Exploring effective way to trade off the range of spatial contexts and computational efficiency is crucial to address challenges of 3D segmentation. To this end, we introduce hierarchical separable convolution (HSC), an integration of view- and group-wise separable convolution, which can simultaneously encode multi-scale context in 3D and reduce memory overhead without sacrificing accuracy. Specifically, typical 3D convolution is replaced with complementary 2D convolutions at multiple scales and thus multiple fields-of-view, which results in a light-weight but stronger model. Moreover, thanks to the decomposed convolutions, we ensemble 3D segmentations with different focal views to further improve segmentation accuracy. Experiments on the BRATS 2017 benchmark showed that our method achieved state-of-the-art performance in Dice, i.e., 0.901, 0.809 and 0.762 for the whole tumor, tumor core and enhancing tumor core, respectively.


Brain tumor segmentation Hierarchical separable convolution Contextual information 


  1. 1.
    Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). Scholar
  2. 2.
    Gao, S., Cheng, M., Zhao, K., et al.: Res2Net: a new multi-scale backbone architecture. In: 32th CVPR, Long Beach, CA (2019)Google Scholar
  3. 3.
    Wang, G., Li, W., Ourselin, S., Vercauteren, T.: Automatic brain tumor segmentation using cascaded anisotropic convolutional neural networks. In: Crimi, A., Bakas, S., Kuijf, H., Menze, B., Reyes, M. (eds.) BrainLes 2017. LNCS, vol. 10670, pp. 178–190. Springer, Cham (2018). Scholar
  4. 4.
    Li, Y., Shen, L.: Deep learning based multimodal brain tumor diagnosis. In: Crimi, A., Bakas, S., Kuijf, H., Menze, B., Reyes, M. (eds.) BrainLes 2017. LNCS, vol. 10670, pp. 149–158. Springer, Cham (2018). Scholar
  5. 5.
    Havaei, M., Davy, A., Warde-Farley, D., et al.: Brain tumor segmentation with deep neural networks. Med. Image Anal. 35, 18–31 (2017)CrossRefGoogle Scholar
  6. 6.
    Kamnitsas, K., Ledig, C., Newcombe, V.F., et al.: Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation. Med. Image Anal. 36, 61–78 (2017)CrossRefGoogle Scholar
  7. 7.
    Isensee, F., Kickingereder, P., Wick, W., Bendszus, M., Maier-Hein, K.H.: Brain tumor segmentation and radiomics survival prediction: contribution to the BRATS 2017 challenge. In: Crimi, A., Bakas, S., Kuijf, H., Menze, B., Reyes, M. (eds.) BrainLes 2017. LNCS, vol. 10670, pp. 287–297. Springer, Cham (2018). Scholar
  8. 8.
    Zhao, X., Wu, Y., Song, G., et al.: A deep learning model integrating FCNNs and CRFs for brain tumor segmentation. Med. Image Anal. 43, 98–111 (2018)CrossRefGoogle Scholar
  9. 9.
    Milletari, F., Navab, N., Ahmadi, S.A.: V-Net: fully convolutional neural networks for volumetric medical image segmentation. In: 4th International Conference on 3D Vision, Stanford, CA, pp. 565–571. IEEE (2016)Google Scholar
  10. 10.
    Casamitjana, A., Catà, M., Sánchez, I., Combalia, M., Vilaplana, V.: Cascaded V-Net using ROI masks for brain tumor segmentation. In: Crimi, A., Bakas, S., Kuijf, H., Menze, B., Reyes, M. (eds.) BrainLes 2017. LNCS, vol. 10670, pp. 381–391. Springer, Cham (2018). Scholar
  11. 11.
    Marcinkiewicz, M., Nalepa, J., Lorenzo, P.R., Dudzik, W., Mrukwa, G.: Segmenting brain tumors from MRI using cascaded multi-modal U-Nets. In: Crimi, A., Bakas, S., Kuijf, H., Keyvan, F., Reyes, M., van Walsum, T. (eds.) BrainLes 2018. LNCS, vol. 11384, pp. 13–24. Springer, Cham (2019). Scholar
  12. 12.
    Pereira, S., Alves, V., Silva, C.A.: Adaptive feature recombination and recalibration for semantic segmentation: application to brain tumor segmentation in MRI. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11072, pp. 706–714. Springer, Cham (2018). Scholar
  13. 13.
    Kamnitsas, K., et al.: Ensembles of multiple models and architectures for robust brain tumour segmentation. In: Crimi, A., Bakas, S., Kuijf, H., Menze, B., Reyes, M. (eds.) BrainLes 2017. LNCS, vol. 10670, pp. 450–462. Springer, Cham (2018). Scholar
  14. 14.
    Islam, M., Ren, H.: Class balanced PixelNet for neurological image segmentation. In: 6th ICBCB, Chengdu, China, pp. 83–87. ACM (2018)Google Scholar
  15. 15.
    Qiu, Z., Yao, T., Mei, T.: Learning spatio-temporal representation with pseudo-3D residual networks. In: 16th ICCV, Venice, Italy, pp. 5533–5541. IEEE (2017)Google Scholar
  16. 16.
    He, K., Zhang, X., Ren, S., et al.: Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Trans. Pattern Anal. Mach. Intell. 37(9), 1904–1916 (2015)CrossRefGoogle Scholar
  17. 17.
    Kingma, D. P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
  18. 18.
    Hu, P., Wu, F., Peng, J., et al.: Automatic 3D liver segmentation based on deep learning and globally optimized surface evolution. Phys. Med. Biol. 61(24), 8676 (2016)CrossRefGoogle Scholar
  19. 19.
    Zhou, H. Y., Gao, B. B., Wu, J.: Adaptive feeding: Achieving fast and accurate detections by adaptively combining object detectors. In: 16th ICCV, Venice, Italy, pp. 3505–3513. IEEE (2017)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.College of Computer Science and TechnologyHuaqiao UniversityXiamenChina

Personalised recommendations