Context Aware 3D CNNs for Brain Tumor Segmentation

  • Siddhartha ChandraEmail author
  • Maria Vakalopoulou
  • Lucas Fidon
  • Enzo Battistella
  • Théo Estienne
  • Roger Sun
  • Charlotte Robert
  • Eric Deutsch
  • Nikos Paragios
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11384)


In this work we propose a novel deep learning based pipeline for the task of brain tumor segmentation. Our pipeline consists of three primary components: (i) a preprocessing stage that exploits histogram standardization to mitigate inaccuracies in measured brain modalities, (ii) a first prediction stage that uses the V-Net deep learning architecture to output dense, per voxel class probabilities, and (iii) a prediction refinement stage that uses a Conditional Random Field (CRF) with a bilateral filtering objective for better context awareness. Additionally, we compare the V-Net architecture with a custom 3D Residual Network architecture, trained on a multi-view strategy, and our ablation experiments indicate that V-Net outperforms the 3D ResNet-18 with all bells and whistles, while fully connected CRFs as post processing, boost the performance of both networks. We report competitive results on the BraTS 2018 validation and test set.


Brain tumor segmentation 3-D fully convolutional CNNs Fully-connected CRFs 


  1. 1.
    Bakas, S., et al.: Advancing the cancer genome atlas glioma MRI collections with expert segmentation labels and radiomic features. Sci. Data 4, 9 (2017)CrossRefGoogle Scholar
  2. 2.
    Bakas, S., et al.: Segmentation labels and radiomic features for the pre-operative scans of the TCGA-GBM collection, July 2017Google Scholar
  3. 3.
    Bakas, S., et al.: Segmentation labels and radiomic features for the pre-operative scans of the TCGA-LGG collection, July 2017Google Scholar
  4. 4.
    Bakas, S., Reyes, M., et al.: Identifying the best machine learning algorithms for brain tumor segmentation, progression assessment, and overall survival prediction in the brats challenge. CoRR, abs/1811.02629 (2018)Google Scholar
  5. 5.
    Holland, E.C.: Progenitor cells and GLIOMA formation. Curr. Opin. Neurol. 14, 683–688 (2002)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Chandra, S., Kokkinos, I.: Fast, exact and multi-scale inference for semantic image segmentation with deep gaussian CRFs. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9911, pp. 402–418. Springer, Cham (2016). Scholar
  7. 7.
    Chen, L., Papandreou, G., Schroff, F., Adam, H.: Rethinking atrous convolution for semantic image segmentation. CoRR, abs/1706.05587 (2017)Google Scholar
  8. 8.
    Chen, L., Yang, Y., Wang, J., Xu, W., Yuille, A.L.: Attention to scale: Scale-aware semantic image segmentation. In: CVPR (2016)Google Scholar
  9. 9.
    Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A.L.: Semantic image segmentation with deep convolutional nets and fully connected CRFs. arXiv preprint arXiv:1412.7062 (2014)
  10. 10.
    Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A.L.: Deeplab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. arXiv:1606.00915 (2016)
  11. 11.
    Chen, L.-C., Papandreou, G., Murphy, K., Yuille, A.L.: Weakly and semi-supervised learning of a deep convolutional network for semantic image segmentation. In: ICCV (2015)Google Scholar
  12. 12.
    Ferlay, J., et al.: Cancer incidence and mortality worldwide (2013)Google Scholar
  13. 13.
    Hadziahmetovic, M., Shirai, K., Chakravarti, A.: Recent advancements in multimodality treatment of gliomas. Future Oncol. 7(10), 1169–1183 (2011)CrossRefGoogle Scholar
  14. 14.
    Hara, K., Kataoka, H., Satoh, Y.: Can spatiotemporal 3D CNNs retrace the history of 2D CNNs and imagenet? In: CVPR (2018)Google Scholar
  15. 15.
    He, K., Gkioxari, G., Dollár, P., Girshick, R.: Mask R-CNN. In: ICCV (2017)Google Scholar
  16. 16.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR (2016)Google Scholar
  17. 17.
    Krähenbühl, P., Koltun, V.: Efficient inference in fully connected CRFs with gaussian edge potentials. In: NIPS (2011)Google Scholar
  18. 18.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: NIPS (2012)Google Scholar
  19. 19.
    Lecun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. In: Proceedings of the IEEE (1998)Google Scholar
  20. 20.
    Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: CVPR, pp. 3431–3440 (2015)Google Scholar
  21. 21.
    Louis, D.N., et al.: The 2016 world health organization classification of tumors of the central nervous system: a summary. Acta Neuropathol. 131(6), 803–820 (2016)CrossRefGoogle Scholar
  22. 22.
    Menze, B.H., et al.: The multimodal brain tumor image segmentation benchmark (brats). IEEE Trans. Med. Imaging 34(10), 1993 (2015)CrossRefGoogle Scholar
  23. 23.
    Milletari, F., Navab, N., Ahmadi, S.-A.: V-net: fully convolutional neural networks for volumetric medical image segmentation. In: 2016 Fourth International Conference on 3D Vision (3DV), pp. 565–571. IEEE (2016)Google Scholar
  24. 24.
    Nyúl, L.G., Udupa, J.K., Zhang, X.: New variants of a method of MRI scale standardization. IEEE Trans. Med. Imaging 19(2), 143–150 (2000)CrossRefGoogle Scholar
  25. 25.
    Paszke, A., et al.: Automatic differentiation in pytorch. In: NIPS-W (2017)Google Scholar
  26. 26.
    Roth, H.R., et al.: A new 2.5D representation for lymph node detection using random sets of deep convolutional neural network observations. In: Golland, P., Hata, N., Barillot, C., Hornegger, J., Howe, R. (eds.) MICCAI 2014. LNCS, vol. 8673, pp. 520–527. Springer, Cham (2014). Scholar
  27. 27.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: ICLR (2015)Google Scholar
  28. 28.
    Sudre, C.H., Li, W., Vercauteren, T., Ourselin, S., Jorge Cardoso, M.: Generalised dice overlap as a deep learning loss function for highly unbalanced segmentations. In: Cardoso, M.J., et al. (eds.) DLMIA/ML-CDS -2017. LNCS, vol. 10553, pp. 240–248. Springer, Cham (2017). Scholar
  29. 29.
    Vakalopoulou, M., et al.: AtlasNet: multi-atlas non-linear deep networks for medical image segmentation. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11073, pp. 658–666. Springer, Cham (2018). Scholar
  30. 30.
    Yu, F., Koltun, V.: Multi-scale context aggregation by dilated convolutions. In: ICLR (2016)Google Scholar
  31. 31.
    Zagoruyko, S., Komodakis, N.: Wide residual networks. In: BMVC (2016)Google Scholar
  32. 32.
    Zhao, H., Shi, J., Qi, X., Wang, X., Jia, J.: Pyramid scene parsing network. CoRR, abs/1612.01105 (2016)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Siddhartha Chandra
    • 1
    Email author
  • Maria Vakalopoulou
    • 1
    • 2
  • Lucas Fidon
    • 1
    • 3
  • Enzo Battistella
    • 1
    • 2
  • Théo Estienne
    • 1
    • 2
  • Roger Sun
    • 1
    • 2
  • Charlotte Robert
    • 2
  • Eric Deutsch
    • 2
  • Nikos Paragios
    • 1
    • 3
  1. 1.CVN, CentraleSupélec, Université Paris-SaclayParisFrance
  2. 2.Gustave Roussy InstituteParisFrance
  3. 3.TheraPanaceaParisFrance

Personalised recommendations