Segmenting Brain Tumors from MRI Using Cascaded Multi-modal U-Nets

  • Michal Marcinkiewicz
  • Jakub NalepaEmail author
  • Pablo Ribalta Lorenzo
  • Wojciech Dudzik
  • Grzegorz Mrukwa
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11384)


Gliomas are the most common primary brain tumors, and their accurate manual delineation is a time- consuming and very user-dependent process. Therefore, developing automated techniques for reproducible detection and segmentation of brain tumors from magnetic resonance imaging is a vital research topic. In this paper, we present a deep learning-powered approach for brain tumor segmentation which exploits multiple magnetic-resonance modalities and processes them in two cascaded stages. In both stages, we use multi-modal fully-convolutional neural nets inspired by U-Nets. The first stage detects regions of interests, whereas the second stage performs the multi-class classification. Our experimental study, performed over the newest release of the BraTS dataset (BraTS 2018) showed that our method delivers accurate brain-tumor delineation and offers very fast processing—the total time required to segment one study using our approach amounts to around 18 s.


Brain tumor Segmentation Deep learning CNN 



This research was supported by the National Centre for Research and Development under the Innomed Research and Development Grant No. POIR.01.02.00-00-0030/15.


  1. 1.
    Aljabar, P., Heckemann, R., Hammers, A., Hajnal, J., Rueckert, D.: Multi-atlas based segmentation of brain images: atlas selection and its effect on accuracy. NeuroImage 46(3), 726–738 (2009)CrossRefGoogle Scholar
  2. 2.
    Bakas, S., et al.: Advancing the cancer genome atlas glioma MRI collections with expert segmentation labels and radiomic features. Nat. Sci. Data 4, 1–13 (2017). Scholar
  3. 3.
    Bakas, S., et al.: Segmentation labels and radiomic features for the pre-operative scans of the TCGA-GBM collection, the Cancer Imaging Archive (2017).
  4. 4.
    Bakas, S., et al.: Segmentation labels and radiomic features for the pre-operative scans of the TCGA-LGG collection, the Cancer Imaging Archive (2017).
  5. 5.
    Bakas, S., Reyes, M., et al.: Identifying the best machine learning algorithms for brain tumor segmentation, progression assessment, and overall survival prediction in the BRATS challenge. CoRR abs/1811.02629 (2018).
  6. 6.
    Bauer, S., Seiler, C., Bardyn, T., Buechler, P., Reyes, M.: Atlas-based segmentation of brain tumor images using a markov random field-based tumor growth model and non-rigid registration. In: Proceedings of IEEE EMBC, pp. 4080–4083 (2010).
  7. 7.
    Chander, A., Chatterjee, A., Siarry, P.: A new social and momentum component adaptive PSO algorithm for image segmentation. Expert Syst. Appl. 38(5), 4998–5004 (2011)CrossRefGoogle Scholar
  8. 8.
    Dong, H., Yang, G., Liu, F., Mo, Y., Guo, Y.: Automatic brain tumor detection and segmentation using U-Net based fully convolutional networks. CoRR abs/1705.03820 (2017).
  9. 9.
    Fan, X., Yang, J., Zheng, Y., Cheng, L., Zhu, Y.: A novel unsupervised segmentation method for MR brain images based on fuzzy methods. In: Liu, Y., Jiang, T., Zhang, C. (eds.) CVBIA 2005. LNCS, vol. 3765, pp. 160–169. Springer, Heidelberg (2005). Scholar
  10. 10.
    Geremia, E., Clatz, O., Menze, B.H., Konukoglu, E., Criminisi, A., Ayache, N.: Spatial decision forests for MS lesion segmentation in multi-channel magnetic resonance images. NeuroImage 57(2), 378–390 (2011)CrossRefGoogle Scholar
  11. 11.
    Ghafoorian, M., et al.: Location sensitive deep convolutional neural networks for segmentation of white matter hyperintensities. CoRR abs/1610.04834 (2016).
  12. 12.
    Ghafoorian, M., et al.: Transfer learning for domain adaptation in MRI: application in brain lesion segmentation. In: Proceedings of MICCAI, pp. 516–524 (2017)Google Scholar
  13. 13.
    Havaei, M., Dutil, F., Pal, C., Larochelle, H., Jodoin, P.-M.: A convolutional neural network approach to brain tumor segmentation. In: Crimi, A., Menze, B., Maier, O., Reyes, M., Handels, H. (eds.) BrainLes 2015. LNCS, vol. 9556, pp. 195–208. Springer, Cham (2016). Scholar
  14. 14.
    Ji, S., Wei, B., Yu, Z., Yang, G., Yin, Y.: A new multistage medical segmentation method based on superpixel and fuzzy clustering. Comp. Math. Meth. Med. 2014, 747549:1–747549:13 (2014)MathSciNetCrossRefGoogle Scholar
  15. 15.
    Kamnitsas, K., et al.: Ensembles of multiple models and architectures for robust brain tumour segmentation. In: Crimi, A., Bakas, S., Kuijf, H., Menze, B., Reyes, M. (eds.) BrainLes 2017. LNCS, vol. 10670, pp. 450–462. Springer, Cham (2018). Scholar
  16. 16.
    Korfiatis, P., Kline, T.L., Erickson, B.J.: Automated segmentation of hyperintense regions in FLAIR MRI using deep learning. Tomogr. J. Imaging Res. 2(4), 334–340 (2016).
  17. 17.
    Ladgham, A., Torkhani, G., Sakly, A., Mtibaa, A.: Modified support vector machines for MR brain images recognition. In: Proceedings of CoDIT, pp. 032–035 (2013).
  18. 18.
    Lorenzo, P.R., Nalepa, J.: Memetic evolution of deep neural networks. In: Proceedings of the Genetic and Evolutionary Computation Conference GECCO 2018, pp. 505–512. ACM, New York (2018)Google Scholar
  19. 19.
    Mei, P.A., de Carvalho Carneiro, C., Fraser, S.J., Min, L.L., Reis, F.: Analysis of neoplastic lesions in magnetic resonance imaging using self-organizing maps. J. Neurol. Sci. 359(1–2), 78–83 (2015)CrossRefGoogle Scholar
  20. 20.
    Menze, B.H., et al.: The multimodal brain tumor image segmentation benchmark (BRATS). IEEE Trans. Med. Imaging 34(10), 1993–2024 (2015). Scholar
  21. 21.
    Moeskops, P., Viergever, M.A., Mendrik, A.M., de Vries, L.S., Benders, M.J.N.L., Isgum, I.: Automatic segmentation of MR brain images with a convolutional neural network. IEEE Trans. Med. Imaging 35(5), 1252–1261 (2016). Scholar
  22. 22.
    Park, M.T.M., et al.: Derivation of high-resolution MRI atlases of the human cerebellum at 3T and segmentation using multiple automatically generated templates. NeuroImage 95, 217–231 (2014)CrossRefGoogle Scholar
  23. 23.
    Pawełczyk, K., et al.: Towards detecting high-uptake lesions from lung CT scans using deep learning. In: Battiato, S., Gallo, G., Schettini, R., Stanco, F. (eds.) ICIAP 2017. LNCS, vol. 10485, pp. 310–320. Springer, Cham (2017). Scholar
  24. 24.
    Pinto, A., Pereira, S., Correia, H., Oliveira, J., Rasteiro, D.M.L.D., Silva, C.A.: Brain tumour segmentation based on extremely rand. forest with high-level features. In: Proceedings of IEEE EMBC, pp. 3037–3040 (2015).
  25. 25.
    Pipitone, J., et al.: Multi-atlas segmentation of the whole hippocampus and subfields using multiple automatically generated templates. NeuroImage 101, 494–512 (2014)CrossRefGoogle Scholar
  26. 26.
    Rajendran, A., Dhanasekaran, R.: Fuzzy clustering and deformable model for tumor segmentation on MRI brain image: a combined approach. Procedia Eng. 30, 327–333 (2012). Scholar
  27. 27.
    Rezaei, M., et al.: Conditional adversarial network for semantic segmentation of brain tumor. CoRR abs/1708.05227, pp. 1–10 (2017)Google Scholar
  28. 28.
    Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. CoRR abs/1505.04597 (2015)Google Scholar
  29. 29.
    Saha, S., Bandyopadhyay, S.: MRI brain image segmentation by fuzzy symmetry based genetic clustering technique. In: Proceedings of IEEE CEC, pp. 4417–4424 (2007)Google Scholar
  30. 30.
    Simi, V., Joseph, J.: Segmentation of glioblastoma multiforme from MR images - a comprehensive review. Egypt. J. Radiol. Nucl. Med. 46(4), 1105–1110 (2015)CrossRefGoogle Scholar
  31. 31.
    Soltaninejad, M., et al.: Automated brain tumour detection and segmentation using superpixel-based extremely randomized trees in FLAIR MRI. Int. J. of Comp. Assist. Radiol. Surg. 12(2), 183–203 (2017)CrossRefGoogle Scholar
  32. 32.
    Taha, A.A., Hanbury, A.: Metrics for evaluating 3D medical image segmentation: analysis, selection, and tool. BMC Med. Imaging 15(1), 29 (2015)CrossRefGoogle Scholar
  33. 33.
    Taherdangkoo, M., Bagheri, M.H., Yazdi, M., Andriole, K.P.: An effective method for segmentation of MR brain images using the ant colony optimization algorithm. J. Digit. Imaging 26(6), 1116–1123 (2013)CrossRefGoogle Scholar
  34. 34.
    Varghese, A., Mohammed, S., Sai, C., Ganapathy, K.: Generative adversarial networks for brain lesion detection. In: Proceedings of SPIE, vol. 10133, p. 10133 (2017)Google Scholar
  35. 35.
    Verma, N., Cowperthwaite, M.C., Markey, M.K.: Superpixels in brain MR image analysis. In: Proceedings of IEEE EMBC, pp. 1077–1080 (2013).
  36. 36.
    Wu, W., Chen, A.Y.C., Zhao, L., Corso, J.J.: Brain tumor detection and segmentation in a CRF (conditional random fields) framework with pixel-pairwise affinity and superpixel-level features. Int. J. of Comp. Assist. Radiol. Surg. 9(2), 241–253 (2014)CrossRefGoogle Scholar
  37. 37.
    Zhao, X., Wu, Y., Song, G., Li, Z., Zhang, Y., Fan, Y.: A deep learning model integrating FCNNs and CRFs for brain tumor segmentation. CoRR abs/1702.04528 (2017)Google Scholar
  38. 38.
    Zhuge, Y., et al.: Brain tumor segmentation using holistically nested neural networks in MRI images. Med. Phys. 44(10), 5234–5243 (2017). Scholar
  39. 39.
    Zikic, D., et al.: Decision forests for tissue-specific segmentation of high-grade gliomas in multi-channel MR. In: Ayache, N., Delingette, H., Golland, P., Mori, K. (eds.) MICCAI 2012. LNCS, vol. 7512, pp. 369–376. Springer, Heidelberg (2012). Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Michal Marcinkiewicz
    • 1
  • Jakub Nalepa
    • 1
    • 2
    Email author
  • Pablo Ribalta Lorenzo
    • 2
  • Wojciech Dudzik
    • 1
    • 2
  • Grzegorz Mrukwa
    • 1
    • 2
  1. 1.Future ProcessingGliwicePoland
  2. 2.Silesian University of TechnologyGliwicePoland

Personalised recommendations