Skip to main content

Brain Tumour Segmentation with a Muti-Pathway ResNet Based UNet

Abstract

Automatic segmentation of brain tumour regions is essential in today’s scenario for proper diagnosis and treatment of the disease. Gliomas can appear in any region and can be of any shape and size, which makes automatic detection challenging. However, now, with the availability of high-quality MRI scans, various strides have been made in this field. In this paper, we propose a novel multi-pathway UNet incorporated with residual networks and skip connections to segment multimodal Magnetic Resonance images into three hierarchical glioma sub-regions. The multi-pathway serves as a medium to decompose the multiclass segmentation problem into subsequent binary segmentation tasks, where each pathway is responsible for segmenting one class from the background. Instead of a cascaded architecture for the hierarchical regions, we propose a shared encoder, followed by separate decoders for each category. Residual connections employed in the model facilitate increasing the performance. Experiments have been carried out on BraTS 2020 dataset and have achieved promising results.

This is a preview of subscription content, access via your institution.

References

  1. 1.

    Ohgaki, H., Kleihues, P.: Population-based studies on incidence, survival rates, and genetic alterations in astrocytic and oligodendroglial gliomas. J. Neuropathol. Exp. Neurol. 64(6), 479–89 (2005). https://doi.org/10.1093/jnen/64.6.479. PMID 15977639

    Article  Google Scholar 

  2. 2.

    Olson, J.D., Riedel, E., DeAngelis, L.M.: Long-term outcome of low-grade oligodendroglioma and mixed glioma. Neurology 54(7), 1442–8 (2000). https://doi.org/10.1212/WNL.54.7.1442 PMID 10751254

    Article  Google Scholar 

  3. 3.

    Bleeker, F.E., Molenaar, R.J., Leenstra, S.: Recent advances in the molecular understanding of glioblastoma. J. Neurooncol. 108(1), 11–27 (2012). https://doi.org/10.1007/s11060-011-0793-0. PMC 3337398. PMID 22270850

    Article  Google Scholar 

  4. 4.

    Hu, Y.: Design and implementation of abnormal behavior detection based on deep intelligent analysis algorithms in massive video surveillance. J. Grid Comput. 18(2), 227–237 (2020)

    Article  Google Scholar 

  5. 5.

    Pengjiang, Q., Xu, K., Wang, T., Qiankun, Z., Yang, H., Atallah, B., Junqing, Z., Bryan, T., F Jr, M.R.: Estimating CT from MR abdominal images using novel generative adversarial networks. J. Grid Comput. 18(2), 211–226 (2020)

    Article  Google Scholar 

  6. 6.

    Havaei, M., Davy, A., Warde-Farley, D., Biard, A., Courville, A., Bengio, Y., Pal, C., Jodoin, P.-M., Larochelle, H.: Brain tumor segmentation with deep neural networks. Med. Image Anal. 35, 18–31 (2017)

    Article  Google Scholar 

  7. 7.

    Pereira, S., Pinto, A., Alves, V., Silva, CA.: Brain tumor segmentation using convolutional neural networks in MRI images. IEEE Trans. Med. Imaging 5, 1240–1251 (2016)

    Article  Google Scholar 

  8. 8.

    Menze, B.H., Jakab, A., Bauer, S., Kalpathy-Cramer, J., Farahani, K., Kirby, J., et al.: The multimodal brain tumor image segmentation benchmark (BRATS). IEEE Trans. Med. Imaging 34(10), 1993–2024 (2015). https://doi.org/10.1109/TMI.2014.2377694

    Article  Google Scholar 

  9. 9.

    Bakas, S., Akbari, H., Sotiras, A., Bilello, M., Rozycki, M., Kirby, J.S., et al.: Advancing the cancer genome atlas glioma MRI collections with expert segmentation labels and radiomic features. Nat. Sci. Data 4, 170117 (2017). https://doi.org/10.1038/sdata.2017.117

    Article  Google Scholar 

  10. 10.

    Bakas, S., Reyes, M., Jakab, A., Bauer, S., Rempfler, M., Crimi, A., et al.: Identifying the best machine learning algorithms for brain tumor segmentation, progression assessment, and overall survival prediction in the brats challenge. arXiv:1811.02629 (2018)

  11. 11.

    Bakas, S., Akbari, H., Sotiras, A., Bilello, M., Rozycki, M., Kirby, J., et al.: Segmentation labels and radiomic features for the pre-operative scans of the TCGA-GBM collection, The Cancer Imaging Archive. https://doi.org/10.7937/K9/TCIA.2017.KLXWJJ1Q (2017)

  12. 12.

    Bakas, S., Akbari, H., Sotiras, A., Bilello, M., Rozycki, M., Kirby, J., et al.: Segmentation labels and radiomic features for the pre-operative scans of the TCGA-LGG collection. The Cancer Imaging Archive. https://doi.org/10.7937/K9/TCIA.2017.GJQ7R0EF (2017)

  13. 13.

    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). https://doi.org/10.1109/CVPR.2016.90, pp 770–778, Las Vegas (2016)

  14. 14.

    Zia, T., Razzaq, S.: Residual recurrent highway networks for learning deep sequence prediction models. J. Grid Comput. 18(1), 169–176 (2020)

    Article  Google Scholar 

  15. 15.

    Springenberg, J.T., Dosovitskiy, A., Brox, T., Riedmiller, M.: Striving for simplicity: The all convolutional net. arXiv:1412.6806 (2014)

  16. 16.

    Milletari, F., Navab, N., Ahmadi, S.-A.: V-net: Fully convolutional neural networks for volumetric medical image segmentation. In: 2016 Fourth International Conference on 3D Vision (3DV). IEEE (2016)

  17. 17.

    Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015, pp 234–241. Springer (2015)

  18. 18.

    Angelini, E., Clatz, O., Konukoglu, E., Capelle, L., Duffau, H.: Glioma dynamics and computational models: A review of segmentation, registration, and in silico growth algorithms and their clinical applications 3 (2007)

  19. 19.

    Bauer, S., Wiest, R., Nolte, L., Reyes, M.: A survey of mri-based medical image analysis for brain tumor studies. Phys. Med. Biol. 58, 97–129 (2013)

    Article  Google Scholar 

  20. 20.

    Kaus, M.R., Warfield, S.K., Nabavi, A., Black, P.M., Jolesz, F.A., Kikinis, R.: Automated segmentation of MR images of brain tumors. Radiology 218(2), 586–591 (2001)

    Article  Google Scholar 

  21. 21.

    Menze, B.H., Van Leemput, K., Lashkari, D., Weber, M.A., Ayache, N., Golland, P.: A generative model for brain tumor segmentation in multi-modal images. In: MICCAI, pp 151–159 (2010)

  22. 22.

    Prastawa, M., Bullit, E., Ho, S., Gerig, G.: A brain tumor segmentation framework based on outlier detection. Med. Image Anayl. 8, 275–283 (2004)

    Article  Google Scholar 

  23. 23.

    Kwon, D., Akbari, H., Da, X., Gaonkar, B., Davatzikos, C.: Multimodal brain tumor image segmentation using glistr. In: Proc of BRATS Challenge - MICCAI (2014)

  24. 24.

    Parisot, S., Duffau, H., Chemouny, S., Paragios, N.: Joint tumor segmentation and dense deformable registration of brain mr images. In: MICCAI, pp 651–658 (2012)

  25. 25.

    Lee, C.H., Schmidt, M., Murtha, A.: Segmenting brain tumors with conditional random fields and support vector machines. In: CVBIA, pp 469–478 (2005)

  26. 26.

    Zikic, D., Glocker, B., Konukoglu, E., Criminisi, A., Demiralp, C., Shotton, J., Thomas, O.M., Das, T., Jena, R., Price, S.J.: Decision forests for tissue-specific segmentation of high-grade gliomas in multi-channel MR. In: MICCAI, pp 369–376 (2012)

  27. 27.

    Gotz, M., Weber, C., Blocher, J., Stieltjes, B., Meinzer, H.P., Maier-Hein, K.: Extremely randomized trees based brain tumor segmentation. In: Proc of BRATS Challenge - MICCAI (2014)

  28. 28.

    Kleesiek, J., Biller, A., Urban, G., Kothe, U., Bendszus, M., Hamprecht, F.A.: Ilastik for multi-modal brain tumor segmentation. In: Proc of BRATS-MICCAI (2014)

  29. 29.

    Menze, B., Reyes, M., Leemput, K.V.: The multimodal brain tumor image segmentation benchmark (brats). IEEE Trans. Med. Imaging. (accepted) (2014)

  30. 30.

    Kamnitsas, K., Ledig, C., Newcombe, V.F.J., Simpson, J.P., Kane, A.D., Menon, D.K., Rueckert, D., Glocker, B.: Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation. Med. Image Anal. 36, 61–78 (2017)

    Article  Google Scholar 

  31. 31.

    Cicek, O., Abdulkadir, A., Lienkamp, S.S., Brox, T., Ronneberger, O.: 3D U-Net: learning dense volumetric segmentation from sparse annotation (2016)

  32. 32.

    Kamnitsas, K., Bai, W., McDonagh, E.F., Sinclair, S., Pawlowski, M., Rajchl, N., Lee, M., Kainz, M., Rueckert, B., Glocker, D.B.: Ensembles of multiple models and architectures for robust brain tumour segmentation. In: International Conf. on Medical Image Computing and Computer Assisted Intervention. Multimodal Brain Tumor Segmentation Challenge (MICCAI). LNCS (2017)

  33. 33.

    Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 3431–3440 (2015)

  34. 34.

    Wang, G., Li, W., Ourselin, S., Vercauteren, T.: Automatic brain tumor segmentation using cascaded anisotropic convolutional neural networks. In: International Conf. on Medical Image Computing and Computer Assisted Intervention. Multimodal Brain Tumor Segmentation Challenge (MICCAI). LNCS (2017)

  35. 35.

    Isensee, F., Kickingereder, P., Wick, W., Bendszus, M., Maier-Hein, K.H.: No newnet. In: International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI 2018). Multimodal Brain Tumor Segmentation Challenge (BraTS 2018). BrainLes 2018 workshop. LNCS. Springer (2018)

  36. 36.

    McKinley, R., Meier, R., Wiest, R.: Ensembles of densely-connected cnns with label-uncertainty for brain tumor segmentation. In: International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI) (2018)

  37. 37.

    Myronenko, A.: 3D MRI Brain Tumor Segmentation Using Autoencoder Regularization, International MICCAI Brainlesion Workshop. Springer, Cham (2018)

    Google Scholar 

  38. 38.

    Feng, X., et al.: Brain tumor segmentation using an ensemble of 3d u-nets and overall survival prediction using radiomic features. Front. Comput. Neurosci. 14, 25 (2020)

    Article  Google Scholar 

  39. 39.

    Yuan, Y.: Automatic brain tumor segmentation with scale attention network. arXiv:2011.03188

  40. 40.

    Lyu, C., Shu, H.: A two-stage cascade model with variational autoencoders and attention gates for MRI brain tumor segmentation. arXiv:2011.02881 (2020)

  41. 41.

    Isensee, F., et al.: NnU-net for brain tumor segmentation. arXiv:2011.00848(2020)

  42. 42.

    Ahmad, P., et al.: Context aware 3D UNet for brain tumor segmentation. arXiv:2010.13082 (2020)

  43. 43.

    Huhdanpaa, H., Hwang, D.H., Gasparian, G.G., Booker, M.T., Cen, Y., Lerner, A., Boyko, O.B., Go, J.L., Kim, P.E., Rajamohan, A., Law, M.: Image coregistration: quantitative processing framework for the assessment of brain lesions. J. Digit. Imaging 27(3), 369–379 (2014)

    Article  Google Scholar 

  44. 44.

    Abbasi, M., Khosravi, M.R.: A robust and accurate particle filter-based pupil detection method for big datasets of eye video. J. Grid Comput. 18(2), 305–325 (2020)

    Article  Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Suresh Chandra Satapathy.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Saha, A., Zhang, YD. & Satapathy, S.C. Brain Tumour Segmentation with a Muti-Pathway ResNet Based UNet. J Grid Computing 19, 43 (2021). https://doi.org/10.1007/s10723-021-09590-y

Download citation

Keywords

  • Glio3ma
  • Multimodal MR images
  • Multiclass segmentation
  • Encoder
  • Decoder