Skip to main content
SpringerLink
Log in

An improved 3D U-Net-based deep learning system for brain tumor segmentation using multi-modal MRI

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Cancer is a lethal illness that requires an initial stage prognosis to enhance patient survival rate. Accurate brain tumor and their sub-structure segmentation through Magnetic Resonance Images (MRIs) is a tough endeavor. Owing to the heterogeneous tumor areas, automatically segmenting brain tumors has proved to be a critical task even for neural network-based algorithms, some tumor regions remain unidentified due to their small size and the variation in area occupancy among tumor sub-classes. Current progress in the area of neural networks has been employed to enhance the segmentation performance. This study designed an intelligent 3D U-Net encoder-decoder-based system for automatic detection and brain tumor sub-structure segmentation. Our proposed 3D model constitutes neural units (the basic building blocks) followed by transition layer blocks and skip connections. BraTS 2018 and private local datasets are used to evaluate the proposed model which segments the Whole Tumor (WT), Tumor Core (TC), and the Enhancing Tumor (ET). The training accuracy, validation accuracy, dice score, sensitivities, and specificities of WT, CT, and ET zones are computed. The experimental results demonstrate that dice scores are 0.913, 0.874, and 0.801 for the BraTS 2018 dataset. The developed models performance was further evaluated by utilizing the dataset from a local hospital containing 71 subjects. The dice scores of 0.891, 0.834, and 0.776 are achieved by the proposed model on the private dataset. The practicability of the proposed model was assessed by the comparative studies of our model with existing literature.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Data availability

The local dataset collected and/or analyzed during this research is not publicly available as it is the proprietary property of the Advanced Diagnostic Center. Any queries regarding to dataset or any raw data of this study can be directed to the first author S.A. at the email: saqibsaleem788@hotmail.com. The BRATS 2018 dataset is publicly available at https://www.med.upenn.edu/sbia/brats2018/data.html.

References

  1. Bauer S, Wiest R, N olte L-P, Reyes M, (2013) A survey of mri-based medical image analysis for brain tumor studies. Phys Med Biol 58(13):R97

    Article  Google Scholar 

  2. Işın A, Direkoğlu C, Şah M (2016) Review of mri-based brain tumor image segmentation using deep learning methods. Procedia Comput Sci 102:317–324

    Article  Google Scholar 

  3. Goetz M, Weber C, Binczyk F, Polanska J, Tarnawski R, Bobek-Billewicz B, Koethe U, Kleesiek J, Stieltjes B, Maier-Hein KH (2015) Dalsa: domain adaptation for supervised learning from sparsely annotated mr images. IEEE Trans Med Imaging 35(1):184–196

    Article  Google Scholar 

  4. Yang G, Jones TL, Barrick TR, Howe FA (2014) Discrimination between glioblastoma multiforme and solitary metastasis using morphological features derived from the p: q tensor decomposition of diffusion tensor imaging. NMR Biomed 27(9):1103–1111

    Article  Google Scholar 

  5. Yang G, Jones TL, Howe FA, Barrick TR (2016) Morphometric model for discrimination between glioblastoma multiforme and solitary metastasis using three-dimensional shape analysis. Magn Reson Med 75(6):2505–2516

    Article  Google Scholar 

  6. Soltaninejad M, Ye X, Yang G, Allinson N, Lambrou T et al (2014) Brain tumour grading in different mri protocols using svm on statistical features

  7. Soltaninejad M, Yang G, Lambrou T, Allinson N, Jones TL, Barrick TR, Howe FA, Ye X (2017) Automated brain tumour detection and segmentation using superpixel-based extremely randomized trees in flair mri. Int J Comput Assist Radiol Surg 12(2):183–203

    Article  Google Scholar 

  8. Soltaninejad M, Yang G, Lambrou T, Allinson N, Jones TL, Barrick TR, Howe FA, Ye X (2018) Supervised learning based multimodal mri brain tumour segmentation using texture features from supervoxels. Comput Methods Programs biomed 157:69–84

    Article  Google Scholar 

  9. Wu W, Chen AYC, Zhao L, Corso JJ (2014) Brain tumor detection and segmentation in a crf (conditional random fields) framework with pixel-pairwise affinity and superpixel-level features. Int J Comput Assist Radiol Surg 9(2):241–253

    Article  Google Scholar 

  10. Amarapur B et al (2019) Cognition-based mri brain tumor segmentation technique using modified level set method. Cogn Technol Work 21(3):357–369

    Article  Google Scholar 

  11. Olabarriaga SD, Smeulders AWM (2001) Interaction in the segmentation of medical images: A survey. Med Image Anal 5(2):127–142

    Article  Google Scholar 

  12. Yao J (2006) Image processing in tumor imaging. New techniques in oncologic imaging, pp 79–102

  13. Menze BH, Jakab A, Bauer S, Kalpathy-Cramer J, Farahani K, Kirby J, Burren Y, Porz N, Slotboom Wiest R et al (2014) The multimodal brain tumor image segmentation benchmark (brats). IEEE Trans Med Imaging 34(10):1993–2024

    Article  Google Scholar 

  14. Iqbal S, Qureshi AN, Aurangzeb K, Alhussein M, Haider SI, Rida I (2023) Amiac: adaptive medical image analyzes and classification, a robust self-learning framework. Neural Comput Appl pp 1–29

  15. Bozkurt F (2023) Skin lesion classification on dermatoscopic images using effective data augmentation and pre-trained deep learning approach. Multimed Tools Appl 82(12):18985–19003

    Article  Google Scholar 

  16. Bozkurt F (2022) A deep and handcrafted features-based framework for diagnosis of covid-19 from chest x-ray images. Concurr Comput Pract Experience 34(5):e6725

    Article  Google Scholar 

  17. Ali S, Li J, Pei Y, Rehman KU (2022) A multi-module 3d u-net learning architecture for brain tumor segmentation. In: International conference on data mining and big data, Springer, pp 57–69

  18. Ali S, Li J, Pei Y, Khurram R, Rehman Ku, Rasool AB (2021) State-of-the-art challenges and perspectives in multi-organ cancer diagnosis via deep learning-based methods. Cancers 13(21):5546

    Article  Google Scholar 

  19. Shelhamer E, Long J, Darrell T (2016) Fully convolutional networks for semantic segmentation. IEEE Trans Pattern Anal Mach Intell 39(4):640–651

    Article  Google Scholar 

  20. Ronneberger O, Fischer P, Brox T (2015) U-net: Convolutional networks for biomedical image segmentation. In: International conference on medical image computing and computer-assisted intervention, Springer, pp 234–241

  21. Qin C, Yujie W, Liao W, Zeng J, Liang S, Zhang X (2022) Improved u-net3+ with stage residual for brain tumor segmentation. BMC Med Imaging 22(1):1–15

    Article  Google Scholar 

  22. Zhao X, Yihong W, Song G, Li Z, Zhang Y, Fan Y (2018) A deep learning model integrating fcnns and crfs for brain tumor segmentation. Med Image Anal 43:98–111

    Article  Google Scholar 

  23. Pereira S, Alves V, Silva CA (2018) Adaptive feature recombination and recalibration for semantic segmentation: application to brain tumor segmentation in mri. In: International conference on medical image computing and computer-assisted intervention, Springer, pp 706–714

  24. Badrinarayanan V, Kendall A, Cipolla R (2017) Segnet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans Pattern Anal Mach Intell 39(12):2481–2495

    Article  Google Scholar 

  25. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556

  26. Iqbal S, Ghani MU, Saba T, Rehman A (2018) Brain tumor segmentation in multi-spectral mri using convolutional neural networks (cnn). Microscopy Res Tech 81(4):419–427

    Article  Google Scholar 

  27. Dong H, Yang G, Liu F, Mo Y, Guo Y (2017) Automatic brain tumor detection and segmentation using u-net based fully convolutional networks. In: Annual conference on medical image understanding and analysis, pp 506–517. Springer

  28. Zhou Z, Rahman Siddiquee MM, Tajbakhsh N, Liang J (2018) Unet++: A nested u-net architecture for medical image segmentation. In: Deep learning in medical image analysis and multimodal learning for clinical decision support, pp 3–11. Springer

  29. Tu Z, Bai X (2009) Auto-context and its application to high-level vision tasks and 3d brain image segmentation. IEEE transactions on pattern analysis and machine intelligence 32(10):1744–1757

    Google Scholar 

  30. Meier R, Bauer S, Slotboom J, Wiest R, Reyes M (2013) A hybrid model for multimodal brain tumor segmentation. Multimodal Brain Tumor Segmentation 31:31–37

    Google Scholar 

  31. Meier R, Bauer S, Slotboom J, Wiest R, Reyes M (2014) Appearance-and context-sensitive features for brain tumor segmentation. Proceedings of MICCAI BRATS Challenge, pp 020–026

  32. Bauer S, Nolte LP, Reyes M (2011) Fully automatic segmentation of brain tumor images using support vector machine classification in combination with hierarchical conditional random field regularization. In: International conference on medical image computing and computer-assisted intervention, pp 354–361. Springer

  33. Sikka K, Sinha N, Singh PK, Mishra AK (2009) A fully automated algorithm under modified fcm framework for improved brain mr image segmentation. Magn Reson Imaging 27(7):994–1004

    Article  Google Scholar 

  34. Pinto A, Pereira S, Correia H, Oliveira J, DMLD Rasteiro, Silva CA (2015) Brain tumour segmentation based on extremely randomized forest with high-level features. In: 2015 37th annual international conference of the IEEE engineering in medicine and biology society (EMBC), pp 3037–3040. IEEE

  35. Hussain S, Anwar SM, Majid M (2017) Brain tumor segmentation using cascaded deep convolutional neural network. In 2017 39th annual International Conference of the IEEE engineering in medicine and biology Society (EMBC), pp 1998–2001. IEEE

  36. Havaei M, Davy A, Farley DW, Biard A, Courville A, Bengio Y, Pal C, Jodoin PM, Larochelle H (2017) Brain tumor segmentation with deep neural networks. Med Image Anal 35:18–31

    Article  Google Scholar 

  37. Hu K, Gan Q, Zhang Y, Deng S, Xiao F, Huang W, Cao C, Gao X (2019) Brain tumor segmentation using multi-cascaded convolutional neural networks and conditional random field. IEEE Access 7:92615–92629

    Article  Google Scholar 

  38. Pereira S, Pinto A, Alves V, Silva CA (2015) Deep convolutional neural networks for the segmentation of gliomas in multi-sequence mri. In: BrainLes 2015, pp 131–143. Springer

  39. Hussain S, Anwar SM, Majid M (2018) Segmentation of glioma tumors in brain using deep convolutional neural network. Neurocomputing 282:248–261

    Article  Google Scholar 

  40. Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z (2016) Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2818–2826

  41. Kamnitsas K, Ferrante E, Parisot S, Ledig C, Nori AV, Criminisi A, Rueckert D, Glocker B (2016) Deepmedic for brain tumor segmentation. In: International workshop on Brainlesion: Glioma, multiple sclerosis, stroke and traumatic brain injuries, Springer, pp 138–149

  42. Wang G, Li W, Ourselin S, Vercauteren T (2017) Automatic brain tumor segmentation using cascaded anisotropic convolutional neural networks. In: International MICCAI brainlesion workshop, pp 178–190. Springer

  43. Albishri AA, Shah SJH, Kang SS, Lee Y (2022) Am-unet: automated mini 3d end-to-end u-net based network for brain claustrum segmentation. Multimed Tools Appl 81(25):36171–36194

    Article  Google Scholar 

  44. Punn NS, Agarwal S (2021) Multi-modality encoded fusion with 3d inception u-net and decoder model for brain tumor segmentation. Multimed Tools Appl 80(20):30305–30320

    Article  Google Scholar 

  45. Raza R, Bajwa UI, Mehmood Y, Anwar MW, Jamal MH (2023) dresu-net: 3d deep residual u-net based brain tumor segmentation from multimodal mri. Biomed Signal Proc Control 79:103861

    Article  Google Scholar 

  46. Li P, Wu W, Liu L, Serry FM, Wang J, Han H (2022) Automatic brain tumor segmentation from multiparametric mri based on cascaded 3d u-net and 3d u-net++. Biomed Signal Proc Control 78:103979

    Article  Google Scholar 

  47. Isensee F, Jäger PF, Kohl SAA, Petersen J, Maier-Hein KH (2019) Automated design of deep learning methods for biomedical image segmentation. arXiv preprint arXiv:1904.08128

  48. Isensee F, Kickingereder P, Wick W, Bendszus M, Maier-Hein KH (2018) No new-net. In: International MICCAI Brainlesion Workshop, pp 234–244. Springer

  49. McKinley R, Meier R, Wiest R (2018) Ensembles of densely-connected cnns with label-uncertainty for brain tumor segmentation. In: International MICCAI Brainlesion Workshop, pp 456–465. Springer

  50. Li H, Li A, Wang M (2019) A novel end-to-end brain tumor segmentation method using improved fully convolutional networks. Comput Biol Med 108:150–160

    Article  Google Scholar 

  51. Kayalibay B, Jensen G, van der Smagt P (2017) Cnn-based segmentation of medical imaging data. arXiv preprint arXiv:1701.03056

  52. Battalapalli D, Rao BP, Yogeeswari P, Kesavadas C, Rajagopalan V (2022) An optimal brain tumor segmentation algorithm for clinical mri dataset with low resolution and non-contiguous slices. BMC Med Imaging 22(1):1–12

    Article  Google Scholar 

  53. Ngo DK, Tran MT, Kim SH, Yang HJ, Lee GS (2020) Multi-task learning for small brain tumor segmentation from mri. Appl Sci 10(21):7790

    Article  Google Scholar 

  54. Myronenko A (2018) 3d mri brain tumor segmentation using autoencoder regularization. In: International MICCAI Brainlesion Workshop, pp 311–320. Springer

  55. Zhou C, Ding C, Wang X, Lu Z, Tao D (2020) One-pass multi-task networks with cross-task guided attention for brain tumor segmentation. IEEE Trans Image Proc 29:4516–4529

    Article  Google Scholar 

  56. Hu Y, Liu X, Wen X, Niu C, Xia Y (2018) Brain tumor segmentation on multimodal mr imaging using multi-level upsampling in decoder. In: International MICCAI Brainlesion Workshop, pp 168–177. Springer

  57. Carver E, Liu C, Zong W, Dai Z, Snyder JM, Lee J, Wen N (2018) Automatic brain tumor segmentation and overall survival prediction using machine learning algorithms. In: International MICCAI Brainlesion Workshop, pp 406–418. Springer

  58. Islam M, Jose V, Ren H (2018) Glioma prognosis: Segmentation of the tumor and survival prediction using shape, geometric and clinical information. In: International MICCAI Brainlesion Workshop, pp 142–153. Springer

Download references

Funding

No external funding was received for this study.

Author information

Authors and Affiliations

  1. Faculty of Information Technology, Beijing University of Technology, 100124, Beijing, China

    Saqib Ali, Rooha Khurram, Anaa Yasin, Zeeshan Shaukat & Zareen Sakhawat

  2. College of Mechatronics and Control Engineering, Shenzhen University, Shenzhen, China

    Khalil ur Rehman

  3. Department of Radiology, Advance Diagnostic Center, Islamabad, Pakistan

    Ghulam Mujtaba

Authors
  1. Saqib Ali
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rooha Khurram
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Khalil ur Rehman
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Anaa Yasin
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Zeeshan Shaukat
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Zareen Sakhawat
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Ghulam Mujtaba
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Each author took part in the present works conception and/or design. Saqib Ali and Rooha Khurram were responsible for carrying out the tasks of material preparation, and original draft writing. Ghulam Mujtaba helps in data collection and data analysis. Khalil ur Rehman and Zareen Sakhawat helped in reviewing and editing the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Saqib Ali.

Ethics declarations

Informed consent statement

Institutional Review Board (IBR) approval was obtained. From, Advanced Diagnostic Center (Pvt) Ltd, and Advanced International Hospital Institutional Review Board (Approval number: MR-20-2543). All procedures performed in this study were by the ethical standards of the institutional and/or local research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards. This study has an IRB approval of an informed consent waiver.

Conflict of interest

The authors declare that there is no conict of interest regarding the publication of this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Saqib Ali and Rooha Khurram contributed equally to this study.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ali, S., Khurram, R., Rehman, K.u. et al. An improved 3D U-Net-based deep learning system for brain tumor segmentation using multi-modal MRI. Multimed Tools Appl (2024). https://doi.org/10.1007/s11042-024-19406-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11042-024-19406-2

Keywords

Search

Navigation