Skip to main content
SpringerLink
Log in

A Three-Stage 2D–3D Convolutional Network Ensemble for Segmenting Malignant Brain Tumors on MRI Images

  • Published:
Cybernetics and Systems Analysis Aims and scope

In this paper, the problem of brain tumor binary semantic segmentation based on MRI images is solved. The pixel-by-pixel determination of the anomaly region boundary is performed given the presence of noise in the training sample and input data. It is shown that in the case of using 2D models for solving 3D segmentation problems, spatial information between neighboring slices is not considered and not utilized. A new approach for optimizing the processing of 3D medical images using ensemble topologies in three stages is proposed. The first stage involves 2D ensemble processing of images in three dimensions to maximize the diversity criterion and accurately capture the region of interest (ROI). The second stage involves ensemble processing of 3D ROI regions extracted by neural networks with different 3D input block sizes to ensure diversity. At the third stage, the extracted abnormal regions (malignant tumors) from the first and the second stages are aggregated by weighted summation and thresholding to obtain the final binary 3D mask of the brain tumor. The proposed approach was tested on the LGG Brain MRI Segmentation Dataset. The segmentation accuracy is significantly improved in terms of dice score and mIoU, reducing the use of computationally expensive 3D networks.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. P. Malhotra, S. Gupta, D. Koundal, A. Zaguia, and W. Enbeyle, “Deep neural networks for medical image segmentation,” J. Healthc. Eng., Vol. 2022, 9580991 (2022). https://doi.org/10.1155/2022/9580991.

    Article  Google Scholar 

  2. R. Wang, T. Lei, R. Cui, B. Zhang, H. Meng, and A. K. Nandi, “Medical image segmentation using deep learning: A survey,” IET Image Process., Vol. 16, Iss. 5, 1243–1267 (2022). https://doi.org/10.1049/ipr2.12419.

  3. I. Rizwan-I-Haque and J. Neubert, “Deep learning approaches to biomedical image segmentation,” Inform. Med. Unlocked, Vol. 18, 100297 (2020). https://doi.org/10.1016/j.imu.2020.100297.

  4. D. Muller and F. Kramer, “MIScnn: a framework for medical image segmentation with convolutional neural networks and deep learning,” BMC Med. Imaging, Vol. 21, 12 (2019). https://doi.org/10.1186/s12880-020-00543-7.

    Article  Google Scholar 

  5. M. Xin and Y. Wang, “Research on image classification model based on deep convolution neural network,” EURASIP J. Image Video Process., Vol. 2019, 40 (2019). https://doi.org/10.1186/s13640-019-0417-8.

    Article  Google Scholar 

  6. M. Manoj krishna, M. Neelima, M. Harshali, and M. Venu Gopala Rao, “Image classification using Deep learning,” International J. of Engineering & Technology, Vol. 7, No. 2.7: Special Iss. 7, 614–617 (2018). https://doi.org/10.14419/ijet.v7i2.7.10892.

  7. S. Minaee, Yu. Boykov, F. Porikli, A. Plaza, A. Kehtarnavaz, and D. Terzopoulos, “Image segmentation using deep learning: A survey,” IEEE Trans. Pattern Anal. Mach. Intell., Vol. 44, Iss. 7, 3523–3542 (2022). https://doi.org/10.1109/TPAMI.2021.3059968.

  8. M. T. Islam, B. M. N. Karim Siddique, S. Rahman, and T. Jabid, “Image recognition with deep learning,” in: Proc. 2018 Intern. Conf. on Intelligent Informatics and Biomedical Sciences (ICIIBMS) (Bangkok, Thailand, October 21–24, 2018), IEEE (2018), pp. 106–110. https://doi.org/10.1109/ICIIBMS.2018.8550021.

  9. J. Chai, H. Zeng, A. Li, and E. W. T. Ngai, “Deep learning in computer vision: A critical review of emerging techniques and application scenarios,” Machine Learning with Applications, Vol. 6, 100134 (2021). https://doi.org/10.1016/j.mlwa.2021.100134.

  10. A. Voulodimos, N. Doulamis, A. Doulamis, and E. Protopapadakis, “Deep learning for computer vision: A brief review,” Comput. Intell. Neurosci., Vol. 2018, 7068349 (2018). https://doi.org/10.1155/2018/7068349.

    Article  Google Scholar 

  11. J. Long, E. Shelhamer, and T. Darrell, “Fully convolutional networks for semantic segmentation,” in: Proc. 2015 IEEE Conf. on Computer Vision and Pattern Recognition (CVPR) (Boston, MA, USA, June 7–12, 2015), IEEE, (2015), pp. 3431–3440. https://doi.org/10.1109/CVPR.2015.7298965.

  12. O. Ronneberger, P. Fischer, and T. Brox, “U-Net: Convolutional networks for biomedical image segmentation,” in: N. Navab, J. Hornegger, W. Wells, and A. Frangi (eds.), Medical Image Computing and Computer-Assisted Intervention — MICCAI 2015, MICCAI 2015; Lecture Notes in Computer Science, Vol. 9351, Springer, Cham (2015), pp. 234–241. https://doi.org/10.1007/978-3-319-24574-4_28.

  13. F. Milletari, N. Navab, and S.-A. Ahmadi, “V-Net: fully convolutional neural networks for volumetric medical image segmentation,” in: Proc. 2016 Fourth Intern. Conf. on 3D Vision (3DV) (Stanford, CA, USA, October 25–28, 2016), IEEE (2016), pp. 565–571. https://doi.org/10.1109/3dv.2016.79.

  14. Y. J. Seol, Y. J. Kim, Y. S. Kim, Y. W. Cheon, and K. G. Kim, “A study on 3D deep learning-based automatic diagnosis of nasal fractures,” Sensors, Vol. 22, Iss. 2, 506 (2022). https://doi.org/10.3390/s22020506.

  15. R. Q. Charles, H. Su, M. Kaichun, and L. J. Guibas, “PointNet: Deep learning on point sets for 3D classification and segmentation,” in: Proc. 2017 IEEE Conf. on Computer Vision and Pattern Recognition (CVPR) (Honolulu, HI, USA, July 21–26, 2017), IEEE (2017), pp. 77–85. https://doi.org/10.1109/CVPR.2017.16.

  16. X. Feng, N. J. Tustison, S. H. Patel, and C. H. Meyer, “Brain tumor segmentation using an ensemble of 3D U-Nets and overall survival prediction using radiomic features,” Front. Comput. Neurosci., Vol. 14, 25 (2020). https://doi.org/10.3389/fncom.2020.00025.

    Article  Google Scholar 

  17. S. Das, S. Bose, G. K. Nayak, and S. Saxena, “Deep learning-based ensemble model for brain tumor segmentation using multi-parametric MR scans,” Open Computer Science, Vol. 12, No. 1, 211–226 (2022). https://doi.org/10.1515/comp-2022-0242.

    Article  Google Scholar 

  18. H. Zheng, Y. Zhang, L. Yang, P. Liang, Z. Zhao, C. Wang, and D. Z. Chen, “A new ensemble learning framework for 3D biomedical image segmentation,” in: Proc. Thirty-Third AAAI Conf. on Artificial Intelligence and Thirty-First Innovative Applications of Artificial Intelligence Conf. and Ninth AAAI Symp. on Educational Advances in Artificial Intelligence (AAAI’19/IAAI’19/EAAI’19) (Honolulu, Hawaii USA, 27 January – 1 February 2019), Vol. 33, No. 01, AAAI Press, Palo Alto, California, USA (2019), pp. 5909–5916. https://doi.org/10.1609/aaai.v33i01.33015909.

  19. H. Cao, H. Liu, E. Song, G. Ma, X. Xu, R. Jin, T. Liu, and Ch.-Ch. Hung, “Multi-branch ensemble learning architecture based on 3D CNN for false positive reduction in lung nodule detection,” IEEE Access, Vol. 7, 67380–67391 (2019). https://doi.org/10.1109/ACCESS.2019.2906116.

    Article  Google Scholar 

  20. Brain MRI Segmentation. LGG Segmentation Dataset. URL: https://www.kaggle.com/datasets/mateuszbuda/lgg-mri-segmentation.

  21. X. Zhou, K. Yamada, R. Takayama, X. Zhou, T. Hara, H. Fujita, S. Wang, and T. Kojima, “Performance evaluation of 2D and 3D deep learning approaches for automatic segmentation of multiple organs on CT images,” in: N. Petrick and K. Mori (eds.), Proc. of SPIE, Medical Imaging 2018: Computer-Aided Diagnosis, Vol. 10575 (2018), pp. 105752C-1–105752C-6. https://doi.org/10.1117/12.2295178.

  22. M. Srikrishna, R. A. Heckemann, J. B. Pereira, G. Volpe, A. Zettergren, S. Kern, E. Westman, I. Skoog, and M. Scholl, “Comparison of two-dimensional- and three-dimensional-based U-Net architectures for brain tissue classification in one-dimensional brain CT,” Front. Comput. Neurosci., Vol. 15, 785244 (2022). https://doi.org/10.3389/fncom.2021.785244.

  23. A. Stamoulakatos, J. Cardona, C. Michie, I. Andonovic, P. Lazaridis, X. Bellekens, R. Atkinson, Md. M. Hossain, and C. Tachtatzis, “A comparison of the performance of 2D and 3D convolutional neural networks for subsea survey video classification,” in: Proc. OCEANS 2021 San Diego — Porto (San Diego, Portugal, September 20–23, 2021), IEEE (2021), pp. 1–10. https://doi.org/10.23919/OCEANS44145.2021.9706125.

  24. L. Yang, Y. Zhang, J. Chen, S. Zhang, and D. Z. Chen, “Suggestive annotation: A deep active learning framework for biomedical image segmentation,” in: M. Descoteaux, L. Maier-Hein, A. Franz, P. Jannin, D. Collins, and S. Duchesne (eds.), Medical Image Computing and Computer Assisted Intervention — MICCAI 2017, MICCAI 2017; Lecture Notes in Computer Science, Vol. 10435, Springer, Cham (2017), pp. 399–407. https://doi.org/10.1007/978-3-319-66179-7_46.

  25. T. D. Bui, J. Shin, and T. Moon, “Skip-connected 3D DenseNet for volumetric infant brain MRI segmentation,” Biomed. Signal Process. Control, Vol. 54, 101613 (2019). https://doi.org/10.1016/j.bspc.2019.101613.

  26. A. K. Anaraki, M. Ayati, and F. Kazemi, “Magnetic resonance imaging-based brain tumor grades classification and grading via convolutional neural networks and genetic algorithms,” Biocybern. Biomed. Eng., Vol. 39, No. 1, 63–74 (2019). https://doi.org/10.1016/j.bbe.2018.10.004.

    Article  Google Scholar 

  27. F. Ozyurt, E. Sert, E. Avci, and E. Dogantekin, “Brain tumor detection based on a Convolutional Neural Network with neutrosophicexpert maximum fuzzy sure entropy,” Measurement, Vol. 147, 106830 (2019). https://doi.org/10.1016/j.measurement.2019.07.058.

  28. S. Ghosh and K. C. Santosh, “Tumor segmentation in brain MRI: U-Nets versus feature pyramid network,” in: 2021 IEEE 34th Intern. Symp. on Computer-Based Medical Systems (CBMS) (Aveiro, Portugal, June 7–9, 2021), IEEE (2021), pp. 31–36. https://doi.org/10.1109/CBMS52027.2021.00013.

    Article  Google Scholar 

  29. Z. Li, Y. Wang, J. Yu, Z. Shi, Y. Guo, L. Chen, and Y. Mao, “Low-grade glioma segmentation based on CNN with fully connected CRF,” J. Healthc. Eng., Vol. 2017, 9283480 (2017). https://doi.org/10.1155/2017/9283480.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. V. M. Glushkov Institute of Cybernetics, National Academy of Sciences of Ukraine, Kyiv, Ukraine

    V. Sineglazov

  2. National Technical University of Ukraine “Igor Sikorsky Kyiv Polytechnic Institute”, Kyiv, Ukraine

    K. Riazanovskiy & O. Klanovets

Authors
  1. V. Sineglazov
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. K. Riazanovskiy
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. O. Klanovets
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to V. Sineglazov.

Additional information

Translated from Kibernetyka ta Systemnyi Analiz, No. 2, March–April, 2023, pp. 27–41.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sineglazov, V., Riazanovskiy, K. & Klanovets, O. A Three-Stage 2D–3D Convolutional Network Ensemble for Segmenting Malignant Brain Tumors on MRI Images. Cybern Syst Anal 59, 199–211 (2023). https://doi.org/10.1007/s10559-023-00555-5

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10559-023-00555-5

Keywords

Search

Navigation