Skip to main content
SpringerLink
Log in

Improving brain tumor segmentation performance using CycleGAN based feature extraction

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

The analysis of brain tumors plays a significant role in medical applications and provides a huge amount of anatomic and functional information. Automatic tumor segmentation is one of the most challenging issues among radiologists and other specialists intent on lowering and eliminating manual detection errors and speeding up the detection of tissue types. In recent years, automatic segmentation combined with deep learning has been proven to be more powerful than traditional approaches. This study introduces a two-step model for the segmentation of brain tumors on multi-channel MRI images based on the Generative Adversarial Network (GAN). First of all, we use CycleGAN and two segmentors added to its structure to train networks that produce new features appropriate for segmentation. Different modules of MRI images are fed into these two-way networks, and the same modules, along with the target tumor segment, are requested at the output.

In addition to segmentation, features created in the middle layers of these networks are capable of mapping images to one another. In the second step, the transfer learning technique is used to extract the related subnets and inject the features produced into the main segmentation network.

In fact, the present study attempts to find features with greater detection power by converting different MRI images to each other. It is assumed that features beneficial in the conversion of different MRI image modules to each other will also improve segmentation performance. The proposed method is evaluated on BraTS 2018, and the results demonstrate the superiority of this method over the majority of existing approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Data Availability

The data sets analyzed during the current study are available at the following repository, which requires registration: MICCAI BraTS DataSets : (https://ipp.cbica.upenn.edu/)

References

  1. Bakas S, Reyes M, Jakab A, Bauer S, Rempfler M, Crimi A, Shinohara RT, Berger C, Ha SM, Rozycki M et al (2018) Identifying the best machine learning algorithms for brain tumor segmentation, progression assessment, and overall survival prediction in the brats challenge. arXiv:1811.02629

  2. Chaddad A (2015) Automated feature extraction in brain tumor by magnetic resonance imaging using gaussian mixture models. Int J Biomed Imaging, vol 2015

  3. Chan TF, Vese LA (2001) Active contours without edges. IEEE Trans Image Process 10(2):266–277

    Article  MATH  Google Scholar 

  4. Chen T, Liu X, Feng R, Wang W, Yuan C, Lu W, Haizhen H, Gao H, Ying H, Chen DZ et al (2021) Discriminative cervical lesion detection in colposcopic images with global class activation and local bin excitation. IEEE J Biomed Health Inf

  5. Chen J, Ying H, Liu X, Gu J, Feng R, Chen T, Gao H, Wu J (2020) A transfer learning based super-resolution microscopy for biopsy slice images: the joint methods perspective. IEEE/ACM Trans Computat Biol Bioinform 18(1):103–113

    Google Scholar 

  6. Çiçek Ö, Abdulkadir A, Lienkamp SS, Brox T, Ronneberger O (2016) 3D u-net: learning dense volumetric segmentation from sparse annotation. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 424–432

  7. Conte GM, Weston AD, Vogelsang DC, Philbrick KA, Cai JC, Barbera M, Sanvito F, Lachance DH, Jenkins RB, Tobin WO et al (2021) Generative adversarial networks to synthesize missing t1 and flair mri sequences for use in a multisequence brain tumor segmentation model. Radiology 299(2):313– 323

    Article  Google Scholar 

  8. Dar SU, Yurt M, Karacan L, Erdem A, Erdem E, Çukur T (2019) Image synthesis in multi-contrast mri with conditional generative adversarial networks. IEEE Trans Med Imaging 38(10):2375–2388

    Article  Google Scholar 

  9. Dhanachandra N, Manglem K, Chanu YJ (2015) Image segmentation using k-means clustering algorithm and subtractive clustering algorithm. Procedia Comput Sci 54:764–771

    Article  Google Scholar 

  10. Dong H, Yang G, Liu F, Mo Y, Guo Y (2017) Automatic brain tumor detection and segmentation using u-net based fully convolutional networks. In: Annual conference on medical image understanding and analysis. Springer, pp 506–517

  11. Gao H, Xu K, Cao M, Xiao J, Xu Q, Yin Y (2021) The deep features and attention mechanism-based method to dish healthcare under social iot systems: an empirical study with a hand-deep local-global net. IEEE Trans Computat Social Syst

  12. Georgiadis P, Cavouras D, Kalatzis I, Daskalakis A, Kagadis GC, Sifaki K, Malamas M, Nikiforidis G, Solomou E (2008) Improving brain tumor characterization on mri by probabilistic neural networks and non-linear transformation of textural features. Comput Methods Programs Biomed 89(1):24–32

    Article  Google Scholar 

  13. Ilunga-Mbuyamba E, Avina–Cervantes JG, Garcia–Perez A, De Jesus Romero–Troncoso R, Aguirre–Ramos H, Cruz–Aceves I, Chalopin C (2017) Localized active contour model with background intensity compensation applied on automatic mr brain tumor segmentation. Neurocomputing 220:84–97

    Article  Google Scholar 

  14. Isola P, Zhu J-Y, Zhou T, Efros AA (2017) Image-to-image translation with conditional adversarial networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1125–1134

  15. Kamnitsas K, Ledig C, Newcombe VF, Simpson JP, Kane AD, Menon DK, Rueckert D, Glocker B (2017) Efficient multi-scale 3d cnn with fully connected crf for accurate brain lesion segmentation. Med Image Anal 36:61–78

    Article  Google Scholar 

  16. Khalid NEA, Ibrahim S, Haniff P (2011) Mri brain abnormalities segmentation using k-nearest neighbors(k-nn). Int J Comput Sci Eng 3(2):980–990

    Google Scholar 

  17. Kharrat A, Gasmi K, Messaoud MB, Benamrane N, Abid M (2010) A hybrid approach for automatic classification of brain mri using genetic algorithm and support vector machine. Leonardo J Sci 17(1):71–82

    Google Scholar 

  18. Kohl S, Bonekamp D, Schlemmer H-P, Yaqubi K, Hohenfellner M, Hadaschik B, Radtke J-P, Maier-Hein K (2017) Adversarial networks for the detection of aggressive prostate cancer. arXiv:1702.08014

  19. Li B, You X, Wang J, Peng Q, Yin S, Qi R, Ren Q, Hong Z (2021) Ias-net: joint intraclassly adaptive gan and segmentation network for unsupervised cross-domain in neonatal brain mri segmentation. Med Phys 48(11):6962–6975

    Article  Google Scholar 

  20. Lu X, Wang W, Ma C, Shen J, Shao L, Porikli F (2019) See more, know more: unsupervised video object segmentation with co-attention siamese networks. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 3623–3632

  21. Lu X, Wang W, Shen J, Crandall D, Luo J (2020) Zero-shot video object segmentation with co-attention siamese networks. IEEE Trans Pattern Anal Mach Intell

  22. Lu X, Wang W, Shen J, Crandall D, Van Gool L (2021) Segmenting objects from relational visual data. IEEE Trans Pattern Anal Mach Intell

  23. Mondal AK, Dolz J, Desrosiers C (2018) Few-shot 3d multi-modal medical image segmentation using generative adversarial learning. arXiv:1810.12241

  24. Nema S, Dudhane A, Murala S, Naidu S (2020) Rescuenet: an unpaired gan for brain tumor segmentation. Biomed Signal Process Contr 55:101641

    Article  Google Scholar 

  25. Oh KT, Lee S, Lee H, Yun M, Yoo SK (2020) Semantic segmentation of white matter in fdg-pet using generative adversarial network. J Digit Imaging 33(4):816–825

    Article  Google Scholar 

  26. Pinto A, Pereira S, Correia H, Oliveira J, Rasteiro DM, Silva CA (2015) Brain tumour segmentation based on extremely randomized forest with high-level features. In: 2015 37th Annual international conference of the IEEE engineering in medicine and biology society (EMBC). IEEE, pp 3037–3040

  27. Rai HM (2021) Chatterjee, k.: 2d mri image analysis and brain tumor detection using deep learning cnn model leu-net. Multimed Tools Appl 80(28):36111–36141

    Article  Google Scholar 

  28. Ramesh S, Sasikala S, Paramanandham N (2021) Segmentation and classification of brain tumors using modified median noise filter and deep learning approaches. Multimed Tools Appl 80(8):11789–11813

    Article  Google Scholar 

  29. Rezaei M, Harmuth K, Gierke W, Kellermeier T, Fischer M, Yang H, Meinel C (2017) A conditional adversarial network for semantic segmentation of brain tumor. In: International MICCAI brainlesion workshop. Springer, pp 241–252

  30. Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 234–241

  31. Roy Choudhury A, Vanguri R, Jambawalikar SR, Kumar P (2018) Segmentation of brain tumors using deeplabv3+. In: International MICCAI brainlesion workshop. Springer, pp 154–167

  32. Saman S, Jamjala Narayanan S (2019) Survey on brain tumor segmentation and feature extraction of mr images. Int J Multimed Inf Retrieval 8(2):79–99

    Article  Google Scholar 

  33. Shi Y, Cheng K, Liu Z (2019) Hippocampal subfields segmentation in brain mr images using generative adversarial networks. Biomed Eng Online 18 (1):1–12

    Article  Google Scholar 

  34. Soleymanifard M, Hamghalam M (2022) Multi-stage glioma segmentation for tumour grade classification based on multiscale fuzzy c-means. Multimed Tools Appl 81(6):8451–8470

    Article  Google Scholar 

  35. Steenwijk MD, Pouwels PJ, Daams M, van Dalen JW, Caan MW, Richard E, Barkhof F, Vrenken H (2013) Accurate white matter lesion segmentation by k nearest neighbor classification with tissue type priors (knn-ttps). NeuroImage: Clinical 3:462–469

    Article  Google Scholar 

  36. Swaraja K, Meenakshi K, Valiveti HB, Karuna G (2022) Segmentation and detection of brain tumor through optimal selection of integrated features using transfer learning. Multimed Tools Appl:1–33

  37. Wang G, Li W, Ourselin S, Vercauteren T (2017) Automatic brain tumor segmentation using cascaded anisotropic convolutional neural networks. In: International MICCAI brainlesion workshop. Springer, pp 178–190

  38. Wu W, Chen AY, Zhao L, Corso JJ (2014) Brain tumor detection and segmentation in a crf (conditional random fields) framework with pixel-pairwise affinity and superpixel-level features. Int J Comput Assisted Radio Surgery 9 (2):241–253

    Article  Google Scholar 

  39. Wu M-N, Lin C-C, Chang C-C (2007) Brain tumor detection using color-based k-means clustering segmentation. In: Third international conference on intelligent information hiding and multimedia signal processing (IIH-MSP 2007). IEEE, vol 2, pp 245–250

  40. Xiao J, Xu H, Gao H, Bian M, Li Y (2021) A weakly supervised semantic segmentation network by aggregating seed cues: the multi-object proposal generation perspective. ACM Trans Multimidia Comput Commun Appl 17(1s):1–19

    Article  Google Scholar 

  41. Xue Y, Xu T, Zhang H, Long LR, Huang X (2018) Segan: adversarial network with multi-scale l1 loss for medical image segmentation. Neuroinformatics 16(3):383–392

    Article  Google Scholar 

  42. Xun S, Li D, Zhu H, Chen M, Wang J, Li J, Chen M, Wu B, Zhang H, Chai X et al (2022) Generative adversarial networks in medical image segmentation: a review. Comput Bio Med 140:105063

    Article  Google Scholar 

  43. Zhao X, Wu Y, Song G, Li Z, Zhang Y, Fan Y (2018) A deep learning model integrating fcnns and crfs for brain tumor segmentation. Med Image Anal 43:98–111

    Article  Google Scholar 

  44. Zhu H, Cao Z, Lian L, Ye G, Gao H, Wu J (2022) Cariesnet: a deep learning approach for segmentation of multi-stage caries lesion from oral panoramic x-ray image. Neural Comput Appl:1–9

  45. Zhu J-Y, Park T, Isola P, Efros AA (2017) Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of the IEEE international conference on computer vision, pp 2223–2232

Download references

Author information

Authors and Affiliations

  1. Department of Electrical and Computer Engineering, University of Zanjan, Zanjan, Iran

    Hamed Mohammadi Azni & Mohsen Afsharchi

  2. Radiology Department, Mazandaran University of Medical Sciences, Sari, Iran

    Armin Allahverdi

Authors
  1. Hamed Mohammadi Azni
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mohsen Afsharchi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Armin Allahverdi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mohsen Afsharchi.

Ethics declarations

Ethics approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Conflict of Interests

The authors declare that they have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Azni, H.M., Afsharchi, M. & Allahverdi, A. Improving brain tumor segmentation performance using CycleGAN based feature extraction. Multimed Tools Appl 82, 18039–18058 (2023). https://doi.org/10.1007/s11042-022-14174-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-14174-3

Keywords

Search

Navigation