Multi-stream Convolutional Autoencoder and 2D Generative Adversarial Network for Glioma Classification

  • Muhaddisa Barat AliEmail author
  • Irene Yu-Hua GuEmail author
  • Asgeir Store JakolaEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11678)


Diagnosis and timely treatment play an important role in preventing brain tumor growth. Deep learning methods have gained much attention lately. Obtaining a large amount of annotated medical data remains a challenging issue. Furthermore, high dimensional features of brain images could lead to over-fitting. In this paper, we address the above issues. Firstly, we propose an architecture for Generative Adversarial Networks to generate good quality synthetic 2D MRIs from multi-modality MRIs (T1 contrast-enhanced, T2, FLAIR). Secondly, we propose a deep learning scheme based on 3-streams of Convolutional Autoencoders (CAEs) followed by sensor information fusion. The rational behind using CAEs is that it may improve glioma classification performance (as comparing with conventional CNNs), since CAEs offer noise robustness and also efficient feature reduction hence possibly reduce the over-fitting. A two-round training strategy is also applied by pre-training on GAN augmented synthetic MRIs followed by refined-training on original MRIs. Experiments on BraTS 2017 dataset have demonstrated the effectiveness of the proposed scheme (test accuracy 92.04%). Comparison with several exiting schemes has provided further support to the proposed scheme.


Brain tumor classification Glioma grading Deep learning Image synthesis Generative Adversarial Networks Multi-stream Convolutional Autoencoders Information fusion 


  1. 1.
    Siegel, R.L., Miller, K.D., Jemal, A.: Cancer statistics, 2017. Cancer J. Clin. 67(1), 7–30 (2017)CrossRefGoogle Scholar
  2. 2.
    Cha, S.: Update on brain tumor imaging: from anatomy to physiology. Am. J. Neuroradiol. 27(3), 475–487 (2006)Google Scholar
  3. 3.
    Vincent, P., Larochelle, H., Lajoie, I., Bengio, Y., Manzagol, P.-A.: Stacked denoising autoencoders: learning useful representations in a deep network with a local denoising criterion. J. Mach. Learn. Res. 11, 3371–3408 (2010)MathSciNetzbMATHGoogle Scholar
  4. 4.
    Masci, J., Meier, U., Cireşan, D., Schmidhuber, J.: Stacked convolutional auto-encoders for hierarchical feature extraction. In: Honkela, T., Duch, W., Girolami, M., Kaski, S. (eds.) ICANN 2011. LNCS, vol. 6791, pp. 52–59. Springer, Heidelberg (2011). Scholar
  5. 5.
    Macyszyn, L., Akbari, H., et al.: Imaging patterns predict patient survival and molecular subtype in glioblastoma via machine learning techniques. Neuro-Oncology 18(3), 417–425 (2015)CrossRefGoogle Scholar
  6. 6.
    Yu, J., Shi, Z., et al.: Noninvasive IDH1 mutation estimation based on a quantitative radiomics approach for grade II glioma. Eur. Radiol. 27(8), 3509–3522 (2017)CrossRefGoogle Scholar
  7. 7.
    Pan, Y., et al.: Brain tumor grading based on neural networks and convolutional neural networks. In: 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Milan, pp. 699–702 (2015)Google Scholar
  8. 8.
    Akkus, Z., Ali, I., et al.: Predicting 1p19q Chromosomal Deletion of Low-Grade Gliomas from MR Images using Deep Learning. arXiv preprint arXiv: 1611.06939 (2016)
  9. 9.
    Goodfellow, I., et al.: Generative adversarial nets. In: Advances in Neural Information Processing Systems, pp. 2672–2680 (2014)Google Scholar
  10. 10.
    Costa, P., et al.: Towards Adversarial Retinal Image Synthesis. arXiv preprint arXiv:1701.08974 (2017)
  11. 11.
    Ben-Cohen, A., Klang, E., Raskin, S.P., Amitai, M.M., Greenspan, H.: Virtual PET images from CT data using deep convolutional networks; Initial results. arXiv preprint arXiv:1707.09585 (2017)
  12. 12.
    Xue, Y., Xu, T., Zhang, H., Long, R., Huang, X.: SegAN: adversarial network with multi-scale L1 loss for medical image segmentation. arXiv preprint arXiv:1706.01805 (2017)
  13. 13.
    Ge, C., Gu, I.Y., Jakola, A.S., Yang, J.: Deep learning and multi-sensor fusion for glioma classification using multistream 2D convolutional networks. In: 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Honolulu, HI, pp. 5894–5897 (2018)Google Scholar
  14. 14.
    Menze, B.H., et al.: The multimodal Brain Tumor Image Segmentation Benchmark (BRATS). IEEE Trans. Med. Imaging 34(10), 1993–2024 (2015)CrossRefGoogle Scholar
  15. 15.
    Chollet, F., et al.: Keras (2015).
  16. 16.
    Radford, A., Metz, L., Chintala, S.: Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv: 1511.06434 (2015)
  17. 17.
    Ye, F., Pu, J., Wang, J., Li, Y., Zha, H.: Glioma grading based on 3D multimodal convolutional neural network and privileged learning. In: 2017 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), Kansas City, MO, pp. 759–763 (2017)Google Scholar
  18. 18.
    Ge, C., Qu, Q., Gu, I.Y., Jakola, A.S.: 3D multi-scale convolutional networks for glioma grading using MR images. In: 2018 25th IEEE International Conference on Image Processing (ICIP), Athens, pp. 141–145 (2018)Google Scholar
  19. 19.
    Diba, A., et al.: Deep temporal linear encoding networks. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1541–1550 (2017)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Department of Electrical EngineeringChalmers University of TechnologyGothenburgSweden
  2. 2.Department of Clinical NeuroscienceSahlgrenska University HospitalGothenburgSweden

Personalised recommendations