Skip to main content
SpringerLink
Log in

Tumor type detection in brain MR images of the deep model developed using hypercolumn technique, attention modules, and residual blocks

  • Original Article
  • Published:
Medical & Biological Engineering & Computing Aims and scope Submit manuscript

Abstract

Brain cancer is a disease caused by the growth of abnormal aggressive cells in the brain outside of normal cells. Symptoms and diagnosis of brain cancer cases are producing more accurate results day by day in parallel with the development of technological opportunities. In this study, a deep learning model called BrainMRNet which is developed for mass detection in open-source brain magnetic resonance images was used. The BrainMRNet model includes three processing steps: attention modules, the hypercolumn technique, and residual blocks. To demonstrate the accuracy of the proposed model, three types of tumor data leading to brain cancer were examined in this study: glioma, meningioma, and pituitary. In addition, a segmentation method was proposed, which additionally determines in which lobe area of the brain the two classes of tumors that cause brain cancer are more concentrated. The classification accuracy rates were performed in the study; it was 98.18% in glioma tumor, 96.73% in meningioma tumor, and 98.18% in pituitary tumor. At the end of the experiment, using the subset of glioma and meningioma tumor images, it was determined which at brain lobe the tumor region was seen, and 100% success was achieved in the analysis of this determination. In this study, a hybrid deep learning model is presented to determine the detection of the brain tumor. In addition, open-source software was proposed, which statistically found in which lobe region of the human brain the brain tumor occurred. The methods applied and tested in the experiments have shown promising results with a high level of accuracy, precision, and specificity. These results demonstrate the availability of the proposed approach in clinical settings to support the medical decision regarding brain tumor detection.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Bisdas S, D’Arco F (2019) In: Barkhof F, Jager R, Thurnher M, Rovira Cañellas A (eds) Pediatric tumor neuroradiology BT - clinical neuroradiology: the ESNR textbook. Springer International Publishing, Cham, pp 1–80

    Google Scholar 

  2. Marszałek A, Szylberg Ł, Wiśniewski S (2016) Pathologic aspects of skull base tumors. Reports Pract Oncol Radiother J Gt Cancer Cent Pozn Polish Soc Radiat Oncol 21:288–303. https://doi.org/10.1016/j.rpor.2016.02.006

    Article  Google Scholar 

  3. Lah TT, Novak M, Breznik B (2020) Brain malignancies: glioblastoma and brain metastases. Semin Cancer Biol 60:262–273. https://doi.org/10.1016/j.semcancer.2019.10.010

    Article  CAS  PubMed  Google Scholar 

  4. Wang J, Pulido JS, O’Neill BP, Johnston PB (2015) Second malignancies in patients with primary central nervous system lymphoma. Neuro-Oncology 17:129–135. https://doi.org/10.1093/neuonc/nou105

    Article  PubMed  Google Scholar 

  5. (2019) Brain and other nervous system cancer — cancer stat facts. In: Natl. Cancer Instıtute https://seer.cancer.gov/statfacts/html/brain.html.

  6. (2019) Brain tumor: statistics | Cancer.Net. In: Am. Cancer Soc. https://www.cancer.net/cancer-types/brain-tumor/statistics. Accessed 4 Feb 2020

  7. Siegel RL, Miller KD, Jemal A (2019) Cancer statistics, 2019. CA Cancer J Clin 69:7–34. https://doi.org/10.3322/caac.21551

    Article  PubMed  Google Scholar 

  8. Minniti G, Filippi AR, Osti MF, Ricardi U (2017) Radiation therapy for older patients with brain tumors. Radiat Oncol 12:101. https://doi.org/10.1186/s13014-017-0841-9

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  9. Hanif F, Muzaffar K, Perveen K et al (2017) Glioblastoma multiforme: a review of its epidemiology and pathogenesis through clinical presentation and treatment. Asian Pac J Cancer Prev 18:3–9. https://doi.org/10.22034/APJCP.2017.18.1.3

    Article  PubMed  PubMed Central  Google Scholar 

  10. Sultan HH, Salem NM, Al-Atabany W (2019) Multi-classification of brain tumor images using deep neural network. IEEE Access 7:69215–69225. https://doi.org/10.1109/ACCESS.2019.2919122

    Article  Google Scholar 

  11. Taylor OG, Brzozowski JS, Skelding KA (2019) Glioblastoma multiforme: an overview of emerging therapeutic targets. Front Oncol 9:963. https://doi.org/10.3389/fonc.2019.00963

    Article  PubMed  PubMed Central  Google Scholar 

  12. Thon N, Tonn J-C, Kreth F-W (2019) The surgical perspective in precision treatment of diffuse gliomas. Onco Targets Ther 12:1497–1508. https://doi.org/10.2147/OTT.S174316

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  13. Lemée J-M, Corniola MV, Da Broi M et al (2019) Extent of resection in meningioma: predictive factors and clinical implications. Sci Rep 9:5944. https://doi.org/10.1038/s41598-019-42451-z

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  14. Ferluga S, Baiz D, Hilton DA, Adams CL, Ercolano E, Dunn J, Bassiri K, Kurian KM, Hanemann CO (2020) Constitutive activation of the EGFR–STAT1 axis increases proliferation of meningioma tumor cells. Neuro-Oncology Adv 2. https://doi.org/10.1093/noajnl/vdaa008

  15. Chatzellis E, Alexandraki KI, Androulakis II, Kaltsas G (2015) Aggressive pituitary tumors. Neuroendocrinology 101:87–104. https://doi.org/10.1159/000371806

    Article  CAS  PubMed  Google Scholar 

  16. Vala H (2013) In: Mesquita JR (ed) The endocrine glands in the dog: from the cell to hormone. IntechOpen, Rijeka, p Ch. 6

    Google Scholar 

  17. de Zakir JC, Casulari LA, Rosa JWC et al (2016) Prognostic value of invasion, markers of proliferation, and classification of giant pituitary tumors, in a georeferred cohort in Brazil of 50 patients, with a long-term postoperative follow-up. Int J Endocrinol 2016:7964523–7964514. https://doi.org/10.1155/2016/7964523

    Article  PubMed  PubMed Central  Google Scholar 

  18. Trouillas J, Jaffrain-Rea ML, Vasiljevic A, Raverot G, Roncaroli F, Villa C (2020) How to classify the pituitary neuroendocrine tumors (PitNET)s in 2020. Cancers (Basel) 12:1–17. https://doi.org/10.3390/cancers12020514

    Article  CAS  Google Scholar 

  19. (2019) Pituitary tumor: MedlinePlus Medical Encyclopedia. In: U.S. Natl. Libr. Med. https://medlineplus.gov/ency/article/000704.htm. Accessed 4 Feb 2020

  20. Iqbal S, Khan MUG, Saba T, Rehman A (2017) Computer-assisted brain tumor type discrimination using magnetic resonance imaging features. Biomed Eng Lett 8:5–28. https://doi.org/10.1007/s13534-017-0050-3

    Article  PubMed  PubMed Central  Google Scholar 

  21. Gupta N, Bhatele P, Khanna P (2019) Glioma detection on brain MRIs using texture and morphological features with ensemble learning. Biomed Signal Proc Cont 47:115–125. https://doi.org/10.1016/j.bspc.2018.06.003

    Article  Google Scholar 

  22. Tamrazi B, Shiroishi MS, Liu C-SJ (2016) Advanced imaging of intracranial meningiomas. Neurosurg Clin N Am 27:137–143. https://doi.org/10.1016/j.nec.2015.11.004

    Article  PubMed  PubMed Central  Google Scholar 

  23. Nour M, Cömert Z, Polat K (2020) A novel medical diagnosis model for COVID-19 infection detection based on deep features and Bayesian optimization. Appl soft Comput 106580. https://doi.org/10.1016/j.asoc.2020.106580

  24. Ahuja AS (2019) The impact of artificial intelligence in medicine on the future role of the physician. PeerJ 7:e7702–e7702. https://doi.org/10.7717/peerj.7702

    Article  PubMed  PubMed Central  Google Scholar 

  25. Toğaçar M, Özkurt KB, Ergen B, Cömert Z (2020) BreastNet: a novel convolutional neural network model through histopathological images for the diagnosis of breast cancer. Phys A Stat Mech its Appl 545:123592. https://doi.org/10.1016/j.physa.2019.123592

    Article  Google Scholar 

  26. Toğaçar M, Ergen B, Cömert Z (2020) BrainMRNet: brain tumor detection using magnetic resonance images with a novel convolutional neural network model. Med Hypotheses 134:109531. https://doi.org/10.1016/j.mehy.2019.109531

    Article  PubMed  Google Scholar 

  27. Tandel GS, Balestrieri A, Jujaray T, Khanna NN, Saba L, Suri JS (2020) Multiclass magnetic resonance imaging brain tumor classification using artificial intelligence paradigm. Comput Biol Med 122:103804. https://doi.org/10.1016/j.compbiomed.2020.103804

    Article  PubMed  Google Scholar 

  28. Meola A, Rao J, Chaudhary N, Sharma M, Chang SD (2018) Gold nanoparticles for brain tumor imaging: a systematic review. Front Neurol 9:328. https://doi.org/10.3389/fneur.2018.00328

    Article  PubMed  PubMed Central  Google Scholar 

  29. Mohsen H, El-Dahshan E-SA, El-Horbaty E-SM, Salem A-BM (2018) Classification using deep learning neural networks for brain tumors. Futur Comput Informatics J 3:68–71. https://doi.org/10.1016/j.fcij.2017.12.001

    Article  Google Scholar 

  30. Pushpa BR, Louies F (2019) Detection and classification of brain tumor using machine learning approaches. Int J Res Pharm Sci 10:2153–2162. https://doi.org/10.26452/ijrps.v10i3.1442

    Article  Google Scholar 

  31. Byale H, Lingaraju GM, Sivasubramanian S (2018) Automatic segmentation and classification of brain tumor using machine learning techniques. Int J Appl Eng Res 13:11686–11692

    Google Scholar 

  32. Sawant A, Bhandari M, Yadav R et al (2018) Brain cancer detection from MRI: a machine learning approach (tensorflow). Int Res J Eng Technol 05:2089

    Google Scholar 

  33. Ahammed Muneer KV, Paul Joseph K (2019) Automation of MR brain image classification for malignancy detection. J Mech Med Biol 19:1–18. https://doi.org/10.1142/S0219519419400025

    Article  Google Scholar 

  34. Kohl SAA, Romera-Paredes B, Meyer C, et al (2018) A probabilistic U-net for segmentation of ambiguous images. Adv Neural Inf Process Syst 2018-Decem:6965–6975

  35. Çiçek Ö, Abdulkadir A, Lienkamp SS, et al (2016) 3D U-net: learning dense volumetric segmentation from sparse annotation. Lect Notes Comput Sci (including Subser Lect Notes Artif Intell Lect Notes Bioinformatics) 9901 lncs:424–432. https://doi.org/10.1007/978-3-319-46723-8_49

  36. Ding Y, Acosta R, Enguix V, Suffren S, Ortmann J, Luck D, Dolz J, Lodygensky GA (2020) Using deep convolutional neural networks for neonatal brain image segmentation. Front Neurosci 14:207. https://doi.org/10.3389/fnins.2020.00207

    Article  PubMed  PubMed Central  Google Scholar 

  37. Mahbod A, Wang C, Chowdhury M, Smedby Ö (2016) Brain segmentation using artificial neural networks with shape context. Under Prep:1–29

  38. Schlemper J, Oktay O, Schaap M, Heinrich M, Kainz B, Glocker B, Rueckert D (2019) Attention gated networks: learning to leverage salient regions in medical images. Med Image Anal 53:197–207. https://doi.org/10.1016/j.media.2019.01.012

    Article  PubMed  Google Scholar 

  39. Germain H, Bourmaud G, Lepetit V (2019) Sparse-to-dense hypercolumn matching for long-term visual localization. Proc - 2019 Int Conf 3D Vision, 3DV 2019 513–523. https://doi.org/10.1109/3DV.2019.00063

  40. Cheng J (2017) Brain tumor dataset. https://doi.org/10.6084/m9.figshare.1512427.v5

    Book  Google Scholar 

  41. Cheng J, Yang W, Huang M, Huang W, Jiang J, Zhou Y, Yang R, Zhao J, Feng Y, Feng Q, Chen W (2016) Retrieval of brain tumors by adaptive spatial pooling and fisher vector representation. PLoS One 11:e0157112

    Article  Google Scholar 

  42. (2019) Converting image. In: GitHub web. https://github.com/shimat/opencvsharp/wiki/Converting-Image. Accessed 5 Feb 2020

  43. Solda F, Fersht N (2017) Radiotherapy for pituitary tumours. NCBI

    Google Scholar 

  44. Toğaçar M (2020) The BrainMRNet model. GitHub, San Francisco https://github.com/happytgcr/brainMRNet.

    Google Scholar 

  45. Aguiar JA, Gong ML, Unocic RR et al (2019) Decoding crystallography from high-resolution electron imaging and diffraction datasets with deep learning. Sci Adv 5:eaaw1949. https://doi.org/10.1126/sciadv.aaw1949

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  46. Meng Z, Li L, Jiao L, Feng Z, Tang X, Liang M (2019) Fully dense multiscale fusion network for hyperspectral image classification. Remote Sens 11:1–20. https://doi.org/10.3390/rs11222718

    Article  CAS  Google Scholar 

  47. Chao H, Fenhua W, Ran Z (2019) Sign language recognition based on CBAM-ResNet. In: Proceedings of the 2019 International Conference on Artificial Intelligence and Advanced Manufacturing. ACM, New York, NY, USA, pp 48:1--48:6

  48. Woo S, Park J, Lee J-Y, Kweon IS CBAM: convolutional block attention module. Eccv2018

  49. Xie S, Girshick R, Dollár P et al (2016) Aggregated residual transformations for deep neural networks. https://doi.org/10.1109/cvpr.2017.634

  50. Yue B, Fu J, Liang J (2018) Residual recurrent neural networks for learning sequential representations. Inf 9. https://doi.org/10.3390/info9030056

  51. Chen G, Chen P, Shi Y, et al (2019) Rethinking the usage of batch normalization and dropout in the training of deep neural networks

  52. Ramachandram DW Taylor G (2018) Skin lesion segmentation using deep hypercolumn descriptors

    Google Scholar 

  53. Hariharan B, Arbeláez P, Girshick R, Malik J (2017) Object instance segmentation and fine-grained localization using hypercolumns. IEEE Trans Pattern Anal Mach Intell 39:627–639. https://doi.org/10.1109/TPAMI.2016.2578328

    Article  PubMed  Google Scholar 

  54. Min J, Lee J, Ponce J, Cho M (2020) Learning to compose hypercolumns for visual correspondence

    Google Scholar 

  55. Sengur A, Budak U, Akbulut Y, et al (2019) 7 - a survey on neutrosophic medical image segmentation. In: Guo Y, Ashour ASBT-NS in MIA (eds). Academic press, pp 145–165

  56. (2019) Image thresholding. In: OpenCV. https://docs.opencv.org/master/d7/d4d/tutorial_py_thresholding.html. Accessed 16 Feb 2020

  57. (2019) Changing color spaces documentation. In: OpenCV. https://docs.opencv.org/3.0-beta/doc/py_tutorials/py_imgproc/py_colorspaces/py_colorspaces.html. Accessed 16 Feb 2020

  58. (2019) Operations on arrays documentation. In: OpenCV. https://docs.opencv.org/2.4/modules/core/doc/operations_on_arrays.html. Accessed 16 Feb 2020

  59. Mabrouk MS, Afify HM, Marzouk SY (2019) Fully automated computer-aided diagnosis system for micro calcifications cancer based on improved mammographic image techniques. Ain Shams Eng J 10:517–527. https://doi.org/10.1016/j.asej.2019.01.009

    Article  Google Scholar 

  60. Zhan Y, Zhang G (2019) An improved OTSU algorithm using histogram accumulation moment for ore segmentation. Symmetry (Basel) 11:11. https://doi.org/10.3390/sym11030431

    Article  Google Scholar 

  61. Engin N, Erman Z (2016) Düzce Üniversitesi Bilim ve Teknoloji Dergisi. Düzce Üniversitesi Bilim ve Teknol Derg 4:293–304

    Google Scholar 

  62. Tumen V, Yildirim O, Ergen B (2018) Recognition of road type and quality for advanced driver assistance systems with deep learning. Elektron ir Elektrotechnika 24:67–74. https://doi.org/10.5755/j01.eie.24.6.22293

    Article  Google Scholar 

  63. Chicco D, Jurman G (2020) The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation. BMC Genomics 21:1–13. https://doi.org/10.1186/s12864-019-6413-7

    Article  Google Scholar 

  64. Cömert Z (2020) Fusing fine-tuned deep features for recognizing different tympanic membranes. Biocybern Biomed Eng 40:40–51. https://doi.org/10.1016/j.bbe.2019.11.001

    Article  Google Scholar 

  65. Gumaei A, Hassan MM, Hassan MR, Alelaiwi A, Fortino G (2019) A hybrid feature extraction method with regularized extreme learning machine for brain tumor classification. IEEE Access 7:36266–36273. https://doi.org/10.1109/access.2019.2904145

    Article  Google Scholar 

  66. Deepak S, Ameer PM (2019) Brain tumor classification using deep CNN features via transfer learning. Comput Biol Med 111:103345. https://doi.org/10.1016/j.compbiomed.2019.103345

    Article  CAS  PubMed  Google Scholar 

Download references

Funding

This work was supported by the Scientific Research Projects Coordination Unit of Firat University. Project number MF.20.11.

Author information

Authors and Affiliations

  1. Department of Computer Technology, Technical Sciences Vocational School, Fırat University, Elazig, Turkey

    Mesut Toğaçar & Burhan Ergen

  2. Department of Software Engineering, Faculty of Engineering, Samsun University, Samsun, Turkey

    Zafer Cömert

Authors
  1. Mesut Toğaçar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Burhan Ergen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zafer Cömert
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mesut Toğaçar.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Code availability

The weblink to download the source codes of the BrainMRNet model written in the Python, the analysis results, and the source codes analyzing which lobe region of the tumor region is located and the relevant sub-dataset; https://github.com/happytgcr/brainMRNet.

Ethical approval

This article does not contain any data or other information from studies or experimentation, with the involvement of human or animal subjects.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Toğaçar, M., Ergen, B. & Cömert, Z. Tumor type detection in brain MR images of the deep model developed using hypercolumn technique, attention modules, and residual blocks. Med Biol Eng Comput 59, 57–70 (2021). https://doi.org/10.1007/s11517-020-02290-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11517-020-02290-x

Keywords

Search

Navigation