Skip to main content
SpringerLink
Log in

Reinforcement learning (RL)-based semantic segmentation and attention based backpropagation convolutional neural network (ABB-CNN) for breast cancer identification and classification using mammogram images

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Breast cancer poses a threat to women’s health and contributes to an increase in mortality rates. Mammography has proven to be an effective tool for the early detection of breast cancer. However, it faces many challenges in early breast cancer detection due to poor image quality, traditional segmentation, and feature extraction. Therefore, this work addresses these issues and proposes an attention-based backpropagation convolutional neural network (ABB-CNN) to detect breast cancer from mammogram images more accurately. The proposed work includes image enhancement, reinforcement learning-based semantic segmentation (RLSS), and multiview feature extraction and classification. The image enhancement is performed by removing noise and artefacts through a hybrid filter (HF), image scaling through a pixel-based bilinear interpolation (PBI), and contrast enhancement through an election-based optimization (EO) algorithm. In addition, the RLSS introduces intelligent segmentation by utilizing a deep Q network (DQN) to segment the region of interest (ROI) strategically. Moreover, the proposed ABB-CNN facilitates multiview feature extraction from the segmented region to classify the mammograms into normal, malignant, and benign classes. The proposed framework is evaluated on the collected and the digital database for screening mammography (DDSM) datasets. The proposed framework provides better outcomes in terms of accuracy, sensitivity, specificity, precision, f-measure, false-negative rate (FNR) and area under the curve (AUC). This work achieved (99.20%, 99.35%), (99.56%, 99.66%), (98.96%, 98.99%), (99.05%, 99.12%), (0.44%, 0.34%), (99.31%, 99.39%) and (99.27%, 99.32%) of accuracy, sensitivity, specificity, precision, FNR, f-measure and AUC on (collected, DDSM datasets), respectively. This research addresses the prevalent challenges in breast cancer identification and offers a robust and highly accurate solution by integrating advanced deep-learning techniques. The evaluated results reveal the proposed framework’s potential in early breast cancer detection.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Algorithm 1
Algorithm 2
Fig. 6
Fig. 7
Algorithm 3
Fig. 8
Fig. 9
Algorithm 4
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20

Similar content being viewed by others

Data availability

The public dataset DDSM [43] is available online http://www.eng.usf.edu/cvprg/Mammography/Database.html, and the collected dataset is confidential due to privacy concerns.

References

  1. Hamed G, Marey M, Amin SE, Tolba MF (2021) Automated breast cancer detection and classification in full field digital mammograms using two full and cropped detection paths approach. IEEE Access 9:116898–116913. https://doi.org/10.1109/ACCESS.2021.3105924

    Article  Google Scholar 

  2. Mahmood T, Li J, Pei Y, Akhtar F, Imran A, Rehman KU (2020) A brief survey on breast cancer diagnostic with deep learning schemes using multi-image modalities. IEEE Access 8:165779–165809. https://doi.org/10.1109/ACCESS.2020.3021343

    Article  Google Scholar 

  3. Meenalochini G, Ramkumar S (2021) Survey of machine learning algorithms for breast cancer detection using mammogram images. Mater Today: Proc 37:2738–2743. https://doi.org/10.1016/j.matpr.2020.08.543

    Article  Google Scholar 

  4. Nagpure R, Chandak S, Pathak N (2020) Breast cancer detection using neural network mammogram. In: International conference on convergence to digital world-quo vadis (ICCDW). IEEE, pp 1–6. https://doi.org/10.1109/ICCDW45521.2020.9318635

  5. Alshammari MM, Almuhanna A, Alhiyafi J (2021) Mammography image-based diagnosis of breast cancer using machine learning: a pilot study. Sensors 22(1):203. https://doi.org/10.3390/s22010203

    Article  Google Scholar 

  6. Eltrass AS, Salama MS (2020) Fully automated scheme for computer-aided detection and breast cancer diagnosis using digitized mammograms. IET Image Proc 14(3):495–505. https://doi.org/10.1049/iet-ipr.2018.5953

    Article  Google Scholar 

  7. Singla C, Sarangi PK, Sahoo AK, Singh PK (2022) Deep learning enhancement on mammogram images for breast cancer detection. Mater Today: Proc 49:3098–3104. https://doi.org/10.1016/j.matpr.2020.10.951

    Article  Google Scholar 

  8. Sajjadnia Z, Khayami R, Moosavi MR (2020) Preprocessing breast cancer data to improve the data quality, diagnosis procedure, and medical care services. Cancer Inform 19:1176935120917955. https://doi.org/10.1177/1176935120917955

    Article  Google Scholar 

  9. Ghosh SK, Mitra A, Ghosh A (2021) A novel intuitionistic fuzzy soft set entrenched mammogram segmentation under multigranulation approximation for breast cancer detection in early stages. Expert Syst Appl 169:114329. https://doi.org/10.1016/j.eswa.2020.114329

    Article  Google Scholar 

  10. Riesaputri DF, Sari CA, De Rosal IMS, Rachmawanto EH (2020) Classification of breast cancer using PNN classifier based on GLCM feature extraction and GMM segmentation. In: International seminar on application for technology of information and communication (iSemantic). IEEE, pp 83–87. https://doi.org/10.1109/iSemantic50169.2020.9234207

  11. Zebari DA, Zeebaree DQ, Abdulazeez AM, Haron H, Hamed HNA (2020) Improved threshold based and trainable fully automated segmentation for breast cancer boundary and pectoral muscle in mammogram images. IEEE Access 8:203097–203116. https://doi.org/10.1109/ACCESS.2020.3036072

    Article  Google Scholar 

  12. Cheng Y, Gao Y, Xie L, Xie X, Lin W (2020) Spatial enhanced rotation aware network for breast mass segmentation in digital mammogram. IEEE Access 10:92559–92570. https://doi.org/10.1109/ACCESS.2020.2978009

    Article  Google Scholar 

  13. Muduli D, Dash R, Majhi B (2020) Automated breast cancer detection in digital mammograms: a moth flame optimization based ELM approach. Biomed Signal Process Control 59:101912. https://doi.org/10.1016/j.bspc.2020.101912

    Article  Google Scholar 

  14. Reddy VN, Rao PS (2020) Breast tumour detection by using new expectation maximization algorithm. In: 3rd International conference on intelligent sustainable systems (ICISS). IEEE, pp 901–906. https://doi.org/10.1109/ICISS49785.2020.9316034

  15. Vijayarajeswari R, Parthasarathy P, Vivekanandan S, Basha AA (2019) Classification of mammogram for early detection of breast cancer using SVM classifier and Hough transform. Measurement 146:800–805. https://doi.org/10.1016/j.measurement.2019.05.083

    Article  Google Scholar 

  16. Wisaeng K (2022) Breast cancer detection in mammogram images using k–means++ clustering based on cuckoo search optimization. Diagnostics 12(12):3088. https://doi.org/10.3390/diagnostics12123088

    Article  Google Scholar 

  17. Jiang G, Wei J, Xu Y, He Z, Zeng H, Wu J, Lu Y (2021) Synthesis of mammogram from digital breast tomosynthesis using deep convolutional neural network with gradient guided cGANs. IEEE Trans Med Imaging 40(8):2080–2091. https://doi.org/10.1109/TMI.2021.3071544

    Article  Google Scholar 

  18. Hsu SY, Wang CY, Kao YK, Liu KY, Lin MC, Yeh LR, Kao FC (2022) Using deep neural network approach for multiple-class assessment of digital mammography. Healthcare 10(12):2382. https://doi.org/10.3390/healthcare10122382

    Article  Google Scholar 

  19. Altameem A, Mahanty C, Poonia RC, Saudagar AKJ, Kumar R (2022) Breast cancer detection in mammography images using deep convolutional neural networks and fuzzy ensemble modeling techniques. Diagnostics 12(8):1812. https://doi.org/10.3390/diagnostics12081812

    Article  Google Scholar 

  20. Shakeel S, Raja G (2021) Classification of breast cancer from mammogram images using deep convolution neural networks. International Bhurban conference on applied sciences and technologies (IBCAST). IEEE, pp 595–599. https://doi.org/10.1109/IBCAST51254.2021.9393191

  21. Kaur P, Singh G, Kaur P (2019) Intellectual detection and validation of automated mammogram breast cancer images by multi-class SVM using deep learning classification. Inform Med Unlocked 16:100151. https://doi.org/10.1016/j.imu.2019.01.001

    Article  Google Scholar 

  22. Loizidou K, Skouroumouni G, Nikolaou C, Pitris C (2020) An automated breast micro-calcification detection and classification technique using temporal subtraction of mammograms. IEEE Access 8:52785–52795. https://doi.org/10.1109/ACCESS.2020.2980616

    Article  Google Scholar 

  23. Soleimani H, Michailovich OV (2020) On segmentation of pectoral muscle in digital mammograms by means of deep learning. IEEE Access 8:204173–204182. https://doi.org/10.1109/ACCESS.2020.3036662

    Article  Google Scholar 

  24. Patil RS, Biradar N (2021) Automated mammogram breast cancer detection using the optimized combination of convolutional and recurrent neural network. Evol Intel 14:1459–1474. https://doi.org/10.1007/s12065-020-00403-x

    Article  Google Scholar 

  25. Toz G, Erdoğmuş P (2021) A novel hybrid image segmentation method for detection of suspicious regions in mammograms based on adaptive multi-thresholding (HCOW). IEEE Access 9:85377–85391. https://doi.org/10.1109/ACCESS.2021.3089077

    Article  Google Scholar 

  26. Chouhan N, Khan A, Shah JZ, Hussnain M, Khan MW (2021) Deep convolutional neural network and emotional learning based breast cancer detection using digital mammography. Comput Biol Med 132:104318. https://doi.org/10.1016/j.compbiomed.2021.104318

    Article  Google Scholar 

  27. Kavitha T, Mathai PP, Karthikeyan C, Ashok M, Kohar R, Avanija J, Neelakandan S (2021) Deep learning based capsule neural network model for breast cancer diagnosis using mammogram images. Interdiscipl Sci Comput Life Sci 2021:1–17. https://doi.org/10.1007/s12539-021-00467-y

    Article  Google Scholar 

  28. Pawar SD, Sharma KK, Sapate SG, Yadav GY (2021) Segmentation of pectoral muscle from digital mammograms with depth-first search algorithm towards breast density classification. Biocybern Biomed Eng 41(3):1224–1241. https://doi.org/10.1016/j.bbe.2021.08.005

    Article  Google Scholar 

  29. Soulami KB, Kaabouch N, Saidi MN, Tamtaoui A (2021) Breast cancer: One-stage automated detection, segmentation, and classification of digital mammograms using UNet model based-semantic segmentation. Biomed Signal Process Control 66:102481. https://doi.org/10.1016/j.bspc.2021.102481

    Article  Google Scholar 

  30. Jebarani PE, Umadevi N, Dang H, Pomplun M (2021) A novel hybrid K-means and GMM machine learning model for breast cancer detection. IEEE Access 9:146153–146162. https://doi.org/10.1109/ACCESS.2021.3123425

    Article  Google Scholar 

  31. Zebari et al (2021) Breast cancer detection using mammogram images with improved multi-fractal dimension approach and feature fusion. Appl Sci 11(24):12122. https://doi.org/10.3390/app112412122

    Article  Google Scholar 

  32. Rehman KU, Li J, Pei Y, Yasin A, Ali S, Saeed Y (2021) Architectural distortion-based digital mammograms classification using depth wise convolutional neural network. Biology 11(1):15. https://doi.org/10.3390/biology11010015

    Article  Google Scholar 

  33. El Houby EM, Yassin NI (2021) Malignant and nonmalignant classification of breast lesions in mammograms using convolutional neural networks. Biomed Signal Process Control 70:102954. https://doi.org/10.1016/j.bspc.2021.102954

    Article  Google Scholar 

  34. Maqsood S, Damaševičius R, Maskeliūnas R (2022) TTCNN: A breast cancer detection and classification towards computer-aided diagnosis using digital mammography in early stages. Appl Sci 12(7):3273. https://doi.org/10.3390/app12073273

    Article  Google Scholar 

  35. Baccouche A, Garcia-Zapirain B, Zheng Y, Elmaghraby AS (2022) Early detection and classification of abnormality in prior mammograms using image-to-image translation and YOLO techniques. Comput Methods Programs Biomed 221:106884. https://doi.org/10.3390/app12073273

    Article  Google Scholar 

  36. Ibrokhimov B, Kang JY (2022) Two-stage deep learning method for breast cancer detection using high-resolution mammogram images. Appl Sci 12(9):4616. https://doi.org/10.3390/app12094616

    Article  Google Scholar 

  37. Zahoor S, Shoaib U, Lali IU (2022) Breast cancer mammograms classification using deep neural network and entropy-controlled whale optimization algorithm. Diagnostics 12(2):557. https://doi.org/10.3390/diagnostics12020557

    Article  Google Scholar 

  38. Almalki YE, Soomro TA, Irfan M, Alduraibi SK, Ali A (2022) Computerized analysis of mammogram images for early detection of breast cancer. Healthcare 5(10):801. https://doi.org/10.3390/healthcare10050801

    Article  Google Scholar 

  39. Mohiyuddin A, Basharat A, Ghani U, Peter V, Abbas S, Naeem OB, Rizwan M (2022) Breast tumor detection and classification in mammogram images using modified YOLOv5 network. Comput Math Methods Med 2022:1–16. https://doi.org/10.1155/2022/1359019

    Article  Google Scholar 

  40. Almalki YE, Soomro TA, Irfan M, Alduraibi SK, Ali A (2022) Impact of image enhancement module for analysis of mammogram images for diagnostics of breast cancer. Sensors 22(5):1868. https://doi.org/10.3390/s22051868

    Article  Google Scholar 

  41. Jakhar AK, Gupta A, Singh M (2023) SELF: a stacked-based ensemble learning framework for breast cancer classification. Evol Intell 2023:1–16. https://doi.org/10.1007/s12065-023-00824-4

    Article  Google Scholar 

  42. Fortis Hospital Kangra, https://www.fortishealthcare.com/india/fortis-hospital-in-kangra-himachal-pradesh

  43. Heath M, Bowyer K, Kopans D, Kegelmeyer P Jr, Moore R, Chang K, Munishkumaran S (1998) Current status of the digital database for screening mammography. Digit Mammogr Nijmegen 1998:457–460. https://doi.org/10.1007/978-94-011-5318-8_75

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science & Engineering, Jaypee University of Information Technology, Solan, HP, India

    Neha Thakur, Pardeep Kumar & Amit Kumar

Authors
  1. Neha Thakur
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Pardeep Kumar
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Amit Kumar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Pardeep Kumar.

Ethics declarations

Conflict of interest

The authors declare no conflict of interest regarding this research work with any other author or organization.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Thakur, N., Kumar, P. & Kumar, A. Reinforcement learning (RL)-based semantic segmentation and attention based backpropagation convolutional neural network (ABB-CNN) for breast cancer identification and classification using mammogram images. Neural Comput & Applic (2024). https://doi.org/10.1007/s00521-024-09721-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00521-024-09721-y

Keywords

Search

Navigation