Skip to main content
SpringerLink
Log in

An ensemble-based transfer learning model for predicting the imbalance heart sound signal using spectrogram images

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Heart sound signal analysis is an important area in healthcare, and the detection of imbalanced heart sounds can provide valuable diagnostic information. However, due to heart sound variability, accurate prediction of imbalanced signals remains challenging. The issue of class inequality has received much attention from numerous scientific domains. The correct classification becomes increasingly challenging as data scale and data imbalance increase. Traditional classifiers tend to favor the dominant class and overlook the minority class, which is frequently considerably more significant when dealing with imbalanced learning problems. We propose an ensemble learning algorithm based on a transfer learning convolutional neural network (CNN) model to solve these challenges to predict imbalanced heart sound signals. We employ spectrogram images and STFT to extract the relevant features from Phonocardiogram (PCG) data. Our model leverages the pre-trained CNN architecture and fine-tunes it on the spectrogram images to improve the prediction performance. Moreover, we incorporate an ensemble approach to improve the model’s robustness and accuracy. Our experimental results on a publicly available PhysioNet PCG dataset demonstrate that the proposed algorithm outperforms existing state-of-the-art methods in terms of accuracy, sensitivity, and specificity. The ensemble methodology comprising AlexNet, SqueezeNet, and VGG19 models was proposed and achieved the highest level of performance, resulting in an accuracy of 99.20% and a sensitivity rate of 99.47%. Our study showcases the potential of leveraging technological advancements to predict unbalanced Phonocardiogram (PCG) signals using spectrogram images. This research opens up promising avenues for future exploration in cardiac diagnostics. Specifically, the ensemble-based transfer learning model proposed in this study holds great promise.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Algorithm 1
Fig. 4
Algorithm 2
Fig. 5
Fig. 6

Similar content being viewed by others

Data Availability

The datasets utilized and/or examined in the current study are publicly accessible through the PhysioNet/CinC Challenge heart sound database [6]. This information is available to other researchers and can potentially contribute to further advancements in the field of heart sound analysis.

References

  1. Classifying heart sounds challenge. https://www.peterjbentley.com/heartchallenge/index.html. Accessed 16 Apr 2023

  2. Ajitkumar Singh S, Dinita Devi N, Majumder S (2022) An improved unsegmented phonocardiogram classification using nonlinear time scattering features. Comput J. https://doi.org/10.1093/comjnl/bxac025

    Article  Google Scholar 

  3. Martinez-Alajarin J, Ruiz-Merino R (2005) Efficient method for events detection in phonocardiographic signals. Bioeng Bioinspired Syst II, vol 5839, p 398. https://doi.org/10.1117/12.608203. October 2014

  4. Hamidi M, Ghassemian H, Imani M (2018) Classification of heart sound signal using curve fitting and fractal dimension. Biomed Signal Process Control 39:351–359. https://doi.org/10.1016/j.bspc.2017.08.002

    Article  Google Scholar 

  5. Zhou X, Liang W, Wang KIK, Wang H, Yang LT, Jin Q (2020) Deep-learning-enhanced human activity recognition for internet of healthcare things. IEEE Internet Things J 7(7):6429–6438. https://doi.org/10.1109/JIOT.2020.2985082

    Article  Google Scholar 

  6. Liu C et al (2016) An open access database for the evaluation of heart sound algorithms. Physiol Meas 37(12):2181. https://doi.org/10.1088/0967-3334/37/12/2181

    Article  Google Scholar 

  7. Yaseen, Son GY, Kwon S (2018) Classification of heart sound signal using multiple features. Appl Sci 8(12). https://doi.org/10.3390/app8122344

  8. Pathak A, Mandana K, Saha G (2022) Ensembled transfer learning and multiple kernel learning for phonocardiogram based atherosclerotic coronary artery disease detection. IEEE J Biomed Heal Inform 26(6):2804–2813. https://doi.org/10.1109/JBHI.2022.3140277

    Article  Google Scholar 

  9. Samanta P, Pathak A, Mandana K, Saha G (2019) Classification of coronary artery diseased and normal subjects using multi-channel phonocardiogram signal. Biocybern Biomed Eng 39(2):426–443. https://doi.org/10.1016/J.BBE.2019.02.003

    Article  Google Scholar 

  10. Li H et al (2020) A fusion framework based on multi-domain features and deep learning features of phonocardiogram for coronary artery disease detection. Comput Biol Med 120:103733. https://doi.org/10.1016/J.COMPBIOMED.2020.103733

    Article  Google Scholar 

  11. Pathak A, Samanta P, Mandana K, Saha G (2020) Detection of coronary artery atherosclerotic disease using novel features from synchrosqueezing transform of phonocardiogram. Biomed Signal Process Control 62:102055. https://doi.org/10.1016/J.BSPC.2020.102055

    Article  Google Scholar 

  12. Singh SA, Majumder S (2019) Classification of unsegmented heart sound recording using knn classifier. J Mech Med Biol 19(4):1950025. https://doi.org/10.1142/S0219519419500258

    Article  Google Scholar 

  13. Mishra M, Banerjee S, Thomas DC, Dutta S, Mukherjee A (2018) Detection of third heart sound using variational mode decomposition. IEEE Trans Instrum Meas 67(7):1713–1721. https://doi.org/10.1109/TIM.2018.2805198

    Article  Google Scholar 

  14. Maity A, Pathak A, Saha G (2023) Transfer learning based heart valve disease classification from Phonocardiogram signal. Biomed Signal Process Control 85:104805. https://doi.org/10.1016/j.bspc.2023.104805. August 2022

  15. Singh SA, Majumder S (2020) Short unsegmented PCG classification based on ensemble classifier. Turkish J Electr Eng Comput Sci 28(2):875–889. https://doi.org/10.3906/elk-1905-165

    Article  Google Scholar 

  16. Babaei S, Geranmayeh A (2009) Heart sound reproduction based on neural network classification of cardiac valve disorders using wavelet transforms of PCG signals. Comput Biol Med 39(1):8–15. https://doi.org/10.1016/J.COMPBIOMED.2008.10.004

    Article  Google Scholar 

  17. Karhade J, Dash S, Ghosh SK, Dash DK, Tripathy RK (2022) Time-Frequency-Domain Deep Learning Framework for the Automated Detection of Heart Valve Disorders Using PCG Signals. IEEE Trans Instrum Meas 71. https://doi.org/10.1109/TIM.2022.3163156

  18. Dominguez-Morales JP, Jimenez-Fernandez AF, Dominguez-Morales MJ, Jimenez-Moreno G (2018) Deep neural networks for the recognition and classification of heart murmurs using neuromorphic auditory sensors. IEEE Trans Biomed Circuits Syst 12(1):24–34. https://doi.org/10.1109/TBCAS.2017.2751545

    Article  Google Scholar 

  19. Gupta S, Agrawal M, Deepak D (2021) Gammatonegram based triple classification of lung sounds using deep convolutional neural network with transfer learning. Biomed Signal Process Control 70:102947. https://doi.org/10.1016/j.bspc.2021.102947

    Article  Google Scholar 

  20. Ismail S, Ismail B (2023) PCG signal classification using a hybrid multi round transfer learning classifier. Biocybern Biomed Eng 43(1):313–334

    Article  Google Scholar 

  21. Jamil S, Roy AM (2023) An efficient and robust Phonocardiography (PCG)-based Valvular Heart Diseases (VHD) detection framework using Vision Transformer (ViT). Comput Biol Med 158:106734

    Article  Google Scholar 

  22. Zang J, Lian C, Xu B, Zhang Z, Su Y, Xue C (2023) AmtNet: Attentional multi-scale temporal network for phonocardiogram signal classification. Biomed Signal Process Control 85:104934

    Article  Google Scholar 

  23. Riccio D, Brancati N, Sannino G, Verde L, Frucci M (2023) CNN-based classification of phonocardiograms using fractal techniques. Biomed Signal Process Control 86:105186

    Article  Google Scholar 

  24. Azam FB, Ansari MI, Nuhash SISK, McLane I, Hasan T (2022) Cardiac anomaly detection considering an additive noise and convolutional distortion model of heart sound recordings. Artif Intell Med 133:102417. https://doi.org/10.1016/j.artmed.2022.102417

    Article  Google Scholar 

  25. Thanaraj P, Parvathavarthini K, Snekhalatha B (2020) Automated heart sound classification system from unsegmented phonocardiogram ( PCG ) using deep neural network. Phys Eng Sci Med 43:505–515. https://doi.org/10.1007/s13246-020-00851-w

    Article  Google Scholar 

  26. Er MB (2021) Heart sounds classification using convolutional neural network with 1D-local binary pattern and 1D-local ternary pattern features. Appl Acoust 180:108152. https://doi.org/10.1016/j.apacoust.2021.108152

    Article  Google Scholar 

  27. Kalidas N, Patidar S, Nesaragi N (2021) Automated detection of abnormal heart sound signals using Fano-factor constrained tunable quality wavelet transform. Biocybern Biomed Eng 41:111–126. https://doi.org/10.1016/j.bbe.2020.12.007

    Article  Google Scholar 

  28. Zhang W, Han J, Deng S (2019) Abnormal heart sound detection using temporal quasi-periodic features and long short-term memory without segmentation. Biomed Signal Process Control 53:101560. https://doi.org/10.1016/j.bspc.2019.101560

    Article  Google Scholar 

  29. Xiao B, Xu Y, Bi X, Zhang J, Ma X (2020) Heart sounds classification using a novel 1-D convolutional neural network with extremely low parameter consumption. Neurocomputing 392:153–159. https://doi.org/10.1016/j.neucom.2018.09.101

    Article  Google Scholar 

  30. Chen Y, Wei S (2020) Classification of heart sounds based on the combination of the modified frequency wavelet transform and convolutional neural network. Med Biol Eng Comput 58:2039–2047

    Article  Google Scholar 

  31. Ding H et al (2023) RVGAN-TL: A generative adversarial networks and transfer learning-based hybrid approach for imbalanced data classification. Inf Sci (Ny) 629:184–203. https://doi.org/10.1016/J.INS.2023.01.147

    Article  Google Scholar 

  32. Khan JS, Kaushik M, Chaurasia A, Dutta MK, Burget R (2022) Cardi-Net: A deep neural network for classification of cardiac disease using phonocardiogram signal. Comput Methods Programs Biomed 219:106727. https://doi.org/10.1016/j.cmpb.2022.106727

    Article  Google Scholar 

  33. Wang C, Deng C, Yu Z, Hui D, Gong X, Luo R (2021) Adaptive ensemble of classifiers with regularization for imbalanced data classification. Inf Fusion 69:81–102. https://doi.org/10.1016/J.INFFUS.2020.10.017

    Article  Google Scholar 

  34. Ren J, Wang Y, Mao M, Ming Cheung Y (2022) Equalization ensemble for large scale highly imbalanced data classification. Knowledge-Based Syst 242:108295. https://doi.org/10.1016/J.KNOSYS.2022.108295

  35. Schmidt SE, Toft E, Holst-Hansen C, Graff C, Struijk JJ (2008) Segmentation of heart sound recordings from an electronic stethoscope by a duration dependent Hidden-Markov model. In computers in cardiology vol. 35, pp 345–348. https://doi.org/10.1109/CIC.2008.4749049

  36. Ismail S, Ismail B, Siddiqi I, Akram U (2022) PCG classification through spectrogram using transfer learning. Biomed Signal Process Control 79:104075. no P1. https://doi.org/10.1016/j.bspc.2022.104075

  37. Jablonski A, Dziedziech K (2022) Intelligent spectrogram - A tool for analysis of complex non-stationary signals. Mech Syst Signal Process 167:108554. no PA. https://doi.org/10.1016/j.ymssp.2021.108554

  38. Chen J, Chen J, Zhang D, Sun Y, Nanehkaran YA (2020) Using deep transfer learning for image-based plant disease identification. Comput Electron Agric 173:105393. https://doi.org/10.1016/j.compag.2020.105393

  39. Zhou R et al (2023) An adaptively weighted ensemble of multiple CNNs for carotid ultrasound image segmentation. Biomed Signal Process Control 83:104673. https://doi.org/10.1016/j.bspc.2023.104673

    Article  Google Scholar 

  40. Li R, Gao R, Suganthan PN (2023) A decomposition-based hybrid ensemble CNN framework for driver fatigue recognition. Inf Sci 624:833–848. https://doi.org/10.1016/j.ins.2022.12.088

  41. Stateczny A, Narahari SC, Vurubindi P, Guptha NS, Srinivas K (2023) Underground water level prediction in remote sensing images using improved hydro index value with ensemble classifier. Remote Sens 15(8):2015. https://doi.org/10.3390/rs15082015

    Article  Google Scholar 

  42. Xie Y, Sun W, Ren M, Chen S, Huang Z, Pan X (2023) Stacking ensemble learning models for daily runoff prediction using 1D and 2D CNNs. Expert Syst Appl 217:119469. https://doi.org/10.1016/J.ESWA.2022.119469

    Article  Google Scholar 

  43. Krizhevsky A, Sutskever I, Hinton GE (2017) ImageNet classification with deep convolutional neural networks. Commun ACM 60(6):84–90. https://doi.org/10.1145/3065386

    Article  Google Scholar 

  44. Iandola FN, Han S, Moskewicz MW, Ashraf K, Dally WJ, Keutzer K (2016) SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and ¡0.5MB model size. Feb. [Online]. Available: arXiv:1602.07360v4. Accessed 24 Apr 2023

  45. Simonyan K, Zisserman A (2014) Very Deep Convolutional Networks for Large-Scale Image Recognition. 3rd Int. Conf. Learn. Represent. ICLR 2015 - Conf Track Proc, Sep. [Online]. Available: arXiv:1409.1556v6. Accessed 24 Apr 2023

  46. Hazeri H, Zarjam P, Azemi G (2021) Classification of normal/abnormal PCG recordings using a time-frequency approach. Analog Integr Circuits Signal Process 109(2):459–465. https://doi.org/10.1007/s10470-021-01867-2

    Article  Google Scholar 

  47. Zeng W, Yuan J, Yuan C, Wang Q, Liu F, Wang Y (2020) A new approach for the detection of abnormal heart sound signals using TQWT, VMD and neural networks. Artif Intell Rev 54:1613–1647. https://doi.org/10.1007/s10462-020-09875-w

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, NIT Manipur, Langol, Imphal, 795004, Manipur, India

    Sinam Ajitkumar Singh & Khelchandra Thongam

  2. Department of Radiation Oncology, RIMS Imphal, Lamphel, Imphal, 795004, Manipur, India

    Ningthoujam Dinita Devi

  3. Department of Electronics and Communication Engineering, IIIT Nagpur, Waranga, Nagpur, 441108, Maharastra, India

    Khuraijam Nelson Singh

  4. Department of Electronics and Communication Engineering, RGMCET, Nandyal, Nandyal, 518501, Andhra Pradesh, India

    Balakrishna Reddy D

  5. Department of Information Technology, Tripura University, Suryamani Nagar, Agartala, 799022, Tripura, India

    Swanirbhar Majumder

Authors
  1. Sinam Ajitkumar Singh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ningthoujam Dinita Devi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Khuraijam Nelson Singh
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Khelchandra Thongam
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Balakrishna Reddy D
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Swanirbhar Majumder
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Swanirbhar Majumder.

Ethics declarations

Ethics approval

There is no issue with Ethical approval and Informed consent

Conflict of Interests

There is no conflict of interest

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Singh, S.A., Devi, N.D., Singh, K.N. et al. An ensemble-based transfer learning model for predicting the imbalance heart sound signal using spectrogram images. Multimed Tools Appl 83, 39923–39942 (2024). https://doi.org/10.1007/s11042-023-17186-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-17186-9

Keywords

Search

Navigation