Skip to main content

Advertisement

SpringerLink
Log in

Transformer-based temporal sequence learners for arrhythmia classification

  • Original Article
  • Published:
Medical & Biological Engineering & Computing Aims and scope Submit manuscript

Abstract

An electrocardiogram (ECG) plays a crucial role in identifying and classifying cardiac arrhythmia. Traditional methods employ handcrafted features, and more recently, deep learning methods use convolution and recursive structures to classify heart signals. Considering the time sequence nature of the ECG signal, a transformer-based model with its high parallelism is proposed to classify ECG arrhythmia. The DistilBERT transformer model, pre-trained for natural language processing tasks, is used in the proposed work. The signals are denoised and then segmented around the R peak and oversampled to get a balanced dataset. The input embedding step is skipped, and only positional encoding is done. The final probabilities are obtained by adding a classification head to the transformer encoder output. The experiments on the MIT-BIH dataset show that the suggested model is excellent in classifying various arrhythmias. The model achieved 99.92% accuracy, 0.99 precision, sensitivity, and F1 score on the augmented dataset with a ROC-AUC score of 0.999.

Graphical Abstract

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Organization WH (2020) The top 10 causes of death https://www.who.int/news-room/fact-sheets/detail/the-top-10-causes-of-death. Accessed 10 Jan 2021

  2. Barrett PM et al (2014) Comparison of 24-hour holter monitoring with 14day novel adhesive patch electrocardiographic monitoring. Am J Med 127(1):95-e11

    Article  Google Scholar 

  3. da S Luz EJ, Schwartz WR, C’amara-Ch’avez G, Menotti D (2016) Ecg-based heartbeat classification for arrhythmia detection: a survey. Comput Methods Programs Biomed 127:144–164

    Article  Google Scholar 

  4. Afkhami RG, Azarnia G, Tinati MA (2016) Cardiac arrhythmia classification using statistical and mixture modeling features of ecg signals. Pattern Recogn Lett 70:45–51

    Article  Google Scholar 

  5. Xie J et al (2021) A signal quality assessment–based ecg waveform delineation method used for wearable monitoring systems. Med Biol Eng Compu 59(10):2073–2084

    Article  Google Scholar 

  6. Varghese A, Sylaja MM, Kurian J (2022) Conception and realization of an iot-enabled deep cnn decision support system for automated arrhythmia classification. J Intell Syst 31(1):407–419

    Google Scholar 

  7. Ebrahimi Z, Loni M, Daneshtalab M, Gharehbaghi A (2020) A review on deep learning methods for ecg arrhythmia classification. Expert Syst Appl: X 7:100033

    Google Scholar 

  8. Majid S et al (2022) Attention based cnn model for fire detection and localization in real-world images. Expert Syst Appl 189:116114

    Article  Google Scholar 

  9. Tomar NK, Jha D, Riegler MA, Johansen HD, Johansen D, Rittscher J, Halvorsen P, Ali S (2022) Fanet: A feedback attention network for improved biomedical image segmentation. IEEE Transactions on Neural Networks and Learning Systems. IEEE

  10. Zhang J et al (2020) Ecg-based multi-class arrhythmia detection using spatio-temporal attention-based convolutional recurrent neural network. Artif Intell Med 106:101856

    Article  PubMed  Google Scholar 

  11. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser L, Polosukhin I (2017) Attention is all you need. Adv Neural Inform Process Syst 30

  12. Li D, Li L, Bissyand´e TF, Klein J, and Le Traon Y (2016) Dsco: A language modeling approach for time series classification. International Conference on Machine Learning and Data Mining in Pattern Recognition. Springer pp 294–310

  13. Sanh V, Debut L, Chaumond J, and Wolf T (2019) Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108. Accessed 10 Oct 2020

  14. Devlin J, Chang M-W, Lee K, and Toutanova K (2018) Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805. Accessed 9 Sept 2020

  15. Moody GB, Mark RG (2001) The impact of the mit-bih arrhythmia database. IEEE Eng Med Biol Mag 20(3):45–50

    Article  CAS  PubMed  Google Scholar 

  16. Ahmad I, Ansari F, Dey U (2013) Power line noise reduction in ecg by butterworth notch filters: A comparative study. Development (IJECIERD) 3(3):65–74

    Google Scholar 

  17. He H, Garcia EA (2009) Learning from imbalanced data. IEEE Trans Knowl Data Eng 21(9):1263–1284

    Article  Google Scholar 

  18. Chawla NV, Bowyer KW, Hall LO, Kegelmeyer WP (2002) Smote: synthetic minority over-sampling technique. J Artif Intell Res 16:321–357

    Article  Google Scholar 

  19. Yan G, Liang S, Zhang Y, and Liu F (2019) Fusing transformer model with temporal features for ecg heartbeat classification. 2019 IEEE International Conference on Bioinformatics and Biomedicine (BIBM). IEEE, pp 898–905

  20. Guan J, Wang W, Feng P, Wang X, and Wang W (2021) Low-dimensional denoising embedding transformer for ecg classification. ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing(ICASSP). IEEE, pp 1285–1289

  21. Meng L et al (2022) Enhancing dynamic ecg heartbeat classification with lightweight transformer model. Artif Intell Med 124:102236

    Article  PubMed  Google Scholar 

  22. Che C, Zhang P, Zhu M, Qu Y, Jin B (2021) Constrained transformer network for ecg signal processing and arrhythmia classification. BMC Med Inform Decis Mak 21(1):1–13

    Article  Google Scholar 

  23. Mousavi S, Afghah F (2019) Inter-and intra-patient ecg heartbeat classification for arrhythmia detection: a sequence to sequence deep learning approach. ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, pp 1308–1312

  24. Yin Y et al (2021) An Algorithm for Locating PVC and SPB in Wearable ECGs. 2021 13th International Conference on Communication Software and Networks (ICCSN). IEEE, pp 89–93

  25. Li J, Si Y, Xu T, Jiang S (2018) Deep convolutional neural network based ecg classification system using information fusion and one-hot encoding techniques. Math Probl Eng 2018:1

    Google Scholar 

  26. Zhai X, Tin C (2018) Automated ecg classification using dual heartbeat coupling based on convolutional neural network. IEEE Access 6:27465–27472

    Article  Google Scholar 

  27. Kiranyaz S, Ince T, Gabbouj M (2017) Personalized monitoring and advance warning system for cardiac arrhythmias. Sci Rep 7(1):1–8

    Article  CAS  Google Scholar 

  28. ECAR, AAMI (1987) Recommended practice for testing and reporting performance results of ventricular arrhythmia detection algorithms. Assoc Adv Med Instrumen 69

Download references

Author information

Authors and Affiliations

  1. Department of Electronics, Cochin University of Science and Technology, Cochin, 682022, Kerala, India

    Ann Varghese, Suraj Kamal & James Kurian

Authors
  1. Ann Varghese
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Suraj Kamal
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. James Kurian
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ann Varghese.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Varghese, A., Kamal, S. & Kurian, J. Transformer-based temporal sequence learners for arrhythmia classification. Med Biol Eng Comput 61, 1993–2000 (2023). https://doi.org/10.1007/s11517-023-02858-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11517-023-02858-3

Keywords

Search

Navigation