Skip to main content
SpringerLink
Log in

Comparison of Machine Learning Algorithms Using Manual/Automated Features on 12-Lead Signal Electrocardiogram Classification: A Large Cohort Study on Students Aged Between 6 to 18 Years Old

  • Original Article
  • Published:
Cardiovascular Engineering and Technology Aims and scope Submit manuscript

Abstract

Propose

An electrocardiogram (ECG) has been extensively used to detect rhythm disturbances. We sought to determine the accuracy of different machine learning in distinguishing abnormal ECGs from normal ones in children who were examined using a resting 12-Lead ECG machine, and we also compared the manual and automated measurement using the modular ECG Analysis System (MEANS) algorithm of ECG features.

Methods

Altogether, 10745 ECGs were recorded for students aged 6 to 18. Manual and automatic ECG features were extracted for each participant. Features were normalized using Z-score normalization and went through the student’s t-test and chi-squared test to measure their relevance. We applied the Boruta algorithm for feature selection and then implemented eight classifier algorithms. The dataset was split into training (80%) and test (20%) partitions. The performance of the classifiers was evaluated on the test data (unseen data) by 1000 bootstrap, and sensitivity (SEN), specificity (SPE), AUC, and accuracy (ACC) were reported.

Results

In univariate analysis, the highest performance was heart rate and RR interval in the manual dataset and heart rate in an automated dataset with AUC of 0.72 and 0.71, respectively. The best classifiers in the manual dataset were random forest (RF) and quadratic-discriminant-analysis (QDA) with AUC, ACC, SEN, and SPE equal to 0.93, 0.98, 0.69, 0.99, and 0.90, 0.95, 0.75, 0.96, respectively. In the automated dataset, QDA (AUC: 0.89, ACC:0.92, SEN:0.71, SPE:0.93) and stack learning (SL) (AUC:0.89, ACC:0.96, SEN:0.61, SPE:0.99) reached best performances.

Conclusion

This study demonstrated that the manual measurement of 12-Lead ECG features had better performance than the automated measurement (MEANS algorithm), but some classifiers had promising results in discriminating between normal and abnormal cases. Further studies can help us evaluate the applicability and efficacy of machine-learning approaches for distinguishing abnormal ECGs in community-based investigations in both adults and children.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Data Availability

The processed ECG data that support the findings of this study are available at Zenodo.org (link).

References

  1. Chow, G. V., J. E. Marine, and J. L. Fleg. Epidemiology of arrhythmias and conduction disorders in older adults. Clin Geriatr Med. 28:539–553, 2012.

    Article  PubMed  PubMed Central  Google Scholar 

  2. Benjamin, E. J., M. J. Blaha, S. E. Chiuve, M. Cushman, S. R. Das, R. Deo, S. D. D. Ferranti, J. Floyd, M. Fornage, C. Gillespie, C. R. Isasi, M. C. Jiménez, L. C. Jordan, S. E. Judd, D. Lackland, J. H. Lichtman, L. Lisabeth, S. Liu, C. T. Longenecker, R. H. Mackey, K. Matsushita, D. Mozaffarian, M. E. Mussolino, K. Nasir, R. W. Neumar, L. Palaniappan, D. K. Pandey, R. R. Thiagarajan, M. J. Reeves, M. Ritchey, C. J. Rodriguez, G. A. Roth, W. D. Rosamond, C. Sasson, A. Towfighi, C. W. Tsao, M. B. Turner, S. S. Virani, J. H. Voeks, J. Z. Willey, J. T. Wilkins, J. H. Wu, H. M. Alger, S. S. Wong, and P. Muntner. Heart disease and stroke statistics-2017 update: a report from the American Heart Association. Circulation. 135:146-e603, 2017.

    Article  Google Scholar 

  3. Sekar, R. P. Epidemiology of arrhythmias in children. Indian Pacing Electrophysiol J. 8:S8, 2008.

    PubMed  PubMed Central  Google Scholar 

  4. Santini, M., S. A. Di Fusco, F. Colivicchi, and A. Gargaro. Electrocardiographic characteristics, anthropometric features, and cardiovascular risk factors in a large cohort of adolescents. Europace. 20:1833–1840, 2018.

    Article  PubMed  PubMed Central  Google Scholar 

  5. Chiu, S. N., J. K. Wang, M. H. Wu, C. W. Chang, C. A. Chen, M. T. Lin, E. T. Wu, Y. C. Hua, and H. C. Lue. Cardiac conduction disturbance detected in a pediatric population. J Pediatr. 152:85–89, 2008.

    Article  PubMed  Google Scholar 

  6. Sox, H. C., Jr., A. M. Garber, and B. Littenberg. The resting electrocardiogram as a screening test. A clinical analysis. Ann. Internal Med. 111:489–502, 1989.

    Article  Google Scholar 

  7. Hu, Y. H., S. Palreddy, and W. J. Tompkins. A patient-adaptable ECG beat classifier using a mixture of experts approach. IEEE Trans Bio-med Eng. 44:891–900, 1997.

    Article  CAS  Google Scholar 

  8. J.C.T.B. Moraes, M.O. Seixas, F.N. Vilani, E.V. Costa, A real time QRS complex classification method using Mahalanobis distance, Computers in Cardiology, 2002, pp. 201-204.

  9. Gonzalez Corcia, M. C., J. Sieira, G. Pappaert, C. de Asmundis, G. B. Chierchia, M. La Meir, A. Sarkozy, and P. Brugada. Implantable cardioverter-defibrillators in children and adolescents with Brugada syndrome. J Am Coll Cardiol. 71:148–157, 2018.

    Article  PubMed  Google Scholar 

  10. G.D. Clifford, C. Liu, B. Moody, H.L. Li-wei, I. Silva, Q. Li, A. Johnson, R.G. Mark, AF classification from a short single lead ECG recording: The PhysioNet/computing in cardiology challenge 2017, 2017 Computing in Cardiology (CinC), IEEE, 2017, pp. 1-4.

  11. Van de Loo, A., W. Arendts, and S. H. Hohnloser. Variability of QT dispersion measurements in the surface electrocardiogram in patients with acute myocardial infarction and in normal subjects. Am J Cardiol. 74:1113–1118, 1994.

    Article  PubMed  Google Scholar 

  12. Massel, D. Observer variability in ECG interpretation for thrombolysis eligibility: experience and context matter. J Thromb Thrombolysis. 15:131–140, 2003.

    Article  PubMed  Google Scholar 

  13. Christov, I., and G. Bortolan. Ranking of pattern recognition parameters for premature ventricular contractions classification by neural networks. Physiol Meas. 25:1281–1290, 2004.

    Article  CAS  PubMed  Google Scholar 

  14. R. Singh, N. Rajpal, R. Mehta, Abnormality detection in ECG using hybrid feature extraction approach, 2018 First International Conference on Secure Cyber Computing and Communication (ICSCCC), IEEE, 2018, pp. 461-466.

  15. Moridani, M., M. A. Zadeh, and Z. S. Mazraeh. An efficient automated algorithm for distinguishing normal and abnormal ECG signal. IRBM. 40:332–340, 2019.

    Article  Google Scholar 

  16. Jadhav, S., S. Nalbalwar, and A. Ghatol. Feature elimination based random subspace ensembles learning for ECG arrhythmia diagnosis. Soft Computing. 18:579–587, 2014.

    Article  Google Scholar 

  17. Venkatesan, C., P. Karthigaikumar, A. Paul, S. Satheeskumaran, and R. Kumar. ECG signal preprocessing and SVM classifier-based abnormality detection in remote healthcare applications. IEEE Access. 6:9767–9773, 2018.

    Article  Google Scholar 

  18. P. Tadejko, W. Rakowski, Mathematical morphology based ECG feature extraction for the purpose of heartbeat classification, 6th International Conference on Computer Information Systems and Industrial Management Applications (CISIM'07), IEEE, 2007, pp. 322-327.

  19. Q. Zhao, L. Zhang, ECG feature extraction and classification using wavelet transform and support vector machines, 2005 International Conference on Neural Networks and Brain, IEEE, 2005, pp. 1089-1092.

  20. S. Mahmoodabadi, A. Ahmadian, M. Abolhasani, ECG feature extraction using Daubechies wavelets, Proceedings of the fifth IASTED International conference on Visualization, Imaging and Image Processing, 2005, pp. 343-348.

  21. Lin, H.-Y., S.-Y. Liang, Y.-L. Ho, Y.-H. Lin, and H.-P. Ma. Discrete-wavelet-transform-based noise removal and feature extraction for ECG signals. Irbm. 35:351–361, 2014.

    Article  Google Scholar 

  22. X. Xu, Y. Liu, ECG QRS complex detection using slope vector waveform (SVW) algorithm, The 26th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, IEEE, 2004, pp. 3597-3600.

  23. Jager, F. Feature extraction and shape representation of ambulatory electrocardiogram using the Karhunen-lòeve transform. Electrotechn. Rev. 69:83–89, 2002.

    Google Scholar 

  24. A. Ahmadian, S. Karimifard, H. Sadoughi, M. Abdoli, An efficient piecewise modeling of ECG signals based on Hermitian basis functions, 2007 29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, IEEE, 2007, pp. 3180-3183.

  25. Van Bemmel, J., J. Kors, and G. Van Herpen. Methods of information in medicine. Methods Inf Med. 29:346–353, 1990.

    PubMed  Google Scholar 

  26. Shiri, I., A. Sadr, M. Amini, Y. Salimi, A. Sanaat, A. Akhavan, B. Razeghi, S. Ferdowsi, A. Saberi, H. Arabi, M. Becker, S. Voloshynovskiy, D. Gündüz, A. Rahmim, and H. Zaidi. Decentralized distributed multi-institutional PET image segmentation using a federated deep learning framework. Clin Nucl Med. 50(4):1034–1050, 2022.

    Google Scholar 

  27. Mohebi, M., M. Amini, M. J. Alemzadeh-Ansari, A. Alizadehasl, A. B. Rajabi, I. Shiri, H. Zaidi, and M. Orooji. Post-revascularization ejection fraction prediction for patients undergoing percutaneous coronary intervention based on myocardial perfusion SPECT imaging radiomics: a preliminary machine learning study. J Digit Imaging. 2023. https://doi.org/10.1007/s10278-023-00820-1.

    Article  PubMed  PubMed Central  Google Scholar 

  28. Sabouri, M., G. Hajianfar, Z. Hosseini, M. Amini, M. Mohebi, T. Ghaedian, S. Madadi, F. Rastgou, M. Oveisi, A. Bitarafan Rajabi, I. Shiri, and H. Zaidi. Myocardial perfusion SPECT imaging radiomic features and machine learning algorithms for cardiac contractile pattern recognition. J Digit Imaging. 36:497–509, 2023.

    Article  PubMed  Google Scholar 

  29. Hajianfar, G., M. Sabouri, Y. Salimi, M. Amini, S. Bagheri, E. Jenabi, S. Hekmat, M. Maghsudi, Z. Mansouri, M. Khateri, M. Hosein Jamshidi, E. Jafari, A. Bitarafan Rajabi, M. Assadi, M. Oveisi, I. Shiri, and H. Zaidi. Artificial intelligence-based analysis of whole-body bone scintigraphy: the quest for the optimal deep learning algorithm and comparison with human observer performance. Z Med Phys. 2023. https://doi.org/10.1016/j.zemedi.2023.01.008.

    Article  PubMed  Google Scholar 

  30. Shiri, I., M. Amini, M. Nazari, G. Hajianfar, A. Haddadi Avval, H. Abdollahi, M. Oveisi, H. Arabi, A. Rahmim, and H. Zaidi. Impact of feature harmonization on radiogenomics analysis: prediction of EGFR and KRAS mutations from non-small cell lung cancer PET/CT images. Comput Biol Med. 142:105230, 2022.

    Article  CAS  PubMed  Google Scholar 

  31. Murat, F., O. Yildirim, M. Talo, U. B. Baloglu, Y. Demir, and U. R. Acharya. Application of deep learning techniques for heartbeats detection using ECG signals-analysis and review. Comput Biol Med. 120:103726, 2020.

    Article  PubMed  Google Scholar 

  32. C. Ye, M.T. Coimbra, B.V. Kumar, Arrhythmia detection and classification using morphological and dynamic features of ECG signals, 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology, IEEE, 2010, pp. 1918-1921.

  33. T. Tabassum, M. Islam, An approach of cardiac disease prediction by analyzing ECG signal, 2016 3rd International Conference on Electrical Engineering and Information Communication Technology (ICEEICT), IEEE, 2016, pp. 1-5.

  34. Hosseini, S., N. Samiei, A. Tabib, H. Bakhshandeh, Y. Rezaei, M. Parsaee, F. R. Ghader, M. Moradian, M. Shojaeifard, and Z. Khajali. Prevalence of structural heart diseases detected by handheld echocardiographic device in school-age children in Iran: the SHED LIGHT study. Glob Heart. 17(1):39, 2022.

    Article  PubMed  PubMed Central  Google Scholar 

  35. Sharma, S., J. A. Drezner, A. Baggish, M. Papadakis, M. G. Wilson, J. M. Prutkin, A. La Gerche, M. J. Ackerman, M. Borjesson, J. C. Salerno, I. M. Asif, D. S. Owens, E. H. Chung, M. S. Emery, V. F. Froelicher, H. Heidbuchel, C. Adamuz, C. A. Asplund, G. Cohen, K. G. Harmon, J. C. Marek, S. Molossi, J. Niebauer, H. F. Pelto, M. V. Perez, N. R. Riding, T. Saarel, C. M. Schmied, D. M. Shipon, R. Stein, V. L. Vetter, A. Pelliccia, and D. Corrado. International recommendations for electrocardiographic interpretation in athletes. J Am Coll Cardiol. 69:1057–1075, 2017.

    Article  PubMed  Google Scholar 

  36. Rautaharju, P. M., B. Surawicz, L. S. Gettes, J. J. Bailey, R. Childers, B. J. Deal, A. Gorgels, E. W. Hancock, M. Josephson, P. Kligfield, J. A. Kors, P. Macfarlane, J. W. Mason, D. M. Mirvis, P. Okin, O. Pahlm, G. van Herpen, G. S. Wagner, and H. Wellens. AHA/ACCF/HRS recommendations for the standardization and interpretation of the electrocardiogram: part IV: the ST segment, T and U waves, and the QT interval: a scientific statement from the American Heart Association Electrocardiography and Arrhythmias Committee, Council on Clinical Cardiology; the American College of Cardiology Foundation; and the Heart Rhythm Society: endorsed by the International Society for Computerized Electrocardiology. Circulation. 119:e241-250, 2009.

    Article  PubMed  Google Scholar 

  37. Maron, B. J., R. A. Friedman, P. Kligfield, B. D. Levine, S. Viskin, B. R. Chaitman, P. M. Okin, J. P. Saul, L. Salberg, G. F. Van Hare, E. Z. Soliman, J. Chen, G. P. Matherne, S. F. Bolling, M. J. Mitten, A. Caplan, G. J. Balady, and P. D. Thompson. Assessment of the 12-lead ECG as a screening test for detection of cardiovascular disease in healthy general populations of young people (12–25 Years of Age): a scientific statement from the American Heart Association and the American College of Cardiology. Circulation. 130:1303–1334, 2014.

    Article  PubMed  Google Scholar 

  38. Kligfield, P., L. S. Gettes, J. J. Bailey, R. Childers, B. J. Deal, E. W. Hancock, G. van Herpen, J. A. Kors, P. Macfarlane, D. M. Mirvis, O. Pahlm, P. Rautaharju, G. S. Wagner, M. Josephson, J. W. Mason, P. Okin, B. Surawicz, and H. Wellens. Recommendations for the standardization and interpretation of the electrocardiogram: part I: the electrocardiogram and its technology a scientific statement from the American Heart Association Electrocardiography and Arrhythmias Committee, Council on Clinical Cardiology; the American College of Cardiology Foundation; and the Heart Rhythm Society endorsed by the International Society for Computerized Electrocardiology. J Am Coll Cardiol. 49:1109–1127, 2007.

    Article  PubMed  Google Scholar 

  39. Hancock, E. W., B. J. Deal, D. M. Mirvis, P. Okin, P. Kligfield, L. S. Gettes, J. J. Bailey, R. Childers, A. Gorgels, M. Josephson, J. A. Kors, P. Macfarlane, J. W. Mason, O. Pahlm, P. M. Rautaharju, B. Surawicz, G. van Herpen, G. S. Wagner, and H. Wellens. AHA/ACCF/HRS recommendations for the standardization and interpretation of the electrocardiogram: part V: electrocardiogram changes associated with cardiac chamber hypertrophy: a scientific statement from the American Heart Association Electrocardiography and Arrhythmias Committee, Council on Clinical Cardiology; the American College of Cardiology Foundation; and the Heart Rhythm Society: endorsed by the International Society for Computerized Electrocardiology. Circulation. 119:e251-261, 2009.

    Article  PubMed  Google Scholar 

  40. Ackerman, M. J., S. G. Priori, S. Willems, C. Berul, R. Brugada, H. Calkins, A. J. Camm, P. T. Ellinor, M. Gollob, R. Hamilton, R. E. Hershberger, D. P. Judge, H. Le Marec, W. J. McKenna, E. Schulze-Bahr, C. Semsarian, J. A. Towbin, H. Watkins, A. Wilde, C. Wolpert, and D. P. Zipes. HRS/EHRA expert consensus statement on the state of genetic testing for the channelopathies and cardiomyopathies: this document was developed as a partnership between the Heart Rhythm Society (HRS) and the European Heart Rhythm Association (EHRA). Europace. 13:1077–1109, 2011.

    Article  PubMed  Google Scholar 

  41. Shrout, P. E., and J. L. Fleiss. Intraclass correlations: uses in assessing rater reliability. Psychol Bull. 86:420, 1979.

    Article  CAS  PubMed  Google Scholar 

  42. Bazett, H. C. An analysis of the time relations of electrocardiograms. Heart. 7:353–370, 1920.

    Google Scholar 

  43. Kursa, M. B., and W. R. Rudnicki. Feature selection with the Boruta package. J Stat Softw. 36:1–13, 2010.

    Article  Google Scholar 

  44. R.C. Team, R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. 2012, 2021.

  45. Ribeiro, A. H., M. H. Ribeiro, G. M. Paixão, D. M. Oliveira, P. R. Gomes, J. A. Canazart, M. P. Ferreira, C. R. Andersson, P. W. Macfarlane, and W. Meira Jr. Automatic diagnosis of the 12-lead ECG using a deep neural network. Nat Commun. 11:1–9, 2020.

    Google Scholar 

Download references

Acknowledgements

The study grant was mainly provided by the National Institute for Medical Research Development (NIMAD) under the identification number 962126. Rajaie Cardiovascular Medical and Research Center, Tehran, Iran, provided the other portion of funding. The SHED LIGHT investigators would like to thank those who helped us during the electrocardiographic evaluations, particularly Ms. Maryam Arvanesh, Mrs. Akram Iranshahi, Mrs. Akram Tabatabaee, and Mrs. Maryam Alibakhshi.

Author information

Authors and Affiliations

  1. Rajaie Cardiovascular Medical and Research Center, Iran University of Medical Sciences, Niayesh Highway, Valiasr Ave., Tehran, 19969111541, Iran

    Ghasem Hajianfar, Mohammadrafie Khorgami, Yousef Rezaei, Niloufar Samiei, Avisa Tabib & Bahareh Kazem Borji

  2. Division of Nuclear Medicine and Molecular Imaging, Geneva University Hospital, Geneva, Switzerland

    Mehdi Amini & Isaac Shiri

  3. Department of Biomedical Engineering and Medical Physics, Shahid Beheshti University of Medical Sciences, Tehran, Iran

    Mehdi Amini

  4. Cardiogenetic Research Center, Rajaie Cardiovascular, Medical and Research Center, Iran University of Medical Sciences, Tehran, Iran

    Samira Kalayinia

  5. Department of Cardiology, Inselspital, University of Bern, Bern, Switzerland

    Isaac Shiri

  6. Heart Valve Disease Research Center, Rajaie Cardiovascular Medical and Research Center, Iran University of Medical Sciences, Tehran, Iran

    Saeid Hosseini

  7. Department of Computer Science, University of British Columbia, Vancouver, BC, Canada

    Mehrdad Oveisi

  8. Behyan Clinic, Pardis New Town, Tehran, Iran

    Yousef Rezaei

Authors
  1. Ghasem Hajianfar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mohammadrafie Khorgami
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yousef Rezaei
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mehdi Amini
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Niloufar Samiei
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Avisa Tabib
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Bahareh Kazem Borji
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Samira Kalayinia
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Isaac Shiri
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Saeid Hosseini
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Mehrdad Oveisi
    View author publications

    You can also search for this author in PubMed Google Scholar

Consortia

The SHED LIGHT Investigators

Contributions

MO, SH, and MK conceptualized and administered the project, acquired funding, and designed the study. GH and IS designed and developed a model and performed the statistical analysis and figure generation. GH and YR drafted the manuscript. MK, YR, IS, SK, and MA interpreted and critically reviewed the results. IS, MA, and SK edited the manuscript. YR, GH, BKB, SK, and MA acquired and prepared data. NS, AT, and MK labeled data. All authors have read and approved the final manuscript.

Corresponding author

Correspondence to Mohammadrafie Khorgami.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Associate Editor Patrick M. Boyle oversaw the review of this article.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (XLSX 1780 kb)

Supplementary file2 (DOCX 16 kb)

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hajianfar, G., Khorgami, M., Rezaei, Y. et al. Comparison of Machine Learning Algorithms Using Manual/Automated Features on 12-Lead Signal Electrocardiogram Classification: A Large Cohort Study on Students Aged Between 6 to 18 Years Old. Cardiovasc Eng Tech 14, 786–800 (2023). https://doi.org/10.1007/s13239-023-00687-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13239-023-00687-x

Keywords

Search

Navigation