Skip to main content
SpringerLink
Log in
Book cover

International Conference on Big Data Analytics

BDA 2017: Big Data Analytics pp 179–194Cite as

Precision/Recall Trade-Off Analysis in Abnormal/Normal Heart Sound Classification

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 10721))

Abstract

Heart sound analysis is a preliminary procedure performed by a physician and involves examining the heart beats to detect the symptoms of cardiovascular diseases (CVDs). With recent developments in clinical science and the availability of devices to capture heart beats, researchers are now exploring the possibility of a machine assisted heart sound analysis system that can augment the clinical expertise of the physician in early detection of CVD. In this paper, we study the application of machine learning algorithms in classifying abnormal/normal heart sounds based on the short (\(\le \)120 s) audio phonocardiogram (PCG) recordings. To this end, we use the largest public audio PCG dataset released as part of the 2016 PhysioNet/Cardiology in Computing Challenge. The data comes from different patients, most of who have had no previous history of cardiac disease and some with known cardiac diseases. In our study, we use these audio recordings to train three different classification algorithms and discuss the effects of class imbalance (normal vs. abnormal) on the precision-recall trade-off of the prediction task. Specifically, our goal is to find a suitable model that takes into account the inherent imbalance and optimize the precision-recall trade-off with a higher emphasis on increasing recall. Bagged random forest models with majority (normal) class under sampling gave us the best configuration resulting in average recall over 91% with nearly 64% average precision.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    Henceforth, we only discuss the results using the random forest approach. Comparisons with the other two classifiers are presented in Sect. 6.

  2. 2.

    We emphasize that prediction of abnormality is made per recording, not per cycle, given a full recording’s multiple cycles together provide the signal for prediction.

  3. 3.

    The notion of accuracy used here is the same as in the 2016 CinC challenge where it is set to (recall+specificity)/2.

  4. 4.

    Even this may not be exact comparison because the numbers of folds were different.

References

  1. American Heart Association: Heart disease and stroke statistics (2017). At-a-glance. https://www.heart.org/idc/groups/ahamah-public/@wcm/@sop/@smd/documents/downloadable/ucm_491265.pdf

  2. Archer, K.J., Kimes, R.V.: Empirical characterization of random forest variable importance measures. Comput. Stat. Data Anal. 52(4), 2249–2260 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  3. Olshausen, B.A.: Aliasing. http://redwood.berkeley.edu/bruno/npb261/aliasing.pdf

  4. Cleveland Clinic: Heart and blood vessels: how does the heart beat. https://my.clevelandclinic.org/health/articles/heart-blood-vessels-heart-beat

  5. Clifford, G.D., Liu, C., Moody, B., Springer, D., Silva, I., Li, Q., Mark, R.G.: Classification of normal/abnormal heart sound recordings: the physionet/computing in cardiology challenge 2016. In: Computing in Cardiology Conference (CinC), pp. 609–612. IEEE (2016)

    Google Scholar 

  6. Goldberger, A.L., Amaral, L.A., Glass, L., Hausdorff, J.M., Ivanov, P.C., Mark, R.G., Mietus, J.E., Moody, G.B., Peng, C.-K., Stanley, H.E.: Physiobank, physiotoolkit, and physionet. Circulation 101(23), e215–e220 (2000)

    Article  Google Scholar 

  7. Hasan, M.R., Jamil, M., Rabbani, M.G., Rahman, M.S.: Speaker identification using mel frequency cepstral coefficients. In: 3rd International Conference on Electrical and Computer Engineering, pp. 565–568 (2004)

    Google Scholar 

  8. Liu, C., Springer, D., Li, Q., Moody, B., Juan, R.A., Chorro, F.J., Castells, F., Roig, J.M., Silva, I., Johnson, A.E., et al.: An open access database for the evaluation of heart sound algorithms. Physiol. Meas. 37(12), 2181 (2016)

    Article  Google Scholar 

  9. Oppenheim, A.V., Verghese, G.C.: Signals, Systems and Inference. Pearson, Boston (2015)

    Google Scholar 

  10. Ozenne, B., Subtil, F., Maucort-Boulch, D.: The precision-recall curve overcame the optimism of the receiver operating characteristic curve in rare diseases. J. Clin. Epidemiol. 68(8), 855–859 (2015)

    Article  Google Scholar 

  11. Potes, C., Parvaneh, S., Rahman, A., Conroy, B.: Ensemble of feature-based and deep learning-based classifiers for detection of abnormal heart sounds. In: Computing in Cardiology Conference (CinC), pp. 621–624. IEEE (2016)

    Google Scholar 

  12. Rubin, J., Abreu, R., Ganguli, A., Nelaturi, S., Matei, I., Sricharan, K.: Classifying heart sound recordings using deep convolutional neural networks and mel-frequency cepstral coefficients. In: Computing in Cardiology Conference (CinC), pp. 813–816. IEEE (2016)

    Google Scholar 

  13. Saito, T., Rehmsmeier, M.: The precision-recall plot is more informative than the roc plot when evaluating binary classifiers on imbalanced datasets. PLoS ONE 10(3), e0118432 (2015)

    Article  Google Scholar 

  14. Springer, D.B., Tarassenko, L., Clifford, G.D.: Logistic regression-hsmm-based heart sound segmentation. IEEE Trans. Biomed. Eng. 63(4), 822–832 (2016)

    Google Scholar 

  15. Wallace, B.C., Small, K., Brodley, C.E., Trikalinos, T.A.: Class imbalance, redux. In: 2011 IEEE 11th International Conference on Data Mining (ICDM), pp. 754–763. IEEE (2011)

    Google Scholar 

  16. Whitaker, B.M., Suresha, P.B., Liu, C., Clifford, G., Anderson, D.: Combining sparse coding and time-domain features for heart sound classification. Physiol. Meas. 38, 1701–1713 (2017)

    Article  Google Scholar 

  17. Zabihi, M., Rad, A.B., Kiranyaz, S., Gabbouj, M., Katsaggelos, A.K.: Heart sound anomaly and quality detection using ensemble of neural networks without segmentation. In: Computing in Cardiology Conference (CinC), pp. 613–616. IEEE (2016)

    Google Scholar 

Download references

Acknowledgements

We thank anonymous reviewers for their honest and constructive criticism of our paper. Our work is primarily supported by the National Library of Medicine through grant R21LM012274. We are also supported by the National Center for Advancing Translational Sciences through grant UL1TR001998. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH.

Author information

Authors and Affiliations

  1. Division of Biomedical Informatics, Department of Internal Medicine, University of Kentucky, Lexington, KY, USA

    Ramakanth Kavuluru

  2. Department of Computer Science, University of Kentucky, Lexington, KY, USA

    Jeevith Bopaiah & Ramakanth Kavuluru

Authors
  1. Jeevith Bopaiah
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ramakanth Kavuluru
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ramakanth Kavuluru .

Editor information

Editors and Affiliations

  1. International Institute of Information Technology, Hyderabad, India

    P. Krishna Reddy

  2. Rajiv Gandhi Education City, Sonepat, India

    Ashish Sureka

  3. University of Texas at Arlington, Arlington, Texas, USA

    Sharma Chakravarthy

  4. University of Aizu, Aizu-Wakamatsu, Japan

    Subhash Bhalla

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Bopaiah, J., Kavuluru, R. (2017). Precision/Recall Trade-Off Analysis in Abnormal/Normal Heart Sound Classification. In: Reddy, P., Sureka, A., Chakravarthy, S., Bhalla, S. (eds) Big Data Analytics. BDA 2017. Lecture Notes in Computer Science(), vol 10721. Springer, Cham. https://doi.org/10.1007/978-3-319-72413-3_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-72413-3_12

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-72412-6

  • Online ISBN: 978-3-319-72413-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation