Skip to main content
SpringerLink
Log in

Emotion classification using temporal and spectral features from IR-UWB-based respiration data

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Emotions play an important part in our daily lives since they influence our feelings and interactions. Because of the involvement of various response channels, emotion identification is a sophisticated and complex process. Emotion recognition has recently attracted the attention of the research industries as well as the scientific communities in the medical area. A variety of approaches have been explored to detect emotions from facial features, speech, text, and physiological signals. This study focuses on the classification of three basic emotions, i.e. happiness, disgust, and fear using a new set of spectral features extracted from the raw chest signal. We have used previously gathered data and biological signals (respiration rate). A structured dataset of 96810 records and 16 columns is maintained for experiments. The prior study’s accuracy was 76%, which served as the baseline. In this investigation, spectral and temporal features are used. Principal component analysis (PCA) is employed to select the best features. Extensive experiments are performed using several machine learning models for different scenarios of using all features, PCA-based selected features, and Chi-square selected features to analyze the efficacy of spectral features, as well as, feature selection approaches. Feature scaling is used to standardize the range of variables. Results indicate that using the PCA-based selective features, an accuracy of 98.52% can be obtained which is competitive with intrusive methods including electroencephalography and galvanic skin response signals while outperforming existing approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Data Availability

The dataset for current research is collected by the authors and is available on requrest.

References

  1. Abdi H, Williams LJ (2010) Principal component analysis. Wiley Interdisciplinary Reviews: Computational Statistics 2(4):433–459

    Article  Google Scholar 

  2. An T-K, Kim M-H (2010) A new diverse adaboost classifier. In: 2010 International conference on artificial intelligence and computational intelligence, vol 1. IEEE, pp 359–363

  3. Antoni J (2006) The spectral kurtosis: a useful tool for characterising non-stationary signals. Mech Syst Signal Process 20(2):282–307

    Article  Google Scholar 

  4. Aslan M (2021) Cnn based efficient approach for emotion recognition. Journal of King Saud University-Computer and Information Sciences

  5. Ayyadevara VK (2018) Gradient boosting machine. In: Pro machine learning algorithms. Springer, pp 117–134

  6. Beauchaine TP (2015) Respiratory sinus arrhythmia: a transdiagnostic biomarker of emotion dysregulation and psychopathology. Curr Opin Psychol 3:43–47

    Article  Google Scholar 

  7. Bhattacharyya A, Tripathy RK, Garg L, Pachori RB (2020) A novel multivariate-multiscale approach for computing eeg spectral and temporal complexity for human emotion recognition. IEEE Sensors J 21(3):3579–3591

    Article  Google Scholar 

  8. Brownlee J (2020) How to develop an extra trees ensemble with python. Machine Learning Mastery

  9. Dhieb N, Ghazzai H, Besbes H, Massoud Y (2019) Extreme gradient boosting machine learning algorithm for safe auto insurance operations. In: 2019 IEEE international conference on vehicular electronics and safety (ICVES). IEEE, pp 1–5

  10. Domínguez-Jiménez JA, Campo-Landines KC, Martínez-Santos JC, Delahoz EJ, Contreras-Ortiz SH (2020) A machine learning model for emotion recognition from physiological signals. Biomed Signal Process Control 55:101646

    Article  Google Scholar 

  11. He L, Lech M, Maddage NC, Allen NB (2011) Study of empirical mode decomposition and spectral analysis for stress and emotion classification in natural speech. Biomed Signal Process Control 6(2):139–146

    Article  Google Scholar 

  12. Holland SM (2008) Principal components analysis (pca). Department of Geology, University of Georgia, Athens, GA, pp 30602–2501

  13. Hosseinzadeh D, Krishnan S (2007) Combining vocal source and mfcc features for enhanced speaker recognition performance using gmms. in 2007. In: IEEE 9th Workshop on multimedia signal processing. IEEE, pp 365–368

  14. Jia Z, Lin Y, Cai X, Chen H, Gou H, Wang J (2020) Sst-emotionnet: spatial-spectral-temporal based attention 3d dense network for eeg emotion recognition. In: Proceedings of the 28th ACM international conference on multimedia, pp 2909–2917

  15. Kanjo E, Younis EM, Ang CS (2019) Deep learning analysis of mobile physiological, environmental and location sensor data for emotion detection. Inform Fus 49:46–56

    Article  Google Scholar 

  16. Khan AN, Ihalage AA, Ma Y, Liu B, Liu Y, Hao Y (2021) Deep learning framework for subject-independent emotion detection using wireless signals. Plos one 16(2):e0242946

    Article  Google Scholar 

  17. Kim J, André E (2008) Emotion recognition based on physiological changes in music listening. IEEE Trans Pattern Anal Mach Intell 30(12):2067–2083

    Article  Google Scholar 

  18. Krishna SR, Rajeswara R, Vizianagaram V (2017) Svm based emotion recognition using spectral features and pca. Int J Pure Appl Math 114 (9):227–235

    Google Scholar 

  19. Lalitha S, Mudupu A, Nandyala BV, Munagala R (2015) Speech emotion recognition using dwt. in 2015. In: IEEE International Conference on Computational Intelligence and Computing Research (ICCIC). IEEE, pp 1–4

  20. Mannepalli K, Sastry PN, Suman M (2017) A novel adaptive fractional deep belief networks for speaker emotion recognition. Alexandria Eng J 56(4):485–497

    Article  Google Scholar 

  21. Noble WS (2006) What is a support vector machine? Nat Biotechnol 24(12):1565–1567

    Article  Google Scholar 

  22. Peterson LE (2009) K-nearest neighbor. Scholarpedia 4(2):1883

    Article  Google Scholar 

  23. Picard RW, Vyzas E, Healey J (2001) Toward machine emotional intelligence: analysis of affective physiological state. IEEE Trans Pattern Anal Mach Intell 23(10):1175–1191

    Article  Google Scholar 

  24. Ramaiah VS, Rao RR (2016) Multi-speaker activity detection using zero crossing rate. In: 2016 International Conference on Communication and Signal Processing (ICCSP). IEEE, pp 0023–0026

  25. Saha DP, Martin TL, Knapp RB (2015) Towards incorporating affective feedback into context-aware intelligent environments. In: 2015 International conference on affective computing and intelligent interaction (ACII). IEEE, pp 49–55

  26. Salau AO, Jain S (2019) Feature extraction: a survey of the types, techniques, applications. In: 2019 International Conference on Signal Processing and Communication (ICSC). IEEE, pp 158–164

  27. Salau AO, Olowoyo TD, Akinola SO (2020) Accent classification of the three major nigerian indigenous languages using 1d cnn lstm network model. In: Advances in computational intelligence techniques. Springer, pp 1–16

  28. Santamaria-Granados L, Munoz-Organero M, Ramirez-Gonzalez G, Abdulhay E, Arunkumar N (2018) Using deep convolutional neural network for emotion detection on a physiological signals dataset (amigos). IEEE Access 7:57–67

    Article  Google Scholar 

  29. Scherer K (2005) What are emotions? And how can they be measured? Soc Sci Inform 44(4):695–729

    Article  Google Scholar 

  30. Siddiqui HUR, Saleem AA, Brown R, Bademci B, Lee E, Rustam F, Dudley S (2021) Non-invasive driver drowsiness detection system. Sensors 21 (14):4833

    Article  Google Scholar 

  31. Siddiqui HUR, Shahzad HF, Saleem AA, Khan Khakwani AB, Rustam F, Lee E, Ashraf I, Dudley S (2021) Respiration based non-invasive approach for emotion recognition using impulse radio ultra wide band radar and machine learning. Sensors 21(24):8336

    Article  Google Scholar 

  32. Tamarit L, Goudbeek M, Scherer K (2008) Spectral slope measurements in emotionally expressive speech. In: Proceedings of speech analysis and processing for knowledge discovery, pp 169–183

  33. Thaseen IS, Kumar CA (2017) Intrusion detection model using fusion of chi-square feature selection and multi class svm. Journal of King Saud University-Computer and Information Sciences 29(4):462–472

    Article  Google Scholar 

  34. Wickramasuriya DS, Tessmer MK, Faghih RT (2019) Facial expression-based emotion classification using electrocardiogram and respiration signals. In: 2019 IEEE Healthcare innovations and point of care technologies, (HI-POCT). IEEE, pp 9–12

  35. Wu S, Falk TH, Chan W-Y (2009) Automatic recognition of speech emotion using long-term spectro-temporal features. In: 2009 16th international conference on digital signal processing. IEEE, pp 1–6

  36. Yantorno RE, Krishnamachari KR, Lovekin JM, Benincasa DS, Wenndt SJ (2001) The spectral autocorrelation peak valley ratio (sapvr)-a usable speech measure employed as a co-channel detection system. In: Proceedings of IEEE International Workshop on Intelligent Signal Processing (WISP), vol 21

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, Khwaja Fareed University of Engineering and Information Technology, Rahim Yar Khan, 64200, Pakistan

    Hafeez Ur Rehman Siddiqui, Kainat Zafar, Adil Ali Saleem & Muhammad Amjad Raza

  2. School of Engineering and Design, London South Bank University, London, UK

    Sandra Dudley

  3. Department of Software Engineering, University of Management and Technology, Lahore, Pakistan

    Furqan Rustam

  4. Information and Communication Engineering, Yeungnam University, Gyeongsan, 38541, Korea

    Imran Ashraf

Authors
  1. Hafeez Ur Rehman Siddiqui
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kainat Zafar
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Adil Ali Saleem
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Muhammad Amjad Raza
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Sandra Dudley
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Furqan Rustam
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Imran Ashraf
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Imran Ashraf.

Ethics declarations

Conflict of Interests

The authors declare that they have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Siddiqui, H.U.R., Zafar, K., Saleem, A.A. et al. Emotion classification using temporal and spectral features from IR-UWB-based respiration data. Multimed Tools Appl 82, 18565–18583 (2023). https://doi.org/10.1007/s11042-022-14091-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-14091-5

Keywords

Search

Navigation