Skip to main content
SpringerLink
Log in

Multi-features Integration for Speech Emotion Recognition

  • Conference paper
  • First Online:
Pattern Recognition and Artificial Intelligence (ICPRAI 2020)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 12068))

  • 1448 Accesses

Abstract

Speech not only conveys the content information but also reveals the emotions of speakers. In order to achieve effective speech emotion recognition, a novel multi-features integration algorithm has been proposed. The statistical Mel frequency cepstrum coefficient (MFCC) features are directly evolved from the original speech. To further mine more useful information among statistical features, sparse groups are presented to extract the discriminative features. For enhancing the nonlinearity of features, we map features to nonlinear space to obtain nonlinear features by the orthogonal matrix. Multiple features integrated enable them to work for speech emotion recognition together. Extensive experiments comparison with state-of-the-art algorithms on CASIA dataset confirm that our algorithm can achieve effective and efficient speech emotion recognition. In addition, the analysis of different features indicates multi-features integration is superior than single type of features, where the MFCC features contribute greater in recognition accuracy and at the same time it also takes more time for features extraction.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 109.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 139.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Albornoz, E.M., Milone, D.H.: Emotion recognition in never-seen languages using a novel ensemble method with emotion profiles. IEEE Trans. Affect. Comput. 8(1), 43–53 (2017)

    Article  Google Scholar 

  2. Kim, J., André, E.: Emotion recognition based on physiological changes in music listening. IEEE Trans. Pattern Anal. Mach. Intell. 30(12), 2067–2083 (2008)

    Article  Google Scholar 

  3. Le, B. V., Lee, S.: Adaptive hierarchical emotion recognition from speech signal for human-robot communication. In: 2014 Tenth International Conference on Intelligent Information Hiding and Multimedia Signal Processing, Kitakyushu, pp. 807–810. IEEE (2014)

    Google Scholar 

  4. Tarunika, K., Pradeeba, R.B., Aruna, P.: Applying machine learning techniques for speech emotion recognition. In: 2018 9th International Conference on Computing, Communication and Networking Technologies (ICCCNT), Bangalore, pp. 1–5. IEEE (2018)

    Google Scholar 

  5. Li, H., Suen, C.Y.: Robust face recognition based on dynamic rank representation. Pattern Recogn. 60(12), 13–24 (2016)

    Article  Google Scholar 

  6. Sun, L., Fu, S., Wang, F.: Decision tree SVM model with Fisher feature selection for speech emotion recognition. EURASIP J. Audio Speech Music Process. 2019(1), 1–14 (2019). https://doi.org/10.1186/s13636-018-0145-5

    Article  Google Scholar 

  7. Sun, L.H., Zou, B., Fu, S., Chen, J., Wang, F.: Speech emotion recognition based on DNN-decision tree SVM model. Speech Commun. 115, 29–37 (2019)

    Article  Google Scholar 

  8. Schuller, B., et al.: Cross-corpus acoustic emotion recognition: variances and strategies. IEEE Trans. Affect. Comput. 1(2), 119–131 (2011)

    Article  Google Scholar 

  9. Wang, K., An, N., Li, B.N., Zhang, Y., Li, L.: Speech emotion recognition using Fourier parameters. IEEE Trans. Affect. Comput. 6(1), 69–75 (2015)

    Article  Google Scholar 

  10. Xu, X.Z., et al.: A two-dimensional framework of multiple kernel subspace learning for recognizing emotion in speech. IEEE-ACM Trans. Audio Speech Lang. Process. 25(7), 1436–1449 (2017)

    Article  Google Scholar 

  11. Alvarez, A., et al.: Feature subset selection based on evolutionary algorithms for automatic emotion recognition in spoken Spanish and standard Basque language. In: 9th International Conference on Text, Speech and Dialogue, Brno, Czech Republic, pp. 565–572 (2006)

    Google Scholar 

  12. Morrison, D., Wang, R., Silva, L.C.D.: Ensemble methods for spoken emotion recognition in call-centres. Speech Commun. 49, 98–112 (2007)

    Article  Google Scholar 

  13. Pao, T.L., Liao, W.Y., Chen, Y.T., Yeh, J.H.: Comparison of several classifiers for emotion recognition from noisy Mandarin speech. In: 3rd International Conference on Intelligent Information Hiding and Multimedia Signal Processing, pp. 23–26. IEEE, Kaohsiung, TaiWan (2007)

    Google Scholar 

  14. Shahin, I., Nassif, A.B., Hamsa, S.: Emotion recognition using hybrid Gaussian mixture model and deep neural network. IEEE Access 7, 26777–26787 (2019)

    Article  Google Scholar 

  15. Li, H.J., Suen, C.Y.: A novel non-local means image denoising method based on grey theory. Pattern Recogn. 49(1), 217–248 (2016)

    Article  Google Scholar 

  16. Absa, A.H.A., Deriche, M.: A two-stage hierarchical bilingual emotion recognition system using a hidden Markov model and neural networks. Arab. J. Sci. Eng. 42(12), 5231–5249 (2017)

    Article  MathSciNet  Google Scholar 

  17. Nogueiras, A., Moreno, A., Bonafonte, A., Mariño, J.B.: Speech emotion recognition using hidden Markov models. In: 7th European Conference on Speech Communication and Technology, pp. 2679–2682 (2001)

    Google Scholar 

  18. Hongjun, L., Ze, Z., Chaobo, L., Shibing, Z.: RCCM: reinforce cycle cascade model for image recognition. IEEE Access 8, 15369–15376 (2020)

    Article  Google Scholar 

  19. Hifny,Y., Ali, A.: Efficient arabic emotion recognition using deep neural networks. In: 44th IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brighton, United Kingdom, pp. 6710–6714. IEEE (2019)

    Google Scholar 

  20. Desai, S., Raghavendra, E.V., Yegnanarayana, B., Black, A.W., Prahallad, K.: Voice conversion using artificial neural networks. In: IEEE International Conference on Acoustics, Speech, and Signal Processing, pp. 3893–3896 (2009)

    Google Scholar 

  21. Zhao, J., Mao, X., Chen, L.: Learning deep features to recognise speech emotion using merged deep CNN. IET Sig. Proc. 12(6), 713–721 (2018)

    Article  Google Scholar 

  22. Neumann, M., Vu, N.T.: Attentive convolutional neural network based Speech emotion recognition: a study on the impact of input features, signal length, and acted speech. In: 18th Annu. Conf. Int. Speech Commun. Assoc, pp. 1263–1267 (2017)

    Google Scholar 

  23. Xie, Y., Liang, R., Liang, Z., Huang, C., Zou, C., Schuller, B.: Speech emotion classification using attention-based LSTM. IEEE-ACM Trans. Audio Speech Lang. Process. 27(11), 1675–1685 (2019)

    Article  Google Scholar 

  24. Greff, K., Srivastava, R.K., Koutník, J., Steunebrink, B.R., Schmidhuber, J.: LSTM: a search space odyssey. IEEE Trans. Neural Netw. Learn. Syst. 28(10), 2222–2232 (2017)

    Article  MathSciNet  Google Scholar 

  25. Zhang, T., Wu, J.: Speech emotion recognition with i-vector feature and RNN model. In: 2015 IEEE China Summit and International Conference on Signal and Information Processing (ChinaSIP), Chengdu, pp. 524–528. IEEE (2015)

    Google Scholar 

  26. Keren G., Schuller, B.: Convolutional RNN: an enhanced model for extracting features from sequential data. In: 2016 International Joint Conference on Neural Networks (IJCNN), Vancouver, BC, Canada, pp. 3412–3419 (2016)

    Google Scholar 

  27. Ren, M.J., Nie, W.Z., Liu, A.A., Su, Y.T.: Multi-modal correlated network for emotion recognition in speech (in press)

    Google Scholar 

  28. Nwe, T.L., Foo, S.W., De Silva, L.C.: Speech emotion recognition using hidden Markov models. Speech Commun. 41(4), 603–623 (2003)

    Article  Google Scholar 

  29. Chen, L.F., Su, W.J., Feng, Y., Wu, M., She, J.H., Hirota, K.: Two-layer fuzzy multiple random forest for speech emotion recognition in human-robot interaction. Inf. Sci. 509, 150–163 (2020)

    Article  Google Scholar 

  30. Liu, Z.T., Wu, M., Cai, W.H., Mao, J.W., Xu, J.P., Tan, G.Z.: Speech emotion recognition based on feature selection and extreme learning machine decision tree. Neurocomputing 273(17), 271–280 (2018)

    Article  Google Scholar 

  31. Chen, L.J., Mao, X., Xue, Y.L., Cheng, L.L.: Speech emotion recognition: features and classification models. Digit. Sig. Proc. 22(6), 1154–1160 (2012)

    Article  MathSciNet  Google Scholar 

  32. Pan, Y.X., Shen, P.P., Shen, L.P.: Speech emotion recognition using support vector machine. Int. J. Smart Home 6(2), 101–108 (2012)

    Google Scholar 

  33. Murty, K.S.R., Yegnanarayana, B.: Combining evidence from residual phase and MFCC features for speaker recognition. IEEE Sig. Process. Lett. 13(1), 52–55 (2006)

    Article  Google Scholar 

  34. Kameoka, H., Ono, N., Sagayama, S.: Speech spectrum modeling for joint estimation of spectral envelope and fundamental frequency. IEEE Trans. Audio Speech Lang. Process. 18(6), 1507–1516 (2010)

    Article  Google Scholar 

  35. Levine, A.: The use of LPC coefficient interpolation to improve the naturalness of sentence formation by word concatenation. J. Acoust. Soc. Am. 66, 1947–1948 (1979)

    Article  Google Scholar 

  36. Zapata, J.G., Díaz Martín, J.C., Vilda, P.G.: Fast formant estimation by complex analysis of LPC coefficients. In: 2004 12th European Signal Processing Conference, Vienna, pp. 737–740. IEEE (2004)

    Google Scholar 

  37. Mundodu, P.K., Ramaswamy, K.: Single channel speech separation based on empirical mode decomposition and Hilbert Transform. IET Signal Proc. 11(5), 579–586 (2017)

    Article  Google Scholar 

  38. Björn, W.: Speech emotion recognition: two decades in a nutshell, benchmarks, and ongoing trends. Commun. ACM 61(5), 90–99 (2018)

    Article  Google Scholar 

  39. Tao, J., Liu, F., Zhang, M., Jia, H.: Design of speech corpus for mandarin text to speech, pp. 1–4 (2018)

    Google Scholar 

Download references

Acknowledgment

This work is supported by National Natural Science Foundation of China (NO. 61871241, NO. 61976120); Ministry of education cooperation in production and education (NO.201802302115, NO. 201901009007, NO. 201901009044); Educational Science Research Subject of China Transportation Education Research Association (Jiaotong Education Research 1802-118); the Science and Technology Program of Nantong (JC2018025, JC2018129); Nantong University-Nantong Joint Research Center for Intelligent Information Technology (KFKT2017B04); Nanjing University State Key Lab. for Novel Software Technology (KFKT2019B15); Postgraduate Research and Practice Innovation Program of Jiangsu Province (KYCX19_2056).

Author information

Authors and Affiliations

  1. School of Information Science and Technology, Nantong University, Nantong, 226019, China

    Hongjun Li, Ze Zhou, Xiaohu Sun & Chaobo Li

  2. State Key Lab, Novel Software Technology, Nanjing University, Nanjing, 210023, China

    Hongjun Li

  3. Nantong Research Institute, Advanced Communication Technologies, Nantong, 226019, China

    Hongjun Li

  4. TONGKE School of Microelectronics, Nantong, 226019, China

    Hongjun Li

Authors
  1. Hongjun Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ze Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiaohu Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chaobo Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hongjun Li .

Editor information

Editors and Affiliations

  1. East China Normal University, Shanghai, China

    Yue Lu

  2. Paris Descartes University, Paris, France

    Nicole Vincent

  3. Hong Kong Baptist University, Kowloon, Hong Kong

    Pong Chi Yuen

  4. Sun Yat-sen University, Guangzhou, China

    Wei-Shi Zheng

  5. Polytechnique Montréal, Montreal, QC, Canada

    Farida Cheriet

  6. Concordia University, Montreal, QC, Canada

    Ching Y. Suen

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Li, H., Zhou, Z., Sun, X., Li, C. (2020). Multi-features Integration for Speech Emotion Recognition. In: Lu, Y., Vincent, N., Yuen, P.C., Zheng, WS., Cheriet, F., Suen, C.Y. (eds) Pattern Recognition and Artificial Intelligence. ICPRAI 2020. Lecture Notes in Computer Science(), vol 12068. Springer, Cham. https://doi.org/10.1007/978-3-030-59830-3_17

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-59830-3_17

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-59829-7

  • Online ISBN: 978-3-030-59830-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation