Abstract
In this paper we propose and evaluate different models for speech emotion classification through audio signal processing, machine learning and deep learning techniques. For this purpose, we have collected from two databases (RAVDESS and TESS), a total of 5252 audio samples with 8 emotional classes (neutral, calm, happy, sad, angry, fearful, disgust and surprised). We have divided our experiments in 3 main stages. In the first stage, we have used feature engineering to extract relevant features from the time, spectral and cepstral domains. Features like ZCR, energy, spectral centroid, chroma, MFCC etc. were used to train a SVM classifier. The best model obtained an accuracy of 91.1%. In the second stage, we only have considered 40 MFCC coefficients for training several Deep Neural Networks such as CNN, LSTM and MLP were trained, the best model obtained an accuracy of 89.5% with an MLP architecture. Finally, for the third stage we have trained an end-to-end CNN network (SampleCNN) at the sample level. This last approach does not require features engineering, but directly the audio signal. In this stage, we achieve a precision of 81.7%. The experiments show that the results achieved are competitive and some experiments have surpassed in accuracy the related works.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Barragán Codina, J.N., Guerra Rodríguez, P., Villalpando Cadena, P.: La economía de la experiencia y el marketing emocional: estrategias contemporáneas de comercialización. Int. J. Good Conscience (2017). www.spentamexico.org/v12-n2/A9.12(2)159-170.pdf
Bisquera, R.: “Cómo educar las emociones” La inteligencia emocional en la infancia y adolescencia. Hospital Sant Joan de Déu, Barcelona (2012)
Basu, S., Chakraborty, J., Aftabuddin, M.: Emotion recognition from speech using convolutional neural network with recurrent neural network architecture. In: 2017 2nd International Conference on Communication and Electronics Systems (ICCES) (2017). https://doi.org/10.1109/CESYS.2017.8321292
Fernadez-Abascal, E.G., Garcia Rodriguez, B., Jiménez Sánchez, M.P., Martín Diaz, M.D., Dominguez Sanchez, F.J.: Psicología de la emoción, 2nd edn., pp. 18–20. Ramón Aceres, Madrid (2010)
Gordillo León, F., Arana Martínez, J.M., Salvador Cruz, J., Mestas Hernández, L.: Emoción y toma de decisiones: teoria y aplicación de IOWA Gambling Task. Revista Electrónica de Psicología Iztacala (2011)
Harár, P., Burget, R., Malay Dutta, K.: Speech emotion recognition with deep learning. In: 2017 4th International Conference on Signal Processing and Integrated Networks (SPIN) (2017). https://doi.org/10.1109/SPIN.2017.8049931
Iqbal, A., Barua, K.: A real-time emotion recognition from speech using gradient boosting. In: 2019 International Conference on Electrical, Computer and Communication Engineering (ECCE) (2019). https://doi.org/10.1109/ECACE.2019.8679271
Kerkeni, L., Serrestou, Y., Mbarki, M., Raoof, K., Mohamed Ali, M., Cleder, C.: Automatic speech emotion recognition using machine learning. Intechopen (2019). https://doi.org/10.5772/intechopen.84856
Lee, J., Park, J., Nam, J., Kim, K.L.: End-to-end deep convolutional neural networks using very small filters for music classification. In: 14th Sound and Music Computing Conference (2017). https://arxiv.org/abs/1703.01789
Livingstone, S.R., Russo, F.A.: The Ryerson audio-visual database of emotional speech and song (RAVDESS): a dynamic, multimodal set of facial and vocal expressions in North American English. PLoS One (2018). https://doi.org/10.1371/journal.pone.0196391
Oviedo Carrascal, E., Oviedo Carrascal, A., Velez Saldarriaga, G.: Minería multimedia: hacia la construcción de una metodología. Sci. Electron. Libr. Online (2016). https://doi.org/10.22395/rium.v16n31a6
Platero Ibañez, C.: Aplicaciones de la inteligencia emocional. Revista electrónica de investigación Docencia Creativa (2016). http://hdl.handle.net/10481/27761
Pichora-Fuller, M.K., Dupuis, K.: Toronto emotional speech set (TESS). Scholars Portal Dataverse (2010). https://doi.org/10.5683/SP2/E8H2MF
Semwal, N., Kumar, A., Narayanan, S.: Automatic speech emotion detection system using multi-domain acoustic feature selection and classification models. In: 2017 IEEE International Conference on Identity, Security and Behavior Analysis (ISBA) (2017). https://doi.org/10.1109/ISBA.2017.7947681
Wang, J., Han, Z.: Research on speech emotion recognition technology based on deep and shallow neural network. In: Chinese Control Conference (CCC) (2019). https://doi.org/10.23919/ChiCC.2019.8866568
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Melo Locumber, N., Fabian, J. (2021). Machine Learning Techniques for Speech Emotion Classification. In: Lossio-Ventura, J.A., Valverde-Rebaza, J.C., Díaz, E., Alatrista-Salas, H. (eds) Information Management and Big Data. SIMBig 2020. Communications in Computer and Information Science, vol 1410. Springer, Cham. https://doi.org/10.1007/978-3-030-76228-5_6
Download citation
DOI: https://doi.org/10.1007/978-3-030-76228-5_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-76227-8
Online ISBN: 978-3-030-76228-5
eBook Packages: Computer ScienceComputer Science (R0)