Skip to main content
SpringerLink
Log in

An Evaluation of Physiological Public Datasets for Emotion Recognition Systems

  • Conference paper
  • First Online:
Information Management and Big Data (SIMBig 2020)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1410))

Included in the following conference series:

Abstract

[Background] The performance of emotion recognition systems depends heavily on datasets used in their training, validation, or testing stages. [Aims] This research aims to evaluate the extent to which public available physiological datasets created for emotion recognition systems meet a set of reference requirements. [Method] Firstly, we analyze the applicability of some reference requirements proposed for stress datasets and adjust the corresponding evaluation criteria. Secondly, nine public physiological datasets were identified from a previous survey. [Results] None of the evaluated datasets satisfy all the reference requirements in order to be considered as a reference dataset for being used in the construction of reliable emotion recognition systems. [Conclusion] Although the evaluated datasets do not support the whole reference requirements, they provide a baseline for further development. Also, a greater effort is needed to establish specific reference requirements that can appropriately guide the creation of physiological datasets for emotion recognition systems.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Bota, P.J., Wang, C., Fred, A.L., Da Silva, H.P.: A review, current challenges, and future possibilities on emotion recognition using machine learning and physiological signals. IEEE Access 7, 140990–141020 (2019)

    Article  Google Scholar 

  2. Carvalho, S., Leite, J., Galdo-Álvarez, S., Gonçalves, Ó.F.: The emotional movie database (EMDB): a self-report and psychophysiological study. Appl. Psychophysiol. Biofeedback 37(4), 279–294 (2012). https://doi.org/10.1007/s10484-012-9201-6

    Article  Google Scholar 

  3. Correa, J.A.M., Abadi, M.K., Sebe, N., Patras, I.: AMIGOS: a dataset for affect, personality and mood research on individuals and groups. IEEE Trans. Affect. Comput. (2018)

    Google Scholar 

  4. Ebner, N.C., Fischer, H.: Emotion and aging: evidence from brain and behavior. Front. Psychol. 5, 996 (2014)

    Google Scholar 

  5. Fischer, A.H., Rodriguez Mosquera, P.M., Van Vianen, A.E., Manstead, A.S.: Gender and culture differences in emotion. Emotion 4(1), 87 (2004)

    Article  Google Scholar 

  6. Gross, J.J., Carstensen, L.L., Pasupathi, M., Tsai, J., Götestam Skorpen, C., Hsu, A.Y.: Emotion and aging: experience, expression, and control. Psychol. Aging 12(4), 590 (1997)

    Article  Google Scholar 

  7. Healey, J.A., Picard, R.W.: Detecting stress during real-world driving tasks using physiological sensors. IEEE Trans. Intell. Transp. Syst. 6(2), 156–166 (2005)

    Article  Google Scholar 

  8. Igual, R., Medrano, C., Plaza, I.: A comparison of public datasets for acceleration-based fall detection. Med. Eng. Phys. 37(9), 870–878 (2015)

    Article  Google Scholar 

  9. Jang, E.H., Park, B.J., Park, M.S., Kim, S.H., Sohn, J.H.: Analysis of physiological signals for recognition of boredom, pain, and surprise emotions. J. Physiol. Anthropol. 34(1), 25 (2015)

    Article  Google Scholar 

  10. Koelstra, S., et al.: DEAP: a database for emotion analysis; using physiological signals. IEEE Trans. Affect. Comput. 3(1), 18–31 (2011)

    Article  Google Scholar 

  11. Kreibig, S.D.: Autonomic nervous system activity in emotion: a review. Biol. Psychol. 84(3), 394–421 (2010)

    Article  Google Scholar 

  12. Kring, A.M., Gordon, A.H.: Sex differences in emotion: expression, experience, and physiology. J. Pers. Soc. Psychol. 74(3), 686 (1998)

    Article  Google Scholar 

  13. Lazarus, R.S.: Stress and Emotion: A New Synthesis. Springer, New York (2006)

    Google Scholar 

  14. Lim, N.: Cultural differences in emotion: differences in emotional arousal level between the east and the west. Integr. Med. Res. 5(2), 105–109 (2016)

    Article  Google Scholar 

  15. Ma, K., Wang, X., Yang, X., Zhang, M., Girard, J.M., Morency, L.P.: ElderReact: a multimodal dataset for recognizing emotional response in aging adults. In: 2019 International Conference on Multimodal Interaction, ICMI 2019, pp. 349–357. Association for Computing Machinery, New York (2019)

    Google Scholar 

  16. Mahesh, B., Prassler, E., Hassan, T., Garbas, J.U.: Requirements for a reference dataset for multimodal human stress detection. In: 2019 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops), pp. 492–498. IEEE (2019)

    Google Scholar 

  17. Marechal, C., et al.: Survey on AI-based multimodal methods for emotion detection. In: Kołodziej, J., González-Vélez, H. (eds.) High-Performance Modelling and Simulation for Big Data Applications. LNCS, vol. 11400, pp. 307–324. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-16272-6_11

    Chapter  Google Scholar 

  18. Picard, R.W., Vyzas, E., Healey, J.: Toward machine emotional intelligence: analysis of affective physiological state. IEEE Trans. Pattern Anal. Mach. Intell. 23(10), 1175–1191 (2001)

    Article  Google Scholar 

  19. Pujol, F.A., Mora, H., Martínez, A.: Emotion recognition to improve e-healthcare systems in smart cities. In: Visvizi, A., Lytras, M.D. (eds.) RIIFORUM 2019. SPC, pp. 245–254. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-30809-4_23

    Chapter  Google Scholar 

  20. Rukavina, S., Gruss, S., Hoffmann, H., Tan, J.W., Walter, S., Traue, H.C.: Affective computing and the impact of gender and age. PLoS ONE 11(3), e0150584 (2016)

    Article  Google Scholar 

  21. Sajjad, M., Nasir, M., Ullah, F.U.M., Muhammad, K., Sangaiah, A.K., Baik, S.W.: Raspberry Pi assisted facial expression recognition framework for smart security in law-enforcement services. Inf. Sci. 479, 416–431 (2019)

    Article  Google Scholar 

  22. Scherer, K.R., Moors, A.: The emotion process: event appraisal and component differentiation. Ann. Rev. Psychol. 70, 719–745 (2019)

    Article  Google Scholar 

  23. Schmidt, P., Reiss, A., Duerichen, R., Marberger, C., Van Laerhoven, K.: Introducing WESAD, a multimodal dataset for wearable stress and affect detection. In: Proceedings of the 20th ACM International Conference on Multimodal Interaction, pp. 400–408 (2018)

    Google Scholar 

  24. Schneegass, S., Pfleging, B., Broy, N., Heinrich, F., Schmidt, A.: A data set of real world driving to assess driver workload. In: Proceedings of the 5th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, pp. 150–157 (2013)

    Google Scholar 

  25. Shu, L., et al.: A review of emotion recognition using physiological signals. Sensors 18(7), 2074 (2018)

    Article  Google Scholar 

  26. Siedlecka, E., Denson, T.F.: Experimental methods for inducing basic emotions: a qualitative review. Emot. Rev. 11(1), 87–97 (2019)

    Article  Google Scholar 

  27. Soleymani, M., Lichtenauer, J., Pun, T., Pantic, M.: A multimodal database for affect recognition and implicit tagging. IEEE Trans. Affect. Comput. 3(1), 42–55 (2011)

    Article  Google Scholar 

  28. Subramanian, R., Wache, J., Abadi, M.K., Vieriu, R.L., Winkler, S., Sebe, N.: ASCERTAIN: emotion and personality recognition using commercial sensors. IEEE Trans. Affect. Comput. 9(2), 147–160 (2016)

    Article  Google Scholar 

  29. Suni Lopez, F., Condori-Fernandez, N.: Design of an adaptive persuasive mobile application for stimulating the medication adherence. In: Poppe, R., Meyer, J.-J., Veltkamp, R., Dastani, M. (eds.) INTETAIN 2016 2016. LNICST, vol. 178, pp. 99–105. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-49616-0_9

    Chapter  Google Scholar 

  30. Tahir, Z., Alexander, R.: Coverage based testing for V&V and safety assurance of self-driving autonomous vehicle: a systematic literature review. In: The Second IEEE International Conference on Artificial Intelligence Testing, York (2020)

    Google Scholar 

  31. Wilkinson, M.D., et al.: The FAIR guiding principles for scientific data management and stewardship. Sci. Data 3 (2016)

    Google Scholar 

  32. Zepf, S., Hernandez, J., Schmitt, A., Minker, W., Picard, R.: Driver emotion recognition for intelligent vehicles: a survey. ACM Comput. Surv. (2020). https://doi.org/10.1145/3388790

  33. Zhang, J., Yin, Z., Chen, P., Nichele, S.: Emotion recognition using multi-modal data and machine learning techniques: a tutorial and review. Inf. Fusion 59, 103–126 (2020)

    Article  Google Scholar 

  34. Zhou, X., Jin, Y., Zhang, H., Li, S., Huang, X.: A map of threats to validity of systematic literature reviews in software engineering. In: 2016 23rd Asia-Pacific Software Engineering Conference (APSEC), pp. 153–160 (2016)

    Google Scholar 

Download references

Acknowledgment

A. Mendoza, A. Cuno, N. Condori-Fernandez and W. Ramos acknowledge financial support from the “Proyecto Concytec - Banco Mundial, Mejoramiento y Ampliación de los Servicios del Sistema Nacional de Ciencia Tecnología e Innovación Tecnológica” 8682-PE, through its executing unit FONDECYT [Contract N\(^\circ \) 014-2019-FONDECYT-BM-INC.INV]. Also, this work has been partially supported by Datos 4.0 (TIN2016-78011-C4-1-R) funded by MINECO-AEI/FEDER-UE.

Author information

Authors and Affiliations

  1. Departamento de Ingeniería de Sistemas e Informática, Universidad Nacional de San Agustín de Arequipa, Arequipa, Peru

    Alexis Mendoza, Alvaro Cuno, Nelly Condori-Fernandez & Wilber Ramos Lovón

  2. Universidad de Coruña, A Coruña, Spain

    Nelly Condori-Fernandez

  3. Vrije Universiteit Amsterdam, Amsterdam, The Netherlands

    Nelly Condori-Fernandez

Authors
  1. Alexis Mendoza
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alvaro Cuno
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Nelly Condori-Fernandez
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Wilber Ramos Lovón
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Nelly Condori-Fernandez or Wilber Ramos Lovón .

Editor information

Editors and Affiliations

  1. Stanford University, Stanford, CA, USA

    Juan Antonio Lossio-Ventura

  2. Visibilia, São Paulo, Brazil

    Jorge Carlos Valverde-Rebaza

  3. University of Valencia, Valencia, Spain

    Eduardo Díaz

  4. Universidad del Pacífico, Lima, Peru

    Hugo Alatrista-Salas

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Mendoza, A., Cuno, A., Condori-Fernandez, N., Lovón, W.R. (2021). An Evaluation of Physiological Public Datasets for Emotion Recognition Systems. In: Lossio-Ventura, J.A., Valverde-Rebaza, J.C., Díaz, E., Alatrista-Salas, H. (eds) Information Management and Big Data. SIMBig 2020. Communications in Computer and Information Science, vol 1410. Springer, Cham. https://doi.org/10.1007/978-3-030-76228-5_7

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-76228-5_7

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-76227-8

  • Online ISBN: 978-3-030-76228-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation