Skip to main content

Real-Life Validation of Emotion Detection System with Wearables

Part of the Lecture Notes in Computer Science book series (LNCS,volume 13259)

Abstract

Emotion recognition in real life is challenging since training machine learning models requires many annotated samples with experienced emotions. Although collecting such data is a difficult task, we may improve the process by utilizing a pre-trained model detecting emotional events. We conducted a study to test whether employing machine learning models that detect intense emotions to trigger self-assessments collects more data than triggering self-reports randomly. We have examined the performance of three models on 13 participants for three months. Results show that our models enhance the data collection and provide on average 21% more emotionally annotated data in the general setup. The personalized model improves the collection even more – by up to 38%.

Keywords

This work was partially supported by the National Science Centre, Poland, project no. 2020/37/B/ST6/03806; by the statutory funds of the Department of Artificial Intelligence, Wroclaw University of Science and Technology; by the Polish Ministry of Education and Science – the CLARIN-PL Project.

This is a preview of subscription content, log in via an institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Change history

  • 24 May 2022

    In an older version of this paper, there was an error in the cited reference no. 12. This has been corrected.

References

  1. Barrett, L.F.: How Emotions Are Made: The Secret Life of the Brain. Pan Macmillan (2017)

    Google Scholar 

  2. Dao, M.S., Dang-Nguyen, D.T., Kasem, A., Tran-The, H.: Healthyclassroom - a proof-of-concept study for discovering students’ daily moods and classroom emotions to enhance a learning-teaching process using heterogeneous sensors. In: ICPRAM (2018)

    Google Scholar 

  3. Dzieżyc, M., et al.: How to catch them all? Enhanced data collection for emotion recognition in the field. In: PerCom 2021, pp. 348–351. IEEE (2021)

    Google Scholar 

  4. Emognition system demo (2022). https://youtu.be/zgJw4krZ5tU. Accessed 25 Feb 2022

  5. Exler, A., Schankin, A., Klebsattel, C., Beigl, M.: A wearable system for mood assessment considering smartphone features and data from mobile ECGs. In: UbiComp 2016, pp. 1153–1161. ACM (2016)

    Google Scholar 

  6. Harper, R., Southern, J.: A Bayesian deep learning framework for end-to-end prediction of emotion from heartbeat. IEEE Trans. Affec. Comput. (2020)

    Google Scholar 

  7. Hernandez, J., Morris, R.R., Picard, R.W.: Call center stress recognition with person-specific models. In: D’Mello, S., Graesser, A., Schuller, B., Martin, J.-C. (eds.) ACII 2011. LNCS, vol. 6974, pp. 125–134. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-24600-5_16

    Chapter  Google Scholar 

  8. Koelstra, S., et al.: DEAP: a database for emotion analysis; using physiological signals. IEEE Trans. Affec. Comput. 3(1), 18–31 (2011)

    Article  Google Scholar 

  9. Larradet, F., Niewiadomski, R., Barresi, G., Caldwell, D.G., Mattos, L.S.: Toward emotion recognition from physiological signals in the wild: approaching the methodological issues in real-life data collection. Front. Psychol., 1111 (2020)

    Google Scholar 

  10. Miranda-Correa, J.A., Abadi, M.K., Sebe, N., Patras, I.: AMIGOS: a dataset for affect, personality and mood research on individuals and groups. IEEE Trans. Affec. Comput. 12(2), 479–493 (2018)

    Article  Google Scholar 

  11. Romeo, L., Cavallo, A., Pepa, L., Berthouze, N., Pontil, M.: Multiple instance learning for emotion recognition using physiological signals. IEEE Trans. Affec. Comput. (2019)

    Google Scholar 

  12. Saganowski, S., et al.: Emognition dataset: emotion recognition with self-reports, facial expressions, and physiology using wearables. Sci. Data 9 (2022). Article No. 158

    Google Scholar 

  13. Saganowski, S., Behnke, M., Komoszyńska, J., Kunc, D., Perz, B., Kazienko, P.: A system for collecting emotionally annotated physiological signals in daily life using wearables. In: ACIIW 2021, pp. 1–3. IEEE (2021)

    Google Scholar 

  14. Saganowski, S., Kunc, D., Perz, B., Komoszyńska, J., Behnke, M., Kazienko, P.: The cold start problem and per-group personalization in real-life emotion recognition with wearables. In: WristSense at PerCom 2022. IEEE (2022, in press)

    Google Scholar 

  15. Santamaria-Granados, L., Munoz-Organero, M., Ramirez-Gonzalez, G., Abdulhay, E., Arunkumar, N.: Using deep convolutional neural network for emotion detection on a physiological signals dataset (AMIGOS). IEEE Access 7, 57–67 (2018)

    Article  Google Scholar 

  16. Schmidt, P., Dürichen, R., Reiss, A., Van Laerhoven, K., Plötz, T.: Multi-target affect detection in the wild: an exploratory study. In: ISWC 2019, pp. 211–219 (2019)

    Google Scholar 

  17. Schmidt, P., Reiss, A., Dürichen, R., Van Laerhoven, K.: UbiComp 2018 (2018)

    Google Scholar 

  18. Shui, X., Zhang, M., Li, Z., Hu, X., Wang, F., Zhang, D.: A dataset of daily ambulatory psychological and physiological recording for emotion research. Sci. Data 8(1), 1–12 (2021)

    Article  Google Scholar 

  19. Siddharth, S., Jung, T.P., Sejnowski, T.J.: Utilizing deep learning towards multi-modal bio-sensing and vision-based affective computing. IEEE Ttans. Affec. Comput. (2019)

    Google Scholar 

  20. Taylor, S., Jaques, N., Nosakhare, E., Sano, A., Picard, R.: Personalized multitask learning for predicting tomorrow’s mood, stress, and health. IEEE Trans. Affec. Comput. 11(2), 200–213 (2020)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Dominika Kunc .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2022 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Kunc, D., Komoszyńska, J., Perz, B., Kazienko, P., Saganowski, S. (2022). Real-Life Validation of Emotion Detection System with Wearables. In: Ferrández Vicente, J.M., Álvarez-Sánchez, J.R., de la Paz López, F., Adeli, H. (eds) Bio-inspired Systems and Applications: from Robotics to Ambient Intelligence. IWINAC 2022. Lecture Notes in Computer Science, vol 13259. Springer, Cham. https://doi.org/10.1007/978-3-031-06527-9_5

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-06527-9_5

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-06526-2

  • Online ISBN: 978-3-031-06527-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics