Skip to main content
SpringerLink
Log in

FILTWAM and Voice Emotion Recognition

  • Conference paper
  • First Online:
Games and Learning Alliance (GALA 2013)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 8605))

Included in the following conference series:

Abstract

This paper introduces the voice emotion recognition part of our framework for improving learning through webcams and microphones (FILTWAM). This framework enables multimodal emotion recognition of learners during game-based learning. The main goal of this study is to validate the use of microphone data for a real-time and adequate interpretation of vocal expressions into emotional states were the software is calibrated with end users. FILTWAM already incorporates a valid face emotion recognition module and is extended with a voice emotion recognition module. This extension aims to provide relevant and timely feedback based upon learner’s vocal intonations. The feedback is expected to enhance learner’s awareness of his or her own behavior. Six test persons received the same computer-based tasks in which they were requested to mimic specific vocal expressions. Each test person mimicked 82 emotions, which led to a dataset of 492 emotions. All sessions were recorded on video. An overall accuracy of our software based on the requested emotions and the recognized emotions is a pretty good 74.6 % for the emotions happy and neutral emotions; but will be improved for the lower values of an extended set of emotions. In contrast with existing software our solution allows to continuously and unobtrusively monitor learners’ intonations and convert these intonations into emotional states. This paves the way for enhancing the quality and efficacy of game-based learning by including the learner’s emotional states, and links these to pedagogical scaffolding.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Anaraki, F.: Developing an effective and efficient elearning platform. Int. J. Comput. Internet Manag. 12(2), 57–63 (2004)

    Google Scholar 

  2. Nagarajan, P., Wiselin, G.J.: Online educational system (e- learning). Int. J. u- e- Serv. Sci. Technol. 3(4), 37–48 (2010)

    Google Scholar 

  3. Norman, G.: Effectiveness, efficiency, and e-learning. J. Adv. Health Sci. Educ. 13(3), 249–251 (2008)

    Article  Google Scholar 

  4. Ebner, M.: E-Learning 2.0 = e-Learning 1.0 + Web 2.0? In: The Second International Conference on Availability, Reliability and Security (ARES), pp. 1235–1239 (2007)

    Google Scholar 

  5. Hrastinski, S.: Asynchronous and synchronous e-learning. Educause Quarterly 31(4), 51–55 (2008)

    Google Scholar 

  6. Kelle, S., Sigurðarson, S., Westera, W., Specht, M.: Game-based life-long learning. In: Magoulas, G.D. (ed.) E-Infrastructures and Technologies for Lifelong Learning: Next Generation Environments, pp. 337–349. IGI Global, Hershey (2011)

    Google Scholar 

  7. Connolly, T.M., Boyle, E.A., MacArthur, E., Hainey, T., Boyle, J.M.: A systematic literature review of empirical evidence on computer games and serious games. Comput. Educ. 59(2), 661–686 (2012)

    Article  Google Scholar 

  8. Reeves, B., Read, J.L.: Total Engagement: Using Games and Virtual Worlds to Change the Way People Work and Business Compete. Harvard Business Press, Boston (2009)

    Google Scholar 

  9. Gee, J.P.: What Video Games have to Teach us About Learning and Literacy. Palgrave Macmillan, New York (2003)

    Google Scholar 

  10. Nadolski, R.J., Hummel, H.G.K., Van den Brink, H.J., Hoefakker, R., Slootmaker, A., Kurvers, H., Storm, J.: EMERGO: methodology and toolkit for efficient development of serious games in higher education. Simul. Gaming 39(3), 338–352 (2008)

    Article  Google Scholar 

  11. Bahreini, K., Nadolski, R., Qi, W., Westera, W.: FILTWAM - A framework for online game-based communication skills training - using webcams and microphones for enhancing learner support. In: Felicia, P. (ed.) The 6th European Conference on Games Based Learning (ECGBL), pp. 39–48. Ireland, Cork (2012)

    Google Scholar 

  12. Avidan, S., Butman, M.: Blind Vision. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3953, pp. 1–13. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  13. Bashyal, S., Venayagamoorthy, G.K.: Recognition of facial expressions using Gabor wavelets and learning vector quantization. Eng. Appl. Artif. Intell. 21, 1056–1064 (2008)

    Article  Google Scholar 

  14. Chibelushi, C.C., Bourel, F.: Facial expression recognition: a brief tutorial overview. In: CVonline: Online in Compendium of Computer Vision, vol. 9 (2003)

    Google Scholar 

  15. Ekman, P., Friesen, W.V.: Facial Action Coding System: Investigator’s Guide. Consulting Psychologists Press, Palo Alto (1978)

    Google Scholar 

  16. Kanade, T.: Picture processing system by computer complex and recognition of human faces. Ph.D. Thesis, Kyoto University, Japan (1973)

    Google Scholar 

  17. Li, S.Z., Jain, A.K.: Handbook of Face Recognition, 2nd edn. Springer, London (2011). ISBN: 978-0-85729-931-4

    Book  MATH  Google Scholar 

  18. Petta, P., Pelachaud, C., Cowie, R.: Emotion-Oriented Systems: The Humaine Handbook. Springer, Berlin (2011)

    Google Scholar 

  19. Chen, L.S.: Joint processing of audio-visual information for the recognition of emotional expressions in human-computer interaction. Ph.D. Thesis, University of Illinois at Urbana-Champaign (2000)

    Google Scholar 

  20. Fong, T., Nourbakhsh, I., Dautenhahn, K.: A survey of socially interactive robots. Robot. Auton. Syst. 42(3–4), 143–166 (2003)

    Article  MATH  Google Scholar 

  21. Sebe, N., Cohen, I.I., Gevers, T., Huang, T.S.: Emotion recognition based on joint visual and audio cues. In: International Conference on Pattern Recognition, Hong Kong, pp. 1136–1139 (2006)

    Google Scholar 

  22. Song, M., Bu, J., Chen, C., Li, N.: Audio-visual based emotion recognition: A new approach. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recogn. 2, 1020–1025 (2004)

    Google Scholar 

  23. Subramanian, R., Staiano, J., Kalimeri, K., Sebe, N., Pianesi, F.: Putting the Pieces Together: Multimodal Analysis of Social Attention in Meetings. ACM Multimedia, Firenze (2010)

    Google Scholar 

  24. Zeng, Z., Pantic, M., Roisman, G.I., Huang, T.S.: A survey of affect recognition methods: Audio, visual, and spontaneous expressions. IEEE Trans. Pattern Anal. Mach. Intell. 31(1), 39–58 (2009)

    Article  Google Scholar 

  25. Sebe, N.: Multimodal interfaces: challenges and perspectives. J. Ambient Intell. Smart Environ. 1(1), 23–30 (2009)

    Google Scholar 

  26. Pekrun, R.: The impact of emotions on learning and achievement: towards a theory of cognitive/motivational mediators. J. Appl. Psychol. 41, 359–376 (1992)

    Article  Google Scholar 

  27. Hager, P.J., Hager, P., Halliday, J.: Recovering Informal Learning: Wisdom, Judgment and Community. Springer, Dordrecht (2006)

    Google Scholar 

  28. Vogt, T., André, E., Bee, N.: EmoVoice - A framework for online recognition of emotions from voice. In: Proceedings of Workshop on Perception and Interactive Technologies for Speech-Based Systems (2008)

    Google Scholar 

  29. Wagner, J., Lingenfelser, F., Andre, E.: The social signal interpretation framework (SSI) for real time signal processing and recognitions. In: Proceedings of INTERSPEECH, Florence, Italy. (2011)

    Google Scholar 

  30. Schuller, B., Manfred, L., Gerhard, R.: Automatic emotion recognition by the speech signal, Institute for Human-Machine-Communication, Technical University of Munich, 80290 (2002)

    Google Scholar 

  31. Bahreini, K., Nadolski, R., Westera, W.: FILTWAM - A framework for online affective computing in serious games. In: The 4th International Conference on Games and Virtual Worlds for Serious Applications (VS-GAMES’12), Procedia Computer Science, Genoa, Italy, vol. 15, pp. 45–52 (2012)

    Google Scholar 

  32. Lang, G., van der Molen, H.T.: Psychologische gespreksvoering. Open University of the Netherlands, Heerlen (2008)

    Google Scholar 

  33. Van der Molen, H.T., Gramsbergen-Hoogland, Y.H.: Communication in Organizations: Basic Skills and Conversation Models. Psychology Press, New York (2005). ISBN: 978-1-84169-556-3

    Google Scholar 

  34. Dai, K., Harriet J.F., MacAuslan, J.: Recognizing emotion in speech using neural networks. In: Telehealth and Assistive Technologies, pp. 31–38 (2008)

    Google Scholar 

Download references

Acknowledgments

We thank our colleagues who participated in the voice emotion recognition proof of concept study. This research is sponsored by The Netherlands Laboratory for Lifelong Learning (NELLL) of the Open University of the Netherlands.

Author information

Authors and Affiliations

  1. Centre for Learning Sciences and Technologies (CELSTEC), Open University Netherlands, Valkenburgerweg 177, 6419 AT, Heerlen, The Netherlands

    Kiavash Bahreini, Rob Nadolski & Wim Westera

Authors
  1. Kiavash Bahreini
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rob Nadolski
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wim Westera
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kiavash Bahreini .

Editor information

Editors and Affiliations

  1. University of Genova, Genova, Italy

    Alessandro De Gloria

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer International Publishing Switzerland

About this paper

Cite this paper

Bahreini, K., Nadolski, R., Westera, W. (2014). FILTWAM and Voice Emotion Recognition. In: De Gloria, A. (eds) Games and Learning Alliance. GALA 2013. Lecture Notes in Computer Science(), vol 8605. Springer, Cham. https://doi.org/10.1007/978-3-319-12157-4_10

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-12157-4_10

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-12156-7

  • Online ISBN: 978-3-319-12157-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation