Assessment of Student’s Emotions in Game-Based Learning

  • Elena Novak
  • Tristan E. Johnson


Research has shown that emotions are directly linked to cognition and there is a strong correlation between affect and learning. This notion along with recent technological advancements has prompted researchers from many disciplines to turn their attention toward adding an affective component to human-computer dialog. This chapter discusses emotion assessment methods, recent empirical research related to examining students’ affective states in entertainment and educational games, and conceptual, methodological, and technological issues associated with developing emotion recognition models. An overview of emotion recognition research suggests that there is little consensus on what emotions should be measured and how to do it. Moreover, it is still not clear how emotions affect human learning and performance.


Emotion Recognition Game Play Intelligent Tutoring System Educational Game Affective Computing 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. Aleven, V., & Koedinger, K. (2002). An effective metacognitive strategy: Learning by doing and explaining with a computer-based cognitive tutor. Cognitive Science, 26(2), 147–179.CrossRefGoogle Scholar
  2. Ball, G., & Breese, J. (1999). Modeling the emotional state of computer users. Paper presented at the UM ‘99, in workshop on ‘Attitude, personality and emotions in user-adapted interaction’, Banff, Canada.CrossRefGoogle Scholar
  3. Baker, R. S., Corbett, A. T., Koedinger, K. R., Evenson, S. E., Roll, I., Wagner, A. Z., et al. (2006). Adapting to when students game an intelligent tutoring system. Paper presented at the Eighth International Conference on Intelligent Tutoring Systems. Jhongli, TaiwanGoogle Scholar
  4. Baker, R. S., Corbett, A. T., Koedinger, K. R., & Wagner, A. Z. (2004). Off-task behavior in the cognitive tutor classroom: When students “Game the System”. Paper presented at the Proceedings of ACM CHI 2004: Computer–Human. Interaction Vienna, Austria.Google Scholar
  5. Baker, R. S., D’Mello, S. K., Rodrigo, M. M. T., & Graesser, A. C. (2010). Better to be frustrated than bored: The incidence, persistence, and impact of learners’ cognitive-affective states during interactions with three different computer-based learning environments. International Journal of Human Computer Studies, 68(4), 223–241.CrossRefGoogle Scholar
  6. Bentley, T., Johnston, L., & von Braggo, K. (2005). Evaluation using cued-recall debrief to elicit information about a user’s affective experiences. Paper presented at the OZCHI, Canberra, Australia.Google Scholar
  7. Blanchard, E., Chalfoun, P., & Frasson, C. (2007). Towards advanced learner modeling: Discussions on quasi real-time adaptation with physiological data. Paper presented at the 7th IEEE International Conference on Advanced Learning Technologies (ICALT 2007), Niigata, Japan.CrossRefGoogle Scholar
  8. Boehner, K., DePaula, R., Dourish, P., & Sengers, P. (2007). How emotion is made and measured. International Journal of Human Computer Studies, 65(4), 275–291.CrossRefGoogle Scholar
  9. Cacioppo, J. T., Berntson, G. G., Larsen, J. T., Poehlmann, K. M., & Ito, T. A. (2000). The psychophysiology of emotion. In M. Lewis & J. M. Haviland-Jones (Eds.), Handbook of Emotions(pp. 173–191). New York: The Guilford Press.CrossRefGoogle Scholar
  10. Canamero, D. (1997). Modeling motivations and emotions as a basis for intelligent behavior. Paper presented at the First International Conference on Autonomous Agents. Marina del Rey, CA.Google Scholar
  11. Castellano, G., Villalba, S. D., & Camurri, A. (2007). Recognizing human emotions from body movement and gesture dynamics. In A. Paiva, R. Prada, & R. W. Picard (Eds.), Affective computing and intelligent interaction (pp. 71–82). Berlin: Springer.CrossRefGoogle Scholar
  12. Chaffar, S., Derbali, L., & Frasson, C. (2009). Towards emotional regulation in intelligent tutoring systems. Paper presented at the AACE World Conference on E-learning in Corporate, Government, Healthcare, & Higher Education: E-LEARN 2009, Vancouver, Canada.Google Scholar
  13. Chen, C.-M., & Wang, H.-P. (2011). Using emotion recognition technology to assess the effects of different multimedia materials on learning emotion and performance. Library and Information Science Research, 33(3), 244–255.CrossRefGoogle Scholar
  14. Coles, G. (1998). Reading lessons: The debate over literacy. New York: Hill & Wang.Google Scholar
  15. Conati, C. (2002). Probabilistic assessment of user’s emotions in educational games. Applied Artificial Intelligence, 16(7), 555–575.CrossRefGoogle Scholar
  16. Conati, C., Chabbal, R., & Maclaren, H. (2003). A study on using biometric sensors for monitoring user emotions in educational games. Paper presented at the Workshop “Assessing and Adapting to User Attitude and Affects: Why, When and How?” In conjunction with UM’03, 9th International Conference on User Modeling, Pittsburgh, PA.Google Scholar
  17. Conati, C., & Maclaren, H. (2009). Empirically building and evaluating a probabilistic model of user affect. User Modeling and User-Adapted Interaction, 19(3), 267–303.CrossRefGoogle Scholar
  18. D’Mello, S. K., Craig, S. D., Witherspoon, A., McDaniel, B., & Graesser, A. C. (2008). Automatic detection of learner’s affect from conversational cues. User Modeling and User-Adapted Interaction, 18(1–2), 45–80.CrossRefGoogle Scholar
  19. D’Mello, S., Lehman, B., Sullins, J., Daigle, R., Combs, R., Vogt, K., et al. (2010). A time for emoting: When affect-sensitivity is and isn’t effective at promoting deep learning. In J. Kay & V. Aleven (Eds.),Proceedings of 10th International Conference on Intelligent Tutoring Systems(Vol. 6094, pp. 245–254). Pittsburgh, PA: Springer Berlin / Heidelberg.CrossRefGoogle Scholar
  20. Damasio, A. (1995). Descartes’ error: Emotion, reason and the human brain. New York: Quill.Google Scholar
  21. de Vicente, A., & Pain, H. (2002). Informing the detection of the students’ motivational state: An empirical study. Paper presented at the ITS2002, Biarritz, France and San Sebastian, Spain.Google Scholar
  22. de Vicente, A., & Pain, H. (2003). Validating the detection of a student’s motivational state. Paper presented at the Second International Conference on Multimedia Information & Communication Technologies in Education (m-ICTE2003). Badajoz, SpainGoogle Scholar
  23. Ekman, P., Levenson, R. V., & Friesen, V. W. (1983). Autonomic nervous system activity distinguishes among emotions. Science 221, 1208–1210.Google Scholar
  24. Erez, A., & Isen, A. M. (2002). The influence of positive affect on the components of expectancy motivation. The Journal of Applied Psychology, 87(6), 1055–1067.CrossRefGoogle Scholar
  25. Gaillard, A. W., & Kramer, A. F. (2000). Theoretical and methodical issues in psycho physiological research. In R. W. Backs & W. Boucsein (Eds.), Engineering psychophysiology. London: Lawrence Erlbaum Associates.Google Scholar
  26. Gee, J. P. (Ed.). (2004). Situated language and learning: A critique of traditional schooling. London: Routledge, Taylor & Francis.Google Scholar
  27. Goleman, D. (1995). Emotional intelligence. New York: Bantam Books.Google Scholar
  28. Graesser, A. C., Person, N., Harter, D., & Group, T. R. (2001). Teaching tactics and dialogue in AutoTutor. International Journal of Artificial Intelligence in Education, 12, 257–279.Google Scholar
  29. Graesser, A. C., Rus, V., D’Mello, S., & Jackson, G. T. (2008). AutoTutor: Learning through natural language dialogue that adapts to the cognitive and affective states of the learner. In D. H. Robinson & G. Schraw (Eds.), Current perspectives on cognition, learning and instruction: Recent innovations in educational technology that facilitate student learning (pp. 95–125). Greenwich: Information Age Publishing.Google Scholar
  30. Graesser, A. C., Witherspoon, A., McDaniel, B., D’Mello, S., Chipman, P., & Gholson, B. (2006). Detection of emotions during learning with AutoTutor. Paper presented at the 28th Annual Meeting of the Cognitive Science Society. Vancouver, BC, CanadaGoogle Scholar
  31. Healey, J. (2000). Wearable and automotive systems for affect recognition from physiology. PhD, MIT, Cambridge, MA.Google Scholar
  32. Hone, K. (2006). Empathic agents to reduce user frustration: The effects of varying agent characteristics. Interacting with Computers, 18, 227–245.CrossRefGoogle Scholar
  33. Hudlicka, E., & McNeese, M. (2002). Assessment of user affective and belief states for interface adaptation: Application to an Air Force pilot task. User Modeling and User Adapted Interaction, 12(1), 1–47.Google Scholar
  34. Isbister, K., Höök, K., Laaksolahti, J., & Sharp, M. (2007). The sensual evaluation instrument: Developing a trans-cultural self-report measure of affect. International Journal of Human Computer Studies, 65(4), 315–328.CrossRefGoogle Scholar
  35. Issroff, K., & del Soldato, T. (1995). Incorporating motivation into computer-supported collaborative learning. Paper presented at the European Conference on Artificial Intelligence in Education, Lisbon.Google Scholar
  36. Kaklauskas, A., Zavadskas, E. K., Pruskus, V., Vlasenko, A., Seniut, M., Kaklauskas, G., et al. (2010). Biometric and intelligent self-assessment of student progress system. Computers in Education, 55(2), 821–833.CrossRefGoogle Scholar
  37. Karaseitanidis, I., Amditis, A., Patel, H., Sharples, S., Bekiaris, E., Bullinger, A., et al. (2006). Evaluation of virtual reality products and applications from individual, organizational and societal perspectives—The “VIEW” case study. International Journal of Human Computer Studies, 64, 251–266.CrossRefGoogle Scholar
  38. Keller, J. M. (1984). Use of the ARCS model of motivation in teacher training. In K. E. Shaw (Ed.), Aspect of educational technology XVII: Staff development and career updating. New York: Nichols.Google Scholar
  39. Koedinger, K., Anderson, J., Hadley, W., & Mark, M. (1997). Intelligent tutoring goes to school in the big city. International Journal of Artificial Intelligence in Education, 8, 30–43.Google Scholar
  40. Kort, B., Reilly, R., & Picard, R. W. (2001). An affective model of interplay between emotions and learning: Reengineering educational pedagogy-building a learning companion. Paper presented at the IEEE International Conference on Advanced Learning Technologies, Madison, USA.Google Scholar
  41. LeDoux, J. E. (1996). The emotional brain. New York: Simon and Schuster.Google Scholar
  42. Liao, W., Zhang, W., Zhu, Z., Ji, Q., & Gray, W. D. (2006). Toward a decision-theoretic framework for affect recognition and user assistance. International Journal of Human Computer Studies, 64(9), 847–873.CrossRefGoogle Scholar
  43. Mandryk, R. L., & Atkins, M. S. (2007). A fuzzy physiological approach for continuously modeling emotion during interaction with play environments. International Journal of Human Computer Studies, 6(4), 329–347.CrossRefGoogle Scholar
  44. Mandryk, R. L., Atkins, M. S., & Inkpen, K. M. (2006). A continuous and objective evaluation of emotional experience with interactive play environments. Paper presented at the Conference on Human Factors in Computing Systems (CHI 2006), Montreal, Canada.Google Scholar
  45. Mandryk, R. L., Inkpen, K. M., & Calvert, T. W. (2006). Using psychophysiological techniques to measure user experience with entertainment technologies. Behaviour and Information Technology (Special Issue on User Experience), 25(2), 141–158.Google Scholar
  46. Mentis, H. M. (2007). Memory of frustrating experiences. In D. Nahl & D. Bilal (Eds.), Information and emotion (pp. 197–210). Medford: Information Today.Google Scholar
  47. Myers, D. G. (2002). Intuition: Its powers and perils. New Haven: Yale University Press.Google Scholar
  48. Oaksford, M., Morris, F., Grainger, B., & Williams, J. M. G. (1996). Mood, reasoning, and central executive process. Journal of Experimental Psychology. Learning, Memory, and Cognition, 22, 477–493.CrossRefGoogle Scholar
  49. Ortony, A., Clore, G. L., & Collins, A. (1988). The cognitive structure of emotions. Cambridge: Cambridge University Press.CrossRefGoogle Scholar
  50. Pagulayan, R. J., Keeker, K., Wixon, D., Romero, R., & Fuller, T. (2002). User-centered design in games. In J. Jacko & A. Sears (Eds.), Handbook for human–computer interaction in interactive systems (pp. 883–906). Mahwah: Lawrence Erlbaum Associates, Inc.Google Scholar
  51. Pekrun, R. (2006). The control-value theory of achievement emotions: Assumptions, corollaries, and implications for educational research and practice. Educational Psychology Review, 18, 315–341.CrossRefGoogle Scholar
  52. Pekrun, R., & Stephens, E. J. (2011). Academic emotions. In K. R. Harris, S. Graham, & T. Urdan (Eds.), APA educational psychology handbook (Vol. 2). Washington: American Psychological Association.Google Scholar
  53. Picard, R. W. (1997). Affective computing. Cambridge: MIT Press.Google Scholar
  54. Picard, R. W., Vyzas, E., & Healey, J. (2001). Toward machine emotional intelligence: Analysis of affective physiological state. IEEE Transactions on Patter Analysis and Machine Intelligence, 23(10), 1175–1191.CrossRefGoogle Scholar
  55. Prendinger, H., Ma, C., & Ishizuka, M. (2007). Eye movements as indices for the utility of life-like interface agents: A pilot study. Interacting with Computers, 19(2), 281–292.CrossRefGoogle Scholar
  56. Reilly, R., & Kort, B. (2004). The science behind the art of teaching science: Emotional state and learning. Paper presented at the Conference on Society for Information Technology and Teacher Education, Atlanta, GA, USA.Google Scholar
  57. Scheirer, J., Fernandez, R., Klein, J., & Picard, R. W. (2002). Frustrating the user on purpose: A step toward building an affective computer. Interacting with Computers, 14(2), 93–118.CrossRefGoogle Scholar
  58. Scherer, K. (2005). What are emotions? And how can they be measured? Social Science Information, 44(4), 695–729.CrossRefGoogle Scholar
  59. Spangler, G., Pekrun, R., Kramer, K., & Hofmann, H. (2002). Students’ emotions, psychological reactions, and coping in academic exams. Anxiety, Stress, and Coping, 15, 413–432.CrossRefGoogle Scholar
  60. Tao, J., & Tan, T. (2005). Affective computing: A review. In J. Tao, T. Tan & R. W. Picard (Eds.), Proceedings of Affective Computing and Intelligent Interaction (ACCII 2005) (Vol. 3784, pp. 981-995). Beijing, China: Springer-Verlag Berlin Heidelberg.CrossRefGoogle Scholar
  61. Vizer, L. M., Zhou, L., & Sears, A. (2009). Automated stress detection using key stroke and linguistic features: An exploratory study. International Journal of Human Computer Studies, 67, 870–886.CrossRefGoogle Scholar
  62. Wagner, J., Vogt, T., & Andr, E. (2007). A systematic comparison of different HMM designs for emotion recognition from acted and spontaneous speech. Paper presented at the Proceedings of the 2nd international conference on Affective Computing and Intelligent Interaction, Lisbon, Portugal.Google Scholar
  63. Weiner, B. (1985). An attributional theory of achievement motivation and emotion. Psychological Review, 92, 548–573.CrossRefGoogle Scholar
  64. Weiner, B. (1992). Human motivation: Metaphors, theories and research. Newbury Park, CA: Sage Publications.Google Scholar
  65. Woolf, B. (2009). Building intelligent interactive tutors. Burlington: Morgan Kaufmann Publishers.Google Scholar
  66. Zakharov, K., Mitrovic, A., & Johnston, L. (2008). Towards emotionally-intelligent pedagogical agents. Paper presented at the Intelligent Tutoring Systems (ITS), 9th International Conference, Montreal, Canada.Google Scholar
  67. Zeidner, M. (2007). Test anxiety in educational context: What I have learned so far. In P. A. Schultz & R. Pekrun (Eds.), Emotion in education (pp. 165–184). San-Diego: Academic.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2012

Authors and Affiliations

  1. 1.Educational Psychology and Learning SystemsFlorida State University, University Center C4600TallahasseeUSA
  2. 2.Graduate School of EngineeringNortheastern UniversityBostonUSA

Personalised recommendations