Identifying Significant Task-Based Predictors of Emotion in Learning

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9935)


Emphatic computing is concerned with enabling a system to recognize a user’s current state and then providing the appropriate response to the user with the intention to support the user emotionally. However, in order to do so, the system must first identify the state of the user. Studies in computer-based tutoring are increasingly investigating ways to incorporate synthetic tutors that are equipped with computational models of empathy – in which these agents are trained to understand learners’ emotions and respond based on the detected learner state. However, cultural differences affect the way people express and detect emotions. This paper attempts to identify the task-based features that could discriminate the learner’s emotions in a Malaysian context. By studying several existing task-based features from literature, and combining them with new features, this study attempts to detect four frequent emotions that accompanies learning, namely, frustration, boredom, uncertainty and neutral. A user study is conducted with 33 students and results revealed that certain features can be used as predictors for the abovementioned emotions. Interestingly, results also showed that there is a tendency for students to choose synthetic tutors of the same race.


Empathy Intelligent tutoring system Task-based features Emotions 



The authors would like to thank Universiti Sains Malaysia for the funding of this work from the grant no. 304/PKOMP/6312153.


  1. 1.
    Davis, M.H.: Empathy: A Social Psychological Approach. Westview Press, Boulder (1994)Google Scholar
  2. 2.
    Sabourin, J., Mott, B., Lester, J.: Computational models of affect and empathy for pedagogical virtual agents. In: Standards in Emotion Modeling, Lorentz Center International Center for Workshops in the SciencesGoogle Scholar
  3. 3.
    Aquino, R.J., Battad, J., Ngo, C.F., Uy, G., Trogo, R., Suarez, M.: Towards empathic support provision for computer users. In: Nishizaki, S.-H., Numao, M., Caro, J., Suarez, M.T. (eds.) Theory and Practice of Computation, vol. 5, pp. 15–27. Springer, Tokyo (2012)CrossRefGoogle Scholar
  4. 4.
    D’Mello, S., Picard, R.W., Graesser, A.: Toward an affect-sensitive AutoTutor. IEEE Intell. Syst. 22, 53–61 (2007)CrossRefGoogle Scholar
  5. 5.
    Rajendran, R., Iyer, S., Murthy, S., Wilson, C., Sheard, J.: A theory-driven approach to predict frustration in an ITS. IEEE Trans. Learn. Technol. 6, 378–388 (2013)CrossRefGoogle Scholar
  6. 6.
    Syed Mohamad, S.J.A.N.: Learning Style among Multi-Ethnic Students in Four Selected Tertiary Institutions in the Klang Valley. Universiti Putra Malaysia (2006)Google Scholar
  7. 7.
    Matsumoto, D., Juang, L.: Culture and Psychology. Cengage Learning, Boston (2012)Google Scholar
  8. 8.
    Pardos, Z.A., Baker, R.S., San Pedro, M., Gowda, S.M., Gowda, S.M.: Affective states and state tests: investigating how affect and engagement during the school year predict end-of-year learning outcomes. J. Learn. Anal. 1, 107–128 (2014)CrossRefGoogle Scholar
  9. 9.
    Wang, Z., Qiao, X., Xie, Y.: An emotional intelligent e-learning system based on mobile agent technology. In: International Conference on Computer Engineering and Technology, 2009, ICCET 2009, pp. 51–54. IEEE (2011)Google Scholar
  10. 10.
    Robison, J., McQuiggan, S., Lester, J.: Evaluating the consequences of affective feedback in intelligent tutoring systems. In: 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops, pp. 1–6. IEEE (2009)Google Scholar
  11. 11.
    Rodrigo, M.M.T., Baker, R.S.: Coarse-grained detection of student frustration in an introductory programming course. In: Proceedings of the Fifth International Workshop on Computing Education Research Workshop, pp. 75–80. ACM (2009)Google Scholar
  12. 12.
    McQuiggan, S.W., Lester, J.C.: Modeling and evaluating empathy in embodied companion agents. Int. J. Hum.-Comput. Stud. 65, 348–360 (2007)CrossRefGoogle Scholar
  13. 13.
    Daradoumis, T., Arguedas, M., Xhafa, F.: Current trends in emotional e-learning: new perspectives for enhancing emotional intelligence. In: 2013 7th International Conference on Complex, Intelligent, and Software Intensive Systems (CISIS), pp. 34–39. IEEE (2013)Google Scholar
  14. 14.
    D’mello, S.K., Craig, S.D., Witherspoon, A., Mcdaniel, B., Graesser, A.: Automatic detection of learner’s affect from conversational cues. User Model. User-Adap. Interact. 18, 45–80 (2008)CrossRefGoogle Scholar
  15. 15.
    Baker, R.S., D’Mello, S.K., Rodrigo, M.M.T., Graesser, A.C.: Better to be frustrated than bored: the incidence, persistence, and impact of learners’ cognitive–affective states during interactions with three different computer-based learning environments. Int. J. Hum.-Comput. Stud. 68, 223–241 (2010)CrossRefGoogle Scholar
  16. 16.
    Forbes-Riley, K., Litman, D., Friedberg, H., Drummond, J.: Intrinsic and extrinsic evaluation of an automatic user disengagement detector for an uncertainty-adaptive spoken dialogue system. In: Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 91–102. Association for Computational Linguistics (2012)Google Scholar
  17. 17.
    Plass, J.L., Heidig, S., Hayward, E.O., Homer, B.D., Um, E.: Emotional design in multimedia learning: effects of shape and color on affect and learning. Learn. Instruct. 29, 128–140 (2014)CrossRefGoogle Scholar
  18. 18.
    Forbes-Riley, K., Litman, D.: Benefits and challenges of real-time uncertainty detection and adaptation in a spoken dialogue computer tutor. Speech Commun. 53, 1115–1136 (2011)CrossRefGoogle Scholar
  19. 19.
    Sabourin, J.L., Lester, J.C.: Affect and engagement in Game-BasedLearning environments. IEEE Trans. Affect. Comput. 5, 45–56 (2014)CrossRefGoogle Scholar
  20. 20.
    McQuiggan, S.W., Robison, J.L., Phillips, R., Lester, J.C.: Modeling parallel and reactive empathy in virtual agents: an inductive approach. In: Proceedings of the 7th International Joint Conference on Autonomous Agents and Multiagent Systems, vol. 1, pp. 167–174. International Foundation for Autonomous Agents and Multiagent Systems (2008)Google Scholar
  21. 21.
    Elliott, C.D.: The affective reasoner: a process model of emotions in a multi-agent system. Ph.D. Northwestern University (1992)Google Scholar
  22. 22.
    Gratch, J., Marsella, S.: A domain-independent framework for modeling emotion. Cogn. Syst. Res. 5, 269–306 (2004)CrossRefGoogle Scholar
  23. 23.
    Picard, R.W., Picard, R.: Affective Computing. MIT Press, Cambridge (1997)CrossRefGoogle Scholar
  24. 24.
    Graesser, A., McDaniel, B., Chipman, P., Witherspoon, A., D’Mello, S., Gholson, B.: Detection of emotions during learning with AutoTutor. In: Proceedings of the 28th Annual Meetings of the Cognitive Science Society, pp. 285–290. Citeseer (2006)Google Scholar
  25. 25.
    Kapoor, A., Picard, R.W.: Multimodal affect recognition in learning environments. In: Proceedings of the 13th Annual ACM International Conference on Multimedia, pp. 677–682. ACM (2005)Google Scholar
  26. 26.
    Zhao, H., Sun, B., Hu, X., Zhu, X.: The study of emotional education based on virtual reality in e-learning. In: 1st International Conference on Information Science and Engineering (ICISE), pp. 3540–3543. IEEE (2009)Google Scholar
  27. 27.
    Hu, Y., Zhao, G.: Virtual classroom with intelligent virtual tutor. In: International Conference on e-Education, e-Business, e-Management, and e-Learning, IC4E 2010, pp. 34–38. IEEE (2010)Google Scholar
  28. 28.
    Chaffar, S., Frasson, C.: Using an emotional intelligent agent to improve the learner’s performance. In: Proceedings of the Workshop on Social and Emotional Intelligence in Learning Environments in Conjunction with Intelligent Tutoring Systems (2004)Google Scholar
  29. 29.
    Lutfi, S.L., Fernández-Martínez, F., Lorenzo-Trueba, J., Barra-Chicote, R., Montero, J.M.: I feel you: the design and evaluation of a domotic affect-sensitive spoken conversational agent. Sensors 13, 10519–10538 (2013)CrossRefGoogle Scholar
  30. 30.
    Hsiao, I.H., Sosnovsky, S., Brusilovsky, P.: Guiding students to the right questions: adaptive navigation support in an e-learning system for Java programming. J. Comput. Assist. Learn. 26, 270–283 (2010)CrossRefGoogle Scholar
  31. 31.
    Gulzar, S., Yahya, F., Nauman, M., Mir, Z., Mujahid, S.H.: Frustration among University Students in Pakistan (2012)Google Scholar
  32. 32.
    Blair, C.: School readiness: integrating cognition and emotion in a neurobiological conceptualization of children’s functioning at school entry. Am. Psychol. 57, 111 (2002)CrossRefGoogle Scholar
  33. 33.
    Pekrun, R., Elliot, A.J., Maier, M.A.: Achievement goals and achievement emotions: testing a model of their joint relations with academic performance. J. Educ. Psychol. 101, 115 (2009)CrossRefGoogle Scholar
  34. 34.
    Valiente, C., Swanson, J., Eisenberg, N.: Linking students’ emotions and academic achievement: when and why emotions matter. Child. Dev. Perspect. 6, 129–135 (2012)CrossRefGoogle Scholar
  35. 35.
    Wong, K.Y., Quek, K.S.: Do Chinese and Malay students report different ways of studying mathematics? 1–13 (2007)Google Scholar
  36. 36.
    Pekrun, R., Goetz, T., Perry, R.P., Kramer, K., Hochstadt, M., Molfenter, S.: Beyond test anxiety: development and validation of the test emotions questionnaire (TEQ). Anxiety Stress Coping 17, 287–316 (2004)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  1. 1.School of Computer SciencesUniversiti Sains MalaysiaMindenMalaysia

Personalised recommendations