Abstract
This paper describes the use of wireless sensors to recognize student emotion and the use of pedagogical agents to respond to students with these emotions. Minimally invasive sensor technology has reached such a maturity level that students engaged in classroom work can us sensors while using a computer-based tutor. The sensors, located on each of 25 student’s chair, mouse, monitor, and wrist, provide data about posture, movement, grip tension, facially expressed mental states and arousal. This data has demonstrated that intelligent tutoring systems can provide adaptive feedback based on an individual student’s affective state. We also describe the evaluation of emotional embodied animated pedagogical agents and their impact on student motivation and achievement. Empirical studies show that students using the agents increased their math value, self-concept and mastery orientation.
Chapter PDF
Similar content being viewed by others
References
Wentzel, K., Asher, S.R.: Academic lives of neglected, rejected, popular, and controversial children. Child Development 66, 754–763 (1995)
Royer, J.M., Walles, R.: Influences of gender, motivation and socioeconomic status on mathematics performance. In: Berch, D.B., Mazzocco, M.M.M. (eds.) Why is math so hard for some children, pp. 349–368. Paul. H. Brookes Publishing Co., Baltimore (2007)
Conati, C., Maclare, H.: Evaluating a Probabilistic Model of Student Affect. In: Lester, J.C., Vicari, R.M., Paraguaçu, F. (eds.) ITS 2004. LNCS, vol. 3220, pp. 55–66. Springer, Heidelberg (2004)
D’Mello, S., Graesser, A.: Mind and Body: Dialogue and Posture for Affect Detection in Learning Environments. Paper presented at the Frontiers in Artificial Intelligence and Applications (2007)
McQuiggan, S., Lester, J.: Diagnosing Self-Efficacy in Intelligent Tutoring Systems: An Empirical Study. In: Ikeda, M., Ashley, K., Chan, T.W. (eds.) Eighth International Conference on Intelligent Tutoring Systems, Jhongli, Taiwan (2006)
Graesser, A.C., Chipman, P., King, B., McDaniel, B., D’Mello, S.: Emotions and Learning with AutoTutor. In: Luckin, R., Koedinger, K., Greer, J. (eds.) 13th International Conference on Artificial Intelligence in Education (AIED 2007), pp. 569–571. IOS Press, Amsterdam (2007)
D’Mello, S.K., Picard, R.W., Graesser, A.C.: Towards an Affect-Sensitive AutoTutor. Special issue on Intelligent Educational Systems. IEEE Intelligent Systems 22(4), 53–61 (2007)
Conati, C., Mclaren, H.: Evaluating A Probabilistic Model of Student Affect. In: Lester, J.C., Vicari, R.M., Paraguaçu, F. (eds.) ITS 2004. LNCS, vol. 3220, pp. 55–66. Springer, Heidelberg (2004)
Burleson, W.: Affective Learning Companions: Strategies for Empathetic Agents with Real-Time Multimodal Affective Sensing to Foster Meta-Cognitive Approaches to Learning, Motivation, and Perseverance. MIT PhD thesis (2006), http://affect.media.mit.edu/pdfs/06.burleson-phd.pdf
Picard, R.W., Scheirer, J.: The galvactivator: A glove that senses and communicates skin conductivity. In: 9th International Conference on Human-Computer Interaction, New Orleans, August 2001, pp. 1538–1542 (2001)
Strauss, M., Reynolds, C., Hughes, S., Park, K., McDarby, G., Picard, R.: The handwave bluetooth skin conductance sensor. In: Tao, J., Tan, T., Picard, R.W. (eds.) ACII 2005. LNCS, vol. 3784, pp. 699–706. Springer, Heidelberg (2005)
Qi, Y., Picard, R.: Context-sensitive bayesian classifiers and application to mouse pressure pattern classification. In: 16th International Conference on Pattern Recognition, Proceedings, vol. 3, pp. 448–451 (2002)
Dennerlein, J., Becker, T., Johnson, P., Reynolds, C., Picard, R.W.: Frustrating computer users increases exposure to physical factors. In: Proceedings of International Ergonomics Association, Seoul, Korea, pp. 24–29 (2003)
Mota, S., Picard, R.W.: Automated posture analysis for detecting learner’s interest level. In: Computer Vision and Pattern Recognition Workshop, vol. 5, p. 49 (2003)
Kapoor, A., Burleson, W., Picard, R.W.: Automatic prediction of frustration. International Journal of Human-Computer Studies 65(8), 724–736 (2007)
Burleson, W., Picard, R.W.: Gender-specific approaches to developing emotionally intelligent learning companions. IEEE Intelligent Systems 22(4), 62–69 (2007)
el Kaliouby, R.: Mind-reading Machines: the automated inference of complex mental states from video. PhD thesis, University of Cambridge (2005)
Arroyo, I., Cooper, D., Burleson, W., Woolf, B.P., Muldner, K.: Empathetic Pedagogical Agents. Submitted to AIED (2009)
Arroyo, I., Ferguson, K., Johns, J., Dragon, T., Mehranian, H., Fisher, D., Barto, A., Mahadevan, S., Woolf, B.: Repairing Disengagement With Non Invasive Interventions. In: International Conference on Artificial Intelligence in Education, Marina del Rey, CA (2007)
Dragon, T., Arroyo, I., Woolf, B.P., Burleson, W., El Kaliouby, R., Eydgahi, H.: Viewing Student Affect and Learning through Classroom Observation and Physical Sensors. In: Woolf, B.P., Aïmeur, E., Nkambou, R., Lajoie, S. (eds.) ITS 2008. LNCS, vol. 5091, pp. 29–39. Springer, Heidelberg (2008)
Graham, S., Weiner, B.: Theories and principles of motivation. In: Berliner, D., Calfee, R. (eds.) Handbook of Educational Psychology, pp. 63–84. Macmillan, New York (1996)
Zimmerman, B.J.: Self-Efficacy: An Essential Motive to Learn. Contemporary Educational Psychology 25, 82–91 (2000)
Picard, R.W., Papert, S., Bender, W., Blumberg, B., Breazeal, C., Cavallo, D., Machover, T., Resnick, M., Roy, D., Strohecker, C.: Affective Learning–A Manifesto. BT Journal 2(4), 253–269 (2004)
Bickmore, T., Picard, R.W.: Establishing and Maintaining Long-Term Human-Computer Relationships. Transactions on Computer-Human Interaction 12(2), 293–327 (2004)
Reeves, B., Nass, C.: The media equation: How people treat computers, television and new media like real people and places. CSLI, New York (1998)
Klein, J., Moon, Y., Picard, R.W.: This Computer Responds to User Frustration: Theory, Design, Results, and Implications. Interacting with Computers 14(2), 119–140 (2002)
Prendinger, H., Ishizuka, M.: The Empathic Companion: A Character-Based Interface that Addresses Users’ Affective States. Applied Artificial Intelligence 19(3-4), 267–285 (2005)
Bickmore, T., Picard, R.W.: Establishing and Maintaining Long-Term Human-Computer Relationships. Transactions on Computer-Human Interaction 12(2), 293–327 (2004)
Baylor, A.: The Impact of Pedagogical Agent Image on Affective Outcomes. In: Proceedings of Workshop on Affective Interactions: Computers in the Affective Loop, International Conference on Intelligent User Interfaces, San Diego, CA (2005)
Dweck, C.: Messages that motivate: How praise molds students’ beliefs, motivation, and performance (In Surprising Ways). In: Aronson, J. (ed.) Improving academic achievement. Academic Press, New York (2002)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Woolf, B., Dragon, T., Arroyo, I., Cooper, D., Burleson, W., Muldner, K. (2009). Recognizing and Responding to Student Affect. In: Jacko, J.A. (eds) Human-Computer Interaction. Ambient, Ubiquitous and Intelligent Interaction. HCI 2009. Lecture Notes in Computer Science, vol 5612. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-02580-8_78
Download citation
DOI: https://doi.org/10.1007/978-3-642-02580-8_78
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-02579-2
Online ISBN: 978-3-642-02580-8
eBook Packages: Computer ScienceComputer Science (R0)