Viewing Student Affect and Learning through Classroom Observation and Physical Sensors

  • Toby Dragon
  • Ivon Arroyo
  • Beverly P. Woolf
  • Winslow Burleson
  • Rana el Kaliouby
  • Hoda Eydgahi
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5091)


We describe technology to dynamically collect information about students’ emotional state, including human observation and real-time multi-modal sensors. Our goal is to identify physical behaviors that are linked to emotional states, and then identify how these emotional states are linked to student learning. This involves quantitative field observations in the classroom in which researchers record the behavior of students who are using intelligent tutors. We study the specific elements of learner’s behavior and expression that could be observed by sensors. The long-term goal is to dynamically predict student performance, detect a need for intervention, and determine which interventions are most successful for individual students and the learning context (problem and emotional state).


Emotional State Classroom Observation Intelligent Tutor System Positive Valence Task Behavior 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Aleven, V., Roll, I., McLaren, B., Ryu, E.J., Koedinger, K.: An architecture to combine meta-cognitive and cognitive tutoring: Pilot testing the Help Tutor. In: Proceedings of the 12th International Conference on AIED (2005)Google Scholar
  2. 2.
    Arroyo, I., Ferguson, K., Johns, J., Dragon, T., Meheranian, H., Fisher, D., Barto, A., Mahadevan, S., Woolf, B.P.: Repairing Disengagement with Non-Invasive Interventions. In: Proceedings of the 13th International Conference of AIED. IOS Press (2007)Google Scholar
  3. 3.
    Baker, R.S.J.d.: Modeling and Understanding Students’ Off-Task Behavior in Intelligent Tutoring Systems. In: Proceedings of ACM CHI 2007. Computer-Human Interaction (2007)Google Scholar
  4. 4.
    Baker, R.S.J.d., Corbett, A., Koedinger, K.: Detecting Student Misuse of Intelligent Tutoring Systems. In: Proceedings of the Seventh International Conference on Intelligent Tutoring Systems, pp. 531–540 (2004)Google Scholar
  5. 5.
    Block, J.: On the relation between IQ, impulsivity and deliquency. Journal of American Psychology 104, 395–398 (1995)Google Scholar
  6. 6.
    Boucsein, W.: Electrodermal activity. Plenum Press, New York (1992)Google Scholar
  7. 7.
    Conati, C.: Probabilistic assessment of users’ emotions in educational games. Journal of Applied Artificial Intelligence, special issue on Merging Cognition and Affect in HCI 16(7-8), 555–575 (2002)Google Scholar
  8. 8.
    Cytowic, R.E.: Synesthesia: A union of the senses. Springer, New York (1989)Google Scholar
  9. 9.
    D’Mello, S.K., Picard, R., Graesser, A.C.: Toward an affect-sensitive AutoTutor. IEEE Intelligent Systems 22(4), 53–61 (2007)CrossRefGoogle Scholar
  10. 10.
    Dweck, C.S.: Self-theories: Their role in motivation, personality and development. The Psychology Press, Philadelphia (1999)Google Scholar
  11. 11.
    Goleman, D.: Emotional intelligence: Why it can matter more than IQ. Bantam, New York (1995)Google Scholar
  12. 12.
    Kaliouby, R., Robinson, P.: Real-time Inference of Complex Mental States from Facial Expressions and Head Gestures. In: Real-Time Vision for HCI, pp. 181–200. Spring-Verlag (2005)Google Scholar
  13. 13.
    Kapoor, A., Mota, S., Picard, R.W.: Towards a learning companion that recognises affect. In: AAAI Fall Symposium 2001, North Falmouth (2001)Google Scholar
  14. 14.
    Kapoor, A., Picard, R.W., Ivanov, Y.: Probabilistic Combination of Multiple Modalities to Detect Interest. In: International Conference on Pattern Recognition, August 2004, Cambridge (2004)Google Scholar
  15. 15.
    McQuiggan, S., Lester, J.: Diagnosing Self-Efficacy in Intelligent Tutoring Systems: An Empirical Study. In: Ikeda, M., Ashley, K.D., Chan, T.-W. (eds.) ITS 2006. LNCS, vol. 4053. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  16. 16.
    Norman, D.A.: Twelve issues for cognitive science. In: Perspectives on cognitive science, pp. 265–295. Erlbaum, Hillsdale (1981)Google Scholar
  17. 17.
    Sheldon-Biddle, E., Malone, L., McBride, D.: Objective measurement of student affect to optimize automated instruction. In: Proceedings of Workshop on Modelling User Attitudes and Affect, User Modeling 2003(2003)Google Scholar
  18. 18.
    Shute, V.J.: Focus on formative feedback. ETS Research Report, #RR-07-11, Princeton (2006)Google Scholar
  19. 19.
    Tekscan: Tekscan Body Pressure Measurement System User’s Manual. Tekscan Inc., South Boston, MA, USA (1997)Google Scholar
  20. 20.
    Wundt, W.: Outlines of Psychology. 2nd rev. English ed. Williams &Norgate, London (1902)Google Scholar
  21. 21.
    Zacks., J.M., Tversky, B., Iyer, G.: Perceiving, remembering, and communicating structure in events. Journal of Experimental Psychology: General 130, 29–58 (2001)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Toby Dragon
    • 1
  • Ivon Arroyo
    • 1
  • Beverly P. Woolf
    • 1
  • Winslow Burleson
    • 2
  • Rana el Kaliouby
    • 3
  • Hoda Eydgahi
    • 4
  1. 1.Department of Computer ScienceUniversity of MassachusettsAmherst 
  2. 2.Arts, Media and Engineering ProgramArizona State University 
  3. 3.Media LabMassachusetts Institute of Technology 
  4. 4.Department of Electrical EngineeringMassachusetts Institute of Technology 

Personalised recommendations