Skip to main content

Predicting Learning from Student Affective Response to Tutor Questions

Part of the Lecture Notes in Computer Science book series (LNPSE,volume 9684)

Abstract

Modeling student learning during tutorial interaction is a central problem in intelligent tutoring systems. While many modeling techniques have been developed to address this problem, most of them focus on cognitive models in conjunction with often-complex domain models. This paper presents an analysis suggesting that observing students’ multimodal behaviors may provide deep insight into student learning at critical moments in a tutorial session. In particular, this work examines student facial expression, electrodermal activity, posture, and gesture immediately following inference questions posed by human tutors. The findings show that for human-human task-oriented tutorial dialogue, facial expression and skin conductance response following tutor inference questions are highly predictive of student learning gains. These findings suggest that with multimodal behavior data, intelligent tutoring systems can make more informed adaptive decisions to support students effectively.

Keywords

  • Facial Expression
  • Skin Conductance Response
  • Intelligent Tutoring System
  • Learning Gain
  • Electrodermal Activity

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-319-39583-8_15
  • Chapter length: 11 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   69.99
Price excludes VAT (USA)
  • ISBN: 978-3-319-39583-8
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   89.99
Price excludes VAT (USA)
Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.

References

  1. Hartley, D., Mitrović, A.: Supporting learning by opening the student model. In: Cerri, S.A., Gouardéres, G., Paraguaçu, F. (eds.) ITS 2002. LNCS, vol. 2363, pp. 453–462. Springer, Heidelberg (2002)

    CrossRef  Google Scholar 

  2. Gluga, R.: Long term student learner modeling and curriculum mapping. In: Aleven, V., Kay, J., Mostow, J. (eds.) ITS 2010, Part II. LNCS, vol. 6095, pp. 227–229. Springer, Heidelberg (2010)

    CrossRef  Google Scholar 

  3. Corbett, A.T., Anderson, J.R.: Student modeling and mastery learning in a computer-based programming tutor. In: Proceedings of the 2nd International Conference on Intelligent Tutoring Systems, pp. 413–420 (1992)

    Google Scholar 

  4. Stevens, R., Soller, A., Cooper, M., Sprang, M.: Modeling the development of problem solving skills in chemistry with a web-based tutor. In: Lester, J.C., Vicari, R.M., Paraguaçu, F. (eds.) ITS 2004. LNCS, vol. 3220, pp. 580–591. Springer, Heidelberg (2004)

    CrossRef  Google Scholar 

  5. Corbett, A.T., Anderson, J.R.: Knowledge tracing: modeling the acquisition of procedural knowledge. User Model. User-Adap. Inter. 4, 253–278 (1995)

    CrossRef  Google Scholar 

  6. Pavlik Jr., P.I., Cen, H., Koedinger, K.R.: Performance factors analysis - a new alternative to knowledge tracing. In: Proceedings of the 14th International Conference on Artificial Intelligence in Education, pp. 531–538 (2009)

    Google Scholar 

  7. Grafsgaard, J.F., Wiggins, J.B., Boyer, K.E., Wiebe, E.N., Lester, J.C.: Automatically recognizing facial expression: predicting engagement and frustration. In: Proceedings of the 6th International Conference on Educational Data Mining, pp. 43–50 (2013)

    Google Scholar 

  8. Grafsgaard, J.F., Wiggins, J.B., Boyer, K.E., Wiebe, E.N., Lester, J.C.: Automatically recognizing facial indicators of frustration: a learning-centric analysis. In: Proceedings of the Humane Association Conference on Affective Computing and Intelligent Interaction, pp. 159–165 (2013)

    Google Scholar 

  9. Scherer, S., Weibel, N., Morency, L.P., Oviatt, S.: Multimodal prediction of expertise and leadership in learning groups. In: Proceedings of the 1st International Workshop on Multimodal Learning Analytics (2012)

    Google Scholar 

  10. Biel, J.I., Teijeiro-Mosquera, L., Gatica-Perez, D.: FaceTube: predicting personality from facial expressions of emotion in online conversational video. In: Proceedings of the 14th International Conference on Multimodal Interaction, pp. 53–56 (2012)

    Google Scholar 

  11. Oviatt, S., Cohen, A.: Written and multimodal representations as predictors of expertise and problem-solving success in mathematics. In: Proceedings of the 15th International Conference on Multimodal Interaction, pp. 599–606 (2013)

    Google Scholar 

  12. D’Mello, S.K., Graesser, A.C.: Multimodal semi-automated affect detection from conversational cues, gross body language, and facial features. User Model. User-Adap. Inter. 20, 147–187 (2010)

    CrossRef  Google Scholar 

  13. Arroyo, I., Cooper, D.G., Burleson, W., Woolf, B.P., Muldner, K., Christopherson, R.: Emotion sensors go to school. In: Proceedings of the 14th International Conference on Artificial Intelligence in Education, pp. 17–24 (2009)

    Google Scholar 

  14. Stein, N.L., Levine, L.J.: Making sense out of emotion: the representation and use of goal-structured knowledge. In: Psychological and Biological Approaches to Emotion, pp. 45–73 (1990)

    Google Scholar 

  15. Piaget, J.: The Origins of Intelligence. International University Press, New York (1952)

    CrossRef  Google Scholar 

  16. Hardy, M., Wiebe, E.N., Grafsgaard, J.F., Boyer, K.E., Lester, J.C.: Physiological responses to events during training: Use of skin conductance to inform future adaptive learning systems. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, pp. 2101–2105 (2013)

    Google Scholar 

  17. Ha, E.Y., Grafsgaard, J.F., Mitchell, C.M., Boyer, K.E., Lester, J.C.: Combining verbal and nonverbal features to overcome the ‘Information Gap’ in task-oriented dialogue. In: Proceedings of the 13th Annual SIGDIAL Meeting on Discourse and Dialogue, pp. 247–256 (2012)

    Google Scholar 

  18. Mitchell, C.M., Ha, E.Y., Boyer, K.E., Lester, J.C.: Learner characteristics and dialogue: recognising effective and student-adaptive tutorial strategies. Int. J. Learn. Technol. 8(4), 382–403 (2013)

    CrossRef  Google Scholar 

  19. Vail, A.K., Boyer, K.E.: Adapting to personality over time: examining the effectiveness of dialogue policy progressions in task-oriented interaction. In: Proceedings of the 15th Annual SIGDIAL Meeting on Discourse and Dialogue, pp. 41–50 (2014)

    Google Scholar 

  20. Vail, A.K., Boyer, K.E.: Identifying effective moves in tutorial dialogue: on the refinement of speech act annotation schemes. In: Proceedings of the 12th International Conference on Intelligent Tutoring Systems, pp. 199–209 (2014)

    Google Scholar 

  21. Graesser, A.C., Olde, B.A.: How does one know whether a person understands a device? The quality of the questions the person asks when the device breaks down. J. Educ. Psychol. 95(3), 524–536 (2003)

    CrossRef  Google Scholar 

  22. Littlewort, G., Whitehill, J., Wu, T., Fasel, I., Frank, M., Javier, M., Bartlett, M.: The computer expression recognition toolbox (CERT). In: Proceedings of the IEEE International Conference on Automatic Face & Gesture Recognition and Workshops, pp. 298–305 (2011)

    Google Scholar 

  23. Lepper, M.R., Woolverton, M.: The wisdom of practice: Lessons learned from the study of highly effective tutors. Impact of Psychological Factors on Education, Improving Academic Achievement, pp. 135–158 (2002)

    Google Scholar 

  24. Boucsein, W.: Electrodermal Activity. Springer Science & Business Media, NewYork (2012)

    CrossRef  Google Scholar 

  25. Benedek, M., Kaernbach, C.: A continuous measure of phasic electrodermal activity. J. Neurosci. Methods 190, 80–91 (2010)

    CrossRef  Google Scholar 

  26. Vural, E., Cetin, M., Ercil, A., Littlewort, G., Bartlett, M., Movellan, J.: Drowsy driver detection through facial movement analysis. In: Proceedings of the 12th International Conference on Human-Computer Interaction, pp. 6–18 (2007)

    Google Scholar 

  27. Mortillaro, M., Mehu, M., Scherer, K.R.: Subtly different positive emotions can be distinguished by their facial expressions. Soc. Psychol. Pers. Sci. 2(3), 262–271 (2011)

    CrossRef  Google Scholar 

  28. Goldin, I.M., Carlson, R.: Learner differences and hint content. In: Lane, H.C., Yacef, K., Mostow, J., Pavlik, P. (eds.) AIED 2013. LNCS, vol. 7926, pp. 522–531. Springer, Heidelberg (2013)

    CrossRef  Google Scholar 

  29. San Pedro, M.O.Z., Baker, R.S.J., Gowda, S.M., Heffernan, N.T.: Towards an understanding of affect and knowledge from student interaction with an intelligent tutoring system. In: Lane, H.C., Yacef, K., Mostow, J., Pavlik, P. (eds.) AIED 2013. LNCS, vol. 7926, pp. 41–50. Springer, Heidelberg (2013)

    CrossRef  Google Scholar 

  30. D’Mello, S., Lehman, B., Pekrun, R., Graesser, A.: Confusion can be beneficial for learning. Learn. Instr. 29, 153–170 (2004)

    CrossRef  Google Scholar 

Download references

Acknowledgements

The authors wish to thank the members of the LearnDialogue and Intellimedia groups at North Carolina State University for their helpful input. This work is supported in part by the Department of Computer Science at North Carolina State University and the National Science Foundation through Grants IIS-1409639, CNS-1453520, and a Graduate Research Fellowship. Any opinions, findings, conclusions, or recommendations expressed in this report are those of the participants, and do not necessarily represent the official views, opinions, or policy of the National Science Foundation.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Alexandria K. Vail .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this paper

Cite this paper

Vail, A.K., Grafsgaard, J.F., Boyer, K.E., Wiebe, E.N., Lester, J.C. (2016). Predicting Learning from Student Affective Response to Tutor Questions. In: Micarelli, A., Stamper, J., Panourgia, K. (eds) Intelligent Tutoring Systems. ITS 2016. Lecture Notes in Computer Science(), vol 9684. Springer, Cham. https://doi.org/10.1007/978-3-319-39583-8_15

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-39583-8_15

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-39582-1

  • Online ISBN: 978-3-319-39583-8

  • eBook Packages: Computer ScienceComputer Science (R0)