Abstract
Frustration is a natural part of learning in AIED systems but remains relatively poorly understood. In particular, it remains unclear how students’ perceptions about the learning activity drive their experience of frustration and their subsequent choices during learning. In this paper, we adopt a mixed-methods approach, using automated detectors of affect to signal classroom researchers to interview a specific student at a specific time. We hand-code the interviews using grounded theory, then distill particularly common associations between interview codes and affective patterns. We find common patterns involving student perceptions of difficulty, system helpfulness, and strategic behavior, and study them in greater depth. We find, for instance, that the experience of difficulty produces shifts from engaged concentration to frustration that lead students to adopt a variety of problem-solving strategies. We conclude with thoughts on both how this can influence the future design of AIED systems, and the broader potential uses of data mining-driven interviews in AIED research and development.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Grawemeyer, B., Wollenschlaeger, A., Santos, S.G., Holmes, W., Mavrikis, M., Poulovassilis, A.: Using Graph-based Modelling to explore changes in students’ affective states during exploratory learning tasks. In: Proceedings of the International Conference on Educational Data Mining, pp. 382–383 (2017)
DeFalco, J.A., et al.: Detecting and addressing frustration in a serious game for military training. Int. J. Artif. Intell. Educ. 28(2), 152–193 (2018)
Sottilare, R., Goldberg, B.: Designing adaptive computer-based tutoring systems to accelerate learning and facilitate retention. Cogn. Technol. 17(1), 19–33 (2012)
Forbes-Riley, K., Rotaru, M., Litman, D.: The relative impact of student affect on performance models in a spoken dialogue tutoring system. User Model. User-Adap. Inter. 18, 11–43 (2007)
D’Mello, S.K., Lehman, B., Person, N.: Monitoring affect states during effortful problem solving activities. Int. J. Artif. Intell. Educ. 20(4), 361–389 (2010)
Pardos, Z.A., Baker, R.S., San Pedro, M.O.C.Z., Gowda, S.M., Gowda, S.M.: Affective states and state tests: investigating how affect and engagement during the school year predict end of year learning outcomes. J. Learn. Anal. 1(1), 107–128 (2014)
Liu, Z., Pataranutaporn, V., Ocumpaugh, J., Baker, R.: Sequences of frustration and confusion, and learning. In: Proceedings of the International Conference on Educational Data Mining (2013)
Gee, J.P.: Good video games+ good learning: Collected essays on video games, learning, and literacy. Peter Lang Pub Incorporated, Bern, Switzerland (2007)
Richey, J.E., et al.: More confusion and frustration, better learning: the impact of erroneous examples. Comput. Educ. 139, 173–190 (2019)
Sabourin, J., Rowe, J.P., Mott, B.W., Lester, J.C.: When off-task is on-task: the affective role of off-task behavior in narrative-centered learning environments. In: Biswas, G., Bull, S., Kay, J., Mitrovic, A. (eds.) AIED 2011. LNCS (LNAI), vol. 6738, pp. 534–536. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-21869-9_93
Baker, R.S., Moore, G.R., Wagner, A.Z., Kalka, J., Salvi, A., Karabinos, M., Yaron, D.: The dynamics between student affect and behavior occurring outside of educational software. In: Proceedings of the International Conference on Affective Computing and Intelligent Interaction, pp. 14–24 (2011)
Valitutti, A.: Action decomposition and frustration regulation in the assisted execution of difficult tasks. In: Proceedings of the AIED 2009 Workshops, Brighton, UK (2009)
Miller, M.K., Mandryk, R.L.: Differentiating in-game frustration from at-game frustration using touch pressure. In: Proceedings of the 2016 ACM International Conference on Interactive Surfaces and Spaces, pp. 225–234 (2016)
McCuaig, J., Pearlstein, M., Judd, A.: Detecting learner frustration: towards mainstream use cases. In: Aleven, V., Kay, J., Mostow, J. (eds.) ITS 2010. LNCS, vol. 6095, pp. 21–30. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-13437-1_3
Buono, S., Zdravkovic, A., Lazic, M., Woodruff, E.: The effect of emotions on self-regulated-learning (SRL) and story comprehension in emerging readers. Front. Educ. 5, 218 (2020)
Huber, G.P., Power, D.J.: Retrospective reports of strategic-level managers: guidelines for increasing their accuracy. Strateg. Manag. J. 6(2), 171–180 (1985)
D’Mello, S.: A selective meta-analysis on the relative incidence of discrete affective states during learning with technology. J. Educ. Psychol. 105(4), 1082 (2013)
D’Mello, S., Graesser, A.: The half-life of cognitive-affective states during complex learning. Cogn. Emot. 25(7), 1299–1308 (2011)
Botelho, A.F., Baker, R., Ocumpaugh, J., Heffernan, N.: Studying affect dynamics and chronometry using sensor-free detectors. In: Proceedings of the 11th International Conference on Educational Data Mining, pp. 157–166 (2018)
Lazar, J., Bessiere, K., Ceaparu, I., Robinson, J., Shneiderman, B.: Help! I’m lost: user frustration in web navigation. IT Soc. 1(3), 18–26 (2003)
Taylor, B., Dey, A., Siewiorek, D., Smailagic, A.: Using physiological sensors to detect levels of user frustration induced by system delays. In: Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing, pp. 517–528 (2015)
Canossa, A., Drachen, A., Sørensen, J.R.M.: Arrrgghh!!! blending quantitative and qualitative methods to detect player frustration. In: Proceedings of the 6th International Conference on Foundations of Digital Games, pp. 61–68 (2011)
Leelawong, K., Biswas, G.: Designing learning by teaching agents: the Betty’s Brain system. Int. J. Artif. Intell. Educ. 18(3), 181–208 (2008)
Munshi, A., Rajendran, R., Ocumpaugh, J., Biswas, G., Baker, R., Paquette, L.: Modeling learners’ cognitive and affective states to scaffold SRL in open-ended learning environments. In: Proceedings of the 25th Conference on User Modeling, Adaptation, and Personalization, pp. 131–138 (2018)
Jiang, Y., et al.: Expert feature-engineering vs. deep neural networks: which is better for sensor-free affect detection? In: Proceedings of the International Conference on Artificial Intelligence in Education, pp. 198–211. Springer, Cham (2018)
Weston, C., Gandell, T., Beauchamp, J., McAlpine, L., Wiseman, C., Beauchamp, C.: Analyzing interview data: the development and evolution of a coding system. Qual. Sociol. 24(3), 381–400 (2001)
Charmaz, K.: The grounded theory method: an explication and interpretation. Contemp. Field Res. 109–126 (1983)
D’Mello, S., Graesser, A.: Dynamics of affective states during complex learning. Learn. Instr. 22(2), 145–157 (2012)
Winne, P.H., Hadwin, A.F.: Studying as self-regulated engagement in learning. In: Hacker, D., Dunlosky, J.,Hillsdale, G.A. (eds.) Metacognition in Educational Theory and Practice, Erlbaum, pp. 277–3048 (1998)
Andres, J.M.A.L., et al.: Affect sequences and learning in Betty's Brain. In: Proceedings of the 9th International Learning Analytics and Knowledge Conference, pp. 383–390 (2019)
Azevedo, R., Gašević, D.: Analyzing multimodal multichannel data about self-regulated learning with advanced learning technologies: issues and challenges. Comput. Hum. Behav. 96, 207–210 (2019)
Chi, M., VanLehn, K., Litman, D., Jordan, P.: Empirically evaluating the application of reinforcement learning to the induction of effective and adaptive pedagogical strategies. User Model. User-Adap. Inter. 21(1), 137–180 (2011)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Baker, R.S. et al. (2021). Affect-Targeted Interviews for Understanding Student Frustration. In: Roll, I., McNamara, D., Sosnovsky, S., Luckin, R., Dimitrova, V. (eds) Artificial Intelligence in Education. AIED 2021. Lecture Notes in Computer Science(), vol 12748. Springer, Cham. https://doi.org/10.1007/978-3-030-78292-4_5
Download citation
DOI: https://doi.org/10.1007/978-3-030-78292-4_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-78291-7
Online ISBN: 978-3-030-78292-4
eBook Packages: Computer ScienceComputer Science (R0)