Face Forward: Detecting Mind Wandering from Video During Narrative Film Comprehension
Attention is key to effective learning, but mind wandering, a phenomenon in which attention shifts from task-related processing to task-unrelated thoughts, is pervasive across learning tasks. Therefore, intelligent learning environments should benefit from mechanisms to detect and respond to attentional lapses, such as mind wandering. As a step in this direction, we report the development and validation of the first student-independent facial feature-based mind wandering detector. We collected training data in a lab study where participants self-reported when they caught themselves mind wandering over the course of completing a 32.5 min narrative film comprehension task. We used computer vision techniques to extract facial features and bodily movements from videos. Using supervised learning methods, we were able to detect a mind wandering with an F1 score of .390, which reflected a 31% improvement over a chance model. We discuss how our mind wandering detector can be used to adapt the learning experience, particularly for online learning contexts.
KeywordsMind wandering Attention aware interfaces
This research was supported by the National Science Foundation (NSF) (DRL 1235958 and IIS 1523091). Any opinions, findings and conclusions, or recommendations expressed in this paper are those of the authors and do not necessarily reflect the views of the NSF.
- 9.Blanchard, N., Bixler, R., Joyce, T., D’Mello, S.: Automated physiological-based detection of mind wandering during learning. In: Trausan-Matu, S., Boyer, K.E., Crosby, M., Panourgia, K. (eds.) ITS 2014. LNCS, vol. 8474, pp. 55–60. Springer, Cham (2014). doi: 10.1007/978-3-319-07221-0_7 CrossRefGoogle Scholar
- 11.Mills, C., Bixler, R., Wang, X., D’Mello, S.K.: Automatic gaze-based detection of mind wandering during film viewing. In: Proceedings of the 9th International Conference on Educational Data Mining. International Educational Data Mining Society, Raleigh (2016)Google Scholar
- 13.Faber, M., Bixler, R., D’Mello, S.K.: An automated behavioral measure of mind wandering during computerized reading. Behav. Res. Methods 1–17 (2017)Google Scholar
- 15.Littlewort, G., Whitehill, J., Wu, T., Fasel, I., Frank, M., Movellan, J., Bartlett, M.: The computer expression recognition toolbox (CERT). In: 2011 IEEE International Conference on Automatic Face & Gesture Recognition and Workshops (FG 2011), pp. 298–305. IEEE (2011)Google Scholar
- 16.Ekman, P., Friesen, W.V.: Facial action coding system (1977)Google Scholar
- 19.Allison, P.D.: Multiple Regression: A Primer. Pine Forge Press, Thousand Oaks (1999)Google Scholar
- 21.Holmes, G., Donkin, A., Witten, I.H.: Weka: a machine learning workbench. In: Proceedings of the 1994 Second Australian and New Zealand Conference on Intelligent Information Systems, pp. 357–361. IEEE (1994)Google Scholar
- 22.Platt, J., et al.: Sequential minimal optimization: a fast algorithm for training support vector machines (1998)Google Scholar