Video-based learners’ observed attention estimates for lecture learning gain evaluation
- 622 Downloads
A significant problem in the field of higher education is maintaining learners’ attention during lectures, which is known to significantly affect their learning outcomes. Attention management is commonly associated with the individual ability of a lecturer to track and respond to the common behaviour of an auditorium; it lacks a detailed estimation of the intra-variability of individual learners’ attention during the course of the lecture. This paper suggests an objective and non-intrusive evaluation of learners’ attention against learning outcomes by introducing an observed attention estimate (OAE). The procedure uses human annotations based on visual cues with a supporting video recording/playback system and a web-based annotation system. This proposed procedure enables us to estimate the attention level of individual learners as observed by human annotators for given time intervals associated with specific concepts covered by the lecture. As part of the procedure, we use an inventory-based lecture gain evaluation and representation based on a novel learning gain matrix. This procedure allows for a detailed analysis of the lecture time flow with regard to learners’ attention. We have verified the applicability of the procedure on a small-scale case study.
KeywordsAttention metering Observed attention estimate Multimedia annotation system Lecture efficiency Learning management Learning outcomes Learning gain Teaching advisory systems
The authors would like to thank the students in the digital signal processing course at the University of Ljubljana for taking part in the study. Special thanks goes to John R. Buck and Kathleen E. Wage for giving us access to the SSCI used in this study.
This study was partially funded by Slovenian Research Agency (P2–0246 B).
Compliance with ethical standards
Conflict of interest
The authors declare that they have no conflict of interest.
- 1.Angelo, T. A., & Cross, K. P. (1993). Classroom Assessment Techniques: A Handbook for College Teachers. ERIC. doi: https://doi.org/10.2307/2943957
- 4.Bligh, D. A. (1998). What's the Use of Lectures? Intellect (UK) doi: https://doi.org/10.1080/03098268508708932
- 6.Chen C-M, Wang J-Y, Yu C-M (2015) Assessing the Attention Levels of Students by Using a Novel Attention Aware System based on Brainwave Signals. (T. Matsuo, K. Hashimoto, T. Mine, & S. Hirokawa, Eds.) Br J Educ Technol:379–384. https://doi.org/10.1109/IIAI-AAI.2015.224
- 8.von Eye A, Mun EY (2004) Analyzing rater agreement: Manifest variable methods. Lawrence Erlbaum Associates Publishers. doi: https://doi.org/10.4324/9781410611024
- 10.Gibbs G (2011) Student attention over an hour. https://epigeum.com/downloads/uct_accessible/uk/01_lecturing1/html/course_files/2_30.html. Accessed 27 February 2017
- 11.Graesser A, McDaniel B, Chipman P, Witherspoon A, D'Mello SK, Gholson B (2006) Detection of emotions during learning with AutoTutor. In: Sun R, Miyake N (eds) Proceedings of the 28th Annual Conference of the Cognitive Science Society. Cognitive Science Society, Austin, pp 285–290Google Scholar
- 16.Johnstone AH, Percival F (1976) Attention Breaks in Lectures. Educ Chem 13:49–50Google Scholar
- 17.Kondermann, D. (2013). Ground Truth Design Principles: An Overview. Proceedings of the International Workshop on Video and Image Ground Truth in Computer Vision Applications (pp. 5:1--5:4). New York: Association for Computing Machinery (ACM). doi: https://doi.org/10.1145/2501105.2501114
- 18.Krippendorff K (2004) Content Analysis: An Introduction to Its Methodology. 2nd Ed. SAGE Publications, London pp 413Google Scholar
- 23.Wilcox RR (2003) Applying Contemporary Statistical Techniques. Academic Press, London, pp 608Google Scholar
- 25.Porta, M., Ricotti, S., & Perez, C. J. (2012). Emotional e-learning through eye tracking. Proceedings of the 2012 I.E. Global Engineering Education Conference (EDUCON). Institute of Electrical and Electronics Engineers (IEEE). doi: https://doi.org/10.1109/educon.2012.6201145
- 34.Yannakakis, G. N., & Martinez, H. P. (2015). Grounding truth via ordinal annotation. 2015 International Conference on Affective Computing and Intelligent Interaction (ACII) (pp. 574-580). Institute of Electrical and Electronics Engineers (IEEE). doi: https://doi.org/10.1109/acii.2015.7344627