Abstract
We know from the literature that high cognitive load can impede performance and educational outcomes. Previous measures of cognitive load have largely relied on subjective scales but few have explored more objective measures. This paper aims to address this issue by examining the validity of electroencephalography (or EEG) as an objective measure of cognitive load in the context of educational video. EEG is an online brain measure of the electrical activity that a population of neurons elicits when they are activated. Theta power is a brain oscillation ranging from 4 to 7 Hz that has been linked to cognitive and memory performance. We manipulated the linguistic complexity of three texts delivered by a speaker on video to induce a simple, a moderate and a complex level of cognitive load amongst a sample of students pursuing postgraduate studies through the medium of English as a second language (n = 35). We specifically measured intrinsic cognitive load via a subjective scale, a recall test, and theta power as an objective measure (average cognitive load). Our results show that the subjective scale was significantly different across the three levels of linguistic complexity. However, theta power and recall were only significantly different between the two most distinct levels of linguistic complexity. Although, this finding suggests theta power may not be as sensitive for average cognitive load as the subjective scale for intrinsic cognitive load, a correlation analysis shows that theta power was correlated with self-reported cognitive load. However, it may need more power compared to the subjective scale. These findings suggest that theta power may be developed into a valid objective measure of average cognitive load although its true potential lies in the possibility to measure online fluctuations in cognitive load or instantaneous cognitive load.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data availability
Data are available at: https://osf.io/ca6kt/.
Notes
For the aims of another study, not this one, the contacts for FC5 and FC6 were swapped with those for P3 and P4, the latter locations normally being used as alternative reference electrodes in addition to the mastoid reference electrodes.
References
Anderson, E. W., Potter, K. C., Matzen, L. E., Shepherd, J. F., Preston, G. A., & Silva, C. T. (2011). A user study of visualization effectiveness using EEG and cognitive load. Computer Graphics Forum,30(3), 791–800. https://doi.org/10.1111/j.1467-8659.2011.01928.x.
Anmarkrud, Ø., Andresen, A., & Bråten, I. (2019). Cognitive load and working memory in multimedia learning: Conceptual and measurement issues. Educational Psychologist. https://doi.org/10.1080/00461520.2018.1554484.
Antonenko, P. D., & Niederhauser, D. S. (2010). The influence of leads on cognitive load and learning in a hypertext environment. Computers in Human Behavior,26(2), 140–150. https://doi.org/10.1016/j.chb.2009.10.014.
Antonenko, P. D., Paas, F., Grabner, R., & van Gog, T. (2010). Using electroencephalography to measure cognitive load. Educational Psychology Review,22(4), 425–438. https://doi.org/10.1007/s10648-010-9130-y.
Badcock, N. A., Mousikou, P., Mahajan, Y., de Lissa, P., Thie, J., & McArthur, G. (2013). Validation of the Emotiv EPOC® EEG gaming system for measuring research quality auditory ERPs. PeerJ,1, e38. https://doi.org/10.7717/peerj.38.
Badcock, N. A., Preece, K. A., de Wit, B., Glenn, K., Fieder, N., Thie, J., et al. (2015). Validation of the Emotiv EPOC EEG system for research quality auditory event-related potentials in children. PeerJ,3, e907. https://doi.org/10.7717/peerj.907.
Baddeley, A. (2012). Working memory: Theories, models, and controversies. Annual Review of Psychology,63(1), 1–29. https://doi.org/10.1146/annurev-psych-120710-100422.
Barry, R. J., Clarke, A. R., Johnstone, S. J., Magee, C. A., & Rushby, J. A. (2007). EEG differences between eyes-closed and eyes-open resting conditions. Clinical Neurophysiology,118(12), 2765–2773. https://doi.org/10.1016/j.clinph.2007.07.028.
Berman, R., & Cheng, L. (2010). English academic language skills: Perceived difficulties by undergraduate and graduate students, and their academic achievement. Canadian Journal of Applied Linguistics/Revue canadienne de linguistique appliquée,4(1), 25–40.
Castro-Meneses, L. J., Johnson, B. W., & Sowman, P. F. (2016). Event-related fields evoked by vocal response inhibition: A comparison of younger and older adults. Experimental Brain Research,234(6), 1525–1535.
Cohen, M. X. (2014). Analyzing neural time series data: Theory and practice. Cambridge: MIT Press.
de Lissa, P., Sörensen, S., Badcock, N., Thie, J., & McArthur, G. (2015). Measuring the face-sensitive N170 with a gaming EEG system: A validation study. Journal of Neuroscience Methods,253, 47–54. https://doi.org/10.1016/j.jneumeth.2015.05.025.
DeLeeuw, K. E., & Mayer, R. E. (2008). A comparison of three measures of cognitive load: Evidence for separable measures of intrinsic, extraneous, and germane load. Journal of Educational Psychology,100(1), 223–234. https://doi.org/10.1037/0022-0663.100.1.223.
Delorme, A., & Makeig, S. (2004). EEGLAB: An open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. Journal of Neuroscience Methods,134(1), 9–21. https://doi.org/10.1016/j.jneumeth.2003.10.009.
Department of Education and Training. (2017). International Student Data, Monthly summary (March-2017). Retrieved June 26, 2019 from https://internationaleducation.gov.au/research/International-Student-Data/Documents/MONTHLY%20SUMMARIES/2017/Mar%202017%20MonthlyInfographic.pdf.
Field, A. (2013). Discovering statistics using IBM SPSS statistics (4th ed.). Thousand Oaks: Sage.
Gerě, I., & Jaušcvec, N. (1999). Multimedia: Differences in cognitive processes observed with EEG. Educational Technology Research and Development,47(3), 5–14. https://doi.org/10.1007/bf02299630.
Gevins, A., & Smith, M. E. (2000). Neurophysiological measures of working memory and individual differences in cognitive ability and cognitive style. Cerebral Cortex,10(9), 829–839. https://doi.org/10.1093/cercor/10.9.829.
Gevins, A., & Smith, M. E. (2003). Neurophysiological measures of cognitive workload during human-computer interaction. Theoretical Issues in Ergonomics Science,4(1–2), 113–131. https://doi.org/10.1080/14639220210159717.
Gevins, A., Smith, M. E., McEvoy, L., & Yu, D. (1997). High-resolution EEG mapping of cortical activation related to working memory: Effects of task difficulty, type of processing, and practice. Cerebral Cortex,7(4), 374–385. https://doi.org/10.1093/cercor/7.4.374.
Graesser, A. C., McNamara, D. S., Louwerse, M. M., & Cai, Z. (2004). Coh-Metrix: Analysis of text on cohesion and language. Behavior research methods, instruments, & computers,36(2), 193–202.
Greene, J. P. (2004). Pursuits of happiness: The social development of early modern British colonies and the formation of American culture. Chapel Hill: Univ of North Carolina Press.
Guo, P. J., Kim, J., & Rubin, R. (2014). How video production affects student engagement: an empirical study of MOOC videos. Paper presented at the Proceedings of the first ACM conference on Learning @ scale conference, Atlanta, Georgia, USA.
Jensen, O., Gelfand, J., Kounios, J., & Lisman, J. E. (2002). Oscillations in the alpha band (9-12 Hz) increase with memory load during retention in a short-term memory task. Cerebral Cortex (New York, N.Y.: 1991),12(8), 877–882. https://doi.org/10.1093/cercor/12.8.877.
Jensen, O., & Tesche, C. D. (2002). Frontal theta activity in humans increases with memory load in a working memory task. European Journal of Neuroscience,15(8), 1395–1399. https://doi.org/10.1046/j.1460-9568.2002.01975.x.
Kahana, M. J., Sekuler, R., Caplan, J. B., Kirschen, M., & Madsen, J. R. (1999). Human theta oscillations exhibit task dependence during virtual maze navigation. Nature,399, 781. https://doi.org/10.1038/21645.
Kalyuga, S. (2012). Instructional benefits of spoken words: A review of cognitive load factors. Educational Research Review,7(2), 145–159. https://doi.org/10.1016/j.edurev.2011.12.002.
Kalyuga, S., Chandler, P., & Sweller, J. (2000). Incorporating learner experience into the design of multimedia instruction. Journal of Educational Psychology,92(1), 126–136. https://doi.org/10.1037/0022-0663.92.1.126.
Kalyuga, S., Chandler, P., & Sweller, J. (2004). When redundant on-screen text in multimedia technical instruction can interfere with learning. Human Factors,46(3), 567–581. https://doi.org/10.1518/hfes.46.3.567.50405.
Kay, R. H. (2012). Exploring the use of video podcasts in education: A comprehensive review of the literature. Computers in Human Behavior,28(3), 820–831. https://doi.org/10.1016/j.chb.2012.01.011.
Khader, P. H., Jost, K., Ranganath, C., & Rösler, F. (2010). Theta and alpha oscillations during working-memory maintenance predict successful long-term memory encoding. Neuroscience Letters,468(3), 339–343. https://doi.org/10.1016/j.neulet.2009.11.028.
Klimesch, W. (1999). EEG alpha and theta oscillations reflect cognitive and memory performance: A review and analysis. Brain Research. Brain Research Reviews,29(2–3), 169–195.
Klimesch, W., Doppelmayr, M., Schwaiger, J., Auinger, P., & Winkler, T. (1999). `Paradoxical’ alpha synchronization in a memory task. Cognitive Brain Research,7(4), 493–501. https://doi.org/10.1016/S0926-6410(98)00056-1.
Klimesch, W., Schack, B., & Sauseng, P. (2005). The functional significance of theta and upper alpha oscillations. Experimental Psychology,52(2), 99–108. https://doi.org/10.1027/1618-3169.52.2.99.
Krause, C. M., Heikki Lang, A., Laine, M., Kuusisto, M., & Pörn, B. (1996). Event-related. EEG desynchronization and synchronization during an auditory memory task. Electroencephalography and Clinical Neurophysiology,98(4), 319–326. https://doi.org/10.1016/0013-4694(96)00283-0.
Kruger, J.-L., & Doherty, S. (2016). Measuring cognitive load in the presence of educational video: Towards a multimodal methodology. Australasian Journal of Educational Technology. https://doi.org/10.14742/ajet.3084.
Leppink, J., Paas, F., van Gog, T., van der Vleuten, C. P. M., & van Merriënboer, J. J. G. (2014). Effects of pairs of problems and examples on task performance and different types of cognitive load. Learning and Instruction,30, 32–42. https://doi.org/10.1016/j.learninstruc.2013.12.001.
Leppink, J., Paas, F., Vleuten, C., Gog, T., & Merriënboer, J. (2013). Development of an instrument for measuring different types of cognitive load. Behavior Research Methods,45(4), 1058–1072. https://doi.org/10.3758/s13428-013-0334-1.
Leppink, J., & van den Heuvel, A. (2015). The evolution of cognitive load theory and its application to medical education. Perspectives on Medical Education,4(3), 119–127. https://doi.org/10.1007/s40037-015-0192-x.
Luck, S. J. (2014). An introduction to the event-related potential technique. Cambridge: MIT press.
Marian, V., Blumenfeld, H. K., & Kaushanskaya, M. (2007). The Language Experience and Proficiency Questionnaire (LEAP-Q): Assessing language profiles in bilinguals and multilinguals. Journal of Speech, Language, and Hearing Research,50(4), 940–967. https://doi.org/10.1044/1092-4388(2007/067).
Mayer, R. E. (2002). Multimedia learning psychology of learning and motivation (Vol. 41, pp. 85–139). Cambridge: Academic Press.
Mayer, R. E. (2014). The Cambridge handbook of multimedia learning (2nd ed.). Cambridge: Cambridge University Press.
McMahon, C. M., Boisvert, I., de Lissa, P., Granger, L., Ibrahim, R., Lo, C. Y., et al. (2016). Monitoring alpha oscillations and pupil dilation across a performance-intensity function. Frontiers in Psychology,7, 745. https://doi.org/10.3389/fpsyg.2016.00745.
Onton, J., Delorme, A., & Makeig, S. (2005). Frontal midline EEG dynamics during working memory. NeuroImage,27(2), 341–356. https://doi.org/10.1016/j.neuroimage.2005.04.014.
Paas, F. (1992). Training strategies for attaining transfer of problem-solving skill in statistics: A cognitive-load approach. Journal of Educational Psychology,84(4), 429. https://doi.org/10.1037/0022-0663.84.4.429.
Paas, F., & Merrienboer, J. (2004). Variability of worked examples and transfer of geometrical problem-solving skills: A cognitive-load approach. Journal of Educational Psychology,86(1), 122.
Paas, F., Tuovinen, J. E., Tabbers, H., & Van Gerven, P. W. (2003). Cognitive load measurement as a means to advance cognitive load theory. Educational Psychologist,38(1), 63–71. https://doi.org/10.1207/S15326985EP3801_8.
Paas, F., Van Merriënboer, J. J., & Adam, J. J. (1994). Measurement of cognitive load in instructional research. Perceptual and Motor Skills,79(1), 419–430.
Pfurtscheller, G., & Lopes da Silva, F. H. (1999). Event-related EEG/MEG synchronization and desynchronization: basic principles. Clinical Neurophysiology,110(11), 1842–1857. https://doi.org/10.1016/S1388-2457(99)00141-8.
Plass, J. L., Moreno, R., & Brünken, R. (2010). Cognitive load theory. Cambridge: Cambridge University Press.
Rouet, J.-F., Lowe, R., & Schnotz, W. (2008). Understanding multimedia documents. Ney York: Springer.
Roux, F., & Uhlhaas, P. J. (2014). Working memory and neural oscillations: Alpha–gamma versus theta–gamma codes for distinct WM information? Trends in Cognitive Sciences,18(1), 16–25. https://doi.org/10.1016/j.tics.2013.10.010.
Rubin, J. (1994). A review of second language listening comprehension research. The Modern Language Journal,78(2), 199–221. https://doi.org/10.1111/j.1540-4781.1994.tb02034.x.
Scharinger, C., Kammerer, Y., & Gerjets, P. (2015). Pupil dilation and EEG alpha frequency band power reveal load on executive functions for link-selection processes during text reading. PLoS ONE,10(6), e0130608. https://doi.org/10.1371/journal.pone.0130608.
Sterman, M., Mann, C., & Kaiser, D. (1993). Quantitative EEG patterns of differential in-flight workload. Paper presented at the Sixth Annual Workshop on Space Operations Applications and Research (SOAR, 1992). https://ntrs.nasa.gov/search.jsp?R=19940007056.
Sweller, J. (1988). Cognitive load during problem solving: Effects on learning. Cognitive Science,12(2), 257–285. https://doi.org/10.1207/s15516709cog1202_4.
Sweller, J. (2010). Element interactivity and intrinsic, extraneous, and Germane cognitive load. Educational Psychology Review,22(2), 123–138. https://doi.org/10.1007/s10648-010-9128-5.
Sweller, J., Ayres, P., & Kalyuga, S. (2011). Cognitive load theory (Vol. 1). New York: Springer.
Sweller, J., van Merrienboer, J., & Paas, F. (1998). Cognitive architecture and instructional design. Educational Psychology Review,10(3), 251–296. https://doi.org/10.1023/A:1022193728205.
Tabbers, H. K., Martens, R. L., & Van Merriënboer, J. J. G. (2004). Multimedia instructions and cognitive load theory: Effects of modality and cueing. British Journal of Educational Psychology,74(1), 71–81. https://doi.org/10.1348/000709904322848824.
Tasker, S., Brezmen, M., & Publications, R.-E. (2003). Aussie, aussie, aussie. Greenwood: Ready-Ed Publications.
Van Overschelde, J. P., & Healy, A. (2001). Learning of nondomain facts in high- and low-knowledge domains. Journal Of Experimental Psychology-Learning Memory And Cognition,27(5), 1160–1171. https://doi.org/10.1037/0278-7393.27.5.1160.
Vigário, R. N. (1997). Extraction of ocular artefacts from EEG using independent component analysis. Electroencephalography and Clinical Neurophysiology,103(3), 395–404. https://doi.org/10.1016/S0013-4694(97)00042-8.
Xie, B., & Salvendy, G. (2000). Review and reappraisal of modelling and predicting mental workload in single- and multi-task environments. Work & Stress,14(1), 74–99. https://doi.org/10.1080/026783700417249.
Zarjam, P. (2013). Estimation of working memory load using EEG signal processing. Sydney: The University of New South Wales.
Funding
This project was jointly funded by the Department of Linguistics, Macquarie University and the School of Humanities & Languages in the University of New South Wales.
Author information
Authors and Affiliations
Corresponding authors
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Castro-Meneses, L.J., Kruger, JL. & Doherty, S. Validating theta power as an objective measure of cognitive load in educational video. Education Tech Research Dev 68, 181–202 (2020). https://doi.org/10.1007/s11423-019-09681-4
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11423-019-09681-4