Skip to main content

Analysis of the “D’oh!” Moments. Physiological Markers of Performance in Cognitive Switching Tasks

  • 877 Accesses

Part of the Lecture Notes in Computer Science book series (LNISA,volume 12884)

Abstract

The link between the body and mind has fascinated philosophers and scientists for ages. The increasing availability of sensor technologies has enabled the possibility to explore this link even deeper, providing some evidence that certain physiological measurements such as galvanic skin response can have in the performance of different learning activities. In this paper, we explore the link between learners’ performance of cognitive tasks and their physiological state with the use of Multimodal Learning Analytics (MMLA). We used MMLA tools and techniques to collect, annotate, and analyse physiological data from 16 participants wearing an Empatica E4 wristband while engaging in task-switching cognitive exercises. The collected data include temperature, blood volume pulse, heart rate variability, galvanic skin response, and screen recording from each participant while performing the exercises. To examine the link between cognitive performance we applied a preliminary qualitative analysis to galvanic skin response and tested different Artificial Intelligence techniques to differentiate between productive and unproductive performance.

Keywords

  • Multimodal learning analytics
  • Psychophysiology
  • Game analytics
  • Sensors

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-86436-1_11
  • Chapter length: 12 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   84.99
Price excludes VAT (USA)
  • ISBN: 978-3-030-86436-1
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   109.99
Price excludes VAT (USA)
Fig. 1.
Fig. 2.

Notes

  1. 1.

    https://www.empatica.com/research/e4/.

  2. 2.

    https://tsfresh.com/.

References

  1. James, W.: The Principles of Psychology, vol. 1. Cosimo, Inc. (2007)

    Google Scholar 

  2. Marmolejo-Ramos, F., et al.: Your face and moves seem happier when i smile. Exp. Psychol. (2020)

    Google Scholar 

  3. Duncan, J.W., Laird, J.D.: Positive and reverse placebo effects as a function of differences in cues used in self-perception. J. Pers. Soc. Psychol. 39(6), 1024 (1980)

    CrossRef  Google Scholar 

  4. Foglia, L., Wilson, R.A.: Embodied cognition. Wiley Interdisc. Rev. Cogn. Sci. 4(3), 319–325 (2013)

    CrossRef  Google Scholar 

  5. Derakshan, N., Eysenck, M.W.: Anxiety, processing efficiency, and cognitive performance (2009)

    Google Scholar 

  6. Sandi, C.: Stress and cognition. Wiley Interdisc. Rev. Cogn. Sci. 4(3), 245–261 (2013)

    CrossRef  Google Scholar 

  7. Raab, M., Araújo, D.: Embodied cognition with and without mental representations: the case of embodied choices in sports. Front. Psychol. 10, 1825 (2019)

    CrossRef  Google Scholar 

  8. Greller, W., Drachsler, H.: Translating learning into numbers: a generic framework for learning analytics. J. Educ. Technol. Soc. 15(3), 42–57 (2012)

    Google Scholar 

  9. Pardo, A., Kloos, C. D.: Stepping out of the box: towards analytics outside the learning management system. In: Proceedings of the 1st International Conference on Learning Analytics and Knowledge, pp. 163–167 (2011)

    Google Scholar 

  10. Di Mitri, D., Schneider, J., Specht, M., Drachsler, H.: From signals to knowledge: a conceptual model for multimodal learning analytics. J. Comput. Assist. Learn. 34(4), 338–349. Ew developments from attentional control theory. Eur. Psychol. 14(2), 168 (2018)

    Google Scholar 

  11. Giannakos, M.N., Sharma, K., Pappas, I.O., Kostakos, V., Velloso, E.: Multimodal data as a means to understand the learning experience. Int. J. Inf. Manage. 48, 108–119 (2019)

    CrossRef  Google Scholar 

  12. Sharma, K., Pappas, I., Papavlasopoulou, S., Giannakos, M.: Towards automatic and pervasive physiological sensing of collaborative learning. In: Lund, K., Niccolai, G.P., Lavoué, E., Gweo, C.H., Baker, M. (eds.) Thirteenth International Conference on Computer Supported Collaborative Learning (CSCL), pp. 684–687 (2019)

    Google Scholar 

  13. Chanel, G., Bétrancourt, M., Pun, T., Cereghetti, D., Molinari, G.: Assessment of computer-supported collaborative processes using interpersonal physiological and eye-movement coupling. In Proceedings of Humaine Association Conference on Affective Computing and Intelligent Interaction, pp. 116–122 (2013)

    Google Scholar 

  14. Spann, C.A., Schaeffer, J., Siemens, G.: Expanding the scope of learning analytics data: preliminary findings on attention and self-regulation using wearable technology. In: LAK17, pp. 203–207 (2017)

    Google Scholar 

  15. Bleck, M., Le, N.T., Pinkwart, N.: Physiology-aware learning analytics using pedagogical agents (2020)

    Google Scholar 

  16. Pijeira-Díaz, H.J., Drachsler, H., Kirschner, P.A., Järvelä, S.: Profiling sympathetic arousal in a physics course: How active is students? J. Comput. Assist. Learn. 34(4), 397–408 (2018)

    CrossRef  Google Scholar 

  17. Larmuseau, C., Vanneste, P., Cornelis, J., Desmet, P., Depaepe, F.: Combining physiological data and subjective measurements to investigate cognitive load during complex learning. Frontline Learn. Res. 7(2), 57–74 (2019)

    CrossRef  Google Scholar 

  18. Worsley, M., Blikstein, P.: What’s an expert? Using learning analytics to identify emergent markers of expertise through automated speech, sentiment and sketch analysis. In: EDM 2011, pp. 235–240 (2011)

    Google Scholar 

  19. Sharma, K., Niforatos, E., Giannakos, M., Kostakos, E.: Assessing cognitive performance using physiological and facial features: generalizing across contexts. Proc. ACM Interact. Mob. Wearable Ubiquit. Technol. 4(3), 1–41 (2020)

    Google Scholar 

  20. Ryu, K., Myung, R.: Evaluation of mental workload with a combined measure based on physiological indices during a dual task of tracking and mental arithmetic. Int. J. Ind. Ergon. 35(11), 991–1009 (2005)

    CrossRef  Google Scholar 

  21. Sharma, K., Giannakos, M.: Multimodal data capabilities for learning: what can multimodal data tell us about learning? Br. J. Edu. Technol. 51(5), 1450–1484 (2020)

    CrossRef  Google Scholar 

  22. Scott, W.A.: Cognitive complexity and cognitive flexibility. Sociometry 405–414 (1962)

    Google Scholar 

  23. Jacob, R., Parkinson, J.: The potential for school-based interventions that target executive function to improve academic achievement: a review. Rev. Educ. Res. 85(4), 512–552 (2015)

    CrossRef  Google Scholar 

  24. Di Mitri, D., Schneider, J., Specht, M.M., Drachsler, H. J.: Multimodal pipeline: a generic approach for handling multimodal data for supporting learning. In: First workshop on AI-based Multimodal Analytics for Understanding Human Learning in Real-world Educational Contexts (2019)

    Google Scholar 

  25. Chelune, G.J., Baer, R.A.: Developmental norms for the Wisconsin card sorting test. J. Clin. Exp. Neuropsychol. 8(3), 219–228 (1986)

    CrossRef  Google Scholar 

  26. Schneider, J., Di Mitri, D., Limbu, B., Drachsler, H.: Multimodal learning hub: a tool for capturing customizable multimodal learning experiences. In: Pammer-Schindler, V., Pérez-Sanagustín, M., Drachsler, H., Elferink, R., Scheffel, M. (eds.) EC-TEL 2018. LNCS, vol. 11082, pp. 45–58. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-98572-5_4

    CrossRef  Google Scholar 

  27. Di Mitri, D., Schneider, J., Klemke, R., Specht, M., Drachsler, H. : Read between the lines: an annotation tool for multimodal data for learning. In: LAK19, pp. 51–60 (2019)

    Google Scholar 

  28. Blasiak, S., Rangwala, H.: A hidden Markov model variant for sequence classification. In: Twenty-Second International Joint Conference on Artificial Intelligence (2011)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Jan Schneider .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

Buraha, T., Schneider, J., Di Mitri, D., Schiffner, D. (2021). Analysis of the “D’oh!” Moments. Physiological Markers of Performance in Cognitive Switching Tasks. In: De Laet, T., Klemke, R., Alario-Hoyos, C., Hilliger, I., Ortega-Arranz, A. (eds) Technology-Enhanced Learning for a Free, Safe, and Sustainable World. EC-TEL 2021. Lecture Notes in Computer Science(), vol 12884. Springer, Cham. https://doi.org/10.1007/978-3-030-86436-1_11

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-86436-1_11

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-86435-4

  • Online ISBN: 978-3-030-86436-1

  • eBook Packages: Computer ScienceComputer Science (R0)