Advertisement

Interdisciplinary Research Methods Used to Investigate Emotions with Advanced Learning Technologies

  • Roger AzevedoEmail author
  • Michelle Taub
  • Nicholas Mudrick
  • Jesse Farnsworth
  • Seth A. Martin
Chapter

Abstract

This chapter provides a synthesis of several research methods used by interdisciplinary researchers to investigate emotions in advanced learning technologies. More specifically, the authors: (1) critique self-report measures used to investigate emotions; (2) briefly describe Scherer’s (2009) model as particularly relevant for investigating emotions due to its complex appraisal system; (3) describe three process-oriented methods (electrodermal activity, facial expressions, and eye-tracking) currently used by interdisciplinary researchers to detect, identify, and classify affective states during learning and briefly highlight strengths and weaknesses of each method; and (4) present major conceptual, methodological, and analytical issues related to investigating emotions during learning with advanced learning technologies.

Keywords

Emotions Affect Advanced learning technologies Measurement Research methods Processes Affect dynamics Temporal sequencing Physiological measures Facial expressions Eye-tracking Models Learning Pedagogical agents Self-regulated learning 

Notes

Acknowledgments

The work in this chapter was supported in part by the National Science Foundation, Institute of Education Sciences, Social Sciences and Humanities Research Council of Canada, and Natural Sciences and Engineering Research Council of Canada.

References

  1. Azevedo, R. (2015). Defining and measuring engagement and learning in science: Conceptual, theoretical, methodological, and analytical issues. Educational Psychologist, 50, 84–94.CrossRefGoogle Scholar
  2. Azevedo, R., & Aleven, V. (Eds.). (2013). International handbook of metacognition and learning technologies. Amsterdam, The Netherlands: Springer.Google Scholar
  3. Azevedo, R., & Chauncey Strain, A. (2011). Integrating cognitive, metacognitive, and affective regulatory processes with MetaTutor. In R. Calvo & S. D’Mello (Eds.), New perspectives on affect and learning technologies (pp. 141–154). Amsterdam, The Netherlands: Springer.CrossRefGoogle Scholar
  4. Azevedo, R., Harley, J., Trevors, G., Duffy, M., Feyzi-Behnagh, R., Bouchet, F., & Landis, R. (2013). Using trace data to examine the complex roles of cognitive, metacognitive, and emotional self-regulatory processes during learning with multi-agent systems. In R. Azevedo & V. Aleven (Eds.), International handbook of metacognition and learning technologies (pp. 427–449). Amsterdam, The Netherlands: Springer.CrossRefGoogle Scholar
  5. Baker, R., & Siemens, G. (2014). Educational data mining and learning analytics. In K. Sawyer (Ed.), Cambridge handbook of the learning sciences (2nd ed., pp. 253–274). Cambridge, MA: Cambridge University Press.CrossRefGoogle Scholar
  6. Barrett, L., & Russell, J. (2015). The psychological construction of emotion. New York, NY: Guilford.Google Scholar
  7. Bondareva, D., Conati, C., Feyzi-Behnagh, R., Harley, J., Azevedo, R., & Bouchet, F. (2013). Inferring learning from gaze data during interaction with an environment to support self-regulated learning. In H. C. Lane, K. Yacef, J. Mostow, & P. Pavlik (Eds.), Lecture Notes in Computer Science, Vol. 7926. Artificial Intelligence in Education (pp. 229–238). Berlin, Germany: Springer.Google Scholar
  8. Calvo, R., & D’Mello, S. (2010). Affect detection: An interdisciplinary review of models, methods, and their applications. IEEE Transactions on Affective Computing, 1, 18–37.CrossRefGoogle Scholar
  9. Calvo, R., D’Mello, S., Gratch, J., & Kappas, A. (Eds.). (2014). The Oxford handbook of affective computing. New York, NY: Oxford University Press.Google Scholar
  10. Chauncey Strain, A., Azevedo, R., & D’Mello, S. (2013). Using a false biofeedback methodology to explore relationships among learners’ affect, metacognition, and performance. Contemporary Educational Psychology, 38, 22–39.CrossRefGoogle Scholar
  11. D’Mello, S. (2013). A selective meta-analysis on the relative incidence of discrete affective states during learning with technology. Journal of Educational Psychology, 105, 1082–1099.CrossRefGoogle Scholar
  12. D’Mello, S., & Graesser, A. (2010). Multimodal semi-automated affect detection from conversational cues, gross body language, and facial features. User Modeling and User-Adapted Interaction, 20, 147–187.CrossRefGoogle Scholar
  13. D’Mello, S., & Graesser, A. (2015). Feeling, thinking, and computing with affect-aware learning technologies. In R. Calvo, S. D’Mello, J. Gratch, & A. Kappas (Eds.), The Oxford handbook of affective computing (pp. 419–434). New York, NY: Oxford University Press.Google Scholar
  14. Efklides, A. (2011). Interactions of metacognition with motivation and affect in self-regulated learning: The MASRL model. Educational Psychologist, 46, 6–25.CrossRefGoogle Scholar
  15. Ekman, O., & Friesen, W. (1978). Facial action coding system: A technique for the measurement of facial movement. Palo Alto, CA: Consulting Psychologists Press.Google Scholar
  16. Figner, B., & Murphy, R. (2011). Using skin conductance in judgment and decision making research. In M. Schulte-Mecklenbeck, A. Kuehberger, & R. Ranyard (Eds.), A handbook of process tracing methods for decision research. New York, NY: Psychology Press.Google Scholar
  17. Graesser, A., & D’Mello, S. (2014). Emotions in advanced learning technologies. In R. Pekrun & L. Linnenbrink-Garcia (Eds.), International handbook of emotions in education (pp. 473–493). New York, NY: Routledge.Google Scholar
  18. Graesser, A., D’Mello, S., & Chauncey Strain, A. (2014). Emotions in advanced learning technologies. In R. Pekrun & L. Linnenbrink-Garcia (Eds.), Handbook of emotions and education (pp. 473–493). New York, NY: Taylor & Francis.Google Scholar
  19. Grafsgaard, J., Wiggins, J., Vail, A., Boyer, K., Wiebe, K., & Lester, J. (2014). The additive value of multimodal features for predicting engagement, frustration, and learning during tutoring. In Proceedings of the Sixteenth ACM International Conference on Multimodal Interaction (pp. 42–49).Google Scholar
  20. Gratch, J., & Marsella, S. (2014). Appraisal models. In R. Calvo, S. D’Mello, J. Gratch, & A. Kappas (Eds.), The Oxford handbook of affective computing (pp. 54–67). New York, NY: Oxford University Press.Google Scholar
  21. Gross, J. (2015). Emotion regulation: Current status and future prospects. Psychological Inquiry, 26, 1–26.CrossRefGoogle Scholar
  22. Harley, J., Bouchet, F., Hussain, S., Azevedo, R., & Calvo, R. (2015). A multi-componential analysis of emotions during complex learning with an intelligent multi-agent system. Computers in Human Behavior, 48, 615–625.CrossRefGoogle Scholar
  23. Healey, J. (2014). Physiological sensing of emotion. In R. Calvo, S. D’Mello, J. Gratch, & A. Kappas (Eds.), The Oxford handbook of affective computing (pp. 204–216). New York, NY: Oxford University Press.Google Scholar
  24. Hot, P., Naveteur, J., Leconte, P., & Sequeira, H. (1999). Diurnal variations of tonic electrodermal activity. International Journal of Psychophysiology, 33, 223–230.CrossRefGoogle Scholar
  25. iMotions® Attention Tool (5.4.3) [Computer software]. (n.d.). Cambridge, MA: iMotions®.Google Scholar
  26. Jaques, N., Conati, C., Harley, J., & Azevedo, R. (2014). Predicting affect from gaze data during interaction with an intelligent tutoring system. In S. Trausan-Matu, K. Boyer, M. Crosby, & K. Panourgia (Eds.), Proceedings of the 12th International Conference on Intelligent Tutoring Systems (ITS 2014) (pp. 29–38). Amsterdam, The Netherlands: Springer.Google Scholar
  27. Lester, J., Mott, B., Robison, J., Rowe, J., & Shores, L. (2013). Supporting self-regulated science learning in narrative-centered learning environments. In R. Azevedo & V. Aleven (Eds.), International handbook of metacognition and learning technologies (pp. 471–483). Amsterdam, The Netherlands: Springer.CrossRefGoogle Scholar
  28. Mudrick, N., Azevedo, R., Taub, M., Feyzi, R., & Bouchet, F. (2014). A study of subjective emotions, self-regulatory processes, and learning gains: Are pedagogical agents effective in fostering learning? In J. Polman et al. (Eds.), Proceedings of the International Conference of the Learning Sciences (pp. 309–316). Boulder, CO: ISLS.Google Scholar
  29. Noldus FaceReader (6.0) [Computer software]. (n.d.). Wageningen, The Netherlands: Noldus Information Technology.Google Scholar
  30. Papadopoulos, F., Corrigan, L., Jones, A., & Castellano, G. (2013). Learner modelling and automatic engagement recognition with robotic tutors. In 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction (ACII) (pp. 740–744), IEEE.Google Scholar
  31. Pekrun, R., Elliot, A., & Maier, M. (2006). Achievement goals and discrete achievement emotions: A theoretical model and prospective test. Journal of Educational Psychology, 98, 583–597.CrossRefGoogle Scholar
  32. Pekrun, R., & Linnenbrink-Garcia, L. (Eds.). (2014). Handbook of emotions and education. New York, NY: Taylor & Francis.Google Scholar
  33. Pekrun, R., Goetz, T., Frenzel, A., Barchfeld, P., & Perry, R. (2011). Measuring emotions in students’ learning and performance: The Achievement Emotions Questionnaire (AEQ). Contemporary Educational Psychology, 36, 36–48.CrossRefGoogle Scholar
  34. Pour, P., Hussain, M., AlZoubi, O., D’Mello, S., & Calvo, R. (2010). The impact of system feedback on learners’ affective and physiological states. In Lecture Notes in Computer Science, Vol. 6094, Intelligent Tutoring Systems (pp. 264–273). New York, NY: Springer.Google Scholar
  35. Scherer, K. (2009). The dynamic architecture of emotion: Evidence for the component process model. Cognition and Emotion, 23, 1307–1351.CrossRefGoogle Scholar
  36. Scherer, K. (2013). The nature of dynamics of relevance and valence appraisals: Theoretical advances and recent evidence. Emotion Review, 5, 150–162.CrossRefGoogle Scholar
  37. Schutz, P., & Zembylas, M. (Eds.). (in press). Methodological advances in research on emotion and education. Amsterdam, The Netherlands: Springer.Google Scholar
  38. Shi, Y., Ruiz, N., Taib, R., Choi, E., & Chen, F. (2007). Galvanic skin response (GSR) as an index of cognitive load. In Proceedings of CHI’07 Extended Abstracts on Human Factors in Computing System (pp. 2651–2656), ACM.Google Scholar
  39. Taub, M., Azevedo, R., Bouchet, F., & Khosravifar, B. (2014). Can the use of cognitive and metacognitive self-regulated learning strategies be predicted by learners’ levels of prior knowledge in hypermedia-learning environments? Computers in Human Behavior, 39, 356–367.CrossRefGoogle Scholar
  40. Venableas, P., & Mitchell, D. (1996). The effects of age, sex and time of testing on skin conductance activity. Journal of Biological Psychology, 43, 87–101.CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Roger Azevedo
    • 1
    Email author
  • Michelle Taub
    • 1
  • Nicholas Mudrick
    • 1
  • Jesse Farnsworth
    • 1
  • Seth A. Martin
    • 1
  1. 1.Department of PsychologyNorth Carolina State UniversityRaleighUSA

Personalised recommendations