Advertisement

Using Data Visualizations to Foster Emotion Regulation During Self-Regulated Learning with Advanced Learning Technologies

  • Roger Azevedo
  • Michelle Taub
  • Nicholas V. Mudrick
  • Garrett C. Millar
  • Amanda E. Bradbury
  • Megan J. Price
Chapter

Abstract

Emotions play a critical role during learning and problem solving with advanced learning technologies. However, learners typically do not accurately monitor and regulate their emotions and may therefore not learn as much, disengage from the task, and not optimize their learning of the instructional material. Despite their importance, relatively few attempts have been made to understand learners’ emotional monitoring and regulation during learning with advanced learning technologies by using data visualizations of their own (and others’) cognitive, affective, metacognitive, and motivational (CAMM) self-regulated learning (SRL) processes to potentially foster their emotion regulation during learning with advanced learning technologies. We present a theoretically-based and empirically-driven conceptual framework that addresses emotion regulation by proposing the use of visualizations of one’s and others’ CAMM-SRL multichannel data (e.g., cognitive strategy use, metacognitive monitoring accuracy, facial expressions of emotions, physiological arousal, eye-movement behaviors, etc.) to facilitate learners’ monitoring and regulation of their emotions during learning with advanced learning technologies. We use examples from several of our laboratory and classroom studies to illustrate a possible mapping between theoretical assumptions, emotion-regulation strategies, and the types of data visualizations that can be used to enhance and scaffold learners’ emotion regulation, including key processes such as emotion flexibility, emotion adaptivity, and emotion efficacy. We conclude with future directions that can lead to a systematic interdisciplinary research agenda that addresses outstanding emotion regulation-related issues by integrating models, theories, methods, and analytical techniques for the areas of cognitive, learning, and affective sciences, human computer interaction, data visualization, big data, data mining, data science, learning analytics, open learner models, and SRL.

Keywords

Self-regulated learning Emotion Emotion regulation Data visualizations Multichannel data Advanced learning technologies Scaffolding 

Notes

Acknowledgements

This chapter was supported by funding from the National Science Foundation (DRL#1431552, DRL#1660878, CMMI#1744351) and the Social Sciences and Humanities Research Council of Canada (SSHRC 895-2011-1006). Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation or Social Sciences and Humanities Research Council of Canada.

The authors would like to thank Carina Tudela, Mitchell Moravec, Alex Haikonen, and Pooja Ganatra from the SMART Lab at NCSU for their assistance.

References

  1. Aldao, A., Nolen-Hoeksema, S., & Schweizer, S. (2015). Emotion-regulation strategies across psychopathology: A meta-analytic review. Clinical Psychology Review, 30, 217–237.CrossRefGoogle Scholar
  2. Antonietti, A., Casellato, C., Garrido, J. A., Luque, N. R., Naveros, F., Ros, E., … Pedrocchi, A. (2015). Spiking neural network with distributed plasticity reproduces cerebellar learning in eye blink conditioning paradigms. IEEE Transactions on Biomedical Engineering, 63, 210–219.CrossRefPubMedGoogle Scholar
  3. Azevedo, R. (2014a). Issues in dealing with sequential and temporal characteristics of self- and socially-regulated learning. Metacognition and Learning, 9, 217–228.CrossRefGoogle Scholar
  4. Azevedo, R. (2014b). Multimedia learning of metacognitive strategies. In R. Mayer (Ed.), The Cambridge handbook of multimedia learning (2nd ed., pp. 647–672). Cambridge, MA: Cambridge University Press.CrossRefGoogle Scholar
  5. Azevedo, R. (2015). Defining and measuring engagement and learning in science: Conceptual, theoretical, methodological, and analytical issues. Educational Psychologist, 50, 84–94.CrossRefGoogle Scholar
  6. Azevedo, R., & Aleven, V. (2013a). Metacognition and learning technologies: An overview of the current interdisciplinary research. In R. Azevedo & V. Aleven (Eds.), International handbook of metacognition and learning technologies (pp. 1–16). Amsterdam, The Netherlands: Springer.CrossRefGoogle Scholar
  7. Azevedo, R., & Aleven, V. (Eds.). (2013b). International handbook of metacognition and learning technologies. Amsterdam, The Netherlands: Springer.Google Scholar
  8. Azevedo, R., & Hadwin, A. F. (2005). Scaffolding self-regulated learning and metacognition—Implications for the design of computer-based scaffolds. Instructional Science, 33, 367–379.CrossRefGoogle Scholar
  9. Azevedo, R., Harley, J., Trevors, G., Feyzi-Behnagh, R., Duffy, M., Bouchet, F., & Landis, R. S. (2013). Using trace data to examine the complex roles of cognitive, metacognitive, and emotional self-regulatory processes during learning with multi-agent systems. In R. Azevedo & V. Aleven (Eds.), International handbook of metacognition and learning technologies (pp. 427–449). Amsterdam, The Netherlands: Springer.CrossRefGoogle Scholar
  10. Azevedo, R., Martin, S. A., Taub, M., Mudrick, N., Millar, G., & Grafsgaard, J. (2016). Are pedagogical agents’ external regulation effective in fostering learning with intelligent tutoring systems? In A. Micarelli, J. Stamper, & K. Panourgia (Eds.), Proceedings of the 13th International Conference on Intelligent Tutoring Systems—Lecture Notes in Computer Science 9684 (pp. 197–207). Amsterdam, The Netherlands: Springer.Google Scholar
  11. Azevedo, R., Taub, M., & Mudrick, N. (2015). Technologies supporting self-regulated learning. In M. Spector, C. Kim, T. Johnson, W. Savenye, D. Ifenthaler, & G. Del Rio (Eds.), The SAGE Encyclopedia of educational technology (pp. 731–734). Thousand Oaks, CA: SAGE.Google Scholar
  12. Azevedo, R., Taub, M., Mudrick, N., Farnsworth, J., & Martin, S. A. (2016). Interdisciplinary research methods used to investigate emotions with advanced learning technologies. In M. Zembylas & P. Schutz (Eds.), Methodological advances in research on emotion and education (pp. 231–243). Amsterdam, The Netherlands: Springer.Google Scholar
  13. Barrett, L. F., Lewis, M., & Haviland-Jones, J. M. (Eds.). (2016). Handbook of emotions. New York, NY: The Guildford Press.Google Scholar
  14. Belland, B., Walker, A., Kim, N., & Lefler, M. (2017). Synthesizing results from empirical research on computer-based scaffolding in STEM education: A meta-analysis. Review of Educational Research, 87(2), 309–344.CrossRefPubMedGoogle Scholar
  15. Biswas, G., Segedy, J. R., & Bunchongchit, K. (2016). From design to implementation to practice—A learning by teaching system: Betty’s brain. International Journal of Artificial Intelligence in Education, 26, 350–364.CrossRefGoogle Scholar
  16. Bonanno, G. A., & Burton, C. L. (2013). Regulatory flexibility an individual differences perspective on coping and emotion regulation. Perspectives on Psychological Science, 8, 591–612.CrossRefPubMedGoogle Scholar
  17. Bosse, T., Gerritsen, C., Man, J. D., & Treur, J. (2013). Learning emotion regulation strategies: A cognitive agent model. In Proceedings of the 2013 IEEE/WIC/ACM International Joint Conferences on Web Intelligence (WI) and Intelligent Agent Technologies (IAT) (Vol. Vol 2, pp. 245–252). Washington, DC: IEEE Computer Society.Google Scholar
  18. Buder, J., & Hesse, F. W. (2017). Informational environments: Cognitive, motivational-affective, and social-affective forays into the digital transformation. In J. Buder & F. W. Hesse (Eds.), Informational environments: Effects of use, effective designs (pp. 1–25). New York: Springer.Google Scholar
  19. Bull, S., & Kay, J. (2016). SMILI: A framework for interfaces to learning data in open learner models, learning analytics and related fields. International Journal of Artificial Intelligence in Education, 26, 293–331.CrossRefGoogle Scholar
  20. Calvo, R. A., D’Mello, S., Gratch, J., & Kappas, A. (Eds.). (2015). The Oxford handbook of affective computing. Oxford, England: Oxford University Press.Google Scholar
  21. D’Mello, S. K. (2013). A selective meta-analysis on the relative incidence of discrete affective states during learning with technology. Journal of Educational Psychology, 105, 1082–1099.CrossRefGoogle Scholar
  22. D’Mello, S., & Graesser, A. (2012). AutoTutor and affective AutoTutor: Learning by talking with cognitively and emotionally intelligent computers that talk back. ACM Transactions on Interactive Intelligent Systems, 2, 1–39.CrossRefGoogle Scholar
  23. D’Mello, S. K., & Graesser, A. C. (2015). Feeling, thinking, and computing with affect-aware learning technologies. In R. A. Calvo, S. K. D’Mello, J. Gratch, & A. Kappas (Eds.), The Oxford handbook of affective computing (pp. 419–434). New York, NY: Oxford University Press.Google Scholar
  24. D’Mello, S., Lehman, B., Pekrun, R., & Graesser, A. (2014). Confusion can be beneficial for learning. Learning and Instruction, 29, 153–170.CrossRefGoogle Scholar
  25. Doré, B. P., Silvers, J. A., & Ochsner, K. N. (2016). Toward a personalized science of emotion regulation. Social and Personality Psychology Compass, 10, 171–187.CrossRefGoogle Scholar
  26. Ekman, P. (1977). Facial expression. In A. Siegman, & S. Feldstein (Eds.), Nonverbal Communication and Behavior (pp. 97–126). New Jersey: Lawrence Erlbaum Association.Google Scholar
  27. Forbes-Riley, K., & Litman, D. (2011). Benefits and challenges of real-time uncertainty detection and adaptation in a spoken dialogue computer tutor. Speech Communication, 53, 1115–1136.CrossRefGoogle Scholar
  28. Graesser, A. C. (2016). Conversations with AutoTutor help students learn. IJAIED, 26, 124–132.Google Scholar
  29. Graesser, A. C., Lippert, A., & Hampton, D. (2017). Successes and failures in building learning environments to promote deep learning: The value of conversational agents. In J. Buder & F. W. Hesse (Eds.), Informational environments: Effects of use, effective designs (pp. 273–298). New York, NY: Springer.Google Scholar
  30. Gratch, J., & Marsella, S. (2015). Appraisal models. In R. A. Calvo, S. K. D’Mello, J. Gratch, & A. Kappas (Eds.), Handbook of affective computing (pp. 57–67). Oxford, UK: Oxford University Press.Google Scholar
  31. Grawemeyer, B., Mavrikis, M., Holmes, W., Gutierrez-Santos, S., Wiedmann, M., & Rummel, N. (2016). Affecting off-task behaviour: How affect-aware feedback can improve student learning. In Proceedings of the 6th Int. Conference on Learning Analytics & Knowledge (pp. 104–113). New York, NY: Association for Computing Machinery (ACM).CrossRefGoogle Scholar
  32. Gross, J. J. (2015a). Emotion regulation: Current status and future prospects. Psychological Inquiry, 26, 1–26.CrossRefGoogle Scholar
  33. Gross, J. J. (2015b). The extended process model of emotion regulation: Elaborations, applications, and future prospects. Psychological Inquiry, 26, 1–26.CrossRefGoogle Scholar
  34. Harley, J. M., Bouchet, F., Hussain, S., Azevedo, R., & Calvo, R. (2015). A multi-componential analysis of emotions during complex learning with an intelligent multi-agent system. Computers in Human Behavior, 48, 615–625.CrossRefGoogle Scholar
  35. Harley, J. M., Poitras, E. G., Jarrell, A., Duffy, M. C., & Lajoie, S. P. (2016). Comparing virtual and location-based augmented reality mobile learning: emotions and learning outcomes. Educational Technology Research and Development, 64(3), 359–388.CrossRefGoogle Scholar
  36. Kappas, A. (2013). Social regulation of emotion: Messy layers. Frontiers in psychology, 4, 1–11.CrossRefGoogle Scholar
  37. Lafford, B. A. (2004). Review of tell me more Spanish. Journal on Language Learning and Technology, 8, 21–34.Google Scholar
  38. Litman, D., & Forbes-Riley, K. (2014). Evaluating a spoken dialogue system that detects and adapts to user affective states. In Proceedings of the 15th Meeting of the Special Interest Group on Discourse and Dialogue (pp. 181–185).CrossRefGoogle Scholar
  39. Mayer, R. (2014). Multimedia instruction. In J. M. Spector, M. D. Merrill, J. Elen, & M. J. Bishop (Eds.), Handbook of research on educational communications and technology (4th ed., pp. 385–400). Amsterdam, The Netherlands: Springer.CrossRefGoogle Scholar
  40. Muldner, K., Wixon, M., Rai, D., Burleson, W., Woolf, B., & Arroyo, I. (2015). Exploring the impact of a learning dashboard on student affect. In C. Conati, N. Heffernan, A. Mitrovic, & M. Verdejo (Eds.), Artificial intelligence in education (pp. 307–317). Heidelberg, Germany: Springer.CrossRefGoogle Scholar
  41. Nye, B. D., Graesser, A. C., & Hu, X. (2014). Multimedia learning with intelligent tutoring systems. In R. Mayer (Ed.), Cambridge handbook of multimedia learning (3rd ed., pp. 705–728). Cambridge, MA: Cambridge University Press.CrossRefGoogle Scholar
  42. Paas, F., & Sweller, J. (2012). An evolutionary upgrade of cognitive load theory: Using the human motor system and collaboration to support the learning of complex cognitive tasks. Educational Psychology Review, 24, 27–45.CrossRefGoogle Scholar
  43. Pekrun, R., & Linnenbrink-Garcia, L. (Eds.). (2014). International handbook of emotions in education. New York, NY: Routledge.Google Scholar
  44. Porayska-Pomsta, K., Mavrikis, M., D’Mello, S., Conati, C., & Baker, R. S. (2013). Knowledge elicitation methods for affect modelling in education. International Journal of Artificial Intelligence in Education, 22, 107–140.Google Scholar
  45. Reeck, C., Ames, D. R., & Ochsner, K. N. (2016). The social regulation of emotion: An integrative, cross-disciplinary model. Trends in Cognitive Sciences, 20, 47–63.CrossRefPubMedGoogle Scholar
  46. Sabourin, J., & Lester, J. (2014). Affect and engagement in game-based learning environments. IEE Transactions on Affective Computing, 5, 45–56.CrossRefGoogle Scholar
  47. Santos, J. L., Govaerts, S., Verbert, K., & Duval, E. (2012). Goal-oriented visualizations of activity tool tracking: A case study with engineering students. In S. Buckingham Shum, D. Gašević, & R. Ferguson (Eds.), Proceedings of the 2nd International Conference on Learning Analytics and Knowledge LAK ′12 (pp. 143–152). New York, NY: ACM.CrossRefGoogle Scholar
  48. Scheiter, K., Fillisch, B., Krebs, M., Leber, J., Ploetzner, R., Renkl, A., … Zimmermann, G. (2017). How to design adaptive multimedia environments to support self-regulated learning. In J. Buder & F. W. Hesse (Eds.), Informational environments: Effects of use, effective designs (pp. 203–233). New York, NY: Springer.Google Scholar
  49. Scherer, K. R. (2009). The dynamic architecture of emotion: Evidence for the component process model. Cognition and Emotions, 7, 1307–1351.CrossRefGoogle Scholar
  50. Schunk, D. H., & Greene, J. A. (Eds.). (in press). Handbook of self-regulation of learning and performance (2nd ed.). New York, NY: Routledge.Google Scholar
  51. Suri, G., & Gross, J. J. (2016). Emotion regulation: A valuation perspective. In L. F. Barrett, M. Lewis, & J. M. Haviland-Jones (Eds.), Handbook of emotions (4th ed.). New York, NY: The Guildford Press.Google Scholar
  52. Taub, M., Mudrick, N., Azevedo, R., Millar, G., Rowe, J., & Lester, J. (2016). Using multi-level modeling with eye-tracking data to predict metacognitive monitoring and self-regulated learning with Crystal Island. In A. Micarelli, J. Stamper, & K. Panourgia (Eds.), Proceedings of the 13th ITS Conference. Lecture Notes in Computer Science 9684 (pp. 240–246). The Netherlands: Springer.Google Scholar
  53. Taub, M., Mudrick, N. V., Azevedo, R., Millar, G. C., Rowe, J., & Lester, J. (2017). Using multi-channel data with multi-level modeling to assess in-game performance during gameplay with CRYSTAL ISLAND. Computers in Human Behavior.Google Scholar
  54. Van Kleef, G. A. (2009). How emotions regulate social life the emotions as social information (EASI) model. Current Directions in Psychological Science, 18, 184–188.CrossRefGoogle Scholar
  55. van Kleef, G. A. (2016). Toward an integrative theory of emotions as social information. Cambridge, MA: Cambridge University Press.Google Scholar
  56. Verbert, K., Duval, E., Klerkx, J., Govaerts, S., & Santos, J. L. (2013). Learning analytics dashboard applications. American Behavioral Scientist, 57, 1500–1509.CrossRefGoogle Scholar
  57. Verbert, K., Govaerts, S., Duval, E., Santos, J. L., Assche, F., Parra, G., & Klerkx, J. (2013). Learning dashboards: An overview and future research opportunities. Personal and Ubiquitous Computing, 18, 1499–1514.Google Scholar
  58. Winne, P. H., & Azevedo, R. (2014). Metacognition. In R. K. Sawyer (Ed.), The Cambridge handbook of the learning sciences (2nd ed., pp. 63–87). Cambridge, England: Cambridge University Press.CrossRefGoogle Scholar
  59. Winne, P. H., & Hadwin, A. F. (1998). Studying as self-regulated learning. In D. J. Hacker, J. Dunlosky, & A. Graesser (Eds.), Metacognition in educational theory and practice (pp. 277–304). Hillsdale, NJ: Erlbaum.Google Scholar
  60. Winne, P. H., Vytaske, J., Patzak, A., Rakovic, M., Marzouk, A., Pakdaman-Sajovi, A., … Nesbit, J. (2017). Designs for learning analytics to support information problem solving. In J. Buder & F. W. Hesse (Eds.), Informational environments: Effects of use, effective designs (pp. 249–272). New York, NY: Springer.Google Scholar
  61. Woolf, B., Burleson, W., Arroyo, I., Dragon, T., Cooper, D., & Picard, R. (2009). Affect-aware tutors: Recognizing and responding to student affect. International Journal of Learning Technology, 4, 129–164.CrossRefGoogle Scholar
  62. Zimmerman, B. J., & Schunk, D. H. (Eds.). (2011). Handbook of self-regulation of learning and performance. New York, NY: Routledge.Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Roger Azevedo
    • 1
  • Michelle Taub
    • 1
  • Nicholas V. Mudrick
    • 1
  • Garrett C. Millar
    • 1
  • Amanda E. Bradbury
    • 1
  • Megan J. Price
    • 1
  1. 1.North Carolina State UniversityRaleighUSA

Personalised recommendations