Abstract
We have developed a computer-based learning environment that helps students learn science by constructing causal concept map models. The system builds upon research in learning-by-teaching (LBT) and has students take on the role and responsibilities of being the teacher to a virtual student named Betty. The environment is structured so that successfully instructing their teachable agents requires the students to learn and understand the science topic for themselves. This learning process is supported through the use of adaptive scaffolding provided by feedback from the two agents in the system: the teachable agent, Betty, and a mentor agent, Mr. Davis. For example, if Betty performs poorly on a quiz, she may tell the student that she needs to learn more about the topics on which she is performing poorly. In addition, Mr. Davis may suggest that students ask Betty questions and get her to explain her answers to help them trace the causal reasoning chains in their map and find out where she may be making mistakes. Thus the system is designed to help students develop and refine their own knowledge construction and monitoring strategies as they teach their agent.
This chapter provides an overview of two studies that were conducted in fifth-grade science classrooms. A description of the analysis techniques that we have developed for interpreting students’ activities in this learning environment is also provided. More specifically, we discuss the generation of hidden Markov models (HMMs) that capture students’ aggregated behavior patterns, which form the basis for analyzing students’ metacognitive strategies in the system. Our study results show that students who utilized LBT versions of our system performed better than students who used a non-teaching version of the system. Further, students’ performances were strongest when the system explicitly provided support to help them develop self-regulated learning strategies. To gain further insight into the students’ reactions to feedback from the two agents, we present results from a second study that employed a think-aloud protocol. Overall, the results from this study illustrated that students were more receptive to the explicit strategy-oriented feedback from the mentor agent. Interestingly, this study also suggested that students had difficulty in correctly applying Betty’s feedback related to metacognitive monitoring activities.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
In the system, the same triggering conditions are used to generate Betty’s and Mr. Davis’ feedback. The system is designed so that the feedback is provided only after the triggering pattern is activated a certain number of times. This number is chosen randomly from a predefined range of values (e.g., [2, 5]) and recomputed after every instance of feedback. The numbers for Betty and Mr. Davis are chosen independently.
- 2.
All statistical comparisons of means among conditions were made with ANOVA post-hoc (Tukey HSD) tests, and effect sizes are computed as Cohen’s \( \widehat{d}\). Further, since some of the differences falling outside of the significance cutoff of p < 0. 05 still had moderately large effect sizes, we report the results for multiple significance cutoff values (p < 0. 1 and p < 0. 05), allowing the reader to make their own determinations based on the reported results.
- 3.
Study 1 and study 2 were conducted in different years.
- 4.
Many student discussions, including some that explicitly referenced the feedback, neither affirmed, dismissed, nor deferred the feedback.
- 5.
Agent role (and consequently relationship with the student) and the content of agent feedback are inextricably linked in this study, making it impossible to attribute student responses to one factor or the other. However, the correlation between students’ responses to agent feedback has useful implications for future system design and experimental study opportunities discussed in this section and the next.
- 6.
We employed a proportion in a window of subsequent actions because agent feedback often suggested a course of action that could involve repeated actions (e.g., edits or reads), and it is not possible to determine precisely whether a student’s action was an attempt to follow agent advice or not.
- 7.
We tested a variety of different window sizes, and all of them produced similar results.
References
Aleven, V., McLaren, B., Roll, I., & Koedinger, K. (2006). Toward meta-cognitive tutoring: A model of help seeking with a Cognitive Tutor. International Journal of Artificial Intelligence in Education, 16(2), 101–128.
Azevedo, R. (2005). Using hypermedia as a metacognitive tool for enhancing student learning? The role of self-regulated learning. Educational Psychologist, 40(4), 199–209.
Azevedo, R., & Witherspoon, A. M. (2009). Self-regulated use of hypermedia. In A. Graesser, J. Dunlosky, & D. Hacker (Eds.), Handbook of metacognition in education. Mahwah, NJ: Erlbaum.
Azevedo, R., Witherspoon, A., Chauncey, A., Burkett, C., & Fike, A. (2009). MetaTutor: A metacognitive tool for enhancing self-regulated learning. Annual Meeting of the American Association for Artificial Intelligence, Symposium on Metacognitive and Cognitive Educational Systems (pp. 4–19). AQ: Please provide symposium location, publisher name and location for reference Azevedo et al. (2009).
Bandura, A. (1997). Self-efficacy: The exercise of control. New York, NY: Freeman.
Biswas, G., Jeong, H., Kinnebrew, J. S., Sulcer, B., & Roscoe, R. (2010). Measuring self-regulated learning skills through social interactions in a teachable agent environment. Research and Practice in Technology-Enhanced Learning (RPTEL), 5(2), 123–152.
Biswas, G., Leelawong, K., Schwartz, D., Vye, N., & Vanderbilt, T. T. A. G. (2005). Learning by teaching: A new agent paradigm for educational software. Applied Artificial Intelligence, 19(3), 363–392.
Blair, K., Schwartz, D. L., Biswas, G., & Leelawong, K. (2007). Pedagogical agents for learning by teaching: teachable agents. Educational Technology & Society: Special Issue on Pedagogical Agents, 47(1). AQ: Please provide page range for reference Blair et al. (2007)
Bransford, J. D., Brown, A. L., & Cocking, R. R. (Eds.) (2000). How people learn. Washington, DC: National Academy Press.
Bransford, J. D., & Schwartz, D. L. (1999). Rethinking transfer: A simple proposal with multiple implications. Review of Research in Education, 24(1), 61.
Brown, A. L., & Palincsar, A. S. (1989). Guided, cooperative learning and individual knowledge acquisition. In L. B. Resnick (Ed.), Knowing, learning, and instruction: Essays in honor of Robert Glaser (pp. 393–451). Hillsdale, NJ: Lawrence Erlbaum Associates.
Burleson, W., Picard, R. W., Perlin, K., & Lippincott, J. (2004) A platform for affective agent research. Workshop on Empathetic Agents, International Conference on Autonomous Agents and Multiagent Systems, New York, NY.
Chase, C. C., Chin, D. B., Oppezzo, M. A., & Schwartz, D. L. (2009). Teachable agents and the protégé effect: Increasing the effort towards learning. Journal of Science Education and Technology, 18(4), 334–352.
D’Mello, S. K., Craig, S. D., Witherspoon, A., Mcdaniel, B., & Graesser, A. (2008). Automatic detection of learners affect from conversational cues. User Modeling and User-Adapted Interaction, 18(1), 45–80.
D’Mello, S., Picard, R. W., & Graesser, A. (2007) Toward an affect-sensitive AutoTutor. IEEE Intelligent Systems, 22(4), 53–61.
Forbus, K. D. (1984) Qualitative process theory. Artificial intelligence, 24(1–3), 85–168.
Hadwin, A. F., Nesbit, J. C., Jamieson-Noel, D., Code, J., & Winne, P. H. (2007). Examining trace data to explore self-regulated learning. Metacognition and Learning, 2(2), 107–124.
Hadwin, A. F., Winne, P. H., Stockley, D. B., Nesbit, J. C., & Woszczyna, C. (2001) Context moderates students’ self-reports about how they study. Journal of Educational Psychology, 93(3), 477–487.
Jeong, H., & Biswas, G. (2008). Mining student behavior models in learning-by-teaching environments. Proceedings of the First International Conference on Educational Data Mining, Montreal, Quebec, Canada (pp. 127–136).
Jonassen, D. H., & Ionas, I. G. (2008). Designing effective supports for causal reasoning. Educational Technology Research and Development, 56(3), 287–308
Leelawong, K., & Biswas, G. (2008). Designing learning by teaching agents: The Betty’s Brain system. International Journal of Artificial Intelligence in Education, 18(3), 181–208.
Lester, J. C., Converse, S. A., Kahler, S. E., Barlow, S. T., Stone, B. A., & Bhogal, R. S. (1997). The persona effect: affective impact of animated pedagogical agents. Proceedings of the SIGCHI conference on Human Factors in Computing Systems (CHI ’97), Atlanta, GA (pp. 359–366).
Li, C., & Biswas, G. (2000). A Bayesian approach to temporal data clustering using hidden Markov models. Proceedings of the Seventeenth International Conference on Machine Learning (pp. 543–550).
Li, C., & Biswas, G. (2002). A Bayesian approach for learning hidden Markov models from data. Scientific Programming: Special Issue on Markov Chain and Hidden Markov Models, 10(3), 201–219.
Paris, S. G., & Paris, A. H. (2001). Classroom applications of research on self-regulated learning. Educational Psychologist, 36(2), 89–101.
Perry, N. E., & Winne, P. H. (2006). Learning from learning kits: gStudy traces of students? Self-regulated engagements with computerized content. Educational Psychology Review, 18(3), 211–228.
Pintrich, P. R. (2000). An achievement goal theory perspective on issues in motivation terminology, theory, and research. Contemporary Educational Psychology, 25(1), 92–104.
Pintrich, P. R. (2002). The role of metacognitive knowledge in learning, teaching, and assessing. Theory into Practice, 41(4), 219–225.
Pintrich, P. R., Marx, R. W., & Boyle, R. A. (1993). Beyond cold conceptual change: The role of motivational beliefs and classroom contextual factors in the process of conceptual change. Review of Educational Research, 63(2), 167–199.
Pintrich, P. R., Smith, D. A. F., Garcia, T., & McKeachie, W. J. (1993). Reliability and predictive validity of the Motivated Strategies for Learning Questionnaire (MSLQ). Educational and psychological measurement, 53(3), 801-813.
Rabiner, L. R. (1989). A tutorial on hidden Markov models and selected applications in speech recognition. Proceedings of the IEEE, 77(2), 257–286.
Roll, I., Aleven, V., Mclaren, B. M., & Koedinger, K. R. (2007). Designing for metacognition: Applying cognitive tutor principles to the tutoring of help seeking. Metacognition and Learning, 2, 125–140.
Roscoe, R. D., & Chi, M. T. H. (2007). Understanding tutor learning: knowledge-building and knowledge-telling in peer tutors’ explanations and questions. Review of Educational Research, 77(4), 534–574.
Schraw, G., Kauffman, D. F., & Lehman, S. (2002). Self-regulated learning theory. In L. Nadel (Ed.), The encyclopedia of cognitive science (pp. 1063–1073), London: Nature Publishing Company.
Schunk, D. H., & Zimmerman, B. J. (1997). Social origins of self-regulatory competence. Educational Psychologist, 32(4), 195–208.
Schwartz, D. L., Blair, K. P., Biswas, G., Leelawong, K., & Davis, J. (2007). Animations of thought: Interactivity in the teachable agent paradigm. In R. Lowe & W. Schnotz (Eds.), Learning with animation: Research and implications for design (pp. 114–140). Cambridge: Cambridge University Press.
Schwartz, D. L., Chase, C., Chin, D. B., Oppezzo, M., Kwong, H., Okita, S. et al. (2009) Interactive metacognition: Monitoring and regulating a teachable agent. In D. J. Hacker, J. Dunlosky, & A. C. Graesser (Eds.), Handbook of metacognition in education (pp. 340–358). New York: Routledge Press.
Schwartz, D. L., & Martin, T. (2004). Inventing to prepare for future learning: The hidden efficiency of encouraging original student production in statistics instruction. Cognition and Instruction, 22(2), 129–184.
Tan, J., Biswas, G., & Schwartz, D. (2006). Feedback for metacognitive support in learning by teaching environments. Proceedings of the 28th Annual Meeting of the Cognitive Science Society, Vancouver, Canada (pp. 828–833).
Wagster, J., Tan, J., Wu, Y., Biswas, G., & Schwartz, D. (2007). Do learning by teaching environments with metacognitive support help students develop better learning behaviors. Proceedings of the 29th Annual Meeting of the Cognitive Science Society, Nashville, TN (pp. 695–700).
Weinstein, C. E., Schulte, A. C., and Palmer, D. R. (1987). The learning and study strategies inventory. Clearwater, FL: H & H Publishing.
Winne, P., & Hadwin, A. (2008). The weave of motivation and self-regulated learning. In D. H. Schunk & B. J. Zimmerman (Eds.), Motivation and self-regulated learning: Theory, research, and applications (pp. 297–314). New York, NY: Taylor & Francis.
Zimmerman, B. J. (1990). Self-regulating academic learning and achievement: The emergence of a social cognitive perspective. Educational Psychology Review, 2(2), 173–201.
Zimmerman, B. J. (2001). Theories of self-regulated learning and academic achievement: An overview and analysis. In B. J. Zimmerman & D. H. Schunk (Eds.), Self-regulated learning and academic achievement: Theoretical perspectives (pp. 1–37). Mahwah, NJ: Erlbaum.
Zimmerman, B. J. (2008). Investigating self-regulation and motivation: Historical background, methodological developments, and future prospects. American Educational Research Journal, 45(1), 166–183.
Zimmerman, B. J., Bandura, A., & Martinez-Pons, M. (1992). Self-motivation for academic attainment: The role of self-efficacy beliefs and personal goal setting. American Educational Research Journal, 29(3), 663–676.
Zimmerman, B. J., & Martinez-Pons, M. (1986). Development of a structured interview for assessing student use of self-regulated learning strategies. American Educational Research Journal, 23(4), 614–628.
Acknowledgments
This work has been supported by Dept. of ED IES grant #R305H060089, NSF REESE Award #0633856, and NSF IIS Award #0904387.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer Science+Business Media New York
About this chapter
Cite this chapter
Kinnebrew, J.S., Biswas, G., Sulcer, B., Taylor, R.S. (2013). Investigating Self-Regulated Learning in Teachable Agent Environments. In: Azevedo, R., Aleven, V. (eds) International Handbook of Metacognition and Learning Technologies. Springer International Handbooks of Education, vol 28. Springer, New York, NY. https://doi.org/10.1007/978-1-4419-5546-3_29
Download citation
DOI: https://doi.org/10.1007/978-1-4419-5546-3_29
Published:
Publisher Name: Springer, New York, NY
Print ISBN: 978-1-4419-5545-6
Online ISBN: 978-1-4419-5546-3
eBook Packages: Humanities, Social Sciences and LawEducation (R0)