Abstract
Betty’s Brain is an open-ended learning environment in which students learn about science topics by teaching a virtual agent named Betty through the construction of a visual causal map that represents the relevant science phenomena. The task is complex, and success requires the use of metacognitive strategies that support knowledge acquisition, causal map construction, and progress monitoring. Previous research has established that middle school students struggle at such tasks without proper scaffolding and feedback. In Betty’s Brain, this feedback is provided by Betty and Mr. Davis, another virtual agent designed to provide guidance and suggestions as students work. This paper discusses our implementation of contextualized conversational (CC) feedback, and then presents the results of an experimental study exploring the effects of this feedback in two 8th-grade science classrooms. The results illustrate some advantages of the CC feedback in comparison with a baseline dialogue mechanism that presents similar strategies in a non-conversational, non-contextualized form. While both groups showed significant pre-to-post test learning gains, the difference in learning gains between the groups was not statistically significant. However, students who received CC feedback more often performed actions in accordance with the advised strategies, and they created higher quality causal maps.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Adams, E. (2010). Fundamentals of game design (2nd ed.). Berkeley, CA: New Riders Pub.
Azevedo, R. (2005). Using hypermedia as a metacognitive tool for enhancing student learning? The role of self-regulated learning. Educational Psychologist, 40(4), 199–209. doi:10.1207/s15326985ep4004_2.
Biswas, G., Leelawong, K., Schwartz, D. L., & Vye, N. (2005). Learning by teaching: Anew agent paradigm for educational software. Applied Artificial Intelligence, 19(3–4), 363–392. doi:10.1080/08839510590910200.
Biswas, G., Jeong, H., Kinnebrew, J. S., Sulcer, B., & Roscoe, R. (2010). Measuring self-regulated learning skills through social interactions in a teachable agent environment. Research and Practice in Technology Enhanced Learning, 5(2), 123–152. doi:10.1142/S1793206810000839.
Bransford, J. D., & Schwartz, D. L. (1999). Rethinking transfer: A simple proposal with multiple implications. In A. Iran-Nejad & P. D. Pearson (Eds.). Review of Research in Education, 24, 61–101. doi:10.3102/0091732X024001061.
Forbus, K. D. (1984). Qualitative process theory. Artificial Intelligence, 24(1–3), 85–168. doi:10.1016/0004-3702(84)90038-9.
Gouli, E., Gogoulou, A., Papanikolaou, K., & Grigoriadou, M. (2004). Compass: An adaptive web-based concept map assessment tool. In A. J. Cañas, J. D. Novak, F. M. González (Eds.), Proceedings of the first international conference on concept mapping, Pamplona, Spain.
Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81–112. doi:10.3102/003465430298487.
Jeong, H., & Biswas, G. (2008). Mining student behavior models in learning-by-teaching environments. In R. S. J. d. Baker, T. Barnes, & J. E. Beck (Eds.), Educational data mining 2008: 1st international conference on educational data mining, proceedings (pp. 127–136). Montreal, Quebec, Canada.
Jonassen, D. H., & Ionas, I. G. (2008). Designing effective supports for causal reasoning. Educational Technology Research and Development, 56(3), 287–308. doi:10.1007/s11423-006-9021-6.
Kinnebrew, J. S., Loretz, K. M., & Biswas, G. (in press). A contextualized, differential sequence mining method to derive students’ learning behavior patterns. Journal of Educational Data Mining.
Koedinger, K. R., & Aleven, V. (2007). Exploring the assistance dilemma in experiments with cognitive tutors. Educational Psychology Review, 19(3), 239–264. doi:10.1007/s10648-007-9049-0.
Land, S. (2000). Cognitive requirements for learning with open-ended learning environments. Educational Technology Research and Development, 48(3), 61–78. doi:10.1007/BF02319858.
Leelawong, K., & Biswas, G. (2008). Designing learning by teaching agents: The Betty’s Brain system. International Journal of Artificial Intelligence in Education, 18(3), 181–208.
Luckie, D., Harrison, S. H., & Ebert-May, D. (2011). Model-based reasoning: Using visual tools to reveal student learning. Advances in Physiology Education, 35, 59–67. doi:10.1152/advan.00016.2010.
Lukasenko, R., Anohina-Naumeca, A., Vilkelis, M., & Grundspenkis, J. (2010). Feedback in the concept map based intelligent knowledge assessment system. Scientific Journal of Riga Technical University, 41, 17–26. doi:10.2478/v10143-010-0020-z.
Mayer, R. E. (2004). Should there be a three-strikes rule against pure discovery learning? American Psychologist, 59(1), 14–19. doi:10.1037/0003-066X.59.1.14.
Mendicino, M., Razzaq, L., & Hefferman, N. T. (2009). A comparison of traditional homework to computer-supported homework. Journal of Research on Technology in Education, 41(3), 331–359.
Rabiner, L. R. (1989). A tutorial on hidden Markov models and selected applications in speech recognition. Proceedings of the IEEE, 77(2), 257–286. IEEE Computer Society.
Roll, I., Aleven, V., McLaren, B. M., & Koedinger, K. R. (2011). Metacognitive practice makes perfect: Improving students’ self-assessment skills with an intelligent tutoring system. In G. Biswas, S. Bull. J. Kay, & T. Mitrovic (Eds.), Proceedings of the 15th international conference on artificial intelligence in education (pp. 288–295), Berlin: Springer.
Segedy, J. R., Kinnebrew, J. S., & Biswas, G. (2011). Modeling learner’s cognitive and metacognitive strategies in an open-ended learning environment. Advances in cognitive systems: Papers from the AAAI Fall Symposium (pp. 297–304). Menlo Park, CA: AAAI Press.
Segedy, J. R., Kinnebrew, J. S., & Biswas, G. (2012). Supporting student learning using conversational agents in a teachable agent environment. In J. van Aalst, K. Thompson, M. J. Jacobson, & P. Reimann (Eds.), The future of learning: Proceedings of the 10th international conference of the learning sciences (ICLS 2012): Vol. 2. Short Papers, Symposia, and Abstracts (pp. 251–255). International Society of the Learning Sciences: Sydney, NSW, Australia.
Shute, V. J. (2008). Focus on formative feedback. Review of Educational Research, 78(1), 153–189. doi:10.3102/0034654307313795.
VanLehn, K. (2006). The behavior of tutoring systems. International Journal of Artificial Intelligence in Education, 16(3), 227–265.
Acknowledgments
This work has been supported by the National Science Foundation’s Information and Intelligent Systems Award #0904387.
Author information
Authors and Affiliations
Corresponding author
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Segedy, J.R., Kinnebrew, J.S. & Biswas, G. The effect of contextualized conversational feedback in a complex open-ended learning environment. Education Tech Research Dev 61, 71–89 (2013). https://doi.org/10.1007/s11423-012-9275-0
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11423-012-9275-0