Skip to main content

Advertisement

SpringerLink
Log in

Analyzing the temporal evolution of students’ behaviors in open-ended learning environments

  • Published:
Metacognition and Learning Aims and scope Submit manuscript

Abstract

Metacognition and self-regulation are important for developing effective learning in the classroom and beyond, but novice learners often lack effective metacognitive and self-regulatory skills. However, researchers have demonstrated that metacognitive processes can be developed through practice and appropriate scaffolding. Betty’s Brain, an open-ended computer-based learning environment, helps students practice their cognitive skills and develop related metacognitive strategies as they learn science topics. In this paper, we analyze students’ activity sequences in a study that compared different categories of adaptive scaffolding in Betty’s Brain. The analysis techniques for measuring students’ cognitive and metacognitive processes extend our previous work on using sequence mining methods to discover students’ frequently-used behavior patterns by (i) developing a systematic approach for interpreting derived behavior patterns using a cognitive/metacognitive task model and (ii) analyzing the evolution of students’ frequent behavior patterns over time. Our results show that it is possible to identify students’ learning behaviors and analyze their evolution as they work in the Betty’s Brain environment. Further, the results illustrate that changes in student behavior were generally consistent with the scaffolding provided, suggesting that these metacognitive strategies can be taught to middle school students in computer-based learning environments.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Notes

  1. In the analysis presented in this paper, we settled on a window of two previous actions after performing qualitative analysis of representative action sequences.

  2. The best map score is the highest map score a student achieved at any time during the intervention.

References

  • Agrawal, R., & Srikant, R. (1995). Mining sequential patterns. In Proceedings of the 11th IEEE international conference on data engineering (ICDE) (pp. 3–14).

  • Aleven, V., McLaren, B., Roll, I., Koedinger, K. (2006). Toward meta-cognitive tutoring: a model of help seeking with a cognitive tutor. International Journal of Artificial Intelligence in Education, 16(2), 101–128.

    Google Scholar 

  • Aleven, V., Roll, I., McLaren, B., Koedinger, K. (2010). Automated, unobtrusive, action-by-action assessment of self-regulation during learning with an intelligent tutoring system. Educational Psychologist, 45(4), 224–233.

    Article  Google Scholar 

  • Amershi, S., & Conati, C. (2009). Combining unsupervised and supervised classification to build user models for exploratory learning environments. Journal of Educational Data Mining, 1(1), 18–71.

    Google Scholar 

  • Azevedo, R. (2005). Using hypermedia as a metacognitive tool for enhancing student learning: the role of self-regulated learning. Educational Psychologist, 40(4), 199–209.

    Article  Google Scholar 

  • Azevedo, R., & Aleven, V. (2013). Metacognition and learning technologies: an overview of current interdisciplinary research. In R. Azevedo & V. Aleven (Eds.), International handbook on metacognition and learning technologies (pp. 1–16). New York: Springer.

  • Azevedo, R., & Witherspoon, A. (2009). Self-regulated use of hypermedia. In A. Graesser, J. Dunlosky, D. Hacker (Eds.), Handbook of metacognition in education. Erlbaum, Mahwah.

  • Azevedo, R., Moos, D.C., Johnson, A.M., Chauncey, A.D. (2010). Measuring cognitive and metacognitive regulatory processes during hypermedia learning: issues and challenges. Educational Psychologist, 45(4), 210–223.

    Article  Google Scholar 

  • Azevedo, R., Cromley, J., Moos, D.C., JGreene, J., Winters, F. (2011). Adaptive content and process scaffolding: a key to facilitating students self-regulated learning with hypermedia. Psychological Testing and Assessment Modeling, 53, 106–140.

    Google Scholar 

  • Azevedo, R., Harley, J., Trevors, G., Duffy, M., Feyzi-Behnagh, R., Bouchet, F., Landis, R. (2013). Using trace data to examine the complex roles of cognitive, metacognitive, and emotional self regulatory processes during learning with multi-agent systems. In R. Azevedo & V. Aleven (Eds.), International handbook on metacognition and learning technologies (pp. 427–449). New York: Springer.

  • Bandura, A. (1982). The assessment and predictive generality of self-percepts of efficacy. Journal of Behavior Therapy and Experimental Psychiatry, 13(3), 195–199.

    Article  Google Scholar 

  • Bandura, A. (1984). Recycling misconceptions of perceived self-efficacy. Cognitive Therapy and Research, 8(3), 231–255.

    Article  Google Scholar 

  • Bandura, A. (1997). Self-efficacy: the exercise of control. New York: Freeman.

  • Biswas, G., Leelawong, K., Schwartz, D., Vye, N., Vanderbilt, T. (2005). Learning by teaching: a new agent paradigm for educational software. Applied Artificial Intelligence, 19(3), 363–392.

    Article  Google Scholar 

  • Biswas, G., Jeong, H., Kinnebrew, J., Sulcer, B., Roscoe, R. (2010). Measuring self-regulated learning skills through social interactions in a teachable agent environment. Research and Practice in Technology Enhanced Learning, 5(02), 123–152.

    Article  Google Scholar 

  • Biswas, G., Kinnebrew, J.S., Segedy, J.R. (2013). Analyzing students’ metacognitive strategies in open-ended learning environments. In: Proceedings of the 35th annual meeting of the cognitive science society. Germany: Berlin.

  • Bransford, J., Brown, A., Cocking, R. (Eds.) (2000). How people learn. Washington, DC: National Academy Press.

    Google Scholar 

  • Brown, A., Bransford, J.D., Ferrara, R.A., Campione, J.C. (1983). Learning, remembering, and understanding. In P. Mussen (Ed.), Handbook of child psychology. Hoboken: John Wiley.

  • Butler, D., & Winne, P. (1995). Feedback and self-regulated learning: a theoretical synthesis. Review of Educational Research, 65(3), 245.

    Article  Google Scholar 

  • Chi, M., Glaser, R., Farr, M. (1988). The nature of expertise. Lawrence Erlbaum Associates, Inc.

  • Chipman, S.F., Schraagen, J.M., Shalin, V.L. (2000). Introduction to cognitive task analysis. In J.M. Schraagen, S.F. Chipman, V.L. Shalin (Eds.), Cognitive task analysis (pp. 3–23). Psychology Press.

  • Cross, D., & Paris, S. (1988). Developmental and instructional analyses of children’s metacognition and reading comprehension. Journal of Educational Psychology, 80(2), 131.

    Article  Google Scholar 

  • Devolder, A., van Braak, J., Tondeur, J. (2012). Supporting self-regulated learning in computer-based learning environments: systematic review of effects of scaffolding in the domain of science education. Journal of Computer Assisted Learning, 28, 557–573.

    Article  Google Scholar 

  • Flavell, J. (1979). Metacognition and cognitive monitoring: a new area of cognitive–developmental inquiry. American Psychologist, 34(10), 906.

    Article  Google Scholar 

  • Ge, X., & Land, S.M. (2003). Scaffolding students problem-solving processes in an ill-structured task using question prompts and peer interactions. Educational Technology Research and Development, 51(1), 21–38.

    Article  Google Scholar 

  • Grotzer, T., & Mittlefehldt, S. (2012). The role of metacognition in students understanding and transfer of explanatory structures in science. Metacognition in science education (pp. 79–99).

  • Hadwin, A., Nesbit, J., Jamieson-Noel, D., Code, J., Winne, P. (2007). Examining trace data to explore self-regulated learning. Metacognition and Learning, 2(2), 107–124.

    Article  Google Scholar 

  • Hennessey, M. (1999). Probing the dimensions of metacognition: implications for conceptual change teaching-learning. In NARST, ERIC.

  • Jain, A., & Dubes, R. (1988). Algorithms for clustering data. Upper Saddle River: Prentice-Hall.

  • Kinnebrew, J.S., & Biswas, G. (2012). Identifying learning behaviors by contextualizing differential sequence mining with action features and performance evolution. In Proceedings of the 5th international conference on educational data mining (EDM). Chania.

  • Kinnebrew, J.S., Loretz, K.M., Biswas, G. (2013a). A contextualized, differential sequence mining method to derive students’ learning behavior patterns. Journal of Educational Data Mining, 5(1), 190–219.

    Google Scholar 

  • Kinnebrew, J.S., Mack, D.L., Biswas, G. (2013b). Mining temporally-interesting learning behavior patterns. In S.K. DMello, R.A. Calvo, A. Olney (Eds.), Proceedings of the 6th international conference on educational data mining (pp. 252–255). Memphis .

  • Kramarski, B., & Mevarech, Z. (2003). Enhancing mathematical reasoning in the classroom: the effects of cooperative learning and metacognitive training. American Educational Research Journal, 40(1), 281–310.

    Article  Google Scholar 

  • Lajoie, S., & Derry, S. (1993). Computers as cognitive tools. Hillsdale: Lawrence Erlbaum Associates.

  • Land, S. (2000). Cognitive requirements for learning with open-ended learning environments. Educational Technology Research and Development, 48(3), 61–78.

    Article  Google Scholar 

  • Larkin, J., McDermott, J., Simon, D., Simon, H. (1980). Expert and novice performance in solving physics problems. Science, 208(4450), 1335–1342.

    Article  Google Scholar 

  • Leelawong, K., & Biswas, G. (2008). Designing learning by teaching agents: the Betty’s Brain system. International Journal of Artificial Intelligence in Education, 18(3), 181–208.

    Google Scholar 

  • Lester, J., Mott, B., Robinson, J., Rowe, J., Shores, L. (2013). Supporting seld-regulated learning in narrative-centered environments. In R. Azevedo, & V. Aleven (Eds.), International handbook on metacognition and learning technologies (pp. 471–483). New York: Springer.

  • Li, C., & Biswas, G. (2002). Unsupervised learning with mixed numeric and nominal data. IEEE Transactions on Knowledge and Data Engineering, 14(4), 673–690.

    Article  Google Scholar 

  • Martinez, M.E. (2006). What is metacognition? Phi Delta Kappan (pp. 696–699).

  • Mashima, D., Kobourov, S., Hu, Y. (2012). Visualizing dynamic data with maps. IEEE Transactions on Visualization and Computer Graphics, 18(9), 1424–1437.

    Article  Google Scholar 

  • Mayer, R.E. (1996). Learning strategies for making sense out of expository text: the SOI model for guiding three cognitive processes in knowledge construction. Educational Psychology Review, 8(4), 357–371.

    Article  Google Scholar 

  • Molenaar, I., Roda, C., van Boxtel, C., Sleegers, P. (2012). Dynamic scaffolding of socially regulated learning in a computer-based learning environment. Computers and Education, 59(2), 515–523.

    Article  Google Scholar 

  • Murtagh, F. (1983). A survey of recent advances in hierarchical clustering algorithms. The Computer Journal, 26(4), 354–359.

    Article  Google Scholar 

  • Nesbit, J., Zhou, M., Xu, Y., Winne, P. (2007). Advancing log analysis of student interactions with cognitive tools. In 12th biennial conference of the european association for research on learning and insruction (EARLI).

  • Perera, D., Kay, J., Koprinska, I., Yacef, K., Zaïane, O. (2009). Clustering and sequential pattern mining of online collaborative learning data. IEEE Transactions on Knowledge and Data Engineering, 21(6), 759–772.

    Article  Google Scholar 

  • Pieschl, S., & Stahl, E. RB. (2013). Adaptation to context as core component of self-regulated learning: the example of complexity and epistemic beliefs. In R. Azevedo, & V. Aleven (Eds.), International handbook on metacognition and learning technologies (pp. 53–65). New York: Springer.

  • Pintrich, P. (2000). An achievement goal theory perspective on issues in motivation terminology, theory, and research* 1. Contemporary Educational Psychology, 25(1), 92–104.

    Article  Google Scholar 

  • Pintrich, P., Smith, D., García, T., McKeachie, W. (1991). A manual for the use of the motivated strategies for learning questionnaire (mslq) (p. 48109:1259). Ann Arbor Michigan: National Center for Research to Improve Postsecondary Teaching and Learning.

  • Plantevit, M., Laurent, A., Teisseire, M. (2006). Hype: mining hierarchical sequential patterns. In Proceedings of the 9th ACM international workshop on data warehousing and OLAP (pp. 19–26). ACM.

  • Pleil, J., Stiegel, M., Madden, M., Sobus, J. (2011). Heat map visualization of complex environmental and biomarker measurements. Chemosphere, 84(5), 716–723.

    Article  Google Scholar 

  • Puntambekar, S., & Hubscher, R. (2005). Tools for scaffolding students in a complex learning environment: what have we gained and what have we missed?Educational Psychologist, 40(1), 1–12.

    Article  Google Scholar 

  • Sabourin, J., Mott, B., Lester, J. (2013). Utilizing dynamic bayes nets to improve early prediction models of self-regulated learning. User Modeling, Adaptation, and Personalization, 228–241.

  • Schraw, G., Kauffman, D., Lehman, S. (2002). Self-regulated learning theory. In L. Nadel (Ed.), The encyclopedia of cognitive science (pp. 1063–1073). London: Nature Publishing Company.

  • Schraw, G., Crippen, K., Hartley, K. (2006). Promoting self-regulation in science education: metacognition as part of a broader perspective on learning. Research in Science Education, 36(1), 111–139.

    Article  Google Scholar 

  • Schraw, G., Olafson, L., Weibel, M., Sewing, D. (2012). Metacognitive knowledge and field-based science learning in an outdoor environmental education program. Metacognition in Science Education, 57–77.

  • Schunk, D., & Zimmerman, B. (1994). Self-regulation of learning and performance: issues and educational applications. Mahwah: Lawrence Erlbaum Associates, Inc.

  • Schunk, D., & Zimmerman, B. (1997). Social origins of self-regulatory competence. Educational Psychologist, 32(4), 195–208.

    Article  Google Scholar 

  • Segedy, J.R., Kinnebrew, J.S., Biswas, G. (2012). Supporting student learning using conversational agents in a teachable agent environment. In Proceedings of the 10th international conference of the learning sciences.

  • Segedy, J.R., Kinnebrew, J.S., Biswas, G. (2013). The effect of contextualized conversational feedback in a complex open-ended learning environment. Educational Technology Research and Development, 61(1), 71–89.

    Article  Google Scholar 

  • Segedy, J.R., Biswas, G., Sulcer, B. (2014). A model-based behavior analysis approach for open-ended environments. Educational Technology & Society, 17(1).

  • Sotiriou, C., Wirapati, P., Loi, S., Harris, A., Fox, S., Smeds, J., Nordgren, H., Farmer, P., Praz, V., Haibe-Kains, B., et al. (2006). Gene expression profiling in breast cancer: understanding the molecular basis of histologic grade to improve prognosis. Journal of the National Cancer Institute, 98(4), 262–272.

    Article  Google Scholar 

  • Sperling, R., Howard, B., Miller, L., Murphy, C. (2002). Measures of children’s knowledge and regulation of cognition. Contemporary Educational Psychology, 27(1), 51–79.

    Article  Google Scholar 

  • Su, J.M., Tseng, S.S., Wang, W., Weng, J.F., Yang, J., Tsai, W.N. (2006). Learning portfolio analysis and mining for scorm compliant environment. Journal of Educational Technology and Society, 9(1), 262–275.

    Google Scholar 

  • Tang, T., & McCalla, G. (2002). Student modeling for a web-based learning environment: a data mining approach. In Proceedings of the 18th national conference on artificial intelligence (pp. 967–968). AAAI Press: Association for the Advancement of Artificial Intelligence.

  • VanLehn, K. (1996). Cognitive skill acquisition. Annual Review of Psychology, 47(1), 513–539.

    Article  Google Scholar 

  • Veenman, M., & Spaans, M. (2005). Relation between intellectual and metacognitive skills: age and task differences. Learning and Individual Differences, 15(2), 159–176.

    Article  Google Scholar 

  • Wagster, J., Tan, J., Wu, Y., Biswas, G., Schwartz, D. (2007). Do learning by teaching environments with metacognitive support help students develop better learning behaviors? In Proceedings of the 29th annual meeting of the cognitive science society (pp. 695–700).

  • Weinstein, C.E., & Mayer, R.E. (1986). The teaching of learning strategies. In M.C. Wittrock (Ed.), Handbook of research on teaching, 3rd edn. (pp. 315–327). New York: Macmillan.

  • Whitebread, D., & Cárdenas, V. (2012). Self-regulated learning and conceptual development in young children: the development of biological understanding. Metacognition in Science Education, 101–132.

  • Wilkinson, L., & Friendly, M. (2009). The history of the cluster heat map. The American Statistician, 63(2), 179–184.

    Article  Google Scholar 

  • Winne, P. (1996). A metacognitive view of individual differences in self-regulated learning. Learning and Individual Differences, 8(4), 327–353.

    Article  Google Scholar 

  • Winne, P. (2008). The weave of motivation and self-regulated learning. In D. Schunk, & B. Zimmerman (Eds.), Motivation and self-regulated learning: theory, research, and applications (pp. 297–314). NY: Taylor & Francis.

  • Winne, P., & Jamieson-Noel, D. (2002). Exploring students’ calibration of self reports about study tactics and achievement. Contemporary Educational Psychology, 27(4), 551–572.

    Article  Google Scholar 

  • Winne, P.H., & Hadwin, A.F. (1998). Studying as self-regulated learning. In D.J. Hacker, J. Dunlosky, A. Graesser (Eds.), Metacognition in educational theory and practice (pp. 277–304). Lawrence Erlbaum Associates Publishers.

  • Zhang, M., & Quintana, C. (2012). Scaffolding strategies for supporting middle school students online inquiry processes. Computers and Education, 58(1), 181–196.

    Article  Google Scholar 

  • Zimmerman, B. (2001). Theories of self-regulated learning and academic achievement. An overview and analysis. In B. Zimmerman, & D. Schunk (Eds.), Self-regulated learning and academic achievement: theoretical perspectives (pp. 1–37). Mahwah: Erlbaum.

  • Zimmerman, B., & Martinez-Pons, M. (1986). Development of a structured interview for assessing student use of self-regulated learning strategies. American Educational Research Journal, 23(4), 614–628.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of EECS/ISIS, Vanderbilt University, Nashville, TN, 37235, USA

    John S. Kinnebrew, James R. Segedy & Gautam Biswas

Authors
  1. John S. Kinnebrew
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. James R. Segedy
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gautam Biswas
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gautam Biswas.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Kinnebrew, J.S., Segedy, J.R. & Biswas, G. Analyzing the temporal evolution of students’ behaviors in open-ended learning environments. Metacognition Learning 9, 187–215 (2014). https://doi.org/10.1007/s11409-014-9112-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11409-014-9112-4

Keywords

Search

Navigation