Learner modeling for adaptive scaffolding in a Computational Thinking-based science learning environment

  • Satabdi Basu
  • Gautam Biswas
  • John S. Kinnebrew


Learner modeling has been used in computer-based learning environments to model learners’ domain knowledge, cognitive skills, and interests, and customize their experiences in the environment based on this information. In this paper, we develop a learner modeling and adaptive scaffolding framework for Computational Thinking using Simulation and Modeling (CTSiM)—an open ended learning environment that supports synergistic learning of science and Computational Thinking (CT) for middle school students. In CTSiM, students have the freedom to choose and coordinate use of the different tools provided in the environment, as they build and test their models. However, the open-ended nature of the environment makes it hard to interpret the intent of students’ actions, and to provide useful feedback and hints that improves student understanding and helps them achieve their learning goals. To address this challenge, we define an extended learner modeling scheme that uses (1) a hierarchical task model for the CTSiM environment, (2) a set of strategies that support effective learning and model building, and (3) effectiveness and coherence measures that help us evaluate student’s proficiency in the different tasks and strategies. We use this scheme to dynamically scaffold learners when they are deficient in performing their tasks, or they demonstrate suboptimal use of strategies. We demonstrate the effectiveness of our approach in a classroom study where one group of 6th grade students received scaffolding and the other did not. We found that students who received scaffolding built more accurate models, used modeling strategies effectively, adopted more useful modeling behaviors, showed a better understanding of important science and CT concepts, and transferred their modeling skills better to new scenarios.


Open ended learning environments Modeling and simulation Learning by modeling Computational Thinking Science education Learner modeling Adaptive scaffolding 



This work was supported by the NSF (NSF Cyber-learning Grants #1124175 and #1441542).


  1. Aleven, V., McLaren, B., Roll, I., Koedinger, K.: Toward tutoring help seeking. In: Intelligent Tutoring Systems, pp. 227–239. Springer, Berlin (2004)Google Scholar
  2. Aleven, V., McLaren, B.M., Roll, I., Koedinger, K.R.: Toward meta-cognitive tutoring: a model of help seeking with a cognitive tutor. I. J. Artif. Intell. Educ. 16(2), 101–128 (2006)Google Scholar
  3. Anderson, J.R., Corbett, A.T., Koedinger, K.R., Pelletier, R.: Cognitive tutors: lessons learned. J. Learn. Sci. 4(2), 167–207 (1995)CrossRefGoogle Scholar
  4. Anderson, J.R., Boyle, C.F., Reiser, B.J.: Intelligent tutoring systems. Science 228, 456–462 (1985)CrossRefGoogle Scholar
  5. Arts, J.A., Gijselaers, W.H., Segers, M.S.: Cognitive effects of an authentic computer-supported, problem-based learning environment. Inst. Sci. 30(6), 465–495 (2002)CrossRefGoogle Scholar
  6. Azevedo, R.: Using hypermedia as a metacognitive tool for enhancing student learning? The role of self-regulated learning. Educ. Psychol. 40(4), 199–209 (2005)CrossRefGoogle Scholar
  7. Azevedo, R., Hadwin, A.F.: Scaffolding self-regulated learning and metacognition-implications for the design of computer-based scaffolds. Instr. Sci. 33(5), 367–379 (2005)CrossRefGoogle Scholar
  8. Baker, R. S., Corbett, A. T., Koedinger, K. R., Wagner, A. Z.: Off-task behavior in the cognitive tutor classroom: when students game the system. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 383–390. ACM, New York (2004)Google Scholar
  9. Bangert-Drowns, R.L., Kulik, C.L.C., Kulik, J.A., Morgan, M.: The instructional effect of feedback in test-like events. Rev. Educ. Res. 61(2), 213–238 (1991)CrossRefGoogle Scholar
  10. Bannert, M., Reimann, P.: Supporting self-regulated hypermedia learning through prompts. Instr. Sci. 40(1), 193–211 (2012)CrossRefGoogle Scholar
  11. Basu, S., Dickes, A., Kinnebrew, J.S., Sengupta, P., Biswas, G.: CTSiM: a computational thinking environment for learning science through simulation and modeling. In: Proceedings of the 5th International Conference on Computer Supported Education, pp. 369–378. Aachen, Germany (2013)Google Scholar
  12. Basu, S., Dukeman, A., Kinnebrew, J., Biswas, G., Sengupta, P.: Investigating student generated computational models of science. In: Proceedings of the 11th International Conference of the Learning Sciences, Boulder, CO (2014a)Google Scholar
  13. Basu, S., Kinnebrew, J., Biswas, G.: Assessing student performance in a computational-thinking based science learning environment. Proceedings of the 12th International Conference on Intelligent Tutoring Systems, pp. 476–481. Springer International Publishing, Honolulu, HI, USA (2014b)CrossRefGoogle Scholar
  14. Basu, S., Sengupta, P., Dickes, A., Biswas, G., Kinnebrew, J.S., Clark, D.: Identifying middle school students’ challenges in computational thinking based science learning. Res. Pract. Technol. Enhanc. Learn. (2016a)Google Scholar
  15. Basu, S., Biswas, G., Kinnebrew, J.S.: Using multiple representations to simultaneously learn computational thinking and middle school science. In: Thirtieth AAAI Conference on Artificial Intelligence, Phoenix, AZ (2016b). doi: 10.1186/s41039-016-0036-2
  16. Biswas, G., Jeong, H., Kinnebrew, J., Sulcer, B., Roscoe, R.: Measuring self-regulated learning skills through social interactions in a teachable agent environment. Res. Pract. Technol. Enhanc. Learn. 5(2), 123–152 (2010)CrossRefGoogle Scholar
  17. Bransford, J., Schwartz, D.: Rethinking transfer: a simple proposal with multiple implications. Rev. Res. Educ. 24(1), 61–101 (1999)CrossRefGoogle Scholar
  18. Brown, J.S., Burton, R.R.: Diagnostic models for procedural bugs in basic mathematical skills. Cogn. Sci. 2(2), 155–192 (1978)CrossRefGoogle Scholar
  19. Brown, J.S., VanLehn, K.: Repair theory: a generative theory of bugs in procedural skills. Cogn. Sci. 4(4), 379–426 (1980)CrossRefGoogle Scholar
  20. Brusilovsky, P., Peylo, C.: Adaptive and intelligent web-based educational systems. Int. J. Artif. Intell. Educ. 13, 159–172 (2003)Google Scholar
  21. Brusilovsky, P., Millán, E.: User models for adaptive hypermedia and adaptive educational systems. In: The Adaptive Web, pp. 3–53. Springer, Berlin (2007)Google Scholar
  22. Carr, B. Goldstein, I.P.: Overlays: a theory of modelling for computer aided instruction (No. AI-M-406). Massachusetts Institute of Technology Cambridge Artificial Intelligence Lab (1977)Google Scholar
  23. Chandler, P., Sweller, J.: The split-attention effect as a factor in the design of instruction. Br. J. Educ. Psychol. 62(2), 233–246 (1992)CrossRefGoogle Scholar
  24. Chrysafiadi, K., Virvou, M.: Student modeling approaches: a literature review for the last decade. Expert Syst. Appl. 40(11), 4715–4729 (2013)CrossRefGoogle Scholar
  25. Clarebout, G., Elen, J.: Advice on tool use in open learning environments. J. Educ. Multimed. Hypermed. 17(1), 81–97 (2008)Google Scholar
  26. Conati, C., Gertner, A., Vanlehn, K.: Using Bayesian networks to manage uncertainty in student modeling. User Model. User Adap. Interact. 12(4), 371–417 (2002)CrossRefzbMATHGoogle Scholar
  27. Conejo, R., Guzman, E., Mill’an, E., Trella, M., P’erez-de-la Cruz, J.L., Rios, A.: SIETTE: a web-based tool for adaptive teaching. Int. J. Artif. Intell. Educ. 14, 29–61 (2004)Google Scholar
  28. Corbett, A.T., Anderson, J.R.: Knowledge tracing: modeling the acquisition of procedural knowledge. User Model. User Adap. Interact. 4(4), 253–278 (1995)CrossRefGoogle Scholar
  29. Desmarais, M.C., d Baker, R.S.: A review of recent advances in learner and skill modeling in intelligent learning environments. User Model. User Adap. Inter. 22(1–2), 9–38 (2012)CrossRefGoogle Scholar
  30. Dolog, P. M. Schaefer.: A framework for browsing, manipulating and maintaining interoperable learner profiles. In: Proceedings of the UM2005—10th International Conference on User Modeling. Edinburgh, UK: Springer, Berlin (2005)Google Scholar
  31. Dolog, P., Simon, B., Nejdl, W., Klobučar, T.: Personalizing access to learning networks. ACM Trans. Internet Technol. 8(2), 3 (2008)CrossRefGoogle Scholar
  32. Duque, R., Bollen, L., Anjewierden, A., Bravo, C.: Automating the analysis of problem-solving activities in learning environments: the co-lab case study. J. UCS 18(10), 1279–1307 (2012)Google Scholar
  33. Elsom-Cook, M.: Student modelling in intelligent tutoring systems. Artif. Intell. Rev. 7(3–4), 227–240 (1993)CrossRefGoogle Scholar
  34. Fedor, D.B., Davis, W.D., Maslyn, J.M., Mathieson, K.: Performance improvement efforts in response to negative feedback: the roles of source power and recipient self-esteem. J. Manag. 27(1), 79–97 (2001)Google Scholar
  35. Gobert, J., Sao Pedro, M., Raziuddin, J., Baker, R.S.: From log files to assessment metrics: measuring students’ science inquiry skills using educational data mining. J. Learn. Sci. 22(4), 521–563 (2013). doi: 10.1080/10508406.2013.837391 CrossRefGoogle Scholar
  36. Goldstein, I.P.: The genetic graph: a representation for the evolution of procedural knowledge. Int. J. Man Mach. Stud. 11(1), 51–77 (1979)CrossRefGoogle Scholar
  37. Grawemeyer, B., Mavrikis, M., Holmes, W., Gutiérrez-Santos,S., Wiedmann, M., Rummel, N. Affective learning. Exploring the impact of affect-aware support on learning and engagement. User Model. User Adapt. Interact. J. Personal. Res. 27 (2017) this issueGoogle Scholar
  38. Grover, S., Pea, R.: Computational Thinking in K-12: a review of the state of the field. Educ. Res. 42(1), 38–43 (2013)CrossRefGoogle Scholar
  39. Grover, S., Cooper, S., Pea, R.: Assessing computational learning in K-12. In: Proceedings of the 2014 Conference on Innovation & Technology in Computer Science Education, pp. 57–62. ACM, New York (2014)Google Scholar
  40. Jeremić, Z., Jovanović, J., Gašević, D.: Student modeling and assessment in intelligent tutoring of software patterns. Expert Syst. Appl. 39(1), 210–222 (2012)CrossRefGoogle Scholar
  41. Jona, K., Wilensky, U., Trouille, L., Horn, MS., Orton, K., Weintrop, D., Beheshti, E.: Embedding computational thinking in science, technology, engineering, and math (CT-STEM). In: Paper Presented at the Future Directions in Computer Science Education Summit Meeting, Orlando, FL (2014)Google Scholar
  42. Jovanović, J., Gasevic, D., Brooks, C., Devedzic, V., Hatala, M., Eap, T., Richards, G.: LOCO-analyst: semantic web technologies in learning content usage analysis. Int. J. Contin. Eng. Educ. Life Long Learn. 18(1), 54–76 (2008)CrossRefGoogle Scholar
  43. Jovanović, J., Gašević, D., Torniai, C., Bateman, S., Hatala, M.: The social semantic web in intelligent learning environments: state of the art and future challenges. Interact. Learn. Environ. 17(4), 273–309 (2009)CrossRefGoogle Scholar
  44. Karabenick, S.A., Knapp, J.R.: Relationship of academic help seeking to the use of learning strategies and other instrumental achievement behavior in college students. J. Educ. Psychol. 83(2), 221 (1991)CrossRefGoogle Scholar
  45. Kinnebrew, J.S., Loretz, K.M., Biswas, G.: A contextualized, differential sequence mining method to derive students’ learning behavior patterns. J. Educ. Data Min. 5(1), 190–219 (2013)Google Scholar
  46. Kinnebrew, J.S., Segedy, J.R., Biswas, G.: Analyzing the temporal evolution of students’ behaviors in open-ended learning environments. Metacogn. Learn. 9(2), 187–215 (2014)CrossRefGoogle Scholar
  47. Kinnebrew, J., Segedy, J.R. Biswas, G.: Integrating model-driven and data-driven techniques for analyzing learning behaviors in open-ended learning environments. IEEE Trans. Learn. Technol. doi: 10.1109/TLT.2015.2513387
  48. Klawe, M., Inkpen, K., Phillips, E., Upitis, R., Rubin, A.: E-GEMS: a project on computer games, mathematics and gender (2002)Google Scholar
  49. Koedinger, K.R., Aleven, V.: Exploring the assistance dilemma in experiments with cognitive tutors. Educ. Psychol. Rev. 19(3), 239–264 (2007)CrossRefGoogle Scholar
  50. Kramarski, B., Gutman, M.: How can self-regulated learning be supported in mathematical E-learning environments? J. Comput. Assist. Learn. 22(1), 24–33 (2006)CrossRefGoogle Scholar
  51. Lajoie, S., Derry, S. (eds.): Computers as Cognitive Tools. Lawrence Erlbaum Associates, Mahwah, NJ (1993)Google Scholar
  52. Land, S.: Cognitive requirements for learning with open-ended learning environments. Educ. Tech. Res. Dev. 48(3), 61–78 (2000)CrossRefGoogle Scholar
  53. Land, S., Hannafin, M., Oliver, K.: Student-centered learning environments: foundations, assumptions and design. In: Jonassen, D., Land, S. (eds.) Theoretical Foundations of Learning Environments, pp. 3–25. Routledge, New York, NY (2012)Google Scholar
  54. Langley, P. Ohlsson, S.: Automated Cognitive Modelling. In: Proceedings of AAAI—84, pp. 193–197 (1984)Google Scholar
  55. Lepper, M.R., Chabay, R.W.: Intrinsic motivation and instruction: conflicting views on the role of motivational processes in computer-based education. Educ. Psychol. 20(4), 217–230 (1985)CrossRefGoogle Scholar
  56. Long, Y. Aleven, V. (2017). Enhancing learning outcomes through self-regulated learning support with an open learner model. User Model. User Adapt. Interact. J. Personal. Res. 27 (2016). doi: 10.1007/s11257-016-9186-6
  57. Luckin, R., du Boulay, B.: Ecolab: the development and evaluation of a vygotskian design framework. Int. J. Artif. Intell. Educ. 10(2), 198–220 (1999)Google Scholar
  58. Mitrovic, A.: Fifteen years of constraint-based tutors: what we have achieved and where we are going. User Model. User Adapt. Interact. 22(1–2), 39–72 (2012)CrossRefGoogle Scholar
  59. Maloney, J., Burd, L., Kafai, Y., Rusk, N., Silverman, B., Resnick, M.: Scratch: A sneak preview. In: Proceedings of Creating, Connecting, and Collaborating Through Computing, pp. 104–109 (2004)Google Scholar
  60. McCalla, G.I., Murtagh, K.: G.E.N.I.U.S.: an experiment in ignorance-based automated program advising. AISB Newsl. 75, 13–20 (1991)Google Scholar
  61. Montalvo, O., Baker, R.S.J., Sao Pedro, M.A., Nakama, A., Gobert, J.D.: Identifying student’ inquiry planning using machine learning. In: Proceedings of the 3rd International Conference on Educational Data Mining, pp. 141–150, Pittsburgh, PA (2010)Google Scholar
  62. Moos, D.C., Honkomp, B.: Adventure learning: motivating students in a Minnesota middle school. J. Res. Technol. Educ. 43(3), 231–252 (2011)CrossRefGoogle Scholar
  63. Ohlsson, S.: Some principles of intelligent tutoring. Instr. Sci. 14(3–4), 293–326 (1986)CrossRefGoogle Scholar
  64. Pelánek R., Papoušek, J., Řihák, J., Stanislav, V., Nižnan, J.: Elo-based learner modeling for adaptive practice of facts. User Model. User Adapt. Interact. J. Personal. Res. (2016). doi: 10.1007/s11257-016-9185-7
  65. Pressley, M., Goodchild, F., Fleet, J., Zajchowski, R., Evansi, E.: The challenges of classroom strategy instruction. Elem. School J. 89, 301–342 (1989)CrossRefGoogle Scholar
  66. Puntambekar, S., Hubscher, R.: Tools for scaffolding students in a complex learning environment: what have we gained and what have we missed? Educ. Psychol. 40(1), 1–12 (2005)CrossRefGoogle Scholar
  67. Roll, I., Aleven, V.,Mclaren, B.M., Koedinger, K.R.: Can help seeking be tutored? Searching for the secret sauce of metacognitive tutoring. In: Artificial Intelligence in Education (AIED 2007), pp. 203–210 (2009)Google Scholar
  68. Schraw, G., Crippen, K.J., Hartley, K.: Promoting self-regulation in science education: metacognition as part of a broader perspective on learning. Res. Sci. Educ. 36(1–2), 111–139 (2006)CrossRefGoogle Scholar
  69. Schwartz, D.L., Arena, D.: Measuring What Matters Most: Choice-Based Assessments for the Digital Age. MIT Press, Cambridge (2013)Google Scholar
  70. Segedy, J.R., Kinnebrew, J.S., Biswas, G.: The effect of contextualized conversational feedback in a complex open-ended learning environment. Educ. Tech. Res. Dev. 61(1), 71–89 (2013)CrossRefGoogle Scholar
  71. Self, J.: The defining characteristics of intelligent tutoring systems research: ITSs care, precisely. Int. J. Artif. Intell. Educ. 10, 350–364 (1998)Google Scholar
  72. Sengupta, P., Kinnebrew, J.S., Basu, S., Biswas, G., Clark, D.: Integrating computational thinking with K-12 science education using agent-based computation: a theoretical framework. Educ. Inf. Technol. 18(2), 351–380 (2013)CrossRefGoogle Scholar
  73. Shang, Y., Shi, H., Chen, S.S.: An intelligent distributed environment for active learning. J. Educ. Resourc. Comput. 1(2es), 4 (2001)Google Scholar
  74. Shute, V.J.: Focus on formative feedback. Rev. Educ. Res. 78(1), 153–189 (2008)CrossRefGoogle Scholar
  75. Sison, R., Shimura, M.: Student modeling and machine learning. Int. J. Artif. Intell. Educ. 9, 128–158 (1998)Google Scholar
  76. Van der Kleij, F.M., Feskens, R.C., Eggen, T.J.: Effects of feedback in a computer-based learning environment on students’ learning outcomes a meta-analysis. Rev. Educ. Res. 85(4), 475–511 (2015)CrossRefGoogle Scholar
  77. VanLehn, K.: Student modeling. In: Polson, M.C., Richardson, J.J. (eds.) Foundations of Intelligent Tutoring Systems, pp. 55–78. Lawrence Erlbaum, Hillsdale, NJ (1988)Google Scholar
  78. Walonoski, J. A., Heffernan, N. T.: Detection and analysis of off-task gaming behavior in intelligent tutoring systems. In: Intelligent Tutoring Systems, pp. 382–391. Springer, Berlin (2006)Google Scholar
  79. Weber, G., Specht, M.: User modeling and adaptive navigation support in WWW-based tutoring systems. In: User Modeling, pp. 289–300. Springer, Vienna (1997)Google Scholar
  80. Wenger, E.: Artificial Intelligence and Tutoring Systems: Computational and Cognitive Approaches to the Communication of Knowledge. Morgan Kaufmann, Los Altos, CA (1987)Google Scholar
  81. Wilensky, U.: NetLogo. Center for Connected Learning and Computer-Based Modeling. Northwestern University, Evanston, IL. ( (1999)
  82. Wing, J.M.: Computational thinking. Commun. ACM 49(3), 33–35 (2006)CrossRefGoogle Scholar
  83. Winne, P.H.: Issues in researching self-regulated learning as patterns of events. Metacogn. Learn. 9(2), 229–237 (2014)CrossRefGoogle Scholar
  84. Winter, M., Brooks, C. A., Greer, J. E.: Towards Best Practices for Semantic Web Student Modelling. In: AIED, pp. 694–701 (2005)Google Scholar
  85. Wood, D., Bruner, J.S., Ross, G.: The role of tutoring in problem solving. J. Child Psychol. Psychiatr. 17(2), 89–100 (1976)CrossRefGoogle Scholar
  86. Woolf, B.P.: Building Intelligent Interactive Tutors: Student-Centered Strategies for Revolutionizing e-Learning. Morgan Kaufmann, Burlington, MA (2009)Google Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2017

Authors and Affiliations

  • Satabdi Basu
    • 1
  • Gautam Biswas
    • 2
  • John S. Kinnebrew
    • 3
  1. 1.SRI InternationalMenlo ParkUSA
  2. 2.Institute for Software Integrated Systems and EECS DepartmentVanderbilt UniversityNashvilleUSA
  3. 3.BridjBostonUSA

Personalised recommendations