Problematizing Helps! A Classroom Study of Computer-Based Guidance for Invention Activities

  • Catherine C. ChaseEmail author
  • Helena Connolly
  • Marianna Lamnina
  • Vincent Aleven


A successful instructional method is to engage learners with exploratory problem-solving before providing explanations of the canonical solutions and foundational concepts. A key question is whether and what type of guidance will lead learners to explore more productively and how this guidance will affect subsequent learning and transfer. We investigate this question through the design and study of the Invention Coach, an adaptive, computer-based learning environment that problematizes students’ understanding as they invent fundamental physics equations. Problematizing guidance (Reiser Journal of the Learning Sciences, 13(3), 273–304, 2004), which encourages learners to grapple with domain complexity, is well-suited to the goals of Invention. However, there are few examples of technology-based learning environments that were explicitly designed to problematize and scant research on their efficacy. In an experimental study, 199 middle schoolers worked with either motivational, task + motivational, or problematizing + task + motivational guidance versions of the Coach while inventing. Students who engaged with the problematizing Coach were better able to transfer their knowledge to novel domains in the short term, and their transfer gains were comparable to those provoked by human tutors. While students in the problematizing condition were less likely to invent the correct solutions, they engaged in more targeted and efficient exploration of the solution space and were less likely to report experiences of difficulty. Findings suggest that problematizing guidance has the potential to effectively support exploratory problem-solving, when the goal is to facilitate productive exploration and transfer from subsequent instruction. The work also has implications for the design of problematizing guidance.


Invention Exploratory learning environments Scaffolding STEM learning Adaptive technologies Productive failure 



We thank Jenna Marks for helping to design the Invention Coach and Octav Popescu and Martin van Velsen for building it. This work was funded by the National Science Foundation (award # 1361062).


  1. Aleven, V., McLaren, B. M., Sewall, J., van Velsen, M., Popescu, O., Demi, S., … Koedinger, K. R. (2016a). Example-tracing tutors: Intelligent tutor development for non-programmers. International Journal of Artificial Intelligence in Education, 26(1), 224–269.
  2. Aleven, V., McLaughlin, E. A., Glenn, R. A., & Koedinger, K. R. (2016b). Instruction based on adaptive learning technologies. Handbook of research on learning and instruction. Routledge.Google Scholar
  3. Aleven, V., Connolly, H., Popescu, O., Marks, J., Lamnina, M., & Chase, C. (2017, June). An adaptive coach for invention activities. In International conference on artificial intelligence in education (pp. 3–14). Cham: Springer.Google Scholar
  4. Alfieri, L., Brooks, P. J., Aldrich, N. J., & Tenenbaum, H. R. (2011). Does discovery-based instruction enhance learning? Journal of Educational Psychology, 103(1), 1–18.Google Scholar
  5. Biswas, G., Leelawong, K., Schwartz, D., Vye, N., & The Teachable Agents Group at Vanderbilt. (2005). Learning by teaching: A new agent paradigm for educational software. Applied Artificial Intelligence, 19(3–4), 363–392.Google Scholar
  6. Bjork, R. A. (1994). Memory and metamemory considerations in the. Metacognition: Knowing about knowing, 185.Google Scholar
  7. Bransford, J. D., & Schwartz, D. L. (1999). Chapter 3: Rethinking transfer: A simple proposal with multiple implications. Review of Research in Education, 24(1), 61–100.Google Scholar
  8. Bransford, J. D., Franks, J. J., Vye, N. J., & Sherwood, R. D. (1989). New approaches to instruction: Because wisdom can’t be told. Similarity and analogical reasoning, 470, 497.Google Scholar
  9. Chase, C. C., Chin, D. B., Oppezzo, M. A., & Schwartz, D. L. (2009). Teachable agents and the protégé effect: Increasing the effort towards learning. Journal of Science Education and Technology, 18(4), 334–352.Google Scholar
  10. Chase, C. C., Marks, J., Bernett, D., Bradley, M., & Aleven, V. (2015, June). Towards the development of the invention coach: A naturalistic study of teacher guidance for an exploratory learning task. In International Conference on Artificial Intelligence in Education (pp. 558–561). Springer, Cham.Google Scholar
  11. Chase, C. C., Malkiewich, L. M., & Kumar, A. (2019). Learning to notice science concepts in engineering activities and transfer situations. Science Education, 103(2), 440–471.Google Scholar
  12. Chen, W., Looi, C.-K., & Wen, Y. (2011). A scaffolded software tool for L2 vocabulary learning: GroupScribbles with graphic organizers. In H. Spada, G. Stahl, N. Miyake & N. Law (Eds.), Proceedings of Computer Supported Collaborative Learning (CSCL) 2011 (Part 1, pp. 414–421). Hong Kong: International Society of the Learning Sciences.Google Scholar
  13. Chi, M. T. (2009). Active-constructive-interactive: A conceptual framework for differentiating learning activities. Topics in Cognitive Science, 1(1), 73–105.Google Scholar
  14. Chi, M. T., De Leeuw, N., Chiu, M. H., & LaVancher, C. (1994). Eliciting self-explanations improves understanding. Cognitive Science, 18(3), 439–477.Google Scholar
  15. Cohen, P. A., Kulik, J. A., & Kulik, C. L. C. (1982). Educational outcomes of tutoring: A meta-analysis of findings. American Educational Research Journal, 19(2), 237–248.Google Scholar
  16. Collins, A., Warnock, E. H., Aiello, N., & Miller, M. L. (1975). Reasoning from incomplete knowledge. In Representation and understanding (pp. 383–415). Morgan Kaufmann.Google Scholar
  17. Day, S. B., & Goldstone, R. L. (2012). The import of knowledge export: Connecting findings and theories of transfer of learning. Educational Psychologist, 47(3), 153–176.Google Scholar
  18. Efstathiou, C., Hovardas, T., Xenofontos, N. A., Zacharia, Z. C., Anjewierden, A., & van Riesen, S. A. (2018). Providing guidance in virtual lab experimentation: The case of an experiment design tool. Educational Technology Research and Development, 66(3), 767–791.Google Scholar
  19. Feng, C. Y., & Chen, M. P. (2014). The effects of goal specificity and scaffolding on programming performance and self-regulation in game design. British Journal of Educational Technology, 45(2), 285–302.Google Scholar
  20. Gick, M. L., & Holyoak, K. J. (1983). Schema induction and analogical transfer. Cognitive Psychology, 15(1), 1–38.Google Scholar
  21. Hicks, D., & Doolittle, P. E. (2008). Fostering analysis in historical inquiry through multimedia embedded scaffolding. Theory & Research in Social Education, 36(3), 206–232.Google Scholar
  22. Holmes, N. G., Day, J., Park, A. H., Bonn, D. A., & Roll, I. (2014). Making the failure more productive: Scaffolding the invention process to improve inquiry behaviors and outcomes in invention activities. Instructional Science, 42(4), 523–538.Google Scholar
  23. Kapur, M. (2008). Productive failure. Cognition and Instruction, 26(3), 379–424.Google Scholar
  24. Kapur, M. (2010). Productive failure in mathematical problem solving. Instructional Science, 38(6), 523–550.Google Scholar
  25. Kapur, M. (2011). A further study of productive failure in mathematical problem solving: Unpacking the design components. Instructional Science, 39(4), 561–579.Google Scholar
  26. Kapur, M. (2014). Productive failure in learning math. Cognitive Science, 38(5), 1008–1022.Google Scholar
  27. Kapur, M., & Bielaczyc, K. (2012). Designing for productive failure. Journal of the Learning Sciences, 21(1), 45–83.Google Scholar
  28. Klahr, D., & Chen, Z. (2011). Finding one’s place in transfer space. Child Development Perspectives, 5(3), 196–204.Google Scholar
  29. Koedinger, K. R., Baker, R. S., Cunningham, K., Skogsholm, A., Leber, B., & Stamper, J. (2010). A data repository for the EDM community: The PSLC DataShop. Handbook of educational data mining, 43, 43–56.Google Scholar
  30. Lamnina, M. & Chase C.C. (2018, April) How different types of uncertainty affect learning, transfer, curiosity, and affect. Poster presented at the annual meeting of the American Educational Research Association, New York, N.Y.Google Scholar
  31. Lepper, M. R., Drake, M. F., & O'Donnell-Johnson, T. (1997). Scaffolding techniques of expert human tutors. In K. Hogan & M. Pressley (Eds.), Advances in learning & teaching. Scaffolding student learning: Instructional approaches and issues (pp. 108–144). Cambridge: Brookline Books.Google Scholar
  32. Linn, M. C., Bell, P., & Davis, E. A. (2004). Internet environments for science education. Mahwah: Lawrence Erlbaum Associates, Inc.Google Scholar
  33. Loibl, K., & Rummel, N. (2014). The impact of guidance during problem-solving prior to instruction on students’ inventions and learning outcomes. Instructional Science, 42, 305–326.Google Scholar
  34. Loibl, K., Roll, I., & Rummel, N. (2017). Towards a theory of when and how problem solving followed by instruction supports learning. Educational Psychology Review, 29(4), 693–715.Google Scholar
  35. Lynch, C., Ashley, K. D., Pinkwart, N., & Aleven, V. (2009). Concepts, structures, and goals: Redefining ill-definedness. International Journal of Artificial Intelligence in Education, 19(3), 253–266.Google Scholar
  36. Manu Kapur, (2012) Productive failure in learning the concept of variance. Instructional Science 40 (4):651–672Google Scholar
  37. Marks, J., Bernett, D., & Chase, C. C. (2016). The invention coach: Integrating data and theory in the Design of an Exploratory Learning Environment. International Journal of Designs for Learning, 7(2).Google Scholar
  38. Mayer, R. E. (2004). Should there be a three-strikes rule against pure discovery learning? American Psychologist, 59(1), 14–19.Google Scholar
  39. Molenaar, I., van Boxtel, C. A., & Sleegers, P. J. (2011). Metacognitive scaffolding in an innovative learning arrangement. Instructional Science, 39(6), 785–803.Google Scholar
  40. Piaget, J. (1977). The development of thought: Equilibration of cognitive structures. New York: Viking.Google Scholar
  41. Reiser, B. J. (2004). Scaffolding complex learning: The mechanisms of structuring and problematizing student work. Journal of the Learning Sciences, 13(3), 273–304.Google Scholar
  42. Roll, I., Holmes, N. G., Day, J., & Bonn, D. (2012). Evaluating metacognitive scaffolding in guided invention activities. Instructional Science, 40(4), 691–710.Google Scholar
  43. Sandoval, W. A. (2003). Conceptual and epistemic aspects of students' scientific explanations. The Journal of the Learning Sciences, 12(1), 5–51.Google Scholar
  44. Schwartz, D. L., & Bransford, J. D. (1998). A time for telling. Cognition and Instruction, 16(4), 475–5223.Google Scholar
  45. Schwartz, D. L., & Martin, T. (2004). Inventing to prepare for future learning: The hidden efficiency of encouraging original student production in statistics instruction. Cognition and Instruction, 22(2), 129–184.Google Scholar
  46. Schwartz, D. L., Chase, C. C., Oppezzo, M. A., & Chin, D. B. (2011). Practicing versus inventing with contrasting cases: The effects of telling first on learning and transfer. Journal of Educational Psychology, 103(4), 759–775.Google Scholar
  47. Shemwell, J. T., Chase, C. C., & Schwartz, D. L. (2015). Seeking the general explanation: A test of inductive activities for learning and transfer. Journal of Research in Science Teaching, 52(1), 58–83.Google Scholar
  48. Shute, V. J., & Zapata-Rivera, D. (2012). Adaptive educational systems. Adaptive technologies for training and education, 7(27), 1–35.Google Scholar
  49. VanLehn, K. (2006). The behavior of tutoring systems. International Journal of Artificial Intelligence in Education, 16(3), 227–265.Google Scholar
  50. Wiedmann, M., Leach, R. C., Rummel, N., & Wiley, J. (2012). Does group composition affect learning by invention? Instructional Science, 40(4), 711–730.Google Scholar
  51. Wood, D., Bruner, J. S., & Ross, G. (1976). The role of tutoring in problem solving. Journal of Child Psychology and Psychiatry, 17(2), 89–100.Google Scholar

Copyright information

© International Artificial Intelligence in Education Society 2019

Authors and Affiliations

  1. 1.Teachers CollegeColumbia UniversityNew YorkUSA
  2. 2.Carnegie Mellon UniversityPittsburghUSA

Personalised recommendations