It is not either or: An initial investigation into combining collaborative and individual learning using an ITS

  • Jennifer K. OlsenEmail author
  • Nikol Rummel
  • Vincent Aleven


Research on Computer-Supported Collaborative Learning (CSCL) has provided significant insights into why collaborative learning is effective and how we can effectively provide support for it. Building on this knowledge, we can investigate when collaboration is beneficial to support learning. Specifically, collaborative and individual learning are often combined in the classroom, and it is important for the CSCL community to understand when a combination is beneficial compared to individual or collaborative learning alone. Before investing significant work into discovering these details, an initial investigation is needed to determine if there may be any value in a combination. In this study, we compared a combined condition to individual or collaborative-only learning conditions using an intelligent tutoring system for fractions. The study was conducted with 382 4th and 5th grade students. Students across all three conditions had significant learning gains, but the combined condition had higher learning gains than the other conditions. However, this difference was restricted to the 4th grade students. By analyzing the hints and errors of students over time from process data, we found that students in the combined condition tended to make fewer errors both when working collaboratively and individually, and asked for fewer hints than the students in the other conditions. Students who collaborated (collaborative and combined conditions) also reported having higher situational interest in the activity. By finding support for the effectiveness of combining collaborative and individual learning, this paper opens a broader line of inquiry into how they can effectively be combined to support learning.


Collaboration Problem-solving Erroneous examples 



  1. Aleven, V., & Koedinger, K. R. (2001). Investigations into help seeking and learning with a cognitive tutor. In Papers of the AIED-2001 workshop on help provision and help seeking in interactive learning environments, 47-58.Google Scholar
  2. Aleven, V., Sewall, J., Popescu, O., van Velsen, M., Demi, S., & Leber, B. (2015). Reflecting on twelve years of ITS authoring tools research with CTAT. Design Recommendations for Adaptive Intelligent Tutoring Systems, 3, 263–283.Google Scholar
  3. Anderson, J. R. (1983). A spreading activation theory of memory. Journal of Verbal Learning and Verbal Behavior, 22(3), 261–295.CrossRefGoogle Scholar
  4. Anderson, J. R., Boyle, C. F., Corbett, A. T., & Lewis, M. W. (1990). Cognitive modeling and intelligent tutoring. Artificial Intelligence, 42, 7–49.CrossRefGoogle Scholar
  5. Aronson, E. (1978). The jigsaw classroom. Sage.Google Scholar
  6. Atkinson, R. K., Derry, S. J., Renkl, A., & Wortham, D. (2000). Learning from examples: Instructional principles from the worked examples research. Review of Educational Research, 70(2), 181–214.CrossRefGoogle Scholar
  7. Baghaei, N., & Mitrovic, A. (2005). COLLECT-UML: supporting individual and collaborative learning of UML class diagrams in a constraint-based intelligent tutoring system. In International Conference on Knowledge-Based and Intelligent Information and Engineering Systems (pp. 458–464). Berlin Heidelberg: Springer.Google Scholar
  8. Celepkolu, M., Wiggins, J.B., Boyer, K.E., McMullen, K. (2017). Think First: Fostering Substantive Contributions in Collaborative Problem-Solving Dialogues. In Proceedings of the 12 th International Conference on Computer Supported Collaborative Learning. (pp. 295-302).Google Scholar
  9. Chen, J., Wang, M., Kirschner, P. A., & Tsai, C. C. (2018). The role of collaboration, computer use, learning environments, and supporting strategies in CSCL: A meta-analysis. Review of Educational Research, 88(6), 799–843.CrossRefGoogle Scholar
  10. Chi, M. T., & Wylie, R. (2014). The ICAP framework: Linking cognitive engagement to active learning outcomes. Educational Psychologist, 49(4), 219–243.CrossRefGoogle Scholar
  11. Cress, U. (2008). The need for considering multilevel analysis in CSCL researchAn appeal for the use of more advanced statistical methods. International Journal of Computer-Supported Collaborative Learning, 3(1), 69–84.CrossRefGoogle Scholar
  12. Dehler, J., Bodemer, D., Buder, J., & Hesse, F. W. (2011). Guiding knowledge communication in CSCL via group knowledge awareness. Computers in Human Behavior, 27(3), 1068–1078.CrossRefGoogle Scholar
  13. Dillenbourg, P. (2002). Over-scripting CSCL: The risks of blending collaborative learning with instructional design. Three worlds of CSCL. Can we support CSCL? 61-91.Google Scholar
  14. Dillenbourg, P. (2004). Framework for integrated learning. Kaleidoscope Network of Excellence Deliverable D23.5.1.Google Scholar
  15. Dillenbourg, P., & Tchounikine, P. (2007). Flexibility in macro-scripts for computer-supported collaborative learning. Journal of Computer Assisted Learning, 23(1), 1–13.CrossRefGoogle Scholar
  16. Diziol, D., Rummel, N., Spada, H., & McLaren, B.M. (2007). Promoting learning in mathematics: Script support for collaborative problem solving with the Cognitive Tutor Algebra. In C.A. Chinn, G. Erkens & S. Puntambekar (Eds.) Mice, minds and society. Proceedings of the Computer Supported Collaborative Learning Conference (pp. 39-41). International Society of the Learning Sciences, Inc.Google Scholar
  17. Diziol, D., Walker, E., Rummel, N., & Koedinger, K. R. (2010). Using intelligent tutor technology to implement adaptive support for student collaboration. Educational Psychology Review, 22(1), 89–102.CrossRefGoogle Scholar
  18. Fischer, F., Kollar, I., Stegmann, K., & Wecker, C. (2013b). Toward a script theory of guidance in computer-supported collaborative learning. Educational Psychologist, 48(1), 56–66.CrossRefGoogle Scholar
  19. Fischer, F., Kollar, I., Stegmann, K., Wecker, C., Zottmann, J., & Weinberger, A. (2013a). Collaboration scripts in computer-supported collaborative learning. The International Handbook of Collaborative Learning, 403-419.Google Scholar
  20. Frank, M. C., & Gibson, E. (2011). Overcoming memory limitations in rule learning. Language, Learning, & Development, 7, 130–148.CrossRefGoogle Scholar
  21. Fraser, B. J., Walberg, H. J., Welch, W. W., & Hattie, J. A. (1987). Syntheses of educational productivity research. International Journal of Educational Research, 11(2), 147–252.CrossRefGoogle Scholar
  22. Hausmann, R. G., Chi, M. T., & Roy, M. (2004) Learning from collaborative problem solving: An analysis of three hypothesized mechanisms. In 26nd annual conference of the Cognitive Science society, 547-552.Google Scholar
  23. Hausmann, R. G., Nokes, T. J., VanLehn, K., & van de Sande, B. (2009). Collaborative Dialog While Studying Worked-out Examples. In International Conference on Artificial Intelligence in Education. (pp. 596-598).Google Scholar
  24. Hausmann, R. G. M., Van de Sande, B., & VanLehn, K. (2008a). Are self-explaining and coached problem solving more effective when done by pairs of students than alone? In B. C. Love, K. McRae, & V. M. Sloutsky (Eds.), Proceedings of the 30th Annual Conference of the Cognitive Science Society (pp. 2369–2374). Austin, TX: Cognitive Science Society.Google Scholar
  25. Hausmann, R. G., van de Sande, B., & VanLehn, K. (2008b). Trialog: How Peer Collaboration Helps Remediate Errors in an ITS. In Proceedings of the 21st International Florida Artificial Intelligence Research Society Conference (pp. 415–420). Menlo Park, CA: AAAI Press.Google Scholar
  26. Hidi, S., & Renninger, K. A. (2006). The four-phase model of interest development. Educational Psychologist, 41(2), 111–127.CrossRefGoogle Scholar
  27. Isotani, S., Adams, D., Mayer, R. E., Durkin, K., Rittle-Johnson, B., & McLaren, B. M. (2011). Can erroneous examples help middle-school students learn decimals? In Towards Ubiquitous Learning (pp. 181–195). Berlin Heidelberg: Springer.CrossRefGoogle Scholar
  28. Janssen, J., & Bodemer, D. (2013). Coordinated computer-supported collaborative learning: Awareness and awareness tools. Educational Psychologist, 48(1), 40–55.CrossRefGoogle Scholar
  29. Jeong, H., Hmelo-Silver, C. E., & Jo, K. (2019). Ten Years of Computer-Supported Collaborative Learning: A meta-analysis of CSCL in STEM education during 2005-2014. Educational Research Review, 100284.Google Scholar
  30. Kapur, M. (2010). Productive failure in mathematical problem solving. Instructional Science, 38(6), 523–550.CrossRefGoogle Scholar
  31. Kapur, M. (2014). Comparing learning from productive failure and vicarious failure. Journal of the Learning Sciences, 23(4), 651–677.CrossRefGoogle Scholar
  32. Koedinger, K. R., Corbett, A. C., & Perfetti, C. (2012). The Knowledge-Learning-Instruction (KLI) framework: Bridging the science-practice chasm to enhance robust student learning. Cognitive Science, 36(5), 757–798.CrossRefGoogle Scholar
  33. Kollar, I., Fischer, F., & Hesse, F. W. (2006). Collaboration scripts–a conceptual analysis. Educational Psychology Review, 18(2), 159–185.CrossRefGoogle Scholar
  34. Kulik, J. A., & Fletcher, J. D. (2015). Effectiveness of Intelligent Tutoring Systems: A Meta-Analytic Review. Review of Educational Research. Scholar
  35. Lester, J. C., Converse, S. A., Kahler, S. E., Barlow, S. T., Stone, B. A., & Bhogal, R. S. (1997). The persona effect: affective impact of animated pedagogical agents. In Proceedings of the ACM SIGCHI Conference on Human factors in computing systems (pp. 359-366). ACM.Google Scholar
  36. Ling, L. M., & Marton, F. (2012). Towards a science of the art of teaching. International journal for lesson and learning studies.Google Scholar
  37. Linnenbrink-Garcia, L., Durik, A. M., Conley, A. M., Barron, K. E., Tauer, J. M., Karabenick, S. A., & Harackiewicz, J. M. (2010). Measuring situational interest in academic domains. Educational and Psychological Measurement.Google Scholar
  38. Lou, Y., Abrami, P. C., & d’Apollonia, S. (2001). Small group and individual learning with technology: A meta-analysis. Review of Educational Research, 71(3), 449–521.CrossRefGoogle Scholar
  39. Lou, Y., Abrami, P. C., Spence, J. C., Poulsen, C., Chambers, B., & d’Apollonia, S. (1996). Within-class grouping: A meta-analysis. Review of Educational Research, 66(4), 423–458.CrossRefGoogle Scholar
  40. Ma, W., Adesope, O. O., Nesbit, J. C., & Liu, Q. (2014). Intelligent tutoring systems and learning outcomes: A meta-analysis. Journal of Educational Psychology, 106(4), 901.CrossRefGoogle Scholar
  41. Magnisalis, I., Demetriadis, S., & Karakostas, A. (2011). Adaptive and intelligent systems for collaborative learning support: A review of the field. IEEE Transactions on Learning Technologies, 4(1), 5–20.CrossRefGoogle Scholar
  42. Mazziotti, C., Loibl, K., & Rummel, N. (2015). Collaborative or Individual Learning within Productive Failure: Does the Social Form of Learning Make a Difference? In O. Lindwall et al. (Eds.), Proceedings of the 11 th International Conference on Computer Supported Collaborative Learning, (570-575). Gothenberg, Sweden.Google Scholar
  43. McLaren, B. M., Adams, D., Durkin, K., Goguadze, G., Mayer, R. E., Rittle-Johnson, B., Sosnovsky, S., Isotani, S., & Van Velsen, M. (2012). To err is human, to explain and correct is divine: A study of interactive erroneous examples with middle school math students. In 21st Century Learning for 21st Century Skills (pp. 222–235). Berlin Heidelberg: Springer.CrossRefGoogle Scholar
  44. Mitchell, M. (1993). Situational interest: Its multifaceted structure in the secondary school mathematics classroom. Journal of Educational Psychology, 85, 424–436.CrossRefGoogle Scholar
  45. Mullins, D., Rummel, N., & Spada, H. (2011). Are two heads always better than one? Differential effects of collaboration on students’ computer-supported learning in mathematics. International Journal of Computer-Supported Collaborative Learning, 6(3), 421–443.CrossRefGoogle Scholar
  46. Olsen, J. K., Aleven, V., & Rummel, N. (2017). Exploring Dual Eye Tracking as a Tool to Assess Collaboration. In Innovative Assessment of Collaboration (pp. 157–172). Cham: Springer.CrossRefGoogle Scholar
  47. Olsen, J. K., Belenky, D. M., Aleven, A., & Rummel, N. (2014b). Using an intelligent tutoring system to support collaborative as well as individual learning. In S. Trausan-Matu, K. E. Boyer, M. Crosby, & K. Panourgia (Eds.), Proceedings of the 12th International Conference on Intelligent Tutoring Systems (pp. 134–143). Berlin, Heidelberg: Springer.CrossRefGoogle Scholar
  48. Olsen, J. K., Belenky, D. M., Aleven, A., Rummel, N., Sewall, J., & Ringenberg, M. (2014a). Authoring tools for collaborative intelligent tutoring system environments. In S. Trausan-Matu, K. E. Boyer, M. Crosby, & K. Panourgia (Eds.), Proceedings of the 12th International Conference on Intelligent Tutoring Systems (pp. 523–528). Berlin, Heidelberg: Springer.CrossRefGoogle Scholar
  49. Olsen, J.K., Rummel, N., & Aleven, V. (2016). Investigating effects of embedding collaboration in an intelligent tutoring system for elementary school students. In the International Conference of the Learning Sciences.Google Scholar
  50. Olsen, J. K., Rummel, N., & Aleven, V. (2018a). Co-designing orchestration support for social plane transitions with teachers: Balancing automation and teacher autonomy. In J. Kay, R. Luckin, & R. (Eds.), Proceedings of the 13th International Conference of the Learning Sciences (pp. 1541–1542). London: International Society of the Learning Sciences.Google Scholar
  51. Olsen, J. K., Sharma, K., Aleven, V., & Rummel, N. (2018b). Combining gaze, dialogue, and action from a collaborative intelligent tutoring system to inform student learning processes. In J. Kay, R. Luckin, & R. (Eds.), Proceedings of the 13th International Conference of the Learning Sciences (pp. 689–696). London: International Society of the Learning Sciences.Google Scholar
  52. Renkl, A. (2005). The worked-out example principle in multimedia learning. In R. Mayer (Ed.), Cambridge Handbook of Multimedia Learning (pp. 229–246). Cambridge: Cambridge University Press.CrossRefGoogle Scholar
  53. Renkl, A., & Atkinson, R. K. (2003). Structuring the transition from example study to problem solving in cognitive skill acquisition: A cognitive load perspective. Educational Psychologist, 38(1), 15–22.CrossRefGoogle Scholar
  54. Rittle-Johnson, B., Siegler, R. S., & Alibali, M. W. (2001). Developing conceptual understanding and procedural skill in mathematics: An iterative process. Journal of Educational Psychology, 93(2), 346–362.CrossRefGoogle Scholar
  55. Rogat, T. K., Linnenbrink-Garcia, L., & DiDonato, N. (2013). Motivation in collaborative groups. In C. E. Hmelo-Silver, C. A. Chinn, C. K. K. Chan, & A. M. O’Donnell (Eds.), The international handbook of collaborative learning (pp. 250–267). New York: Routledge.Google Scholar
  56. Rogers, J. L., Howard, K. I., & Vessey, J. T. (1993). Using significance tests to evaluate equivalence between two experimental groups. Psychological Bulletin, 113(3), 553.CrossRefGoogle Scholar
  57. Rummel, N. (2018). One framework to rule them all? Carrying forward the conversation started by Wise and Schwarz. International Journal of Computer-Supported Collaborative Learning, 13(1), 123–129.CrossRefGoogle Scholar
  58. Rummel, N., Weinberger, A., Wecker, C., Fischer, F., Meier, A., Voyiatzaki, E., … & Koedinger, K. R. (2008). New challenges in CSCL: Towards adaptive script support. In Proceedings of the 8th International Conference for the Learning Sciences, 3, (pp. 338-345). International Society of the Learning Sciences.Google Scholar
  59. Schraw, G., & Lehman, S. (2001). Situational interest: A review of the literature and directions for future research. Educational Psychology Review, 13(1), 23–52.CrossRefGoogle Scholar
  60. Schwarz, B. B., Neuman, Y., & Biezuner, S. (2000). Two wrongs may make a right... If they argue together! Cognition and Instruction, 18(4), 461–494.CrossRefGoogle Scholar
  61. Siegler, R. S. (1995). How does change occur: A microgenetic study of number conservation. Cognitive Psychology, 28(3), 225–273.CrossRefGoogle Scholar
  62. Slavin, R. E. (1989). Cooperative learning and student achievement: Six theoretical perspectives. In Advances in Motivation and Achievement, 6, 161-177. Greenwich: JAI Press, Inc..Google Scholar
  63. Slavin, R. E. (1996). Research on cooperative learning and achievement: What we know, what we need to know. Contemporary Educational Psychology, 21(1), 43–69.CrossRefGoogle Scholar
  64. Springer, L., Stanne, M. E., & Donovan, S. S. (1999). Effects of small-group learning on undergraduates in science, mathematics, engineering, and technology: A meta-analysis. Review of Educational Research, 69(1), 21–51.CrossRefGoogle Scholar
  65. Teasley, S. D. (1995). The role of talk in children's peer collaborations. Developmental Psychology, 31(2), 207–220.CrossRefGoogle Scholar
  66. Tsovaltzi, D., Melis, E., Mclaren, B. M., Dietrich, M., Goguadze, G., & Meyer, A. K. (2009). Erroneous examples: A preliminary investigation into learning benefits. In Learning in the Synergy of Multiple Disciplines (pp. 688–693). Berlin Heidelberg: Springer.CrossRefGoogle Scholar
  67. Tsovaltzi, D., Melis, E., McLaren, B. M., Meyer, A.-K., Dietrich, M., & Goguadze, G. (2010). Learning from erroneous examples: When and how do students benefit from them? In M. Wolpers, P. A. Kirschner, M. Scheffel, S. Lindstaedt, & V. Dimitrova (Eds.), Proceedings of the 5th European Conference on Technology Enhanced Learning, Sustaining TEL: From Innovation to Learning and Practice, Barcelona, Spain (pp. 357–373). Berlin Heidelberg: Springer-Verlag.CrossRefGoogle Scholar
  68. Van Gog, T., Rummel, N., & Renkl, A. (2019). Learning how to solve problems by studying examples. In J. Dunlosky & K. Rawson (Eds.), The Cambridge Handbook of Cognition and Education. New York: Cambridge University Press.Google Scholar
  69. VanLehn, K. (2006). The behavior of tutoring systems. International Journal of Artificial Intelligence in Education, 16(3), 227–265.Google Scholar
  70. Vogel, F., Wecker, C., Kollar, I., & Fischer, F. (2017). Socio-cognitive scaffolding with computer-supported collaboration scripts: A meta-analysis. Educational Psychology Review, 29(3), 477–511.CrossRefGoogle Scholar
  71. Walker, E., Rummel, N., & Koedinger, K. R. (2011). Designing automated adaptive support to improve student helping behaviors in a peer tutoring activity. International Journal of Computer-Supported Collaborative Learning, 6(2), 279–306.CrossRefGoogle Scholar
  72. Wang, H. C., Rosé, C. P., & Chang, C. Y. (2011). Agent-based dynamic support for learning from collaborative brainstorming in scientific inquiry. International Journal of Computer-Supported Collaborative Learning, 6(3), 371.CrossRefGoogle Scholar
  73. Wise, A. F., & Schwarz, B. B. (2017). Visions of CSCL: Eight provocations for the future of the field. International Journal of Computer-Supported Collaborative Learning, 12(4), 423–467.CrossRefGoogle Scholar

Copyright information

© International Society of the Learning Sciences, Inc. 2019

Authors and Affiliations

  1. 1.École Polytechnique Fédérale de LausanneLausanneSwitzerland
  2. 2.Carnegie Mellon UniversityPittsburghUSA
  3. 3.Institute of Educational ResearchRuhr-Universität BochumBochumGermany

Personalised recommendations