Advertisement

Quantitative approach to collaborative learning: performance prediction, individual assessment, and group composition

  • Ling Cen
  • Dymitr Ruta
  • Leigh Powell
  • Benjamin Hirsch
  • Jason Ng
Article
First Online:

Abstract

The benefits of collaborative learning, although widely reported, lack the quantitative rigor and detailed insight into the dynamics of interactions within the group, while individual contributions and their impacts on group members and their collaborative work remain hidden behind joint group assessment. To bridge this gap we intend to address three important aspects of collaborative learning focused on quantitative evaluation and prediction of group performance. First, we use machine learning techniques to predict group performance based on the data of member interactions and thereby identify whether, and to what extent, the group’s performance is driven by specific patterns of learning and interaction. Specifically, we explore the application of Extreme Learning Machine and Classification and Regression Trees to assess the predictability of group academic performance from live interaction data. Second, we propose a comparative model to unscramble individual student performances within the group. These performances are then used further in a generative mixture model of group grading as an explicit combination of isolated individual student grade expectations and compared against the actual group performances to define what we coined as collaboration synergy - directly measuring the improvements of collaborative learning. Finally the impact of group composition of gender and skills on learning performance and collaboration synergy is evaluated. The analysis indicates a high level of predictability of group performance based solely on the style and mechanics of collaboration and quantitatively supports the claim that heterogeneous groups with the diversity of skills and genders benefit more from collaborative learning than homogeneous groups.

Keywords

Collaborative learning Performance prediction Machine learning Performance modeling Group composition 
This is a preview of subscription content, log in to check access.

References

  1. Araque, F., Roldan, C., & Salguero, A. (2009). Factors influencing university drop out rates. Computer and Education, 53, 563–574.CrossRefGoogle Scholar
  2. Baker, R., & Yacef, K. (2009). The state of educational data mining in 2009: a review and future visions. Journal of Educational Data Mining, 1(1), 3–17.Google Scholar
  3. Barber, R., Sharkey, M. (2012). Course correction: using analytics to predict course success. Proceedings of the Second International Conference on Learning Analytics and Knowledge, ACM. Google Scholar
  4. Barkley, E. F., Cross, K. P., Major, C. H. (2004). Collaborative learning techniques: A handbook for college faculty. San Francisco: Jossey-Bass.Google Scholar
  5. Bhardwaj, B.K., & Pal, S. (2011). Data mining: a prediction for performance improvement using classification. International Journal of Computer Science and Information Security, 9(4), 136–140.Google Scholar
  6. Bishop, C. (2006). Pattern recognition and machine learning. Berlin: Springer.Google Scholar
  7. Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (1984). Classification and regression trees. Monterey: Wadsworth, Inc.Google Scholar
  8. Bruckman, A., Jensen, C., Debonte, A. (2002). Gender and programming achievement in a CSCL environment. Proceedings of the Conference on Computer Support for Collaborative Learning: Foundations for a CSCL Community, 119–127.Google Scholar
  9. Bruffee, K. (1993). Collaborative learning. Baltimore: The Johns Hopkins University Press.Google Scholar
  10. Cen, H., Koedinger, K., Junker, B. (2006). Learning factors analysis - a general method for cognitive model evaluation and improvement. International Conference on Intelligent Tutoring Systems, 164–175.Google Scholar
  11. Cen, L., Ruta, D., Powell, L., Ng, J. (2014a). Learning alone or in a group - an empirical case study of the collaborative learning patterns and their impact on student grades. International Conference on Interactive Collaborative Learning, 627–632.Google Scholar
  12. Cen, L., Ruta, D., Powell, L., Ng, J. (2014b). Does gender matter for collaborative learning? International iCampus Forum 2014 on Smart Education for the 21st Century, IEEE International Conference on Teaching, Assessment, and Learning for Engineering, 433–440, New Zealand.Google Scholar
  13. Chennabathni, R., Rgskind, G. (1998). Gender issues in collaborative learning. Canadian Women Studies, 17(4), 44–46.Google Scholar
  14. Chiu, M. M. (2000). Group problem solving processes: social interactions and individual actions. Journal for the Theory of Social Behaviour, 30(1), 27–50.CrossRefGoogle Scholar
  15. Chiu, M. M. (2008). Flowing toward correct contributions during groups’ mathematics problem solving: a statistical discourse analysis. Journal of the Learning Sciences, 17(3), 415–463.CrossRefGoogle Scholar
  16. Coffrin, C., Corrin, L., Barba, P., Kennedy, G. (2014). Visualizing patterns of student engagement and performance in MOOCs. Proceedings of the fourth international conference on learning analytics and knowledge, ACM, 83–92.Google Scholar
  17. Cohen, E. G., Lotan, R. A., Abram, P. L., Scarloss, B. A., & Schultz, S. E. (2002). Can groups learn? Teachers College Record, 104(6), 1045–1068.CrossRefGoogle Scholar
  18. Cress, U. (2008). The need for considering multilevel analysis in CSCL research: an appeal for the use of more advanced statistical methods. International Journal of Computer-Supported Collaborative Learning, 3(1), 69–84.CrossRefGoogle Scholar
  19. Crosswell, L., & Hunter, L. (2012). Navigating the muddy waters of the research into single sex class-rooms in co-educational middle years settings. Australian Journal of Middle Schooling, 12(2), 16–27.Google Scholar
  20. Davidson, J. E., Sternberg, R. J. (Eds.) (2003). The psychology of problem solving. Cambridge, United Kingdom: Cambridge University Press.Google Scholar
  21. Dillenbourg, P. (1999). What do you mean by ‘collaborative learning’? In P. Dillenbourg (Ed.), Collaborative-learning: Cognitive and computational approaches ch.1, 1–15. Oxford: Elsevier.Google Scholar
  22. Dillenbourg, P., Baker, M., Blaye, A., OMalley, C. (1996). The evolution of research on collaborative learning. In E. Spada, P. Reiman (Eds.), Learning in humans and machine: Towards an interdisciplinary learning science, (189–211). Oxford: Elsevier. Proceedings of Computer Support for Collaborative Learning, 119–127.Google Scholar
  23. Dirkx, J. M., Smith, R. O. (2004). Thinking out of a bowl of spaghetti: Learning to learn in online collaborative groups. In T. S. Roberts (Ed.), Online collaborative learning: Theory and practice (pp. 132–159). Hershey, PA: Information Science Publishing.Google Scholar
  24. Fall, R., Webb, N., & Chudowsky, N. (2000). Group discussion and large-scale language arts assessment: effects on students comprehension. American educational research journal, 37 (4), 911–941.Google Scholar
  25. Feichtner, S. B., & Davis, E. A. (1991). Why some groups fail: a survey of students experiences with learning groups. The Organizational Behavior Teaching Review, 9(4), 75–88.Google Scholar
  26. Feng, M., Heffernan, N., & Koedinger, K. (2009). Addressing the assessment challenge with an online system that tutors as it assesses. User Modeling and User-Adapted Interaction, 19(3), 243–266.CrossRefGoogle Scholar
  27. Ferguson, R., Shum, S.M. (2011). Learning analytics to identify exploratory dialogue within synchronous text chat. Proceedings of the 1st International Conference on Learning Analytics and Knowledge, ACM, 99–103.Google Scholar
  28. Goggins, S., Xing, W., Chen, X., Chen, B., & Wadholm, B. (2015). Learning analytics at “small” scale: exploring a complexity-grounded model for assessment automation. Journal of Universal Computer Science, 21(1), 66–92.Google Scholar
  29. Goldstein, J., Puntambekar, S. (2004). The brink of change: gender in technology-rich collaborative learning environments. Journal of Science Education and Technology, 13(4), 505–522.Google Scholar
  30. Golub, J. (Ed.). (1988). Focus on collaborative learning. Urbana: National Council of Teachers of English.Google Scholar
  31. Goode, W., & Caicedo, G. (2014). Online collaboration: individual involvement used to predict team performance. Learning and Collaboration Technologies, Technology-Rich Environments for Learning and Collaboration, Lecture Notes in Computer Science, 8524, 408–416.Google Scholar
  32. Gordon, A. (2000). In a class of their own: boys benefit even more than girls from single-sex schools, a-level grades study reveals. The Mail on Sunday (UK), June 11 (2000), p. 42.Google Scholar
  33. Gress, C. L. Z., Fior, M., Hadwin, A. F., & Winne, P. H. (2010). Measurement and assessment in computer-supported collaborative learning. Computers in Human Behavior, 26(5), 806–814.CrossRefGoogle Scholar
  34. Gunnarsson, B. L., Alterman, R. (2012). Predicting failure: a case study in co-blogging. Proceedings of the 2nd International Conference on Learning Analytics and Knowledge, ACM, 263–266.Google Scholar
  35. Hackman, J. R., & Morris, C. G. (1975). Group tasks, group interaction process, and group performance effectiveness: A review and proposed integration. In L. Berkowitz (Ed.), Advances in experimental social psychology (p. 8). New York: Academic.Google Scholar
  36. Hämäläinen, W., & Vinni, M. (2011). Classifiers for educational data mining. London: Chapman and Hall/CRC.Google Scholar
  37. Hirsch, B., Hitt, G. W., Powell, L., Khalaf, K., Balawi, S. (2013). Collaborative learning in action. Proceedings of the IEEE International Conference on Teaching, Assessment and Learning for Engineering, Bali, Indonesia.Google Scholar
  38. Huang, G. B., Zhu, Q. Y., & Siew, C. K. (2006). Extreme learning machine: theory and applications. Neurocomputing, 70, 489–501.CrossRefGoogle Scholar
  39. Johnson, D.W., Johnson, R.T. (1988). Two heads learn better than one. Transforming Education: In Context, #18, Winter 1988, p. 34. Google Scholar
  40. Johnson, D. W., Johnson, R. T. (1998). Cooperative learning and social interdependence theory. Retrieved from http://www.co-operation.org/pages/SIT.html.
  41. Johnson, D. W., Johnson, R. T., & Smith, K. A. (1991). Cooperative learning: Increasing college faculty instructional productivity. Washington: The George Washington University, School of Education and Human Development.Google Scholar
  42. Johnson, D. W., Johnson, R. T., & Stanne, M. E. (2000). Cooperative learning methods: a meta analysis. Minnesota: University of Minneapolis.Google Scholar
  43. Kagan, S. (1994). Cooperative learning. San Clemente: Kagan Cooperative Publishing.Google Scholar
  44. Kapur, M., Kinzer, C.K. (2009). Productive failure in CSCL groups. International Journal of Computer-Supported Collaborative Learning, 4(1), 21–46.Google Scholar
  45. Kapur, M., Voiklis, J., Kinzer, C.K. (2011). A complexity-grounded model for the emergence of convergence in CSCL groups. Analyzing interactions in CSCL , Volume 12 of the series Computer-Supported Collaborative Learning Series (pp. 3–23). New York, NY: Springer.Google Scholar
  46. Koschmann, T. (2002). Dewey’s contribution to the foundations of CSCL research. Procedings of the Conference on Computer Support for Collaborative Learning: Foundation for a CSCL Community, 17–2.Google Scholar
  47. Kotsiantis, S., Patriarcheas, K., & Xenos, M. (2010). A combinational incremental ensemble of classifiers as a technique for predicting students’ performance in distance education. Knowledge-Based Systems, 23(6), 529–535.CrossRefGoogle Scholar
  48. Lai, E. R. (2011). Collaboration: A literature review research report. Retrieved from: http://www.pearsonassessments.com/.
  49. Mael, F., Alonso, A., Gibson, D., Rogers, K., & Smith, M. (2005). Single-sex versus coeducational schooling: A systematic review. Washington: American Institutes for Research.Google Scholar
  50. McNely, B.J., Gestwicki, P., Hill, J.H., Parli-Horne, P., Johnson, E. (2012). Learning analytics for collaborative writing: A prototype and case study. Proceedings of the 2nd International Conference on Learning Analytics and Knowledge, 222–225.Google Scholar
  51. Mitnik, R., Recabarren, M., Nussbaum, M., & Soto, A. (2009). Collaborative robotic instruction: a graph teaching experience. Computers and Education, 53(2), 330–342.CrossRefGoogle Scholar
  52. Morse, S. (1998). Separated by sex: A critical look at single-sex education for girls. Washington: American Association of University Women Educational Foundation.Google Scholar
  53. Oakley, B., Felder, R. M., Brent, R., & Elhajj, I. (2004). Turning student groups into effective teams. Journal of Student Centred Learning, 2(1), 9–34.Google Scholar
  54. Race, P. (2001). Assessment: a briefing on self, peer, and group assessment. LTSN Generic Centre Assessment Series No. 9, York: LTSN Generic Centre.Google Scholar
  55. Romero, C., & Ventura, S. (2010). Educational data mining: a review of the state of the art. IEEE Transactions on Systems, Man, and Cybernetics Part C: Applications and Reviews, 40(6), 601–618.CrossRefGoogle Scholar
  56. Romero, C., Ventura, S., Espejo, P.G., & Hervs, C. (2008). Data mining algorithms to classify students. Proceedings of the 1st International Conference on Educational Data Mining, 8–17.Google Scholar
  57. Romero, C., López, M., Luna, J., & Ventura, S. (2013). Predicting students’ final performance from participation in on-line discussion forums. Computers and Education, 68, 458–472.CrossRefGoogle Scholar
  58. Roschelle, J., & Teasley, S. D. (1995). The construction of shared knowledge in collaborative problem-solving. In C. E. O’Malley (Ed.), Computer-supported collaborative learning (pp. 69–97). Berlin: Springer.CrossRefGoogle Scholar
  59. Ruta, D., Powell, L., Wang, D., Hirsch, B., Ng, J. (2013). Self-organising P2P learning for 21C education. International Symposium on Smart Learning for the Next Generation, 66–69, Abu Dhabi, UAE.Google Scholar
  60. Safin, S., Verschuere, A., Burkhardt, J., Dtienne, F., & Hbert, A. M. (2010). Quality of collaboration in a distant collaborative architectural educational setting. International Reports on Socio-Informatics, 7(1), 40–48.Google Scholar
  61. Saner, H., McCaffrey, D., Stecher, B., Klein, S., & Bell, R. (1994). The effects of working in pairs in science performance assessments. Educational Assessment, 2(4), 325–338.CrossRefGoogle Scholar
  62. Savicki, V., Kelley, M., & Lingenfelter, D. (1996). Gender, group composition, and task type in small task groups using computer-mediated comunication. Computers in Human Behavior (Elsevier), 12(4), 549–565.Google Scholar
  63. Sembiring, S., Zarlis, M., Hartama, D., Ramliana, S., Wani, E. (2011). Prediction of student academic performance by an application of data mining techniques. Proceedings of International Conference on Management and Artificial Intelligence, 6, 110–114, Bali, Indonesia.Google Scholar
  64. Slavin, R. (1990). Cooperative learning: Theory, research and practice. New Jersey: Prentice Hall.Google Scholar
  65. Slavin, R., & Cooper, R. (1999). Improving intergroup relations: lessons learned from cooperative learning programs. Journal of Social Issues, 55(4), 647–663.CrossRefGoogle Scholar
  66. Smith, I. (1996). Good for boys and bad for girls? Empirical evidence on the coeducation/single-sex schooling debate. Forum of Education, 51(2), 44–51.Google Scholar
  67. Stahl, G., Koschmann, T., & Suthers, D. (2006). Computer-supported collaborative learning: A historical perspective. In R. K. Sawyer (Ed.), Cambridge handbook of the learning sciences (pp. 409–426). Cambridge: Cambridge University Press.Google Scholar
  68. Strijbos, J. W. (2011). Assessment of (computer-supported) collaborative learning. IEEE Transactions on Learning Technologies, 4(1), 59–73.CrossRefGoogle Scholar
  69. Stump, G. S., Hilpert, J. C., Husman, J., Chung, W. T., & Kim, W. (2011). Collaborative learning in engineering students: gender and achievement. Journal of Engineering Education, 100(3), 475–497.CrossRefGoogle Scholar
  70. Suthers, D. D. (2006). Technology affordances for intersubjective meaning-making: a research agenda for CSCL. International Journal of Computer Supported Collaborative Learning, 1(2), 315–337.CrossRefGoogle Scholar
  71. Thai-Nghe, N., Janecek, P., Haddawy, P. (2007). A comparative analysis of techniques for predicting academic performance. Proceedings of the 37th IEEE Frontiers in Education Conference, T2G7–T2G12.Google Scholar
  72. Thai-Nghe, N., Busche, A., Schmidt-Thieme, L. (2009). Improving academic performance prediction by dealing with class imbalance. Proceedings of 9th IEEE International Conference on Intelligent Systems Design and Applications, 878–883.Google Scholar
  73. Thai-Nghe, Drumond, N.L., Krohn-Grimberghe, A., Schmidt-Thieme, L. (2010). Recommender system for predicting student performance. Proceedings of the 1st workshop on Recommender Systems for Technology Enhanced Learning, 1, 2811–2819.Google Scholar
  74. Thai-Nghe, N., et al. (2011a). Multi-relational factorization models for predicting student performance. KDD 2011 Workshop on Knowledge Discovery in Educational Data.Google Scholar
  75. Thai-Nghe, N., Horvath, N., Schmidt-Thieme, L. (2011b). Factorization models for forecasting student performance. Proceedings of the 4th International Conference on Educational Data Mining.Google Scholar
  76. Van Boxtel, C., Van der Linden, J., & Kanselaar, G. (2000). Collaborative learning tasks and the elaboration of conceptual knowledge. Learning and Instruction, 10(4), 311–330.CrossRefGoogle Scholar
  77. Vita, G. D. (2005). Fostering intercultural learning through multicultural group work. In J. Carroll & J. Ryan (Eds.), Teaching international students: Improving learning for all (pp. 75–83). Abingdon: Routledge.Google Scholar
  78. Ward, C. (2006). International students: Interpersonal, institutional and community impacts. Wellington: New Zealand Ministry of Education.Google Scholar
  79. Webb, N. M. (1991). Task-related verbal interaction and mathematical learning in small groups. Research in Mathematics Education, 22(5), 366–389.Google Scholar
  80. Webb, N. M. (1993). Collaborative group versus individual assessment in mathematics: processes and outcomes. Educational Assessment, 1(2), 131–152.CrossRefGoogle Scholar
  81. Webb, N. M. (1995). Group collaboration in assessment: multiple objectives, processes, and outcomes. Educational Evaluation and Policy Analysis, 17(2), 239–261.CrossRefGoogle Scholar
  82. Webb, N. M., Nemer, K. M., & Chizhik, A. W. (1998). Equity issues in collaborative group assessment: group composition and performance. American Educational Research Journal, 35(4), 607–651.CrossRefGoogle Scholar
  83. Wolff, A., et al. (2013). Improving retention: predicting at-risk students by analysing clicking behaviour in a virtual learning environment. Proceedings of the Third International Conference on Learning Analytics and Knowledge, ACM.Google Scholar
  84. Xing, W., Wadholm, B., Goggins, S. (2014). Learning analytics in CSCL with a focus on assessment: an exploratory study of activity theory-informed cluster analysis. Proceedings of the Fourth International Conference on Learning Analytics and Knowledge, 59–67.Google Scholar
  85. Xing, W., Guo, R., Petakovic, E., & Goggins, S. (2015). Participation-based student final performance prediction model through interpretable genetic programming: integrating learning analytics, educational data mining and theory. Computers in Human Behavior, Elsevier, 47, 168–181.CrossRefGoogle Scholar
  86. Yadav, S. K., & Pal, S. (2012). Data mining: a prediction for performance improvement of engineering students using classification. World of Computer Science and Information Technology Journal, 2(2), 51–56.Google Scholar
  87. Zeid, A., El-Bahey, R. (2011). Impact of introducing single-gender classrooms in higher education on student achievement levels- a case study in software engineering courses in the GCC region. Proceedings of the 41st ASEE/IEEE Frontiers in Education Conference.Google Scholar
  88. Zheng, L., Huang, R. (2016). The effects of sentiments and co-regulation on group performance in computer supported collaborative learning. The internet and higher education, Elsevier, 28, 59–67.Google Scholar
  89. Zhu, C. (2012). Student satisfaction, performance, and knowledge construction in online collaborative learning. Journal of Educational Technology and Society, 15(1), 127–136.Google Scholar

Copyright information

© International Society of the Learning Sciences, Inc. 2016

Authors and Affiliations

  • Ling Cen
    • 1
  • Dymitr Ruta
    • 1
  • Leigh Powell
    • 1
  • Benjamin Hirsch
    • 1
  • Jason Ng
    • 1
  1. 1.Emirates ICT Innovation CenterKhalifa University of Science, Technology and ResearchAbu DhabiUAE

Personalised recommendations

Cite article

Buy options