Skip to main content
SpringerLink
Log in

Quantitative approach to collaborative learning: performance prediction, individual assessment, and group composition

  • Published:
International Journal of Computer-Supported Collaborative Learning Aims and scope Submit manuscript

Abstract

The benefits of collaborative learning, although widely reported, lack the quantitative rigor and detailed insight into the dynamics of interactions within the group, while individual contributions and their impacts on group members and their collaborative work remain hidden behind joint group assessment. To bridge this gap we intend to address three important aspects of collaborative learning focused on quantitative evaluation and prediction of group performance. First, we use machine learning techniques to predict group performance based on the data of member interactions and thereby identify whether, and to what extent, the group’s performance is driven by specific patterns of learning and interaction. Specifically, we explore the application of Extreme Learning Machine and Classification and Regression Trees to assess the predictability of group academic performance from live interaction data. Second, we propose a comparative model to unscramble individual student performances within the group. These performances are then used further in a generative mixture model of group grading as an explicit combination of isolated individual student grade expectations and compared against the actual group performances to define what we coined as collaboration synergy - directly measuring the improvements of collaborative learning. Finally the impact of group composition of gender and skills on learning performance and collaboration synergy is evaluated. The analysis indicates a high level of predictability of group performance based solely on the style and mechanics of collaboration and quantitatively supports the claim that heterogeneous groups with the diversity of skills and genders benefit more from collaborative learning than homogeneous groups.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  • Araque, F., Roldan, C., & Salguero, A. (2009). Factors influencing university drop out rates. Computer and Education, 53, 563–574.

    Article  Google Scholar 

  • Baker, R., & Yacef, K. (2009). The state of educational data mining in 2009: a review and future visions. Journal of Educational Data Mining, 1(1), 3–17.

    Google Scholar 

  • Barber, R., Sharkey, M. (2012). Course correction: using analytics to predict course success. Proceedings of the Second International Conference on Learning Analytics and Knowledge, ACM.

  • Barkley, E. F., Cross, K. P., Major, C. H. (2004). Collaborative learning techniques: A handbook for college faculty. San Francisco: Jossey-Bass.

  • Bhardwaj, B.K., & Pal, S. (2011). Data mining: a prediction for performance improvement using classification. International Journal of Computer Science and Information Security, 9(4), 136–140.

  • Bishop, C. (2006). Pattern recognition and machine learning. Berlin: Springer.

    Google Scholar 

  • Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (1984). Classification and regression trees. Monterey: Wadsworth, Inc.

    Google Scholar 

  • Bruckman, A., Jensen, C., Debonte, A. (2002). Gender and programming achievement in a CSCL environment. Proceedings of the Conference on Computer Support for Collaborative Learning: Foundations for a CSCL Community, 119–127.

  • Bruffee, K. (1993). Collaborative learning. Baltimore: The Johns Hopkins University Press.

  • Cen, H., Koedinger, K., Junker, B. (2006). Learning factors analysis - a general method for cognitive model evaluation and improvement. International Conference on Intelligent Tutoring Systems, 164–175.

  • Cen, L., Ruta, D., Powell, L., Ng, J. (2014a). Learning alone or in a group - an empirical case study of the collaborative learning patterns and their impact on student grades. International Conference on Interactive Collaborative Learning, 627–632.

  • Cen, L., Ruta, D., Powell, L., Ng, J. (2014b). Does gender matter for collaborative learning? International iCampus Forum 2014 on Smart Education for the 21st Century, IEEE International Conference on Teaching, Assessment, and Learning for Engineering, 433–440, New Zealand.

  • Chennabathni, R., Rgskind, G. (1998). Gender issues in collaborative learning. Canadian Women Studies, 17(4), 44–46.

  • Chiu, M. M. (2000). Group problem solving processes: social interactions and individual actions. Journal for the Theory of Social Behaviour, 30(1), 27–50.

    Article  Google Scholar 

  • Chiu, M. M. (2008). Flowing toward correct contributions during groups’ mathematics problem solving: a statistical discourse analysis. Journal of the Learning Sciences, 17(3), 415–463.

    Article  Google Scholar 

  • Coffrin, C., Corrin, L., Barba, P., Kennedy, G. (2014). Visualizing patterns of student engagement and performance in MOOCs. Proceedings of the fourth international conference on learning analytics and knowledge, ACM, 83–92.

  • Cohen, E. G., Lotan, R. A., Abram, P. L., Scarloss, B. A., & Schultz, S. E. (2002). Can groups learn? Teachers College Record, 104(6), 1045–1068.

    Article  Google Scholar 

  • Cress, U. (2008). The need for considering multilevel analysis in CSCL research: an appeal for the use of more advanced statistical methods. International Journal of Computer-Supported Collaborative Learning, 3(1), 69–84.

    Article  Google Scholar 

  • Crosswell, L., & Hunter, L. (2012). Navigating the muddy waters of the research into single sex class-rooms in co-educational middle years settings. Australian Journal of Middle Schooling, 12(2), 16–27.

    Google Scholar 

  • Davidson, J. E., Sternberg, R. J. (Eds.) (2003). The psychology of problem solving. Cambridge, United Kingdom: Cambridge University Press.

  • Dillenbourg, P. (1999). What do you mean by ‘collaborative learning’? In P. Dillenbourg (Ed.), Collaborative-learning: Cognitive and computational approaches ch.1, 1–15. Oxford: Elsevier.

    Google Scholar 

  • Dillenbourg, P., Baker, M., Blaye, A., OMalley, C. (1996). The evolution of research on collaborative learning. In E. Spada, P. Reiman (Eds.), Learning in humans and machine: Towards an interdisciplinary learning science, (189–211). Oxford: Elsevier. Proceedings of Computer Support for Collaborative Learning, 119–127.

  • Dirkx, J. M., Smith, R. O. (2004). Thinking out of a bowl of spaghetti: Learning to learn in online collaborative groups. In T. S. Roberts (Ed.), Online collaborative learning: Theory and practice (pp. 132–159). Hershey, PA: Information Science Publishing.

  • Fall, R., Webb, N., & Chudowsky, N. (2000). Group discussion and large-scale language arts assessment: effects on students comprehension. American educational research journal, 37 (4), 911–941.

  • Feichtner, S. B., & Davis, E. A. (1991). Why some groups fail: a survey of students experiences with learning groups. The Organizational Behavior Teaching Review, 9(4), 75–88.

    Google Scholar 

  • Feng, M., Heffernan, N., & Koedinger, K. (2009). Addressing the assessment challenge with an online system that tutors as it assesses. User Modeling and User-Adapted Interaction, 19(3), 243–266.

    Article  Google Scholar 

  • Ferguson, R., Shum, S.M. (2011). Learning analytics to identify exploratory dialogue within synchronous text chat. Proceedings of the 1st International Conference on Learning Analytics and Knowledge, ACM, 99–103.

  • Goggins, S., Xing, W., Chen, X., Chen, B., & Wadholm, B. (2015). Learning analytics at “small” scale: exploring a complexity-grounded model for assessment automation. Journal of Universal Computer Science, 21(1), 66–92.

    Google Scholar 

  • Goldstein, J., Puntambekar, S. (2004). The brink of change: gender in technology-rich collaborative learning environments. Journal of Science Education and Technology, 13(4), 505–522.

  • Golub, J. (Ed.). (1988). Focus on collaborative learning. Urbana: National Council of Teachers of English.

    Google Scholar 

  • Goode, W., & Caicedo, G. (2014). Online collaboration: individual involvement used to predict team performance. Learning and Collaboration Technologies, Technology-Rich Environments for Learning and Collaboration, Lecture Notes in Computer Science, 8524, 408–416.

    Google Scholar 

  • Gordon, A. (2000). In a class of their own: boys benefit even more than girls from single-sex schools, a-level grades study reveals. The Mail on Sunday (UK), June 11 (2000), p. 42.

  • Gress, C. L. Z., Fior, M., Hadwin, A. F., & Winne, P. H. (2010). Measurement and assessment in computer-supported collaborative learning. Computers in Human Behavior, 26(5), 806–814.

    Article  Google Scholar 

  • Gunnarsson, B. L., Alterman, R. (2012). Predicting failure: a case study in co-blogging. Proceedings of the 2nd International Conference on Learning Analytics and Knowledge, ACM, 263–266.

  • Hackman, J. R., & Morris, C. G. (1975). Group tasks, group interaction process, and group performance effectiveness: A review and proposed integration. In L. Berkowitz (Ed.), Advances in experimental social psychology (p. 8). New York: Academic.

    Google Scholar 

  • Hämäläinen, W., & Vinni, M. (2011). Classifiers for educational data mining. London: Chapman and Hall/CRC.

    Google Scholar 

  • Hirsch, B., Hitt, G. W., Powell, L., Khalaf, K., Balawi, S. (2013). Collaborative learning in action. Proceedings of the IEEE International Conference on Teaching, Assessment and Learning for Engineering, Bali, Indonesia.

  • Huang, G. B., Zhu, Q. Y., & Siew, C. K. (2006). Extreme learning machine: theory and applications. Neurocomputing, 70, 489–501.

    Article  Google Scholar 

  • Johnson, D.W., Johnson, R.T. (1988). Two heads learn better than one. Transforming Education: In Context, #18, Winter 1988, p. 34.

  • Johnson, D. W., Johnson, R. T. (1998). Cooperative learning and social interdependence theory. Retrieved from http://www.co-operation.org/pages/SIT.html.

  • Johnson, D. W., Johnson, R. T., & Smith, K. A. (1991). Cooperative learning: Increasing college faculty instructional productivity. Washington: The George Washington University, School of Education and Human Development.

    Google Scholar 

  • Johnson, D. W., Johnson, R. T., & Stanne, M. E. (2000). Cooperative learning methods: a meta analysis. Minnesota: University of Minneapolis.

    Google Scholar 

  • Kagan, S. (1994). Cooperative learning. San Clemente: Kagan Cooperative Publishing.

    Google Scholar 

  • Kapur, M., Kinzer, C.K. (2009). Productive failure in CSCL groups. International Journal of Computer-Supported Collaborative Learning, 4(1), 21–46.

  • Kapur, M., Voiklis, J., Kinzer, C.K. (2011). A complexity-grounded model for the emergence of convergence in CSCL groups. Analyzing interactions in CSCL , Volume 12 of the series Computer-Supported Collaborative Learning Series (pp. 3–23). New York, NY: Springer.

  • Koschmann, T. (2002). Dewey’s contribution to the foundations of CSCL research. Procedings of the Conference on Computer Support for Collaborative Learning: Foundation for a CSCL Community, 17–2.

  • Kotsiantis, S., Patriarcheas, K., & Xenos, M. (2010). A combinational incremental ensemble of classifiers as a technique for predicting students’ performance in distance education. Knowledge-Based Systems, 23(6), 529–535.

    Article  Google Scholar 

  • Lai, E. R. (2011). Collaboration: A literature review research report. Retrieved from: http://www.pearsonassessments.com/.

  • Mael, F., Alonso, A., Gibson, D., Rogers, K., & Smith, M. (2005). Single-sex versus coeducational schooling: A systematic review. Washington: American Institutes for Research.

    Google Scholar 

  • McNely, B.J., Gestwicki, P., Hill, J.H., Parli-Horne, P., Johnson, E. (2012). Learning analytics for collaborative writing: A prototype and case study. Proceedings of the 2nd International Conference on Learning Analytics and Knowledge, 222–225.

  • Mitnik, R., Recabarren, M., Nussbaum, M., & Soto, A. (2009). Collaborative robotic instruction: a graph teaching experience. Computers and Education, 53(2), 330–342.

    Article  Google Scholar 

  • Morse, S. (1998). Separated by sex: A critical look at single-sex education for girls. Washington: American Association of University Women Educational Foundation.

    Google Scholar 

  • Oakley, B., Felder, R. M., Brent, R., & Elhajj, I. (2004). Turning student groups into effective teams. Journal of Student Centred Learning, 2(1), 9–34.

    Google Scholar 

  • Race, P. (2001). Assessment: a briefing on self, peer, and group assessment. LTSN Generic Centre Assessment Series No. 9, York: LTSN Generic Centre.

  • Romero, C., & Ventura, S. (2010). Educational data mining: a review of the state of the art. IEEE Transactions on Systems, Man, and Cybernetics Part C: Applications and Reviews, 40(6), 601–618.

    Article  Google Scholar 

  • Romero, C., Ventura, S., Espejo, P.G., & Hervs, C. (2008). Data mining algorithms to classify students. Proceedings of the 1st International Conference on Educational Data Mining, 8–17.

  • Romero, C., López, M., Luna, J., & Ventura, S. (2013). Predicting students’ final performance from participation in on-line discussion forums. Computers and Education, 68, 458–472.

    Article  Google Scholar 

  • Roschelle, J., & Teasley, S. D. (1995). The construction of shared knowledge in collaborative problem-solving. In C. E. O’Malley (Ed.), Computer-supported collaborative learning (pp. 69–97). Berlin: Springer.

    Chapter  Google Scholar 

  • Ruta, D., Powell, L., Wang, D., Hirsch, B., Ng, J. (2013). Self-organising P2P learning for 21C education. International Symposium on Smart Learning for the Next Generation, 66–69, Abu Dhabi, UAE.

  • Safin, S., Verschuere, A., Burkhardt, J., Dtienne, F., & Hbert, A. M. (2010). Quality of collaboration in a distant collaborative architectural educational setting. International Reports on Socio-Informatics, 7(1), 40–48.

    Google Scholar 

  • Saner, H., McCaffrey, D., Stecher, B., Klein, S., & Bell, R. (1994). The effects of working in pairs in science performance assessments. Educational Assessment, 2(4), 325–338.

    Article  Google Scholar 

  • Savicki, V., Kelley, M., & Lingenfelter, D. (1996). Gender, group composition, and task type in small task groups using computer-mediated comunication. Computers in Human Behavior (Elsevier), 12(4), 549–565.

  • Sembiring, S., Zarlis, M., Hartama, D., Ramliana, S., Wani, E. (2011). Prediction of student academic performance by an application of data mining techniques. Proceedings of International Conference on Management and Artificial Intelligence, 6, 110–114, Bali, Indonesia.

  • Slavin, R. (1990). Cooperative learning: Theory, research and practice. New Jersey: Prentice Hall.

    Google Scholar 

  • Slavin, R., & Cooper, R. (1999). Improving intergroup relations: lessons learned from cooperative learning programs. Journal of Social Issues, 55(4), 647–663.

    Article  Google Scholar 

  • Smith, I. (1996). Good for boys and bad for girls? Empirical evidence on the coeducation/single-sex schooling debate. Forum of Education, 51(2), 44–51.

    Google Scholar 

  • Stahl, G., Koschmann, T., & Suthers, D. (2006). Computer-supported collaborative learning: A historical perspective. In R. K. Sawyer (Ed.), Cambridge handbook of the learning sciences (pp. 409–426). Cambridge: Cambridge University Press.

    Google Scholar 

  • Strijbos, J. W. (2011). Assessment of (computer-supported) collaborative learning. IEEE Transactions on Learning Technologies, 4(1), 59–73.

    Article  Google Scholar 

  • Stump, G. S., Hilpert, J. C., Husman, J., Chung, W. T., & Kim, W. (2011). Collaborative learning in engineering students: gender and achievement. Journal of Engineering Education, 100(3), 475–497.

    Article  Google Scholar 

  • Suthers, D. D. (2006). Technology affordances for intersubjective meaning-making: a research agenda for CSCL. International Journal of Computer Supported Collaborative Learning, 1(2), 315–337.

    Article  Google Scholar 

  • Thai-Nghe, N., Janecek, P., Haddawy, P. (2007). A comparative analysis of techniques for predicting academic performance. Proceedings of the 37th IEEE Frontiers in Education Conference, T2G7–T2G12.

  • Thai-Nghe, N., Busche, A., Schmidt-Thieme, L. (2009). Improving academic performance prediction by dealing with class imbalance. Proceedings of 9th IEEE International Conference on Intelligent Systems Design and Applications, 878–883.

  • Thai-Nghe, Drumond, N.L., Krohn-Grimberghe, A., Schmidt-Thieme, L. (2010). Recommender system for predicting student performance. Proceedings of the 1st workshop on Recommender Systems for Technology Enhanced Learning, 1, 2811–2819.

  • Thai-Nghe, N., et al. (2011a). Multi-relational factorization models for predicting student performance. KDD 2011 Workshop on Knowledge Discovery in Educational Data.

  • Thai-Nghe, N., Horvath, N., Schmidt-Thieme, L. (2011b). Factorization models for forecasting student performance. Proceedings of the 4th International Conference on Educational Data Mining.

  • Van Boxtel, C., Van der Linden, J., & Kanselaar, G. (2000). Collaborative learning tasks and the elaboration of conceptual knowledge. Learning and Instruction, 10(4), 311–330.

    Article  Google Scholar 

  • Vita, G. D. (2005). Fostering intercultural learning through multicultural group work. In J. Carroll & J. Ryan (Eds.), Teaching international students: Improving learning for all (pp. 75–83). Abingdon: Routledge.

    Google Scholar 

  • Ward, C. (2006). International students: Interpersonal, institutional and community impacts. Wellington: New Zealand Ministry of Education.

    Google Scholar 

  • Webb, N. M. (1991). Task-related verbal interaction and mathematical learning in small groups. Research in Mathematics Education, 22(5), 366–389.

    Google Scholar 

  • Webb, N. M. (1993). Collaborative group versus individual assessment in mathematics: processes and outcomes. Educational Assessment, 1(2), 131–152.

    Article  Google Scholar 

  • Webb, N. M. (1995). Group collaboration in assessment: multiple objectives, processes, and outcomes. Educational Evaluation and Policy Analysis, 17(2), 239–261.

    Article  Google Scholar 

  • Webb, N. M., Nemer, K. M., & Chizhik, A. W. (1998). Equity issues in collaborative group assessment: group composition and performance. American Educational Research Journal, 35(4), 607–651.

    Article  Google Scholar 

  • Wolff, A., et al. (2013). Improving retention: predicting at-risk students by analysing clicking behaviour in a virtual learning environment. Proceedings of the Third International Conference on Learning Analytics and Knowledge, ACM.

  • Xing, W., Wadholm, B., Goggins, S. (2014). Learning analytics in CSCL with a focus on assessment: an exploratory study of activity theory-informed cluster analysis. Proceedings of the Fourth International Conference on Learning Analytics and Knowledge, 59–67.

  • Xing, W., Guo, R., Petakovic, E., & Goggins, S. (2015). Participation-based student final performance prediction model through interpretable genetic programming: integrating learning analytics, educational data mining and theory. Computers in Human Behavior, Elsevier, 47, 168–181.

    Article  Google Scholar 

  • Yadav, S. K., & Pal, S. (2012). Data mining: a prediction for performance improvement of engineering students using classification. World of Computer Science and Information Technology Journal, 2(2), 51–56.

    Google Scholar 

  • Zeid, A., El-Bahey, R. (2011). Impact of introducing single-gender classrooms in higher education on student achievement levels- a case study in software engineering courses in the GCC region. Proceedings of the 41st ASEE/IEEE Frontiers in Education Conference.

  • Zheng, L., Huang, R. (2016). The effects of sentiments and co-regulation on group performance in computer supported collaborative learning. The internet and higher education, Elsevier, 28, 59–67.

  • Zhu, C. (2012). Student satisfaction, performance, and knowledge construction in online collaborative learning. Journal of Educational Technology and Society, 15(1), 127–136.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Emirates ICT Innovation Center, Khalifa University of Science, Technology and Research, Abu Dhabi, UAE

    Ling Cen, Dymitr Ruta, Leigh Powell, Benjamin Hirsch & Jason Ng

Authors
  1. Ling Cen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dymitr Ruta
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Leigh Powell
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Benjamin Hirsch
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jason Ng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ling Cen.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cen, L., Ruta, D., Powell, L. et al. Quantitative approach to collaborative learning: performance prediction, individual assessment, and group composition. Intern. J. Comput.-Support. Collab. Learn 11, 187–225 (2016). https://doi.org/10.1007/s11412-016-9234-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11412-016-9234-6

Keywords

Search

Navigation