Abstract
(Quasi-)experimental designs play an important role in CSCL research. By actively manipulating one or several independent variables while keeping other influencing factors constant and through the use of randomization, they allow to determine the causal effects of such independent variables on one or more dependent variable(s) that may be of interest to CSCL researchers. So far, (quasi-)experimental CSCL studies have mainly looked at the effects of certain tools and scaffolds on the occurrence of hoped-for learning process and outcomes variables. While earlier CSCL research mainly ignored the interdependence of data from learners who learned in the same group, more recent research uses more advanced statistical methods to analyze the effects of different CSCL settings on learning processes and outcomes (such as multilevel modeling). Because of the replication crisis in psychology, preregistration and the open science movement are becoming increasingly important also for CSCL research that uses (quasi-)experimental designs.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Ackerman, R. A., & Kenny, D. A. (2016). APIMPower: An interactive tool for actor-partner interdependence model power analysis [Computer software]. Retrieved from https://robert-a-ackerman.shinyapps.io/apimpower/
Bause, I. M., Brich, I. R., Wesslein, A.-K., & Hesse, F. W. (2018). Using technological functions on a multi-touch table and their affordances to counteract biases and foster collaborative problem solving. International of Computer-Supported Collaborative Learning, 13, 7–33. https://doi.org/10.1007/s11412-018-9271-4.
Brom, C., Sisler, V., Slussareff, M., Selmbecherova, T., & Hlavka, Z. (2016). You like it, you learn it: Affectivity and learning in competitive social role play gaming. International Journal of Computer-Supported Collaborative Learning, 11, 313–348. https://doi.org/10.1007/s11412-016-9237-3.
Cesareni, D., Cacciamani, S., & Fujita, N. (2016). Role taking and knowledge building in a blended university course. International Journal of Computer-Supported Collaborative Learning, 11, 9–39. https://doi.org/10.1007/s11412-015-9224-0.
Chen, J., Wang, M., Kirschner, P., & Tsai, C.-C. (2018). The role of collaboration, computer use, learning environments, and supporting strategies in CSCL: A meta-analysis. Review of Educational Research, 88(6), 799–843. https://doi.org/10.3102/0034654318791584.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences. Routledge.
Cook, T. D., & Campbell, D. T. (1979). Quasi-experimentation: Design and analysis issues for field settings. Rand McNally.
Cress, U. (2008). The need for considering multilevel analysis in CSCL research: An appeal for the use of more advanced statistical methods. International Journal of Computer-Supported Collaborative Learning, 3, 69–84. https://doi.org/10.1007/s11412-007-9032-2.
Cresswell, J. W. (2008). Research designs: Quantitative, qualitative, and mixed methods approaches. Thousand Oaks: SAGE.
Cuendet, S., Dehler-Zufferey, J., Ortoleva, G., & Dillenbourg, P. (2015). An integrated way of using a tangible user interface in a classroom. International Journal of Computer Supported Collaborative Learning, 10, 183–208. https://doi.org/10.1007/s11412-015-9213-3.
De Wever, B., Van Keer, H., Schellens, T., & Valcke, M. (2007). Applying multilevel modelling to content analysis data: Methodological issues in the study of role assignment in asynchronous discussion groups. Learning and Instruction, 17, 436–447. https://doi.org/10.1016/j.learninstruc.2007.04.001.
Du, H., & Wang, L. (2016). The impact of the number of dyads on estimation of dyadic data analysis using multilevel modeling. Methodology, 12, 21–31. https://doi.org/10.1027/1614-2241/a000105.
Erkens, M., Bodemer, D., & Hoppe, H. U. (2016). Improving collaborative learning in the classroom: Text mining based grouping and representing. International Journal of Computer-Supported Collaborative Learning, 11, 387–415. https://doi.org/10.1007/s11412-016-9243-5.
Field, A. (2018). Discovering statistics using IBM SPSS statistics (5th ed.). Thousand Oaks: SAGE.
Field, A., Miles, J., & Field, Z. (2012). Discovering statistics using R. Thousand Oaks SAGE.
Fischer, F., Kollar, I., Stegmann, K., & Wecker, C. (2013). Toward a script theory of guidance in computer-supported collaborative learning. Educational Psychologist, 48(1), 56–66. https://doi.org/10.1080/00461520.2012.748005.
Fischer, F., & Mandl, H. (2005). Knowledge convergence in computer-supported collaborative learning: The role of external representation tools. Journal of the Learning Sciences, 14, 405–441. https://doi.org/10.1207/s15327809jls1403_3.
Gress, C. L. Z., Fior, M., Hadwin, A. F., & Winne, P. H. (2010). Measurement and assessment in computer-supported collaborative learning. Computers in Human Behavior, 26, 806–814. https://doi.org/10.1016/j.chb.2007.05.012.
Harney, O. M., Hogan, M. J., Broome, B., Hall, T., & Ryan, C. (2015). Investigating the effects of prompts on argumentation style, consensus and perceived efficacy in collaborative learning. International Journal of Computer-Supported Collaborative Learning, 10, 367–394. https://doi.org/10.1007/s11r412-015-9223-1.
Harney, O. M., Hogan, M. J., & Quinn, S. (2017). Investigating the effects of peer to peer prompts on collaborative argumentation, consensus and perceived efficacy in collaborative learning. International Journal of Computer-Supported Collaborative Learning, 12, 307–336. https://doi.org/10.1007/s11412-017-9263-9.
Henrich, J., Heine, S. J., & Norenzayan, A. (2010). The weirdest people in the world? Behavioral and Brain Sciences, 33(2–3), 61–83. https://doi.org/10.1017/S0140525X0999152X.
Hox, J. (2003). Multilevel analysis: Techniques and applications. Mahwah: Erlbaum.
Inkpen, K., Booth, K., Klawe, M., & Upitis, R. (1995). Playing together beats playing apart, especially for girls. In J. L. Schnase & E. L. Cunnius (Eds.), Proceedings of CSCL ‘95: The first international conference on computer support for collaborative learning (pp. 177–181). Erlbaum. https://doi.org/10.3115/222020.222164.
Ioannidis, J. P. A. (2005). Why most published research findings are false. PLoS Medicine, 2, e124. https://doi.org/10.1371/journal.pmed.0020124.
Janssen, J., & Bodemer, D. (2013). Coordinated computer-supported collaborative learning: Awareness and awareness tools. Educational Psychologist, 48(1), 40–55. https://doi.org/10.1080/00461520.2012.749153.
Janssen, J., Cress, U., Erkens, G., & Kirschner, P. A. (2013). Multilevel analysis for the analysis of collaborative learning. In C. E. Hmelo-Silver, C. A. Chinn, C. Chan, & A. M. O’Donnell (Eds.), The international handbook of collaborative learning (pp. 112–115). Routledge.
Jeong, H., & Chi, M. T. H. (2007). Knowledge convergence and collaborative learning. Instructional Science, 35, 287–315. https://doi.org/10.1007/s11251-006-9008-z.
Jeong, H., Hmelo-Silver, C. E., & Yu, Y. (2014). An examination of CSCL methodological practices and the influence of theoretical frameworks 2005–2009. International Journal of Computer-Supported Collaborative Learning, 9, 305–334. https://doi.org/10.1007/s11412-014-9198-3.
Johnson, R. B., & Onwuegbuzie, A. J. (2004). Mixed methods research: A paradigm whose time has come. Educational Researcher, 33(7), 14–26. https://doi.org/10.3102/0013189X033007014.
Kenny, D. A., Kashy, D. A., & Cook, W. L. (2006). Dyadic data analysis. New York: Guilford Press.
Kollar, I., Ufer, S., Reichersdorfer, E., Vogel, F., Fischer, F., & Reiss, K. (2014). Effects of collaboration scripts and heuristic worked examples on the acquisition of mathematical argumentation skills of teacher students with different levels of priorachievement. Learning and Instruction, 32, 22–36. https://doi.org/10.1016/j.learninstruc.2014.01.003.
Kolloffel, B., Eysink, T. H. S., & de Jong, T. (2011). Comparing the effects of representational tools in collaborative and individual inquiry learning. International Journal for Computer-Supported Collaborative Learning, 6(2), 223–251. https://doi.org/10.1007/s11412-011-9110-3.
Koole, S. L., & Lakens, D. (2012). Rewarding replications: A sure and simple way to improve psychological science. Perspectives on Psychological Science, 7, 608–614. https://doi.org/10.1177/1745691612462586.
Lakens, D. (2014). Performing high-powered studies efficiently with sequential analyses. European Journal of Social Psychology, 44, 701–710. https://doi.org/10.1002/ejsp.2023.
Levelt Committee (2012). Flawed science: The fraudulent research practices of social psychologist Diederik Stapel. Retrieved from https://poolux.psychopool.tu-dresden.de/mdcfiles/gwp/Reale%20F%C3%A4lle/Stapel%20-%20Final%20Report.pdf
Lin, J.-W., Mai, L.-J., & Lai, Y.-C. (2015). Peer interaction and social network analysis of online communities with the support of awareness of different contexts. International Journal of Computer-Supported Collaborative Learning, 10, 139–159. https://doi.org/10.1007/s11412-015-9212-4.
Lincoln, Y. S., & Guba, E. G. (2000). Paradigmatic controversies, contradictions, and emerging confluences. In N. K. Denzin & Y. S. Lincoln (Eds.), Handbook of qualitative research (pp. 163–188). SAGE.
Maas, C. J. M., & Hox, J. J. (2005). Sufficient sample sizes for multilevel modeling. Methodology, 1, 86–92. https://doi.org/10.1027/1614-2241.1.3.86.
Makel, M. C., & Plucker, J. A. (2014). Facts are more important than novelty: Replication in the education sciences. Educational Researcher, 43(6), 304–316. https://doi.org/10.3102/0013189X14545513.
Mende, S., Proske, A., Körndle, H., & Narciss, S. (2017). Who benefits from a low versus high guidance CSCL script and why? Instructional Science, 45, 439–468. https://doi.org/10.1007/s11251-017-9411-7.
Resendes, M., Scardamalia, M., Bereiter, C., Chen, B., & Halewood, C. (2015). Group-level formative feedback and metadiscourse. International Journal of Computer-Supported Collaborative Learning, 10, 309–336. https://doi.org/10.1007/s11412-015-9219-x.
Scardamalia, M., & Bereiter, C. (2006). Knowledge building: Theory, pedagogy, and technology. In K. Sawyer (Ed.), The Cambridge handbook of the learning sciences (pp. 97–115). Cambridge: Cambridge University Press.
Schellens, T., Van Keer, H., & Valcke, M. (2005). The impact of role assignment on knowledge construction in asynchronous discussion groups: A multilevel analysis. Small Group Research, 36, 704–745. https://doi.org/10.1177/1046496405281771.
Schwaighofer, M., Vogel, F., Kollar, I., Ufer, S., Strohmaier, A., Terwedow, I., Ottinger, S., Reiss, K., & Fischer, F. (2017). How to combine collaboration scripts and heuristic worked examples to foster mathematical argumentation—When working memory matters. International Journal of Computer-Supported Collaborative Learning, 12, 281–305. https://doi.org/10.1007/s11412-017-9260-z.
Serrano-Cámara, L. M., Paredes-Velasco, M., Alcover, C.-M., & Velazquez-Iturbide, J. Á. (2014). An evaluation of students’ motivation in computer-supported collaborative learning of programming concepts. Computers in Human Behavior, 31, 499–508. https://doi.org/10.1016/j.chb.2013.04.030.
Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Boston: Houghton Mifflin Company.
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22, 1359–1366. https://doi.org/10.1177/2F0956797611417632.
Slavin, R. E., & Smith, D. (2009). The relationship between sample sizes and effect sizes in systematic reviews in education. Educational Evaluation and Policy Analysis, 31, 500–506. https://doi.org/10.3102/0162373709352369.
Snijders, T. A. B. (2005). Power and sample size in multilevel linear models. In B. S. Everitt & D. C. Howell (Eds.), Encyclopedia of statistics in behavioral science (Vol. 3, pp. 1570–1573). Chicester: Wiley.
Snijders, T. A. B., & Bosker, R. J. (1999). Multilevel analysis: An introduction to basic and advanced multilevel modeling. Thousand Oaks: SAGE.
Stahl, G., Koschmann, T., & Suthers, D. (2006). Computer-supported collaborative learning: An historical perspective. In R. K. Sawyer (Ed.), Cambridge handbook of the learning sciences (pp. 409–426). Cambridge: Cambridge University Press.
Stasser, G., & Titus, W. (1985). Pooling of unshared information in group decision making: Biased information sampling during discussion. Journal of Personality and Social Psychology, 48, 1467–1478. https://doi.org/10.1037/0022-3514.48.6.1467.
Strijbos, J.-W., Martens, R. L., Jochems, W. M. G., & Broers, N. J. (2007). The effect of functional roles on perceived group efficiency during computer-supported collaborative learning: A matter of triangulation. Computers in Human Behavior, 23, 353–380. https://doi.org/10.1016/j.chb.2004.10.016.
Tegos, S., Demetriadis, S., Papadopoulos, P. M., & Weinberger, A. (2016). Conversational agents for academically productive talk: A comparison of directed and undirected agent interventions. International Journal of Computer-Supported Collaborative Learning, 11, 417–440. https://doi.org/10.1007/s11412-016-9246-2.
Uttamchandani, S., & Lester, J. N. (this volume). Qualitative approaches to language in CSCL. In U. Cress, C. Rosé, A. F. Wise, & J. Oshima (Eds.), International handbook of computer-supported collaborative learning. Cham: Springer.
van der Zee, T., & Reich, J. (2018). Open education science. AERA Open, 4, 1–15. https://doi.org/10.1177/2332858418787466.
van’t Veer, A. E., & Giner-Sorolla, R. (2016). Pre-registration in social psychology: A discussion and suggested template. Journal of Experimental Social Psychology. https://doi.org/10.1016/j.jesp.2016.03.004.
Vogel, F., Wecker, C., Kollar, I., & Fischer, F. (2017). Socio-cognitive scaffolding with collaboration scripts: A meta-analysis. Educational Psychology Review, 29, 477–511. https://doi.org/10.1007/s10648-016-9361-7.
Walker, E., Rummel, N., & Koedinger, K. R. (2011). Designing automated adaptive support to improve student helping behaviors in a peer tutoring activity. International Journal of Computer-Supported Collaborative Learning, 6(2), 279–306. https://doi.org/10.1007/s11412-011-9111-2.
Wang, X., Wallace, M. P., & Wang, Q. (2017). Rewarded and unrewarded competition in a CSCL environment: A coopetition design with a social cognitive perspective using PLS-SEM analysis. Computers in Human Behavior, 72, 140–151. https://doi.org/10.1016/j.chb.2017.02.045.
Weinberger, A., Stegmann, K., & Fischer, F. (2010). Learning to argue online: Scripted groups surpass individuals (unscripted groups do not). Computers in Human Behavior, 26, 506–515. https://doi.org/10.1016/j.chb.2009.08.007.
Wise, A. F., Knight, S., & Buckingham Shum, S. (this volume). Collaborative learning analytics. In U. Cress, C. Rosé, A. F. Wise, & J. Oshima (Eds.), International handbook of computer-supported collaborative learning. Cham: Springer.
Yilmaz, F. G. K., & Yilmaz, R. (2019). Impact of pedagogic agent-mediated metacognitive support towards increasing task and group awareness in CSCL. Computers & Education, 134, 1–14. https://doi.org/10.1016/j.compedu.2019.02.001.
Zahn, C., Ruf, A., & Goldman, R. (this volume). Video data collection and video analyses in CSCL research. In U. Cress, C. Rosé, A. F. Wise, & J. Oshima (Eds.), International handbook of computer-supported collaborative learning. Cham: Springer.
Further Readings
Chen, J., Wang, M., Kirschner, P., & Tsai, C.-C. (2018). The role of collaboration, computer use, learning environments, and supporting strategies in CSCL: A meta-analysis. Review of Educational Research, 88(6), 799–843. https://doi.org/10.3102/0034654318791584. This article provides a meta-analytical synthesis of prior (quasi-)experimental research in the field of computer-supported collaborative learning (CSCL). It describes that most (quasi-)experimental CSCL research falls into three categories: (a) studies that look at the effects of collaborative versus individual computer-supported learning, (b) studies that investigate the effects of computer use during collaboration (vs. no computer use), and (c) studies that are interested in the effects of purposefully designed learning environments, tools, and scaffolds.
De Wever, B. (n.d.). NAPLeS webinar series: 15 minutes about selecting statistical methods for the learning sciences and reporting their results. Retrieved from http://isls-naples.psy.lmu.de/video-resources/guided-tour/15-minutes-dewever/index.html. In this webinar, Bram De Wever (Ghent University, Belgium) discusses relevant recommendations for conducting quantitative experimental CSCL studies. The webinar, for example, discusses the hierarchical grouping of participants that necessitates the use of multilevel analysis.
Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Houghton Mifflin Company. This book describes the theory behind experimentation, causality, and validity. It, furthermore, offers an exhaustive description of different experimental and quasi-experimental designs and their strengths and weaknesses.
Strijbos, J.-W., Martens, R. L., Jochems, W. M. G., & Broers, N. J. (2004). The effect of functional roles on group efficiency: Using multilevel modeling and content analysis to investigate computer-supported collaboration in small groups. Small Group Research, 35, 195–229. https://doi.org/10.1177/1046496403260843. This study was perhaps the first CSCL study to use multilevel analysis. Strijbos et al. compared role and nonrole groups with respect grades, efficiency, and collaboration. The article also describes the rationale for using multilevel analysis in a CSCL study.
Weinberger, A., Ertl, B., Fischer, F., & Mandl, H. (2005). Epistemic and social scripts in computer-supported collaborative learning. Instructional Science, 33, 1–30. https://doi.org/10.1007/s11251-004-2322-4. This article presents two classical studies on the effects of computer-supported collaboration scripts on knowledge acquisition. In both studies, epistemic and social scripts were varied in a 2 x 2 design. While study 1 implemented the different conditions in an asynchronous, text-based learning environment, study 2 realized a synchronous, video-based environment. Both studies showed positive effects of the social scripts on knowledge acquisition, but no or even negative effects of the epistemic scripts that were used.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Janssen, J., Kollar, I. (2021). Experimental and Quasi-Experimental Research in CSCL. In: Cress, U., Rosé, C., Wise, A.F., Oshima, J. (eds) International Handbook of Computer-Supported Collaborative Learning. Computer-Supported Collaborative Learning Series, vol 19. Springer, Cham. https://doi.org/10.1007/978-3-030-65291-3_27
Download citation
DOI: https://doi.org/10.1007/978-3-030-65291-3_27
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-65290-6
Online ISBN: 978-3-030-65291-3
eBook Packages: EducationEducation (R0)