The need for considering multilevel analysis in CSCL research—An appeal for the use of more advanced statistical methods
- 657 Downloads
Per definition, CSCL research deals with the data of individuals nested in groups, and the influence of a specific learning setting on the collaborative process of learning. Most well-established statistical methods are not able to analyze such nested data adequately. This article describes the problems which arise when standard methods are applied and introduces multilevel modelling (MLM) as an alternative and adequate statistical approach in CSCL research. MLM enables testing interactional effects of predictor variables varying within groups (for example, the activity of group members in a chat) and predictors varying between groups (for example, the group homogeneity created by group members’ prior knowledge). So it allows taking into account that an instruction, tool or learning environment has different but systematic effects on the members within the groups on the one hand and on the groups on the other hand. The underlying statistical model of MLM is described using an example from CSCL. Attention is drawn to the fact that MLM requires large sample sizes which are not provided in most CSCL research. A proposal is made for the use of some analyses which are useful.
KeywordsMultilevel models Hierarchical linear models Quantitative analysis for CSCL
The author would like to thank three anonymous reviewers for their helpful comments on an earlier version of this article.
- Bryk, A. S., & Raudenbush, S. W. (1992). Hierarchical linear models. Newbury Park, CA: Sage.Google Scholar
- Burstein, L., Kim, S. S., & Delandshere, G. (1989). Multilevel investigations of systematically varying slopes: Issues, alternatives, and consequences. In D. Bock (Ed.) Multilevel analysis of educational data (pp. 233–279). San Diego: Academic.Google Scholar
- Hox, J. J. (2002). Multilevel analysis: techniques and applications. Mahwah, NJ: Lawrence Erlbaum Associates.Google Scholar
- Kashy, D. A., & Kenny, D. A. (2000). The analysis of data from dyads and groups. In H. T. Reis, & C. M. Judd (Eds.) Handbook of research methods in social and personality psychology. Cambridge, UK: Cambridge University Press.Google Scholar
- Kenny, D. A., Kashy, D. A., & Bolger, N. (1998). Data analysis in social psychology. In D. T. Gilbert, S. T. Fiske, & G. Lindzey (Eds.) The handbook of social psychology (4th ed., Vol. 1, pp. 233–265). New York: McGraw Hill.Google Scholar
- Kimmerle, J. & Cress (2008). Group Awareness and Self-Presentation in Computer-Supported Information Exchange. International Journal of Computer-Supported Collaborative Learning DOI 10.1007/s11412-007-9027-z.
- Kreft, I. (1996). Are multilevel techniques necessary? An overview, including simulation studies. Retrieved March, 20, 2007 from http://www.calstatela.edu/faculty/ikreft/quarterly/quarterly.html.
- Raudenbush, S., & Bryk, A. (2002). Hierarchical linear models: applications and data analysis methods. Thousand Oaks: Sage.Google Scholar
- Snijders, T. A. B., & Bosker, R. J. (1999). Multilevel analysis. London: Sage.Google Scholar
- Stevens, J. (1996). Applied multivariate statistics for the social sciences (3rd ed.). Mahwah, NJ: Lawrence Erlbaum.Google Scholar