Abstract
In this chapter we report on the development of a rating scheme for the analysis of collaborative process data, and on its implementation in diverse CSCL settings. The rating scheme is composed of nine dimensions measuring different aspects of collaboration quality: sustaining mutual understanding, dialogue management, information pooling, reaching consensus, task division, time management, technical coordination, reciprocal interaction, and individual task orientation. It can be applied to recordings (video, audio, screen recordings, or log data) of student interaction and does not necessarily require transcripts or written records. While the rating scheme was originally developed in the context of a specific CSCL setting (video-based interdisciplinary problem-solving in the medical domain; Meier et al. (2007)), we demonstrate in our chapter that it can successfully be adapted to other CSCL settings. First, we introduce the initial rating scheme and its dimensions. Next we describe the process of adapting it to data from a very different CSCL setting (chat-based interaction in computer science classes). We briefly report on a study that used the ratings of collaboration quality as basis for adaptive feedback to students on how to improve their collaboration. Finally, we describe how we have integrated our rating scheme with ActivityLens (Avouris et al. 2007), a software tool which allows for a combined analysis of multiple sources of data (e.g., logfiles, audio and video recordings). Several tool modifications were made to permit analysis of collaborative process data from yet another CSCL study in which high-school students collaborated face-to-face on solving algebra problems with support from an intelligent tutoring system. We conclude our chapter with a discussion of practical implications for practitioners who may wish to adapt and apply our rating scheme.
Further information about the rating scheme and materials for rating can be obtained from the first author.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Anderson, A. H., O’Malley, C., Doherty-Sneddon, G., Langton, S., Newlands, A., Mullin, J., et al. (1997). The impact of VMC on collaborative problem solving: An analysis of task performance, communicative process, and user satisfaction. In K. E. Finn, A. J. Sellen, & S. B. Wilbur (Eds.), Video-mediated communication (pp. 133–156). Mahwah: Lawrence Erlbaum Associates.
Angiolillo, J. S., Blanchard, H. E., Israelski, E. W., & Mané, A. (1997). Technology constraints of videomediated communication. In K. E. Finn, A. J. Sellen, & S. B. Wilbur (Eds.), Video-mediated communication (pp. 51–74). Mahwah: Lawrence Erlbaum Associates.
Avouris, N., Fiotakis, G., Kahrimanis, G., Margaritis, M., & Komis, V. (2007). Beyond logging of fingertip actions: Analysis of collaborative learning using multiple sources of data. Journal of Interactive Learning Research JILR, 18(2), 231–250.
Avouris, N., Margaritis, M., & Komis, V. (2004). Modelling interaction during small-group synchronous problem solving activities: The Synergo approach. In 2nd International Workshop on Designing Computational Models of Collaborative Learning Interaction, ITS 2004, Brazil.
Barron, B. (2000). Achieving coordination in collaborative problem-solving groups. Journal of the Learning Sciences, 9, 403–436.
Clark, H. H. (1996). Using language. Cambridge: Cambridge University Press.
Clark, H. H., & Brennan, S. E. (1991). Grounding in communication. In L. B. Resnick, J. M. Levine, & S. D. Teasley (Eds.), Perspectives on socially shared cognition (pp. 127–148). Washington: American Psychological Association.
Deutsch, M. (2003). Cooperation and conflict: A personal perspective on the history of the social psychological study of conflict resolution. In M. A. West, D. Tjosvold, & K. G. Smith (Eds.), International handbook of organizational teamwork and cooperative working (pp. 9–43). Chichester: Wiley.
Dillenbourg, P. (1999). Introduction: What do you mean by “collaborative learning”? In P. Dillenbourg (Ed.), Collaborative learning: Cognitive and computational approaches (pp. 1–19). Amsterdam: Pergamon.
Diziol, D., Rummel, N., Spada, H., & McLaren, B. (2007). Promoting learning in mathematics: Script support for collaborative problem solving with the Cognitive Tutor Algebra. In Chinn, C. A., Erkens, G., & Puntambekar, S. (Eds.), Mice, minds, and society. Proceedings of the Computer Supported Collaborative Learning (CSCL) Conference 2007 I (Vol. 8, pp. 39–41). International Society of the Learning Sciences, Inc.
Erkens, G., Jaspers, J., Prangsma, M., & Kanselaar, G. (2005). Coordination processes in computer supported collaborative writing. Computers in Human Behavior, 21, 463–486.
Fischer, F., & Mandl, H. (2003). Being there or being where? Videoconferencing and cooperative learning. In H. van Oostendorp (Ed.), Cognition in a digital world (pp. 205–223). Mahwah: Lawrence Erlbaum Associates.
Heckhausen, H. (1989). Motivation und Handeln. [Motivation and behavior]. Berlin, Heidelberg, New York: Springer.
Hermann, F., Rummel, N., & Spada, H. (2001). Solving the case together: The challenge of net-based interdisciplinary collaboration. In P. Dillenbourg, A. Eurelings, & K. Hakkarainen (Eds.), Proceedings of the first European conference on Computer-Supported Collaborative Learning (pp. 293–300). Maastricht: McLuhan Institute.
Hinsz, V. B., Tindale, R. S., & Vollrath, D. A. (1997). The emerging conceptualization of groups as information processors. Psychological Bulletin, 121(1), 43–64.
Janis, I. L. (1982). Groupthink. Boston: Houghton Mifflin.
Johnson, D. W., & Johnson, R. T. (2003). Training for cooperative group work. In M. A. West, D. Tjosvold, & K. G. Smith (Eds.), International handbook of organizational teamwork and cooperative working (pp. 167–183). Chichester: Wiley.
Jucks, R., Bromme, R., & Runde, A. (2003). Audience Design von Experten in der netzgestützten Kommunikation: Die Rolle von Heuristiken über das geteilte Vorwissen. [Audience design of experts in net-based communication: The role of heuristics about shared knowledge]. Zeitschrift für Psychologie, 211(2), 60–74.
Kerlinger, F. N., & Lee, H. B. (2000). Foundations of behavioral research. Fort Worth: Harcourt College.
Kneser, C., & Ploetzner, R. (2001). Collaboration on the basis of complementary domain knowledge: Observed dialogue structures and their relation to learning success. Learning and Instruction, 11(1), 53–83.
Larson, J. R., & Christensen, C. (1993). Groups as problem-solving units: Toward a new meaning of social cognition. The British Journal of Social Psychology, 32, 5–30.
Mayring, P. (2003). Qualitative Inhaltsanalyse. Grundlagen und Techniken [Qualitative content analysis. Foundations and techniques. Weinheim: Beltz.
Malone, T. W., & Crowston, K. (1990). What is coordination theory and how can it help design cooperative work systems? In Proceedings of the conference on computer-supported cooperative work, Los Angeles (pp. 357–370). New York: ACM.
Malone, T. W., & Crowston, K. (1994). The interdisciplinary study of coordination. ACM Computing Surveys, 26(1), 87–119.
Meier, A., Spada, H., & Rummel, N. (2007). A rating scheme for assessing the quality of computer-supported collaboration processes. International Journal of Computer-Supported Collaborative Learning, 2(1), 63–86.
Meier, A., Voyiatzaki, E., Kahrimanis, G., Rummel, N., Spada, H., & Avouris, N. (2008). Teaching students how to improve their collaboration: Assessing collaboration quality and providing adaptive feedback in a CSCL setting. Paper presented as part of the symposium by Rummel, N., & Weinberger, A. New challenges in CSCL: Towards adaptive script support. Proceedings of the 2008 International Conference of the Learning Sciences, Utrecht, The Netherlands.
Moreland, R. L., & Myaskovsky, L. (2000). Exploring the performance benefits of group training: Transactive memory or improved communication? Organizational Behavior and Human Decision Processes, 82(1), 117–133.
Nickerson, R. S. (1999). How we know—and sometimes misjudge—what others know: Imputing one’s own knowledge to others. Psychological Bulletin, 125(6), 737–759.
O’Conaill, B., & Whittaker, S. (1997). Characterizing, predicting, and measuring video-mediated communication: A conversational approach. In K. E. Finn, A. J. Sellen, & S. B. Wilbur (Eds.), Videomediated communication (pp. 107–132). Mahwah: Lawrence Erlbaum Associates.
Postmes, T., Spears, R., & Cihangir, S. (2001). Quality of decision making and group norms. Journal of Personality and Social Psychology, 80(6), 918–930.
Rummel, N., Diziol, D., & Spada, H. (2010). Scripted collaborative learning with the Cognitive Tutor Algebra: An experimental classroom study. Manuscript under revision.
Rummel, N., Hauser, S., & Spada, H. (2007). How does net-based interdisciplinary collaboration change with growing domain expertise? In Chinn, C. A., Erkens, G., & Puntambekar, S. (Eds.), Mice, minds, and sociecty. Proceedings of the Computer Supported Collaborative Learning (CSCL) Conference 2007 (pp. 611–620). International Society of the Learning Sciences, Inc. ISSN 1819-0146.
Rummel, N., & Spada, H. (2005a). Instructional support for collaboration in desktop videoconferencing settings: How it can be achieved and assessed. In R. Bromme, F. W. Hesse, & H. Spada (Eds.), Barriers and biases in computer-mediated knowledge communication—and how they may be overcome (pp. 59–88). Berlin, Heidelberg, New York: Springer.
Rummel, N., & Spada, H. (2005b). Learning to collaborate: An instructional approach to promoting collaborative problem-solving in computer-mediated settings. Journal of the Learning Sciences, 14(2), 201–241.
Rummel, N., Spada, H., & Hauser, S. (2009). Learning to collaborate from being scripted or from observing a model. International Journal of Computer-Supported Collaborative Learning, 4(1), 69–92.
Sacks, H., Schegloff, E., & Jefferson, G. (1974). A simplest systematic for the organization of turn-taking in conversation. Language, 50, 696–753.
Sosa y Fink, S. (2003). Merkmale gelungener Kooperation. Eine qualitative Analyse netzgestützter Zusammenarbeit. [Characteristics of successful cooperation. A qualitative analysis of net-based collaboration.] Diploma thesis, Albert-Ludwigs-Universität, Freiburg.
Stasser, G., Stewart, D., & Wittenbaum, G. (1995). Expert roles and information exchange during discussion: The importance of knowing who knows what. Journal of Experimental Social Psychology, 31, 244–265.
Stasser, G., & Titus, W. (1985). Pooling of unshared information in group decision making: Biased information sampling during group discussion. Journal of Personality and Social Psychology, 48, 1467–1478.
Steiner, I. D. (1972). Group process and productivity. New York: Academic.
Strijbos, J.-W., Martens, R. L., Prins, F. J., & Jochems, W. M. G. (2006). Content analysis: What are they talking about? Computers and Education, 46, 29–48.
Tindale, R. S., Kameda, T., & Hinsz, V. B. (2003). Group decision making. In M. A. Hogg & J. Cooper (Eds.), Sage handbook of social psychology (pp. 381–403). London: Sage.
Voyiatzaki, E., Meier, A., Kahrimanis, G., Rummel, N., Spada, H., & Avouris, N. (2008a). Rating the quality of collaboration during networked problem solving activities. In Proceedings of the 6th International Conference on Networked Learning (pp. 409–416).
Voyiatzaki, E., Papadakis, S., Rossiou, E., Avouris, N., Paparizzos, K., & Hadzilacos, T. (2008b). One size does not fit all: A case study of combining networked learning methods and tools. In Proceedings of the 6th International Conference on Networked Learning, 5–6 May 2008, Halkidiki, Greece. http://www.networkedlearningconference.org.uk/abstracts/PDFs/Voyiatzaki_847-848.pdf. Retrieved 5 November 2008.
Webb, N. M. (1989). Peer interaction and learning in small groups. Int J Educ Res, 13, 21–39.
Wegner, D. M. (1987). Transactive memory: A contemporary analysis of the group mind. In B. Mullen & G. R. Goethals (Eds.), Theories of group behavior (pp. 185–208). Berlin, Heidelberg, New York: Springer.
Wirtz, M., & Caspar, F. (2002). Beurteilerübereinstimmung und Beurteilerreliabilität. [Inter-rater agreement and inter-rater reliability]. Göttingen: Verlag für Psychologie.
Wittenbaum, G. M., Vaughan, S. I., & Stasser, G. (1998). Coordination in task performing groups. In R. S. Tindale et al. (Eds.), Theory and research on small groups (pp. 177–204). New York: Plenum.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer Science+Business Media, LLC
About this chapter
Cite this chapter
Rummel, N., Deiglmayr, A., Spada, H., Kahrimanis, G., Avouris, N. (2011). Analyzing Collaborative Interactions Across Domains and Settings: An Adaptable Rating Scheme. In: Puntambekar, S., Erkens, G., Hmelo-Silver, C. (eds) Analyzing Interactions in CSCL. Computer-Supported Collaborative Learning Series, vol 12. Springer, Boston, MA. https://doi.org/10.1007/978-1-4419-7710-6_17
Download citation
DOI: https://doi.org/10.1007/978-1-4419-7710-6_17
Published:
Publisher Name: Springer, Boston, MA
Print ISBN: 978-1-4419-7709-0
Online ISBN: 978-1-4419-7710-6
eBook Packages: Humanities, Social Sciences and LawEducation (R0)