Advertisement

Time is precious: Variable- and event-centred approaches to process analysis in CSCL research

  • Peter Reimann
Article
First Online:

Abstract

Although temporality is a key characteristic of the core concepts of CSCL—interaction, communication, learning, knowledge building, technology use—and although CSCL researchers have privileged access to process data, the theoretical constructs and methods employed in research practice frequently neglect to make full use of information relating to time and order. This is particularly problematic when collaboration and learning processes are studied in groups that work together over weeks, and months, as is often the case. The quantitative method dominant in the social and learning sciences—variable-centred variance theory—is of limited value for studying change on longer time scales. We introduce the event-centred view of process as a more generally applicable approach, not only for quantitative analysis, but also for providing closer links between qualitative and quantitative research methods. A number of methods for variable- and event-centred analysis of process data are described and compared, using examples from CSCL research. I conclude with suggestions on how experimental, descriptive, and design-oriented research orientations can become better integrated.

Keywords

Process analysis Qualitative methods Quantitative methods Research methods 
This is a preview of subscription content, log in to check access.

Notes

Acknowledgements

I would like to thank Anindito Aditomo and Fides-Ronja Voss for conducting the duration analysis on the CSCL 2005 and 2007 conference papers, and a number of colleagues for feedback on earlier versions of this paper.

References

  1. Abbott, A. (1984). Event sequence and event duration: Colligation and measurement. Historical Methods, 17, 192–204.Google Scholar
  2. Abbott, A. (1990). A primer on sequence methods. Organization Science, 1(4), 375–392.CrossRefGoogle Scholar
  3. Abbott, A., & Hrycak, A. (1990). Measuring resemblance in sequence data: An optimal matching analysis of musicians’ careers. American Journal of Sociology, 96, 144–185.CrossRefGoogle Scholar
  4. Abell, P. (1987). The syntax of social life: The theory and method of comparative narratives. Oxford: Clarendon.Google Scholar
  5. Abell, P. (2004). Narrative explanations: An alternative to variable-centered explanation? Annual Review of Sociology, 30, 287–310.CrossRefGoogle Scholar
  6. Aristotle. (1941). In R. McKeon (Ed.), The basic works of Aristotle. New York: Random House.Google Scholar
  7. Arrow, H., McGrath, J. E., & Behrdal, J. (2000). Small groups as complex systems: Formation, co-ordination, development and adaptation. Thousand Oaks: Sage.Google Scholar
  8. Box, G., & Jenkins, G. (1976). Time series analysis: Forecasting and control. San Francisco: Holden-Day.Google Scholar
  9. Cassandras, C. G. (1993). Discrete event systems. Homewood: Richard D. Irwin, Inc.Google Scholar
  10. Chinn, C., Erkens, G., & Puntambekar, S. (Eds.). (2007). CSCL 2007–The Computer Supported Collaborative (CSCL) Conference 2007 (Rutgers, NJ, July 16th–21st, 2007): International Society of the Learning Sciences.Google Scholar
  11. Coleman, J. S. (1964). Introduction to mathematical sociology. New York: Free.Google Scholar
  12. Cress, U. (2008). The need for considering multilevel analysis in CSCL research–An appeal for the use of more advanced statistical methods. International Journal of Computer-supported Collaborative Learning, 3, 69–84.CrossRefGoogle Scholar
  13. Danto, A. C. (1985). Narration and knowledge. New York: Columbia University Press.Google Scholar
  14. Dillenbourg, P., & Hong, F. (2008). The mechanics of CSCL macro scripts. International Journal of Computer-supported Collaborative Learning, 3(1), 5–24.CrossRefGoogle Scholar
  15. Gill, A. (1962). Introduction to the theory of finite-state machines. New York: McGraw-Hill.Google Scholar
  16. Goodwin, C. (2000). Action and embodiment within situated human interaction. Journal of Pragmatics, 32, 1489–1522.CrossRefGoogle Scholar
  17. Holmes, M. E. (1997). Optimal matching analysis of negotiation phase sequences in simulated and authentic hostage negotiations. Communication Reports, 10, 1–9.Google Scholar
  18. Jones, C., Dirckinck-Holmfeld, L., & Linstroem, B. (2006). A relational, indirect, meso-level approach to CSCL design in the next decade. Computer-supported Collaborative Learning, 1(1), 35–56.CrossRefGoogle Scholar
  19. Kapur, M., Voiklis, J., & Kinzer, C. K. (2005). Problem solving as a complex, evolutionary activity: a methodological framework for analyzing problem-solving processes in a computer-supported collaborative environment. In T. Koschmann, D. Suthers & T.-W. Chan (Eds.), Computer-supported collaborative learning 2005: The next 10 years! (pp. 252–261). Mahway: Erlbaum.Google Scholar
  20. Kay, J., Maisonneuve, N., Yacef, K., & Zaiane, O. R. (2006). Mining patterns of events in students’ teamwork data. Paper presented at the Proceedings of Educational Data Mining Workhop held in conjunction with ITS’06, Jhongli, Taiwan.Google Scholar
  21. Kay, J., Yacef, K., & Reimann, P. (2007). Visualisations for team learning: small teams working on long-term projects. In C. Chinn, G. Erkens & S. Puntambekar (Eds.), Minds, mind, and society. Proceedings of the 6th International Conference on Computer-supported Collaborative Learning (CSCL 2007) (pp. 351–353). New Brunswick: International Society of the Learning Sciences.Google Scholar
  22. Kemeny, J. G., & Snell, J. L. (1976). Finite Markov chains. New York: Springer.Google Scholar
  23. Kollar, I., Fischer, F., & Hesse, F. W. (2006). Collaboration scripts - a conceptual analysis. Educational Psychological Review, 18, 159–185.CrossRefGoogle Scholar
  24. Koschmann, T. (2001). Revising the paradigms of instructional technology. Paper presented at the Meeting at the Crossroads. 8th Annual Conference of the Australasian Society for Computers in Learning in Tertiary Education (ASCILITE).Google Scholar
  25. Koschmann, T., Suthers, D., & Chan, T.-W. (Eds.). (2005). CSCL 2005–the next 10 years. Proceedings of the International Conference on Computer Supported Collaborative Learning 2005. Mahwah: Erlbaum.Google Scholar
  26. Langley, A. (1999). Strategies for theorizing from process data. Academy of Management Review, 24(4), 691–710.CrossRefGoogle Scholar
  27. Markauskaite, L., & Reimann, P. (2008). Enhancing and scaling-up design-based research: The potential of e-research. Paper presented at the International Learning Science Conference–ICLS 2008, Utrecht, The Netherlands (24–28 July 2008) from http://www.fi.uu.nl/en/icls2008/343/paper343.pdf
  28. McGrath, J. E., & Tschan, F. (2004). Temporal matters in social psychology: Examining the role of time in the lives of groups and individuals. Washington: American Psychological Association.CrossRefGoogle Scholar
  29. Mercer, N. (2008). The seeds of time: Why classroom dialogue needs a temporal analysis. The Journal of the Learning Sciences, 17, 33–59.CrossRefGoogle Scholar
  30. Miura, R. M. (1986). Some mathematical questions in biology: DNA sequence analysis (Vol. 17). Providence: American Mathematical Society.Google Scholar
  31. Mochizuki, T., Kato, H., Yaegashi, K., Nishimori, T., Nagamori, Y., & Fujita, S. (2007). ProBoPortable: Does the cellular phone software promote emergent division of labor in project-based learning? In C. Chinn, G. Erkens & S. Puntambekar (Eds.), Minds, mind, and society. Proceedings of the 6th International Conference on Computer-supported Collaborative Learning (CSCL 2007) (pp. 512–514). New Brunswick: International Society of the Learning Sciences.Google Scholar
  32. Mohr, L. (1982). Explaining organizational behavior. San Francisco: Jossey-Bass.Google Scholar
  33. Muukkonen, H., Hakkarainen, K., Konsonen, K., Jalonen, S., Heikkil, A., Lonka, K., et al. (2007). Process- and context-sensitive research on academic knowledge practices: Developing CASS-tools and methods. In C. Chinn, G. Erkens & S. Puntambekar (Eds.), Minds, mind, and society. Proceedings of the 6th International Conference on Computer-supported Collaborative Learning (CSCL 2007) (pp. 541–543). New Brunswick: International Society of the Learning Sciences.Google Scholar
  34. Olson, G. M., Herbsleb, J. D., & Rueter, H. H. (1994). Characterizing the sequential structure of interactive behaviors through statistical and grammatical techniques. Human-Computer Interaction, 9(3/4), 427–472.CrossRefGoogle Scholar
  35. Poole, M. S., & DeSanctis, G. (2004). Structuration theory in information systems research: methods and controversies. In M. E. Whitman & Woszcynski (Eds.), The handbook of information systems research (pp. 206–249). Idea Group Publishing.Google Scholar
  36. Poole, M. S., & Roth, J. (1989). Decision making in small groups IV: A typology of group decision paths. Human Communication Research, 15(3), 323–356.CrossRefGoogle Scholar
  37. Poole, M. S., van de Ven, A., Dooley, K., & Holmes, M. E. (2000). Organizational change and innovation processes. Theories and methods for research. New Oxford: Oxford University Press.Google Scholar
  38. Rabiner, L. R. (1989). A tutorial on hidden markov models and selected applications in speech recognition. Proceedings of the IEEE, 77(2), 257–286.CrossRefGoogle Scholar
  39. Reimann, P., Frerejean, J., & Thompson, K. (2009). Using process mining to identify decision making processes in virtual teams (accepted). Paper presented at the International Conference on Computer-supported collaborative learning (CSCL2009). Rhodes/Greece.Google Scholar
  40. Reisig, W. (1985). Petri Nets. An introduction. Berlin: Springer.Google Scholar
  41. Ritter, F. E., Nerb, J., Lehtinen, E., & O’Shea, T. (Eds.). (2007). In order to learn: How the sequences of topics influences learning. Oxford: Oxford.Google Scholar
  42. Sanderson, P. M., & Fisher, C. (1994). Exploratory sequential data analysis: Foundations. Human-Computer Interaction, 9(3/4), 251–317.CrossRefGoogle Scholar
  43. Schegloff, E. A. (2007). Sequence organization in interaction. New York: Cambridge University Press.Google Scholar
  44. Schümmer, T., Strijbos, J.-W., & Berkel, T. (2005). A new direction for log file analysis in CSCL: Experiences with a spatio-temporal metric. In T. Koschmann, D. Suthers & T. W. Chan (Eds.), Computer Supported Collaborative Learning 2005: The next 10 years! (pp. 567–576). Mahwah: Erlbaum.Google Scholar
  45. Sloane, F. C. (2008). Multilevel models in design research. A case from mathematics education. In A. E. Kelly, R. A. Lesh & J. Y. Baek (Eds.), Handbook of design research methods in education (pp. 459–476). New York: Routledge.Google Scholar
  46. Sloane, F. C., & Kelly, A. E. (2008). Design research and the study of change. In A. E. Kelly, R. A. Lesh & J. Y. Baek (Eds.), Handbook of design research methods in education (pp. 441–448). New York: Routledge.Google Scholar
  47. Soller, A., Wiebe, J., & Lesgold, A. (2002). A machine learning approach to assessing knowledge sharing during collaborative learning activities. In G. Stahl (Ed.), Proceedings of Computer Support for Collaborative Learning (pp. 128–137). Hillsdale: Erlbaum.Google Scholar
  48. Stahl, G. (2006). Group cognition: Computer support for building collaborative knowledge. Cambridge: MIT.Google Scholar
  49. Strijbos, J. W., Martens, R. L., Prins, F. J., & Jochems, W. M. G. (2006). Content analysis: What are they talking about? Computers & Education, 46, 29–48.CrossRefGoogle Scholar
  50. Suppes, P., & Atkinson, R. C. (1960). Markov learning models for multiperson interactions. Stanford: Stanford University Press.Google Scholar
  51. Suthers, D. D. (2006). A qualitative analysis of collaborative knowledge construction through shared representations. Research and Practice in Technology Enhanced Learning, 1(2), 115–142.CrossRefGoogle Scholar
  52. Suthers, D. D., Dwyer, N., Medina, R., & Vatrapu, R. (2007). A framework for eclectic analysis of collaborative interaction. In C. Chinn, G. Erkens & S. Puntambekar (Eds.), Minds, mind, and society. Proceedings of the 6th International Conference on Computer-supported Collaborative Learning (CSCL 2007) (pp. 694–703). New Brunswick: International Society of the Learning Sciences.Google Scholar
  53. Van der Aalst, W. M. P., & Günther, C. W. (2007). Finding structure in unstructured processes: the case of process mining. In T. Basten, G. Juhas & S. Shukla (Eds.), Proceedings the 7th International Conference on Applications of Concurrency to System Design (ACSD 2007; Bratislava, Slovak Republic) (pp. 3–12). Los Alamitos: IEEE Computer Society.CrossRefGoogle Scholar
  54. Wever, B. d., Schellens, T., Valcke, M., & Keer, H. v. (2006). Content analysis schemes to analyze transcripts of online asynchronous discussion groups: A review. Computers & Education, 46, 6–28.CrossRefGoogle Scholar
  55. Zumbach, J., & Reimann, P. (2003). Influence of feedback on distributed problem based learning. In B. Wasson, S. Ludvigsen & U. Hoppe (Eds.), Designing for change in networked learning environments (pp. 219–228). Dordrecht: Kluwer.Google Scholar

Copyright information

© International Society of the Learning Sciences, Inc.; Springer Science + Business Media, LLC 2009

Authors and Affiliations

  1. 1.Research Centre for Computer-supported Learning and Cognition (CoCo)University of SydneySydneyAustralia

Personalised recommendations

Cite article

Buy options