Enhancing student knowledge acquisition from online learning conversations

  • Evren Eryilmaz
  • Jakko van der Pol
  • Terry Ryan
  • Philip Martin Clark
  • Justin Mary


This article reports a theory-driven experimental study that evaluates the effects of an annotation functionality on online social interaction and individual learning outcomes. The central hypothesis of this study is that directly addressing a part of a text by annotating it and then connecting each annotation with its related discussion can decrease coordinative interaction costs and result in a higher-quality discussion that favors greater gains in individual learning outcomes. To reach our objective, we carried out a theory-driven experimental study that compares two versions of an anchored discussion system: one with annotation functionality and one without it, both displaying the learning material side by side with its associated discussion in one window. Participants were 106 students enrolled in two sections of a blended-format course in health education. We assigned each section to a software condition. The examination of students’ online social interaction centered on a fine-grained content analysis of coordination and knowledge construction activities as well as sequential analysis of knowledge construction activities. The results indicate that annotation functionality decreased coordinative interaction costs and stimulated more elaborated discussions that favored greater gains in individual learning outcomes. Implications for research and practice are discussed.


Anchored discussion Annotation functionality Online social interaction Interaction cost Coordination Knowledge construction Individual learning outcomes 


  1. Ackerman, M. S., & Halverson, C. (2004). Sharing expertise: The next step for knowledge management. In V. Wulf & M. H. Huysman (Eds.), Social capital and information technology (pp. 273–299). Cambridge: MIT.Google Scholar
  2. Alamargot, D., & Andriessen, J. (2002). The “power” of text production activity in collaborative modelling: Nine recommendations to make a computer supported situation work. In M. Baker, P. Brna, K. Stenning, & A. Tiberghien (Eds.), The role of communication in learning to model (pp. 275–302). Mahwah: Lawrence Erlbaum Associates.Google Scholar
  3. Amadieu, F., Tricot, A., & Mariné, C. (2009). Prior knowledge in learning from a non-linear electronic document: Disorientation and coherence of the reading sequences. Computers in Human Behavior, 25(2), 381–388.CrossRefGoogle Scholar
  4. Andriessen, J., & Sandberg, J. (1999). Where is education heading and how about AI. International Journal of Artificial Intelligence in Education, 10, 130–150.Google Scholar
  5. Ayres, P. (2006). Using subjective measures to detect variations of intrinsic cognitive load within problems. Learning and Instruction, 16(5), 389–400.CrossRefGoogle Scholar
  6. Baker, M., Hansen, T., Joiner, R., & Traum, D. (1999). The role of grounding in collaborative learning tasks. In P. Dillenbourg (Ed.), Collaborative learning: Cognitive and computational approaches (pp. 31–63). Elsevier Science Publishers.Google Scholar
  7. Baltes, B. B., Dickson, M., Sherman, M. P., Bauer, C. C., & LaGanke, J. (2002). Computer-mediated communication and group decision making: A meta-analysis. Organizational Behavior and Human Decision Processes, 87(1), 156–179.CrossRefGoogle Scholar
  8. Boekaerts, M. & Simons, P. R. J. (1995). Leren en instructie: Psychologie van de leerling en het leerproces [Learning and instruction: Psychology of the learner and his learning process]. Assen: Dekker & Van de Vegt.Google Scholar
  9. Brush, B. A. J., Bargeron, D., Grudin, J., Borning, A., & Gupta, A. (2002). Supporting interaction outside of class: Anchored discussion vs. discussion boards. In G. Stahl (Ed.), Computer support for collaborative learning: Foundations for a CSCL community. Proceedings of CSCL 2002 (pp. 425–434). Hillsdale: Lawrence Erlbaum Associates.CrossRefGoogle Scholar
  10. Cakir, M. P., Zemel, A., & Stahl, G. (2009). The joint organization of interaction within a multimodal CSCL medium. International Journal of Computer-Supported Collaborative Learning, 4(2), 115–149.CrossRefGoogle Scholar
  11. Carlon, S., Woods, D. B., Berg, B., Claywell, L., LeDuc, K., Marcisz, N., et al. (2012). The community of inquiry instrument: Validation and results in online health care disciplines. Computers in Education, 59(2), 215–221.CrossRefGoogle Scholar
  12. Cierniak, G., Scheiter, K., & Gerjects, P. (2009). Explaining the split-attention effect: Is the reduction of extraneous cognitive load accompanied by an increase in germane cognitive load. Computers in Human Behavior, 25(2), 315–324.CrossRefGoogle Scholar
  13. Clark, H. H., & Wilkes-Gibbs, D. (1986). Referring as a collaborative process. Cognition, 22(1), 1–39.Google Scholar
  14. Clark, J. (2001). Stimulation collaboration and discussion in online learning environments. Internet and Higher Education, 4(2), 119–124.CrossRefGoogle Scholar
  15. Cobos, R., & Pifarré, M. (2008). Collaborative knowledge construction in the web supported by the KnowCat system. Computers in Education, 50(3), 962–978.CrossRefGoogle Scholar
  16. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum.Google Scholar
  17. Cress, U., & Kimmerle, J. (2008). A systemic and cognitive view on collaborative knowledge building with wikis. International Journal of Computer-Supported Collaborative Learning, 3(2), 105–122.CrossRefGoogle Scholar
  18. De Jong, F., Kollöffel, B., Van der Meijden, H., Staarman, J. K., & Janssen, J. (2005). Regulative processes in individual, 3D and computer supported cooperative learning contexts. Computers in Human Behavior, 21(4), 645–670.CrossRefGoogle Scholar
  19. Dennen, V. (2008). Looking for evidence of learning: Assessment and analysis methods for online discourse. Computers in Human Behavior, 24(2), 205–219.CrossRefGoogle Scholar
  20. Dillenbourg, P. (1999). What do you mean by “collaborative learning”? In P. Dillenbourg (Ed.), Collaborative learning: Cognitive and computational approaches (pp. 1–19). Oxford: Elsevier.Google Scholar
  21. Dillenbourg, P., & Bétrancourt, M. (2006). Collaboration load. In J. Elen & R. E. Clark (Eds.), Handling complexity in learning environments: Research and theory (pp. 141–165). Amsterdam: Elsevier.Google Scholar
  22. Ding, N. (2009). Visualizing the sequential process of knowledge elaboration in computer-supported collaborative problem solving. Computers in Education, 52(2), 509–519.CrossRefGoogle Scholar
  23. Engelmann, T., Dehler, J., Bodemer, D., & Buder, J. (2009). Knowledge awareness in CSCL: A psychological perspective. Computers in Human Behavior, 25(4), 949–960.CrossRefGoogle Scholar
  24. Erkens, G., Jaspers, J., Prangsma, M., & Kanselaar, G. (2005). Coordination processes in computer supported collaborative writing. Computers in Human Behavior, 21(3), 463–486.CrossRefGoogle Scholar
  25. Eryilmaz, E., Alrushiedat, N., Kasemvilas, S., Mary, J., & Van der Pol, J. (2009). The effect of anchoring online discussion on collaboration and cognitive load. In: Proceedings of 15th Americas Conference on Information Systems, San Francisco, California.Google Scholar
  26. Eryilmaz, E., Ryan, T., Van der Pol, J., Kasemvilas, S., & Mary, J. (2013). Fostering quality and flow of online learning conversations by artifact-centered discourse systems. Journal of the Association for Information Systems, 14(1).Google Scholar
  27. Fiedler, K., Schott, M., & Meiser, T. (2011). What mediation analysis can (not) do. Journal of Experimental Social Psychology, 47(6), 1231–1236.CrossRefGoogle Scholar
  28. Garrison, D. R., & Arbaugh, J. B. (2007). Researching the community of inquiry framework: Review, issues and future directions. The Internet and Higher Education, 10(3), 157–172.CrossRefGoogle Scholar
  29. Garrison, D. R., Innes, M. C., & Fung, T. S. (2010). Exploring causal relationships among teaching, cognitive and social presence: Student perceptions of the community of inquiry framework. Internet and Higher Education, 13(1–2), 31–36.CrossRefGoogle Scholar
  30. Gunawardena, C. N. (1995). Social presence theory and implications for interaction and collaborative learning in computer conferences. International Journal of Educational Telecommunications, 1(2), 147–166.Google Scholar
  31. Guzdial, M., & Turns, J. (2000). Effective discussion through a computer-mediated anchored forum. The Journal of the Learning Sciences, 9(4), 437–469.CrossRefGoogle Scholar
  32. Häkkinen, P., & Järvelä, S. (2006). Sharing and constructing perspectives in web-based conferencing. Computers in Education, 47(4), 433–447.CrossRefGoogle Scholar
  33. Harasim, L. (2000). Shift happens online education as a new paradigm in learning. The Internet and Higher Education, 3(1–2), 41–61.CrossRefGoogle Scholar
  34. Hayes, A. F. (2012). PROCESS: A versatile computational tool for observed variable mediation, moderation, and conditional process modeling. Manuscript submitted for publication. http://www.afhayes.com/public/process2012.pdf.
  35. Hayes, A. F. (2009). Beyond baron and kenny: Statistical mediation analysis in the new millennium. Communication Monographs, 76(4), 408–420.CrossRefGoogle Scholar
  36. Hayes, A. F., & Krippendorff, K. (2007). Answering the call for a standard reliability measure for coding data. Communication Methods and Measures, 1(1), 77–89.CrossRefGoogle Scholar
  37. Herrmann, T., & Kienle, A. (2008). Context-oriented communication and the design of computer-supported discursive learning. International Journal of Computer-Supported Collaborative Learning, 3(3), 273–299.CrossRefGoogle Scholar
  38. Hewitt, J. (2003). How habitual online practices affect the development of asynchronous discussion threads. Journal of Educational Computing Research, 28(1), 31–45.CrossRefGoogle Scholar
  39. Hewitt, J. (2005). Toward an understanding of how threads die in asynchronous computer conferences. The Journal of the Learning Sciences, 14(4), 567–589.CrossRefGoogle Scholar
  40. Hiltz, S. R., Coppola, N., Rotter, N., & Turoff, M. (2000). Measuring the importance of collaborative learning for the effectiveness of ALN: A multi-measure, multi-method approach. Journal of Asynchronous Learning Networks, 4(2), 103–125.Google Scholar
  41. Jamaludin, A., Chee, Y., & Mei Lin Ho, C. (2009). Fostering argumentative knowledge construction through enactive role play in Second Life. Computers in Education, 53(2), 317–329.CrossRefGoogle Scholar
  42. Janssen, J., Erkens, G., & Kanselaar, G. (2007). Visualization of agreement and discussion processes during computer-supported collaborative learning. Computers in Human Behavior, 23(3), 1105–1125.CrossRefGoogle Scholar
  43. Janssen, J., Kirschner, F., Erkens, G., Kirschner, P. A., & Paas, F. (2010). Making the black box of collaborative learning transparent: Combining process-oriented and cognitive load approaches. Educational Psychology Review, 22(2), 139–154.CrossRefGoogle Scholar
  44. Jeong, A., & Frazier, S. (2008). How day of posting affects level of critical discourse in asynchronous discussions and computer supported collaborative argumentation. British Journal of Educational Technology, 39(5), 875–887.CrossRefGoogle Scholar
  45. Kapur, M. (2011). Temporality matters: Advancing a method for analyzing problem-solving processes in a computer-supported collaborative environment. International Journal of Computer-Supported Collaborative Learning, 6(1), 39–56.CrossRefGoogle Scholar
  46. Kapur, M., & Kinzer, C. K. (2007). Examining the effect of problem type in a synchronous computer-supported collaborative learning (CSCL) environment. Educational Technology Research and Development, 55(5), 439–459.CrossRefGoogle Scholar
  47. King, A. (2007). Scripting collaborative learning processes: A cognitive perspective. Scripting computer-supported collaborative learning, 13–37. doi:10.1007/978-0-387-36949-5_2.
  48. King, A. (2002). Structuring peer interaction to promote high-level cognitive processing. Theory into Practice, 41(1), 33–39.CrossRefGoogle Scholar
  49. Kirschner, P. A., Martens, R. L., & Strijbos, J. W. (2004). CSCL in higher education? A framework for designing multiple collaborative environments (pp. 3–30) Kluwer Academic Publishers.Google Scholar
  50. Kirschner, F., Paas, F., & Kirschner, P. A. (2009a). A cognitive load approach to collaborative learning: United brains for complex tasks. Educational Psychology Review, 21(1), 31–42.CrossRefGoogle Scholar
  51. Kirschner, F., Paas, F., & Kirschner, P. A. (2009b). Individual and group-based learning from complex cognitive tasks: Effects on retention and transfer efficiency. Computers in Human Behavior, 25(2), 306–314.CrossRefGoogle Scholar
  52. Kobbe, L., Weinberger, A., Dillenbourg, P., Harrer, A., Hämäläinen, R., Häkkinen, P., et al. (2007). Specifying computer-supported collaboration scripts. International Journal of Computer-Supported Collaborative Learning, 2(2), 211–224.CrossRefGoogle Scholar
  53. Mäkitalo, K., Salo, P., Häkkinen, P., & Järvelä, S. 2001. Analysing the mechanisms of common ground in collaborative web-based interaction. In P. Dillenbourg, A. Eurelings & K. Hakkarainen (Eds.), European perspectives on computer-supported collaborative learning. Proceedings of the First European Conference on Computer-Supported Collaborative Learning (pp. 445–453). Maastricht, The Netherlands: University of Maastricht.Google Scholar
  54. Mayer, R. H. (1999). Designing instruction for constructivist learning. In C. M. Reigeluth (Ed.), Instructional-design theories and models: A new paradigm of instructional theory, Volume II (pp. 141–160). Mahwah: Lawrence Erlbaum Associates.Google Scholar
  55. Meier, A., Spada, H., & Rummel, N. (2007). A rating scheme for assessing the quality of computer-supported collaboration processes. International Journal of Computer-Supported Collaborative Learning, 2(1), 63–86.CrossRefGoogle Scholar
  56. Morgan, R., Whorton, J., & Gunsalus, C. (2000). A comparison of short-term and long-term retention: Lecture combined with discussion versus cooperative learning. Journal of Instructional Psychology, 27(1), 53–58.Google Scholar
  57. Mühlpfordt, M., & Wessner, M. (2005). Explicit referencing in chat supports collaborative learning. In T. Kosschmann, D. Suthers, & T. W. Chan (Eds.), Computer supported collaborative learning: The next 10 years (pp. 460–469). Mahwah: Lawrance Erlbaum.Google Scholar
  58. Murphy, E., & Manzanares, R. M. A. (2005). Reading between the lines: Understanding the role of latent content in the analysis of online asynchronous discussions. International Journal of Instructional Technology and Distance Learning, 2(6), 23–32.Google Scholar
  59. Murray, B. (2000). Reinventing class discussion online. Monitor on Psychology, 31(4), 54–56.Google Scholar
  60. Onrubia, J., & Engel, A. (2009). Strategies for collaborative writing and phases of knowledge construction in CSCL environments. Computers in Education, 53(4), 1256–1265.CrossRefGoogle Scholar
  61. Paas, F. G. W. C., & Van Merriënboer, J. J. G. (1994). Variability of worked examples and transfer of geometrical problem-solving skills: A cognitive-load approach. Journal of Educational Psychology, 86(1), 122–133.CrossRefGoogle Scholar
  62. Pena-Shaff, J., & Nicholls, C. (2004). Analyzing student interactions and meaning construction in computer bulletin board discussions. Computers in Education, 42(3), 243–265.CrossRefGoogle Scholar
  63. Pfister, H. R., & Mühlpfordt, M. (2002). Supporting discourse in a synchronous learning environment: The learning protocol approach. In G. Stahl (Ed.), Proceedings of the Conference on Computer Supported Collaborative Learning (pp. 581–582). Hillsdale: Lawrence Erlbaum.Google Scholar
  64. Phillips, D. C. (1995). The good, the bad, and the ugly: The many faces of constructivism. Educational researcher, 24(7), 5–12.Google Scholar
  65. Poesio, M., & Muskens, R. (1997). The dynamics of discourse situations. In P. Dekker, M. Stokhof (Eds.), Proceedings of the 11th Amsterdam Colloquium, (pp. 247–252). University of Amsterdam, ILLC, December.Google Scholar
  66. Ponnusawmy, H., & Santally, M. I. (2008). Promoting (quality) participation in online forums: A study of the use of forums in two online modules at the University of Mauritius. International Journal of Instructional Technology & Distance Learning, 5(4), 1550–6908.Google Scholar
  67. Preacher, K. J., & Hayes, A. F. (2008). Asymptotic and resampling strategies for assessing and comparing indirect effects in multiple mediator models. Behavior Research Methods, 40(3), 879–891.CrossRefGoogle Scholar
  68. Preacher, K. J., & Kelley, K. (2011). Effect size measures for mediation models: Quantitative strategies for communicating indirect effects. Psychological methods, 16(2), 93.Google Scholar
  69. Riffe, D., Lacy, S., & Fico, F. (1998). Analyzing media messages: Using quantitative content analysis in research. Mahwah: Lawrence Erlbaum Associates.Google Scholar
  70. Roschelle, J. (1992). Learning by collaborating: Convergent conceptual change. The Journal of the Learning Sciences, 2(3), 235–276.CrossRefGoogle Scholar
  71. Rothkopf, E. Z. (1970). The concept of mathemagenic activities. Review of Educational Research, 40(3), 325–336.Google Scholar
  72. Schnotz, W., & Kürschner, C. (2007). A reconsideration of cognitive load theory. Educational Psychology Review, 19(4), 469–508.CrossRefGoogle Scholar
  73. Schwartz, D. L., & Lin, X. D. (2000). Computers, productive agency, and the effort after shared meaning. Journal of Computing in Higher Education, 12(2), 3–33.Google Scholar
  74. Shea, P., & Bidjerano, T. (2009). Community of inquiry as a theoretical framework to foster epistemic engagement and cognitive presence in online education. Computers in Education, 52(3), 543–553.CrossRefGoogle Scholar
  75. Simons, P. R. J. (2000). Towards a constructivistic theory of self-directed learning. Self-learning, 1–12. http://igitur-archive.library.uu.nl/ivlos/2005-0622-190617/5701.pdf.
  76. Slof, B., Erkens, G., Kirschner, P. A., Jaspers, J., & Janssen, J. (2010). Guiding students' online complex learning-task behavior through representational scripting. Computers in Human Behavior, 26(5), 927–939.CrossRefGoogle Scholar
  77. Slof, B., Erkens, G., Kirschner, P. A., Janssen, J., & Phielix, C. (2010). Fostering complex learning-task performance through scripting student use of computer supported representational tools. Computers in Education, 55(4), 1707–1720.CrossRefGoogle Scholar
  78. Stahl, G. (2000). A model of collaborative knowledge building. Proc. Fourth International Conference of the Learning Sciences (ICLS 2000) (pp. 70–77). Ann Arbor, MI.Google Scholar
  79. Strijbos, J. W., Martens, R. L., Prins, F. J., & Jochems, W. M. G. (2006). Content analysis: What are they talking about? Computers in Education, 46(1), 29–48.CrossRefGoogle Scholar
  80. Suthers, D. (2001). Collaborative representations: Supporting face-to-face and online knowledge-building discourse. Paper presented at the 34th Hawaii International Conference on System Sciences, Hawaii, USA.Google Scholar
  81. Suthers, D. (2006). Technology affordances for intersubjective meaning making: A research agenda for CSCL. International Journal of Computer-Supported Collaborative Learning, 1(3), 315–337.CrossRefGoogle Scholar
  82. Suthers, D., & Xu, J. (2002). Kukakuka: An online environment for artifact-centered discourse. Paper presented at the Eleventh World Wide Web Conference (WWW 2002). Honolulu, Hawaii, USA.Google Scholar
  83. Suthers, D., Girardeau, L., & Hundhausen, C. (2003). Deictic roles of external representations in face-to-face and online collaboration. In B. Wasson, S. Ludvigsen, & U. Hoppe (Eds.), Designing for change in networked learning environments, Proceedings of the International Conference on Computer Support for Collaborative Learning (pp. 173–182). Dordrecht: Kluwer Academic Publishers.Google Scholar
  84. Suthers, D., Vatrapu, R., Medina, R., Joseph, S., & Dwyer, N. (2008). Beyond threaded discussion: Representational guidance in asynchronous collaborative learning environments. Computers in Education, 50(4), 1103–1127.CrossRefGoogle Scholar
  85. Suthers, D. D., Dwyer, N., Medina, R., & Vatrapu, R. (2010). A framework for conceptualizing, representing, and analyzing distributed interaction. International Journal of Computer-Supported Collaborative Learning, 5(1), 5–42.CrossRefGoogle Scholar
  86. Sweller, J., & Chandler, P. (1994). Why some material is difficult to learn. Cognition and Instruction, 12(3), 185–233.CrossRefGoogle Scholar
  87. Takeda, T., & Suthers, D. (2002). Online workspaces for annotation and discussion of documents. Poster presented at the International Conference on Computers in Education, 2002.Google Scholar
  88. Timmers, C., & Veldkamp, B. (2011). Attention paid to feedback provided by a computer-based assessment for learning on information literacy. Computers in Education, 56(3), 923–930.CrossRefGoogle Scholar
  89. Toulmin, S. (1958). The uses of argument. Cambridge: Cambridge University Press.Google Scholar
  90. Uren, V., Buckingham, S., Li, G., Domingue, J., & Motta, E. (2003). Scholarly publishing and argument in hyperspace. Presented at the 12th International World Wide Web Conference. Budapest, Hungary.Google Scholar
  91. Van der Pol, J., Admiraal, W. F., & Simons, P. R. J. (2006a). The affordance of anchored discussion for the collaborative processing of academic texts. International Journal of Computer-Supported Collaborative Learning, 1(3), 339–357.CrossRefGoogle Scholar
  92. Van der Pol, J., Admiraal, W., & Simons, P. R. J. (2006b). Context enhancement for co-intentionality and co-reference in asynchronous CMC. AI & Society, 20(3), 301–313.CrossRefGoogle Scholar
  93. Van Merriënboer, J. P., Kirschner, P. A., & Loesbeth, K. (2003). Taking the load off a learners mind: Instructional design for complex learning. Educational Psychologist, 38(1), 5–13.CrossRefGoogle Scholar
  94. Veerman, A., Andriessen, J., & Kanselaar, G. (1999). Collaborative learning through computer-mediated argumentation. International Society of the Learning Sciences, 31(4), 77–86.Google Scholar
  95. Vygotsky, L. S. (1978). Mind and society: The development of higher psychological processes. Cambridge: Harvard University Press.Google Scholar
  96. Whitworth, B., Gallupe, B., & McQueen, R. (2000). A cognitive three-process model of computer-mediated group interaction. Group Decision and Negotiation, 9(5), 431–456.CrossRefGoogle Scholar
  97. Wolfe, J. (2008). Annotations and the collaborative digital library: Effects of an aligned annotation interface on student argumentation and reading strategies. International Journal of Computer-Supported Collaborative Learning, 3(2), 141–164.CrossRefGoogle Scholar
  98. Yang, Y., Newby, T., & Robert, B. (2008). Facilitating interactions through structured web-based bulletin boards: A quasi-experimental study on promoting learners’ critical thinking skills. Computers in Education, 50(4), 1572–1585.CrossRefGoogle Scholar

Copyright information

© International Society of the Learning Sciences, Inc. and Springer Science+Business Media New York 2013

Authors and Affiliations

  • Evren Eryilmaz
    • 1
  • Jakko van der Pol
    • 2
  • Terry Ryan
    • 1
  • Philip Martin Clark
    • 3
  • Justin Mary
    • 4
  1. 1.Social Learning Software Laboratory, School of Information Systems and TechnologyClaremont Graduate UniversityClaremontUSA
  2. 2.Centrum voor Nascholing (CNA)Hogeschool van Amsterdam - Theo ThijssenhuisAmsterdamThe Netherlands
  3. 3.Faculty of PharmacyYeditepe UniversityIstanbulTurkey
  4. 4.School of Behavioral and Organizational SciencesClaremont Graduate UniversityClaremontUSA

Personalised recommendations