Towards Combined Network and Text Analytics of Student Discourse in Online Discussions

  • Rafael FerreiraEmail author
  • Vitomir Kovanović
  • Dragan Gašević
  • Vitor Rolim
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10947)


This paper presents a novel method for the evaluation of students’ use of asynchronous discussions in online learning environments. In particular, the paper shows how students’ cognitive development across different course topics can be examined using the combination of natural language processing and graph-based analysis techniques. Drawing on the theoretical foundation of the community of inquiry model, we show how topic modeling and epistemic network analysis can provide qualitatively new insight into students’ development of critical and deep thinking skills. We also show how the same method can be used to investigate the effectiveness of instructional interventions and its effect on student learning. The results of this study and its practical implications are further discussed.


Community of inquiry model Epistemic network analysis Content analysis Instructional interventions Online discussions 


  1. 1.
    Blei, D.M.: Topic modeling in digital humanities. J. Digit. Humanit. 2(1) (2012). Special IssueGoogle Scholar
  2. 2.
    Akyol, Z., Garrison, D.R.: Assessing metacognition in an online community of inquiry. Internet High. Educ. 14(3), 183–190 (2011).
  3. 3.
    Anderson, T., Dron, J.: Three generations of distance education pedagogy. Int. Rev. Res. Open Distance Learn. 12(3), 80–97 (2010).
  4. 4.
    Anderson, T., Rourke, L., Garrison, D.R., Archer, W.: Assessing teaching presence in a computer conferencing context. J. Asynchronous Learn. Netw. 5, 1–17 (2001)Google Scholar
  5. 5.
    Arbaugh, J., Cleveland-Innes, M., Diaz, S.R., Garrison, D.R., Ice, P., Richardson, J.C., Swan, K.P.: Developing a community of inquiry instrument: testing a measure of the community of inquiry framework using a multi-institutional sample. Internet High. Educ. 11(3–4), 133–136 (2008).
  6. 6.
    Azevedo, R., Moos, D.C., Greene, J.A., Winters, F.I., Cromley, J.G.: Why is externally-facilitated regulated learning more effective than self-regulated learning with hypermedia? Educ. Technol. Res. Dev. 56(1), 45–72 (2008).
  7. 7.
    Blei, D.M., Ng, A.Y., Jordan, M.I.: Latent dirichlet allocation. J. Mach. Learn. Res. 3, 993–1022 (2003).
  8. 8.
    Cohen, J.: The analysis of variance. In: Statistical Power Analysis for the Behavioral Sciences, pp. 273–406. L. Erlbaum Associates, Hillsdale (1988)Google Scholar
  9. 9.
    Dewey, J.: My pedagogical creed. Sch. J. 54(3), 77–80 (1897)Google Scholar
  10. 10.
    Feldman, R., Sanger, J.: The Text Mining Handbook: Advanced Approaches in Analyzing Unstructured Data. Cambridge University Press, Cambridge (2007)Google Scholar
  11. 11.
    Fischer, F., Kollar, I., Mandl, H., Haake, J.M.: Scripting Computer-Supported Collaborative Learning: Cognitive. Computational and Educational Perspectives. Springer Science & Business Media, New York (2007).
  12. 12.
    Garrison, D.R.: Cognitive presence for effective asynchronous online learning: the role of reflective inquiry, self-direction and metacognition. Elem. Qual. Online Educ. Pract. Dir. 4(1), 47–58 (2003)Google Scholar
  13. 13.
    Garrison, D.R., Anderson, T., Archer, W.: Critical inquiry in a text-based environment: computer conferencing in higher education. Internet High. Educ. 2(2–3), 87–105 (1999).
  14. 14.
    Garrison, D.R., Anderson, T., Archer, W.: Critical thinking, cognitive presence, and computer conferencing in distance education. Am. J. Distance Educ. 15(1), 7–23 (2001).
  15. 15.
    Garrison, D.R., Anderson, T., Archer, W.: The first decade of the community of inquiry framework: a retrospective. Internet High. Educ. 13(1–2), 5–9 (2010).
  16. 16.
    Gašević, D., Adesope, O., Joksimović, S., Kovanović, V.: Externally-facilitated regulation scaffolding and role assignment to develop cognitive presence in asynchronous online discussions. Internet High. Educ. 24, 53–65 (2015).
  17. 17.
    Heo, H., Lim, K.Y., Kim, Y.: Exploratory study on the patterns of online interaction and knowledge co-construction in project-based learning. Comput. Educ. 55(3), 1383–1392 (2010).
  18. 18.
    Kovanović, V., Joksimović, S., Gašević, D., Hatala, M.: Automated cognitive presence detection in online discussion transcripts. In: Proceedings of the Workshops at the LAK 2014 Conference Co-located with 4th International Conference on Learning Analytics and Knowledge (LAK 2014), Indianapolis, IN (2014).
  19. 19.
    Kovanović, V., Joksimović, S., Gašević, D., Hatala, M., Siemens, G.: Content analytics: the definition, scope, and an overview of published research. In: Lang, C., Siemens, G., Wise, A., Gašević, D. (eds.) Handbook of Learning Analytics and Educational Data Mining, pp. 77–92. SoLAR, Edmonton (2017).
  20. 20.
    Kovanović, V., Joksimović, S., Gašević, D., Siemens, G., Hatala, M.: What public media reveals about MOOCs: a systematic analysis of news reports. Br. J. Educ. Technol. 46(3), 510–527 (2015).
  21. 21.
    Kovanović, V., Joksimović, S., Poquet, O., Hennis, T., Čukić, I., de Vries, P., Hatala, M., Dawson, S., Siemens, G., Gašević, D.: Exploring communities of inquiry in massive open online courses. Comput. Educ. 119, 44–58 (2018).
  22. 22.
    Kovanović, V., Joksimović, S., Waters, Z., Gašević, D., Kitto, K., Hatala, M., Siemens, G.: Towards automated content analysis of discussion transcripts: a cognitive presence case. In: Proceedings of the Sixth International Conference on Learning Analytics & Knowledge (LAK 2016), pp. 15–24. ACM, New York (2016).
  23. 23.
    Lipman, M.: Thinking in Education. Cambridge University Press, Cambridge (1991)Google Scholar
  24. 24.
    Manning, C.D., Schütze, H.: Foundations of Statistical Natural Language Processing, vol. 999. MIT Press, Cambridge (1999)zbMATHGoogle Scholar
  25. 25.
    Nash, P., Shaffer, D.W.: Mentor modeling: the internalization of modeled professional thinking in an epistemic game. J. Comput. Assist. Learn. 27(2), 173–189 (2011).
  26. 26.
    Ramage, D., Rosen, E., Chuang, J., Manning, C.D., McFarland, D.A.: Topic modeling for the social sciences. In: NIPS 2009 Workshop on Applications for Topic Models: Text and Beyond, Whistler, Canada (2009)Google Scholar
  27. 27.
    Reich, J., Tingley, D., Leder-Luis, J., Roberts, M.E., Stewart, B.: Computer-assisted reading and discovery for student generated text in massive open online courses. J. Learn. Analytics 2(1), 156–184 (2014).
  28. 28.
    Rourke, L., Anderson, T., Garrison, D.R., Archer, W.: Assessing social presence in asynchronous text-based computer conferencing. J. Distance Educ. 14(2), 50–71 (1999).
  29. 29.
    Shaffer, D.W.: Epistemic frames for epistemic games. Comput. Educ. 46(3), 223–234 (2006).
  30. 30.
    Shaffer, D.W.: Epistemic frames and islands of expertise: learning from infusion experiences. In: Proceedings of the 6th International Conference on Learning Sciences, ICLS 2004, pp. 473–480. International Society of the Learning Sciences, Santa Monica (2004).
  31. 31.
    Shaffer, D.W., Collier, W., Ruis, A.R.: A tutorial on epistemic network analysis: analyzing the structure of connections in cognitive, social, and interaction data. J. Learn. Analytics 3(3), 9–45 (2016).
  32. 32.
    Shaffer, D.W., Hatfield, D., Svarovsky, G.N., Nash, P., Nulty, A., Bagley, E., Frank, K., Rupp, A.A., Mislevy, R.: Epistemic network analysis: a prototype for 21st-century assessment of learning. Int. J. Learn. Media 1(2), 33–53 (2009).
  33. 33.
    Stahl, G., Koschmann, T., Suthers, D., Sawyer, R.K.: Computer-supported collaborative learning: a historical perspective. In: The Cambridge Handbook of the Learning Sciences, pp. 409–426. Cambridge University Press, Cambridge, New York (2006)Google Scholar
  34. 34.
    Wallach, H.M.: Topic modeling: beyond bag-of-words. In: Proceedings of the 23rd International Conference on Machine Learning, ICML 2006, pp. 977–984. ACM, New York (2006).
  35. 35.
    Waters, Z., Kovanović, V., Kitto, K., Gašević, D.: Structure matters: adoption of structured classification approach in the context of cognitive presence classification. In: Zuccon, G., Geva, S., Joho, H., Scholer, F., Sun, A., Zhang, P. (eds.) AIRS 2015. LNCS, vol. 9460, pp. 227–238. Springer, Cham (2015). Scholar
  36. 36.
    Winne, P.H., Hadwin, A.F.: Studying as self-regulated learning. In: Hacker, D.J., Dunlosky, J., Graesser, A.C. (eds.) Metacognition in educational theory and practice. The Educational Psychology Series, pp. 277–304. Lawrence Erlbaum Associates Publishers, Mahwah (1998)Google Scholar
  37. 37.
    Yang, D., Wen, M., Rose, C.: Towards identifying the resolvability of threads in MOOCs. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 21–31 (2014).

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Rafael Ferreira
    • 1
    • 2
    Email author
  • Vitomir Kovanović
    • 3
  • Dragan Gašević
    • 2
    • 4
  • Vitor Rolim
    • 1
  1. 1.Federal Rural University of PernambucoRecifeBrazil
  2. 2.University of EdinburghEdinburghUK
  3. 3.University of South AustraliaAdelaideAustralia
  4. 4.Monash UniversityClaytonAustralia

Personalised recommendations