Abstract
Online collaborative discussion (OCD) focuses on promoting individual knowledge inquiry and group knowledge construction through active peer interactions and communications. In practice, it is necessary to explore how different modes of OCD come into play, in which student engagement can function as an evaluating indicator. To identify student engagement in OCD, prior research has identified and categorized various types of student roles. However, although students usually change their engagement during the learning process and across learning occasions, most existing research focuses on examining unchanging student roles or developing roles in similar collaborative activities, which might overlook the probable role transitions brought by engagement changes. To fill this gap, this research proposes an integrated probabilistic clustering approach to detect student roles, role transitions, and fine-grained attributes of transitions across the asynchronous and synchronous OCD modes. The results demonstrate four roles (Knowledge Constructor, Task Follower, Isolated Explorer, and Lurker), four transition categories (Maintenance of inactive participant, Transferring to inactive participant, Maintenance of active participant, and Transferring to active participant), and the code co-occurrence structures of four transition categories. This research deepens the understanding of the complexity of student engagement in online collaborative discussions and offers both analytical and practical implications for improving student engagement.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Availability of data and materials
The datasets are available from the corresponding author on reasonable request.
Abbreviations
- AOD:
-
Asynchronous online discussion
- ENA:
-
Epistemic network analysis
- LPA:
-
Latent profile analysis
- LTA:
-
Latent transition analysis
- OCD:
-
Online collaborative discussion
- SOD:
-
Synchronous online discussion
References
Abarda, A., Dakkon, M., Azhari, M., Zaaloul, A., & Khabouze, M. (2020). Latent transition analysis (LTA): A method for identifying differences in longitudinal change among unobserved groups. Procedia Computer Science, 170, 1116–1121. https://doi.org/10.1016/j.procs.2020.03.059
Akaike, H. (1987). Factor analysis and AIC. Psychometrika, 52(3), 317–332. https://doi.org/10.1007/BF02294359
Appleton, J. J., Christenson, S. L., & Furlong, M. J. (2008). Student engagement with school: Critical conceptual and methodological issues of the construct. Psychology in the Schools, 45(5), 369–386. https://doi.org/10.1002/pits.20303
Barthakur, A., Kovanovic, V., Joksimovic, S., Siemens, G., Richey, M., & Dawson, S. (2021). Assessing program-level learning strategies in MOOCs. Computers in Human Behavior, 117, 106674. https://doi.org/10.1016/j.chb.2020.106674
Bauer, J. (2022). A primer to latent profile and latent class analysis. In M. Goller, E. Kyndt, S. Paloniemi, & C. Damşa (Eds.), Methods for researching professional learning and development (Vol. 33, pp. 243–268). Springer International Publishing. https://doi.org/10.1007/978-3-031-08518-5_11
Boroujeni, M. S., Hecking, T., Hoppe, H. U., & Dillenbourg, P. (2017). Dynamics of MOOC discussion forums. In Proceedings of the seventh international learning analytics & knowledge conference, pp. 128–137. https://doi.org/10.1145/3027385.3027391
Branon, R. F., & Essex, C. (2001). Synchronous and asynchronous communication tools in distance education: A survey of instructors. TechTrends, 45(1), 36–36. https://doi.org/10.1007/BF02763377
Broadbent, J., & Fuller-Tyszkiewicz, M. (2018). Profiles in self-regulated learning and their correlates for online and blended learning students. Educational Technology Research and Development, 66(6), 1435–1455. https://doi.org/10.1007/s11423-018-9595-9
Celeux, G., & Soromenho, G. (1996). An entropy criterion for assessing the number of clusters in a mixture model. Journal of Classification, 13(2), 195–212. https://doi.org/10.1007/BF01246098
Chen, C.-M., Li, M.-C., Chang, W.-C., & Chen, X.-X. (2021). Developing a topic analysis instant feedback system to facilitate asynchronous online discussion effectiveness. Computers & Education, 163, 104095. https://doi.org/10.1016/j.compedu.2020.104095
Chen, Y., Chen, N.-S., & Tsai, C.-C. (2009). The use of online synchronous discussion for web-based professional development for teachers. Computers & Education, 53(4), 1155–1166. https://doi.org/10.1016/j.compedu.2009.05.026
Collins, L. M., & Lanza, S. T. (2009). Repeated-measures latent class analysis and latent transition analysis. In L. M. Collins & S. T. Lanza (Eds.), Latent class and latent transition analysis: With applications in the social, behavioral, and health sciences. John Wiley & Sons. https://doi.org/10.1002/9780470567333.ch7
Csanadi, A., Eagan, B., Kollar, I., Shaffer, D. W., & Fischer, F. (2018). When coding-and-counting is not enough: Using epistemic network analysis (ENA) to analyze verbal data in CSCL research. International Journal of Computer-Supported Collaborative Learning, 13(4), 419–438. https://doi.org/10.1007/s11412-018-9292-z
Damşa, C. I. (2014). The multi-layered nature of small-group learning: Productive interactions in object-oriented collaboration. International Journal of Computer-Supported Collaborative Learning, 9(3), 247–281. https://doi.org/10.1007/s11412-014-9193-8
de Lima, D. P. R., Gerosa, M. A., Conte, T. U., & Netto, J. F. M. (2019). What to expect, and how to improve online discussion forums: The instructors’ perspective. Journal of Internet Services and Applications, 10(1), 22. https://doi.org/10.1186/s13174-019-0120-0
De Wever, B., & Strijbos, J.-W. (2021). Roles for structuring groups for collaboration. In U. Cress, C. Rose, A. F. Wise, & J. Oshima (Eds.), International handbook of computer-supported collaborative learning (pp. 315–331). Springer International Publishing. https://doi.org/10.1007/978-3-030-65291-3_17
Delaney, D., Kummer, T.-F., & Singh, K. (2019). Evaluating the impact of online discussion boards on student engagement with group work: Evaluating impact of online discussion boards. British Journal of Educational Technology, 50(2), 902–920. https://doi.org/10.1111/bjet.12614
Dowell, N. M. M., Nixon, T. M., & Graesser, A. C. (2019). Group communication analysis: A computational linguistics approach for detecting sociocognitive roles in multiparty interactions. Behavior Research Methods, 51(3), 1007–1041. https://doi.org/10.3758/s13428-018-1102-z
Driskell, T., Driskell, J. E., Burke, C. S., & Salas, E. (2017). Team roles: A review and integration. Small Group Research, 48(4), 482–511. https://doi.org/10.1177/1046496417711529
Du, X., Zhang, L., Hung, J.-L., Li, H., Tang, H., & Dai, M. (2022). Analyzing the effects of instructional strategies on students’ on-task status from aspects of their learning behaviors and cognitive factors. Journal of Computing in Higher Education. https://doi.org/10.1007/s12528-022-09345-y
Fredricks, J. A., Blumenfeld, P. C., & Paris, A. H. (2016). School engagement: Potential of the concept, state of the evidence. Review of Educational Research, 74(1), 59–109. https://doi.org/10.3102/003465430740010
Garcia, C., & Badia, A. (2020). Posting messages and acquiring knowledge in collaborative online tasks. Technology, Pedagogy and Education, 29(3), 377–388. https://doi.org/10.1080/1475939X.2020.1778076
Garrison, D. R., Anderson, T., & Archer, W. (1999). Critical inquiry in a text-based environment: Computer conferencing in higher education. The Internet and Higher Education, 2(2–3), 87–105. https://doi.org/10.1016/S1096-7516(00)00016-6
Gijlers, H., Saab, N., Van Joolingen, W. R., De Jong, T., & Van Hout-Wolters, B. H. A. M. (2009). Interaction between tool and talk: How instruction and tools support consensus building in collaborative inquiry-learning environments. Journal of Computer Assisted Learning, 25(3), 252–267. https://doi.org/10.1111/j.1365-2729.2008.00302.x
Grabowski, B. L. (2011). Framework for asynchronous discussion design decisions: Applied principles from special issue authors. Journal of Computing in Higher Education, 23(2–3), 187–199. https://doi.org/10.1007/s12528-011-9048-5
Hew, K. F., Cheung, W. S., & Ng, C. S. L. (2010). Student contribution in asynchronous online discussion: A review of the research and empirical exploration. Instructional Science, 38(6), 571–606. https://doi.org/10.1007/s11251-008-9087-0
Hickendorff, M., Edelsbrunner, P. A., McMullen, J., Schneider, M., & Trezise, K. (2018). Informative tools for characterizing individual differences in learning: Latent class, latent profile, and latent transition analysis. Learning and Individual Differences, 66, 4–15. https://doi.org/10.1016/j.lindif.2017.11.001
Hou, H.-T., & Wu, S.-Y. (2011). Analyzing the social knowledge construction behavioral patterns of an online synchronous collaborative discussion instructional activity using an instant messaging tool: A case study. Computers & Education, 57(2), 1459–1468. https://doi.org/10.1016/j.compedu.2011.02.012
Huang, B., Hwang, G.-J., Hew, K. F., & Warning, P. (2019). Effects of gamification on students’ online interactive patterns and peer-feedback. Distance Education, 40(3), 350–379. https://doi.org/10.1080/01587919.2019.1632168
Järvelä, S., & Rose, C. P. (2022). Novel CSCL design processes prompted by exploration of role-playing and emotions. International Journal of Computer-Supported Collaborative Learning, 17(4), 457–461. https://doi.org/10.1007/s11412-022-09383-9
Jeong, S., & Feldon, D. F. (2023). Changes in self-regulated learning profiles during an undergraduate peer-based intervention: A latent profile transition analysis. Learning and Instruction, 83, 101710. https://doi.org/10.1016/j.learninstruc.2022.101710
Jiao, P., Ouyang, F., Zhang, Q., & Alavi, A. H. (2022). Artificial intelligence-enabled prediction model of student academic performance in online engineering education. Artificial Intelligence Review, 55, 6321–6344. https://doi.org/10.1007/s10462-022-10155-y
Katz, L. G. (1993). Dispositions as educational goals. ERIC Digest. ERIC. https://eric.ed.gov/?id=ED363454.
Kemp, N., & Grieve, R. (2014). Face-to-face or face-to-screen? Undergraduates’ opinions and test performance in classroom vs. online learning. Frontiers in Psychology. https://doi.org/10.3389/fpsyg.2014.01278
Kim, M. K., & Ketenci, T. (2019). Learner participation profiles in an asynchronous online collaboration context. The Internet and Higher Education, 41, 62–76. https://doi.org/10.1016/j.iheduc.2019.02.002
Lau, C., Sinclair, J., Taub, M., Azevedo, R., & Jang, E. E. (2017). Transitioning self-regulated learning profiles in hypermedia-learning environments. In Wise, A., Winne, P. H., Lynch, G. (Eds.), Proceedings of the seventh international learning analytics & knowledge conference (pp. 198–202). Association for Computing Machinery. https://doi.org/10.1145/3027385.3027443
Liu, S., Kang, L., Liu, Z., Zhao, L., Yang, Z., & Su, Z. (2022). Exploring the relationships between students’ network characteristics, discussion topics and learning outcomes in a course discussion forum. Journal of Computing in Higher Education. https://doi.org/10.1007/s12528-022-09335-0
Liu, T., & Israel, M. (2022). Uncovering students’ problem-solving processes in game-based learning environments. Computers & Education, 182, 104462. https://doi.org/10.1016/j.compedu.2022.104462
Luo, H., Chen, Y., Chen, T., Koszalka, T. A., & Feng, Q. (2023). Impact of role assignment and group size on asynchronous online discussion: An experimental study. Computers & Education, 192, 104658. https://doi.org/10.1016/j.compedu.2022.104658
Malmberg, J., Järvelä, S., & Järvenoja, H. (2017). Capturing temporal and sequential patterns of self-, co-, and socially shared regulation in the context of collaborative learning. Contemporary Educational Psychology, 49, 160–174. https://doi.org/10.1016/j.cedpsych.2017.01.009
McLachlan, G. J. (1987). On Bootstrapping the likelihood ratio test stastistic for the number of components in a normal mixture. Journal of the Royal Statistical Society. Series C (Applied Statistics), 36(3), 318–324. https://doi.org/10.2307/2347790
Medina, E., Vega, D., Meseguer, R., Medina, H., Ochoa, S. F., & Magnani, M. (2016). Using indirect blockmodeling for monitoring students roles in collaborative learning networks. In Shen, W., Liu, X., Yang, C., Barthès, J. P., Luo, J., Chen, L., Yong, J. (Eds.), 2016 IEEE 20th international conference on computer supported cooperative work in design (CSCWD). IEEE, pp. 164–169. https://doi.org/10.1109/CSCWD.2016.7565982
Medina, R., & Stahl, G. (2021). Analysis of group practices. In U. Cress, C. Rosé, A. F. Wise, & J. Oshima (Eds.), International handbook of computer-supported collaborative learning (pp. 199–218). Springer International Publishing. https://doi.org/10.1007/978-3-030-65291-3_11
Nylund, K. L., Asparouhov, T., & Muthén, B. O. (2007). Deciding on the number of classes in latent class analysis and growth mixture modeling: A Monte Carlo simulation study. Structural Equation Modeling, 14(4), 535–569. https://doi.org/10.1080/10705510701575396
Oh, E. G., Huang, W.-H.D., HedayatiMehdiabadi, A., & Ju, B. (2018). Facilitating critical thinking in asynchronous online discussion: Comparison between peer- and instructor-redirection. Journal of Computing in Higher Education, 30(3), 489–509. https://doi.org/10.1007/s12528-018-9180-6
Ouyang, F., & Chang, Y. (2019). The relationships between social participatory roles and cognitive engagement levels in online discussions. British Journal of Educational Technology, 50(3), 1396–1414. https://doi.org/10.1111/bjet.12647
Ouyang, F., Chen, Z., Cheng, M., Tang, Z., & Su, C.-Y. (2021). Exploring the effect of three scaffoldings on the collaborative problem-solving processes in China’s higher education. International Journal of Educational Technology in Higher Education, 18(1), 1–22. https://doi.org/10.1186/s41239-021-00273-y
Ouyang, F., Li, X., Sun, D., Jiao, P., & Yao, J. (2020). Learners’ discussion patterns, perceptions, and preferences in a Chinese Massive Open Online Course (MOOC). The International Review of Research in Open and Distributed Learning, 21(3), 264–284. https://doi.org/10.19173/irrodl.v21i3.4771
Ouyang, F., Wu, M., Zhang, L., Xu, W., Zheng, L., & Cukurova, M. (2023). Making strides towards AI-supported regulation of learning in collaborative knowledge construction. Computers in Human Behavior, 142, 107650. https://doi.org/10.1016/j.chb.2023.107650
Ouyang, F., & Xu, W. (2022). The effects of three instructor participatory roles on a small group’s collaborative concept mapping. Journal of Educational Computing Research, 60(4), 930–959. https://doi.org/10.1177/07356331211057283
Oztok, M., Zingaro, D., Brett, C., & Hewitt, J. (2013). Exploring asynchronous and synchronous tool use in online courses. Computers & Education, 60(1), 87–94. https://doi.org/10.1016/j.compedu.2012.08.007
Palloff, R. M., & Pratt, K. (2007). Building online learning communities: Effective strategies for the virtual classroom (2nd ed.). Jossey-Bass.
Park, Y., Yu, J. H., & Jo, I.-H. (2016). Clustering blended learning courses by online behavior data: A case study in a Korean higher education institute. The Internet and Higher Education, 29, 1–11. https://doi.org/10.1016/j.iheduc.2015.11.001
Raković, M., Marzouk, Z., Liaqat, A., Winne, P. H., & Nesbit, J. C. (2020). Fine grained analysis of students’ online discussion posts. Computers & Education, 157, 103982. https://doi.org/10.1016/j.compedu.2020.103982
Rovai, A. P. (2007). Facilitating online discussions effectively. The Internet and Higher Education, 10(1), 77–88. https://doi.org/10.1016/j.iheduc.2006.10.001
Ryoo, J. H., Wang, C., Swearer, S. M., Hull, M., & Shi, D. (2018). Longitudinal model building using latent transition analysis: An example using school bullying data. Frontiers in Psychology, 9, 675. https://doi.org/10.3389/fpsyg.2018.00675
Sandoval, W. A., & Reiser, B. J. (2004). Explanation-driven inquiry: Integrating conceptual and epistemic scaffolds for scientific inquiry. Science Education, 88(3), 345–372. https://doi.org/10.1002/sce.10130
Saqr, M., & López-Pernas, S. (2021). The longitudinal trajectories of online engagement over a full program. Computers & Education, 175, 104325. https://doi.org/10.1016/j.compedu.2021.104325
Saqr, M., & López-Pernas, S. (2022). How CSCL roles emerge, persist, transition, and evolve over time: A four-year longitudinal study. Computers & Education, 189, 104581. https://doi.org/10.1016/j.compedu.2022.104581
Saqr, M., & Viberg, O. (2020). Using diffusion network analytics to examine and support knowledge construction in CSCL settings. In C. Alario-Hoyos, M. J. Rodríguez-Triana, M. Scheffel, I. Arnedillo-Sánchez, & S. M. Dennerlein (Eds.), Addressing global challenges and quality education (Vol. 12315, pp. 158–172). Springer International Publishing. https://doi.org/10.1007/978-3-030-57717-9_12
Scardamalia, M., & Bereiter, C. (2006). Knowledge building: Theory, pedagogy, and technology. In Cambridge handbook of the learning sciences, pp. 97–115. https://doi.org/10.1017/CBO9781139519526.025
Schellens, T., Van Keer, H., De Wever, B., & Valcke, M. (2007). Scripting by assigning roles: Does it improve knowledge construction in asynchronous discussion groups? International Journal of Computer-Supported Collaborative Learning, 2(2–3), 225–246. https://doi.org/10.1007/s11412-007-9016-2
Schwarz, G. (1978). Estimating the dimension of a model. Annals of Statistics. https://doi.org/10.1214/aos/1176344136
Shaffer, D. W., Collier, W., & Ruis, A. R. (2016). A tutorial on epistemic network analysis: Analyzing the structure of connections in cognitive, social, and interaction data. Journal of Learning Analytics, 3(3), 9–45. https://doi.org/10.18608/jla.2016.33.3
Sorgente, A., Lanz, M., Serido, J., Tagliabue, S., & Shim, S. (2019). Latent transition analysis: Guidelines and an application to emerging adults’ social development. Testing, Psychometrics, Methodology in Applied Psychology, 26(1), 39–72. https://doi.org/10.4473/TPM26.1.3
Spurk, D., Hirschi, A., Wang, M., Valero, D., & Kauffeld, S. (2020). Latent profile analysis: A review and “how to” guide of its application within vocational behavior research. Journal of Vocational Behavior, 120, 103445. https://doi.org/10.1016/j.jvb.2020.103445
Strijbos, J.-W., & De Laat, M. F. (2010). Developing the role concept for computer-supported collaborative learning: An explorative synthesis. Computers in Human Behavior, 26(4), 495–505. https://doi.org/10.1016/j.chb.2009.08.014
Susnjak, T., Ramaswami, G. S., & Mathrani, A. (2022). Learning analytics dashboard: A tool for providing actionable insights to learners. International Journal of Educational Technology in Higher Education, 19(1), 12. https://doi.org/10.1186/s41239-021-00313-7
Vygotsky, L. S. (1978). Mind in society: The development of higher psychological processes. Harvard University Press.
Wang, J., & Wang, Y. (2021). Compare synchronous and asynchronous online instruction for science teacher preparation. Journal of Science Teacher Education, 32(3), 265–285. https://doi.org/10.1080/1046560X.2020.1817652
Wang, Q., & Woo, H. L. (2007). Comparing asynchronous online discussions and face-to-face discussions in a classroom setting. British Journal of Educational Technology, 38(2), 272–286. https://doi.org/10.1111/j.1467-8535.2006.00621.x
Weinberger, A., Stegmann, K., & Fischer, F. (2010). Learning to argue online: Scripted groups surpass individuals (unscripted groups do not). Computers in Human Behavior, 26(4), 506–515. https://doi.org/10.1016/j.chb.2009.08.007
Winne, P. H., & Hadwin, A. F. (1998). Studying as self-regulated learning. In D. J. Hacker, J. Dunlosky, & A. C. Graesser (Eds.), Metacognition in educational theory and practice (pp. 277–304). Erlbau.
Wood, D., Bruner, J. S., & Ross, G. (1976). The role of tutoring in problem solving. Journal of Child Psychology and Psychiatry, 17(2), 89–100. https://doi.org/10.1111/j.1469-7610.1976.tb00381.x
Xie, Y., Huang, Y., Luo, W., Bai, Y., Qiu, Y., & Ouyang, Z. (2023). Design and effects of the teacher-student interaction model in the online learning spaces. Journal of Computing in Higher Education, 35(1), 69–90. https://doi.org/10.1007/s12528-022-09348-9
Xu, W., Wu, Y., & Ouyang, F. (2023). Multimodal learning analytics of collaborative patterns during pair programming in higher education. International Journal of Educational Technology in Higher Education, 20(1), 8. https://doi.org/10.1186/s41239-022-00377-z
Yilmaz, R., & Karaoglan Yilmaz, F. G. (2019). Assigned roles as a structuring tool in online discussion groups: Comparison of transactional distance and knowledge sharing behaviors. Journal of Educational Computing Research, 57(5), 1303–1325. https://doi.org/10.1177/0735633118786855
Zhao, H., Sullivan, K. P. H., & Mellenius, I. (2014). Participation, interaction and social presence: An exploratory study of collaboration in online peer review groups. British Journal of Educational Technology, 45(5), 807–819. https://doi.org/10.1111/bjet.12094
Acknowledgements
We appreciate the students participated in this research. We thank Liyin Zhang for her preliminary data analysis work.
Funding
The authors acknowledge the financial support from National Natural Science Foundation of China (62177041), the Fundamental Research Funds for the Central Universities (Year 2023), Zhejiang University, China as well as Zhejiang University graduate education research project (20220310).
Author information
Authors and Affiliations
Contributions
MW collected, analyzed data and wrote the manuscript. FO conceptualized the research idea, supervised the research project, and modified the manuscript. All of the authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no competing interests.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Wu, M., Ouyang, F. Using an integrated probabilistic clustering approach to detect student engagement across asynchronous and synchronous online discussions. J Comput High Educ (2024). https://doi.org/10.1007/s12528-023-09394-x
Accepted:
Published:
DOI: https://doi.org/10.1007/s12528-023-09394-x