Abstract
Social annotation plays a crucial role in nurturing and sustaining a collaborative reading community, offering the potential to enhance students’ motivation and performance within socially supportive learning environments. Nonetheless, research on the dynamic changes in student engagement in social annotation remains limited. This study aims to unveil the temporal changes in students’ behavioral, cognitive, affective, and social engagement within the context of social annotation. In addition, it examines the disparities in social annotation behaviors between students with different engagement profiles. Using a multivariate time series clustering approach to analyze a dataset comprising 91 undergraduate students interacting with 29 reading materials, this study identified two distinct engagement profiles. Both profiles revealed a decline in behavioral engagement as the number of reading activities increased. However, students’ cognitive, affective, and social engagement levels in social annotation remained relatively stable across these activities. Subsequent analyses showed that students exhibiting a declining engagement profile displayed higher levels of aggregated behavioral, cognitive, and social engagement. Furthermore, they posted a significantly greater number of annotations and responses to peers’ annotations compared to students characterized by a low engagement profile. Potential explanations and pedagogical implications of these findings were discussed.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data availability
The datasets generated during and/or analyzed during the current study are available from the corresponding author upon reasonable request.
References
Adams, B., & Wilson, N. (2020). Building community in asynchronous online higher education courses through collaborative annotation. Journal of Educational Technology Systems, 49(2), 250–261. https://doi.org/10.1177/0047239520946422.
Aghabozorgi, S., Shirkhorshidi, A. S., & Wah, T. Y. (2015). Time-series clustering–A decade review. Information Systems, 53, 16–38. https://doi.org/10.1016/j.is.2015.04.007.
Anderson, T., Rourke, L., Garrison, D. R., & Archer, W. (2001). Assessing teaching presence in a computer conferencing context. Journal of Asynchronous Learning Networks, 5(2), 1–17.
Berndt, D. J., & Clifford, J. (1994, July). Using dynamic time warping to find patterns in time series. In KDD workshop (Vol. 10, No. 16, pp. 359–370).
Boyd, R. L., Ashokkumar, A., Seraj, S., & Pennebaker, J. W. (2022). The development and psychometric properties of LIWC-22. University of Texas at Austin. https://www.liwc.app.
Brown, M., & Croft, B. (2020). Social annotation and an inclusive praxis for open pedagogy in the college classroom. Journal of Interactive Media in Education, 2020(1), 1–8. https://doi.org/10.5334/jime.561.
Cattuto, C., Barrat, A., Baldassarri, A., Schehr, G., & Loreto, V. (2009). Collective dynamics of social annotation. Proceedings of the National Academy of Sciences, 106(26), 10511–10515. https://doi.org/10.1073/pnas.0901136106.
Cecchinato, G., & Foschi, L. C. (2020). Perusall: University learning-teaching innovation employing social annotation and machine learning. QWERTY, 15(2), 45–67. https://doi.org/10.30557/QW000030.
Chang, C. K., & Hsu, C. K. (2011). A mobile-assisted synchronously collaborative translation–annotation system for English as a foreign language (EFL) reading comprehension. Computer Assisted Language Learning, 24(2), 155–180. https://doi.org/10.1080/09588221.2010.536952.
Chen, C. M., & Chen, F. Y. (2014). Enhancing digital reading performance with a collaborative reading annotation system. Computers & Education, 77, 67–81. https://doi.org/10.1016/j.compedu.2014.04.010.
Chen, J., Wang, M., Kirschner, P. A., & Tsai, C. C. (2018). The role of collaboration, computer use, learning environments, and supporting strategies in CSCL: A meta-analysis. Review of Educational Research, 88(6), 799–843. https://doi.org/10.3102/0034654318791584.
Collins, L. M., & Lanza, S. T. (2013). Latent class and latent transition analysis: With applications in the social, behavioral, and health sciences. John Wiley.
Deng, R., Benckendorff, P., & Gannaway, D. (2020). Learner engagement in MOOCs: Scale development and validation. British Journal of Educational Technology, 51(1), 245–262. https://doi.org/10.1111/bjet.12810.
Finn, J. D., & Zimmer, K. S. (2012). Student engagement: What is it? Why does it matter? Handbook of research on student engagement (pp. 97–131). Springer. https://doi.org/10.1007/978-1-4614-2018-7_5.
Fredricks, J. A., Blumenfeld, P. C., & Paris, A. H. (2004). School engagement: Potential of the concept, state of the evidence. Review of Educational Research, 74(1), 59–109. https://doi.org/10.3102/00346543074001059.
Fredricks, J., McColskey, W., Meli, J., Mordica, J., Montrosse, B., & Mooney, K. (2011). Measuring student engagement in upper elementary through high school: A description of 21 instruments (issues & answers Report, REL 2011-No. 098). U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Southeast.
Gao, F. (2013). A case study of using a social annotation tool to support collaboratively learning. The Internet and Higher Education, 17, 76–83. https://doi.org/10.1016/j.iheduc.2012.11.002.
Garrison, D. R., & Arbaugh, J. B. (2007). Researching the community of inquiry framework: Review, issues and future directions. The Internet and Higher Education, 10(3), 157–172. https://doi.org/10.1016/j.iheduc.2007.04.001.
Garrison, D. R., Anderson, T., & Archer, W. (2000). Critical inquiry in a text-based environment: Computer conferencing in higher education model. The Internet and Higher Education, 2(2–3), 87–105. https://doi.org/10.1016/S1096-7516(00)00016-6.
Garrison, D. R., Anderson, T., & Archer, W. (2001). Critical thinking, cognitive presence, and computer conferencing in distance education. American Journal of Distance Education, 15(1), 7–23. https://doi.org/10.1080/08923640109527071.
Ghadirian, H., Salehi, K., & Ayub, A. F. M. (2018). Social annotation tools in higher education: A preliminary systematic review. International Journal of Learning Technology, 13(2), 130–162. https://doi.org/10.1504/IJLT.2018.092096.
Hickendorff, M., Edelsbrunner, P. A., McMullen, J., Schneider, M., & Trezise, K. (2018). Informative tools for characterizing individual differences in learning: Latent class, latent profile, and latent transition analysis. Learning and Individual Differences, 66, 4–15. https://doi.org/10.1016/j.lindif.2017.11.001.
Hollister, B., Nair, P., Hill-Lindsay, S., & Chukoskie, L. (2022). Engagement in online learning: Student attitudes and behavior during COVID-19. Frontiers in Education, 7, 851019. https://doi.org/10.3389/feduc.2022.851019.
How Perusall scoring works. (n.d.). Retrieved from https://perusall.com/hubfs/downloads/scoring-details.pdf.
Howard, M. C., & Hoffman, M. E. (2018). Variable-centered, person-centered, and person-specific approaches: Where theory meets the method. Organizational Research Methods, 21(4), 846–876. https://doi.org/10.1177/1094428117744021.
Hu, M., & Li, H. (2017, June). Student engagement in online learning: A review. In 2017 International Symposium on Educational Technology (ISET) (pp. 39–43). IEEE. https://doi.org/10.1109/ISET.2017.17.
Kalir, J., & Garcia, A. (2019). Annotation. MIT Press Open.
Kalir, J. H., Morales, E., Fleerackers, A., & Alperin, J. P. (2020). When I saw my peers annotating Student perceptions of social annotation for learning in multiple courses. Information and Learning Sciences, 121(3/4), 207–230. https://doi.org/10.1108/ILS-12-2019-0128.
Lämsä, J., Hämäläinen, R., Koskinen, P., Viiri, J., & Lampi, E. (2021). What do we do when we analyse the temporal aspects of computer-supported collaborative learning? A systematic literature review. Educational Research Review, 33, 100387. https://doi.org/10.1016/j.edurev.2021.100387.
Lazzara, J., & Clinton-Lisell, V. (2022). Using social annotation to enhance student engagement in psychology courses. Scholarship of Teaching and Learning in Psychology. https://doi.org/10.1037/stl0000335.
Lee, Y., Jang, B. G., & Conradi Smith, K. (2021). A systematic review of reading engagement research: What do we mean, what do we know, and where do we need to go? Reading Psychology, 42(5), 540–576. https://doi.org/10.1080/02702711.2021.1888359.
Li, S. C., & Lai, T. K. (2022). Unfolding knowledge co-construction processes through social annotation and online collaborative writing with text mining techniques. Australasian Journal of Educational Technology, 38(1), 148–163. https://doi.org/10.14742/ajet.6834.
Li, M., & Li, J. (2022). Using Perusall to motivate students’ curriculum-based academic reading. Journal of Computers in Education. https://doi.org/10.1007/s40692-022-00234-y.
Li, S. C., Pow, J. W., & Cheung, W. C. (2015). A delineation of the cognitive processes manifested in a social annotation environment. Journal of Computer Assisted Learning, 31(1), 1–13. https://doi.org/10.1111/jcal.12073.
Lin, J. W., & Lai, Y. C. (2014). Using collaborative annotating and data mining on formative assessments to enhance learning efficiency. Computer Applications in Engineering Education, 22(2), 364–374. https://doi.org/10.1002/cae.20561.
Marissa, K. L. (2021). Using an online social annotation tool in a content-based instruction (CBI) classroom. International Journal of TESOL Studies, 3(2), 5–23. https://doi.org/10.46451/ijts.2021.06.02.
Marshall, C. C. (1997). Annotation: From paper books to the digital library. In Proceedings of the second ACM international conference on Digital libraries (pp. 131–140). New York, NY: ACM Press.
Martin, F., & Bolliger, D. U. (2018). Engagement matters: Student perceptions on the importance of engagement strategies in the online learning environment. Online Learning, 22(1), 205–222. https://doi.org/10.24059/olj.v22i1.1092.
Martin, F., & Borup, J. (2022). Online learner engagement: Conceptual definitions, research themes, and supportive practices. Educational Psychologist, 57(3), 162–177. https://doi.org/10.1080/00461520.2022.2089147.
Mendenhall, A., & Johnson, T. E. (2010). Fostering the development of critical thinking skills, and reading comprehension of undergraduates using a web 2.0 tool coupled with a learning system. Interactive Learning Environments, 18(3), 263–276. https://doi.org/10.1080/10494820.2010.500537.
Miller, K., Zyto, S., Karger, D., Yoo, J., & Mazur, E. (2016). Analysis of student engagement in an online annotation system in the context of a flipped introductory physics class. Physical Review Physics Education Research, 12(2), 020143. https://doi.org/10.1103/PhysRevPhysEducRes.12.020143.
Miller, K., Lukoff, B., King, G., & Mazur, E. (2018). Use of a social annotation platform for pre-class reading assignments in a flipped introductory physics class. Frontiers in Education. https://doi.org/10.3389/feduc.2018.00008.
Miller, A. N., Sellnow, D. D., & Strawser, M. G. (2021). Pandemic pedagogy challenges and opportunities: Instruction communication in remote, HyFlex, and BlendFlex courses. Communication Education, 70(2), 202–204. https://doi.org/10.1080/03634523.2020.1857418.
Morales, E., Kalir, J. H., Fleerackers, A., & Alperin, J. P. (2022). Using social annotation to construct knowledge with others: A case study across undergraduate courses. F1000Research, 11, https://doi.org/10.12688/f1000research.109525.2.
Morin, A. J., Bujacz, A., & Gagné, M. (2018). Person-centered methodologies in the organizational sciences: Introduction to the feature topic. Organizational Research Methods, 21(4), 803–813. https://doi.org/10.1177/1094428118773856.
Moubayed, A., Injadat, M., Shami, A., & Lutfiyya, H. (2020). Student engagement level in an e-learning environment: Clustering using k-means. American Journal of Distance Education, 34(2), 137–156. https://doi.org/10.1080/08923647.2020.1696140.
Mu, X. (2010). Towards effective video annotation: An approach to automatically link notes with video content. Computers & Education, 55(4), 1752–1763. https://doi.org/10.1016/j.compedu.2010.07.021.
Novak, E., Razzouk, R., & Johnson, T. E. (2012). The educational use of social annotation tools in higher education: A literature review. The Internet and Higher Education, 15(1), 39–49. https://doi.org/10.1016/j.iheduc.2011.09.002.
Palvia, S., Aeron, P., Gupta, P., Mahapatra, D., Parida, R., Rosner, R., & Sindhi, S. (2018). Online education: Worldwide status, challenges, trends, and implications. Journal of Global Information Technology Management, 21(4), 233–241. https://doi.org/10.1080/1097198X.2018.1542262.
Pennebaker, J. W., Booth, R. J., Boyd, R. L., & Francis, M. E. (2015). Linguistic inquiry and word count: LIWC 2015 operator’s manual. Pennebaker Conglomerates.
Petitjean, F., Ketterlin, A., & Gançarski, P. (2011). A global averaging method for dynamic time warping, with applications to clustering. Pattern Recognition, 44(3), 678–693. https://doi.org/10.1016/j.patcog.2010.09.013.
Rai, P., & Singh, S. (2010). A survey of clustering techniques. International Journal of Computer Applications, 7(12), 1–5.
Rimm-Kaufman, S. E., Baroody, A. E., Larsen, R. A. A., Curby, T. W., & Abry, T. (2015). To what extent do teacher–student interaction quality and student gender contribute to fifth graders’ engagement in mathematics learning? Journal of Educational Psychology, 107(1), 170–185. https://doi.org/10.1037/a0037252.
Roman, T. A., Brantley-Dias, L., Dias, M., & Edwards, B. (2022). Addressing student engagement during COVID-19: Secondary STEM teachers attend to the affective dimension of lear ner needs. Journal of Research on Technology in Education, 54(sup1), S65–S93. https://doi.org/10.1080/15391523.2021.1920519.
Rourke, L., Anderson, T., Garrison, D. R., & Archer, W. (1999). Assessing social presence in asynchronous, text-based computer conferencing. Journal of Distance Education, 14(3), 51–70. https://www.learntechlib.org/p/92000/.
Saqr, M., & López-Pernas, S. (2021). The longitudinal trajectories of online engagement over a full program. Computers & Education, 175, 104325. https://doi.org/10.1016/j.compedu.2021.104325.
Sardá-Espinosa, A. (2017). Comparing time-series clustering algorithms in r using the dtwclust package. R Development Core Team.
Sardá-Espinosa, A. (2019). Time-series clustering in R using the dtwclust package. The R Journal. https://doi.org/10.32614/RJ-2019-023.
Spitzer, M. W. H., Gutsfeld, R., Wirzberger, M., & Moeller, K. (2021). Evaluating students’ engagement with an online learning environment during and after COVID-19 related school closures: A survival analysis approach. Trends in Neuroscience and Education, 25, 100168. https://doi.org/10.1016/j.tine.2021.100168.
Suhre, C., Winnips, K., De Boer, V., Valdivia, P., & Beldhuis, H. (2019, July). Students’ experiences with the use of a social annotation tool to improve learning in flipped classrooms. In HEAD’19. 5th International Conference on Higher Education Advances (pp. 955–964). Editorial Universitat Politècnica de València.
Sun, Y., & Gao, F. (2017). Comparing the use of a social annotation tool and a threaded discussion forum to support online discussions. The Internet and Higher Education, 32, 72–79. https://doi.org/10.1016/j.iheduc.2016.10.001.
Sun, C., Hwang, G. J., Yin, Z., Wang, Z., & Wang, Z. (2023). Trends and issues of social annotation in education: A systematic review from 2000 to 2020. Journal of Computer Assisted Learning, 39(2), 329–350. https://doi.org/10.1111/jcal.12764.
Unrau, N. J., & Quirk, M. (2014). Reading motivation and reading engagement: Clarifying commingled conceptions. Reading Psychology, 35(3), 260–284. https://doi.org/10.1080/02702711.2012.684426.
Wang, W., & Zhang, Y. (2007). On fuzzy cluster validity indices. Fuzzy Sets and Systems, 158(19), 2095–2117. https://doi.org/10.1016/j.fss.2007.03.004.
Willms, J. D. (2003). Student engagement at school: A sense of belonging and participation. Organisation for Economic Co-Operation & Development.
Wood, D., & O’Malley, C. (1996). Collaborative learning between peers: An overview. Educational Psychology in Practice, 11(4), 4–9. https://doi.org/10.1080/0266736960110402.
Yang, Y. F., & Lin, Y. Y. (2015). Online collaborative note-taking strategies to foster EFL beginners’ literacy development. System, 52, 127–138. https://doi.org/10.1016/j.system.2015.05.006.
Zarzour, H., & Sellami, M. (2018). Effects of a linked data-based annotation approach on students’ learning achievement and cognitive load. Interactive Learning Environments, 26(8), 1090–1099. https://doi.org/10.1080/10494820.2018.1446989.
Zhu, X., Shui, H., & Chen, B. (2023). Beyond reading together: Facilitating knowledge construction through participation roles and social annotation in college classrooms. The Internet and Higher Education, 100919. https://doi.org/10.1016/j.iheduc.2023.100919.
Funding
No funding was received for conducting this study.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Ethics approval
Ethical approval was obtained from the Research Ethics Board Office at McGill University.
Conflict of interest
The authors declare that they have no competing interests.
Informed consent
All the participants have given their written informed consent.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Chen, F., Li, S., Lin, L. et al. Identifying temporal changes in student engagement in social annotation during online collaborative reading. Educ Inf Technol (2024). https://doi.org/10.1007/s10639-024-12494-5
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s10639-024-12494-5