Abstract
Knowledge building (KB) competencies are crucial for undergraduates’ creative knowledge work and academic success. While there is substantial research on KB discourse, there are limited efforts in examining how KB competencies in the conceptual, metacognitive, socio-emotional, and epistemic dimensions are demonstrated in KB discourse and how the competencies can be scaffolded. Previous studies suggest the effectiveness of reflective assessment on sustainable and productive KB discourse. This study developed a framework for analyzing KB competencies using KB discourse moves. It also examined whether a KB design augmented by reflective assessment enriched by analytic tools and artifacts could foster undergraduates’ KB competencies, and if so, how. This KB design involves principle-based pedagogy with the participants engaging in collaborative inquiry and discussion on Knowledge Forum, and reflective assessment using (a) super synthesis notes, (b) KB interaction rubrics, and (c) learning analytics visualization tools. Qualitative tracking and lag sequential pattern analysis of Knowledge Forum’s discourse revealed that implementing reflective assessment supported by analytics and artifacts could help the undergraduate students develop KB competencies manifested in discourse with evidence of conceptual advance, epistemic engagement, metacognition, and productive socio-emotional interactions with a collective focus. The thematic analysis illustrated the dynamics through which the design enriched by standards and visualizations helped the undergraduates develop KB competencies: leveraging synthesis super notes to promote conceptual and metacognitive advancement and epistemic engagement; employing KB interaction rubrics to cultivate metacognitive, socio-emotional, and epistemic competencies; and harnessing KBDeX visualizations to promote metacognitive and conceptual advancement and to facilitate epistemic engagement. The implications of scaffolding students’ epistemic agency, metacognition, productive collaborative inquiry, and developing KB competencies in a technology-supported metacognitive learning environment are discussed.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data availability
These data will be made available to other researchers on a case-by-case basis.
References
Andrews-Todd, J., & Forsyth, C. M. (2020). Exploring social and cognitive dimensions of collaborative problem solving in an open online simulation-based task. Computers in Human Behavior, 104, Article 105759. https://doi.org/10.1016/j.chb.2018.10.025
Bell, T., Urhahne, D., Schanze, S., & Ploetzner, R. (2010). Collaborative inquiry learning: Models, tools, and challenges. International Journal of Science Education, 32(3), 349–377. https://doi.org/10.1080/09500690802582241
Bereiter, C., & Scardamalia, M. (2016). “Good Moves” in knowledge-creating dialogue. QWERTY-Interdisciplinary Journal of Technology, Culture and Education, 11(2), 12–26.
Bjuland, R., & Helgevold, N. (2018). Dialogic processes that enable student teachers’ learning about pupil learning in mentoring conversations in a Lesson Study field practice. Teaching and Teacher Education, 70, 246–254. https://doi.org/10.1016/j.tate.2017.11.026
Borge, M., Ong, Y. S., & Rosé, C. P. (2018). Learning to monitor and regulate collective thinking processes. International Journal of Computer-Supported Collaborative Learning, 13, 61–92. https://doi.org/10.1007/s11412-018-9270-5
Chalkiadaki, A. (2018). A systematic literature review of 21st century skills and competencies in primary education. International Journal of Instruction, 11(3), 1-16. https://doi.org/10.12973/iji.2018.1131a
Chan, C. K., & van Aalst, J. (2018). Knowledge building: Theory, design, and analysis. In International handbook of the learning sciences (pp. 295–307). Routledge.
Chen, B., & Hong, H.-Y. (2016). Schools as knowledge-building organizations: Thirty years of design research. Educational Psychologist, 51, 266–288. https://doi.org/10.1080/00461520.2016.1175306
Chen, B. (2017). Fostering scientific understanding and epistemic beliefs through judgments of promisingness. Educational Technology Research & Development, 65, 255–277. https://doi.org/10.1007/s11423-016-9467-0
Chen, B., Scardamalia, M., & Bereiter, C. (2015). Advancing knowledge-building discourse through judgments of promising ideas. International Journal of Computer-Supported Collaborative Learning, 10(4), 345–366. https://doi.org/10.1007/s11412-015-9225-z
Chen, Y., Andrews, C. D., Hmelo-Silver, C. E., & D’Angelo, C. (2019). Coding schemes as lenses on collaborative learning. Information and Learning Sciences, 121(1/2), 1–18. https://doi.org/10.1108/ILS-08-2019-0079
Chuy, M., Resendes, M., Tarchi, C., Chen, B., Scardamalia, M., & Bereiter, C. (2011). Ways of contributing to an explanation-seeking dialogue in science and history. QWERTY: Journal of Technology and Culture, 6(2), 242–260.
Collins, A. (2017). What’s worth teaching? Rethinking curriculum in the Age of technology. Teachers College Press.
Collins, A., Joseph, D., & Bielaczyc, K. (2004). Design research: Theoretical and methodological Issues. The Journal of the Learning Sciences, 13(1), 15–42.
Damrow, A. L., & Sweeney, J. S. (2019). Beyond the bubble: Preparing preservice teachers through dialogue across distance and difference. Teaching and Teacher Education, 80, 255–265. https://doi.org/10.1016/j.tate.2019.02.003
Damşa, C. I. (2014). The multi-layered nature of small-group learning: Productive interactions in object-oriented collaboration. International Journal of Computer-Supported Collaborative Learning, 9, 247-281. https://doi.org/10.1007/s11412-014-9193-8
De Backer, L., Van Keer, H., & Valcke, M. (2021). The functions of shared metacognitive regulation and their differential relation with collaborative learners’ understanding of the learning content. Learning and Instruction, 77, 101527. https://doi.org/10.1016/j.learninstruc.2021.101527
Dorfsman, M. I. (2018). The development of discourse in the online environment: Between technology and multiculturalism. International Journal of Educational Technology in Higher Education, 15(31), https://doi.org/10.1186/s41239-018-0110-5.
Ewell, P. T. (2005). Can assessment serve accountability? It depends on the question. Achieving Accountability in Higher Education, 1-24.
González-Salamanca, J. C., Agudelo, O. L., & Salinas, J. (2020). Key competences, education for sustainable development and strategies for the development of 21st century skills. A systematic literature review. Sustainability, 12(24), 10366. https://doi.org/10.3390/su122410366
Hilton, M. L., & Pellegrino, J. W. (Eds.). (2012). Education for life and work: Developing transferable knowledge and skills in the 21st century. National Academies Press.
Hmelo-Silver, C. E., & Barrows, H. S. (2008). Facilitating collaborative knowledge building. Cognition and Instruction, 26(1), 48–94. https://doi.org/10.1080/07370000701798495
Hod, Y., & Katz, S. (2020). Fostering highly engaged knowledge building communities in socio-emotional and sociocognitive hybrid learning spaces. British Journal of Educational Technology, 51(4), 1117–1135. https://doi.org/10.1111/bjet.12910
Hong, H. Y., Lin, P. Y., Chai, C. S., Hung, G. T., & Zhang, Y. (2019). Fostering design-oriented collective reflection among preservice teachers through principle-based knowledge building activities. Computers & Education, 130, 105–120. https://doi.org/10.1016/j.compedu.2018.12.001
House, J. (2013). Developing pragmatic competence in English as a lingua franca: Using discourse markers to express intersubjectivity and connectivity. Journal of Pragmatics, 59, 57–67. https://doi.org/10.1016/j.pragma.2013.03.001
Järvenoja, H., & Järvelä, S. (2009). Emotion control in collaborative learning situations: Do students regulate emotions evoked by social challenges. British Journal of Educational Psychology, 79(3), 463–481. https://doi.org/10.1348/000709909X402811
Järvelä, S., Kirschner, P. A., Hadwin, A., Järvenoja, H., Malmberg, J., Miller, M., & Laru, J. (2016). Socially shared regulation of learning in CSCL: Understanding and prompting individual-and group-level shared regulatory activities. International Journal of Computer-Supported Collaborative Learning, 11, 263–280. https://doi.org/10.1007/s11412-016-9238-2
Jeong, H., Hmelo-Silver, C. E., & Yu, Y. (2014). An examination of CSCL methodological practices and the influence of theoretical frameworks 2005–2009. International Journal of Computer-Supported Collaborative Learning, 9, 305–334. https://doi.org/10.1007/s11412-014-9198-3
Kim, D., & Lim, C. (2018). Promoting socially shared metacognitive regulation in collaborative project-based learning: A framework for the design of structured guidance. Teaching in Higher Education, 23(2), 194–211. https://doi.org/10.1080/13562517.2017.1379484
Kulikowich, J. M., & Alexander, P. A. (2003). Cognitive Assessment. In L. Nadel (Ed.), The Encyclopedia of Cognitive Science (Vol. 1, pp. 526–532). Nature Publishing Group.
Lee, E. Y., Chan, C. K. K., & Van Aalst, J. (2006). Students assessing their own collaborative knowledge building. International Journal of Computer-Supported Collaborative Learning, 1(1), 57–87. https://doi.org/10.1007/s11412-006-6844-4
Lei, C., & Chan, C. K. K. (2018). Developing meta-discourse through reflective assessment in knowledge building environments. Computers & Education, 126, 153–169. https://doi.org/10.1016/j.compedu.2018.07.006
Näykki, P., Isohätälä, J., Järvelä, S., Pöysä-Tarhonen, J., & Häkkinen, P. (2017). Facilitating socio-cognitive and socio-emotional monitoring in collaborative learning with a regulation macro script–an exploratory study. International Journal of Computer-Supported Collaborative Learning, 12, 251–279. https://doi.org/10.1007/s11412-017-9259-5
National Research Council (NRC). (2012). A framework for K-12 science education: Practices, crosscutting concepts, and core ideas. The National Academies Press
OECD. (2023). OECD core competency framework. https://www.oecd.org/careers/OECD-Core-Competency-Framework.pdf. Accessed 12 Jan 2024.
Oshima, J., Oshima, R., & Matsuzawa, Y. (2012). Knowledge Building Discourse Explorer: a social network analysis application for knowledge building discourse. Educational Technology Research and Development, 60, 903–921. https://doi.org/10.1007/s11423-012-9265-2
Pettersson, F. (2018). On the issues of digital competence in educational contexts–a review of literature. Education and Information Technologies, 23(3), 1005–1021. https://doi.org/10.1007/s10639-017-9649-3
Resendes, M., Scardamalia, M., Bereiter, C., Chen, B., & Halewood, C. (2015). Group-level formative feedback and metadiscourse. International Journal of Computer-Supported Collaborative Learning, 10, 309–336. https://doi.org/10.1007/s11412-015-9219-x
Rogat, T. K., & Adams-Wiggins, K. R. (2025). Interrelation between regulatory and socio-emotional processes within collaborative groups characterized by facilitative and directive other-regulation. Computers in Human Behavior, 52, 589–600. https://doi.org/10.1016/j.chb.2015.01.026
Rogat, T. K., & Linnenbrink-Garcia, L. (2011). Socially shared regulation in collaborative groups: An analysis of the interplay between quality of social regulation and group processes. Cognition and Instruction, 29, 375–415. https://doi.org/10.1080/07370008.2011.607930
Scardamalia, M. (2002). Collective cognitive responsibility for the advancement of knowledge. Liberal Education in a Knowledge Society, 97, 67–98.
Scardamalia, M., & Bereiter, C. (2014). Knowledge building and knowledge creation: theory, pedagogy, and technology. In R. K. Sawyer (Ed.), The Cambridge handbook of the learning sciences (2nd ed., pp. 397-417). New York, NY: Cambridge University Press. https://doi.org/10.1017/CBO9781139519526.025
Snell, J., & Lefstein, A. (2018). “Low Ability”, participation, and identity in dialogic pedagogy. American Educational Research Journal, 55, 40–78. https://doi.org/10.3102/0002831217730010
Stahl, G. (2015). A decade of CSCL. International Journal of Computer-Supported Collaborative Learning, 10, 337–344. https://doi.org/10.1007/s11412-015-9222-2
Sun, C., Shute, V. J., Stewart, A., Yonehiro, J., Duran, N., & D’Mello, S. (2020). Towards a generalized competency model of collaborative problem solving. Computers & Education, 143, 103672. https://doi.org/10.1016/j.compedu.2019.103672
Tao, D., & Zhang, J. (2018). Forming shared inquiry structures to support knowledge building in a grade 5 community. Instructional Science, 46, 563–592. https://doi.org/10.1007/s11251-018-9462-4
Tao, D., & Zhang, J. (2021). Agency to transform: how did a grade 5 community co-configure dynamic knowledge building practices in a yearlong science inquiry? International Journal of Computer-Supported Collaborative Learning, 16, 403–404. https://doi.org/10.1007/s11412-021-09353-7
Tchounikine, P. (2019). Learners’ agency and CSCL technologies: towards an emancipatory perspective. International Journal of Computer-Supported Collaborative Learning, 14, 237–250. https://doi.org/10.1007/s11412-019-09302-5
Toth, E. E., Suthers, D. D., & Lesgold, A. M. (2002). “Mapping to know”: The effects of representational guidance and reflective assessment on scientific inquiry. Science Education, 86(2), 264–286. https://doi.org/10.1002/sce.10004
Van Aalst, J. (2009). Distinguishing knowledge-sharing, knowledge-construction and knowledge-creation discourse. International Journal of Computer Supported Collaborative Learning, 4, 259–287. https://doi.org/10.1007/s11412-009-9069-5
Van Aalst, J., & Chan, C. K. K. (2007). Student-directed assessment of knowledge building using electronic portfolios. Journal of the Learning Sciences, 16(2), 175–220. https://doi.org/10.1080/10508400701193697
Van Uum, M. S., Verhoeff, R. P., & Peeters, M. (2016). Inquiry-based science education: towards a pedagogical framework for primary school teachers. International Journal of Science Education, 38(3), 450–469. https://doi.org/10.1080/09500693.2016.1147660
Voogt, J., & Roblin, N. P. (2012). A comparative analysis of international frameworks for 21st century competencies: Implications for national curriculum policies. Journal of Curriculum Studies, 44(3), 299–321. https://doi.org/10.1080/00220272.2012.668938
White, B., & Frederiksen, J. (1998). Inquiry, modelling, and metacognition: Making science accessible to all students. Cognition and Instruction, 16, 3–118. https://doi.org/10.1207/s1532690xci1601_2
Wise, A. F., & Schwartz, B. (2017). Visions of CSCL: Eight provocations for the future of the field. International Journal of Computer-Supported Collaborative Learning, 12, 423–467. https://doi.org/10.1007/s11412-017-9267-5
Yang, Y. (2019). Reflective assessment for epistemic agency of academically low-achieving students. Journal of Computer Assisted Learning, 35, 459–475. https://doi.org/10.1111/jcal.12343
Yang, Y., Chen, Y., Feng, X., Sun, D., & Pang, S. (2024). Investigating the mechanisms of analytics-supported reflective assessment for fostering collective knowledge. Journal of Computing in Higher Education. https://doi.org/10.1007/s12528-024-09398-1
Yang, Y., Feng, X., Zhu, G., & Sun, D. (2023). Exploring the mechanisms of data-supported reflective assessment for pre-service teachers’ knowledge building. Interactive Learning Environments. https://doi.org/10.1080/10494820.2023.2223238
Yang, Y., Chen, Q., Yu, Y., Feng, X., & van Aalst, J. (2020). Collective reflective assessment for shared epistemic agency by undergraduates in knowledge building. British Journal of Educational Technology, 51(4), 1136–1154. https://doi.org/10.1111/bjet.12909
Yang, Y., Du, Y., van Aalst, J., Sun, D., & Ouyang, F. (2020a). Self-directed refective assessment for collective empowerment among pre-service teachers. British Journal of Educational Technology, 51(6), 1961–1981.
Yang, Y., van Aalst, J., & Chan, C. K. K. (2020c). Dynamics of reflective assessment and knowledge building for academically low-achieving students. American Educational Research Journal, 57(3), 1241–1289. https://doi.org/10.3102/0002831219872444
Yang, Y., van Aalst, J., Chan, C. K. K., & Tian, W. (2016). Reflective assessment in knowledge building by students with low academic achievement. International Journal of Computer-Supported Collaborative Learning, 11, 281–311. https://doi.org/10.1007/s11412-016-9239-1
Yang, Y., Zhu, G., Sun, D., & Chan, C. K. K. (2022). Collaborative analytics-supported reflective assessment for scaffolding pre-service teachers’ collaborative inquiry and knowledge building. International Journal of Computer-Supported Collaborative Learning, 17, 249–292. https://doi.org/10.1007/s11412-022-09372-y
Yuan, G., Zhang, J., & Chen, M.-C. (2022). Cross-community knowledge building with Idea Thread Mapper. International Journal of Computer-Supported Collaborative Learning, 17(2), 293–326. https://doi.org/10.1007/s11412-022-09371-z
Zhang, J., Scardamalia, M., Lamon, M., Messina, R., & Reeve, R. (2007). Socio-cognitive dynamics of knowledge building in the work of 9-and 10-year-olds. Educational Technology Research and Development, 55, 117–145. https://doi.org/10.1007/s11423-006-9019-0
Zhang, J., Tian, Y., Yuan, G., & Tao, D. (2022). Epistemic agency for costructuring expansive knowledge-building practices. Science Education, 106, 890– 923. https://doi.org/10.1002/sce.21717
Zhang, J., Yuan, G., & Bogouslavsky, M. (2020). Give student ideas a larger stage: Support cross-community interaction for knowledge building. International Journal of Computer-Supported Collaborative Learning, 15(4), 389–410. https://doi.org/10.1007/s11412-020-09332-4
Zhu, G., & Lin, F. (2023). Teachers scaffold student discourse and emotions in knowledge building classrooms. Interactive Learning Environments, 1–18. https://doi.org/10.1080/10494820.2023.2172046
Acknowledgments
The authors are indebted to the participating students.
Funding
National Natural Science Foundation of China, Grant/Award Number: 62107020; Central China Normal University, Grant/Award Numbers: CCNUTEIII 2021-11, CCNUAI&FE2022-03-15; Collaborative Innovation Center for Informatization and Balanced Development of K-12 Education by MOE, Grant/Award Number: xtzd2022-002; Ministry of Education of the People’s Republic of China, Grant/Award Number: 21YJA880078.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Electronic supplementary material
Below is the link to the electronic supplementary material.
Appendix A
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Yang, Y., Chan, C.K.K., Zhu, G. et al. Reflective assessment using analytics and artifacts for scaffolding knowledge building competencies among undergraduate students. Intern. J. Comput.-Support. Collab. Learn (2024). https://doi.org/10.1007/s11412-024-09421-8
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s11412-024-09421-8