Abstract
Developing critical thinking is becoming increasingly important as is giving and receiving feedback during the learning process. The aim of this work is to study how technology can scaffold peer assessment activities to develop critical thinking among pre-service teachers and study the relevance of giving and receiving feedback. A series of practice and application activities were introduced using technology-scaffolded peer assessment. Technological scaffolding minimized classroom logistics, while at the same time resolved any personal issues between peers as the tasks were assigned at random. Mixed-methods analysis revealed that technology-scaffolded peer assessment with anonymous feedback aided the significant development of critical thinking activities. It also showed that the feedback that was given was a predictor of the success of these activities. The added value of this work is that we show that for pre-service teachers, in a Reading Methods course, we can improve critical thinking skills with technology scaffolded peer assessment, and that giving feedback shows to be more relevant than receiving it.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data Availability
The datasets generated during and/or analysed during the current study are available from the corresponding author on reasonable request.
References
Amalia, Q., Hartono, Y., & Indaryanti, I. (2019). Students’ critical thinking skills in modeling based learning. Journal of Physics: Conference Series, 1166(1), 012017. https://doi.org/10.1088/1742-6596/1166/1/012017
Aminudin, M., & Kusmaryono, I. (2019). mathematical teacher efforts to develop student’s critical thinking skill. Math Didactic: Jurnal Pendidikan Matematika, 5(3), 248–258. https://doi.org/10.33654/math.v5i3.773
Arsal, Z. (2015). The effects of microteaching on the critical thinking dispositions of pre-service teachers. Australian Journal of Teacher Education, 40(3), 9. https://doi.org/10.14221/ajte.2014v40n3.9
Baker, K. M. (2016). Peer review as a strategy for improving students’ writing process. Active Learning in Higher Education, 17(3), 179–192. https://doi.org/10.1177/1469787416654794
Barak, M. (2017). Science teacher education in the twenty-first century: A pedagogical framework for technology-integrated social constructivism. Research in Science Education, 47(2), 283–303. https://doi.org/10.1007/s11165-015-9501-y
Berndt, M., Strijbos, J., & Fischer, F. (2018). Effects of written peer-feedback content and sender’s competence on perceptions, performance, and mindful cognitive processing. European Journal of Psychology of Education, 33, 31–49. https://doi.org/10.1007/s10212-017-0343-z
Bonett, D. G. (2019). Point-biserial correlation: Interval estimation, hypothesis testing, meta-analysis, and sample size determination. British Journal of Mathematical and Statistical Psychology. https://doi.org/10.1111/bmsp.12189
Broadbent, J., Panadero, E., & Boud, D. (2018). Implementing summative assessment with a formative flavour: A case study in a large class. Assessment & Evaluation in Higher Education, 43(2), 307–322. https://doi.org/10.1080/02602938.2017.1343455
Brookhart, S. M., & Chen, F. (2015). The quality and effectiveness of descriptive rubrics. Educational Review, 67(3), 343–368. https://doi.org/10.1080/00131911.2014.929565
Cáceres, M., Nussbaum, M., & Ortiz, J. (2020). Integrating critical thinking into the classroom: A teacher’s perspective. Thinking Skills and Creativity, 37, 100674. https://doi.org/10.1016/j.tsc.2020.100674
Cao, Z., Yu, S., & Huang, J. (2019). A qualitative inquiry into undergraduates’ learning from giving and receiving peer feedback in L2 writing: Insights from a case study. Studies in Educational Evaluation, 63, 102–112. https://doi.org/10.1016/j.stueduc.2019.08.001
Chang, S. C., Hsu, T. C., & Jong, M. S. Y. (2020). Integration of the peer assessment approach with a virtual reality design system for learning earth science. Computers & Education, 146, 103758. https://doi.org/10.1016/j.compedu.2019.103758
Chasteen, S. V., & Scherr, R. E. (2020). Developing the Physics Teacher Education Program Analysis rubric: Measuring features of thriving programs. Physical Review Physics Education Researchers Suggest, 16(1), 010115. https://doi.org/10.1103/PhysRevPhysEducRes.16.010115
Chechile, R. A. (2018). A Bayesian analysis for the Wilcoxon signed-rank statistic. Communications in Statistics-Theory and Methods, 47(21), 5241–5254. https://doi.org/10.1213/ANE.0b013e31827f53d7
Chen, M. R. A., & Hwang, G. J. (2020). Effects of a concept mapping-based flipped learning approach on EFL students’ English-speaking performance, critical thinking awareness and speaking anxiety. British Journal of Educational Technology, 51(3), 817–834.
Clark, V. L. P. (2019). Meaningful integration within mixed methods studies: Identifying why, what, when, and how. Contemporary Educational Psychology, 57, 106–111. https://doi.org/10.1016/j.cedpsych.2019.01.007
Creswell, J. W. (2014). A concise introduction to mixed methods research. SAGE publications.
Cruz, G., Payan-Carreira, R., Dominguez, C., Silva, H., & Morais, F. (2021). What critical thinking skills and dispositions do new graduates need for professional life? Views from Portuguese employers in different fields. Higher Education Research & Development, 40(4), 721–737. https://doi.org/10.1080/07294360.2020.1785401
Danczak, S. M., Thompson, C. D., & Overton, T. L. (2020). Development and validation of an instrument to measure undergraduate chemistry students’ critical thinking skills. Chemistry Education Research and Practice, 21(1), 62–78. https://doi.org/10.1039/C8RP00130H
Divine, G., Norton, H. J., Hunt, R., & Dienemann, J. (2013). A review of analysis and sample size calculation considerations for Wilcoxon tests. Anesthesia & Analgesia, 117(3), 699–710. https://doi.org/10.1213/ANE.0b013e31827f53d7
Double, K. S., McGrane, J. A., & Hopfenbeck, T. N. (2020). The impact of peer assessment on academic performance: A meta-analysis of control group studies. Educational Psychology Review, 32(2), 481–509. https://doi.org/10.1007/s10648-019-09510-3
El Soufi, N., & See, B. H. (2019). Does explicit teaching of critical thinking improve critical thinking skills of English language learners in higher education? A critical review of causal evidence. Studies in Educational Evaluation, 60, 140–162. https://doi.org/10.1016/j.stueduc.2018.12.006
Ennis, R. H. (2018). Critical thinking across the curriculum: A vision. Topoi, 37(1), 165–184. https://doi.org/10.1007/s11245-016-9401-4
Facione, P. A. (1990). Critical Thinking: A Statement of Expert Consensus for Purposes of Educational Assessment and Instruction Executive Summary “The Delphi Report (vol. 423(c), pp. 1–19). The California Academic Press. https://doi.org/10.1016/j.tsc.2009.07.002
Fang, J. W., Chang, S. C., Hwang, G. J., et al. (2021). An online collaborative peer-assessment approach to strengthening pre-service teachers’ digital content development competence and higher-order thinking tendency. Educational Technology Research and Development, 69, 1155–1181. https://doi.org/10.1007/s11423-021-09990-7
Fertelli, T. K. (2019). Peer assessment in learning of nursing process: Critical thinking and peer support. International Journal of Caring Sciences, 12(1), 331–339.
Filius, R. M., de Kleijn, R. A., Uijl, S. G., Prins, F. J., van Rijen, H. V., & Grobbee, D. E. (2018). Strengthening dialogic peer feedback aiming for deep learning in SPOCs. Computers & Education, 125, 86–100.
Fu, Q. K., Lin, C. J., & Hwang, G. J. (2019). Research trends and applications of technology-supported peer assessment: A review of selected journal publications from 2007 to 2016. Journal of Computers in Education, 6(2), 191–213. https://doi.org/10.1007/s40692-019-00131-x
Giacumo, L. A., & Savenye, W. (2020). Asynchronous discussion forum design to support cognition: Effects of rubrics and instructor prompts on learner’s critical thinking, achievement, and satisfaction. Educational Technology Research and Development, 68, 37–66. https://doi.org/10.1007/s11423-019-09664-5
Gielen, M., & De Wever, B. (2015). Structuring peer assessment: Comparing the impact of the degree of structure on peer feedback content. Computers in Human Behavior, 52, 315–325. https://doi.org/10.1016/j.chb.2015.06.019
Gill-Simmen, L. (2020). Developing critical thinking skills: Using Edward de Bono’s six thinking hats in formative peer assessment & feedback. Journal of Applied Learning and Teaching, 3(1), 138–141. https://doi.org/10.37074/jalt.2020.3.1.5
Golightly, A. (2021). Self-and peer assessment of preservice geography teachers’ contribution in problem-based learning activities in geography education. International Research in Geographical and Environmental Education, 30(1), 75–90. https://doi.org/10.1080/10382046.2020.1744242
Goodsett, M. (2020). Best practices for teaching and assessing critical thinking in information literacy online learning objects. The Journal of Academic Librarianship, 46(5), 102163. https://doi.org/10.1016/j.acalib.2020.102163
Hanrahan, S. J., & Isaacs, G. (2001). Assessing self-and peer-assessment: The students’ views. Higher Education Research & Development, 20(1), 53–70. https://doi.org/10.1080/07294360123776
Harland, T., Wald, N., & Randhawa, H. (2017). Assessment & evaluation in higher education student peer review: Enhancing formative feedback with a rebuttal Student peer review: Enhancing formative feedback with a rebuttal. Assessment & Evaluation in Higher Education, 42(5), 801–811. https://doi.org/10.1080/02602938.2016.1194368
Hwang, G., & Chang, S. (2021). Facilitating knowledge construction in mobile learning contexts: A bi-directional peer-assessment approach. British Journal of Educational Technology, 52(1), 337–357. https://doi.org/10.1111/bjet.13001
Hwang, G.-J., Hung, C.-M., & Chen, N.-S. (2014). Improving learning achievements, motivations and problem-solving skills through a peer assessment-based game development approach. Educational Technology Research & Development, 62(2), 129–145. https://doi.org/10.1007/s11423-013-9320-7
Hursen, C. (2020). The effect of problem-based learning method supported by web 2.0 tools on academic achievement and critical thinking skills in teacher education. Technology, Knowledge and Learning. https://doi.org/10.1007/s10758-020-09458-2
Ion, G., Sánchez Martí, A., & Agud Morell, I. (2019). Giving or receiving feedback: Which is more beneficial to students’ learning? Assessment & Evaluation in Higher Education, 44(1), 124–138. https://doi.org/10.1080/02602938.2018.1484881
Janssen, E. M., Mainhard, T., Buisman, R. S., Verkoeijen, P. P., Heijltjes, A. E., van Peppen, L. M., & van Gog, T. (2019). Training higher education teachers’ critical thinking and attitudes towards teaching it. Contemporary Educational Psychology, 58, 310–322. https://doi.org/10.1016/j.cedpsych.2019.03.007
Kimberlin, C. L., & Winterstein, A. G. (2008). Validity and reliability of measurement instruments used in research. American Journal of Health-System Pharmacy, 65(23), 2276–2284. https://doi.org/10.2146/ajhp070364
Kostiainen, E., Ukskoski, T., Ruohotie-Lyhty, M., Kauppinen, M., Kainulainen, J., & Mäkinen, T. (2018). Meaningful learning in teacher education. Teaching and Teacher Education, 71, 66–77. https://doi.org/10.1016/j.tate.2017.12.009
Kvalseth, T. (1991). A coefficient of agreement for nominal scales: An asymmetric version of Kappa. Educational and Psychological Measurement, 51(1), 95–101.
Lai, C. L., & Hwang, G. J. (2015). An interactive peer-assessment criteria development approach to improving students’ art design performance using handheld devices. Computers & Education, 85, 149–159. https://doi.org/10.1016/j.compedu.2015.02.011
Li, P., Chang, L., Chua, T. H. H., & Loh, R. S. M. (2018). “Likes” as KPI: An examination of teenage girls’ perspective on peer feedback on Instagram and its influence on coping response. Telematics and Informatics, 35(7), 1994–2005. https://doi.org/10.1016/j.tele.2018.07.003
Liu, J., McBride, R. E., Xiang, P., & Scarmardo-Rhodes, M. (2018). Physical education pre-service teachers’ understanding, application, and development of critical thinking. Quest, 70(1), 12–27. https://doi.org/10.1080/00336297.2017.1330218
Lin, G. Y. (2018). Anonymous versus identified peer assessment via a Facebook-based learning application: Effects on quality of peer feedback, perceived learning, perceived fairness, and attitude toward the system. Computers & Education, 116, 81–92. https://doi.org/10.1016/j.compedu.2017.08.010
Li, L., & Grion, V. (2019). The Power of giving feedback ad receiving feedback in peer assessment. All Ireland Journal of Teaching and Learning in Higher Education, 11(2), 1–17. https://ojs.aishe.org/index.php/aishe-j/article/view/413/671
Li, H., Xiong, Y., Hunter, C. V., Guo, X., & Tywoniw, R. (2020). Does peer assessment promote student learning? A meta-analysis. Assessment & Evaluation in Higher Education, 45(2), 193–211. https://doi.org/10.1080/02602938.2019.1620679
Latifi, S., Noroozi, O., & Talaee, E. (2021). Peer feedback or peer feedforward? Enhancing students’ argumentative peer learning processes and outcomes. British Journal of Educational Technology, 52(2), 768–784. https://doi.org/10.1111/bjet.13054
Lee, Y. F., Lin, C. J., Hwang, G. J., Fu, Q. K., & Tseng, W. H. (2021). Effects of a mobile-based progressive peer-feedback scaffolding strategy on students’ creative thinking performance, metacognitive awareness, and learning attitude. Interactive Learning Environments. https://doi.org/10.1080/10494820.2021.1916763
Mulder, R., Baik, C., Naylor, R., & Pearce, J. (2014). How does student peer review influence perceptions, engagement and academic outcomes? A case study. Assessment & Evaluation in Higher Education, 39(6), 657–677. https://doi.org/10.1080/02602938.2013.860421
Mercader, C., Ion, G., & Díaz-Vicario, A. (2020). Factors influencing students’ peer feedback uptake: Instructional design matters. Assessment & Evaluation in Higher Education. https://doi.org/10.1080/02602938.2020.1726283
Nowell, L. S., Norris, J. M., White, D. E., & Moules, N. J. (2017). Thematic analysis: Striving to meet the trustworthiness criteria. International Journal of Qualitative Methods, 16(1), 1609406917733847.
Polit, D. F., & Beck, C. T. (2006). The content validity index: Are you sure you know what’s being reported? Critique and recommendations. Research in Nursing & Health, 29(5), 489–497. https://doi.org/10.1002/nur.20147
Palinkas, L. A., Horwitz, S. M., Green, C. A., Wisdom, J. P., Duan, N., & Hoagwood, K. (2015). Purposeful sampling for qualitative data collection and analysis in mixed method implementation research. Administration and Policy in Mental Health and Mental Health Services Research, 42(5), 533–544. https://doi.org/10.1007/s10488-013-0528-y
Planas-Lladó, A., Feliu, L., Castro, F., Fraguell, R. M., Arbat, G., Pujol, J., Suñol, J. J., & Daunis-i-Estadella, P. (2018). Using peer assessment to evaluate teamwork from a multidisciplinary perspective. Assessment & Evaluation in Higher Education, 43(1), 14–30. https://doi.org/10.1080/02602938.2016.1274369
Panadero, E., & Alqassab, M. (2019). An empirical review of anonymity effects in peer assessment, peer feedback, peer review, peer evaluation and peer grading. Assessment & Evaluation in Higher Education, 44(8), 1253–1278. https://doi.org/10.1080/02602938.2019.1600186
Puntambekar, S. (2022). Distributed scaffolding: scaffolding students in classroom environments. Educational Psychology Review, 34(1), 451–472. https://doi.org/10.1007/s10648-021-09636-3
Reddy, K., Harland, T., Wass,R., $ Wald, N. (2021). Student peer review as a process of knowledge creation through dialogue. Higher Education Research & Development, 40(4), 825–837. https://doi.org/10.1080/07294360.2020.1781797
Sharma, P., & Hannafin, M. J. (2007). Scaffolding in technology-enhanced learning environments. Interactive Learning Environments, 15(1), 27–46. https://doi.org/10.1080/10494820600996972
Stupple, E. J. N., Maratos, F. A., Elander, J., Hunt, T. E., Cheung, K. Y. F., & Aubeeluck, A. V. (2017). Development of the Critical Thinking Toolkit (CriTT): A measure of student attitudes and beliefs about critical thinking. Thinking Skills and Creativity, 23, 91–100. https://doi.org/10.1016/j.tsc.2016.11.007
Shen, B., Bai, B., & Xue, W. (2020). The effects of peer assessment on learner autonomy: An empirical study in a Chinese college English writing class. Studies in Educational Evaluation, 64, 100821. https://doi.org/10.1016/j.stueduc.2019.100821
Topping, K. (2018). Using peer assessment to inspire reflection and learning. Routledge.
Tai, J., & Adachi, C. (2019). The transformative role of self-and peer-assessment in developing critical thinkers. In C. Bryan & K. Clegg (Eds.), Innovative assessment in higher education (pp. 64–73). Routledge.
Tan, J. S., & Chen, W. (2022). Peer feedback to support collaborative knowledge improvement: What kind of feedback feed-forward? Computers & Education. https://doi.org/10.1016/j.compedu.2022.104467
Veliz, L., & Veliz-Campos, M. (2019). An interrogation of the role of critical thinking in English language pedagogy in Chile. Teaching in Higher Education, 24(1), 47–62. https://doi.org/10.1080/13562517.2018.1456424
van Heerden, M., & Bharuthram, S. (2021). Knowing me, knowing you: The effects of peer familiarity on receiving peer feedback for undergraduate student writers. Assessment & Evaluation in Higher Education, 46(8), 1191–1201. https://doi.org/10.1080/02602938.2020.1863910
Wass, R., Harland, T., & Mercer, A. (2011). Scaffolding critical thinking in the zone of proximal development. Higher Education Research & Development, 30(3), 317–328.
Winstone, N. E., Nash, R. A., Parker, M., & Rowntree, J. (2017). Supporting learners’ agentic engagement with feedback: A systematic review and a taxonomy of recipience processes. Educational Psychologist, 52(1), 17–37. https://doi.org/10.1080/00461520.2016.1207538
Wu, Y., & Schunn, C. D. (2020). From feedback to revisions: Effects of feedback features and perceptions. Contemporary Educational Psychology, 60, 101826. https://doi.org/10.1016/j.cedpsych.2019.101826
Wang, C., OuYang, J., & Wu, F. (2021). Subgroups of assessor and assessee: The relationship between students’ peer assessment roles and perceptions of MSCL in science education. Journal of Science Education and Technology, 30(6), 816–828.
Yu, S. (2020). Giving genre-based peer feedback in academic writing: Sources of knowledge and skills, difficulties and challenges. Assessment & Evaluation in Higher Education. https://doi.org/10.1080/02602938.2020.1742872
Yu, S., $ Liu, C. (2021). Improving student feedback literacy in academic writing: An evidence-based framework. Assessing Writing, 48, 100525. https://doi.org/10.1016/j.asw.2021.100525
Yuan, R., Liao, W., Wang, Z., Kong, J., & Zhang, Y. (2022). How do English-as-a-foreign-language (EFL) teachers perceive and engage with critical thinking: A systematic review from 2010 to 2020. Thinking Skills and Creativity. https://doi.org/10.1016/j.tsc.2022.101002
Zaidi, N. L. B., Grob, K. L., Monrad, S. M., Kurtz, J. B., Tai, A., Ahmed, A. Z., Gruppen, L. D., & Santen, S. A. (2018). Pushing critical thinking skills with multiple-choice questions: Does bloom’s taxonomy work? Academic Medicine, 93(6), 856–859.
Zhu, Q., & Carless, D. (2018). Dialogue within peer feedback processes: Clarification and negotiation of meaning. Higher Education Research & Development, 37(4), 883–897. https://doi.org/10.1080/07294360.2018.1446417
Zheng, L., Chen, N. S., Cui, P., & Zhang, X. (2019). A systematic review of technology-supported peer assessment research: An activity theory approach. International Review of Research in Open and Distributed Learning, 20(5), 168–191. https://doi.org/10.1973/irrodl.v20i5.4333
Zhang, H., Yuan, R., & He, X. (2020). Investigating University EFL teachers’ perceptions of critical thinking and its teaching: voices from China. The Asia-Pacific Education Researcher. https://doi.org/10.1007/s40299-020-00500-6
Acknowledgements
This work was funded by ANID / CONICYT 1180024 and 2116576.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Ethical approval
The study received approval from the University’s ethics committee (protocol identifier number: 160915003). Before participating in the first peer assessment activity, the students were provided with information on the purpose of the research, the specific activities that were involved. The voluntary nature of participation in the study was also highlighted.
Informed consent
The students were requested to sign an informed consent form.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Below is the link to the electronic supplementary material.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Barahona, C., Nussbaum, M., Martin, V. et al. Technology-scaffolded peer assessment for developing critical thinking in pre-service teacher training: the importance of giving feedback. Education Tech Research Dev 71, 667–688 (2023). https://doi.org/10.1007/s11423-022-10173-1
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11423-022-10173-1