An online collaborative peer-assessment approach to strengthening pre-service teachers’ digital content development competence and higher-order thinking tendency

Abstract

The competences of making good use of digital information and technology as well as making critical judgments and communicating with others have been considered as important educational objectives, in particular, for teacher education. Peer assessment is a frequently adopted learning strategy to assist students in rating and offering instant feedback to peers from the perspective of instructors, which has good potential for fostering students’ critical thinking. However, the conventional peer-assessment approach mainly focuses on communications between reviewers and reviewees, while peer communications for collaboratively providing comments or suggestions are generally ignored. As a result, the present study proposed a Collaborative Feedback-based Peer-Assessment (CFPA) learning approach; moreover, a learning system was developed for evaluating the effectiveness of the different collaborative peer-assessment approaches by conducting a quasi-experiment in a pre-service teacher training program. Two classes of students participated in the experiment. One class including 48 students was the experimental group learning with the CFPA approach, while the other class with 49 students was the control group learning with the Non-Collaborative Peer Assessment (NCPA) approach. The findings indicated that the pre-service teachers who learned with the proposed approach showed significantly better instructional video development quality and commenting quality as well as higher self-efficacy and critical thinking tendency than those learning with the Non-Collaborative Peer Assessment approach.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

References

  1. Ananiadou, K., & Claro, M. (2009). 21st century skills and competences for new millennium learners in OECD countries. France.

  2. Beaubien, J. M., & Baker, D. P. (2004). The use of simulation for training teamwork skills in health care: How low can you go? BMJ Quality and Safety, 13(suppl 1), i51–i56.

    Article  Google Scholar 

  3. Boersma, A., ten Dam, G., Volman, M., & Wardekker, W. (2010). This baby… it isn’t alive. Towards a community of learners for vocational orientation. British Educational Research Journal, 36(1), 3–25.

    Article  Google Scholar 

  4. Boud, D. (1990). Assessment and the promotion of academic values. Studies in Higher Education, 15(1), 101–111.

    Article  Google Scholar 

  5. Boud, D. (2000). Sustainable assessment: Rethinking assessment for the learning society. Studies in Continuing Education, 22, 151–167.

    Article  Google Scholar 

  6. Boud, D., & Soler, R. (2016). Sustainable assessment revisited. Assessment and Evaluation in Higher Education, 41(3), 400–413.

    Article  Google Scholar 

  7. Chai, C. S., Deng, F., Tsai, P. S., Koh, J. H. L., & Tsai, C. C. (2015). Assessing multidimensional students’ perceptions of twenty-first-century learning practices. Asia Pacific Education Review, 16(3), 389–398.

    Article  Google Scholar 

  8. Chang, S. C., Hsu, T. C., & Jong, M. S. Y. (2020). Integration of the peer assessment approach with a virtual reality design system for learning earth science. Computers and Education. https://doi.org/10.1016/j.compedu.2019.103758.

    Article  Google Scholar 

  9. Chen, N. S., Wei, C. W., Wu, K. T., & Uden, L. (2009). Effects of high level prompts and peer assessment on online learners’ reflection levels. Computers and Education, 52(2), 283–291.

    Article  Google Scholar 

  10. Cho, Y. H., & Cho, K. (2011). Peer reviewers learn from giving comments. Instructional Science, 39(5), 629–643.

    Article  Google Scholar 

  11. Chou, C. L., Masters, D. E., Chang, A., Kruidering, M., & Hauer, K. E. (2013). Effects of longitudinal small-group learning on delivery and receipt of communication skills feedback. Medical Education, 47(11), 1073–1079.

    Article  Google Scholar 

  12. Choudhury, B., & Gouldsborough, I. (2012). The use of electronic media to develop transferable skills in science students studying anatomy. Anatomical Sciences education, 5(3), 125–131.

    Article  Google Scholar 

  13. Cohen, E. G. (1994). Restructuring the classroom: Conditions for productive small groups. Review of Educational Research, 64(1), 1–35.

    Article  Google Scholar 

  14. Cunningham, C., Odonoghue, G., & Jennings, D. (2016). Introduction of wikis to foster collaboration in health professional education. Focus on Health Professional Education: A Multi-Disciplinary Journal, 17(1), 30–44.

    Article  Google Scholar 

  15. Cuseo, J. (1992). Collaborative and cooperative learning in higher education: A proposed taxonomy. Cooperative Learning and College Teaching, 2(2), 2–4.

    Google Scholar 

  16. Denton, P., Madden, J., Roberts, M., & Rowe, P. (2008). Students’ response to traditional and computer-assisted formative feedback: A comparative case study. British Journal of Educational Technology, 39(3), 486–500.

    Article  Google Scholar 

  17. Down, B., Smyth, J., & Robinson, J. (2019). Problematising vocational education and training in schools: Using student narratives to interrupt neoliberal ideology. Critical Studies in Education, 60(4), 443–461.

    Article  Google Scholar 

  18. Du, J., Wang, C., Zhou, M., Xu, J., Fan, X., & Lei, S. (2018). Group trust, communication media, and interactivity: Toward an integrated model of online collaborative learning. Interactive Learning Environments, 26(2), 273–286.

    Article  Google Scholar 

  19. Elshami, W., & Abdalla, M. E. (2016). Diagnostic radiography students’ perceptions of formative peer assessment within a radiographic technique module. Radiography, 23(1), 9–13.

    Article  Google Scholar 

  20. Falchikov, N. (2007). The place of peers in learning and assessment. Rethinking assessment in higher education: Learning for the longer term, 128–143.

  21. Garrison, D. R., Anderson, T., & Archer, W. (2001). Critical thinking, cognitive presence, and computer conferencing in distance education. American Journal of Distance Education, 15(1), 7–23.

    Article  Google Scholar 

  22. Gielen, M., & De Wever, B. (2015). Structuring the peer assessment process: A multilevel approach for the impact on product improvement and peer feedback quality. Journal of Computer Assisted Learning, 31(5), 435–449.

    Article  Google Scholar 

  23. Girvan, C., & Savage, T. (2010). Identifying an appropriate pedagogy for virtual worlds: A communal constructivism case study. Computers and Education, 55(1), 342–349.

    Article  Google Scholar 

  24. Grice, G. R., Gattas, N. M., Sailors, J., Murphy, J. A., Tiemeier, A. M., Hurd, P. D., & Duncan, W. (2013). Health literacy: Use of the four habits model to improve student pharmacists’ communication. Patient Education and Counseling, 90(1), 23–28.

    Article  Google Scholar 

  25. Harris, L. R., & Brown, G. T. (2013). Opportunities and obstacles to consider when using peer-and self-assessment to improve student learning: Case studies into teachers’ implementation. Teaching and Teacher Education, 36, 101–111.

    Article  Google Scholar 

  26. Hsia, L. H., Huang, I., & Hwang, G. J. (2016). Effects of different online peer-feedback approaches on students’ performance skills, motivation and self-efficacy in a dance course. Computers and Education, 96, 55–71.

    Article  Google Scholar 

  27. Hou, H., Chang, K., & Sung, Y. (2007). An analysis of peer assessment online discussions within a course that uses project-based learning. Interactive Learning Environments, 15(3), 237–251.

    Article  Google Scholar 

  28. Hsu, C. K., & Hwang, G. J. (2014). A context-aware ubiquitous learning approach for providing instant learning support in personal computer assembly activities. Interactive Learning Environments, 22(6), 687–703.

    Article  Google Scholar 

  29. Huang, M. Y., Tu, H. Y., Wang, W. Y., Chen, J. F., Yu, Y. T., & Chou, C. C. (2017). Effects of cooperative learning and concept mapping intervention on critical thinking and basketball skills in elementary school. Thinking Skills and Creativity, 23, 207–216.

    Article  Google Scholar 

  30. Huisman, B., Saab, N., Van Driel, J., & Van Den Broek, P. (2018). Peer feedback on academic writing: Undergraduate students’ peer feedback role, peer feedback perceptions and essay performance. Assessment and Evaluation in Higher Education, 43(6), 955–968.

    Article  Google Scholar 

  31. Hwang, G. J., Tu, N. T., & Wang, X. M. (2018). Creating interactive E-books through learning by design: The impacts of guided peer-feedback on students’ learning achievements and project outcomes in science courses. Journal of Educational Technology and Society, 21(1), 25–36.

    Google Scholar 

  32. Ingeç, ŞK. (2009). Analysing concept maps as an assessment tool in teaching physics and comparison with the achievement tests. International Journal of Science Education, 31(14), 1897–1915.

    Article  Google Scholar 

  33. James, R., McInnis, C., & Devlin, M. (2002). Assessing learning in Australian universities. . Melbourne: The University of Melbourne Centre for the Study of Higher Education.

    Google Scholar 

  34. Johnson, D. W., & Johnson, R. T. (1999). Making cooperative learning work. Theory Into Practice, 38(2), 67–73.

    Article  Google Scholar 

  35. Johnson, J. J., Hrycaiko, D. W., Johnson, G. V., & Halas, J. M. (2004). Self-talk and female youth soccer performance. The Sport Psychologist, 18(1), 44–59.

    Article  Google Scholar 

  36. Kearney, S. (2013). Improving engagement: The use of ‘Authentic self-and peer-assessment for learning to enhance the student learning experience. Assessment and Evaluation in Higher Education, 38(7), 875–891.

    Article  Google Scholar 

  37. Kearney, S., Perkins, T., & Kennedy-Clark, S. (2016). Using self-and peer-assessments for summative purposes: Analyzing the relative validity of the AASL (Authentic Assessment for Sustainable Learning) model. Assessment and Evaluation in Higher Education, 41(6), 840–853.

    Article  Google Scholar 

  38. Khaled, A., Gulikers, J., Biemans, H., & Mulder, M. (2015). How authenticity and self-directedness and student perceptions thereof predict competence development in hands-on simulations. British Educational Research Journal, 41(2), 265–286.

    Article  Google Scholar 

  39. Kollar, I., & Fischer, F. (2010). Peer assessment as collaborative learning: A cognitive perspective. Learning and Instruction, 20(4), 344–348.

    Article  Google Scholar 

  40. Kreijns, K., Kirschner, P. A., & Jochems, W. (2003). Identifying the pitfalls for social interaction in computer-supported collaborative learning environments: A review of the research. Computers in Human Behavior, 19(3), 335–353.

    Article  Google Scholar 

  41. Ku, H. Y., Tseng, H. W., & Akarasriworn, C. (2013). Collaboration factors, teamwork satisfaction, and student attitudes toward online collaborative learning. Computers in Human Behavior, 29(3), 922–929.

    Article  Google Scholar 

  42. Lai, C. L., & Hwang, G. J. (2014). Effects of mobile learning time on students’ conception of collaboration, communication, complex problem-solving, meta-cognitive awareness and creativity. International Journal of Mobile Learning and Organisation, 8(3), 276–291.

    Article  Google Scholar 

  43. Lamprianou, I., & Athanasou, J. A. (2009). A teacher’s guide to educational assessment: Revised Edition. Brill Sense.

  44. Le, H., Janssen, J., & Wubbels, T. (2018). Collaborative learning practices: Teacher and student perceived obstacles to effective student collaboration. Cambridge Journal of Education, 48(1), 103–122.

    Article  Google Scholar 

  45. Li, L., Liu, X., & Steckelberg, A. L. (2010). Assessor or assessee: How student learning improves by giving and receiving peer feedback. British Journal of Educational Technology, 41(3), 525–536.

    Article  Google Scholar 

  46. Li, L., Liu, X., & Zhou, Y. (2012). Give and take: A re-analysis of assessor and assessee’s roles in technology-facilitated peer assessment. British Journal of Educational Technology, 43(3), 376–384.

    Article  Google Scholar 

  47. Liaw, S. S., & Huang, H. M. (2000). Enhancing interactivity in web-based instruction: A review of the literature. Educational Technology, 40(3), 41–45.

    Google Scholar 

  48. Lu, J., & Law, N. (2012). Online peer assessment: Effects of cognitive and affective feedback. Instructional Science, 40(2), 257–275.

    Article  Google Scholar 

  49. Lu, J., & Zhang, Z. (2012). Understanding the effectiveness of online peer assessment: A path model. Journal of Educational Computing Research, 46(3), 313–333.

    Article  Google Scholar 

  50. Lynch, R., Mcnamara, P. M., & Seery, N. (2012). Promoting deep learning in a teacher education programme through self- and peer-assessment and feedback. European Journal of Teacher Education, 35(2), 179–197.

    Article  Google Scholar 

  51. Margaryan, A., Littlejohn, A., & Vojt, G. (2011). Are digital natives a myth or reality? University students’ use of digital technologies. Computers and Education, 56(2), 429–440.

    Article  Google Scholar 

  52. Menon, D., Chandrasekhar, M., Kosztin, D., & Steinhoff, D. C. (2019). Impact of mobile technology-based physics curriculum on preservice elementary teachers’ technology self-efficacy. Science Education. https://doi.org/10.1002/sce.21554.

    Article  Google Scholar 

  53. Nelson, M. M., & Schunn, C. D. (2009). The nature of feedback: How different types of peer feedback affect writing performance. Instructional Science, 37(4), 375–401.

    Article  Google Scholar 

  54. Northrup, P. (2001). A framework for designing interactivity into web-based instruction. Educational Technology, 41(2), 31–39.

    Google Scholar 

  55. Orozco, M., Gijbels, D., & Timmerman, C. (2019). Empirical conceptualisation of integrative learning. A focus on theory-practice integration in technical vocational education and training. Vocations and Learning, 12(3), 1–20.

    Article  Google Scholar 

  56. Osman, G., Duffy, T., Chang, J.-Y., & Lee, J. (2011). Learning through collaboration: Student perspectives. Asia Pacific Education Review, 12(4), 547–558.

    Article  Google Scholar 

  57. Panadero, E. (2016). Is it safe? Social, interpersonal, and human effects of peer assessment. Handbook of Human and Social Conditions in Assessment, 247.

  58. Patchan, M. M., Schunn, C. D., & Clark, R. J. (2018). Accountability in peer assessment: Examining the effects of reviewing grades on peer ratings and peer feedback. Studies in Higher Education, 43(12), 2263–2278.

    Article  Google Scholar 

  59. Piaget, J. (1988). Extracts from Piaget’s theory (G. Gellerier & J. Langer, Trans.). Cognitive development to adolescence: A reader, 3–18.

  60. Pintrich, P.R., Smith, D.A.F., Garcia, T., & McKeachie, W.J. (1991). A manual for the use of the motivated strategies for learning questionnaire (MSLQ). MI: National Center for Research to Improve Postsecondary Teaching and Learning. (ERIC Document Reproduction Service No. ED 338122)

  61. Rotsaert, T., Panadero, E., & Schellens, T. (2018). Anonymity as an instructional scaffold in peer assessment: Its effects on peer feedback quality and evolution in students’ perceptions about peer assessment skills. European Journal of Psychology of Education, 33(1), 75–99.

    Article  Google Scholar 

  62. Saxton, E., Belanger, S., & Becker, W. (2012). The critical thinking analytic rubric (CTAR): Investigating intra-rater and inter-rater reliability of a scoring mechanism for critical thinking performance assessments. Assessing Writing, 17(4), 251–270.

    Article  Google Scholar 

  63. Scardamalia, M., & Bereiter, C. (1994). Computer support for knowledge-building communities. The Journal of the Learning Sciences, 3(3), 265–283.

    Article  Google Scholar 

  64. Schellens, T., & Valcke, M. (2005). Collaborative learning in asynchronous discussion groups: What about the impact on cognitive processing? Computers in Human Behavior, 21(6), 957–975.

    Article  Google Scholar 

  65. Segers, M., Dochy, F., & Cascallar, E. (2003). The era of assessment engineering: Changing perspectives on teaching and learning and the role of new modes of assessment. Optimising new modes of assessment: In search of qualities and standards. (pp. 1–12). Dordrecht: Springer.

    Google Scholar 

  66. Sun, J. C. Y., & Chen, A. Y. Z. (2016). Effects of integrating dynamic concept maps with interactive response system on elementary school students’ motivation and learning outcome: The case of anti-phishing education. Computers and Education, 102, 117–127.

    Article  Google Scholar 

  67. Tam, M. (2000). Constructivism, instructional design, and technology: Implications for transforming distance learning. Educational Technology and Society, 3(2), 50–60.

    Google Scholar 

  68. Tondeur, J., Aesaert, K., Prestridge, S., & Consuegra, E. (2018). A multilevel analysis of what matters in the training of pre-service teacher’s ICT competencies. Computers and Education, 122, 32–42.

    Article  Google Scholar 

  69. Tondeur, J., van Braak, J., & Valcke, M. (2007). Towards a typology of computer use in primary education. Journal of Computer Assisted Learning, 23(3), 197–206.

    Article  Google Scholar 

  70. Topping, K. J. (2010). Methodological quandaries in studying process and outcomes in peer assessment. Learning and Instruction, 20(4), 339–343.

    Article  Google Scholar 

  71. Tricio, J. A., Woolford, M. J., & Escudier, M. P. (2016). Fostering dental students’ academic achievements and reflection skills through clinical peer assessment and feedback. Journal of dental education, 80(8), 914–923.

    Article  Google Scholar 

  72. Usher, M., & Barak, M. (2018). Peer assessment in a project-based engineering course: Comparing between on-campus and online learning environments. Assessment and Evaluation in Higher Education, 43(5), 745–759.

    Article  Google Scholar 

  73. van den Bos, A. H., & Tan, E. (2019). Effects of anonymity on online peer review in second-language writing. Computers and Education. https://doi.org/10.1016/j.compedu.2019.103638.

    Article  Google Scholar 

  74. van Gennip, N. A., Segers, M. S., & Tillema, H. H. (2009). Peer assessment for learning from a social perspective: The influence of interpersonal variables and structural features. Educational Research Review, 4(1), 41–54.

    Article  Google Scholar 

  75. Van Gennip, N. A., Segers, M. S., & Tillema, H. H. (2010). Peer assessment as a collaborative learning activity: The role of interpersonal variables and conceptions. Learning and Instruction, 20(4), 280–290.

    Article  Google Scholar 

  76. Van Popta, E., Kral, M., Camp, G., Martens, R. L., & Simons, P. R. J. (2017). Exploring the value of peer feedback in online learning for the provider. Educational Research Review, 20, 24–34.

    Article  Google Scholar 

  77. Wang, X., & Mu, J. (2017). Introduction to collaboration scripts. Flexible scripting to facilitate knowledge construction in computer-supported collaborative learning. (pp. 13–24). Singapore: Springer.

    Google Scholar 

  78. Yu, F. Y., & Wu, C. P. (2013). Predictive effects of online peer feedback types on performance quality. Journal of Educational Technology and Society, 16(1), 332–341.

    Google Scholar 

  79. Yu, F. Y., & Wu, C. P. (2016). Predictive effects of the quality of online peer-feedback provided and received on primary school students’ quality of question-generation. Educational Technology and Society, 19(3), 234–246.

    Google Scholar 

  80. Zheng, L., Cui, P., Li, X., & Huang, R. (2018). Synchronous discussion between assessors and assessees in web-based peer assessment: Impact on writing performance, feedback quality, meta-cognitive awareness and self-efficacy. Assessment and Evaluation in Higher Education, 43(3), 500–514.

    Article  Google Scholar 

Download references

Acknowledgements

This study is supported in part by the Ministry of Science and Technology of the Republic of China under contract number MOST-108-2511-H-011-005-MY3. This study is also supported by the Education Department of Zhejiang Province, the People’s Republic of China with project title “An Empirical Study on the Peer Assessment Strategies to Improve Pre-service Teachers’ Informational Teaching Ability” (Contract Number: Y201941092) as well as the Education Science Planning Department of Zhejiang Province, the People’s Republic of China with project title “Exploring the reform of linking employment and university students’ science and innovation in local universities” (Contract Number: 2014SCG341). The authors would like to declare that there is no conflict of interest in this study.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Gwo-Jen Hwang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Fang, JW., Chang, SC., Hwang, GJ. et al. An online collaborative peer-assessment approach to strengthening pre-service teachers’ digital content development competence and higher-order thinking tendency. Education Tech Research Dev 69, 1155–1181 (2021). https://doi.org/10.1007/s11423-021-09990-7

Download citation

Keywords

  • Peer assessment
  • Collaborative learning
  • Pre-service teacher
  • Higher-order thinking tendency
  • Commenting quality