Towards an Annotation System for Collaborative Peer Review

  • Sebastian MaderEmail author
  • François Bry
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 1007)


Peers providing feedback on their peers’ work is called peer review which has been shown to have beneficial effects on students’ learning. This article presents a novel approach to peer review where reviewers, reviewees, and lecturers alike have access to the same documents, and reviews are created in form of annotations which are automatically shared among all users, can be up- or downvoted, and can themselves be commented on. Working on a same document and seeing annotations immediately after they have been created enables various forms of collaboration among learners: Between reviewers who can agree or disagree with a review by up- or downvoting or comment on the review thus providing further insight, between reviewers and reviewees by allowing the reviewees to inquire about reviews and reviewers to provide clarifications and justifications, and between lecturers and reviewers, when lecturers take the role of reviewers and up- or downvote and comment on reviews. The contributions of this article are twofold: First, a report on the conception and implementation of a collaborative annotation environment that supports the collaborative peer review described in this article, and second, an analysis of the communication that happened during collaborative peer review in three courses pointing to the approach’s effectiveness but uncovering problems of the collaborative annotation system as well.


Peer review Computer-supported collaborative learning Virtual learning environments 



The authors are thankful to Nikolai Gruschke for implementing the notification mechanisms described in Sect. 5 as part of his bachelor’s thesis.


  1. 1.
    Ashenafi, M.M.: Peer-assessment in higher education-twenty-first century practices, challenges and the way forward. Assess. Eval. High. Educ. 42(2), 226–251 (2017)Google Scholar
  2. 2.
    Cho, K., Schunn, C.D.: Scaffolded writing and rewriting in the discipline: a web-based reciprocal peer review system. Comput. Educ. 48(3), 409–426 (2007)Google Scholar
  3. 3.
    Cockburn, A., Karlson, A., Bederson, B.B.: A review of overview+ detail, zooming, and focus+ context interfaces. ACM Comput. Surv. (CSUR) 41(1), 2 (2009)Google Scholar
  4. 4.
    Glover, I., Hardaker, G., Xu, Z.: Collaborative annotation system environment (CASE) for online learning. Campus-Wide Inf. Syst. 21(2), 72–80 (2004)Google Scholar
  5. 5.
    Hiltz, S.R., Turoff, M.: Structuring computer-mediated communication systems to avoid information overload. Commun. ACM 28(7), 680–689 (1985)Google Scholar
  6. 6.
    Howard, C.D., Barrett, A.F., Frick, T.W.: Anonymity to promote peer feedback: pre-service teachers’ comments in asynchronous computer-mediated communication. J. Educ. Comput. Res. 43(1), 89–112 (2010)Google Scholar
  7. 7.
    Jonsson, A., Svingby, G.: The use of scoring rubrics: reliability, validity and educational consequences. Educ. Res. Rev. 2(2), 130–144 (2007)Google Scholar
  8. 8.
    Linden, G., Smith, B., York, J.: recommendations: item-to-item collaborative filtering. IEEE Internet Comput. 1, 76–80 (2003)Google Scholar
  9. 9.
    Nicol, D., Thomson, A., Breslin, C.: Rethinking feedback practices in higher education: a peer review perspective. Assess. Eval. High. Educ. 39(1), 102–122 (2014)Google Scholar
  10. 10.
    Nokelainen, P., Miettinen, M., Kurhila, J., Floréen, P., Tirri, H.: A shared document-based annotation tool to support learner-centred collaborative learning. Br. J. Educ. Technol. 36(5), 757–770 (2005)Google Scholar
  11. 11.
    Paas, F., Renkl, A., Sweller, J.: Cognitive load theory and instructional design: recent developments. Educ. Psychol. 38(1), 1–4 (2003)Google Scholar
  12. 12.
    Popham, W.J.: What’s wrong-and what’s right-with rubrics. Educ. Leadersh. 55, 72–75 (1997)Google Scholar
  13. 13.
    Schriver, K.A.: Evaluating text quality: the continuum from text-focused to reader-focused methods. IEEE Trans. Prof. Commun. 32(4), 238–255 (1989)Google Scholar
  14. 14.
    Stahl, G., Koschmann, T.D., Suthers, D.D.: Computer-supported collaborative learning (2006)Google Scholar
  15. 15.
    Topping, K.: Peer assessment between students in colleges and universities. Rev. Educ. Res. 68(3), 249–276 (1998)Google Scholar
  16. 16.
    Turoff, M.: Computer-mediated communication requirements for group support. J. Organ. Comput. Electron. Commer. 1(1), 85–113 (1991)Google Scholar
  17. 17.
    Weng, C., Gennari, J.H.: Asynchronous collaborative writing through annotations. In: Proceedings of the 2004 ACM Conference on Computer Supported Cooperative Work, pp. 578–581. ACM (2004)Google Scholar
  18. 18.
    Yang, S.J., Zhang, J., Su, A.Y., Tsai, J.J.: A collaborative multimedia annotation tool for enhancing knowledge sharing in CSCL. Interact. Learn. Environ. 19(1), 45–62 (2011)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Institute for InformaticsLudwig Maximilian University of MunichMunichGermany

Personalised recommendations