Peer Review in Mentorship: Perception of the Helpfulness of Review and Reciprocal Ratings

  • Oluwabunmi Adewoyin
  • Roberto Araya
  • Julita Vassileva
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9684)


Peer review is the main mechanism for quality evaluation and peer-mentoring in the research community. Yet, it has been criticized with respect to its summative function, as being prone to bias and inconsistency and approaches had been proposed to improve it (e.g. double blind review). However, relatively less attention has been paid on how well it meets its formative objective, i.e. providing useful feedback to help the authors improve their quality of work. In our previous work we proposed a modified peer review process, which involved a back-evaluation of reviews by the authors. This paper reports the results of a study of the application of this peer review process to support a group of teachers in Chile engage in group peer mentorship in the context of a summer continuing education course. The objectives are to find out if authors reciprocate their reviews feedback in the back-evaluation given to their reviewers, and if the review length affects the helpfulness and authors’ satisfaction with the reviews. Our results showed that peers did not reciprocate their ratings and review length did not affect peers’ satisfaction with the reviews.


Peer review Continuing education Collaborative learning Peer mentorship 



We thank Paulina Sepúlveda for implementing the peer review system and to Abelino Jiménez and Josefina Hernández for initial data preprocessing. We thank Francisco Gutierrez for his help with the translation of user comments from Spanish to English. This research was possible with funding from NSERC Discovery Grants Program to the 3rd author and from Basal Funds for Centers of Excellence Project BF 0003 from the Associative Research Program of CONICYT to the 2nd author.


  1. 1.
    Adewoyin, O., Vassileva, J.: Can online peer-review systems support group mentorship? In: Lane, H., Yacef, K., Mostow, J., Pavlik, P. (eds.) AIED 2013. LNCS, vol. 7926, pp. 737–741. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  2. 2.
    Adewoyin, O., Vassileva, J.: Ethics of scientific peer review: are we judging or helping the review recipients? In: IEEE International Symposium on Ethics in Engineering, Science, and Technology, Ethics 2014 (2014)Google Scholar
  3. 3.
    Cestone, C.M., Levine, R.E., Lane, D.R.: Peer assessment and evaluation in team-based learning. New Directions Teaching Learn. 116, 69–78 (2008)CrossRefGoogle Scholar
  4. 4.
    Farrell, K.: Collegial feedback on teaching: a guide to peer review. Centre for the Study of Higher Education, The University of Melbourne (2011)Google Scholar
  5. 5.
    Fernandez, C.E., Yu, J.: Peer review of teaching. J. Chiropractic Educ. 21(2), 154–161 (2007)CrossRefGoogle Scholar
  6. 6.
    Gutknecht-Gmeiner, M.: Peer review in education. Peer review in initial VET, Leornado da Vinci project, Austria (2005)Google Scholar
  7. 7.
    Houry, D., Green, S., Callaham, M.: Does mentoring new peer reviewers improve review quality? A randomized trial. BMC Med. Educ. 12(1), 83 (2012)CrossRefGoogle Scholar
  8. 8.
    Hutchings, P.: Peer review of teaching. “From Idea to Prototype”. In: AAHE Bulletin, November 1994Google Scholar
  9. 9.
    Kronick, D.A.: Peer review in 18th century scientific journalism. J. Am. Medical Assoc. (JAMA) 263, 1321–1322 (1990)CrossRefGoogle Scholar
  10. 10.
    Levine, R.E.: Peer evaluation in team-based learning (2010). (Accessed on 17 October 2015)
  11. 11.
    Pearce, J., Mulder, R., Baik, C.: Involving students in peer review: case studies and practical strategies for University teaching. University of Melbourne, Victoria (2009). (Accessed on 17 October 2015)
  12. 12.
    Ranalli, B.: A prehistory of peer review: religious blueprints from the hartlib circle spontaneous generations. J. History Philos. Sci. 5(1), 12–18 (2011)MathSciNetGoogle Scholar
  13. 13.
    Sachs, J., Parsell, M.: Peer review of learning and teaching in higher education. British J. Educ. Technol. 45(3) (2014)Google Scholar
  14. 14.
    Schroter, S., Black, N., Evans, S., Carpenter, J., Godlee, F., Smith, R.: Effects of training on quality of peer review: randomised controlled trial. BMJ 2004(328), 673 (2004)CrossRefGoogle Scholar
  15. 15.
    Searby, M., Ewers, T.: An evaluation of the use of peer assessment in higher education: a case study in the school of music. Assess. Eval. Higher Educ. 22(4), 371–383 (1997). Kingston UniversityCrossRefGoogle Scholar
  16. 16.
    Smith, R.: Peer review: a flawed process at the heart of science and journals. J. R. Soc. Med. 99, 178–182 (2006)CrossRefGoogle Scholar
  17. 17.
    Turner, S.; Perez-Quinones, M.A., Chase, J.: Peer review in CS2: conceptual learning. In: SIGCSE 2010 Proceedings of the 41st ACM Technical Symposium on Computer Science Education, pp. 331–335Google Scholar
  18. 18.
    Van Rooyen, S., Godlee, F., Evans, S., Black, N., Smith, R.: Effect of open peer review on quality of reviews and on reviewers’ recommendations: a randomised trial. Br. Med. J. 318, 23–27 (1999)CrossRefGoogle Scholar
  19. 19.
    Vygotsky, L.S.: Mind in Society: The Development of Higher Psychological Processes. Harvard University Press, Cambridge (1978)Google Scholar
  20. 20.
    Wagner, W., Steinzor, R.: Rescuing Science from Politics Regulation and the Distortion of Scientific Research, 1st edn. Cambridge University Press, Cambridge (2006)CrossRefGoogle Scholar
  21. 21.
    Xiong, W., Litman, D., Schunn, C.: Assessing reviewers’ performance based on mining problem localization in peer-review data. In: Proceedings of the Third International Conference on Educational Data Mining (EDM 2010), pp. 211–220 (2010)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Oluwabunmi Adewoyin
    • 1
  • Roberto Araya
    • 2
  • Julita Vassileva
    • 1
  1. 1.University of SaskatchewanSaskatoonCanada
  2. 2.CIAE University of ChileSantiagoChile

Personalised recommendations