Comparative Judgment Within Online Assessment: Exploring Students Feedback Reactions

  • Anneleen V. Mortier
  • Marije Lesterhuis
  • Peter Vlerick
  • Sven De Maeyer
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 571)


Nowadays, comparative judgment (CJ) emerged as an alternative method for assessing competences and performances (e.g. Pollitt, 2012). In this method, various assessors compare independently several representations of different students and decide each time which of them demonstrate the best performance of the given competence. This study investigated students’ attitudes (honesty, relevancy and trustworthiness) towards feedback that is based upon this method. Additionally, it studied the importance of specific tips in CJ-based feedback.


Online Tool Comparative Judgment Feedback Report Specific Argument Governmental Official 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. Anseel, F., Lievens, F.: The mediating role of feedback acceptance in the relationship between feedback and attitudinal and performance outcomes. Int. J. Sel. Assess. 17(4), 362–376 (2009). doi: 10.1111/j.1468-2389.2009.00479.x CrossRefGoogle Scholar
  2. Bejar, I.I.: Rater cognition: implications for validity. Educ. Meas. Issues Pract. 31(3), 2–9 (2012). doi: 10.1111/j.1745-3992.2012.00238.x CrossRefGoogle Scholar
  3. Bradley, R.A., Terry, M.E.: Rank analysis of incomplete block designs, 1. the method of paired comparisons. Biometrika 39, 324–345 (1952). doi: 10.2307/2334029 zbMATHMathSciNetGoogle Scholar
  4. Bramley, T.: Paired comparisons methods. In: Newton, P., Baird, J.-A., Goldstein, H., Patrick, H., Tymms, P. (eds.) Techniques for Monitoring the Comparability of Examination Standards, pp. 246–294. Qualification and authority, London (2007)Google Scholar
  5. Braun, V., Clarke, V.: Using thematic analysis in psychology. Qual. Res. Psychol. 3(2), 77–102 (2006). doi: 10.1191/1478088706qp063oa CrossRefGoogle Scholar
  6. Crisp, V.: Do assessors pay attention to appropriate features of student work when making assessment judgments? Paper presented at the International Association for Educational Assessment, Annual Conference, Baku, Azerbaijan (2007)Google Scholar
  7. Kimbell, R., Wheeler, T., Stables, K., Shepard, T., Davies, D., Martin, F., Pollitt, A., Whitehouse, G.: E-scape portfolio assessment: phase 3 report. University of London, Goldsmiths College(2009)Google Scholar
  8. Laming, D.: Human Judgment: The Eye of the Beholder. Thomson Learning, London (2004)Google Scholar
  9. Luce, R.D.: Individual Choice Behaviors: A Theoretical Analysis. Wiley, New York (1959)zbMATHGoogle Scholar
  10. Pollitt, A.: Lets stop marking exams. Paper presented at the International Association for Educational Assessment, Annual Conference, Philadelphia, United States of America (2004)Google Scholar
  11. Pollitt, A.: The method of adaptive comparative judgment. Assess. Educ. Principles Policy Pract. 19(3), 1–20 (2012). doi: 10.1080/0969594X.2012.665354 Google Scholar
  12. Sadler, D.R.: Transforming holistic assessment and grading into a vehicle for complex learning. In: Joughin, G. (ed.) Assessment, Learning, and Judgment in Higher Education, pp. 1–19. Springer, Netherlands (2009)CrossRefGoogle Scholar
  13. Thurstone, L.L.: The law of comparative judgment. Psychol. Rev. 34(4), 273–286 (1927). doi: 10.1037/h0070288 CrossRefGoogle Scholar
  14. van der Schaaf, M.F., Stokking, K.M., Verloop, N.: Cognitive representations in raters assessment of teacher portfolios. Stud. Educ. Eval. 31, 27–55 (2005). doi: 10.1016/j.stueduc.2005.02.005 CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Anneleen V. Mortier
    • 1
  • Marije Lesterhuis
    • 2
  • Peter Vlerick
    • 1
  • Sven De Maeyer
    • 2
  1. 1.Department of Personnel Management, Work and Organizational PsychologyGhent UniversityGhentBelgium
  2. 2.EduBROnUniversity of AntwerpAntwerpBelgium

Personalised recommendations