Application of Fuzzy Ordinal Peer Assessment in Formative Evaluation

  • Nicola CapuanoEmail author
  • Francesco Orciuoli
Conference paper
Part of the Lecture Notes on Data Engineering and Communications Technologies book series (LNDECT, volume 13)


Peer assessment has been used for many years as a tool to improve learning outcomes but, only recently, it is becoming an increasingly used support also in students evaluation. Many approaches have been proposed so far to make peer assessment as reliable as possible even in case of incorrect or inaccurate evaluations proposed by students. Among these approaches, Fuzzy Ordinal Peer Assessment (FOPA) relies on ordinal evaluations (rather than cardinal ones) and on the application of models coming from Fuzzy Set Theory and Group Decision Making. FOPA has already demonstrated good results in in-silico experiments. To complement these results, in the work presented in this paper, we experiment the same model in a University context to support formative evaluation. Obtained results show better performance of FOPA with respect to competitor models and a general attitude of peer assessment models to approximate instructor ratings.


  1. 1.
    Black, P., Wiliam, D.: Assessment for learning in the classroom. In: Assessment and Learning, pp. 9–15. SAGE Publications (2006)Google Scholar
  2. 2.
    Bransford, J.D., Brown, A., Cocking, R.: How People Learn: Mind, Brain, Experience and School. National Academy Press, Washington, DC (2000)Google Scholar
  3. 3.
    Capuano, N., Caballé, S., Miguel, J.: Improving peer grading reliability with graph mining techniques. Int. J. Emerg. Technol. Learn. 11(7), 24–33 (2016)CrossRefGoogle Scholar
  4. 4.
    Sadler, P.M., Good, E.: The impact of self- and peer-grading on student learning. Educ. Assess. 11(1), 1–31 (2006)CrossRefGoogle Scholar
  5. 5.
    Glance, D.G., Forsey, M., Riley, M.: The pedagogical foundations of massive open online courses. First Monday 18(5) (2013)Google Scholar
  6. 6.
    Capuano, N., Caballé, S.: Towards adaptive peer assessment for MOOCs. In: Proceedings of the 10th International Conference on P2P, Parallel, GRID, Cloud and Internet Computing (3PGCIC 2015), Krakow, Poland (2015)Google Scholar
  7. 7.
    Bouzidi, L., Jaillet, A.: Can online peer assessment be trusted? Educ. Technol. Soc. 12(4), 257–268 (2009)Google Scholar
  8. 8.
    Raman, K., Joachims, T.: Methods for ordinal peer grading. In: Proceedings of the 20th SIGKDD International Conference on Knowledge Discovery and Data Mining (2014)Google Scholar
  9. 9.
    Capuano, N., Loia, V., Orciuoli, F.: A fuzzy group decision making model for ordinal peer assessment. IEEE Trans. Learn. Technol. 10(2), 247–259 (2017)CrossRefGoogle Scholar
  10. 10.
    Carlson, P.A., Berry, F.C.: Calibrated peer review™ and assessing learning outcomes. In: Proceedings of the 33rd International Conference Frontiers in Education (2003)Google Scholar
  11. 11.
    Piech, C., Huang, J., Chen, Z., Do, C., Ng, A., Koller, D.: Tuned models of peer assessment in MOOCs. In: Proceedings of the 6th International Conference on Educational Data Mining (2013)Google Scholar
  12. 12.
    Goldin, I.M.: Accounting for peer reviewer bias with bayesian models. In: Proceedings of the 11th International Conference on Intelligent Tutoring Systems (2012)Google Scholar
  13. 13.
    Walsh, T.: The peerrank method for peer assessment. In: Proceedings of the 21st European Conference on Artificial Intelligence (2014)Google Scholar
  14. 14.
    Albano, G., Capuano, N., Pierri, A.: Adaptive peer grading and formative assessment. J. e-Learn. Knowl. Soc. 13(1), 147–161 (2017)Google Scholar
  15. 15.
    Caragiannis, I., Krimpas, A., Voudouris, A.A.: Aggregating partial rankings with applications to peer grading in massive online open courses. In: Proceedings of the International Conference on Autonomous Agents and Multiagent Systems, Istanbul (2015)Google Scholar
  16. 16.
    Borda, J.C.: Memoire sur les elections au scrutin. Histoire de l’Académie Royale des Sciences (1781)Google Scholar
  17. 17.
    Mallows, C.L.: Non-null ranking models. I. Biometrika 44(1), 114 (1957)CrossRefzbMATHMathSciNetGoogle Scholar
  18. 18.
    Bradley, R.A., Terry, M.E.: Rank analysis of incomplete block designs: I. The method of paired comparisons. Biometrika 39(3), 324 (1952)zbMATHMathSciNetGoogle Scholar
  19. 19.
    Plackett, R.L.: The analysis of permutations. Appl. Stat. 24(2), 193 (1975)CrossRefMathSciNetGoogle Scholar
  20. 20.
    Capuano, N., Chiclana, F., Fujita, H., Herrera-Viedma, E., Loia, V.: Fuzzy group decision making with incomplete information guided by social influence. IEEE Trans. Fuzzy Syst. PP(99), 1 (2017)Google Scholar

Copyright information

© Springer International Publishing AG 2018

Authors and Affiliations

  1. 1.Department of Information Engineering, Electric Engineering and Applied MathematicsUniversity of SalernoFiscianoItaly
  2. 2.Department of Business Sciences, Management and Innovation SystemsUniversity of SalernoFiscianoItaly

Personalised recommendations