Peering Inside Peer Review with Bayesian Models

  • Ilya M. Goldin
  • Kevin D. Ashley
Conference paper

DOI: 10.1007/978-3-642-21869-9_14

Volume 6738 of the book series Lecture Notes in Computer Science (LNCS)
Cite this paper as:
Goldin I.M., Ashley K.D. (2011) Peering Inside Peer Review with Bayesian Models. In: Biswas G., Bull S., Kay J., Mitrovic A. (eds) Artificial Intelligence in Education. AIED 2011. Lecture Notes in Computer Science, vol 6738. Springer, Berlin, Heidelberg

Abstract

Instructors and students would benefit more from computer-supported peer review, if instructors received information on how well students have understood the conceptual issues underlying the writing assignment. Our aim is to provide instructors with an evaluation of both the students and the criteria that students used to assess each other. Here we develop and evaluate several hierarchical Bayesian models relating instructor scores of student essays to peer scores based on two peer assessment rubrics. We examine model fit and show how pooling across students and different representations of rating criteria affect model fit and how they reveal information about student writing and assessment criteria. Finally, we suggest how our Bayesian models may be used by an instructor or an ITS.

Keywords

computer-supported peer review evaluation of assessment criteria Bayesian models 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Ilya M. Goldin
    • 1
  • Kevin D. Ashley
    • 1
  1. 1.Intelligent Systems Program and Learning Research and Development CenterUniversity of PittsburghPittsburghUSA