Peering Inside Peer Review with Bayesian Models
- Cite this paper as:
- Goldin I.M., Ashley K.D. (2011) Peering Inside Peer Review with Bayesian Models. In: Biswas G., Bull S., Kay J., Mitrovic A. (eds) Artificial Intelligence in Education. AIED 2011. Lecture Notes in Computer Science, vol 6738. Springer, Berlin, Heidelberg
Instructors and students would benefit more from computer-supported peer review, if instructors received information on how well students have understood the conceptual issues underlying the writing assignment. Our aim is to provide instructors with an evaluation of both the students and the criteria that students used to assess each other. Here we develop and evaluate several hierarchical Bayesian models relating instructor scores of student essays to peer scores based on two peer assessment rubrics. We examine model fit and show how pooling across students and different representations of rating criteria affect model fit and how they reveal information about student writing and assessment criteria. Finally, we suggest how our Bayesian models may be used by an instructor or an ITS.
Keywordscomputer-supported peer review evaluation of assessment criteria Bayesian models
Unable to display preview. Download preview PDF.