Psychometrika

, Volume 74, Issue 3, pp 531–553

Why Are Experts Correlated? Decomposing Correlations Between Judges

Theory and Methods

Abstract

We derive an analytic model of the inter-judge correlation as a function of five underlying parameters. Inter-cue correlation and the number of cues capture our assumptions about the environment, while differentiations between cues, the weights attached to the cues, and (un)reliability describe assumptions about the judges. We study the relative importance of, and interrelations between these five factors with respect to inter-judge correlation. Results highlight the centrality of the inter-cue correlation. We test the model’s predictions with empirical data and illustrate its relevance. For example, we show that, typically, additional judges increase efficacy at a greater rate than additional cues.

Keywords

information aggregation correlation dependence expert advice 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Copyright information

© The Psychometric Society 2009

Authors and Affiliations

  1. 1.Department of PsychologyUniversity of IllinoisChampaignUSA
  2. 2.Department of PsychologyFordham UniversityThe BronxUSA

Personalised recommendations