Why Are Experts Correlated? Decomposing Correlations Between Judges
- 230 Downloads
We derive an analytic model of the inter-judge correlation as a function of five underlying parameters. Inter-cue correlation and the number of cues capture our assumptions about the environment, while differentiations between cues, the weights attached to the cues, and (un)reliability describe assumptions about the judges. We study the relative importance of, and interrelations between these five factors with respect to inter-judge correlation. Results highlight the centrality of the inter-cue correlation. We test the model’s predictions with empirical data and illustrate its relevance. For example, we show that, typically, additional judges increase efficacy at a greater rate than additional cues.
Keywordsinformation aggregation correlation dependence expert advice
Unable to display preview. Download preview PDF.
- Budescu, D.V. (2006). Confidence in aggregation of opinions from multiple sources. In K. Fiedler & P. Juslin (Eds.), Information sampling and adaptive cognition (pp. 327–354). Cambridge: Cambridge University Press. Google Scholar
- Hammond, K.R., & Stewart, T.R. (2001). The essential Brunswik: beginnings, explications, application. London: Oxford University Press. Google Scholar
- Miller, S. (2008). Supporting joint human-computer judgment under uncertainty. Unpublished Dissertation at the University of Illinois at Urbana-Champaign. Google Scholar
- Shanteau, J. (2001). What does it mean when experts disagree? In E. Salas & G. Klein (Eds.), Linking expertise and naturalistic decision making. Earlbaum: Mahwa. Google Scholar
- Weiss, D.J., & Shanteau, J. (2003a). The vice of consensus and the virtue of consistency. In J. Shanteau, P. Johnson, & C. Smith (Eds.), Psychological explorations of competent decision making. Cambridge: Cambridge University Press. Google Scholar