The Group Calibration Index: a group-based approach for assessing forecasters’ expertise when external outcome data are missing
- 86 Downloads
The Group Calibration Index (GCI) provides a means of assessing the quality of forecasters’ predictions in situations that lack external feedback or outcome data. The GCI replaces the missing outcome data with aggregated ratings of a well-defined reference group. A simulation study and two experiments show how the GCI classifies forecaster performance and distinguishes between forecasters with restricted information and those with complete information. The results also show that under certain circumstances, where members of the reference group have high-quality information, the new GCI will outperform expert classification that is based on traditional calibration indices.
KeywordsGroup decisions Experts Calibration Uncertainty
Unable to display preview. Download preview PDF.
- Dawes R. M. (1994) House of cards: Psychology and psychotherapy built of myth. Free Press, New YorkGoogle Scholar
- Harvey N., Fischer I. (2005) Development of experience-based judgment and decision making: The role of Outcome Feedback. In: Betsch T., Haberstroh S. (eds) The routines of decision making. Erlbaum, Mahwah, NJ, pp 119–137Google Scholar
- Oskamp, S. (1965). Overconfidence in case-study judgments. The Journal of Consulting Psychology, 29, 261–265. Reprinted in: Kahneman, D., Slovic, P., Tversky, A. (1982). Judgment under Uncertainty, 287–293. (Cambridge: Cambridge University Press).Google Scholar
- Rowe G., Wright G. (2001) Expert opinions in Forecasting: The Role of the Delphi Technique. In: Armstrong J. S. (eds) Principles of forecasting—a handbook for researchers and practitioners. Kluwer Academic Publishers, Boston, pp 125–144Google Scholar
- Stevens S. S. (1975) Psychophysics: Introduction to its perceptual, neural, and social prospects. John Wiley & Sons, New York, pp 13–36Google Scholar
- Vincenz, S. (1955). Bałaguły, On the High Uplands, translated by H. C. Stevens (pp. 248–267). New York: RoyGoogle Scholar