Exploiting User Model Diversity in Forecast Aggregation

Abstract

In many contexts, people generate forecasts about events of interest, and decision-makers wish to aggregate these forecasts to improve their accuracy. These forecasts differ from signals in the physical sciences. In particular, sensor signals are noisy samples from a common underlying distribution, while human-generated forecasts are based on cognitive models that vary from one informant to another. As a result, human forecasts, unlike physical signals, are not guaranteed to be statisticallyindependent conditioned on the true outcome. These differences both provide new opportunities for aggregation, and impose restrictions that do not apply to physical signals. This paper describes the difference between forecasts and physical signals, outlines a strategy for exploiting these differences in aggregation, and demonstrates modest but statistically significant gains in the accuracy of aggregated forecasts using data from a large ongoing experiment in forecasting world events.