Combinatorial Prediction Markets: An Experimental Study
Prediction markets produce crowdsourced probabilistic forecasts through a market mechanism in which forecasters buy and sell securities that pay off when events occur. Prices in a prediction market can be interpreted as consensus probabilities for the corresponding events. There is strong empirical evidence that aggregate forecasts tend to be more accurate than individual forecasts, and that prediction markets are among the most accurate aggregation methods. Combinatorial prediction markets allow forecasts not only on base events, but also on conditional events (e.g., “A if B”) and/or Boolean combinations of events. Economic theory suggests that the greater expressivity of combinatorial prediction markets should improve accuracy by capturing dependencies among related questions. This paper describes the DAGGRE combinatorial prediction market and reports on an experimental study to compare combinatorial and traditional prediction markets. The experiment challenged participants to solve a “whodunit” murder mystery by using a prediction market to arrive at group consensus probabilities for characteristics of the murderer, and to update these consensus probabilities as clues were revealed. A Bayesian network was used to generate the “ground truth” scenario and to provide “gold standard" probabilistic predictions. The experiment compared predictions using an ordinary flat prediction market with predictions using a combinatorial market. Evaluation metrics include accuracy of participants’ predictions and the magnitude of market updates. The murder mystery scenario provided a more concrete, realistic, intuitive, believable, and dynamic environment than previous empirical work on combinatorial prediction markets.
KeywordsCombinatorial prediction markets crowdsourcing Bayesian networks combining expert judgment
Unable to display preview. Download preview PDF.
- 1.Meehl, P.E.: Clinical Versus Statistical Prediction: A Theoretical Analysis and a Review of the Evidence. University of Minnesota Press (1954)Google Scholar
- 3.Marchese, M.C.: Clinical Versus Actuarial Prediction: a Review of the Literature. Perceptual and Motor Skills 75(2), 583–594 (1992)Google Scholar
- 5.Tetlock, P.: Expert Political Judgment: How Good Is It? How Can We Know? Princeton University Press (2005)Google Scholar
- 6.Silver, N.: The Signal and the Noise: Why So Many Predictions Fail — but Some Don’t, 1st edn. Penguin Press HC (2012)Google Scholar
- 7.Surowiecki, J.: The wisdom of crowds. Anchor (2005)Google Scholar
- 9.Hanson, R.: Logarithmic market scoring rules for modular combinatorial information aggregation. The Journal of Prediction Markets 1(1), 3–15 (2007)Google Scholar
- 12.Sun, W., Hanson, R., Laskey, K.B., Twardy, C.: Probability and Asset Updating using Bayesian Networks for Combinatorial Prediction Markets. In: Proceedings of the Twenty-Eighth Conference on Uncertainty in Artificial Intelligence, Catalina Island, USA (2012)Google Scholar
- 13.Berea, A., Maxwell, D., Twardy, C.: Improving Forecasting Accuracy Using Bayesian Network Decomposition in Prediction Markets. In: Proceedings of the AAAI Fall Symposium Series (2012)Google Scholar