Combinatorial Prediction Markets: An Experimental Study

  • Walter A. Powell
  • Robin Hanson
  • Kathryn B. Laskey
  • Charles Twardy
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8078)

Abstract

Prediction markets produce crowdsourced probabilistic forecasts through a market mechanism in which forecasters buy and sell securities that pay off when events occur. Prices in a prediction market can be interpreted as consensus probabilities for the corresponding events. There is strong empirical evidence that aggregate forecasts tend to be more accurate than individual forecasts, and that prediction markets are among the most accurate aggregation methods. Combinatorial prediction markets allow forecasts not only on base events, but also on conditional events (e.g., “A if B”) and/or Boolean combinations of events. Economic theory suggests that the greater expressivity of combinatorial prediction markets should improve accuracy by capturing dependencies among related questions. This paper describes the DAGGRE combinatorial prediction market and reports on an experimental study to compare combinatorial and traditional prediction markets. The experiment challenged participants to solve a “whodunit” murder mystery by using a prediction market to arrive at group consensus probabilities for characteristics of the murderer, and to update these consensus probabilities as clues were revealed. A Bayesian network was used to generate the “ground truth” scenario and to provide “gold standard" probabilistic predictions. The experiment compared predictions using an ordinary flat prediction market with predictions using a combinatorial market. Evaluation metrics include accuracy of participants’ predictions and the magnitude of market updates. The murder mystery scenario provided a more concrete, realistic, intuitive, believable, and dynamic environment than previous empirical work on combinatorial prediction markets.

Keywords

Combinatorial prediction markets crowdsourcing Bayesian networks combining expert judgment 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Meehl, P.E.: Clinical Versus Statistical Prediction: A Theoretical Analysis and a Review of the Evidence. University of Minnesota Press (1954)Google Scholar
  2. 2.
    Dawes, R., Faust, D., Meehl, P.E.: Clinical Versus Actuarial Judgment. Science 243(4899), 1668–1674 (1989)CrossRefGoogle Scholar
  3. 3.
    Marchese, M.C.: Clinical Versus Actuarial Prediction: a Review of the Literature. Perceptual and Motor Skills 75(2), 583–594 (1992)Google Scholar
  4. 4.
    Grove, W.M., Zald, D.H., Lebow, B.S., Snitz, B.E., Nelson, C.: Clinical Versus Mechanical Prediction: a Meta-Analysis. Psychological Assessment 12(1), 19–30 (2000)CrossRefGoogle Scholar
  5. 5.
    Tetlock, P.: Expert Political Judgment: How Good Is It? How Can We Know? Princeton University Press (2005)Google Scholar
  6. 6.
    Silver, N.: The Signal and the Noise: Why So Many Predictions Fail — but Some Don’t, 1st edn. Penguin Press HC (2012)Google Scholar
  7. 7.
    Surowiecki, J.: The wisdom of crowds. Anchor (2005)Google Scholar
  8. 8.
    Hanson, R.: Combinatorial information market design. Information Systems Frontiers 5(1), 107–119 (2003)CrossRefGoogle Scholar
  9. 9.
    Hanson, R.: Logarithmic market scoring rules for modular combinatorial information aggregation. The Journal of Prediction Markets 1(1), 3–15 (2007)Google Scholar
  10. 10.
    Pennock, D.M., Wellman, M.P.: Graphical models for groups: Belief aggregation and risk sharing. Decision Analysis 3, 148–164 (2005)CrossRefGoogle Scholar
  11. 11.
    Chen, Y., Pennock, D.M.: Designing markets for prediction. AI Magazine 31(4), 42–52 (2010)MathSciNetGoogle Scholar
  12. 12.
    Sun, W., Hanson, R., Laskey, K.B., Twardy, C.: Probability and Asset Updating using Bayesian Networks for Combinatorial Prediction Markets. In: Proceedings of the Twenty-Eighth Conference on Uncertainty in Artificial Intelligence, Catalina Island, USA (2012)Google Scholar
  13. 13.
    Berea, A., Maxwell, D., Twardy, C.: Improving Forecasting Accuracy Using Bayesian Network Decomposition in Prediction Markets. In: Proceedings of the AAAI Fall Symposium Series (2012)Google Scholar
  14. 14.
    Berea, A., Twardy, C.: Automated Trading in Prediction Markets. In: Greenberg, A.M., Kennedy, W.G., Bos, N.D. (eds.) SBP 2013. LNCS, vol. 7812, pp. 111–122. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  15. 15.
    Brier, G.W.: Verification of forecasts expressed in terms of probability. Monthly Weather Review 75, 1–3 (1950)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Walter A. Powell
    • 1
  • Robin Hanson
    • 1
  • Kathryn B. Laskey
    • 1
  • Charles Twardy
    • 1
  1. 1.Volgenau School of EngineeringGeorge Mason UniversityFairfaxUSA

Personalised recommendations