Abstract
Field tests were conducted on two new information aggregation mechanism designs. The mechanisms were designed to collect information held as intuitions about opening weekend box office revenues for movies in Australia. The principles on which the mechanisms operate and their capacity to collect information are explored. A pari-mutuel mechanism produces a predicted probability distribution over box office amounts that is, with the exception of very small films, indistinguishable from the actual revenues. The second mechanism is based on guessing the guesses of others and when applied under conditions where incentives for accuracy are unavailable still performs well against data.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
The number of buckets is dictated by screen size. The size of the buckets is determined by how the film is classified. “Art House” films have the smallest buckets, followed by “Regular” and then “Blockbuster” which is the largest. The movies, classifications, and buckets are discussed in Sect. 5.
The implications of heterogeneous preferences over outcomes and incentives for outcome manipulation within market systems is addressed by Rausser et al. (2015).
This idea was motivated by insights contained in Prelec (2004) and Weaver and Prelec (2013). The property could be a consequence of an “availability heuristic” or, alternatively a “recognition heuristic.” The idea also appears as a substantive principle in “false consensus” research. We use the property axiomatically and take no stand on competing explanations or the conditions under which it might be reliable as a model.
Papers by Axelrod et al. (2009) and Plott and Roust (2009) added both understanding and features. Axelrod et al. (2009) demonstrated that the addition of a time clock and an increasing price of pari-mutuel tickets would increase the speed with which information flowed into the system. Plott and Roust (2009) demonstrated that poor information aggregation was related to weak signals. When the signals are weak, information aggregation is poor, primarily because risk aversion prevented agents acting on poor information. That information exists to be reflected in the decisions of others is not revealed. Those two studies lead to the major features of the architecture implemented in Intel and in Boxoffice Prophecy. More recently the results of Kalovcova and Ortmann (2009) and Koessler et al. (2012) have added depth to the understanding.
Foutz and Jank (2010) is an exception who investigate the price path to assist in forecasts of box office revenues.
We compute the Mean BOP Forecast with a weighted average of the value assigned to each bin, weighted by the number of tickets in that bin. Specifically, if the kth bin pays off if revenues are between \(x_{k-1}\) and \(x_{k}\), we assign bin k the value of \(v_{k} = (x_{k-1}+x_{k})/2\). Since bins are equally spaced, we assign the first bin a value of (\(x_{1} + (x_{1} - (x_{2 }-x_{1})))/2\) and treat the last bin symmetrically. There are several different ways to label these extreme bins and our results are robust to reasonable treatments (obviously, labeling the last bin as having an infinite value would be problematic). Given these defined values, if the kth bin has \(\eta _{k}\) tickets in it, then the Mean BOP Forecast is simply:
$$\begin{aligned} \hbox {BOP Mean}={\frac{1}{{\sum \nolimits _{k=1}^{1}} \eta _k}} {\mathop {\sum }\limits _{k=1}^{k}} \eta _k v_k \end{aligned}$$.
We drop 10 observations due to issues related to censoring. For example, Art House films that ended up in the lowest bucket when the buckets were designed too large ex ante. Dropped titles are noted in “Appendix 1” table.
Our application of the Kolmogorov–Smirnoff test represents a calibration test that could be manipulated if the IAM were to report a uniform distribution over sales for all movies, an outcome that was not observed in our sample. Technically, the p values for the Kolmogorov–Smirnoff test implicitly assumes independence across sessions, which is unlikely to hold in the current application. Unfortunately, testing distributional equivalence with serial dependence and heteroscedasticity presents an open question in statistical hypothesis testing that is beyond the scope of our current analysis.
The Long-Shot Bias arises when market odds overstate the likelihood of low-probability events, an issue that has been discussed extensively in the literature (including four chapters in the Handbook of Sports and Lottery Markets (Hausch and Ziemba (2008)). Researchers have also observed the opposite pattern, a Reverse Long-Shot Bias, which presents as market odds overstating the likelihood of high-probability events and understating the probability of low-probability events. A number of strategic or behavioral features of prediction markets might drive these phenomena, including risk aversion (Jullien and Salanie 2000), probability weighting (Snowberg and Wolfers 2010), heterogeneous beliefs (Gandhi and Serrano-Padial 2012), and strategic models (Ottaviani and Sørensen 2010).
References
Asur, S., Huberman, B.A.: Predicting the future with social media. In: Proceedings of the 2010 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology, pp. 492–499 (2010)
Axelrod, B.S., Kulick, B.J., Plott, C.R., Roust, K.A.: The design of improved parimutuel-type information aggregation mechanisms: inaccuracies and the long-shot bias as disequilibrium phenomena. J. Econ. Behav. Organ. 69(2), 170–181 (2009)
Berg, J., Forsythe, R., Nelson, F., Rietz, T.: Results from a Dozen Years of Election Futures Markets Research, Volume 1 of Handbook of Experimental Economics Results, Chapter 80. Elsevier, pp. 742–751 (2008)
Chen, K.Y., Plott, C.R.: Information Aggregation Mechanisms: Concept, Design, and Implementation for a Sales Forecasting Problem, Mimeo (2002)
De Vany, A.S.: Hollywood Economics: How Extreme Uncertainty Shapes the Film Industry. Routledge, London (2004)
Diebold, F.X., Mariano, R.S.: Comparing predictive accuracy. J. Bus. Econ. Stat. 13(3), 253–263 (1995)
Doshi, L., Krauss, J., Nann, S., Glor, P.: Predicting movie prices through dynamic social network analysis. Proc. Soc. Behav. Sci. 2, 6423–6433 (2010)
Eliashberg, J., Elberse, A., Leenders, M.: The motion picture industry: critical issues in practice, current research, and new research directions. Mark. Sci. 25(6), 638–661 (2006)
Foster, D.P., Vohra, R.: Asymptotic calibration. Biometrika 85(2), 379–390 (1998)
Foutz, N., Jank, W.: The wisdom of crowds: pre-release forecasting via functional shape analysis of the online virtual stock market. Mark. Sci. 29(3), 568–579 (2010)
Fair, R.C., Shiller, R.J.: Comparing information in forecasts from econometric models. Am. Econ. Rev. 80(3), 375–389 (1990)
Gandhi, A., Serrano-Padial, R.: Are Beliefs Heterogeneous? The Case of the Longshot Bias. Working paper (2012)
Gillen, B., Plott, C. R., Shum, M.: A parimutuel-like mechanism for information aggregation: a field test inside intel. J. Polit. Econ. (forthcoming), California Institute of Technology, Working Paper No. 1367 (2015)
Google.: Quantifying Movie Magic with Google Search. Google Whitepaper: Industry Perspectives + User Insights (2013)
Hadida, A.: Motion picture performance: a review and research agenda. Int. J. Manag. Rev. 11(3), 297–335 (2008)
Hausch, D.B., Ziemba, W.T.: Handbook of Sports and Lottery Markets. North-Holland, Amsterdam (2008)
Jullien, A., Salanie, B.: Estimating preferences under risk: the case of racetrack bettors. J. Polit. Econ. 108, 503–530 (2000)
Kalovcova, K., Ortmann, A.: Understanding the Plott–Wit–Yang Paradox. CERGE-EI. Working Papers WP397, The Center for Economic Research and Graduate Education—Economic Institute, Prague (2009)
Koessler, F., Noussair, C., Ziegelmeyer, A.: Information aggregation and belief elicitation in experimental parimutuel betting markets. J. Econ. Behav. Organ. 83, 195–208 (2012)
Krishna, V.: Auction Theory, 2nd edn. Academic Press, San Diego (2010)
McKenzie, J.: The economics of movies: a literature survey. J. Econ. Surv. 26(1), 42–70 (2012)
McKenzie, J.: Predicting box office with and without markets: do internet users know anything? Inf. Econ. Policy 25, 70–80 (2013)
Mestyan, M., Yasseri, T., Kertesz, J.: Early prediction of movie box office success based on wikipedia activity big data. PLoS One 8(8), e71226 (2013)
Mishne, G., Glance, N.: Predicting movie sales from blogger sentiment. In: Proceedings of AAAI 2006 Spring Symposium on Computational Approaches to Analyzing Weblogs (AAAI-CAAW), pp. 155–158 (2006)
Nagel, R.: Unraveling in guessing games: an experimental study. Am. Econ. Rev. 85(5), 1313–1326 (1995)
Ottaviani, M., Sørensen, P.N.: Noise, information, and the favorite-longshot bias in parimutuel predictions. Am. Econ. J. Microecon. 2(1), 58–85 (2010)
Pennock, D., Lawrence, S., Giles, C.L., Nielsen, F.A.: The real power of artificial markets. Science 291(5506), 987–988. [A longer version of this article is also available at http://artificialmarkets.com/am/pennock-neci-tr-2000-168/] (2001)
Plott, C.R., Roust, K.A.: The design of information aggregation mechanisms: a two-stage parimutuel market to avoid mirages bubbles, Mimeo (2009)
Plott, C.R.: Markets as information gathering tools. South. Econ. J. 67(1), 1–15 (2000)
Plott, C.R., Sunder, S.: Efficiency of experimental security markets with insider information: an application of rational-expectations models. J. Polit. Econ. 90(4), 663–698 (1982)
Plott, C.R., Sunder, S.: Rational expectations and the aggregation of diverse information in laboratory security markets. Econometrica 56(5), 1085–1118 (1988)
Plott, C.R., Wit, J., Yang, W.C.: Parimutuel betting markets as information aggregation devices: experimental results. Econ. Theory 22(2), 311–351 (2003)
Prelec, D.: A Bayesian truth serum for subjective data. Science 306(5695), 462–466 (2004)
Rausser, G.C., Simon, L.K., Zhao, J.: Rational exaggeration and counter-exaggeration in information aggregation games. Econ. Theory 591, 109–146 (2015)
Snowberg, E., Wolfers, J.: Explaining the favorite-long shot bias: is it risk-love or misperceptions? J. Polit. Econ. 118(4), 723–746 (2010)
Spann, M., Skiera, B.: Internet-based virtual stock markets in business forecasting. Manag. Sci. 49(10), 1310–1326 (2003)
Weaver, S., Prelec, D.: Creating truth-telling incentives with the Bayesian truth serum. J. Mark. Res. 1, 289–302 (2013)
Acknowledgements
We thank the Gordon and Betty Moore Foundation; the Lee Center; Australian Research Council (Linkage Grant LP110200336); University of Sydney; Australian Film, Television and Radio School (AFTRS); and the Caltech Laboratory for Experimental Economics and Political Science. The computer and software development skills of Hsing Yang Lee and Travis Maron are acknowledged. Their skills and dedication made the research possible. The comments of Matt Shum were very helpful.
Author information
Authors and Affiliations
Corresponding author
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Court, D., Gillen, B., McKenzie, J. et al. Two information aggregation mechanisms for predicting the opening weekend box office revenues of films: Boxoffice Prophecy and Guess of Guesses. Econ Theory 65, 25–54 (2018). https://doi.org/10.1007/s00199-017-1036-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00199-017-1036-1