Abstract
Pricing decisions are increasingly in the “hands” of artificial algorithms. Scholars and competition authorities have voiced concerns that those algorithms are capable of sustaining collusive outcomes more effectively than can human decision makers. If this is so, then our traditional policy tools for fighting collusion may have to be reconsidered. We discuss these issues by critically surveying the relevant law, economics, and computer science literature.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
British Airways seems to have been the first company to use pricing algorithms in the 1970s.
The European Commission’s 2017 “Final report on the E-commerce Sector Inquiry” concludes that “A majority of retailers track the online prices of competitors. Two-thirds of them use software programs that autonomously adjust their own prices based on the observed prices of competitors”.
AP could provide a competing explanation (and thus an identification challenge) for the evidence of higher online (relative to offline) prices in some markets. The prevalent explanation is that of an increase in the match quality (Ellison and Ellison 2018). AP could also speak to the question of what is causing (online) price dispersion both in the cross-section and in the time-series in seemingly homogenous product markets (Chen et al. 2016).
For example, the New Yorker asked what happens “When bots collude” (April 25, 2015), and the Financial Times wrote of “Digital cartels” (January 8, 2017).
The Acting Chairman of the U.S. Federal Trade Commission M. Ohlhausen “Should We Fear the Things That Go Beep in the Night? Some Initial Thoughts on the Intersection of Antitrust Law and Algorithmic Pricing,” remarks at the “Concurrences Antitrust in the Financial Sector Conference,” New York, May 23, 2017. OECD Roundtable on Algorithms and Collusion, June 2017. The European Commissioner for Competition M. Vestager, “Algorithms and Competition,” remarks at the Bundeskarellamt 18th Conference on Competition, Berlin, March 16, 2017.
Wired Magazine; U.S. v. Topkins, 2015 and CMA case 2015 n. 50223.
See Olivia Solon, 2015, “How a book about flies came to be priced $24 million on Amazon”.
Harrington (2017) develops a legal approach for collusion with AP that is grounded in economic analysis.
The term may be imprecise as these algorithms learn only in a very limited sense, as we shall clarify below.
Whether this result survives when market conditions are not stationary-- and hence the estimation function of the pricing algorithm is active-- is an open question.
For example, the second-hand book episode seems to have been generated by adaptive algorithms which would set the own price as a multiple of the rival’s price. If two firms adopt a pricing rule of the type \(p_{i} = a_{i} p_{j}\), the system explodes whenever \(a_{i} a_{j} > 1.\) As a results prices may get very high indeed, but the outcome will be poor in terms of profit maximization.
Sannikov and Skrzypacz (2007) derive this result by assuming that agents optimally extract the signal from the noisy information that they receive. Whether the result continues to be true also for AP remains to be investigated.
ML techniques have been developed and are currently adopted for a large number of applications. By far the most popular in the social sciences are those of classifying tasks (with supervised learning) and those meant to uncover hidden structure in big datasets (with unsupervised learning).
If the environment is stationary, experimentation is crucial initially but may optimally vanish eventually; in a dynamic environment, in contrast, it may be optimal to keep experimenting forever.
For a textbook introduction to Q-learning see Sutton and Barto (1998).
To economists, the analysis of a MDP may be reminiscent of the analysis of Markov Perfect industry dynamics, which are summarized in Doraszelski and Pakes (2007). There are however substantial differences: In that literature, the numerical methods for equilibrium identification rely on the industry structure and solve systems of firms’ optimality conditions. Here instead, the algorithms are model free and learn with experimentation.
Q-learning is also related to the idea of active learning in game theory (see Fudenberg and Levine 2016). It is normally associated with the more general class of “reinforcement learning” models where an agent learns by interacting with the environment, perceiving its state, and taking actions with trials and errors. Those actions that are associated with a positive consequence (reward) then have higher chances of being chosen in the future: They are reinforced by the agent’s behavior.
Still, Busoniu et al. (2008) account for cases in which these algorithms have show good performances in terms of convergence.
For a comprehensive survey see Busoniu et al. (2008).
Note that this does not imply that the values of the Q matrix remain unchanged, but only that the ranking between Qi(s,1) and Qi(s,2) is unchanged.
In a celebrated experiment Mnih et al. (2015) show how a deep reinforcement ML algorithm learned to solve complex tasks such as playing classic Atari videogames better than humans using as the state the color of 210 × 160 pixels on the screen with a 128-colour palette and the scoreplay.
Other recent and promising examples about cooperation in a multi-agent setting is the work at Facebook AI Research: e.g., by Lerer and Peysakhovich (2018).
They are not sufficient because, for example, the Max operator in the formula for Q-learning may lead to non-unique solutions.
Cooper and Kühn (2014) have shown that communication between humans helps cooperation by clarifying how individuals think about the environment and whether they really mean punishing deviations and by making social punishments and rewards explicit.
Salcedo (2015) presents a theoretical model that shows that collusion with algorithms is not only possible, but also inevitable. However, the result relies on algorithms’ ability to read into other algorithms and thus learn their “intentions,” and on some commitment not to revise the algorithms in use.
With this respect it is interesting to notice that the use of learning and optimizing software may induce firms to behave more like the dynamic profit‐maximizing firms of standard economic theory.
References
Bloembergen, D., Tuyls, K., Hennes, D., & Kaisers, M. (2015). Evolutionary dynamics of multi-agent learning: A survey. Journal of Artificial Intelligence Research, 53, 659–697.
Busoniu, L., Babuska, R., & De Schutter, B. (2008). A comprehensive survey of multiagent reinforcement learning. IEEE Transactions on Systems, Man, and Cybernetics-Part C: Applications and Reviews, 38(2), 156–172.
Calvano, E., Calzolari, G., Denicolò, V., & Pastorello, S. (2018). Artificial intelligence, algorithmic pricing and collusion. CEPR Discussion Paper13405.
Chen, L., Mislove, A., & Wilson, C. (2016). An empirical analysis of algorithmic pricing on Amazon marketplace. In Proceedings of the 25th international conference on world wide web. International World Wide Web Conferences Steering Committee.
Cooper, R. W., DeJong, D. V., Forsythe, R., & Ross, T. W. (1990). Selection criteria in coordination games: Some experimental results. The American Economic Review, 80(1), 218–233.
Cooper, D. J., & Kühn, K. U. (2014). Communication, renegotiation, and the scope for collusion. American Economic Journal: Microeconomics, 6(2), 247–278.
Crandall, J. W., Oudah, M., Tennom, Ishowo-Oloko, F., Abdallah, S., Bonnefon, J., et al. (2018). Cooperating with machines. Nature Communications, 9(233), 2018.
Dogan, I., & Guner, A. R. (2015). A reinforcement learning approach to competitive ordering and pricing problem. Expert Systems, 32, 39–47.
Doraszelski, U., & Pakes, A. (2007). A framework for applied dynamic analysis in IO. In M. Armstrong & R. H. Porter (Eds.), Handbook of industrial organization (Vol. 3, Chapter 4). Amsterdam: Elsevier Science.
Ellison, G., & Ellison, S. F. (2018). Match quality, search, and the Internet market for used books (No. w24197). National Bureau of Economic Research.
Erev, I., & Roth, A. E. (1998). Predicting how people play games: Reinforcement learning in experimental games with unique, mixed strategy equilibria. American Economic Review, 88, 848–881.
Ezrachi, A., & Stucke, M. E. (2015). Artificial intelligence and collusion: When computers inhibit competition. Oxford Legal Studies Research Paper No. 18/2015, University of Tennessee Legal Studies Research Paper No. 267.
Fudenberg, D., & Levine, D. K. (2016). Whither game theory? Towards a theory of learning in games. The Journal of Economic Perspectives, 30(4), 151–169.
Harrington, J. E. (2017). Developing competition law for collusion by autonomous agents. Working paper, The Wharton School, University of Pennsylvania.
Hu, J., & Wellman, M. P. (2003). Nash Q-learning for general-sum stochastic games. Journal of machine learning research, 4, 1039–1069.
Lerer, A., & Peysakhovich, A. (2018). Maintaining cooperation in complex social dilemmas using deep reinforcement learning. arXiv preprint arXiv:1707.01068.
Mehra, S. K. (2016). Antitrust and the robo-seller: Competition in the time of algorithms. Minnesota Law Review, 100, 1323–75.
Milgrom, P. R., & Roberts, D. J. (1990). Rationalizability, learning, and equilibrium in games with strategic complementarities. Econometrica, 58, 1255–1277.
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., et al. (2015). Human-level control through deep reinforcement learning. Nature, 518(7540), 529–533. https://doi.org/10.1038/nature14236. (PMID: 25719670).
Roth, A. E., & Erev, I. (1995). Learning in extensive-form games: Experimental data and simple dynamic models in the intermediate term. Games and Economic Behavior, 8, 164–212.
Salcedo, B. (2015). Pricing algorithms and tacit collusion. Working paper Pennsylvania State University.
Sannikov, Y., & Skrzypacz, A. (2007). Impossibility of collusion under imperfect monitoring with flexible production. American Economic Review, 97, 1794–1823.
Sarin, R., & Vahid, F. (2001). Predicting how people play games: A simple dynamic model of choice. Games and Economic Behavior, 34, 104–122.
Sukhbaatar, S., Szlam, A., & Fergus, R. (2016). Learning multiagent communication with backpropagation. arXiv, preprint arXiv:1605.07736.
Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning: An introduction. Cambridge: MIT press.
Tampuu, A., Matiisen, T., Kodelja, D., Kuzovkin, I., Korjus, K., Aru, J., et al. (2017). Multiagent cooperation and competition with deep reinforcement learning. PLoS ONE, 12(4), e0172395.
Tesauro, G., & Kephart, J. O. (2002). Pricing in agent economics using multi-agent Q-learning. Autonomous Agents and Multi-Agent Systems, 5, 289–304.
United States v. David Topkins. (2015). Plea agreement. https://www.justice.gov/atr/casedocument/file/513586/download
Waltman, L., & Kaymak, U. (2008). Q-learning agents in a Cournot oligopoly model. Journal of Economic Dynamics and Control, 32(10), 3275–3293.
Watkins, C. J. C. H., & Dayan, P. (1992). Q-learning. Machine Learning, 8(279), 292.
Xie, M., & Chen, J. (2004). Studies on horizontal competition among homogeneous retailers through agent-based simulations. Journal of Systems Science and Systems Engineering, 13, 490–505.
Acknowledgements
We thank the editor Larry White, the guest editors Christos Genakos, Michael Pollitt, and the discussant Patrick Legros and participants at the conference “Celebrating 25 Years of the EU Single Market” organized by the Review of Industrial Organization in Cambridge Judge Business School, 2018. Financial support from the Digital Chair of the Toulouse school of economics is gratefully acknowledged.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Calvano, E., Calzolari, G., Denicolò, V. et al. Algorithmic Pricing What Implications for Competition Policy?. Rev Ind Organ 55, 155–171 (2019). https://doi.org/10.1007/s11151-019-09689-3
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11151-019-09689-3