Abstract
We use a large-scale internet experiment to explore how subjects learn to play against computers that are programmed to follow one of a number of standard learning algorithms. The learning theories are (unbeknown to subjects) a best response process, fictitious play, imitation, reinforcement learning, and a trial & error process. We explore how subjects’ performances depend on their opponents’ learning algorithm. Furthermore, we test whether subjects try to influence those algorithms to their advantage in a forward-looking way (strategic teaching). We find that strategic teaching occurs frequently and that all learning algorithms are subject to exploitation with the notable exception of imitation.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Alós-Ferrer C., Ania A.B.: The evolutionary stability of perfectly competitive behavior. Econ Theory 26, 497–516 (2005)
Apesteguia J., Huck S., Oechssler J.: Imitation: Theory and experimental evidence. J Econ Theory 136, 217–235 (2007)
Brown G.W.: Iterative solutions of games by fictitious play. In: Koopmans, T.C. (eds) Activity Analysis of Production and Allocation, Wiley, New York (1951)
Camerer C., Ho T.H.: Strategic learning and teaching in games. In: Hoch, S., Kunreuther, H. (eds) Wharton on Decision Making, Wiley, New York (2001)
Camerer C., Ho T.H., Chong J.K.: Sophisticated experience-weighted attraction Learning and strategic teaching in repeated games. J Econ Theory 104, 137–188 (2002)
Cason T., Sharma T.: Recommended play and correlated equilibria: An experimental study. Econ Theory 33, 11–27 (2007)
Coricelli, G.: Strategic interaction in iterated zero-sum games, Mimeo (2005)
Cournot, A.: Researches into the mathematical principles of the theory of wealth, transl. by Bacon, N.T., MacMillan Company, New York, 1927 (1838)
Drehmann M., Oechssler J., Roider A.: Herding and contrarian behavior in financial markets. Am Econ Rev 95(5), 1403–1426 (2005)
Duersch, P., Kolb, A., Oechssler, J., Schipper, B.: Rage against the machines—How Subjects Learn to Play Against Computers, AWI-Discussion Paper No. 423, Department of Economics, University of Heidelberg (2008a)
Duersch, P., Oechssler, J., Schipper, B.: Experimenting on the internet: Does it make a difference? University of Heidelberg, Mimeo (2008b)
Ellison G.: Learning from personal experience: One rational guy and the justification of myopia. Game Econ Behav 19, 180–210 (1997)
Erev, I., Haruvy, E.: Learning and the Economics of Small Decisions. In: Kagel, J.H., Roth, A.E. (eds.) The Handbook of Experimental Economics, vol. 2 (forthcoming) (2008)
Fudenberg D., Levine D.K.: Reputation and equilibrium selection in games with a patient player. Econometrica 57, 759–778 (1989)
Fudenberg D., Levine D.K.: The Theory of Learning in Games. MIT Press, Cambridge (1998)
Huck S., Normann H.T., Oechssler J.: Learning in Cournot oligopoly: An experiment. Econ J 109, C80–C95 (1999)
Huck S., Normann H.T., Oechssler J.: Through trial & error to collusion. Int Econ Rev 45, 205–224 (2004a)
Huck S., Normann H.T., Oechssler J.: Two are few and four are many: Number effects in experimental oligopoly. J Econ Behav Organ 53, 435–446 (2004b)
Ianni A.: Reinforcement learning and the power law of practice: Some analytical results. University of Southampton, Southampton (2002)
Laslier J.-F., Topol R., Walliser B.: A behavioral learning process in games. Game Econ Behav 37, 340–366 (2001)
Monderer D., Shapley L.: Potential games. Game Econ Behav 14, 124–143 (1996)
Offerman T., Potters J., Sonnemans J.: Imitation and belief learning in an oligopoly experiment. Rev Econ Studies 69, 973–997 (2002)
Possajennikov A.: Evolutionary foundation of aggregative-taking behavior. Econ Theory 21, 921–928 (2003)
Robinson J.: An iterative method of solving games. Ann Math 54, 296–301 (1951)
Roth A., Erev I.: Learning in extensive form games: Experimental data and simple dynamic models in the intermediate term. Game Econ Behav 8, 164–212 (1995)
Sarin R., Vahid F.: Strategic similarity and coordination. Econ J 114, 506–527 (2004)
Schipper B.C.: Submodularity and the evolution of Walrasian behavior. Int J Game Theory 32, 471–477 (2003)
Schipper B.C.: Strategic control of myopic best reply in repeated games. University of California, Davis (2006)
Schipper B.C.: Imitators and optimizers in Cournot oligopoly. University of California, Davis (2008)
Selten R., Buchta J.: Experimental sealed bid first price auctions with directly observed bid functions. In: Budescu, D., Erev, I., Zwick, R. (eds) Games and human behavior: Essays in honor of Amnon Rapoport, Lawrence Erlbaum Associates, Mahwah (1998)
Shachat J., Swarthout J.T.: Learning about learning in games through experimental control of strategic independence. University of Arizona, Arizona (2002)
Siegel S., Castellan N.J. Jr: Nonparametric Statistics for the Behavioral Sciences. McGraw-Hill, Singapore (1988)
Tanaka Y.: Long run equilibria in an asymmetric oligopoly. Econ Theory 14, 705–715 (1999)
Vega-Redondo F.: The evolution of Walrasian behavior. Econometrica 65, 375–384 (1997)
Open Access
This article is distributed under the terms of the Creative Commons Attribution Noncommercial License which permits any noncommercial use, distribution,and reproduction in any medium, provided the original author(s) and source are credited.
Author information
Authors and Affiliations
Corresponding author
Additional information
Financial support by the DFG through SFB/TR 15 and SFB 504 is gratefully acknowledged. We thank two anonymous referees, David Cooper, Drew Fudenberg, Tim Grebe, Aaron Lowen, and seminar participants in Edinburgh, Heidelberg, Mannheim, Vienna, Tsukuba, the University of Arizona, and at the ESA Meetings 2005 in Tucson for helpful comments.
Rights and permissions
Open Access This is an open access article distributed under the terms of the Creative Commons Attribution Noncommercial License (https://creativecommons.org/licenses/by-nc/2.0), which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.
About this article
Cite this article
Duersch, P., Kolb, A., Oechssler, J. et al. Rage against the machines: how subjects play against learning algorithms. Econ Theory 43, 407–430 (2010). https://doi.org/10.1007/s00199-009-0446-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00199-009-0446-0
Keywords
- Learning
- Fictitious play
- Imitation
- Reinforcement
- Trial & error
- Strategic teaching
- Cournot duopoly
- Experiments
- Internet