Skip to main content

Rage against the machines: how subjects play against learning algorithms

Abstract

We use a large-scale internet experiment to explore how subjects learn to play against computers that are programmed to follow one of a number of standard learning algorithms. The learning theories are (unbeknown to subjects) a best response process, fictitious play, imitation, reinforcement learning, and a trial & error process. We explore how subjects’ performances depend on their opponents’ learning algorithm. Furthermore, we test whether subjects try to influence those algorithms to their advantage in a forward-looking way (strategic teaching). We find that strategic teaching occurs frequently and that all learning algorithms are subject to exploitation with the notable exception of imitation.

References

  • Alós-Ferrer C., Ania A.B.: The evolutionary stability of perfectly competitive behavior. Econ Theory 26, 497–516 (2005)

    Article  Google Scholar 

  • Apesteguia J., Huck S., Oechssler J.: Imitation: Theory and experimental evidence. J Econ Theory 136, 217–235 (2007)

    Article  Google Scholar 

  • Brown G.W.: Iterative solutions of games by fictitious play. In: Koopmans, T.C. (eds) Activity Analysis of Production and Allocation, Wiley, New York (1951)

    Google Scholar 

  • Camerer C., Ho T.H.: Strategic learning and teaching in games. In: Hoch, S., Kunreuther, H. (eds) Wharton on Decision Making, Wiley, New York (2001)

    Google Scholar 

  • Camerer C., Ho T.H., Chong J.K.: Sophisticated experience-weighted attraction Learning and strategic teaching in repeated games. J Econ Theory 104, 137–188 (2002)

    Article  Google Scholar 

  • Cason T., Sharma T.: Recommended play and correlated equilibria: An experimental study. Econ Theory 33, 11–27 (2007)

    Article  Google Scholar 

  • Coricelli, G.: Strategic interaction in iterated zero-sum games, Mimeo (2005)

  • Cournot, A.: Researches into the mathematical principles of the theory of wealth, transl. by Bacon, N.T., MacMillan Company, New York, 1927 (1838)

  • Drehmann M., Oechssler J., Roider A.: Herding and contrarian behavior in financial markets. Am Econ Rev 95(5), 1403–1426 (2005)

    Article  Google Scholar 

  • Duersch, P., Kolb, A., Oechssler, J., Schipper, B.: Rage against the machines—How Subjects Learn to Play Against Computers, AWI-Discussion Paper No. 423, Department of Economics, University of Heidelberg (2008a)

  • Duersch, P., Oechssler, J., Schipper, B.: Experimenting on the internet: Does it make a difference? University of Heidelberg, Mimeo (2008b)

  • Ellison G.: Learning from personal experience: One rational guy and the justification of myopia. Game Econ Behav 19, 180–210 (1997)

    Article  Google Scholar 

  • Erev, I., Haruvy, E.: Learning and the Economics of Small Decisions. In: Kagel, J.H., Roth, A.E. (eds.) The Handbook of Experimental Economics, vol. 2 (forthcoming) (2008)

  • Fudenberg D., Levine D.K.: Reputation and equilibrium selection in games with a patient player. Econometrica 57, 759–778 (1989)

    Article  Google Scholar 

  • Fudenberg D., Levine D.K.: The Theory of Learning in Games. MIT Press, Cambridge (1998)

    Google Scholar 

  • Huck S., Normann H.T., Oechssler J.: Learning in Cournot oligopoly: An experiment. Econ J 109, C80–C95 (1999)

    Article  Google Scholar 

  • Huck S., Normann H.T., Oechssler J.: Through trial & error to collusion. Int Econ Rev 45, 205–224 (2004a)

    Article  Google Scholar 

  • Huck S., Normann H.T., Oechssler J.: Two are few and four are many: Number effects in experimental oligopoly. J Econ Behav Organ 53, 435–446 (2004b)

    Article  Google Scholar 

  • Ianni A.: Reinforcement learning and the power law of practice: Some analytical results. University of Southampton, Southampton (2002)

    Google Scholar 

  • Laslier J.-F., Topol R., Walliser B.: A behavioral learning process in games. Game Econ Behav 37, 340–366 (2001)

    Article  Google Scholar 

  • Monderer D., Shapley L.: Potential games. Game Econ Behav 14, 124–143 (1996)

    Article  Google Scholar 

  • Offerman T., Potters J., Sonnemans J.: Imitation and belief learning in an oligopoly experiment. Rev Econ Studies 69, 973–997 (2002)

    Article  Google Scholar 

  • Possajennikov A.: Evolutionary foundation of aggregative-taking behavior. Econ Theory 21, 921–928 (2003)

    Article  Google Scholar 

  • Robinson J.: An iterative method of solving games. Ann Math 54, 296–301 (1951)

    Article  Google Scholar 

  • Roth A., Erev I.: Learning in extensive form games: Experimental data and simple dynamic models in the intermediate term. Game Econ Behav 8, 164–212 (1995)

    Article  Google Scholar 

  • Sarin R., Vahid F.: Strategic similarity and coordination. Econ J 114, 506–527 (2004)

    Article  Google Scholar 

  • Schipper B.C.: Submodularity and the evolution of Walrasian behavior. Int J Game Theory 32, 471–477 (2003)

    Google Scholar 

  • Schipper B.C.: Strategic control of myopic best reply in repeated games. University of California, Davis (2006)

    Google Scholar 

  • Schipper B.C.: Imitators and optimizers in Cournot oligopoly. University of California, Davis (2008)

    Google Scholar 

  • Selten R., Buchta J.: Experimental sealed bid first price auctions with directly observed bid functions. In: Budescu, D., Erev, I., Zwick, R. (eds) Games and human behavior: Essays in honor of Amnon Rapoport, Lawrence Erlbaum Associates, Mahwah (1998)

    Google Scholar 

  • Shachat J., Swarthout J.T.: Learning about learning in games through experimental control of strategic independence. University of Arizona, Arizona (2002)

    Google Scholar 

  • Siegel S., Castellan N.J. Jr: Nonparametric Statistics for the Behavioral Sciences. McGraw-Hill, Singapore (1988)

    Google Scholar 

  • Tanaka Y.: Long run equilibria in an asymmetric oligopoly. Econ Theory 14, 705–715 (1999)

    Article  Google Scholar 

  • Vega-Redondo F.: The evolution of Walrasian behavior. Econometrica 65, 375–384 (1997)

    Article  Google Scholar 

Download references

Open Access

This article is distributed under the terms of the Creative Commons Attribution Noncommercial License which permits any noncommercial use, distribution,and reproduction in any medium, provided the original author(s) and source are credited.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Jörg Oechssler.

Additional information

Financial support by the DFG through SFB/TR 15 and SFB 504 is gratefully acknowledged. We thank two anonymous referees, David Cooper, Drew Fudenberg, Tim Grebe, Aaron Lowen, and seminar participants in Edinburgh, Heidelberg, Mannheim, Vienna, Tsukuba, the University of Arizona, and at the ESA Meetings 2005 in Tucson for helpful comments.

Rights and permissions

Open Access This is an open access article distributed under the terms of the Creative Commons Attribution Noncommercial License (https://creativecommons.org/licenses/by-nc/2.0), which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.

Reprints and Permissions

About this article

Cite this article

Duersch, P., Kolb, A., Oechssler, J. et al. Rage against the machines: how subjects play against learning algorithms. Econ Theory 43, 407–430 (2010). https://doi.org/10.1007/s00199-009-0446-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00199-009-0446-0

Keywords

  • Learning
  • Fictitious play
  • Imitation
  • Reinforcement
  • Trial & error
  • Strategic teaching
  • Cournot duopoly
  • Experiments
  • Internet

JEL Classification

  • C72
  • C91
  • C92
  • D43
  • L13