Advertisement

Economic Theory

, Volume 43, Issue 3, pp 407–430 | Cite as

Rage against the machines: how subjects play against learning algorithms

  • Peter Duersch
  • Albert Kolb
  • Jörg Oechssler
  • Burkhard C. Schipper
Open Access
Research Article

Abstract

We use a large-scale internet experiment to explore how subjects learn to play against computers that are programmed to follow one of a number of standard learning algorithms. The learning theories are (unbeknown to subjects) a best response process, fictitious play, imitation, reinforcement learning, and a trial & error process. We explore how subjects’ performances depend on their opponents’ learning algorithm. Furthermore, we test whether subjects try to influence those algorithms to their advantage in a forward-looking way (strategic teaching). We find that strategic teaching occurs frequently and that all learning algorithms are subject to exploitation with the notable exception of imitation.

Keywords

Learning Fictitious play Imitation Reinforcement Trial & error Strategic teaching Cournot duopoly Experiments Internet 

JEL Classification

C72 C91 C92 D43 L13 

Notes

Open Access

This article is distributed under the terms of the Creative Commons Attribution Noncommercial License which permits any noncommercial use, distribution,and reproduction in any medium, provided the original author(s) and source are credited.

References

  1. Alós-Ferrer C., Ania A.B.: The evolutionary stability of perfectly competitive behavior. Econ Theory 26, 497–516 (2005)CrossRefGoogle Scholar
  2. Apesteguia J., Huck S., Oechssler J.: Imitation: Theory and experimental evidence. J Econ Theory 136, 217–235 (2007)CrossRefGoogle Scholar
  3. Brown G.W.: Iterative solutions of games by fictitious play. In: Koopmans, T.C. (eds) Activity Analysis of Production and Allocation, Wiley, New York (1951)Google Scholar
  4. Camerer C., Ho T.H.: Strategic learning and teaching in games. In: Hoch, S., Kunreuther, H. (eds) Wharton on Decision Making, Wiley, New York (2001)Google Scholar
  5. Camerer C., Ho T.H., Chong J.K.: Sophisticated experience-weighted attraction Learning and strategic teaching in repeated games. J Econ Theory 104, 137–188 (2002)CrossRefGoogle Scholar
  6. Cason T., Sharma T.: Recommended play and correlated equilibria: An experimental study. Econ Theory 33, 11–27 (2007)CrossRefGoogle Scholar
  7. Coricelli, G.: Strategic interaction in iterated zero-sum games, Mimeo (2005)Google Scholar
  8. Cournot, A.: Researches into the mathematical principles of the theory of wealth, transl. by Bacon, N.T., MacMillan Company, New York, 1927 (1838)Google Scholar
  9. Drehmann M., Oechssler J., Roider A.: Herding and contrarian behavior in financial markets. Am Econ Rev 95(5), 1403–1426 (2005)CrossRefGoogle Scholar
  10. Duersch, P., Kolb, A., Oechssler, J., Schipper, B.: Rage against the machines—How Subjects Learn to Play Against Computers, AWI-Discussion Paper No. 423, Department of Economics, University of Heidelberg (2008a)Google Scholar
  11. Duersch, P., Oechssler, J., Schipper, B.: Experimenting on the internet: Does it make a difference? University of Heidelberg, Mimeo (2008b)Google Scholar
  12. Ellison G.: Learning from personal experience: One rational guy and the justification of myopia. Game Econ Behav 19, 180–210 (1997)CrossRefGoogle Scholar
  13. Erev, I., Haruvy, E.: Learning and the Economics of Small Decisions. In: Kagel, J.H., Roth, A.E. (eds.) The Handbook of Experimental Economics, vol. 2 (forthcoming) (2008)Google Scholar
  14. Fudenberg D., Levine D.K.: Reputation and equilibrium selection in games with a patient player. Econometrica 57, 759–778 (1989)CrossRefGoogle Scholar
  15. Fudenberg D., Levine D.K.: The Theory of Learning in Games. MIT Press, Cambridge (1998)Google Scholar
  16. Huck S., Normann H.T., Oechssler J.: Learning in Cournot oligopoly: An experiment. Econ J 109, C80–C95 (1999)CrossRefGoogle Scholar
  17. Huck S., Normann H.T., Oechssler J.: Through trial & error to collusion. Int Econ Rev 45, 205–224 (2004a)CrossRefGoogle Scholar
  18. Huck S., Normann H.T., Oechssler J.: Two are few and four are many: Number effects in experimental oligopoly. J Econ Behav Organ 53, 435–446 (2004b)CrossRefGoogle Scholar
  19. Ianni A.: Reinforcement learning and the power law of practice: Some analytical results. University of Southampton, Southampton (2002)Google Scholar
  20. Laslier J.-F., Topol R., Walliser B.: A behavioral learning process in games. Game Econ Behav 37, 340–366 (2001)CrossRefGoogle Scholar
  21. Monderer D., Shapley L.: Potential games. Game Econ Behav 14, 124–143 (1996)CrossRefGoogle Scholar
  22. Offerman T., Potters J., Sonnemans J.: Imitation and belief learning in an oligopoly experiment. Rev Econ Studies 69, 973–997 (2002)CrossRefGoogle Scholar
  23. Possajennikov A.: Evolutionary foundation of aggregative-taking behavior. Econ Theory 21, 921–928 (2003)CrossRefGoogle Scholar
  24. Robinson J.: An iterative method of solving games. Ann Math 54, 296–301 (1951)CrossRefGoogle Scholar
  25. Roth A., Erev I.: Learning in extensive form games: Experimental data and simple dynamic models in the intermediate term. Game Econ Behav 8, 164–212 (1995)CrossRefGoogle Scholar
  26. Sarin R., Vahid F.: Strategic similarity and coordination. Econ J 114, 506–527 (2004)CrossRefGoogle Scholar
  27. Schipper B.C.: Submodularity and the evolution of Walrasian behavior. Int J Game Theory 32, 471–477 (2003)Google Scholar
  28. Schipper B.C.: Strategic control of myopic best reply in repeated games. University of California, Davis (2006)Google Scholar
  29. Schipper B.C.: Imitators and optimizers in Cournot oligopoly. University of California, Davis (2008)Google Scholar
  30. Selten R., Buchta J.: Experimental sealed bid first price auctions with directly observed bid functions. In: Budescu, D., Erev, I., Zwick, R. (eds) Games and human behavior: Essays in honor of Amnon Rapoport, Lawrence Erlbaum Associates, Mahwah (1998)Google Scholar
  31. Shachat J., Swarthout J.T.: Learning about learning in games through experimental control of strategic independence. University of Arizona, Arizona (2002)Google Scholar
  32. Siegel S., Castellan N.J. Jr: Nonparametric Statistics for the Behavioral Sciences. McGraw-Hill, Singapore (1988)Google Scholar
  33. Tanaka Y.: Long run equilibria in an asymmetric oligopoly. Econ Theory 14, 705–715 (1999)CrossRefGoogle Scholar
  34. Vega-Redondo F.: The evolution of Walrasian behavior. Econometrica 65, 375–384 (1997)CrossRefGoogle Scholar

Copyright information

© The Author(s) 2009

Authors and Affiliations

  • Peter Duersch
    • 1
  • Albert Kolb
    • 1
  • Jörg Oechssler
    • 1
  • Burkhard C. Schipper
    • 2
  1. 1.Department of EconomicsUniversity of HeidelbergHeidelbergGermany
  2. 2.Department of EconomicsUniversity of California, DavisDavisUSA

Personalised recommendations