Advertisement

Learning and Cognition in Financial Markets: A Paradigm Shift for Agent-Based Models

  • Johann LussangeEmail author
  • Alexis Belianin
  • Sacha Bourgeois-Gironde
  • Boris Gutkin
Conference paper
  • 90 Downloads
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 1252)

Abstract

The history of research in finance and economics has been widely impacted by the field of Agent-based Computational Economics (ACE). While at the same time being popular among natural science researchers for its proximity to the successful methods of physics and chemistry for example, the field of ACE has also received critics by a part of the social science community for its lack of empiricism. Yet recent trends have shifted the weights of these general arguments and potentially given ACE a whole new range of realism. At the base of these trends are found two present-day major scientific breakthroughs: the steady shift of psychology towards a hard science due to the advances of neuropsychology, and the progress of reinforcement learning due to increasing computational power and big data. We outline here the main lines of a computational research study where each agent would trade by reinforcement learning.

Keywords

Financial markets Agent-based models Multi-agent systems Reinforcement learning 

Notes

Acknowledgment

This work was supported by the RFFI grant nr. 16-51-150007 and CNRS PRC nr. 151199, and received support from FrontCog ANR-17-EURE-0017. A preliminary preprint of this work was uploaded to arXiv  [150].

References

  1. 1.
    Bollerslev, T.: CREATES research paper 2008, p. 49 (2008)Google Scholar
  2. 2.
    Engle, R.F.: Econometrica 50(4), 987 (1982)MathSciNetGoogle Scholar
  3. 3.
    Brownlees, C.T., Engle, R.F., Kelly, B.T.: J. Risk 14(2), 3 (2011)Google Scholar
  4. 4.
    Sbordone, A.M., Tambalotti, A., Rao, K., Walsh, K.J.: Econ. Policy Rev. 16(2) (2010) Google Scholar
  5. 5.
    Evans, G.W., Honkapohja, S.: Learning and Expectations in Macroeconomics. Princeton University Press, Princeton (2001)Google Scholar
  6. 6.
    Eusepi, S., Preston, B.: Am. Econ. Rev. 101, 2844 (2011)Google Scholar
  7. 7.
    Massaro, D.: J. Econ. Dyn. Control 37, 680 (2013)MathSciNetGoogle Scholar
  8. 8.
    Farmer, J.D., Foley, D.: Nature 460(7256), 685 (2009)Google Scholar
  9. 9.
    Grauwe, P.D.: Public Choice 144(3–4), 413 (2010)Google Scholar
  10. 10.
    Tesfatsion, L., Judd, K.L.: Handbook of Computational Economics: Agent-Based Computational Economics, vol. II. Elsevier, Amsterdam (2006)zbMATHGoogle Scholar
  11. 11.
    Samanidou, E., Zschischang, E., Stauffer, D., Lux, T.: Rep. Prog. Phys. 70(3), 409 (2007)Google Scholar
  12. 12.
    LeBaron, B.: Building the Santa Fe Artificial Stock Market (2002)Google Scholar
  13. 13.
    Bonabeau, E.: Harvard Bus. Rev. 80(3), 109 (2002)Google Scholar
  14. 14.
    Smith, E., Farmer, D.J., Gillemot, L., Krishnamurthy, S.: Quant. Finance 3, 481 (2003)MathSciNetGoogle Scholar
  15. 15.
    Huang, W., Lehalle, C.-A., Rosenbaum, M.: J. Am. Stat. Assoc. 110, 509 (2015)Google Scholar
  16. 16.
    Mota, R., Larralde, H.: arXiv:1601.00229 (2016)
  17. 17.
    Macal, C.M., North, M.J.: J. Simul. 4, 151 (2010)Google Scholar
  18. 18.
    Axelrod, R.M.: The Complexity of Cooperation: Agent-Based Models of Competition and Collaboration. Princeton University Press, Princeton (1997)Google Scholar
  19. 19.
    Grimm, V., Berger, U., Bastiansen, F., Eliassen, S., Ginot, V., Giske, J., Goss-Custard, J., Grand, T., Heinz, S.K., Huse, G., et al.: Ecol. Model. 198(1), 115 (2006)Google Scholar
  20. 20.
    Bush, R.R., Mosteller, F.: Stochastic Models for Learning. Wiley, Oxford (1955)zbMATHGoogle Scholar
  21. 21.
    Smith, J., Price, D.: Nature 246, 15 (1973)Google Scholar
  22. 22.
    Taylor, P.D., Jonker, L.B.: Math. Biosci. 40, 145 (1978)MathSciNetGoogle Scholar
  23. 23.
    Mookherjee, D., Sopher, B.: Games Econ. Behav. 7, 62 (1994)Google Scholar
  24. 24.
    Erev, I., Roth, A.E.: Am. Econ. Rev. 88, 848 (1998)Google Scholar
  25. 25.
    Erev, I., Roth, A.E.: PNAS 111, 10818 (2014)Google Scholar
  26. 26.
    Camerer, C.F., Ho, T.H.: PNAS 67, 827 (1999)Google Scholar
  27. 27.
    Fudenberg, D., Levine, D.: The Theory of Learning in Games. MIT Press, Cambridge (1998)zbMATHGoogle Scholar
  28. 28.
    Colander, D., Howitt, P., Kirman, A., Leijonhufvud, A., Mehrling, P.: Am. Econ. Rev. 236–240 (2008) Google Scholar
  29. 29.
    Dosi, G., Fagiolo, G., Napoletano, M., Roventini, A.: J. Econ. Dyn. Control 37(8), 1598 (2013)Google Scholar
  30. 30.
    Gualdi, S., Tarzia, M., Zamponi, F., Bouchaud, J.-P.: J. Econ. Interact. Coord. 1–31 (2016)Google Scholar
  31. 31.
    Gualdi, S., Tarzia, M., Zamponi, F., Bouchaud, J.-P.: J. Econ. Dyn. Control 50, 29 (2015)Google Scholar
  32. 32.
    Westerhoff, F.H.: Jahrbucher Fur Nationalokonomie Und Statistik 228(2), 195 (2008)Google Scholar
  33. 33.
    Xu, H.-C., Zhang, W., Xiong, X., Zhou, W.-X.: Math. Prob. Eng. 2014, 563912 (2014)Google Scholar
  34. 34.
    Boero, R., Morini, M., Sonnessa, M., Terna, P.: Agent-Based Models of the Economy, From Theories to Applications. Palgrave Macmillan, New York (2015)Google Scholar
  35. 35.
    LeBaron, B.: Agent-based computational finance. In: The Handbook of Computational Economics, vol. 2. Elsevier, Amsterdam (2005)Google Scholar
  36. 36.
    Heylighen, F.: Complexity and Self-Organization. CRC Press, Boca Raton (2008)Google Scholar
  37. 37.
    Plerou, V., Gopikrishnan, P., Stanley, H.E.: Nature 421, 130 (2003)Google Scholar
  38. 38.
    Hamill, L., Gilbert, N.: Agent-Based Modelling in Economics. Wiley, Hoboken (2016)Google Scholar
  39. 39.
    Wilcox, D., Gebbie, T.: arXiv:1408.5585 (2014)
  40. 40.
    Hanson, T.A.: Midwest finance association 2012 annual meetings paper (2011)Google Scholar
  41. 41.
    Bartolozzi, M.: Eur. Phys. J. B 78, 265 (2010)Google Scholar
  42. 42.
    Wah, E., Wellman, M.P.: Proceedings of the Fourteenth ACM Conference on Electronic Commerce, pp. 855–872 (2013)Google Scholar
  43. 43.
    Paddrik, M.E., Hayes, R.L., Todd, A., Yang, S.Y., Scherer, W., Beling, P.: SSRN 1932152 (2011)Google Scholar
  44. 44.
    Aloud, M., Tsang, E., Olsen, R.: Business science reference, Hershey (2013)Google Scholar
  45. 45.
    Challet, D., Marsili, M., Zhang, Y.-C.: Minority Games: Interacting Agents in Financial Markets. Oxford University Press, Oxford (2005)zbMATHGoogle Scholar
  46. 46.
    Martino, A.D., Marsili, M.: J. Phys. A 39, 465 (2006)MathSciNetGoogle Scholar
  47. 47.
    Kyle, A.S., O. A.: Economertrica forthcoming (2016) Google Scholar
  48. 48.
    Preis, T., Golke, S., Paul, W., Schneider, J.J.: Europhys. Lett. 75(3), 510 (2006)MathSciNetGoogle Scholar
  49. 49.
    Farmer, J.D., Patelli, P., Zovko, I.I.: Proc. Natl. Acad. Sci. U.S.A. 102(6), 2254 (2005)Google Scholar
  50. 50.
    Maslov, S.: Phys. A 278(3), 571 (2000)MathSciNetGoogle Scholar
  51. 51.
    Challet, D., Stinchcombe, R.: Quant. Finance 3(3), 155 (2003)Google Scholar
  52. 52.
    Schmitt, T.A., Schfer, R., Mnnix, M.C., Guhr, T.: Europhys. Lett. 100 (2012)Google Scholar
  53. 53.
    Lux, T., Marchesi, M.: J. Theor. Appl. Finance 3, 67 (2000)Google Scholar
  54. 54.
    Cont, R.: Volatility clustering in financial markets: empirical facts and agent-based models. Springer (2007)Google Scholar
  55. 55.
    Bertella, M.A., Pires, F.R., Feng, L., Stanley, H.E.: PLoS ONE 9(1), e83488 (2014)Google Scholar
  56. 56.
    Alfi, V., Cristelli, M., Pietronero, L., Zaccaria, A.: Eur. Phys. J. B 67(3), 385 (2009)Google Scholar
  57. 57.
    Kim, G., Markowitz, H.M.: J. Portfolio Manag. 16, 45 (1989)Google Scholar
  58. 58.
    Levy, M., Solomon, S.: Int. J. Mod. Phys. C 7, 595 (1996)Google Scholar
  59. 59.
    Levy, M., Levy, H., Solomon, S.: Econ. Lett. 45, 103 (1994)Google Scholar
  60. 60.
    Levy, M., Levy, H., Solomon, S.: J. Phys. I 5, 1087 (1995)Google Scholar
  61. 61.
    Levy, M., Solomon, S.: Int. J. Mod. Phys. C 7, 65 (1996)Google Scholar
  62. 62.
    Levy, M., Persky, N., Solomon, S.: Int. J. High Speed Comput. 8, 93 (1996)Google Scholar
  63. 63.
    Levy, M., Levy, H., Solomon, S.: Phys. A 242, 90 (1997)Google Scholar
  64. 64.
    Levy, M., Levy, H., Solomon, S.: Microscopic Simulation of Financial Markets. Academic Press, New York (2000)zbMATHGoogle Scholar
  65. 65.
    Cont, R., Bouchaud, J.P.: Macroecon. Dyn. 4, 170 (2000)Google Scholar
  66. 66.
    Solomon, S., Weisbuch, G., de Arcangelis, L., Jan, N., Stauffer, D.: Phys. A 277(1), 239 (2000)Google Scholar
  67. 67.
    Lux, T., Marchesi, M.: Nature 397, 498 (1999)Google Scholar
  68. 68.
    Donangelo, R., Hansen, A., Sneppen, K., Souza, S.R.: Phys. A 283, 469 (2000)Google Scholar
  69. 69.
    Donangelo, R., Sneppen, K.: Phys. A 276, 572 (2000)Google Scholar
  70. 70.
    Bak, P., Norrelykke, S., Shubik, M.: Phys. Rev. E 60, 2528 (1999)Google Scholar
  71. 71.
    Bak, P., Norrelykke, S., Shubik, M.: Quant. Finance 1, 186 (2001)MathSciNetGoogle Scholar
  72. 72.
    Huang, Z.F., Solomon, S.: Eur. Phys. J. B 20, 601 (2000)Google Scholar
  73. 73.
    Lipski, J., Kutner, R.: arXiv:1310.0762 (2013)
  74. 74.
    Barde, S.: School of economics discussion papers 04. University of Kent (2015)Google Scholar
  75. 75.
    Potters, M., Bouchaud, J.-P.: Phys. A 299, 60 (2001)Google Scholar
  76. 76.
    Plerou, V., Gopikrishnan, P., Amaral, L.A., Meyer, M., Stanley, H.E.: Phys. Rev. E 60(6), 6519 (1999)Google Scholar
  77. 77.
    Cristelli, M.: Complexity in Financial Markets. Springer, Cham (2014)Google Scholar
  78. 78.
    Weron, R.: Int. J. Mod. Phys. C 12, 209 (2001)Google Scholar
  79. 79.
    Eisler, Z., Kertesz, J.: Eur. Phys. J. B 51, 145 (2006)Google Scholar
  80. 80.
    Mandelbrot, B.: J. Bus. 394–419 (1963)Google Scholar
  81. 81.
    Cont, R.: Quant. Finance 1, 223 (2001)Google Scholar
  82. 82.
    Bouchaud, J., Cont, R., Potters, M.: Scale invariance and beyond. In: Proceedings of CNRS Workshop on Scale Invariance. Springer, Les Houches (1997)Google Scholar
  83. 83.
    Ding, Z., Engle, R., Granger, C.: J. Empir. Finance 1, 83 (1993)Google Scholar
  84. 84.
    Lobato, I.N., Savin, N.E.: J. Bus. Econ. Stat. 16, 261 (1998)Google Scholar
  85. 85.
    Vandewalle, N., Ausloos, M.: Phys. A 246, 454 (1997)Google Scholar
  86. 86.
    Mandelbrot, B., Fisher, A., Calvet, L.: A multifractal model of asset returns. Cowles Foundation for Research and Economics (1997)Google Scholar
  87. 87.
    de Vries, C., Leuven, K.: Stylized facts of nominal exchange rate returns. Working Papers from Purdue University, Krannert School of Management Center for International Business Education and Research (CIBER) (1994)Google Scholar
  88. 88.
    Pagan, A.: J. Empir. Finance 3, 15 (1996)Google Scholar
  89. 89.
    Cont, R.: Volatility clustering in financial markets: empirical facts and agent-based models. In: Kirman, A., Teyssiere, G. (eds.) Long Memory in Economics. Springer (2005)Google Scholar
  90. 90.
    Fama, E.: J. Finance 25, 383 (1970)Google Scholar
  91. 91.
    Bera, A.K., Ivliev, S., Lillo, F.: Financial Econometrics and Empirical Market Microstructure. Springer, Cham (2015)Google Scholar
  92. 92.
    Wiesinger, J., Sornette, D., Satinover, J.: Comput. Econ. 41(4), 475 (2012)Google Scholar
  93. 93.
    Andersen, J.V., Sornette, D.: Europhys. Lett. 70(5), 697 (2005)MathSciNetGoogle Scholar
  94. 94.
    Zhang, Q.: Disentangling financial markets and social networks: models and empirical tests. Ph.D. thesis, ETH Zurich (2013)Google Scholar
  95. 95.
    Friedman, M.: Essays in Positive Economics. Chicago University Press, Chicago (1953)Google Scholar
  96. 96.
    Canova, F., Sala, L.: J. Monetary Econ. 56(4), 431 (2009)Google Scholar
  97. 97.
    Chiarella, C., Iori, G., Perello, J.: J. Econ. Dyn. Control 33, 525 (2009)Google Scholar
  98. 98.
    Leal, S.J., Napoletano, M., Roventini, A., Fagiolo, G.: J. Evol. Econ. 26, 49 (2016)Google Scholar
  99. 99.
    Fabretti, A.: J. Econ. Interact. Coord. 8, 277 (2013)Google Scholar
  100. 100.
    Axtell, R.: Center on social and economic dynamics working paper 17 (2000)Google Scholar
  101. 101.
    Gilli, M., Winker, P.: Comput. Stat. Data Anal. 42, 299 (2003)Google Scholar
  102. 102.
    Farmer, J.D., Joshi, S.: J. Econ. Behav. Organ. 49, 149 (2002)Google Scholar
  103. 103.
    Kirman, A.: Epidemics of opinion and speculative bubbles in financial markets. In: Money and Financial Markets. Macmillan, New York (1991)Google Scholar
  104. 104.
    Glimcher, P.W., Camerer, C.F., Fehr, E., Poldrack, R.A.: Neuroeconomics: Decision Making and the Brain. Academic Press, Cambridge (2009)Google Scholar
  105. 105.
    Camerer, C.: J. Econ. Lit. 51(4), 1155 (2013)Google Scholar
  106. 106.
    Martino, B.D., Doherty, J.P.O., Ray, D., Bossaerts, P., Camerer, C.: Neuron 79(6), 1222 (2013)Google Scholar
  107. 107.
    Camerer, C.: Ann. Rev. Econ. 5, 425 (2013)Google Scholar
  108. 108.
    Camerer, C.: Neuroscience, game theory, monkeys. TEDx talk (2013)Google Scholar
  109. 109.
    Kahneman, D., Tversky, A.: Econometrica 47(2), 263 (1979)MathSciNetGoogle Scholar
  110. 110.
    Frydman, C., Barberis, N., Camerer, C., Bossaerts, P., Rangel, A.: NBER working paper 18562 (2012)Google Scholar
  111. 111.
    Camerer, C.F.: Behavioral Game Theory: Experiments on Strategic Interaction. Princeton University Press, Princeton (2003)zbMATHGoogle Scholar
  112. 112.
    Silver, D., Hubert, T., Schrittwieser, J., Antonoglou, I., Lai, M., Guez, A., Lanctot, M., Sifre, L., Kumaran, D., Graepel, T., et al.: Science 362, 1140 (2018)MathSciNetGoogle Scholar
  113. 113.
    Silver, D., Schrittwieser, J., Simonyan, K., Antonoglou, I., Huang, A., Guez, A., Hubert, T., Baker, L., Lai, M., Bolton, A., et al.: Nature 550, 354 (2017)Google Scholar
  114. 114.
    Doll, B.B., Duncan, K.D., Simon, D.A., Shohamy, D.S., Daw, N.D.: Nat. Neurosci. 18, 767 (2015)Google Scholar
  115. 115.
    Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. MIT Press, Cambridge (2016)zbMATHGoogle Scholar
  116. 116.
    Schmidhuber, J.: Neural Netw. 61, 85 (2015)Google Scholar
  117. 117.
    Turchenko, V., Beraldi, P., Simone, F.D., Grandinetti, L.: The 6th IEEE International Conference on Intelligent Data Acquisition and Advanced Computing Systems: Technology and Applications (2011)Google Scholar
  118. 118.
    Silver, D., Huang, A., Maddison, C.J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., et al.: Nature 529, 484 (2016)Google Scholar
  119. 119.
    Tuyls, K., Weiss, G.: AI Mag. Fall (2012) Google Scholar
  120. 120.
    Heinrich, J., Silver, D.: AAAI Workshop (2014)Google Scholar
  121. 121.
    Heinrich, J., Silver, D.: IJCAI (2015)Google Scholar
  122. 122.
    Reed, S., Akata, Z., Yan, X., Logeswaran, L., Lee, H., Schiele, B.: ICML (2016)Google Scholar
  123. 123.
    Lerer, A., Gross, S., Fergus, R.: ICML (2016)Google Scholar
  124. 124.
    Biondo, A.E.: J. Econ. Interact. Coord. 14(3) (2018)Google Scholar
  125. 125.
    Spooner, T., Fearnley, J., Savani, R., Koukorinis, A.: Proceedings of the 17th AAMAS (2018)Google Scholar
  126. 126.
    Ganesh, S., Vadori, N., Xu, M., Zheng, H., Reddy, P., Veloso, M.: arXiv:1911.05892 (2019)
  127. 127.
    Lefebvre, G., Lebreton, M., Meyniel, F., BourgeoisGironde, S., Palminteri, S.: Nat. Hum. Behav. 1, 1 (2017)Google Scholar
  128. 128.
    Duncan, K., Doll, B.B., Daw, N.D., Shohamy, D.: Neuron 98, 645 (2018)Google Scholar
  129. 129.
    Momennejad, I., Russek, E., Cheong, J., Botvinick, M., Daw, N.D., Gershman, S.J.: Nat. Hum. Behav. 1, 680–692 (2017)Google Scholar
  130. 130.
    Palminteri, S., Khamassi, M., Joffily, M., Coricelli, G.: Nat. Commun. 1–14 (2015)Google Scholar
  131. 131.
    Laibson, D.: Q. J. Econ. 112(2), 443 (1997)Google Scholar
  132. 132.
    The financial crisis inquiry report. Official government edition (2011)Google Scholar
  133. 133.
    Fouque, J.-P., Langsam, J.A.: Handbook on Systemic Risk. Cambridge University Press, Cambridge (2013)zbMATHGoogle Scholar
  134. 134.
    Bikhchandani, S., Sharma, S.: Int. Monetary Fund. 47(3) (2001)Google Scholar
  135. 135.
    Bikhchandani, S., Hirshleifer, D., Welch, I.: J. Polit. Econ. 100(5), 992 (1992)Google Scholar
  136. 136.
    Fama, E.: J. Bus. 38, 34 (1965)Google Scholar
  137. 137.
    Sornette, D.: arXiv:1404.0243v1 (2014)
  138. 138.
    da Costa Pereira, C., Mauri, A., Tettamanzi, A.G.B.: IEEE Computer Society WIC ACM (2009)Google Scholar
  139. 139.
    Kyle, A.S.: Econometrica 53, 1315 (1985)Google Scholar
  140. 140.
    Sanford, G.J., Miller, M.H.: J. Finance 43, 617 (1988)Google Scholar
  141. 141.
    Grossman, S.J., Stiglitz, J.E.: Am. Econ. Rev. 70, 393 (1980)Google Scholar
  142. 142.
    Cason, T.N., Friedman, D.: Exp. Econ. 2, 77 (1999)Google Scholar
  143. 143.
    Evstigneev, I.V., Hens, T., Schenk-Hopp, K.R.: Evolutionary finance. In: Handbook of Financial Markets, Dynamics and Evolution. North-Holland, Elsevier (2009)Google Scholar
  144. 144.
    Saichev, A., Malevergne, Y., Sornette, D.: Theory of Zipf’s Law and Beyond. Lecture Notes in Economics and Mathematical Systems, vol. 632. Springer, Heidelberg (2010)zbMATHGoogle Scholar
  145. 145.
    Malevergne, Y., Saichev, A., Sornette, D.: J. Econ. Dyn. Control 37(6), 1195 (2013)Google Scholar
  146. 146.
    Hasanhodzic, J., Lo, A.W., Viola, E.: Quant. Finance 11(7), 1043 (2011)MathSciNetGoogle Scholar
  147. 147.
    Malkiel, B.G.: A Random Walk Down Wall Street: The Time-Tested Strategy for Successful Investing, 10th revised edn. W. W. Norton and Company (2012)Google Scholar
  148. 148.
    Black, F.: J. Finance 41(3), 529 (1985)Google Scholar
  149. 149.
    Lussange, J., Bourgeois-Gironde, S., Palminteri, S., Gutkin, B.: arXiv:1909.07748 (2019)
  150. 150.
    Lussange, J., Belianin, A., Bourgeois-Gironde, S., Gutkin, B.: arXiv:1801.08222 (2018)

Copyright information

© Springer Nature Switzerland AG 2021

Authors and Affiliations

  • Johann Lussange
    • 1
    Email author
  • Alexis Belianin
    • 2
  • Sacha Bourgeois-Gironde
    • 3
    • 4
  • Boris Gutkin
    • 1
    • 5
  1. 1.Laboratoire des Neurosciences Cognitives, INSERM U960, Département des Études Cognitives, École Normale Supérieure PSL UniversityParisFrance
  2. 2.ICEF, National Research University Higher School of Economics and Primakov Institute for World Economy and International RelationsRussian Academy of SciencesMoscowRussia
  3. 3.Institut Jean-Nicod, UMR 8129, Département des Études Cognitives, École Normale Supérieure PSL UniversityParisFrance
  4. 4.Laboratoire d’Économie Mathématique et de Microéconomie Appliquée, EA 4442, Université Paris II Panthéon-AssasParisFrance
  5. 5.Center for Cognition and Decision Making, Department of PsychologyNU University Higher School of EconomicsMoscowRussia

Personalised recommendations