Skip to main content
SpringerLink
Log in

Multi-agent Financial Systems with RL: A Pension Ecosystem Case

  • Conference paper
  • First Online:
Multi-Agent-Based Simulation XXIV (MABS 2023)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 14558))

Abstract

This paper introduces a multi-agent reinforcement learning (MARL) model for the pension ecosystem, aiming to optimise the contributor’s saving and investment strategies. The multi-agent approach enables the examination of endogenous and exogenous shocks, business cycle impacts, and policy decisions on contributor behaviour. The model generates synthetic income trajectories to develop inclusive savings strategies for a broad population. Additionally, this research innovates by developing a multi-agent model capable of adapting to various environmental changes, contrasting with traditional econometric models that assume stationary employment and market dynamics. The non-stationary nature of the model allows for a more realistic representation of economic systems, enabling a better understanding of the complex interplay between agents and their responses to evolving economic conditions (A variation of this article was included as a chapter in the PhD Thesis of Ozhamaratli, F. submitted on 22 Jan 2024).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Asano, Y.M., Kolb, J.J., Heitzig, J., Farmer, J.D.: Emergent inequality and endogenous dynamics in a simple behavioral macroeconomic model (Jul 2019). http://arxiv.org/abs/1907.02155, arXiv:1907.02155 [econ, q-fin]

  2. Bank of England: Announcement of additional measures to support market functioning (10 2022). https://www.bankofengland.co.uk/news/2022/october/bank-of-england-announces-additional-measures-to-support-market-functioning

  3. Bradbury, J., et al.: JAX: composable transformations of Python+NumPy programs (2018). http://github.com/google/jax

  4. Campanale, C., Fugazza, C., Gomes, F.: Life-cycle portfolio choice with liquid and illiquid financial assets. Journal of Monetary Economics 71, 67–83 (Apr 2015). https://doi.org/10.1016/j.jmoneco.2014.11.008, https://linkinghub.elsevier.com/retrieve/pii/S0304393214001652

  5. Campbell, J.Y., Viceira, L.M.: Strategic asset allocation: portfolio choice for long-term investors. Oxford University Press, New York (2002)

    Google Scholar 

  6. Cocco, J.F., Gomes, F.J., Maenhout, P.J.: Consumption and Portfolio Choice over the Life Cycle. Rev. Finan. Stud. 18(2), 491–533 (2005). https://doi.org/10.1093/rfs/hhi017, https://academic.oup.com/rfs/article-lookup/doi/10.1093/rfs/hhi017

  7. Cont, R., Wagalath, L.: FIRE SALES FORENSICS: MEASURING ENDOGENOUS RISK: FIRE SALES FORENSICS: MEASURING ENDOGENOUS RISK. Mathematical Finance 26(4), 835–866 (Oct 2016). https://doi.org/10.1111/mafi.12071, https://onlinelibrary.wiley.com/doi/10.1111/mafi.12071

  8. Frostig, R., Johnson, M.J., Leary, C.: Compiling machine learning programs via high-level tracing. Syst. Mach. Learn. 4(9) (2018)

    Google Scholar 

  9. Gibbons, R., Waldman, M.: Task-specific human capital. American Econom. Rev. 94(2), 203–207 (2004). http://www.jstor.org/stable/3592883

  10. Gu, S., Holly, E., Lillicrap, T., Levine, S.: Deep reinforcement learning for robotic manipulation with asynchronous off-policy updates. In: 2017 IEEE International Conference on Robotics and Automation (ICRA), pp. 3389–3396. IEEE (2017)

    Google Scholar 

  11. Guvenen, F., Ozkan, S., Song, J.: The Nature of Countercyclical Income Risk (May 2012). https://doi.org/10.3386/w18035, https://www.nber.org/papers/w18035

  12. Harris, C.R., et al.: Array programming with NumPy. Nature 585(7825), 357–362 (Sep 2020). https://doi.org/10.1038/s41586-020-2649-2, https://doi.org/10.1038/s41586-020-2649-2

  13. Heek, J., et al.: Flax: A neural network library and ecosystem for JAX (2020). http://github.com/google/flax

  14. Impavido, G., Tower, I.: How the financial crisis affects pensions and insurance and why the impacts matter (Jul 2009)

    Google Scholar 

  15. Lowe, R., Wu, Y., Tamar, A., Harb, J., Abbeel, P., Mordatch, I.: Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments (Mar 2020). https://doi.org/10.48550/arXiv.1706.02275, http://arxiv.org/abs/1706.02275, arXiv:1706.02275 [cs]

  16. Merton, R.C.: Optimum consumption and portfolio rules in a continuous-time model. J. Econom. Theor. 3(4), 373–413 (Dec 1971). https://doi.org/10.1016/0022-0531(71)90038-X, https://linkinghub.elsevier.com/retrieve/pii/002205317190038X

  17. Mnih, V., et al.: Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602 (2013)

  18. Mnih, V., et al.: Human-level control through deep reinforcement learning. Nature 518(7540), 529–533 (2015)

    Google Scholar 

  19. Narvekar, S., Peng, B., Leonetti, M., Sinapov, J., Taylor, M.E., Stone, P.: Curriculum learning for reinforcement learning domains: A framework and survey. arXiv preprint arXiv:2003.04960 (2020)

  20. Papaioannou, M.G., Rentsendorj, B.: Sovereign wealth fund asset allocations-some stylized facts on the norway pension fund global. Procedia Econom. Finance 29, 195–199 (2015). https://doi.org/10.1016/S2212-5671(15)01122-3, https://linkinghub.elsevier.com/retrieve/pii/S2212567115011223

  21. Pichler, A., Farmer, J.D.: Simultaneous supply and demand constraints in input-output networks: the case of Covid-19 in Germany, Italy, and Spain. Econom. Syst. Res. 34(3), 273–293 (Jul 2022). https://doi.org/10.1080/09535314.2021.1926934, https://doi.org/10.1080/09535314.2021.1926934

  22. Rashid, T., Samvelyan, M., de Witt, C.S., Farquhar, G., Foerster, J., Whiteson, S.: QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning (Jun 2018). https://doi.org/10.48550/arXiv.1803.11485, http://arxiv.org/abs/1803.11485, arXiv:1803.11485 [cs, stat]

  23. Ravi, S., Larochelle, H.: Optimization as a model for few-shot learning. In: International Conference on Learning Representations (2017)

    Google Scholar 

  24. del Rio-Chanona, R.M., Mealy, P., Beguerisse-Díaz, M., Lafond, F., Farmer, J.D.: Occupational mobility and automation: a data-driven network model. J. Royal Society Interface 18(174), 20200898 (Jan 2021). https://doi.org/10.1098/rsif.2020.0898, https://royalsocietypublishing.org/doi/10.1098/rsif.2020.0898

  25. Samvelyan, M., et al.: The StarCraft Multi-Agent Challenge (Dec 2019), http://arxiv.org/abs/1902.04043, arXiv:1902.04043 [cs, stat]

  26. Schulman, J., Wolski, F., Dhariwal, P., Radford, A., Klimov, O.: Proximal policy optimization algorithms (2017). https://doi.org/10.48550/ARXIV.1707.06347, https://arxiv.org/abs/1707.06347

  27. Schulman, J., Wolski, F., Dhariwal, P., Radford, A., Klimov, O.: Proximal Policy Optimization Algorithms (Aug 2017). https://doi.org/10.48550/arXiv.1707.06347, http://arxiv.org/abs/1707.06347, arXiv:1707.06347 [cs]

  28. Son, K., Kim, D., Kang, W.J., Hostallero, D.E., Yi, Y.: QTRAN: Learning to Factorize with Transformation for Cooperative Multi-Agent Reinforcement Learning (May 2019). https://doi.org/10.48550/arXiv.1905.05408, http://arxiv.org/abs/1905.05408, arXiv:1905.05408 [cs, stat]

  29. Yang, Y., Wang, J.: An Overview of Multi-Agent Reinforcement Learning from Game Theoretical Perspective (Mar 2021). http://arxiv.org/abs/2011.00583, arXiv:2011.00583 [cs]

  30. Yu, C., Velu, A., Vinitsky, E., Wang, Y., Bayen, A., Wu, Y.: The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Games (Jul 2022). http://arxiv.org/abs/2103.01955, arXiv:2103.01955 [cs]

  31. Zheng, S., et al.: The AI Economist: Improving Equality and Productivity with AI-Driven Tax Policies. arXiv:2004.13332 [cs, econ, q-fin, stat] (Apr 2020), http://arxiv.org/abs/2004.13332, arXiv: 2004.13332

Download references

Author information

Authors and Affiliations

  1. University College London, 66-72 Gower Street, WC1E 6EA, London, UK

    Fatih Ozhamaratli & Paolo Barucca

Authors
  1. Fatih Ozhamaratli
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Paolo Barucca
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fatih Ozhamaratli .

Editor information

Editors and Affiliations

  1. Clermont Auvergne University, Saint‐Étienne, France

    Luis G. Nardin

  2. Grantham Research Institute on Climate Change and the Environment, London, UK

    Sara Mehryar

7 Appendix

7 Appendix

Table 2. Input-Output Matrix of Sectors excluding inv_cash
Full size table
Table 3. Consumption Vector
Full size table
Table 4. Model Card Parameters
Full size table

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Ozhamaratli, F., Barucca, P. (2024). Multi-agent Financial Systems with RL: A Pension Ecosystem Case. In: Nardin, L.G., Mehryar, S. (eds) Multi-Agent-Based Simulation XXIV. MABS 2023. Lecture Notes in Computer Science(), vol 14558. Springer, Cham. https://doi.org/10.1007/978-3-031-61034-9_5

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-61034-9_5

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-61033-2

  • Online ISBN: 978-3-031-61034-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation