Advertisement

Dynamic Games and Applications

, Volume 9, Issue 4, pp 1100–1125 | Cite as

Risk-Sensitive Mean Field Games via the Stochastic Maximum Principle

  • Jun MoonEmail author
  • Tamer Başar
Article
  • 168 Downloads

Abstract

In this paper, we consider risk-sensitive mean field games via the risk-sensitive maximum principle. The problem is analyzed through two sequential steps: (i) risk-sensitive optimal control for a fixed probability measure, and (ii) the associated fixed-point problem. For step (i), we use the risk-sensitive maximum principle to obtain the optimal solution, which is characterized in terms of the associated forward–backward stochastic differential equation (FBSDE). In step (ii), we solve for the probability law induced by the state process with the optimal control in step (i). In particular, we show the existence of the fixed point of the probability law of the state process determined by step (i) via Schauder’s fixed-point theorem. After analyzing steps (i) and (ii), we prove that the set of N optimal distributed controls obtained from steps (i) and (ii) constitutes an approximate Nash equilibrium or \(\epsilon \)-Nash equilibrium for the N player risk-sensitive game, where \(\epsilon \rightarrow 0\) as \(N \rightarrow \infty \) at the rate of \(O(\frac{1}{N^{1/(n+4)}})\). Finally, we discuss extensions to heterogeneous (non-symmetric) risk-sensitive mean field games.

Keywords

Mean field game theory Risk-sensitive optimal control Forward–backward stochastic differential equations Decentralized control 

Notes

Acknowledgements

The authors would like to thank the Associate Editor and the two anonymous reviewers for careful reading of and helpful suggestions on the earlier version of the manuscript. This research was supported in part by the National Research Foundation of Korea (NRF) Grant funded by the Ministry of Science and ICT, South Korea (NRF-2017R1E1A1A03070936, NRF-2017R1A5A1015311), in part by Institute for Information and Communications Technology Promotion (IITP) Grant funded by the Korea government (MSIT), South Korea (No. 2018-0-00958), and in part by the Office of Naval Research (ONR) MURI Grant N00014-16-1-2710.

References

  1. 1.
    Achdou Y, Capuzzo-Dolcetta I (2010) Mean field games: numerical methods. SIAM J Numer Anal 48(3):1136–1162MathSciNetzbMATHGoogle Scholar
  2. 2.
    Achdou Y, Camilli F, Capuzzo-Dolcetta I (2012) Mean field games: numerical methods for the planning problem. SIAM J Control Opt 50(1):77–109MathSciNetzbMATHGoogle Scholar
  3. 3.
    Ahuja S (2016) Wellposedness of mean field games with common noise under a weak monotonicity condition. SIAM J Control Opt 54(1):30–48MathSciNetzbMATHGoogle Scholar
  4. 4.
    Andersson D, Djehiche B (2010) A maximum principle for SDEs of mean-field type. Appl Math Opt 63(3):341–356MathSciNetzbMATHGoogle Scholar
  5. 5.
    Başar T (1999) Nash equilibria of risk-sensitive nonlinear stochastic differential games. J Opt Theory Appl 100(3):479–498MathSciNetzbMATHGoogle Scholar
  6. 6.
    Başar T, Olsder GJ (1999) Dynamic noncooperative game theory, 2nd edn. SIAM, PhiladelphiazbMATHGoogle Scholar
  7. 7.
    Başar T, Bernhard P (1995) \(\text{ H }^\infty \) optimal control and related minimax design problems, 2nd edn. Birkhäuser, BostonzbMATHGoogle Scholar
  8. 8.
    Bardi M, Priuli FS (2014) Linear-quadratic N-person and mean-field games with ergodic cost. SIAM J Control Opt 52(5):3022–3052MathSciNetzbMATHGoogle Scholar
  9. 9.
    Bauso D, Tembine H, Başar T (2016) Robust mean field games. Dyn Games Appl 6(3):277–303MathSciNetzbMATHGoogle Scholar
  10. 10.
    Bauso D, Tembine H, Başar T (2016) Opinion dynamics in social networks through mean-field games. SIAM J Control Opt 54(6):3225–3257MathSciNetzbMATHGoogle Scholar
  11. 11.
    Bensoussan A, Frehse J, Yam P (2013) Mean field games and mean field type control theory. Springer, New YorkzbMATHGoogle Scholar
  12. 12.
    Bensoussan A, Sung KCJ, Yam SCP, Yung SP (2014) Linear–quadratic mean field games. arXiv:1404.5741
  13. 13.
    Bensoussan A, Chau MHM, Yam SCP (2015) Mean field Stackelberg games: aggregation of delayed instructions. SIAM J Control Opt 53(4):2237–2266MathSciNetzbMATHGoogle Scholar
  14. 14.
    Billingsley P (1999) Convergence of probability measures, 2nd edn. Wiley, New YorkzbMATHGoogle Scholar
  15. 15.
    Bolley F (2008) Separability and completeness for the Wasserstein distance. Seminarire de probabilities XLI, Lecture Notes Math, pp 371–377Google Scholar
  16. 16.
    Cardaliaguet P (2012) Notes on mean field games. Technical reportGoogle Scholar
  17. 17.
    Cardaliaguet P, Lehalle CA (2018) Mean field game of controls and an application to trade crowding. Math Financ Econ 12(3):335–363MathSciNetzbMATHGoogle Scholar
  18. 18.
    Carmona R, Delarue F (2013) Mean field forward–backward stochastic differential equations. Electron Commun Probab 18(68):1–15MathSciNetzbMATHGoogle Scholar
  19. 19.
    Carmona R, Delarue F (2013) Probabilistic analysis of mean-field games. SIAM J Control Opt 51(4):2705–2734MathSciNetzbMATHGoogle Scholar
  20. 20.
    Carmona R, Delarue F (2015) Forward–backward stochastic differential equations and controlled McKean–Vlasov dynamics. Ann Probab 43(5):2647–2700MathSciNetzbMATHGoogle Scholar
  21. 21.
    Carmona R, Delarue F, Lachapelle A (2012) Control of McKean–Vlasov dynamics versus mean field games. Math Financ Econ 7(2):131MathSciNetzbMATHGoogle Scholar
  22. 22.
    Conway JB (2000) A course in functional analysis. Springer, BerlinGoogle Scholar
  23. 23.
    Cormen TH, Leiserson CE, Rivest RL, Stein C (2009) Introduction to algorithms, 3rd edn. The MIT Press, LondonzbMATHGoogle Scholar
  24. 24.
    Couillet R, Perlaza SM, Tembine H, Debbah M (2012) Electrical vehicles in the smart grid: a mean field game analysis. IEEE J Sel Areas Commun 30(6):1086–1096Google Scholar
  25. 25.
    Delarue F (2002) On the existence and uniqueness of solutions to FBSDEs in a non-degenerate case. Stoch Process Appl 99:209–286MathSciNetzbMATHGoogle Scholar
  26. 26.
    Djehiche B, Hamadene S (2016) Optimal control and zero-sum stochastic differential game problems of mean-field type. arXiv:1603.06071v3
  27. 27.
    Duncan TE (2013) Linear–exponential–quadratic Gaussian control. IEEE Trans Autom Control 58(11):2910–2911MathSciNetzbMATHGoogle Scholar
  28. 28.
    Fleming W, Rishel R (1975) Deterministic and stochastic optimal control. Springer, BerlinzbMATHGoogle Scholar
  29. 29.
    Fleming W, Soner HM (2006) Controlled Markov processes and viscosity solutions, 2nd edn. Springer, BerlinzbMATHGoogle Scholar
  30. 30.
    Fleming WH, James MR (1995) The risk-sensitive index and the \(\text{ H }_2\) and \(\text{ H }_\infty \) norms for nonlinear systems. Math Control Signals Syst 8:199–221Google Scholar
  31. 31.
    Horowitz J, Karandikar R (1994) Mean rate of convergence of empirical measures in the Wasserstein metric. J Comput Appl Math 55:261–273MathSciNetzbMATHGoogle Scholar
  32. 32.
    Huang J, Li N (2018) Linear–quadratic mean-field game for stochastic delayed systems. IEEE Trans Autom Control 63(8):2722–2729MathSciNetzbMATHGoogle Scholar
  33. 33.
    Huang M (2010) Large-population LQG games involving a major player: the Nash certainty equivalence principle. SIAM J Control Opt 48(5):3318–3353MathSciNetzbMATHGoogle Scholar
  34. 34.
    Huang M, Caines PE, Malhamé RP (2003) Individual and mass behaviour in large population stochastic wireless power control problems: centralized and Nash equilibrium solutions. In: Proceedings of the 42nd IEEE conference on decision and control, pp 98–103Google Scholar
  35. 35.
    Huang M, Malhamé Roland P, Caines Peter E (2006) Large population stochastic dynamic games: closed-loop McKean–Vlasov systems and the Nash certainty equivalence principle. Commun Inf Syst 6(3):221–252MathSciNetzbMATHGoogle Scholar
  36. 36.
    Huang M, Caines PE, Malhamé RP (2007) Large-population cost-coupled LQG problems with nonuniform agents: individual-mass behavior and decentralized \(\epsilon \)-Nash equilibria. IEEE Trans Autom Control 52(9):1560–1571MathSciNetzbMATHGoogle Scholar
  37. 37.
    Jacobson D (1973) Optimal stochastic linear systems with exponential performance criteria and their relation to deterministic differential games. IEEE Trans Autom Control 18(2):124–131MathSciNetzbMATHGoogle Scholar
  38. 38.
    Jordain B, Meleard S, Woyczynski W (2008) Nonlinear SDEs driven by Levy processes and related PDEs. Latin Am J Probab 4:1–29zbMATHGoogle Scholar
  39. 39.
    Karatzas I, Shreve SE (2000) Brownian motion and stochastic calculus. Springer, BerlinzbMATHGoogle Scholar
  40. 40.
    Lasry JM, Lions PL (2007) Mean field games. Jpn J Math 2(1):229–260MathSciNetzbMATHGoogle Scholar
  41. 41.
    Li T, Zhang JF (2008) Asymptotically optimal decentralized control for large population stochastic multiagent systems. IEEE Trans Autom Control 53(7):1643–1660MathSciNetzbMATHGoogle Scholar
  42. 42.
    Lim EBA, Zhou XY (2005) A new risk-sensitive maximum principle. IEEE Trans Autom Control 50(7):958–966MathSciNetzbMATHGoogle Scholar
  43. 43.
    Luenberger DG (1969) Optimization by vector space methods. Wiley, New YorkzbMATHGoogle Scholar
  44. 44.
    Ma J, Yong J (1999) Forward–backward stochastic differential equations and their applications. Springer, BerlinzbMATHGoogle Scholar
  45. 45.
    Ma J, Callaway DS, Hiskens IA (2013) Decentralized charging control of large populations of plug-in electric vehicles. IEEE Trans Control Syst Technol 21(1):67–78Google Scholar
  46. 46.
    Moon J, Başar T (2015) Linear-quadratic stochastic differential Stackelberg games with a high population of followers. In: Proceedings of the 54th IEEE conference on decision and control, pp 2270–2275Google Scholar
  47. 47.
    Moon J, Başar T (2016) Robust mean field games for coupled Markov jump linear systems. Int J Control 89(7):1367–1381MathSciNetzbMATHGoogle Scholar
  48. 48.
    Moon J, Başar T (2017) Linear quadratic risk-sensitive and robust mean field games. IEEE Trans Autom Control 62(3):1062–1077MathSciNetzbMATHGoogle Scholar
  49. 49.
    Moon J, Başar T (2018) Linear quadratic mean field Stackelberg differential games. Automatica 97:200–213MathSciNetzbMATHGoogle Scholar
  50. 50.
    Nourian M, Caines P (2013) \(\epsilon \)-Nash mean field game theory for nonlinear stochastic dynamical systems with major and minor agents. SIAM J Control Opt 51(4):3302–3331MathSciNetzbMATHGoogle Scholar
  51. 51.
    Nourian M, Caines PE, Malhamé RP, Huang Minyi (2013) Nash, social and centralized solutions to consensus problems via mean field control theory. IEEE Trans Autom Control 58(3):639–653MathSciNetzbMATHGoogle Scholar
  52. 52.
    Nourian M, Caines PE, Malhamé RP (2014) A mean field game synthesis of initial mean consensus problems: a continuum approach for non-Gaussian behavior. IEEE Trans Autom Control 59(2):449–455MathSciNetzbMATHGoogle Scholar
  53. 53.
    Pardoux E, Tang S (1999) Forward–backward stochastic differential equations and quasilinear parabolic PDEs. Probab Theory Relat Fields 114:123–150MathSciNetzbMATHGoogle Scholar
  54. 54.
    Parise F, Grammatico S, Colombino M, Lygeros J (2014) Mean field constrained charging policy for large populations of plug-in electric vehicles. In: Proceedings of 53rd IEEE conference on decision and control, pp 5101–5106Google Scholar
  55. 55.
    Peng S, Wu Z (1999) Fully coupled forward–backward stochastic differential equations and applications to optimal control. SIAM J Control Opt 37(3):825–843MathSciNetzbMATHGoogle Scholar
  56. 56.
    Pham H (2009) Continuous-time stochastic control and optimization with financial applications. Springer, BerlinzbMATHGoogle Scholar
  57. 57.
    Rachev ST, Ruschendorf L (1998) Mass transportation theory: volume I: theory. Springer, BerlinzbMATHGoogle Scholar
  58. 58.
    Smart DR (1974) Fixed point theorems. Cambridge University Press, CambridgezbMATHGoogle Scholar
  59. 59.
    Tembine H, Zhu Q, Başar T (2014) Risk-sensitive mean field games. IEEE Trans Autom Control 59(4):835–850MathSciNetzbMATHGoogle Scholar
  60. 60.
    Touzi N (2013) Optimal stochastic control, stochastic target problems, and backward SDE. Springer, BerlinzbMATHGoogle Scholar
  61. 61.
    Wang B, Zhang J (2012) Mean field games for large-population multiagent systems with Markov jump parameters. SIAM J Control Opt 50(4):2308–2334MathSciNetzbMATHGoogle Scholar
  62. 62.
    Weintraub GY, Benkard CL, Van Roy B (2008) Markov perfect industry dynamics with many firms. Econometrica 76(6):1375–1411MathSciNetzbMATHGoogle Scholar
  63. 63.
    Whittle P (1990) Risk-sensitive optimal control. Wiley, New YorkzbMATHGoogle Scholar
  64. 64.
    Yin H, Mehta PG, Meyn SP, Shanbhag UV (2012) Synchronization of coupled oscillators is a game. IEEE Trans Autom Control 57(4):920–935MathSciNetzbMATHGoogle Scholar
  65. 65.
    Yong J, Zhou XY (1999) Stochastic controls: Hamiltonian systems and HJB equations. Springer, BerlinzbMATHGoogle Scholar
  66. 66.
    Zhou XY (1996) Sufficient conditions of optimality for stochastic systems with controllable diffusions. IEEE Trans Autom Control 41(8):1176–1179MathSciNetzbMATHGoogle Scholar
  67. 67.
    Zhu Q, Başar T (2013) Multi-resolution large population stochastic differential games and their application to demand response management in the smart grid. Dyn Games Appl 3(1):66–88MathSciNetzbMATHGoogle Scholar
  68. 68.
    Zhu Q, Tembine H, Başar T (2011) Hybrid risk-sensitive mean-field stochastic differential games with application to molecular biology. In: Proceedings of the 50th IEEE CDC and ECC, pp 4491–4497Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.School of Electrical and Computer EngineeringUlsan National Institute of Science and Technology (UNIST)UlsanSouth Korea
  2. 2.Coordinated Science LaboratoryUniversity of Illinois at Urbana-ChampaignUrbanaUSA

Personalised recommendations