Skip to main content

Mean Field Control Hierarchy

Applied Mathematics & Optimization volume 76, pages 93–135 (2017)Cite this article

Abstract

In this paper we model the role of a government of a large population as a mean field optimal control problem. Such control problems are constrained by a PDE of continuity-type, governing the dynamics of the probability distribution of the agent population. We show the existence of mean field optimal controls both in the stochastic and deterministic setting. We derive rigorously the first order optimality conditions useful for numerical computation of mean field optimal controls. We introduce a novel approximating hierarchy of sub-optimal controls based on a Boltzmann approach, whose computation requires a very moderate numerical complexity with respect to the one of the optimal control. We provide numerical experiments for models in opinion formation comparing the behavior of the control hierarchy.

This is a preview of subscription content, access via your institution.

We’re sorry, something doesn't seem to be working properly.

Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.

Fig. 1
Fig. 2
Fig. 3

References

  1. Achdou, Y., Laurière, M.: Mean field type control with congestion. Appl. Math. Optim. 73(3), 393–418 (2016)

    MathSciNet  Article  MATH  Google Scholar 

  2. Achdou, Y., Camilli, F., Capuzzo-Dolcetta, I.: Mean field games: numerical methods for the planning problem. SIAM J. Control Optim. 50(1), 77–109 (2012)

    MathSciNet  Article  MATH  Google Scholar 

  3. Albi, G., Pareschi, L.: Binary interaction algorithms for the simulation of flocking and swarming dynamics. Multiscale Model. Simul. 11, 1–29 (2013)

    MathSciNet  Article  MATH  Google Scholar 

  4. Albi, G., Pareschi, L.: Modeling of self-organized systems interacting with a few individuals: from microscopic to macroscopic dynamics. Appl. Math. Lett. 26, 397–401 (2013)

    MathSciNet  Article  MATH  Google Scholar 

  5. Albi, G., Pareschi, L., Zanella, M.: Boltzmann-type control of opinion consensus through leaders. Philos. Trans. R. Soc. A 372, 20140138/1–20140138/18 (2014)

  6. Albi, G., Herty, M., Pareschi, L.: Kinetic description of optimal control problems and applications to opinion consensus. Commun. Math. Sci. 13(6), 1407–1429 (2015)

    MathSciNet  Article  MATH  Google Scholar 

  7. Albi, G., Bongini, M., Cristiani, E., Kalise, D.: Invisible control of self-organizing agents leaving unknown environments. SIAM J. Appl. Math. 76, 1683–1710 (2016)

    MathSciNet  Article  MATH  Google Scholar 

  8. Albi, G., Pareschi, L., Toscani, G., Zanella, M.: Recent advances in opinion modeling: control and social influence. In: Bellomo, N., Degond, P., Tadmor, E. (eds.) Active Particles, Vol. 1: Theory, Methods, and Applications. Birkhauser-Springer, Boston (2016)

  9. Albi, G., Pareschi, L., Zanella, M.: Opinion dynamics over complex networks: kinetic modeling and numerical methods. Kinet. Relat. Mod. 10(1), 1–32 (2017)

    MATH  Google Scholar 

  10. Aletti, G., Naldi, G., Toscani, G.: First-order continuous models of opinion formation. SIAM J. Appl. Math. 67(3), 837–853 (2007)

    MathSciNet  Article  MATH  Google Scholar 

  11. Alla, A., Falcone, M., Kalise, D.: An efficient policy iteration algorithm for dynamic programming equations. SIAM J. Sci. Comput. 37(1), A181–A200 (2015)

    MathSciNet  Article  MATH  Google Scholar 

  12. Ballerini, M., Cabibbo, N., Candelier, R., Cavagna, A., Cisbani, E., Giardina, L., Lecomte, L., Orlandi, A., Parisi, G., Procaccini, A., Viale, M., Zdravkovic, V.: Interaction ruling animal collective behavior depends on topological rather than metric distance: evidence from a field study. PNAS 105(4), 1232–1237 (2008)

    Article  Google Scholar 

  13. Bellman, R., Kalaba, R.E.: Dynamic Programming and Modern Control Theory, vol. 81. Citeseer (1965)

  14. Benamou, J.-D., Brenier, Y.: A computational fluid mechanics solution to the Monge-Kantorovich mass transfer problem. Numer. Math. 84(3), 375–393 (2000)

    MathSciNet  Article  MATH  Google Scholar 

  15. Bensoussan, A., Frehse, J., Yam, P.: Mean Field Games and Mean Field Type Control Theory. Springer, New York (2013)

    Book  MATH  Google Scholar 

  16. Bobylev, A., Nanbu, K.: Theory of collision algorithms for gases and plasmas based on the Boltzmann equation and the Landau-Fokker-Planck equation. Phys. Rev. E 61(4), 4576 (2000)

    Article  Google Scholar 

  17. Bongini, M., Fornasier, M.: Sparse stabilization of dynamical systems driven by attraction and avoidance forces. Netw. Heterog. Media 9(1), 1–31 (2014)

    MathSciNet  Article  MATH  Google Scholar 

  18. Bongini, M., Fornasier, M.: Sparse control of multiagent systems. In: Bellomo, N., Degond, P., Tadmor, E. (eds.) Active Particles, vol. 1: Theory, Methods, and Applications. Birkhauser-Springer, Boston (2016)

  19. Buet, C., Dellacherie, S.: On the Chang and Cooper scheme applied to a linear Fokker-Planck equation. Commun. Math. Sci. 8(4), 1079–1090 (2010)

    MathSciNet  Article  MATH  Google Scholar 

  20. Burger, M., Francesco, M.D., Markowich, P.A., Wolfram, M.-T.: Mean field games with nonlinear mobilities in pedestrian dynamics. Discret. Contin. Dyn. Syst. Ser. B 19(5), 1311–1333 (2014)

    MathSciNet  Article  MATH  Google Scholar 

  21. Camazine, S., Deneubourg, J., Franks, N., Sneyd, J., Theraulaz, G., Bonabeau, E.: Self-organization in Biological Systems. Princeton University Press, Princeton (2003)

    MATH  Google Scholar 

  22. Camilli, F., Jakobsen, E.R.: A finite element like scheme for integro-partial differential Hamilton-Jacobi-Bellmann equations. SIAM J. Numer. Anal. 47(4), 2407–2431 (2009)

    MathSciNet  Article  MATH  Google Scholar 

  23. Cañizo, J.A., Carrillo, J.A., Rosado, J.: A well-posedness theory in measures for some kinetic models of collective motion. Math. Models Methods Appl. Sci. 21(3), 515–539 (2011)

    MathSciNet  Article  MATH  Google Scholar 

  24. Caponigro, M., Fornasier, M., Piccoli, B., Trélat, E.: Sparse stabilization and optimal control of the Cucker-Smale model. Math. Control Relat. Fields 3, 447–466 (2013)

    MathSciNet  Article  MATH  Google Scholar 

  25. Cardaliaguet, P., Hadikhanloo, S.: Learning in mean field games: the fictitious play. ESAIM: COCV 23(2), 569–591 (2017)

  26. Carlini, E., Silva, F.J.: A fully discrete semi-Lagrangian scheme for a first order mean field game problem. SIAM J. Numer. Anal. 52(1), 45–67 (2014)

    MathSciNet  Article  MATH  Google Scholar 

  27. Carrillo, J.A., D’Orsogna, M.R., Panferov, V.: Double milling in self-propelled swarms from kinetic theory. Kinet. Relat. Models 2(2), 363–378 (2009)

    MathSciNet  Article  MATH  Google Scholar 

  28. Carrillo, J.A., Fornasier, M., Toscani, G., Vecil, F.: Particle, kinetic, and hydrodynamic models of swarming. In: Naldi, G., Pareschi, L., Toscani, G., Bellomo, N. (eds.) Mathematical Modeling of Collective Behavior in Socio-Economic and Life Sciences. Modeling and Simulation in Science, Engineering and Technology, pp. 297–336. Birkhäuser Boston, Boston (2010)

  29. Carrillo, J.A., Choi, Y.-P., Hauray, M.: The derivation of swarming models: mean-field limit and Wasserstein distances. In: Muntean, A., Toschi, F. (eds.) Collective Dynamics from Bacteria to Crowds. CISM International Centre for Mechanical Sciences, pp. 1–46. Springer, New York (2014)

  30. Carrillo, J.A., Choi, Y.-P., Pérez, S.: A review on attractive–repulsive hydrodynamics for consensus in collective behavior. In: Bellomo, N., Degond, P., Tadmor, E. (eds.) Active Particles. Modeling and Simulation in Science, Engineering and Technology, vol. 1, pp. 259–298. Birkhäuser, Cham (2017)

  31. Chang, J., Cooper, G.: A practical difference scheme for Fokker-Planck equations. J. Comput. Phys. 6(1), 1–16 (1970)

    Article  MATH  Google Scholar 

  32. Choi, Y.-P.: Global classical solutions of the Vlasov-Fokker-Planck equation with local alignment forces. Nonlinearity 29(7), 1887–1916 (2016)

    MathSciNet  Article  MATH  Google Scholar 

  33. Choi, Y.-P., Ha, S.-Y., Li, Z.: Emergent dynamics of the Cucker–Smale flocking model and its variants. In: Bellomo, N., Degond, P., Tadmor, E. (eds.) Active Particles. Modeling and Simulation in Science, Engineering and Technology, vol 1, pp. 299–331. Birkhäuser, Cham (2017)

  34. Chuang, Y., D’Orsogna, M., Marthaler, D., Bertozzi, A., Chayes, L.: State transition and the continuum limit for the 2D interacting, self-propelled particle system. Physica D 232, 33–47 (2007)

    MathSciNet  Article  MATH  Google Scholar 

  35. Chuang, Y., Huang, Y., D’Orsogna, M., Bertozzi, A.: Multi-vehicle flocking: scalability of cooperative control algorithms using pairwise potentials. In: IEEE International Conference on Robotics and Automation, pp. 2292–2299 (2007)

  36. Cordier, S., Pareschi, L., Toscani, G.: On a kinetic model for a simple market economy. J. Stat. Phys. 120(1–2), 253–277 (2005)

    MathSciNet  Article  MATH  Google Scholar 

  37. Couzin, I., Franks, N.: Self-organized lane formation and optimized traffic flow in army ants. Proc. R. Soc. Lond. B 270, 139–146 (2002)

    Article  Google Scholar 

  38. Couzin, I., Krause, J., Franks, N., Levin, S.: Effective leadership and decision making in animal groups on the move. Nature 433, 513–516 (2005)

    Article  Google Scholar 

  39. Cristiani, E., Piccoli, B., Tosin, A.: Modeling self-organization in pedestrians and animal groups from macroscopic and microscopic viewpoints. In: Naldi, G., Pareschi, L., Toscani, G., Bellomo, N. (eds.) Mathematical Modeling of Collective Behavior in Socio-Economic and Life Sciences, Modeling and Simulation in Science. Engineering and Technology. Birkhäuser Boston, Boston (2010)

    Google Scholar 

  40. Cristiani, E., Piccoli, B., Tosin, A.: Multiscale modeling of granular flows with application to crowd dynamics. Multiscale Model. Simul. 9(1), 155–182 (2011)

    MathSciNet  Article  MATH  Google Scholar 

  41. Cucker, F., Dong, J.-G.: A general collision-avoiding flocking framework. IEEE Trans. Autom. Control 56(5), 1124–1129 (2011)

    MathSciNet  Article  Google Scholar 

  42. Cucker, F., Mordecki, E.: Flocking in noisy environments. J. Math. Pures Appl. (9) 89(3), 278–296 (2008)

  43. Cucker, F., Smale, S.: Emergent behavior in flocks. IEEE Trans. Autom. Control 52(5), 852–862 (2007)

    MathSciNet  Article  Google Scholar 

  44. Cucker, F., Smale, S., Zhou, D.: Modeling language evolution. Found. Comput. Math. 4(5), 315–343 (2004)

    MathSciNet  MATH  Google Scholar 

  45. Degond, P., Herty, M., Liu, J.-G.: Meanfield games and model predictive control. Comm. Math. Sci. 15(5), 1403–1422 (2017)

  46. Duan, R., Fornasier, M., Toscani, G.: A kinetic flocking model with diffusion. Commun. Math. Phys. 300, 95–145 (2010)

    MathSciNet  Article  MATH  Google Scholar 

  47. Falcone, M., Ferretti, R.: Semi-Lagrangian Approximation Schemes for Linear and Hamilton-Jacobi Equations. Society for Industrial and Applied Mathematics, Philadelphia (2013)

    Book  MATH  Google Scholar 

  48. Festa, A.: Reconstruction of independent sub-domains for a class of Hamilton–Jacobi equations and application to parallel computing. ESAIM: M2AN 50(4), 1223–1240 (2016)

  49. Festa, A., Wolfram, M.-T.: Collision avoidance in pedestrian dynamics. In: 2015 54th IEEE Conference on Decision and Control (CDC), pp. 3187–3192 (2015)

  50. Filippov, A.F.: Differential Equations with Discontinuous Righthand Sides. Mathematics and Its Applications. Kluwer, Dordrecht (1988)

    Book  Google Scholar 

  51. Fornasier, M., Solombrino, F.: Mean-field optimal control. ESAIM Control Optim. Calc. Var. 20(4), 1123–1152 (2014)

    MathSciNet  Article  MATH  Google Scholar 

  52. Grégoire, G., Chaté, H.: Onset of collective and cohesive motion. Phys. Rev. Lett. 92(2), 025702 (2004)

    Article  Google Scholar 

  53. Hegselmann, R., Krause, U.: Opinion dynamics and bounded confidence: models, analysis and simulation. J. Artif. Soc. Soc. Simul. 5(3), 1–33 (2002)

    Google Scholar 

  54. Hofbauer, J., Sigmund, K.: Evolutionary Games and Population Dynamics. Cambridge University Press, Cambridge (1998)

    Book  MATH  Google Scholar 

  55. Huang, M., Caines, P., Malhamé, R.: Individual and mass behaviour in large population stochastic wireless power control problems: centralized and Nash equilibrium solutions. In: Proceedings of the 42nd IEEE Conference on Decision and Control Maui, Hawaii, USA, December 2003, pp. 98–103 (2003)

  56. Jadbabaie, A., Lin, J., Morse, A.S.: Correction to: “Coordination of groups of mobile autonomous agents using nearest neighbor rules”. IEEE Trans. Autom. Control 48(9), 1675 (2003)

    Article  MATH  Google Scholar 

  57. Kalise, D., Kröner, A., Kunisch, K.: Local minimization algorithms for dynamic programming equations. SIAM J. Sci. Comput. 38(3), A1587–A1615 (2016)

    MathSciNet  Article  MATH  Google Scholar 

  58. Ke, J., Minett, J., Au, C.-P., Wang, W.-Y.: Self-organization and selection in the emergence of vocabulary. Complexity 7, 41–54 (2002)

    MathSciNet  Article  Google Scholar 

  59. Keller, E.F., Segel, L.A.: Initiation of slime mold aggregation viewed as an instability. J. Theor. Biol. 26(3), 399–415 (1970)

    Article  MATH  Google Scholar 

  60. Koch, A., White, D.: The social lifestyle of myxobacteria. Bioessays 20, 1030–1038 (1998)

    Article  Google Scholar 

  61. Lacker, D.: Limit theory for controlled McKean-Vlasov dynamics. SIAM J. Control Optim. 55(3), 1641–1672 (2017)

    MathSciNet  Article  MATH  Google Scholar 

  62. Lasry, J.-M., Lions, P.-L.: Mean field games. Jpn. J. Math. (3) 2(1), 229–260 (2007)

  63. Leonard, N., Fiorelli, E.: Virtual leaders, artificial potentials and coordinated control of groups. In: Proceeding of 40th IEEE Conference on Decision and Control, pp. 2968–2973 (2001)

  64. Mayne, D.Q., Rawlings, J.B., Rao, C.V., Scokaert, P.O.M.: Constrained model predictive control: stability and optimality. Autom. J. IFAC 36(6), 789–814 (2000)

    MathSciNet  Article  MATH  Google Scholar 

  65. Motsch, S., Tadmor, E.: Heterophilious dynamics enhances consensus. SIAM Rev. 56(4), 577–621 (2014)

    MathSciNet  Article  MATH  Google Scholar 

  66. Niwa, H.: Self-organizing dynamic model of fish schooling. J. Theor. Biol. 171, 123–136 (1994)

    Article  Google Scholar 

  67. Nuorian, M., Caines, P., Malhamé, R.: Synthesis of Cucker–Smale type flocking via mean field stochastic control theory: Nash equilibria. In: Proceedings of the 48th Allerton Conference on Communication, Control, and Computing, Monticello, Illinois, pp. 814–819, September 2010, pp. 814–815 (2010)

  68. Nuorian, M., Caines, P., Malhamé, R.: Mean field analysis of controlled Cucker–Smale type flocking: linear analysis and perturbation equations. In: Proceedings of 18th IFAC World Congress Milano (Italy), 28 August–2 September 2011, pp. 4471–4476 (2011)

  69. Pareschi, L., Toscani, G.: Interacting Multi-agent Systems. Kinetic Equations & Monte Carlo Methods. Oxford University Press, Oxford (2013)

    Google Scholar 

  70. Parrish, J., Edelstein-Keshet, L.: Complexity, pattern, and evolutionary trade-offs in animal aggregation. Science 294, 99–101 (1999)

    Article  Google Scholar 

  71. Parrish, J., Viscido, S., Gruenbaum, D.: Self-organized fish schools: an examination of emergent properties. Biol. Bull. 202, 296–305 (2002)

    Article  Google Scholar 

  72. Perea, L., Gómez, G., Elosegui, P.: Extension of the Cucker-Smale control law to space flight formations. AIAA J. Guid. Control Dyn. 32, 527–537 (2009)

    Article  Google Scholar 

  73. Perthame, B.: Transport Equations in Biology. Birkhäuser, Basel (2007)

    MATH  Google Scholar 

  74. Romey, W.: Individual differences make a difference in the trajectories of simulated schools of fish. Ecol. Model. 92, 65–77 (1996)

    Article  Google Scholar 

  75. Roy, S., Annunziato, M., Borzì, A.: A Fokker-Planck feedback control-constrained approach for modeling crowd motion. J. Comput. Theor. Transp. 45(6), 442–458 (2016)

    MathSciNet  Article  Google Scholar 

  76. Short, M.B., D’Orsogna, M.R., Pasour, V.B., Tita, G.E., Brantingham, P.J., Bertozzi, A.L., Chayes, L.B.: A statistical model of criminal behavior. Math. Models Methods Appl. Sci. 18(suppl.), 1249–1267 (2008)

  77. Sugawara, K., Sano, M.: Cooperative acceleration of task performance: foraging behavior of interacting multi-robots system. Physica D 100, 343–354 (1997)

    Article  MATH  Google Scholar 

  78. Sznajd-Weron, K., Sznajd, J.: Opinion evolution in closed community. Int. J. Mod. Phys. C 11(06), 1157–1165 (2000)

    Article  MATH  Google Scholar 

  79. Toner, J., Tu, Y.: Long-range order in a two-dimensional dynamical xy model: how birds fly together. Phys. Rev. Lett. 75, 4326–4329 (1995)

    Article  Google Scholar 

  80. Toscani, G.: Kinetic models of opinion formation. Commun. Math. Sci. 4(3), 481–496 (2006)

    MathSciNet  Article  MATH  Google Scholar 

  81. Vicsek, T., Zafeiris, A.: Collective motion. Phys. Rep. 517, 71–140 (2012)

    Article  Google Scholar 

  82. Vicsek, T., Czirok, A., Ben-Jacob, E., Cohen, I., Shochet, O.: Novel type of phase transition in a system of self-driven particles. Phys. Rev. Lett. 75, 1226–1229 (1995)

    MathSciNet  Article  Google Scholar 

  83. Villani, C.: On a new class of weak solutions to the spatially homogeneous Boltzmann and Landau equations. Arch. Ration. Mech. Anal. 143(3), 273–307 (1998)

    MathSciNet  Article  MATH  Google Scholar 

  84. Villani, C.: Optimal Transport, vol. 338 of Grundlehren der Mathematischen Wissenschaften [Fundamental Principles of Mathematical Sciences]. Springer, Berlin (2009)

  85. Yates, C., Erban, R., Escudero, C., Couzin, L., Buhl, J., Kevrekidis, L., Maini, P., Sumpter, D.: Inherent noise can facilitate coherence in collective swarm motion. Proc. Natl Acad. Sci. U.S.A. 106, 5464–5469 (2009)

    Article  Google Scholar 

  86. Zeidler, E.: Applied Functional Analysis. Applied Mathematical Sciences. Springer, New York (1995)

    MATH  Google Scholar 

Download references

Acknowledgements

G.A., Y.P.C., and M.F. acknowledge the support of the ERC-Starting Grant HDSPCONTR “High-Dimensional Sparse Optimal Control”. Y.P.C. is also supported by the Alexander Humboldt Foundation through the Humboldt Research Fellowship for Postdoctoral Researchers. D.K. acknowledges the support of the ERC-Advanced Grant OCLOC “From Open-Loop to Closed-Loop Optimal Control of PDEs”.

Author information

Authors and Affiliations

  1. Department of Computer Science, University of Verona, Str. Le Grazie,15, 37134, Verona, Italy

    Giacomo Albi

  2. Department of Mathematics, Inha University, Incheon, 402-751, Republic of Korea

    Young-Pil Choi

  3. Department of Mathematics, TU München, Boltzmannstr. 3, 85748, Garching bei München, Germany

    Massimo Fornasier

  4. RICAM, Austrian Academy of Sciences, Altenbergerstr. 69, 4040, Linz, Austria

    Dante Kalise

Authors
  1. Giacomo Albi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Young-Pil Choi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Massimo Fornasier
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Dante Kalise
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Massimo Fornasier.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Albi, G., Choi, YP., Fornasier, M. et al. Mean Field Control Hierarchy. Appl Math Optim 76, 93–135 (2017). https://doi.org/10.1007/s00245-017-9429-x

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00245-017-9429-x

Keywords

  • PDE constrained optimization multi-agent systems
  • Boltzmann equations
  • Kinetic equations