Abstract
We are concerned with stochastic control systems composed of a large number of N interacting objects sharing a common environment. The evolution of each object is determined by a stochastic difference equation where the random disturbance density \(\rho \) is unknown for the controller. We present the Markov control model (N-model) associated to the proportions of objects in each state, which is analyzed according to the mean field theory. Thus, combining convergence results as \(N\rightarrow \infty \) (the mean field limit) with a suitable statistical estimation method for \(\rho \), we construct the so-named eventually asymptotically optimal policies for the N-model under a discounted optimality criterion. A consumption-investment problem is analyzed to illustrate our results.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Achdou, I., Capuzzo-Dolcetta, I.: Mean field games: numerical methods. SIAM J. Numer. Anal. 48, 1136–1162 (2010)
Aoki, M.: New macroeconomic modeling approaches. Hierarchical dynamics and mean field approximation. J. Econ. Dyn. Control 18, 865–877 (1994)
Bensoussan, A., Frehse, J., Yam, P.: Mean Field Games and Mean Field Control Theory. Springer, New York (2010)
Bertsekas, D.P.: Dynamic Programming: Deterministic and Stochastic Models. Prentice-Hall, Englewood Cliffs (1987)
Budhiraja, A., Dupuis, P., Fischer, M.: Large deviation properties of weakly interacting processes via weak convergence methods. Ann. Probab. 40, 74–102 (2012)
Carmona, R., Fouque, J.P., Vestal, D.: Interacting particle systems for the computation of rare credit portfolio losses. Financ. Stoch. 13, 613–633 (2009)
Gast, N., Gaujal, B.: A mean field approach for optimization in discrete time. Discret. Event Dyn. Syst. 21, 63–101 (2011)
Gast, N., Gaujal, B., Le Boudec, J.Y.: Mean field for Markov decision processes: from discrete to continuous optimization. IEEE Trans. Autom. Control 57, 2266–2280 (2012)
Gomes, D.A., Mohr, J., Souza, R.R.: Discrete time, finite state space mean field games. J. Math. Pures Appl. 93, 308–328 (2010)
Gordienko, E.I., Minjárez-Sosa, J.A.: Adaptive control for discrete-time Markov processes with unbounded costs: discounted criterion. Kybernetika 34, 217–234 (1998)
Hernández-Lerma, O.: Adaptive Markov Control Processes. Springer, New York (1989)
Hernández-Lerma, O., Lasserre, J.B.: Discrete-Time Markov Control Processes: Basic Optimality Criteria. Springer, New York (1996)
Hilgert, N., Minjárez-Sosa, J.A.: Adaptive control of stochastic systems with unknown disturbance distribution: discounted criterion. Math. Method Oper. Res. 63, 443–460 (2006)
Huang, M.: Large-population LQG games involving a major player: the Nash certainty equivalence principle. SIAM J. Control Optim. 48, 3318–3353 (2010)
Kolokoltsov, V.N., Troeva, M., Yang, W.: On the rate of convergence for the mean-field approximation of controlled diffusions with large number of players. Dyn. Games Appl. 4, 208–230 (2014)
Lachapelle, A., Salomon, J., Turinici, G.: Computation of mean field equilibria in economics. Math. Models Methods Appl. Sci. 20, 567–588 (2010)
Le Boudec, J.Y., McDonald, D., Mundinger, J.: A generic mean field convergence result for systems of interacting objects. In: 4th International Conference Quantitative Evaluation of Systems (2007). ISBN:0-7695-2883-X/07
Lasry, J.M., Lions, P.L.: Mean field games. Jap. J. Math. 2, 229–260 (2007)
Minjárez-Sosa, J.A., Vega-Amaya, O.: Asymptotically optimal strategies for adaptive zero-sum discounted Markov games. SIAM J. Control Optim. 48, 1405–1421 (2009)
Peyrard, N., Sabbadin, R.: Mean field approximation of the policy iteration algorithm for graph-based Markov decision processes. In: Biewka, G., Coradeschi, S., Perini, A., Traverso, P. (eds.) Frontiers in Artificial Intelligence and Applications, pp. 595–599. IOS Press, Amsterdam (2006)
Puterman, M.L.: Markov Decision Processes. Discrete Stochastic Dynamic Programming. Wiley, New York (1994)
Acknowledgments
Work partially supported by Consejo Nacional de Ciencia y Tecnologia (CONACyT) under Grants CB2015/254306 and CB2015/238045.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Higuera-Chan, C.G., Jasso-Fuentes, H. & Minjárez-Sosa, J.A. Discrete-Time Control for Systems of Interacting Objects with Unknown Random Disturbance Distributions: A Mean Field Approach. Appl Math Optim 74, 197–227 (2016). https://doi.org/10.1007/s00245-015-9312-6
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00245-015-9312-6