Stochastic Differential Games and Intricacy of Information Structures

Part of the Dynamic Modeling and Econometrics in Economics and Finance book series (DMEF, volume 16)

Abstract

This chapter discusses, in both continuous time and discrete time, the issue of certainty equivalence in two-player zero-sum stochastic differential/dynamic games when the players have access to state information through a common noisy measurement channel. For the discrete-time case, the channel is also allowed to fail sporadically according to an independent Bernoulli process, leading to intermittent loss of measurements, where the players are allowed to observe past realizations of this process. A complete analysis of a parametrized two-stage stochastic dynamic game is conducted in terms of existence, uniqueness and characterization of saddle-point equilibria (SPE), which is shown to admit SPE of both certainty-equivalent (CE) and non-CE types, in different regions of the parameter space; for the latter, the SPE involves mixed strategies by the maximizer. The insight provided by the analysis of this game is used to obtain through an indirect approach SPE for three classes of differential/dynamic games: (i) linear-quadratic-Gaussian (LQG) zero-sum differential games with common noisy measurements, (ii) discrete-time LQG zero-sum dynamic games with common noisy measurements, and (iii) discrete-time LQG zero-sum dynamic games with intermittently missing perfect state measurements. In all cases CE is a generalized notion, requiring two separate filters for the players, even though they have a common communication channel. Discussions on extensions to other classes of stochastic games, including nonzero-sum stochastic games, and on the challenges that lie ahead conclude the chapter.

Keywords

Covariance Assure Nash 

Notes

Acknowledgements

This work was supported in part by the AFOSR MURI Grant FA9550-10-1-0573, and in part by NSF under grant number CCF 11-11342.

References

  1. Altman, E., Kambley, V., & Silva, A. (2009). Stochastic games with one step delay sharing information pattern with application to power control. In Proc. internat. conf. game theory for networks (GameNets’09), Turkey, Istanbul (pp. 124–129). CrossRefGoogle Scholar
  2. Bansal, R., & Başar, T. (1987). Stochastic teams with nonclassical information revisited: when is an affine law optimal? IEEE Transactions on Automatic Control, 32(6), 554–559. CrossRefGoogle Scholar
  3. Başar, T. (1978a). Two-criteria LQG decision problems with one-step delay observation sharing pattern. Information and Control, 38(1), 21–50. CrossRefGoogle Scholar
  4. Başar, T. (1978b). Decentralized multicriteria optimization of linear stochastic systems. IEEE Transactions on Automatic Control, 23(2), 233–243. CrossRefGoogle Scholar
  5. Başar, T. (1981). On the saddle-point solution of a class of stochastic differential games. Journal of Optimization Theory and Applications, 33(4), 539–556. CrossRefGoogle Scholar
  6. Başar, T. (2008). Variations on the theme of the Witsenhausen counterexample. In Proc. 47th IEEE conf. decision and control (pp. 1614–1619). Google Scholar
  7. Başar, T., & Bernhard, P. (1995). Systems & control: foundations and applications series. H optimal control and related minimax design problems: a dynamic game approach. Boston: Birkhäuser. Google Scholar
  8. Başar, T., & Mintz, M. (1972). On the existence of linear saddle-point strategies for a two-person zero-sum stochastic game. In Proc. 1972 IEEE conf. decision and control (pp. 188–192). Google Scholar
  9. Başar, T., & Mintz, M. (1973). A multistage pursuit-evasion game that admits a Gaussian random process as a maximin control policy. Stochastics, 1(1–4), 25–69. Google Scholar
  10. Başar, T., & Olsder, G. J. (1999). SIAM series in classics in applied mathematics. Dynamic non-cooperative game theory. Philadelphia: Society for Industrial and Applied Mathematics. Google Scholar
  11. Behn, R., & Ho, Y.-C. (1968). On a class of linear stochastic differential games. IEEE Transactions on Automatic Control, 13(3), 227–240. CrossRefGoogle Scholar
  12. Fleming, W. H., & Soner, H. M. (1993). Controlled Markov processes and viscosity solutions. Berlin: Springer. Google Scholar
  13. Hespanha, J., & Prandini, M. (2001). Nash equilibria in partial-information games on Markov chains. In Proc. 40th IEEE conf. decision and control (pp. 2102–2107). Google Scholar
  14. Ho, Y.-C. (1974). On the minimax principle and zero-sum stochastic differential games. Journal of Optimization Theory and Applications, 13(3), 343–361. CrossRefGoogle Scholar
  15. Ho, Y.-C. (1980). Team decision theory and information structures. Proceedings of the IEEE, 68(6), 644–654. CrossRefGoogle Scholar
  16. Nayyar, A., & Başar, T. (2012). Dynamic stochastic games with asymmetric information. In Proc. 51st IEEE conf. decision and control, Maui, Hawaii (pp. 7145–7150). Google Scholar
  17. Nayyar, A., Mahajan, A., & Teneketzis, D. (2011). Optimal control strategies in delayed sharing information structures. IEEE Transactions on Automatic Control, 57(7), 1606–1620. CrossRefGoogle Scholar
  18. Pan, Z., & Başar, T. (1995). H control of Markovian jump systems and solutions to associated piecewise-deterministic differential games. In G. J. Olsder (Ed.), Annals of dynamic games (Vol. 2, pp. 61–94). Boston: Birkhäuser. Google Scholar
  19. Rhodes, I., & Luenberger, D. (1969). Differential games with imperfect state information. IEEE Transactions on Automatic Control, 14(1), 29–38. CrossRefGoogle Scholar
  20. Shi, L., Epstein, M., & Murray, R. M. (2010). Kalman filtering over a packet-dropping network: a probabilistic perspective. IEEE Transactions on Automatic Control, 55(1), 594–604. Google Scholar
  21. Willman, W. (1969). Formal solutions for a class of stochastic pursuit-evasion games. IEEE Transactions on Automatic Control, 14(5), 504–509. CrossRefGoogle Scholar
  22. Witsenhausen, H. (1968). A counterexample in stochastic optimum control. SIAM Journal on Control, 6, 131–147. CrossRefGoogle Scholar
  23. Witsenhausen, H. (1971a). Separation of estimation and control for discrete time systems. Proceedings of the IEEE, 59(11), 1557–1566. CrossRefGoogle Scholar
  24. Witsenhausen, H. (1971b). On information structures, feedback and causality. SIAM Journal on Control, 9(2), 149–160. CrossRefGoogle Scholar
  25. Yüksel, S., & Başar, T. (2013). Systems & control: foundations and applications series. Stochastic networked control systems: stabilization and optimization under information constraints. Boston: Birkhäuser. CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  1. 1.Department of Electrical and Computer Engineering, Coordinated Science LaboratoryUniversity of IllinoisUrbanaUSA

Personalised recommendations