Stochastic Differential Games and Intricacy of Information Structures
This chapter discusses, in both continuous time and discrete time, the issue of certainty equivalence in two-player zero-sum stochastic differential/dynamic games when the players have access to state information through a common noisy measurement channel. For the discrete-time case, the channel is also allowed to fail sporadically according to an independent Bernoulli process, leading to intermittent loss of measurements, where the players are allowed to observe past realizations of this process. A complete analysis of a parametrized two-stage stochastic dynamic game is conducted in terms of existence, uniqueness and characterization of saddle-point equilibria (SPE), which is shown to admit SPE of both certainty-equivalent (CE) and non-CE types, in different regions of the parameter space; for the latter, the SPE involves mixed strategies by the maximizer. The insight provided by the analysis of this game is used to obtain through an indirect approach SPE for three classes of differential/dynamic games: (i) linear-quadratic-Gaussian (LQG) zero-sum differential games with common noisy measurements, (ii) discrete-time LQG zero-sum dynamic games with common noisy measurements, and (iii) discrete-time LQG zero-sum dynamic games with intermittently missing perfect state measurements. In all cases CE is a generalized notion, requiring two separate filters for the players, even though they have a common communication channel. Discussions on extensions to other classes of stochastic games, including nonzero-sum stochastic games, and on the challenges that lie ahead conclude the chapter.
KeywordsDifferential Game Dynamic Game Stochastic Game Policy Space Riccati Differential Equation
This work was supported in part by the AFOSR MURI Grant FA9550-10-1-0573, and in part by NSF under grant number CCF 11-11342.
- Başar, T. (2008). Variations on the theme of the Witsenhausen counterexample. In Proc. 47th IEEE conf. decision and control (pp. 1614–1619). Google Scholar
- Başar, T., & Bernhard, P. (1995). Systems & control: foundations and applications series. H ∞ optimal control and related minimax design problems: a dynamic game approach. Boston: Birkhäuser. Google Scholar
- Başar, T., & Mintz, M. (1972). On the existence of linear saddle-point strategies for a two-person zero-sum stochastic game. In Proc. 1972 IEEE conf. decision and control (pp. 188–192). Google Scholar
- Başar, T., & Mintz, M. (1973). A multistage pursuit-evasion game that admits a Gaussian random process as a maximin control policy. Stochastics, 1(1–4), 25–69. Google Scholar
- Başar, T., & Olsder, G. J. (1999). SIAM series in classics in applied mathematics. Dynamic non-cooperative game theory. Philadelphia: Society for Industrial and Applied Mathematics. Google Scholar
- Fleming, W. H., & Soner, H. M. (1993). Controlled Markov processes and viscosity solutions. Berlin: Springer. Google Scholar
- Hespanha, J., & Prandini, M. (2001). Nash equilibria in partial-information games on Markov chains. In Proc. 40th IEEE conf. decision and control (pp. 2102–2107). Google Scholar
- Nayyar, A., & Başar, T. (2012). Dynamic stochastic games with asymmetric information. In Proc. 51st IEEE conf. decision and control, Maui, Hawaii (pp. 7145–7150). Google Scholar
- Pan, Z., & Başar, T. (1995). H ∞ control of Markovian jump systems and solutions to associated piecewise-deterministic differential games. In G. J. Olsder (Ed.), Annals of dynamic games (Vol. 2, pp. 61–94). Boston: Birkhäuser. Google Scholar
- Shi, L., Epstein, M., & Murray, R. M. (2010). Kalman filtering over a packet-dropping network: a probabilistic perspective. IEEE Transactions on Automatic Control, 55(1), 594–604. Google Scholar