Stochastic Games

Abstract

Sequential decision processes represent the mathematical description of the behavior of competing decision makers, who interact over time under uncertainty. An important class of sequential decision processes, with a rich mathematical theory, are the “stochastic games.” In this section, we examine some important formulations of stochastic games and in the last section we concentrate on a problem which stems from economic theory (equilibrium theory in particular) and develop and analyze two mathematical models which blend game theory with economic analysis. The two recurring themes of this chapter are the “dynamic programming methodology” and the “Markovian nature” of the dynamic system. The dynamic programming is a simple mathematical technique that has been used for many years by mathematicians, engineers and social scientists to analyze a large group of sequential decision problems. It was Bellman who first in the early 1950’s realized the power of the dynamic programming technique and developed it in a systematic way into a powerful tool of optimization theory. By means of dynamic programming, the analysis of an optimization problem is reduced to the study of a nonlinear functional equation, known as “dynamic programming equation” or “Bellman’s equation” which, under appropriate assumptions, is a necessary (and often sufficient) condition for an optimum. It is safe to say that the optimality theory of controlled Markov process essentially is the study of the theory of the dynamic programming functional equation. Moreover, in dynamic programming we deal with a family of optimization problems depending parametrically on the initial state of the system and we study this family as a whole. Thus, in a sense the dynamic programming formalism has a global character.

Laboring citizens are condemned to be mere passive instruments of production for the exclusive profit of the capitalist.

—Ponciano Arriaga, to the 1856–1857 Mexican Constitutional Convention

... to supply cheap labor to an international market that demands cheap products

—Eduardo Galeano, “Open Veins of Latin America”

The erratum of this chapter is available at http://dx.doi.org/10.1007/978-1-4615-4665-8_15