A Minimum Principle for Decentralized Stochastic Control Problems

  • Pravin Varaiya
  • Jean Walrand
Part of the Mathematical Concepts and Methods in Science and Engineering book series (MCSENG)


The notion of the “state” of a dynamical system has turned out, in retrospect, to be the single most important concept in system theory, and the representation of systems in state space form is now almost universally accepted as the correct framework for posing questions of control and for formulating and analyzing proposed solutions. The power of the state concept derives from two properties. First, the current value of the state is a sufficient (and “minimal”) statistic of all the past history of the system that is relevant for predicting future behavior. Second, the system’s optimal future behavior depends only on the current state, so it is a sufficient statistic for controlling future behavior in an optimal manner. As is well known, however, these properties can be exploited only when the state is directly observed—the so-called complete observation case—and so the utility of the state concept is reduced considerably when it can be observed only indirectly—the partial observation case.


Control Input SIAM Journal Minimum Principle State Concept Learning Function 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Striebel, C., Sufficient statistics in the optimum control of stochastic systems, Journal of Mathematical Analysis and Applications, Vol. 12, pp. 576–592, 1965.CrossRefGoogle Scholar
  2. 2.
    Witsenhausen, H. S., A standard form for sequential stochastic control, Mathematical Systems Theory, Vol. 7, pp. 5–11, 1973.CrossRefGoogle Scholar
  3. 3.
    Feldbaum, A. A., Optimal Control Systems, Academic Press, New York, 1965.Google Scholar
  4. 4.
    Bar-Shalom, Y., and Tse, E., Dual effect, certainty equivalence, and separation in stochastic control, IEEE Transactions on Automatic Control, Vol. AC-19, pp, 494–500, 1974.CrossRefGoogle Scholar
  5. 5.
    Kendrick, D., Adaptive control of macroeconomic models with measurement error, in Optimal Control of Econometric Models. Edited by S. Holly, B. Rustem and M. B. Zarrop, Macmillan, New York (forthcoming).Google Scholar
  6. 6.
    Wittenmark, B., Stochastic adaptive control methods: A survey, International Journal of Control, Vol. 21, pp. 705–730, 1975.CrossRefGoogle Scholar
  7. 7.
    Mandl, P., Estimation and control in Markov chains, Advances in Applied Probability, Vol. 6, pp. 30–50, 1974.CrossRefGoogle Scholar
  8. 8.
    Elliott, R. J., The optimal control of a stochastic system, SIAM Journal on Control and Optimization, Vol. 15, pp. 756–778, 1977.CrossRefGoogle Scholar
  9. 9.
    Thompson, G. L., Signaling strategies in n-person games, in Contributions to the Theory of Games II. Edited by H. W. Kuhn and A. W. Tucker, Princeton University Press, Princeton, New Jersey, 1953.Google Scholar
  10. 10.
    Sandell, N. R., Jr., and Athans, M., A finite-state, finite-memory minimum principle, Journal of Optimization Theory and Applications, Vol. 25, pp. 289–306, 1978.CrossRefGoogle Scholar
  11. 11.
    Ho, Y.-C., Kastner, M. P., and Wong, E., Teams, signaling and information theory, IEEE Transactions on Automatic Control, Vol. AC-23, pp. 305–312, 1978.Google Scholar
  12. 12.
    Rishel, R., Necessary and sufficient dynamic programming conditions for continuous-time stochastic optimal control, SIAM Journal on Control and Optimization, Vol. 8, pp. 559–571, 1970.CrossRefGoogle Scholar
  13. 13.
    Davis, M. H. A., and Varaiya, P., Dynamic programming conditions for partially observable stochastic systems, SIAM Journal on Control and Optimization, Vol. 11, pp. 226–261, 1973.CrossRefGoogle Scholar
  14. 14.
    Striebel, C., Martingale conditions for the optimal control of continuous-time stochastic systems, unpublished, 1974.Google Scholar
  15. 15.
    Samuelson, P. A., Proof that properly discounted present values of assets vibrate randomly, Bell Journal of Economics and Management Science, Vol. 4, pp. 369–374, 1973.CrossRefGoogle Scholar
  16. 16.
    Samuelson, P. A., Lifetime portfolio selection by dynamic stochastic programming, The Review of Economics and Statistics, Vol. 51, pp. 239–246, 1969.CrossRefGoogle Scholar
  17. 17.
    Samuelson, P. A., and McKean, H. P., Jr., Rational theory of warrant pricing, Industrial Management Review, Vol. 6, pp. 791–810, 1964.Google Scholar
  18. 18.
    Benes, V. E., Existence of optimal strategies based on specified information, for a class of stochastic decision problems, SIAM Journal on Control, Vol. 8, pp. 179–188, 1970.CrossRefGoogle Scholar

Copyright information

© Plenum Press, New York 1980

Authors and Affiliations

  • Pravin Varaiya
    • 1
  • Jean Walrand
    • 1
  1. 1.Department of Electrical Engineering and Computer Sciences, and the Electronics Research LaboratoryUniversity of CaliforniaBerkeleyUSA

Personalised recommendations