Skip to main content
SpringerLink
Log in

Risk-sensitive and risk-neutral control for continuous-time hidden Markov models

  • Published:
Applied Mathematics and Optimization Aims and scope Submit manuscript

Abstract

In this paper the optimal control of a continuous-time hidden Markov model is discussed. The risk-sensitive problem involves a cost function which has an exponential form and a risk parameter, and is solved by defining an appropriate information state and dynamic programming. As the risk parameter tends to zero, the classical risk-neutral optimal control problem is recovered. The limits are proved using viscosity solution methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. A. Bensoussan and R. J. Elliott, A Finite-Dimensional Risk-Sensitive Control Problem, SIAM J. Control Optim., to appear.

  2. A. Bensoussan and J. H. van Schuppen, Optimal Control of Partially Observable Stochastic Systems with an Exponential-of-Integral Performance Index, SIAM J. Control Optim., 23 (1985), 599–613.

    Google Scholar 

  3. J. M. C. Clarke, The Design of Robust Approximations to the Stochastic Differential Equations of Nonlinear Filtering, in Communication Systems and Random Processes Theory (Darlington, 1977) (ed. J. K. Skwirzynski), Sijhoff and Nordhoff, Alphen aan den Rijn, 1978.

    Google Scholar 

  4. M. G. Crandall, H. Ishii, and P. L. Lions, A Users Guide to Viscosity Solutions, Bull. Amer. Math. Soc, 27 (1992), 1–67.

    Google Scholar 

  5. R. J. Elliott, Stochastic Calculus and Applications, Springer-Verlag, New York, 1982.

    Google Scholar 

  6. R. J. Elliott, L. Aggoun, and J. B. Moore, Hidden Markov Models: Estimation and Control, Springer-Verlag, New York, 1995.

    Google Scholar 

  7. W. H. Fleming and H. M. Soner, Controlled Markov Processes and Viscosity Solutions, Springer-Verlag, New York, 1993.

    Google Scholar 

  8. D. H. Jacobson, Optimal Stochastic Linear Systems with Exponential Performance Criteria and Their Relation to Deterministic Differential Games, IEEE Trans. Automat. Control, 18(2) (1973), 124–131.

    Google Scholar 

  9. M. R. James, Asymptotic Analysis of Nonlinear Stochastic Risk-Sensitive Control and Differential Games, Math. Control Signals Systems, 5(4) (1992), 401–417.

    Google Scholar 

  10. M. R. James and J. S. Baras, RobustH ∞ Output Feedback Control for Nonlinear Systems, IEEE Trans. Automat. Control, 40(6) (1995), 1007–1017.

    Google Scholar 

  11. M. R. James, J. S. Baras, and R. J. Elliott, Output Feedback Risk-Sensitive Control and Differential Games for Continuous-Time Nonlinear Systems, Proc. 32nd IEEE CDC, San Antonio, Dec. 1993, pp. 3357–3360.

  12. M. R. James, J. S. Baras, and R. J. Elliott, Risk-Sensitive Control and Dynamic Games for Partially Observed Discrete-Time Nonlinear Systems, IEEE Trans. Automat. Control, 39(4) (1994), 780–792.

    Google Scholar 

  13. P. L. Lions, Optimal Control of Diffusion Processes and Hamilton-Jacobi-Bellman Equations, Parts I and II, Comm. Partial Differential Equations 8 (1983), 1101–1134 and 1229–1276.

    Google Scholar 

  14. P. L. Lions, Viscosity Solutions of Fully Nonlinear Second-Order Equations and Optimal Stochastic Control in Infinite Dimensions, Part II, Optimal Control of Zakai's Equation, in Stochastic Partial Differential Equations and Applications, II, LNM, Vol. 1350, Springer-Verlag, Berlin, 1989.

    Google Scholar 

  15. P. Whittle, Risk-Sensitive Linear/Quadratic/Gaussian Control, Adv. in Appl. Probab., 13 (1981), 764–777.

    Google Scholar 

  16. E. Wong and B. Hajek, Stochastic Processes in Engineering Systems, Springer-Verlag, New York, 1985.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Engineering and Cooperative Research Centre for Robust and Adaptive Systems, Australian National University, 0200, Canberra, ACT, Australia

    M. R. James

  2. Department of Statistics and Applied Probability, University of Alberta, T6G 2G1, Edmonton, Alberta, Canada

    R. J. Elliott

Authors
  1. M. R. James
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. R. J. Elliott
    View author publications

    You can also search for this author in PubMed Google Scholar

Additional information

The first author wishes to acknowledge the funding of the activities of the Cooperative Research Centre for Robust and Adaptive Systems by the Australian Commonwealth Government under the Cooperative Research Centers Program. The support of NSERC Grant A7964 is acknowledged by the second author, as is the hospitality of the Department of Systems Engineering and the Cooperative Research Centre for Robust and Adaptive Systems, Australian National University, in July 1993.

Rights and permissions

Reprints and permissions

About this article

Cite this article

James, M.R., Elliott, R.J. Risk-sensitive and risk-neutral control for continuous-time hidden Markov models. Appl Math Optim 34, 37–50 (1996). https://doi.org/10.1007/BF01182472

Download citation

  • Accepted:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF01182472

Key words

AMS classification

Search

Navigation