Skip to main content
SpringerLink
Log in

Infinite horizon indefinite stochastic linear quadratic control for discrete-time systems

  • Published:
Control Theory and Technology Aims and scope Submit manuscript

Abstract

This paper discusses discrete-time stochastic linear quadratic (LQ) problem in the infinite horizon with state and control dependent noise, where the weighting matrices in the cost function are assumed to be indefinite. The problem gives rise to a generalized algebraic Riccati equation (GARE) that involves equality and inequality constraints. The well-posedness of the indefinite LQ problem is shown to be equivalent to the feasibility of a linear matrix inequality (LMI). Moreover, the existence of a stabilizing solution to the GARE is equivalent to the attainability of the LQ problem. All the optimal controls are obtained in terms of the solution to the GARE. Finally, we give an LMI -based approach to solve the GARE via a semidefinite programming.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. R. E. Kalman. Contributions to the theory of optimal control. Boletin de la Sociedad Matematica Mexicana, 1960, 5: 102–119.

    MathSciNet  Google Scholar 

  2. B. D. O. Anderson, J. B. Moore. Optimal Control–Linear Quadratic Methods. New York: Prentice-Hall, 1989.

    Google Scholar 

  3. F. L. Lewis. Optimal Control. New York: John Wiley & Sons, 1986.

    MATH  Google Scholar 

  4. L. Ntogramatzidis, A. Ferrante. On the solution of the Riccati differential equation arising from the LQ optimal control problem. Systems & Control Letters, 2010, 59(2): 114–121.

    Article  MathSciNet  MATH  Google Scholar 

  5. A. Ferrante, L. Ntogrammatzidis. The generalized continuous algebraic Riccati equation and impulse-free continuous-time LQ optimal control. Automatica, 2014, 50(4): 1176–1180.

    Article  MathSciNet  MATH  Google Scholar 

  6. W. M. Wonham. On a matrix Riccati equation of stochastic control. SIAM Journal on Control, 1968, 6: 312–326.

    Article  MathSciNet  MATH  Google Scholar 

  7. D. H. Jacobson, D. H. Martin, M. Pachter, et al. Extensions of Linear–Quadratic Control Theory. Lecture Notes in Control and Information Sciences. Berlin: Springer, 1980.

    Chapter  Google Scholar 

  8. A. Bensoussan. Stochastic Control of Partially Observed Systems. Cambridge, U.K.: Cambridge University Press, 1992.

    Book  Google Scholar 

  9. M. H. A. Davis. Linear Estimation and Stochastic Control. London: Chapman and Hall, 1977.

    MATH  Google Scholar 

  10. E. Yaz. Infinite horizon quadratic optimal control of a class of nonlinear stochastic systems. IEEE Transactions on Automatic Control, 1989, 34(11): 1176–1180.

    Article  MathSciNet  MATH  Google Scholar 

  11. W. Zhang, B. Chen. On stabilizability and exact observability of stochastic systems with their applications. Automatica, 2004, 40(1): 87–94.

    Article  MathSciNet  MATH  Google Scholar 

  12. W. Zhang, H. Zhang, B. Chen. Generalized Lyapunov equation approach to state-dependent stochastic stabilization/detectability criterion. IEEE Transactions on Automatic Control, 2008, 53(7): 1630–1642.

    Article  Google Scholar 

  13. Y. Hu, H. Jin, X. Zhou. Time-inconsistent stochastic linearquadratic control. SIAM Journal on Control and Optimization, 2012, 50(3): 1548–1572.

    Article  MathSciNet  MATH  Google Scholar 

  14. J. M. Yong. Linear-quadratic optimal control problems for meanfield stochastic differential equations. SIAM Journal on Control and Optimization, 2013, 51(4): 2809–2838.

    Article  MathSciNet  MATH  Google Scholar 

  15. S. Chen, X. Li, X. Zhou. Stochastic linear quadratic regulators with indefinite control weight costs. SIAM Journal on Control and Optimization, 1998, 36(5): 1685–1702.

    Article  MathSciNet  MATH  Google Scholar 

  16. Z. Qian, X. Zhou. Existence of solutions to a class of indefinite stochastic Riccati equations. SIAM Journal on Control and Optimization, 2013, 51(1): 221–229.

    Article  MathSciNet  MATH  Google Scholar 

  17. Q. Meng. Linear quadratic optimal stochastic control problem driven by a Brownian motion and a Poisson random martingale measure with random cofficients. Stochastic Analysis and Applications, 2014, 32(1): 88–109.

    Article  MathSciNet  MATH  Google Scholar 

  18. R. S. Burachik, C. Y. Kaya, S. N. Majeed. A duality approach for solving control-constrained linear-quadratic optimal control problems. SIAM Journal on Control and Optimization, 2014, 52(3): 1423–1456.

    Article  MathSciNet  MATH  Google Scholar 

  19. Y. Huang, W. Zhang, H. Zhang. Infinite horizon linear quadratic optimal control for discrete-time stochastic systems. Asian Journal of Control, 2008, 10(5): 608–615.

    Article  MathSciNet  Google Scholar 

  20. R. T. Ku, M. Athans. Further results on the uncertainty threshold principle. IEEE Transactions on Automatic Control, 1977, 22(5): 866–868.

    Article  MathSciNet  MATH  Google Scholar 

  21. A. Beghi, D. D. Alessandro. Discrete-time optimal control with control-dependent noise and generalized Riccati difference equations. Automatica, 1998, 34(8): 1031–1034.

    Article  MathSciNet  MATH  Google Scholar 

  22. M. Ait Rami, X. Chen, X. Zhou. Discrete-time indefinite LQ control with state and control dependent noises. Journal of Global Optimization, 2002, 23(3/4): 245–265.

    Article  MathSciNet  MATH  Google Scholar 

  23. G. Li, W. Zhang. Study on indefinite stochastic linear quadratic optimal control with inequality constraint. Journal of Applied Mathematics, 2013: DOI 10.1155/2013/805829.

    Google Scholar 

  24. W. Zhang, G. Li. Discrete-time indefinite stochastic linear quadratic optimal control with second moment constraints. Mathmatical Problems in Engineering, 2014: DOI 10.1155/2014/278142.

    Google Scholar 

  25. X. Liu, Y. Li, W. Zhang. Stochastic linear quadratic optimal control with constraint for discrete-time systems. Applied Mathematics and Computation, 2014, 228: 264–270.

    Article  MathSciNet  Google Scholar 

  26. R. Penrose. A generalized inverse for matrices. Mathematical Proceedings of the Cambridge Philosophical Society, 1955, 51(3): 406–413.

    Article  MathSciNet  MATH  Google Scholar 

  27. A. Albert. Conditions for positive and nonnegative definiteness in terms of pseudo-inverse. SIAM Journal on Applied Mathematics, 1969, 17(2): 434–440.

    Article  MathSciNet  MATH  Google Scholar 

  28. L. Vandenberghe, S. Boyd. Semidefinite programming. SIAM Review, 1996, 38(1): 49–95.

    Article  MathSciNet  MATH  Google Scholar 

  29. M. Ait Rami, X. Zhou, J. B. Moore. Well-posedness and attainability of indefinite stochastic linear quadratic control in infinite time horizon. Systems & Control Letters, 2000, 41(2): 123–133.

    Article  MathSciNet  Google Scholar 

  30. B. R. Copeland, M. G. Safonov. Zero cancelling compensation for singular control problems and their application to the innerouter factorization problem. International Journal of Robust and Nonlinear Control, 1992, 2(2): 139–164.

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. College of Electrical Engineering and Automation, Shandong University of Science and Technology, Qingdao Shandong, 266590, China

    Weihai Zhang & Yan Li

  2. College of Mathematics and Systems Science, Shandong University of Science and Technology, Qingdao Shandong, 266590, China

    Xikui Liu

Authors
  1. Weihai Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yan Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xikui Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Weihai Zhang.

Additional information

This work was supported by the National Natural Science Foundation of China (Nos. 61174078, 61170054, 61402265) and the Research Fund for the Taishan Scholar Project of Shandong Province of China.

Weihai ZHANG received his Ph.D. degree from Zhejiang University in 1998, Hangzhou, China. He is currently a professor of Shandong University of Science and Technology. His research interests include stochastic optimal control, robust H∞ control, stochastic stability and stabilization.

Yan LI received the M.Sc. and Ph.D. degrees from Shandong University of Science and Technology, China, in 2006 and 2015, respectively. She is a lecturer of Shandong University of Science and Technology. Her research interests include linear and nonlinear stochastic control.

Xikui LIU received the M.Sc. degree from Shandong University of Science and Technology, and the Ph.D. degree from Huazhong University of Science and Technology, China, in 2000 and 2004, respectively. He is a associate professor of Shandong University of Science and Technology. His interests include graph theory and DNA computing.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, W., Li, Y. & Liu, X. Infinite horizon indefinite stochastic linear quadratic control for discrete-time systems. Control Theory Technol. 13, 230–237 (2015). https://doi.org/10.1007/s11768-015-4147-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11768-015-4147-x

Keywords

Search

Navigation