Optimal bounded control of quasi-nonintegrable Hamiltonian systems using stochastic maximum principle

Abstract

A new procedure for designing optimal bounded control of quasi-nonintegrable Hamiltonian systems with actuator saturation is proposed based on the stochastic averaging method for quasi-nonintegrable Hamiltonian systems and the stochastic maximum principle. First, the stochastic averaging method for controlled quasi-nonintegrable Hamiltonian systems is introduced. The original control problem is converted into one for a partially averaged equation of system energy together with a partially averaged performance index. Then, the adjoint equation and the maximum condition of the partially averaged control problem are derived based on the stochastic maximum principle. The bounded optimal control forces are obtained from the maximum condition and solving the forward–backward stochastic differential equations (FBSDE). For infinite time-interval ergodic control, the adjoint variable is stationary process, and the FBSDE is reduced to an ordinary differential equation. Finally, the stationary probability density of the Hamiltonian and other response statistics of optimally controlled system are obtained by solving the Fokker–Plank–Kolmogorov equation associated with the fully averaged Itô equation of the controlled system. For comparison, the bang–bang control is also presented. An example of two degree-of-freedom quasi-nonintegrable Hamiltonian system is worked out to illustrate the proposed procedure and its effectiveness. Numerical results show that the proposed control strategy has higher control efficiency and less discontinuous control force than the corresponding bang–bang control at the price of slightly less control effectiveness.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

References

  1. 1.

    Agrawal, A.K., Yang, J.N.: Optimal polynomial control of seismically excited linear structures. J. Eng. Mech. 122, 753–761 (2000)

    Article  Google Scholar 

  2. 2.

    Crandall, M.G., Lions, P.L.: Viscostity solutions of Hamilton–Jacobi equations. Trans. Am. Math. Soc. 277, 1–42 (1983)

    Article  MATH  MathSciNet  Google Scholar 

  3. 3.

    Crespo, L.G., Sun, J.Q.: Stochastic optimal control of nonlinear systems via short-time gaussian approximation and cell mapping. Nonlinear Dyn. 28, 323–342 (2002)

    Article  MATH  MathSciNet  Google Scholar 

  4. 4.

    Crespo, L.G., Sun, J.Q.: Stochastic optimal control via bellman’s principle. Automatica 39, 2109–2114 (2003)

    Article  MATH  MathSciNet  Google Scholar 

  5. 5.

    Fleming, W.H., Soner, H.M.: Controlled Markov process and viscosity solutions. Springer, New York (1992)

    Google Scholar 

  6. 6.

    Gu, X.D., Zhu, W.Q., Xu, W.: Stochastic optimal control of quasi non-integrable hamiltonian systems with maximum principle. Nonlinear Dyn. 70, 779–787 (2012)

    Article  MATH  MathSciNet  Google Scholar 

  7. 7.

    Kovaleva, A.S.: Asymptotic solution of the optimal control problem for non-linear oscillations in the neighbourhood of resonance. Prikl. Math. Mekh. 62(6), 913–922 (1999)

    MathSciNet  Google Scholar 

  8. 8.

    Kushner, H.J.: Optimality conditions for the average cost per unit time problem with a diffusion model. SIAM J. Control Optim. 16, 330–346 (1978)

    Article  MATH  MathSciNet  Google Scholar 

  9. 9.

    Ma, J., Shen, J., Zhao, Y.H.: On numerical approximations of forward–backward stochastic differential equations. SIAM J. Numer. Anal. 46, 2636–2661 (2008)

    Article  MATH  MathSciNet  Google Scholar 

  10. 10.

    Ma, J., Yong, J.M.: Forward–backward stochastic differential equations and their applications. Springer, Berlin (1999)

    MATH  Google Scholar 

  11. 11.

    Milstein, G.N., Tretyakov, M.V.: Numerical algorithms for forward–backward stochastic differential equations. SIAM J. Sci. Comput. 39, 1535–1546 (2006)

    MathSciNet  Google Scholar 

  12. 12.

    Peng, S.G.: A general stochastic maximum principle for optimal control problems. SIAM J. Control Optim. 28(4), 966–979 (1990)

    Article  MATH  MathSciNet  Google Scholar 

  13. 13.

    To, C.W.S., Lin, R.: Bifurcations in a stochastically disturbed nonlinear two-degree-of-freedom system. Struct. Saf. 6, 223–231 (1989)

    Article  Google Scholar 

  14. 14.

    Wong, E., Zakai, W.: On the relation between ordinary and stochastic differential equations. Int. J. Eng. Sci. 3, 213–229 (1965)

    Article  MATH  MathSciNet  Google Scholar 

  15. 15.

    Yong, J.M., Zhou, X.Y.: Stochastic controls, hamiltonian systems and HJB equations. Springer, New York (1999)

    MATH  Google Scholar 

  16. 16.

    Zhu, W.Q.: Nonlinear stochastic dynamics and control in hamiltonian formulation. Appl. Mech. Rev. 59(4), 230–247 (2006)

    Article  Google Scholar 

  17. 17.

    Zhu, W.Q., Deng, M.L.: Optimal bounded control for minimizing the response of quasi-integrable hamiltonian systems. Int. J. Non-linear Mech. 39, 1535–1546 (2004)

    Article  MATH  MathSciNet  Google Scholar 

  18. 18.

    Zhu, W.Q., Huang, Z.L.: Stochastic stabilization of quasi-partially integrable hamiltonian systems by using lyapunov exponent. Nonlinear Dyn. 33, 209–224 (2003)

    Article  MATH  MathSciNet  Google Scholar 

  19. 19.

    Zhu, W.Q., Yang, Y.G.: Stochastic averaging of quasi non-integrable hamiltonian systems. ASME J. Appl. Mech. 64, 157–164 (1997)

    Article  MATH  MathSciNet  Google Scholar 

  20. 20.

    Zhu, W.Q., Deng, M.L., Huang, Z.L.: Optimal bounded control of first-passage failure of quasi-integrable hamiltonian systems with wide-band random excitation. Nonlinear Dyn. 33, 189–207 (2003)

    Article  MATH  MathSciNet  Google Scholar 

  21. 21.

    Zhu, W.Q., Ying, Z.G., Soong, T.T.: An optimal nonlinear feedback control strategy for randomly excited structural systems. Nonlinear Dyn. 24, 31–51 (2001)

    Article  MATH  MathSciNet  Google Scholar 

Download references

Acknowledgments

This study was supported by the National Nature Science Foundation of China under NSFC Grant Nos. 10932009, 11072212, 11272279 and the Basic Research Fund of Northwestern Polytechnical University under Grant No. JC201242.

Author information

Affiliations

Authors

Corresponding author

Correspondence to W. Q. Zhu.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Gu, X.D., Zhu, W.Q. Optimal bounded control of quasi-nonintegrable Hamiltonian systems using stochastic maximum principle. Nonlinear Dyn 76, 1051–1058 (2014). https://doi.org/10.1007/s11071-013-1188-x

Download citation

Keywords

  • Quasi-nonintegrable Hamiltonian system
  • Stochastic optimal control
  • Stochastic averaging
  • Stochastic maximum principle