Skip to main content
SpringerLink
Log in

Initialization of the Shooting Method via the Hamilton-Jacobi-Bellman Approach

  • Published:
Journal of Optimization Theory and Applications Aims and scope Submit manuscript

Abstract

The aim of this paper is to investigate from the numerical point of view the coupling of the Hamilton-Jacobi-Bellman (HJB) equation and the Pontryagin minimum principle (PMP) to solve some control problems. A rough approximation of the value function computed by the HJB method is used to obtain an initial guess for the PMP method. The advantage of our approach over other initialization techniques (such as continuation or direct methods) is to provide an initial guess close to the global minimum. Numerical tests involving multiple minima, discontinuous control, singular arcs and state constraints are considered.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Büskens, C.: NUDOCCCS—User’s manual. Universität Münster (1996)

  2. Diehl, M., Leineweber, D.B., Schäfer, A.A.S.: MUSCOD-II Users’ Manual. University of Heidelberg, Germany (2001)

    Google Scholar 

  3. Wächter, A., Biegler, L.T.: On the implementation of a primal-dual interior point filter line search algorithm for large-scale nonlinear programming. Math. Program. 106, 25–57 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  4. Chudej, K., Büskens, Ch., Graf, T.: Solution of a hard flight path optimization problem by different optimization codes. In: Breuer, M., Durst, F., Zenger, C. (eds.) High-Performance Scientific and Engineering Computing. Lecture Notes in Computational Science and Engineering, vol. 21, pp. 289–296. Springer, New York (2002)

    Google Scholar 

  5. Maurer, H., Pesch, H.J.: Direct optimization methods for solving a complex state-constrained optimal control problem in microeconomics. Appl. Math. Comput. 204, 568–579 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  6. Clarke, F., Vinter, R.B.: The relationship between the maximum principle and dynamic programming. SIAM J. Control Optim. 25, 1291–1311 (1987)

    Article  MathSciNet  MATH  Google Scholar 

  7. Cannarsa, P., Frankowska, H.: Some characterizations of optimal trajectories in control theory. SIAM J. Control Optim. 28, 1322–1347 (1991)

    Article  MathSciNet  Google Scholar 

  8. Cannarsa, P., Frankowska, H., Sinestrari, C.: Optimality conditions and synthesis for the minimum time problem. Set-Valued Anal. 8, 127–148 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  9. Cernea, A., Frankowska, H.: A connection between the maximum principle and dynamic programming for constrained control problems. SIAM J. Control Optim. 44, 673–703 (2006)

    Article  MathSciNet  Google Scholar 

  10. Pesch, H.J.: A practical guide to the solution of real-life optimal control problems. Control Cybern. 23, 7–60 (1994)

    MathSciNet  MATH  Google Scholar 

  11. Pesch, H.J., Plail, M.: The maximum principle of optimal control: a history of ingenious idea and missed opportunities. Control and Cybern. (to appear)

  12. Deuflhard, P.: Newton Methods for Nonlinear Problems. Springer Series in Computational Mathematics, vol 35. Springer, Berlin (2004)

    MATH  Google Scholar 

  13. Bellman, R.E.: The theory of dynamic programming. Bull. Am. Math. Soc. 60, 503–515 (1954)

    Article  MATH  Google Scholar 

  14. Falcone, M.: Numerical solution of dynamic programming equations. Appendix A in [21]

  15. Falcone, M.: Numerical methods for differential games based on partial differential equations. Int. Game Theory Rev. 8, 231–272 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  16. Pontryagin, L.S., Boltyanski, V.G., Gamkrelidze, R.V., Mishtchenko, E.F.: The Mathematical Theory of Optimal Processes. Wiley–Interscience, New York (1962)

    MATH  Google Scholar 

  17. Bryson, A.E., Ho, Y.-C.: Applied Optimal Control. Hemisphere, New York (1975)

    Google Scholar 

  18. Pesch, H.J.: Real-time computation of feedback controls for constrained optimal control problems II: a correction method based on multiple shooting. Optim. Control Appl. Methods 10, 147–171 (1989)

    Article  MathSciNet  MATH  Google Scholar 

  19. Bettiol, P., Frankowska, H.: Normality of the maximum principle for nonconvex constrained Bolza problems. J. Differ. Equ. 243, 256–269 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  20. Chitour, Y., Jean, F., Trélat, E.: Genericity results for singular curves. J. Differ. Geom. 73, 45–73 (2006)

    MATH  Google Scholar 

  21. Bardi, M., Capuzzo Dolcetta, I.: Optimal Control and Viscosity Solutions of Hamilton-Jacobi-Bellman Equations. Birkhäuser, Boston (1997)

    Book  MATH  Google Scholar 

  22. Cristiani, E., Falcone, M.: Fast semi-Lagrangian schemes for the Eikonal equation and applications. SIAM J. Numer. Anal. 45, 1979–2011 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  23. Bokanowski, O., Martin, S., Munos, R., Zidani, H.: An anti-diffusive scheme for viability problems. Appl. Numer. Math. 56, 1147–1162 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  24. Bokanowski, O., Cristiani, E., Zidani, H.: An efficient data structure and accurate scheme to solve front propagation problems. J. Sci. Comput. 42, 251–273 (2010)

    Article  MathSciNet  Google Scholar 

  25. Tsai, Y.R., Cheng, L.T., Osher, S., Zhao, H.: Fast sweeping algorithms for a class of Hamilton-Jacobi equations. SIAM J. Numer. Anal. 41, 673–694 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  26. Bardi, M., Falcone, M., Soravia, P.: Fully discrete schemes for the value function of pursuit-evasion games. In Basar, T., Haurie, A. (eds) Advances in Dynamic Games and Applications, Annals of the International Society of Dynamic Games, vol. 1, pp. 89–105 (1994)

  27. Soravia, P.: Estimates of convergence of fully discrete schemes for the Isaacs equation of pursuit-evasion differential games via maximum principle. SIAM J. Control Optim. 36, 1–11 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  28. More, J.J., Garbow, B.S., Hillstrom, K.E.: User Guide for MINIPACK-1. Argonne National Laboratory Report ANL-80-74 (1980)

  29. Hairer, E., Nørsett, S.P., Wanner, G.: Solving Ordinary Differential Equations I. Springer Series in Computational Mathematics, vol. 8. Springer, Berlin (1993)

    Google Scholar 

  30. Gergaud, J., Martinon, P.: Using switching detection and variational equations for the shooting method. Optim. Control Appl. Methods 28, 95–116 (2007)

    Article  MathSciNet  Google Scholar 

  31. Goddard, R.H.: A method of reaching extreme altitudes. Smithsonian Inst. Misc. Coll. 71 (1919)

  32. Maurer, H.: Numerical solution of singular control problems using multiple shooting techniques. J. Optim. Theory Appl. 18, 235–257 (1976)

    Article  MathSciNet  MATH  Google Scholar 

  33. Oberle, H.J.: Numerical computation of singular control functions in trajectory optimization. J. Guid. Control Dyn. 13, 153–159 (1990)

    Article  MathSciNet  MATH  Google Scholar 

  34. Tsiotras, P., Kelley, H.J.: Drag-law effects in the Goddard problem. Automatica 27, 481–490 (1991)

    Article  MathSciNet  Google Scholar 

  35. Seywald, H., Cliff, E.M.: Goddard problem in presence of a dynamic pressure limit. J. Guid. Control Dyn. 16, 776–781 (1993)

    Article  MATH  Google Scholar 

  36. Bonnans, F., Martinon, P., Trélat, E.: Singular arcs in the generalized Goddard’s problem. J. Optim. Theory Appl. 139, 439–461 (2008)

    Article  MathSciNet  Google Scholar 

  37. Camilli, F., Falcone, M., Lanucara, P., Seghini, A.: A domain decomposition method for Bellman equations. In: Keyes, D.E., Xu, J. (eds.) Domain Decomposition Methods in Scientific and Engineering Computing. Contemporary Mathematics, vol. 180, pp. 477–483. AMS, Providence (1994)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. CEMSAC, Università di Salerno, Fisciano, SA, Italy

    E. Cristiani

  2. IAC-CNR, Rome, Italy

    E. Cristiani

  3. INRIA and CMAP École Polytechnique, Palaiseau, France

    P. Martinon

Authors
  1. E. Cristiani
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. P. Martinon
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to E. Cristiani.

Additional information

Communicated by H.J. Pesch.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Cristiani, E., Martinon, P. Initialization of the Shooting Method via the Hamilton-Jacobi-Bellman Approach. J Optim Theory Appl 146, 321–346 (2010). https://doi.org/10.1007/s10957-010-9649-6

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10957-010-9649-6

Keywords

Search

Navigation