Equations for the Missing Boundary Values in the Hamiltonian Formulation of Optimal Control Problems

Abstract

Partial differential equations for the unknown final state and initial costate arising in the Hamiltonian formulation of regular optimal control problems with a quadratic final penalty are found. It is shown that the missing boundary conditions for Hamilton’s canonical ordinary differential equations satisfy a system of first-order quasilinear vector partial differential equations (PDEs), when the functional dependence of the H-optimal control in phase-space variables is explicitly known. Their solutions are computed in the context of nonlinear systems with ℝⁿ-valued states. No special restrictions are imposed on the form of the Lagrangian cost term. Having calculated the initial values of the costates, the optimal control can then be constructed from on-line integration of the corresponding 2n-dimensional Hamilton ordinary differential equations (ODEs). The off-line procedure requires finding two auxiliary n×n matrices that generalize those appearing in the solution of the differential Riccati equation (DRE) associated with the linear-quadratic regulator (LQR) problem. In all equations, the independent variables are the finite time-horizon duration T and the final-penalty matrix coefficient S, so their solutions give information on a whole two-parameter family of control problems, which can be used for design purposes. The mathematical treatment takes advantage from the symplectic structure of the Hamiltonian formalism, which allows one to reformulate Bellman’s conjectures concerning the “invariant-embedding” methodology for two-point boundary-value problems. Results for LQR problems are tested against solutions of the associated differential Riccati equation, and the attributes of the two approaches are illustrated and discussed. Also, nonlinear problems are numerically solved and compared against those obtained by using shooting techniques.