New results on the relationship between dynamic programming and the maximum principle

Abstract

The dynamic programming approach to optimal control theory attempts to characterize the value functionV as a solution to the Hamilton-Jacobian-Bellman equation. Heuristic arguments have long been advanced relating the Pontryagin maximum principle and dynamic programming according to the equation (H(t, x^*(t), u^*(t), p(t)),−p(t))=√V(t,x^*(t)), where (x^*, u^*) is the optimal control process under consideration,p(t), is the coextremal, andH is the Hamiltonian. The relationship has previously been verified under only very restrictive hypotheses. We prove new results, establishing the relationship, now expressed in terms of the generalized gradient ofV, for a large class of nonsmooth problems.