Abstract
Connections between the principle of least action and optimal control are explored with a view to describing the trajectories of energy conserving systems, subject to temporal boundary conditions, as solutions of corresponding systems of characteristics equations on arbitrary time horizons. Motivated by the relaxation of least action to stationary action for longer time horizons, due to loss of convexity of the action functional, a corresponding relaxation of optimal control problems to stationary control problems is considered. In characterizing the attendant stationary controls, corresponding to generalized velocity trajectories, an auxiliary stationary control problem is posed with respect to the characteristic system of interest. Using this auxiliary problem, it is shown that the controls rendering the action functional stationary on arbitrary time horizons have a state feedback representation, via a verification theorem, that is consistent with the optimal control on short time horizons. An example is provided to illustrate application via a simple mass-spring system.
Similar content being viewed by others
References
Baggett, L.W.: Functional Analysis. Dekker, London (1991)
Bardi, M., Capuzzo-Dolcetta, I.: Optimal Control and Viscosity Solutions of Hamilton-Jacobi-Bellman Equations. Systems & Control: Foundations & Application. Birkhauser, Basel (1997)
Basco, V., Frankowska, H.: Lipschitz continuity of the value function for the infinite horizon optimal control problem under state constraints. In: Trends in Control Theory and Partial Differential Equations, Springer INdAM Series, pp. 17–38 (2019)
Cannarsa, P., Sinestrari, C.: Semiconcave Functions, Hamilton-Jacobi Equations, and Optimal Control. Progress in Nonlinear DIfferential Equations and Their Applications. Birkhaüser, Basel (2004)
Dower, P.M., McEneaney, W.M.: On existence and uniqueness of stationary action trajectories. In: Proceedings 22nd International Symposium on Mathematical Theory of Networks and Systems (Minneapolis MN, USA), pp. 624–631 (2016)
Dower, P.M., McEneaney, W.M.: Solving two-point boundary value problems for a wave equation via the principle of stationary action and optimal control. SIAM J. Control Optim. 55(4), 2151–2205 (2017)
Dower, P.M., McEneaney, W.M.: Verifying fundamental solution groups for lossless wave equations via stationary action and optimal control. Appl. Math. Optim. (2020). https://doi.org/10.1007/s00245-020-09700-4
Dower, P.M., McEneaney, W.M., Yegorov, I.: Exploiting characteristics in stationary action problems. In: Proceedings of SIAM Conference on Control & Its Applications (Chengdu, China), pp. 75–82 (2019)
Gray, C.G., Taylor, E.F.: When action is not least. Am. J. Phys. 75(5), 434–458 (2007)
McEneaney, W.M., Dower, P.M.: The principle of least action and fundamental solutions of mass-spring and \(n\)-body two-point boundary value problems. SIAM J. Control Optim. 53(5), 2898–2933 (2015)
McEneaney, W.M., Dower, P.M.: Staticization, its dynamic program, and solution propagation. Automatica 81, 56–67 (2017)
McEneaney, W.M., Dower, P.M.: Static duality and a stationary action application. J. Differ. Equ. 264(2), 525–549 (2018)
McEneaney, W.M., Dower, P.M.: Staticization and associated Hamilton-Jacobi and Riccati equations. SIAM Conf. Control Theory Appl. 2015, 376–383 (2015)
McEneaney, W.M., Dower, P.M.: Staticization-based representations for Schrödinger equations driven by Coulomb potentials. In: \(3^{rd}\) IFAC Workshop on Thermodynamic Foundations for a Mathematical Systems Theory (Louvain-la-Neuve, Belgium). https://doi.org/10.1016/j.ifacol.2019.07.005 (2019)
Pazy, A.: Semigroups of Linear Operators and Applications to Partial Differential Equations. Applied Mathematical Sciences, vol. 44. Springer, New York (1983)
Subbotin, A.I.: Generalized Solutions of First-Order PDEs: The Dynamical Optimization Perspective. Systems & Control: Foundations & Application. Springer, New York (1995)
Yegorov, I., Dower, P.M.: Perspectives on characteristics based curse-of-dimensionality-free numerical approaches to solving Hamilton Jacobi equations. Appl. Math. Optim. (2018)
Funding
This research was supported by the US Air Force Office of Scientific Research.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supported by AFOSR Grant FA2386-16-1-4066. A preliminary version of this work appeared in [8].
Appendices
Appendix
Proofs of Lemmas 2, 3, 4, and 5
Proof (Lemma 2)
The proof employs a standard fixed point argument, exploiting global Lipschitz continuity of f of (22), see for example [15, Theorem 5.1, p. 127]. Note that global Lipschitz continuity of \({\nabla }V(x)\) in (22) follows directly from the second bound assumed in (4). \(\square \)
Proof (Lemma 3)
Fix \(T\in {\mathbb {R}}_{\ge 0}\), \(t\in [0,T]\), and \(Y,h\in {\mathscr {X}}^2\). Applying Lemma 2, there exist unique classical solutions \({\overline{X}}(Y)\) and \({\overline{X}}(Y+h)\) to (24) satisfying respectively \({\overline{X}}(Y)_t = Y\) and \({\overline{X}}(Y+h)_t = Y+h\). In integral form,
so that
for all \(s\in [t,T]\). Consequently, as f is globally Lipschitz by inspection of (22),
in which \(\alpha \in {\mathbb {R}}_{\ge 0}\) is the associated Lipschitz constant. Applying Gronwall’s inequality, and recalling the definition of \(\Vert \cdot \Vert _\infty \), yields
so that (26) holds. As \(Y,h\in {\mathscr {X}}^2\) are arbitrary, the asserted continuity follows. \(\square \)
Proof (Lemma 4)
Fix \(T\in {\mathbb {R}}_{\ge 0}\), \(t\in [0,T]\), and \({\overline{X}}\in C({\mathscr {X}}^2;C([t,T];{\mathscr {X}}^2))\) as per Lemma 3. Fix \(Y\in {\mathscr {X}}^2\) and \(s\mapsto A(Y)_s\) as per (29), and note that (28) follows by [15, Theorem 5.2, p. 128]. Fix any \(h\in {\mathscr {X}}^2\), \(s\in [t,T]\), and note by inspection of (22) that \(A(Y)_s = D f({\overline{X}}(Y)_s)\). Hence, recalling (52),
Define \(\bar{I}_f:C([t,T];{\mathscr {X}}^2)\rightarrow C([t,T];{\mathscr {X}}^2)\) by
for all \(X\in C([t,T];{\mathscr {X}}^2)\). Note that \(Y\mapsto f(Y)\) is twice Fréchet differentiable by (4), with \(D^2 f(Y)\in {\mathcal {L}}({\mathscr {X}}^2;{\mathcal {L}}({\mathscr {X}}^2)) = {\mathcal {L}}({\mathscr {X}}^2\times {\mathscr {X}}^2;{\mathscr {X}}^2)\) for all \(Y\in {\mathscr {X}}^2\). Again by (4), there exists an \(M\in {\mathbb {R}}_{>0}\) such that
Hence, by the mean value theorem, given \(X,\delta \in C([t,T];{\mathscr {X}}^2)\),
That is,
so that \(\bar{I}_f\) is Fréchet differentiable with derivative
for all \(X,\delta \in C([t,T];{\mathscr {X}}^2)\), \(s\in [t,T]\). So, recalling (53), and (1),
Noting that \(L\doteq \sup _{\sigma \in [t,T]} \Vert D f({\overline{X}}(Y)_\sigma ) \Vert _{{\mathcal {L}}({\mathscr {X}}^2)} < \infty \), taking the norm of both sides yields
Hence, by Gronwall’s inequality,
or, with \(\theta _Y(h) \doteq \Vert d[{\bar{I}_f}]_{{\overline{X}}(Y)} ( {\overline{X}}(Y+h) - {\overline{X}}(Y) ) \Vert _\infty \),
As \(\theta _Y\) is continuous at 0, there exists an \(r>0\) sufficiently small such that \(\Vert h\Vert < r\) implies that \(\theta _Y(h) \exp (L\, (T-t)) < {{\textstyle {\frac{1}{2}}}}\). Hence, with \(\Vert h\Vert < r\),
in which \(Q \doteq 2\, \sup _{s\in [t,T]} \Vert U_{s,t}(Y) \Vert _{{\mathcal {L}}({\mathscr {X}}^2)}\, \exp (L\, (T-t))\). Consequently, taking a limit,
That is, \(Y\mapsto {\overline{X}}(Y)\) is Fréchet differentiable, with the indicated derivative. \(\square \)
Proof (Lemma 5)
Fix \(T\in {\mathbb {R}}_{>0}\), \(t\in [0,T]\) as per the lemma statement. It is first demonstrated that \(Y\mapsto U_{s,r}(Y)\) is continuous, uniformly in \(r,s\in [t,T]\), as this motivates the subsequent proof of continuous differentiability. Fix \(r,s\in [t,T]\), \(h,\hat{h}\in {\mathscr {X}}^2\). As \(U_{s,r}(Y)\in {\mathcal {L}}({\mathscr {X}}^2)\) is an element of the two-parameter family of evolution operators generated by \(A(Y)_s\in {\mathcal {L}}({\mathscr {X}}^2)\), see (29),
so that
Hence, by the triangle inequality,
Recalling (4), and in particular the uniform bound on \(x\mapsto D{\nabla }^2 V(x)\), given \(x,\bar{x}\in {\mathscr {X}}^2\), the mean value theorem implies that \({\nabla }^2 V(x) - {\nabla }^2 V(\bar{x}) = ( \int _0^1 D{\nabla }^2 V(\bar{x} + \eta \, (x - \bar{x})) \, d\eta ) (x - \bar{x})\), so that \(\Vert {\nabla }^2 V(x) - {\nabla }^2 V(\bar{x})\Vert _{{\mathcal {L}}({\mathscr {X}})} \le \frac{K}{2}\, \Vert x - \bar{x}\Vert \). Hence, by (29), there exists an \(\alpha _1\in {\mathbb {R}}_{\ge 0}\) such that \(\varLambda :{\mathscr {X}}^2\rightarrow {\mathcal {L}}({\mathscr {X}}^2)\) satisfies \(\Vert \varLambda (Z) - \varLambda (\bar{Z}) \Vert _{{\mathcal {L}}({\mathscr {X}}^2)} \le \alpha _1 \Vert Z - \bar{Z} \Vert \) for all \(Z,\bar{Z}\in {\mathscr {X}}^2\). So, applying Lemma 3, there exists an \(\alpha \in {\mathbb {R}}_{\ge 0}\), \(L_0 \doteq \sup _{\sigma \in [t,T]}\Vert A(0)_\sigma \Vert _{{\mathcal {L}}({\mathscr {X}}^2)}<\infty \), \(L_1 \doteq \alpha _1\, \exp (\alpha \, (T-t))<\infty \), such that
in which the second inequality follows from the first, via the triangle inequality, by selecting \(\hat{h} = -Y\). Note further that as \(\sigma \mapsto A(Y)_\sigma \) is continuous, \(L_2 \doteq \sup _{\sigma \in [t,T]} \Vert U_{\sigma , t} (Y) \Vert _{{\mathcal {L}}({\mathscr {X}}^2)} < \infty \), see [15, Theorem 5.2, p.128]. Hence, substituting these inequalities in (57) yields
Gronwall’s inequality subsequently implies that
Continuity of \(Y\mapsto U_{s,r}(Y)\), uniformly in \(r,s\in [t,T]\), thus follows.
Next, \(Y\mapsto U_{s,r}(Y)\) is shown to be Fréchet differentiable, uniformly in \(r,s\in [t,T]\). Appealing to the contraction theorem and Picard’s principle, for any \(t\le r<s\le T\) and \(Y\in {\mathscr {X}}\), consider the two-parameter family of operators \(V_{s,r}(Y)\in {\mathcal {L}}({\mathscr {X}}^2;{\mathcal {L}}({\mathscr {X}}^2))\) solving
for all \(h, \hat{h}\in {\mathscr {X}}^2\), \(r,s\in [t,T]\), in which \(D_Y A(Y)_\sigma = D \varLambda ({\overline{X}}(Y)_\sigma )\, U_{\sigma ,t}(Y) \in {\mathcal {L}}({\mathscr {X}}^2;{\mathcal {L}}({\mathscr {X}}^2))\) by the chain rule and Lemma 4. Note in particular by (4), (29), and Lemma 3 that
Applying the triangle inequality to (60), and recalling the definitions of \(L_0\), \(L_1\), \(L_2\), yields
so that by Gronwall’s inequality,
As \(\hat{h}, h\in {\mathscr {X}}^2\) are arbitrary, it follows immediately that \(V_{s,r}(Y)\in {\mathcal {L}}({\mathscr {X}}^2;{\mathcal {L}}({\mathscr {X}}^2))\) for all \(r,s\in [t,T]\). Recalling (56), observe by adding and subtracting terms that
and the last term in square brackets is zero by definition (60) of \(V_{s,r}(Y)\). Define \(\hat{A}:C([t,T];{\mathscr {X}}^2)\rightarrow C([t,T];{\mathcal {L}}({\mathscr {X}}^2))\) by \(\hat{A}(X)_\sigma \doteq A(X_\sigma )\) for all \(X\in C([t,T];{\mathscr {X}}^2)\), and note that the range of \(\hat{A}\) follows by (4), (29). Fix \(X,\delta \in C([t,T];{\mathscr {X}}^2)\), and (for convenience) write \(X_\sigma = ([X_1]_\sigma ], [X_2]_\sigma ])\in {\mathscr {X}}^2\), \(\delta _\sigma = ([\delta _1]_\sigma , [\delta _2]_\sigma )\in {\mathscr {X}}^2\) for all \(\sigma \in [t,T]\), with \(X_1,X_2,\delta _1,\delta _2\in C([t,T];{\mathscr {X}})\). Combining (4), (29) with the mean value theorem, there exists \(\hat{\alpha }\in {\mathbb {R}}_{\ge 0}\) such that
for all \(X,\delta \in C([t,T];{\mathscr {X}}^2)\). Dividing both sides by \(\Vert \delta \Vert _{C([t,T];{\mathscr {X}}^2)}\) and taking the limit as \(\Vert \delta \Vert _{C([t,T];{\mathscr {X}}^2)}\rightarrow 0\) subsequently yields that \(\hat{A}\) is Fréchet differentiable with derivative \(D\hat{A}(X)\in {\mathcal {L}}(C([t,T];{\mathscr {X}}^2);C([t,T];{\mathcal {L}}({\mathscr {X}}^2)\). Hence, taking the norm of both sides of (61), applying the triangle inequality, (59), (62), and recalling the definitions of \(L_1\), \(L_2\), \(L_3\),
in which \(d(\hat{A}\circ {\overline{X}})_Y(\cdot )\) is defined via (1). Hence, by Gronwall’s inequality,
As \(\hat{h}, h\in {\mathscr {X}}^2\) are arbitrary,
Hence, \(Y\mapsto U_{s,r}(Y)\) is Fréchet differentiable, uniformly in \(r,s\in [t,T]\), with derivative \(V_{s,r}(Y)\).
It remains to be shown that \(Y\mapsto U_{s,r}(Y)\) is twice Fréchet differentiable via (60). To this end, define \(\upsilon _s\doteq V_{s,r}(Y) \, \hat{h}\in {\mathcal {L}}({\mathscr {X}}^2)\) and \(w_s\doteq D_Y A(Y)_s\, \hat{h}\, U_{s,r}(Y)\in {\mathcal {L}}({\mathscr {X}}^2)\) for all \(s\in [t,T]\), and note by (60) that
for all \(s\in [t,T]\), recalling that \(h\in {\mathscr {X}}\) in (60) is arbitrary. Equivalently, \(s\mapsto \upsilon _s\) is the unique solution of the IVP \(\dot{\upsilon }_s = A(Y)_s \, \upsilon _s + w_s\) for all \(s\in (t,T)\), subject to \(\upsilon _r = 0\in {\mathcal {L}}({\mathscr {X}}^2)\). By definition, \(s\mapsto A(Y)_s\) generates the two-parameter family \(U_{s,r}(Y)\), \(r,s\in [t,T]\), so that \(s\mapsto \upsilon _s = V_{s,r}(Y)\, \hat{h}\) satisfies
for all \(r,s\in [t,T]\), in which the third equality follows as \(v_r = V_{r,r}(Y)\, \hat{h} = 0\in {\mathcal {L}}({\mathscr {X}}^2)\), either by (60) or directly as \(V_{r,r}(Y) \doteq D_Y U_{r,r}(Y) = D_Y I = 0\). Hence, by inspection of (64), the map \(Y\mapsto V_{s,r}(Y)\doteq D_Y U_{s,r}(Y)\) is also Fréchet differentiable, with
in which \(D_Y^2 A(Y)_\sigma \in {\mathcal {L}}({\mathscr {X}}^2\times {\mathscr {X}}^2;{\mathcal {L}}({\mathscr {X}}^2))\), \(\sigma \in [t,T]\), exists by (29) and (4).
\(\square \)
An Auxiliary Statement of Proposition 2
Proposition 3
Given \(T\in {\mathbb {R}}_{>0}\), \(t\in [0,T)\), \(x,p\in {\mathscr {X}}\), and \((\bar{x}_s, \bar{p}_s) \doteq {\overline{X}}(Y_p(x))_s\) for all \(s\in [t,T]\), the maps \(s\mapsto {\nabla }_p \bar{J}_T(s,\bar{x}_s, \bar{p}_s)\) and \(s\mapsto {\nabla }_x \bar{J}_T(s,\bar{x}_s, \bar{p}_s)\) are continuously differentiable, with derivatives given by
for all \(s\in (t,T)\). Moreover, \(s\mapsto {\nabla }_p \bar{J}_T(s,\bar{x}_s, \bar{p}_s)\) is twice continuously differentiable, and satisfies
for all \(s\in (t,T)\).
Proof
Fix \(T\in {\mathbb {R}}_{>0}\), \(x,p\in {\mathscr {X}}\), and let \((\bar{x}_s, \bar{p}_s)\in {\mathscr {X}}^2\), \(s\in [t,T]\), be as per the lemma statement. Fix \(h\in {\mathscr {X}}\). Applying Proposition 1, \((s,x,p)\mapsto \bar{J}_T(s,x,p)\) is twice continuously differentiable, and the order of differentiation may be swapped. In particular,
Meanwhile, \(\bar{J}_T\) satisfies (41) by Theorem 5, i.e.
for all \(s\in (t,T)\), \(x,p\in {\mathscr {X}}\). Differentiating (69) with respect to p,
Evaluating along the trajectory \(s\mapsto (\bar{x}_s,\bar{p}_s)\) corresponding to \({\overline{X}}(Y_p(x))\), i.e. as per (20), yields
Substitution in (68) subsequently yields
Recalling that \(h\in {\mathscr {X}}\) is arbitrary immediately yields (65).
Similarly, for (66), observe that
Differentiating (69) with respect to x,
Evaluating along the trajectory \(s\mapsto (\bar{x}_s,\bar{p}_s)\) corresponding to \({\overline{X}}(Y_p(x))\), i.e. as per (20), yields
Substitution in (71) subsequently yields
Recalling that \(h\in {\mathscr {X}}\) is arbitrary immediately yields (66).
The remaining assertion regarding twice differentiability is immediate by inspection of (65), (66), with
as required. \(\square \)
Rights and permissions
About this article
Cite this article
Basco, V., Dower, P.M., McEneaney, W.M. et al. Exploiting Characteristics in Stationary Action Problems. Appl Math Optim 84 (Suppl 1), 733–765 (2021). https://doi.org/10.1007/s00245-021-09784-6
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00245-021-09784-6
Keywords
- Stationary action
- Optimal control
- Characteristics
- Hamilton–Jacobi–Bellman partial differential equations