Abstract
In this paper, a family of discrete optimal control problems that depend on parameters is considered. The control problems are reformulated as parametric optimization problems. By establishing/exploiting abstract results on subdifferentials of optimal value functions of parametric optimization problems, we derive formulas for estimating/computing subdifferentials of optimal value functions of parametric discrete optimal control problems in both nonconvex and convex cases. Namely, for control problems with nonconvex costs, upper-evaluations on the regular subdifferential and the limiting (Mordukhovich) subdifferential of the optimal value function are obtained without using the (strict) differentiability of the costs. Meanwhile, for control problems with convex costs, besides results on estimating/computing the subdifferential (in the sense of convex analysis) of the optimal value function, it is worth pointing out that some properties of the optimal value function are first discussed in this paper.
References
An, D.T.V., Jourani, A.: Subdifferentials of the marginal functions in parametric convex optimization via intersection formulas. J. Optim. Theory Appl. 192, 82–96 (2022)
An, D.T.V., Toan, N.T.: Differential stability of convex discrete optimal control problems. Acta Math. Vietnam. 43, 201–217 (2018)
An, D.T.V., Yao, J.-C.: Further results on differential stability of convex optimization problems. J. Optim. Theory Appl. 170, 28–42 (2016)
An, D.T.V., Yao, J.-C., Yen, N.D.: Differential stability of a class of convex optimal control problems. Appl. Math. Optim. 81, 1–22 (2020)
An, D.T.V., Yen, N.D.: Differential stability of convex optimization problems under inclusion constraints. Appl. Anal. 94, 108–128 (2015)
Aris, R.: Discrete Dynamic Programming; An Introduction to the Optimization of Staged Processes. Blaisdell Publishing Company, New York (1964)
Bellman, R.: Dynamic Programming. Princeton University Press, Princeton (1957)
Bellman, R., Dreyfus, S.E.: Applied Dynamic Programming. Princeton University Press, Princeton (1962)
Bertsekas, D.P.: Dynamic Programming and Optimal Control, vol. I. Athena Scientific, Belmont (1995)
Bonnans, J.F., Shapiro, A.: Perturbation Analysis of Optimization Problems. Springer, New York (2000)
Chieu, N.H., Yao, J.-C.: Subgradients of the optimal value function in a parametric discrete optimal control problem. J. Ind. Manag. Optim. 6, 401–410 (2010)
Hilscher, R., Zeidan, V.: Discrete optimal control: the accessory problem and necessary optimality conditions. J. Math. Anal. Appl. 243, 429–452 (2000)
Hilscher, R., Zeidan, V.: Second order sufficiency criteria for a discrete optimal control problem. J. Differ. Equ. Appl. 8, 573–603 (2002)
Ioffe, A.D., Tihomirov, V.M.: Theory of Extremal Problems. North-Holland Publishing Company, North-Holland (1979)
Kien, B.T., Liou, Y.C., Wong, N.-C., Yao, J.-C.: Subgradients of value functions in parametric dynamic programming. Eur. J. Oper. Res. 193, 12–22 (2009)
Malanowski, K.: Differential sensitivity of solutions of convex constrained optimal control problems for discrete systems. J. Optim. Theory Appl. 53, 429–449 (1987)
Malanowski, K.: Stability and sensitivity of solutions to nonlinear optimal control problems. Appl. Math. Optim. 32, 111–141 (1995)
Marinkovíc, B.: Sensitivity analysis for discrete optimal control problems. Math. Methods Oper. Res. 63, 513–524 (2006)
Marinkovíc, B.: Optimality conditions for discrete optimal control problems. Optim. Methods Softw. 22, 959–969 (2007)
Mordukhovich, B.S.: Variational Analysis and Generalized Differentiation, Volume I: Basic Theory. Springer, Berlin (2006)
Mordukhovich, B.S.: Variational Analysis and Generalized Differentiation, Volume II: Applications. Springer, Berlin (2006)
Mordukhovich, B.S.: Variational Analysis and Applications. Springer, Switzerland (2018)
Mordukhovich, B.S., Nam, N.M.: An Easy Path to Convex Analysis and Applications. Morgan & Claypool, Williston (2014)
Mordukhovich, B.S., Nam, N.M.: Geometric approach to convex subdifferential calculus. Optimization 66, 839–873 (2017)
Mordukhovich, B.S, Nam, N.M.: Convex Analysis and Beyond: Volume I: Basic Theory. Springer Series in Operations Research and Financial Engineering (2022)
Mordukhovich, B.S., Nam, N.M., Rector, R.B., Tran, T.: Variational geometric approach to generalized differential and conjugate calculi in convex analysis. Set-Valued Var. Anal. 25, 731–755 (2017)
Mordukhovich, B.S., Nam, N.M., Yen, N.D.: Subgradients of marginal functions in parametric mathematical programming. Math. Program. Ser. B 116, 369–396 (2009)
Pearson, J.B., Sridhar, R.: A discrete optimal control problem. IEEE Trans. Automat. Control 2, 171–174 (1966)
Penot, J.-P.: Calculus Without Derivatives. Graduate Texts In Mathematics 266. Springer, New York (2013)
Pindyck, R.S.: An application of the linear quadratic tracking problem to economic stabilization policy. IEEE Trans. Automat. Control 17, 287–300 (1972)
Qui, N.T., Wachsmuth, D.: Subgradients of marginal functions in parametric control problems of partial differential equations. SIAM J. Optim. 30, 1724–1755 (2020)
Rockafellar, R.T., Wets, R.J.-B.: Variational Analysis. Springer, Berlin (2004)
Seeger, A.: Subgradients of optimal-value functions in dynamic programming: the case of convex systems without optimal paths. Math. Oper. Res. 21, 555–575 (1996)
Toan, N.T.: Differential stability of discrete optimal control problems with mixed constraints. Positivity 25, 1229–1254 (2021)
Toan, N.T., Ansari, Q.H., Yao, J.-C: Second-order necessary optimality conditions for a discrete optimal control problem. J. Optim. Theory Appl. 165, 812–836 (2015)
Toan, N.T., Thuy, L.Q.: Second-order necessary optimality conditions for a discrete optimal control problem with mixed constraints. J. Global Optim. 64, 533–562 (2016)
Toan, N.T., Yao, J.-C.: Mordukhovich subgradients of the value function to a parametric discrete optimal control problem. J. Global Optim. 58, 595–612 (2014)
Tu, P.N.V.: Introductory Optimization Dynamics. Springer, Berlin (1991)
Zălinescu, C.: Convex Analysis in General Vector Spaces. World Scientific, London (2002)
Acknowledgements
The authors were indebted to the anonymous referee for his/her valuable suggestions and comments which greatly improved the presentation of this manuscript.
Funding
Hong-Kun Xu was supported in part by National Natural Science Foundation of China (grant number U1811461) and by Australian Research Council (grant number DP200100124).
Author information
Authors and Affiliations
Contributions
All authors contributed equally, read and approved the final manuscript.
Corresponding author
Ethics declarations
Conflict of interest
The authors have no relevant financial or non-financial interests to disclose.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
An, D.T.V., Huong, V.T. & Xu, H.K. Differential Stability of Discrete Optimal Control Problems with Possibly Nondifferentiable Costs. Appl Math Optim 86, 37 (2022). https://doi.org/10.1007/s00245-022-09905-9
Accepted:
Published:
DOI: https://doi.org/10.1007/s00245-022-09905-9
Keywords
- Parametric discrete optimal control problems
- Optimal value functions
- Nondifferentiable costs
- Linear-convex constraint maps
- Subdifferentials