Solving inverse optimal control problems via value functions to global optimality

Abstract

In this paper, we show how a special class of inverse optimal control problems of elliptic partial differential equations can be solved globally. Using the optimal value function of the underlying parametric optimal control problem, we transfer the overall hierarchical optimization problem into a nonconvex single-level one. Unfortunately, standard regularity conditions like Robinson’s CQ are violated at all the feasible points of this surrogate problem. It is, however, shown that locally optimal solutions of the problem solve a Clarke-stationarity-type system. Moreover, we relax the feasible set of the surrogate problem iteratively by approximating the lower level optimal value function from above by piecewise affine functions. This allows us to compute globally optimal solutions of the original inverse optimal control problem. The global convergence of the resulting algorithm is shown theoretically and illustrated by means of a numerical example.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4

References

  1. 1.

    Adams, R.A., Fournier, J.J.F.: Sobolev Spaces. Elsevier Science, Oxford (2003)

    Google Scholar 

  2. 2.

    Albrecht, S., Leibold, M., Ulbrich, M.: A bilevel optimization approach to obtain optimal cost functions for human arm movements. Numer. Algebra Control Optim. 2(1), 105–127 (2012). https://doi.org/10.3934/naco.2012.2.105

    MathSciNet  Article  MATH  Google Scholar 

  3. 3.

    Albrecht, S., Ulbrich, M.: Mathematical programs with complementarity constraints in the context of inverse optimal control for locomotion. Optim. Methods Softw. 32(4), 670–698 (2017). https://doi.org/10.1080/10556788.2016.1225212

    MathSciNet  Article  MATH  Google Scholar 

  4. 4.

    Bard, J.F.: Practical Bilevel Optimization: Algorithms and Applications. Kluwer Academic, Dordrecht (1998)

    Google Scholar 

  5. 5.

    Benita, F., Mehlitz, P.: Bilevel optimal control with final-state-dependent finite-dimensional lower level. SIAM J. Optim. 26(1), 718–752 (2016). https://doi.org/10.1137/15M1015984

    MathSciNet  Article  MATH  Google Scholar 

  6. 6.

    Bonnans, J.F., Shapiro, A.: Perturbation Analysis of Optimization Problems. Springer, New York (2000)

    Google Scholar 

  7. 7.

    Dempe, S.: Foundations of Bilevel Programming. Kluwer, Dordrecht (2002)

    Google Scholar 

  8. 8.

    Dempe, S., Dutta, J.: Is bilevel programming a special case of a mathematical program with complementarity constraints? Math. Program. 131(1), 37–48 (2012). https://doi.org/10.1007/s10107-010-0342-1

    MathSciNet  Article  MATH  Google Scholar 

  9. 9.

    Dempe, S., Franke, S.: On the solution of convex bilevel optimization problems. Comput. Optim. Appl. 63(3), 685–703 (2016). https://doi.org/10.1007/s10589-015-9795-8

    MathSciNet  Article  MATH  Google Scholar 

  10. 10.

    Dempe, S., Kalashnikov, V., Pérez-Valdéz, G., Kalashnykova, N.: Bilevel Programming Problems—Theory, Algorithms and Applications to Energy Networks. Springer, Berlin (2015)

    Google Scholar 

  11. 11.

    Fiacco, A.V., Kyparisis, J.: Convexity and concavity properties of the optimal value function in parametric nonlinear programming. J. Optim. Theory Appl. 48(1), 95–126 (1986). https://doi.org/10.1007/BF00938592

    MathSciNet  Article  MATH  Google Scholar 

  12. 12.

    Fisch, F., Lenz, J., Holzapfel, F., Sachs, G.: On the solution of bilevel optimal control problems to increase the fairness in air races. J. Guid. Control Dyn. 35(4), 1292–1298 (2012). https://doi.org/10.2514/1.54407

    Article  Google Scholar 

  13. 13.

    Flegel, M.L., Kanzow, C.: On M-stationary points for mathematical programs with equilibrium constraints. J. Math. Anal. Appl. 310(1), 286–302 (2005). https://doi.org/10.1016/j.jmaa.2005.02.011

    MathSciNet  Article  MATH  Google Scholar 

  14. 14.

    Haraux, A.: How to differentiate the projection on a convex set in Hilbert space. Some applications to variational inequalities. J. Math. Soc. Jpn. 29(4), 615–631 (1977). https://doi.org/10.2969/jmsj/02940615

    MathSciNet  Article  MATH  Google Scholar 

  15. 15.

    Harder, F., Wachsmuth, G.: Comparison of optimality systems for the optimal control of the obstacle problem. GAMM Mitt. 40(4), 312–338 (2018). https://doi.org/10.1002/gamm.201740004

    MathSciNet  Article  Google Scholar 

  16. 16.

    Harder, F., Wachsmuth, G.: Optimality conditions for a class of inverse optimal control problems with partial differential equations. Optimization (2018). https://doi.org/10.1080/02331934.2018.1495205

    MATH  Article  Google Scholar 

  17. 17.

    Hatz, K.: Efficient Numerical Methods for Hierarchical Dynamic Optimization with Application to Cerebral Palsy Gait Modeling. Ph.D. thesis, University of Heidelberg, Germany (2014)

  18. 18.

    Hatz, K., Schlöder, J.P., Bock, H.G.: Estimating parameters in optimal control problems. SIAM J. Sci. Comput. 34(3), A1707–A1728 (2012). https://doi.org/10.1137/110823390

    MathSciNet  Article  MATH  Google Scholar 

  19. 19.

    Hinze, M., Pinnau, R., Ulbich, M., Ulbrich, S.: Optimization with PDE Constraints. Springer, Berlin (2009)

    Google Scholar 

  20. 20.

    Holler, G., Kunisch, K., Barnard, R.C.: A bilevel approach for parameter learning in inverse problems. Inverse Probl. 34(11), 1–28 (2018). https://doi.org/10.1088/1361-6420/aade77

    MathSciNet  Article  MATH  Google Scholar 

  21. 21.

    Horst, R., Thoai, N.V.: DC programming: overview. J. Optim. Theory Appl. 103(1), 1–43 (1999). https://doi.org/10.1023/A:1021765131316

    MathSciNet  Article  MATH  Google Scholar 

  22. 22.

    Kalashnikov, V., Benita, F., Mehlitz, P.: The natural gas cash-out problem: a bilevel optimal control approach. Math. Probl. Eng. (2015). https://doi.org/10.1155/2015/286083

    MathSciNet  MATH  Article  Google Scholar 

  23. 23.

    Kinderlehrer, D., Stampacchia, G.: An Introduction to Variational Inequalities and Their Applications. Academic Press, New York (1980)

    Google Scholar 

  24. 24.

    Knauer, M., Büskens, C.: Hybrid Solution Methods for Bilevel Optimal Control Problems with Time Dependent Coupling. In: M. Diehl, F. Glineur, E. Jarlebring, W. Michiels (eds.) Recent Advances in Optimization and its Applications in Engineering: The 14th Belgian-French-German Conference on Optimization, pp. 237–246. Springer, Berlin (2010). https://doi.org/10.1007/978-3-642-12598-0_20

  25. 25.

    Lewis, F.L., Vrabie, D., Syrmos, V.L.: Optimal Control. Wiley, Hoboken (2012)

    Google Scholar 

  26. 26.

    Mehlitz, P.: Necessary optimality conditions for a special class of bilevel programming problems with unique lower level solution. Optimization 66(10), 1533–1562 (2017). https://doi.org/10.1080/02331934.2017.1349123

    MathSciNet  Article  MATH  Google Scholar 

  27. 27.

    Mehlitz, P., Wachsmuth, G.: Weak and strong stationarity in generalized bilevel programming and bilevel optimal control. Optimization 65(5), 907–935 (2016). https://doi.org/10.1080/02331934.2015.1122007

    MathSciNet  Article  MATH  Google Scholar 

  28. 28.

    Mombaur, K., Truong, A., Laumond, J.P.: From human to humanoid locomotion—an inverse optimal control approach. Auton. Robots 28(3), 369–383 (2010). https://doi.org/10.1007/s10514-009-9170-7

    Article  Google Scholar 

  29. 29.

    Nožička, F., Guddat, J., Hollatz, H., Bank, B.: Theorie der Linearen Parametrischen Optimierung. Akademie, Berlin (1974)

    Google Scholar 

  30. 30.

    Outrata, J.V.: On the numerical solution of a class of Stackelberg problems. Z. für Oper. Res. 34(4), 255–277 (1990). https://doi.org/10.1007/BF01416737

    MathSciNet  MATH  Article  Google Scholar 

  31. 31.

    Robinson, S.M.: Stability theory for systems of inequalities, part II: differentiable nonlinear systems. SIAM J. Numer. Anal. 13(4), 497–513 (1976). https://doi.org/10.1137/0713043

    MathSciNet  Article  MATH  Google Scholar 

  32. 32.

    Shimizu, K., Ishizuka, Y., Bard, J.F.: Nondifferentiable and Two-Level Mathematical Programming. Kluwer Academic, Dordrecht (1997)

    Google Scholar 

  33. 33.

    Tröltzsch, F.: Optimal Control of Partial Differential Equations. Vieweg, Wiesbaden (2009)

    Google Scholar 

  34. 34.

    Troutman, J.L.: Variational Calculus and Optimal Control. Springer, New York (1996)

    Google Scholar 

  35. 35.

    Ye, J.J.: Necessary conditions for bilevel dynamic optimization problems. SIAM J. Control Optim. 33(4), 1208–1223 (1995). https://doi.org/10.1137/S0363012993249717

    MathSciNet  Article  MATH  Google Scholar 

  36. 36.

    Ye, J.J.: Optimal strategies for bilevel dynamic problems. SIAM J. Control Optim. 35(2), 512–531 (1997). https://doi.org/10.1137/S0363012993256150

    MathSciNet  Article  MATH  Google Scholar 

  37. 37.

    Ye, J.J., Zhu, D.L.: Optimality conditions for bilevel programming problems. Optimization 33(1), 9–27 (1995). https://doi.org/10.1080/02331939508844060

    MathSciNet  Article  MATH  Google Scholar 

  38. 38.

    Zowe, J., Kurcyusz, S.: Regularity and stability for the mathematical programming problem in Banach spaces. Appl. Math. Optim. 5(1), 49–62 (1979). https://doi.org/10.1007/BF01442543

    MathSciNet  Article  MATH  Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Patrick Mehlitz.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This work is supported by the DFG Grant Analysis and Solution Methods for Bilevel Optimal Control Problems (Grant Nos. DE 650/10-1 and WA3636/4-1) within the Priority Program SPP 1962 (Non-smooth and Complementarity-based Distributed Parameter Systems: Simulation and Hierarchical Optimization).

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Dempe, S., Harder, F., Mehlitz, P. et al. Solving inverse optimal control problems via value functions to global optimality. J Glob Optim 74, 297–325 (2019). https://doi.org/10.1007/s10898-019-00758-1

Download citation

Keywords

  • Bilevel optimal control
  • Global optimization
  • Inverse optimal control
  • Optimality conditions
  • Solution algorithm

Mathematics Subject Classification

  • 49K20
  • 49M20
  • 49N45
  • 90C26
  • 90C48