Algorithm for Overcoming the Curse of Dimensionality For Time-Dependent Non-convex Hamilton–Jacobi Equations Arising From Optimal Control and Differential Games Problems

Abstract

In this paper we develop a parallel method for solving possibly non-convex time-dependent Hamilton–Jacobi equations arising from optimal control and differential game problems. The subproblems are independent so they can be implemented in an embarrassingly parallel fashion, which usually has an ideal parallel speedup. The algorithm is proposed to overcome the curse of dimensionality (Bellman in Adaptive control processes: a guided tour. Princeton University Press, Princeton, 1961; Dynamic programming. Princeton University Press, Princeton, 1957) when solving HJ PDE . We extend previous work Chow et al. (Algorithm for overcoming the curse of dimensionality for certain non-convex Hamilton–Jacobi equations, Projections and differential games, UCLA CAM report, pp 16–27, 2016) and Darbon and Osher (Algorithms for overcoming the curse of dimensionality for certain Hamilton–Jacobi equations arising in control theory and elsewhere, UCLA CAM report, pp 15–50, 2015) and apply a generalized Hopf formula to solve HJ PDE involving time-dependent and perhaps non-convex Hamiltonians. We suggest a coordinate descent method for the minimization procedure in the Hopf formula. This method is preferable when even the evaluation of the function value itself requires some computational effort, and also when we handle higher dimensional optimization problem. For an integral with respect to time inside the generalized Hopf formula, we suggest using a numerical quadrature rule. Together with our suggestion to perform numerical differentiation to minimize the number of calculation procedures in each iteration step, we are bound to have numerical errors in our computations. These errors can be effectively controlled by choosing an appropriate mesh-size in time and the method does not use a mesh in space. The use of multiple initial guesses is suggested to overcome possibly multiple local extrema in the case when non-convex Hamiltonians are considered. Our method is expected to have wide application in control theory and differential game problems, and elsewhere.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

References

  1. 1.

    Bellman, R.: Adaptive Control Processes: A Guided Tour. Princeton University Press, Princeton (1961)

    Google Scholar 

  2. 2.

    Bellman, R.: Dynamic Programming. Princeton University Press, Princeton (1957)

    Google Scholar 

  3. 3.

    Crandall, M.G., Lions, P.-L.: Some properties of viscosity solutions of Hamilton–Jacobi equations. Trans. AMS 282(2), 487–502 (1984)

    Article  MATH  MathSciNet  Google Scholar 

  4. 4.

    Crandall, M.G., Lions, P.-L.: Viscosity solutions of Hamilton-Jacobi equations. Trans. AMS 277(1), 1–42 (1983)

    Article  MATH  MathSciNet  Google Scholar 

  5. 5.

    Chow, Y.T., Darbon, J., Osher, S., Yin, W.: Algorithm for overcoming the curse of dimensionality for certain non-convex Hamilton–Jacobi equations, projections and differential games, UCLA CAM report, pp. 16–27 (2016)

  6. 6.

    Darbon, J., Osher, S.: Algorithms for overcoming the curse of dimensionality for certain Hamilton–Jacobi equations arising in control theory and elsewhere. UCLA CAM report, pp. 15–50 (2015). (preprint)

  7. 7.

    Davis, D., Yin, W.: A three-operator splitting scheme and its optimization applications. UCLA CAM report, pp. 15–13 (2015). (preprint)

  8. 8.

    Dijkstra, E.W.: A note on two problems in connexion with graphs. Numer. Math. 1, 269–271 (1959)

    Article  MATH  MathSciNet  Google Scholar 

  9. 9.

    Dolcetta, I.C.: Representation of solutions of Hamilton–Jacobi equations. In: Nonlinear Equations: Methods, Models and Applications, pp. 79–90. Birkhauser Basel (2003)

  10. 10.

    Elishakoff, I. (ed.): Whys and Hows in Uncertainty Modelling. Springer, Wien (1999)

    Google Scholar 

  11. 11.

    Evans, L.C.: Partial differential equations. In: Graduate Studies in Mathematics, vol. 19. AMS (2010)

  12. 12.

    Evans, L.C., Souganidis, P.E.: Differential games and representation formulas for solutions of Hamilton–Jacobi–Isaacs equations. Indiana Univ. Math. J. 38, 773–797 (1984)

    Article  MATH  MathSciNet  Google Scholar 

  13. 13.

    Friedlander, M.P., Macedo, I., Pong, T.K.: Gauge optimization and duality. SIAM J. Optim. 24(4), 1999–2022 (2014)

    Article  MATH  MathSciNet  Google Scholar 

  14. 14.

    Friedman, J., Hastie, T., Hofling, H., Tibshirani, R.: Pathwise coordinate optimization. Ann. Appl. Stat. 1, 302–332 (2007)

    Article  MATH  MathSciNet  Google Scholar 

  15. 15.

    Gabay, D., Mercier, B.: A dual algorithm for the solution of nonlinear variational problems via finite element approximation. Comput. Math. Appl. 2(1), 17–40 (1976)

    Article  MATH  Google Scholar 

  16. 16.

    Glowinski, R., Marroco, A.: On the approximation by finite elements of order one, and resolution, penalisation-duality for a class of nonlinear Dirichlet problems. ESAIM Math. Model. Numer. Anal. 9(R2), 41–76 (1975)

    MATH  Google Scholar 

  17. 17.

    Grippo, L., Sciandrone, M.: On the convergence of the block nonlinear Gauss–Seidel method under convex constraints. Oper. Res. Lett. 26(3), 127–136 (2000)

    Article  MATH  MathSciNet  Google Scholar 

  18. 18.

    Goldstein, T., Osher, S.: The split Bregman method for L1-regularized problems. SIAM J. Imaging Sci. 2(2), 323–343 (2009)

    Article  MATH  MathSciNet  Google Scholar 

  19. 19.

    Hildreth, C.: A quadratic programming procedure. Nav. Res. Logist. Q. 4(1), 79–85 (1957)

    Article  MathSciNet  Google Scholar 

  20. 20.

    Hiriart-Urruty, J.-B., Lemaréchal, C.: Fundamentals of Convex Analysis, Grundlehren Text Editions. Springer, New York (2001)

    Google Scholar 

  21. 21.

    Hopf, E.: Generalized solutions of nonlinear equations of the first order. J. Math. Mech. 14, 951–973 (1965)

    MATH  MathSciNet  Google Scholar 

  22. 22.

    Horowitz, M.B., Damle, A., Burdick, J. W.: Linear Hamilton Jacobi Bellman equations in high dimensions. arXiv:1404.1089 (2014)

  23. 23.

    Kang, W., Wilcox, L.C.: Mitigating the curse of dimensionality: sparse grid characteristics method for optimal feedback control and HJB equations. arXiv:1507.04769 (2015). (preprint)

  24. 24.

    Kenney, C.S., Lepnik, R.B.: Integration of the differential matrix Riccati equation. lEEE Trans. Autom. Control AC–30(10), 962–970 (1985)

    Article  MATH  MathSciNet  Google Scholar 

  25. 25.

    Li, Y., Osher, S.: Coordinate descent optimization for \(l^1\) minimization with application to compressed sensing: a greedy algorithm. Inverse Probl. Imaging 3(3), 487–503 (2009)

    Article  MATH  MathSciNet  Google Scholar 

  26. 26.

    Luo, Z.-Q., Tseng, P.: On the convergence of the coordinate descent method for convex differentiable minimization. J. Optim. Theory Appl. 72(1), 7–35 (1992)

    Article  MATH  MathSciNet  Google Scholar 

  27. 27.

    Mitchell, I.M., Bayen, A.M., Tomlin, C.J.: A time-dependent Hamilton–Jacobi formulation of reachable sets for continuous dynamic games. Automatic control. IEEE Trans. Autom. Control 50(7), 947–957 (2005)

    Article  MATH  Google Scholar 

  28. 28.

    Mitchell, I.M., Tomlin, C.J.: Overapproximating reachable sets by Hamilton–Jacobi projections. J. Sci Comput. 19(1–3), 323–346 (2003)

    Article  MATH  MathSciNet  Google Scholar 

  29. 29.

    Moreau, J.J.: Proximit\(\acute{\text{ e }}\) et dualit\(\acute{\text{ e }}\) dans un espace hilbertien. Bull. Soc. Math. Fr. 93, 273–299 (1965)

    Article  Google Scholar 

  30. 30.

    Ortega, J.M., Rheinboldt, W.C.: Iterative Solution of Nonlinear Equations in Several Variables. Academic Press, New York (1970)

    Google Scholar 

  31. 31.

    Osher, S., Chu, C.-W.: High order essentially non-oscillatory schemes for Hamilton–Jacobi equations. SIAM J. Numer. Anal. 28(4), 907–922 (1991)

    Article  MATH  MathSciNet  Google Scholar 

  32. 32.

    Osher, S., Sethian, J.A.: Fronts propagating with curvature dependent speech: algorithms based on Hamilton–Jacobi formulations. J. Comput. Phys. 79(1), 12–49 (1988)

    Article  MATH  MathSciNet  Google Scholar 

  33. 33.

    Osher, S., Merriman, B.: The Wulff shape as the asymptotic limit of a growing crystalline interface. Asian J. Math. 1(3), 560–571 (1997)

    Article  MATH  MathSciNet  Google Scholar 

  34. 34.

    Rockafellar, R.T.: Convex Analysis, Princeton Landmarks in Mathematics. Princeton University Press, Princeton (1997)

    Google Scholar 

  35. 35.

    Rublev, I.V.: Generalized Hopf formulas for the nonautonomous Hamilton–Jacobi equation. Comput. Math. Model. 11(4), 391–400 (2000)

    MATH  MathSciNet  Google Scholar 

  36. 36.

    Shen, Y., Wen, Z., Zhang, Y.: Augmented lagrangian alternating direction method for matrix separation based on low-rank factorization. Optim. Methods Softw. 29(2), 239–263 (2014)

    Article  MATH  MathSciNet  Google Scholar 

  37. 37.

    Sun, D.L., Fevotte, C.: Alternating direction method of multipliers for non-negative matrix factorization with the beta-divergence. In: IEEE International Conference on ICASSP, pp. 6201–6205 (2014)

  38. 38.

    Tseng, P.: Convergence of a block coordinate descent method for nondifferentiable minimization. J. Optim. Theory Appl. 109(3), 475–494 (2001)

    Article  MATH  MathSciNet  Google Scholar 

  39. 39.

    Tsai, Y.H.R., Cheng, L.T., Osher, S., Zhao, H.K.: Fast sweeping algorithms for a class of Hamilton–Jacobi equations. SIAM J. Numer. Anal 41(2), 673–694 (2003)

    Article  MATH  MathSciNet  Google Scholar 

  40. 40.

    Tsitsiklis, J.N.: Efficient algorithms for globally optimal trajectories. IEEE Trans. Autom. Control 40(9), 1528–1538 (1995)

    Article  MATH  MathSciNet  Google Scholar 

  41. 41.

    Warga, J.: Minimizing certain convex functions. J. Soc. Indus. Appl. Math. 11(3), 588–593 (1963)

    Article  MATH  MathSciNet  Google Scholar 

  42. 42.

    Wang, Y., Yin, W., Zeng, J.: Global convergence of ADMM in nonconvex nonsmooth optimization, UCLA CAM report, pp. 15–62 (2015). (preprint)

  43. 43.

    Wen, Z., Yang, C., Liu, X., Marchesini, S.: Alternating direction methods for classical and ptychographic phase retrieval. Inverse Probl. 28(11), 115010 (2012)

    Article  MATH  MathSciNet  Google Scholar 

  44. 44.

    Xu, Y., Yin, W.: A block coordinate descent method for regularized multiconvex optimization with applications to nonnegative tensor factorization and completion. SIAM J. Imaging Sci. 6(3), 1758–1789 (2013)

    Article  MATH  MathSciNet  Google Scholar 

  45. 45.

    Xu, Y. Yin, W.: A globally convergent algorithm for nonconvex optimization based on block coordinate update. arXiv preprint arXiv:1410.1386 (2014)

  46. 46.

    Xu, Y., Yin, W., Wen, Z., Zhang, Y.: An alternating direction algorithm for matrix completion with nonnegative factors. Front. Math. China 7(2), 365–384 (2012)

    Article  MATH  MathSciNet  Google Scholar 

  47. 47.

    Yang, L., Pong, T.K., Chen, X.: Alternating direction method of multipliers for nonconvex background/foreground extraction. arXiv:1506.07029 (2015)

  48. 48.

    Yin, W., Osher, S., Goldfarb, D., Darbon, J.: Bregman iterative algorithms for \(l_1\) minimization with applications to compressed sensing. SIAM J. Imaging Sci. 1(1), 143–168 (2008)

    Article  MATH  MathSciNet  Google Scholar 

Download references

Acknowledgements

We sincerely thank Dr. Gary Hewer and his colleagues (China Lake Naval Center) for providing help and guidance in practical optimal control and differential game problems. We also thank Prof. Jianliang Qian for reminding us of the literature [35] and the technical condition for the generalized Hopf formula.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Yat Tin Chow.

Additional information

Research supported by ONR Grant N000141410683, N000141210838, N000141712162, DOE Grant DE-SC00183838, and NSF Grant ECCS-1462398.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Chow, Y.T., Darbon, J., Osher, S. et al. Algorithm for Overcoming the Curse of Dimensionality For Time-Dependent Non-convex Hamilton–Jacobi Equations Arising From Optimal Control and Differential Games Problems. J Sci Comput 73, 617–643 (2017). https://doi.org/10.1007/s10915-017-0436-5

Download citation

Keywords

  • Hamilton–Jacobi equations
  • Hopf–Lax formula
  • Optimal control
  • Differential game