# Algorithm for Overcoming the Curse of Dimensionality For Time-Dependent Non-convex Hamilton–Jacobi Equations Arising From Optimal Control and Differential Games Problems

## Abstract

In this paper we develop a parallel method for solving possibly non-convex time-dependent Hamilton–Jacobi equations arising from optimal control and differential game problems. The subproblems are independent so they can be implemented in an embarrassingly parallel fashion, which usually has an ideal parallel speedup. The algorithm is proposed to overcome the curse of dimensionality (Bellman in Adaptive control processes: a guided tour. Princeton University Press, Princeton, 1961; Dynamic programming. Princeton University Press, Princeton, 1957) when solving HJ PDE . We extend previous work Chow et al. (Algorithm for overcoming the curse of dimensionality for certain non-convex Hamilton–Jacobi equations, Projections and differential games, UCLA CAM report, pp 16–27, 2016) and Darbon and Osher (Algorithms for overcoming the curse of dimensionality for certain Hamilton–Jacobi equations arising in control theory and elsewhere, UCLA CAM report, pp 15–50, 2015) and apply a generalized Hopf formula to solve HJ PDE involving time-dependent and perhaps non-convex Hamiltonians. We suggest a coordinate descent method for the minimization procedure in the Hopf formula. This method is preferable when even the evaluation of the function value itself requires some computational effort, and also when we handle higher dimensional optimization problem. For an integral with respect to time inside the generalized Hopf formula, we suggest using a numerical quadrature rule. Together with our suggestion to perform numerical differentiation to minimize the number of calculation procedures in each iteration step, we are bound to have numerical errors in our computations. These errors can be effectively controlled by choosing an appropriate mesh-size in time and the method does not use a mesh in space. The use of multiple initial guesses is suggested to overcome possibly multiple local extrema in the case when non-convex Hamiltonians are considered. Our method is expected to have wide application in control theory and differential game problems, and elsewhere.

This is a preview of subscription content, log in to check access.

## References

1. 1.

Bellman, R.: Adaptive Control Processes: A Guided Tour. Princeton University Press, Princeton (1961)

2. 2.

Bellman, R.: Dynamic Programming. Princeton University Press, Princeton (1957)

3. 3.

Crandall, M.G., Lions, P.-L.: Some properties of viscosity solutions of Hamilton–Jacobi equations. Trans. AMS 282(2), 487–502 (1984)

4. 4.

Crandall, M.G., Lions, P.-L.: Viscosity solutions of Hamilton-Jacobi equations. Trans. AMS 277(1), 1–42 (1983)

5. 5.

Chow, Y.T., Darbon, J., Osher, S., Yin, W.: Algorithm for overcoming the curse of dimensionality for certain non-convex Hamilton–Jacobi equations, projections and differential games, UCLA CAM report, pp. 16–27 (2016)

6. 6.

Darbon, J., Osher, S.: Algorithms for overcoming the curse of dimensionality for certain Hamilton–Jacobi equations arising in control theory and elsewhere. UCLA CAM report, pp. 15–50 (2015). (preprint)

7. 7.

Davis, D., Yin, W.: A three-operator splitting scheme and its optimization applications. UCLA CAM report, pp. 15–13 (2015). (preprint)

8. 8.

Dijkstra, E.W.: A note on two problems in connexion with graphs. Numer. Math. 1, 269–271 (1959)

9. 9.

Dolcetta, I.C.: Representation of solutions of Hamilton–Jacobi equations. In: Nonlinear Equations: Methods, Models and Applications, pp. 79–90. Birkhauser Basel (2003)

10. 10.

Elishakoff, I. (ed.): Whys and Hows in Uncertainty Modelling. Springer, Wien (1999)

11. 11.

Evans, L.C.: Partial differential equations. In: Graduate Studies in Mathematics, vol. 19. AMS (2010)

12. 12.

Evans, L.C., Souganidis, P.E.: Differential games and representation formulas for solutions of Hamilton–Jacobi–Isaacs equations. Indiana Univ. Math. J. 38, 773–797 (1984)

13. 13.

Friedlander, M.P., Macedo, I., Pong, T.K.: Gauge optimization and duality. SIAM J. Optim. 24(4), 1999–2022 (2014)

14. 14.

Friedman, J., Hastie, T., Hofling, H., Tibshirani, R.: Pathwise coordinate optimization. Ann. Appl. Stat. 1, 302–332 (2007)

15. 15.

Gabay, D., Mercier, B.: A dual algorithm for the solution of nonlinear variational problems via finite element approximation. Comput. Math. Appl. 2(1), 17–40 (1976)

16. 16.

Glowinski, R., Marroco, A.: On the approximation by finite elements of order one, and resolution, penalisation-duality for a class of nonlinear Dirichlet problems. ESAIM Math. Model. Numer. Anal. 9(R2), 41–76 (1975)

17. 17.

Grippo, L., Sciandrone, M.: On the convergence of the block nonlinear Gauss–Seidel method under convex constraints. Oper. Res. Lett. 26(3), 127–136 (2000)

18. 18.

Goldstein, T., Osher, S.: The split Bregman method for L1-regularized problems. SIAM J. Imaging Sci. 2(2), 323–343 (2009)

19. 19.

Hildreth, C.: A quadratic programming procedure. Nav. Res. Logist. Q. 4(1), 79–85 (1957)

20. 20.

Hiriart-Urruty, J.-B., Lemaréchal, C.: Fundamentals of Convex Analysis, Grundlehren Text Editions. Springer, New York (2001)

21. 21.

Hopf, E.: Generalized solutions of nonlinear equations of the first order. J. Math. Mech. 14, 951–973 (1965)

22. 22.

Horowitz, M.B., Damle, A., Burdick, J. W.: Linear Hamilton Jacobi Bellman equations in high dimensions. arXiv:1404.1089 (2014)

23. 23.

Kang, W., Wilcox, L.C.: Mitigating the curse of dimensionality: sparse grid characteristics method for optimal feedback control and HJB equations. arXiv:1507.04769 (2015). (preprint)

24. 24.

Kenney, C.S., Lepnik, R.B.: Integration of the differential matrix Riccati equation. lEEE Trans. Autom. Control AC–30(10), 962–970 (1985)

25. 25.

Li, Y., Osher, S.: Coordinate descent optimization for $$l^1$$ minimization with application to compressed sensing: a greedy algorithm. Inverse Probl. Imaging 3(3), 487–503 (2009)

26. 26.

Luo, Z.-Q., Tseng, P.: On the convergence of the coordinate descent method for convex differentiable minimization. J. Optim. Theory Appl. 72(1), 7–35 (1992)

27. 27.

Mitchell, I.M., Bayen, A.M., Tomlin, C.J.: A time-dependent Hamilton–Jacobi formulation of reachable sets for continuous dynamic games. Automatic control. IEEE Trans. Autom. Control 50(7), 947–957 (2005)

28. 28.

Mitchell, I.M., Tomlin, C.J.: Overapproximating reachable sets by Hamilton–Jacobi projections. J. Sci Comput. 19(1–3), 323–346 (2003)

29. 29.

Moreau, J.J.: Proximit$$\acute{\text{ e }}$$ et dualit$$\acute{\text{ e }}$$ dans un espace hilbertien. Bull. Soc. Math. Fr. 93, 273–299 (1965)

30. 30.

Ortega, J.M., Rheinboldt, W.C.: Iterative Solution of Nonlinear Equations in Several Variables. Academic Press, New York (1970)

31. 31.

Osher, S., Chu, C.-W.: High order essentially non-oscillatory schemes for Hamilton–Jacobi equations. SIAM J. Numer. Anal. 28(4), 907–922 (1991)

32. 32.

Osher, S., Sethian, J.A.: Fronts propagating with curvature dependent speech: algorithms based on Hamilton–Jacobi formulations. J. Comput. Phys. 79(1), 12–49 (1988)

33. 33.

Osher, S., Merriman, B.: The Wulff shape as the asymptotic limit of a growing crystalline interface. Asian J. Math. 1(3), 560–571 (1997)

34. 34.

Rockafellar, R.T.: Convex Analysis, Princeton Landmarks in Mathematics. Princeton University Press, Princeton (1997)

35. 35.

Rublev, I.V.: Generalized Hopf formulas for the nonautonomous Hamilton–Jacobi equation. Comput. Math. Model. 11(4), 391–400 (2000)

36. 36.

Shen, Y., Wen, Z., Zhang, Y.: Augmented lagrangian alternating direction method for matrix separation based on low-rank factorization. Optim. Methods Softw. 29(2), 239–263 (2014)

37. 37.

Sun, D.L., Fevotte, C.: Alternating direction method of multipliers for non-negative matrix factorization with the beta-divergence. In: IEEE International Conference on ICASSP, pp. 6201–6205 (2014)

38. 38.

Tseng, P.: Convergence of a block coordinate descent method for nondifferentiable minimization. J. Optim. Theory Appl. 109(3), 475–494 (2001)

39. 39.

Tsai, Y.H.R., Cheng, L.T., Osher, S., Zhao, H.K.: Fast sweeping algorithms for a class of Hamilton–Jacobi equations. SIAM J. Numer. Anal 41(2), 673–694 (2003)

40. 40.

Tsitsiklis, J.N.: Efficient algorithms for globally optimal trajectories. IEEE Trans. Autom. Control 40(9), 1528–1538 (1995)

41. 41.

Warga, J.: Minimizing certain convex functions. J. Soc. Indus. Appl. Math. 11(3), 588–593 (1963)

42. 42.

Wang, Y., Yin, W., Zeng, J.: Global convergence of ADMM in nonconvex nonsmooth optimization, UCLA CAM report, pp. 15–62 (2015). (preprint)

43. 43.

Wen, Z., Yang, C., Liu, X., Marchesini, S.: Alternating direction methods for classical and ptychographic phase retrieval. Inverse Probl. 28(11), 115010 (2012)

44. 44.

Xu, Y., Yin, W.: A block coordinate descent method for regularized multiconvex optimization with applications to nonnegative tensor factorization and completion. SIAM J. Imaging Sci. 6(3), 1758–1789 (2013)

45. 45.

Xu, Y. Yin, W.: A globally convergent algorithm for nonconvex optimization based on block coordinate update. arXiv preprint arXiv:1410.1386 (2014)

46. 46.

Xu, Y., Yin, W., Wen, Z., Zhang, Y.: An alternating direction algorithm for matrix completion with nonnegative factors. Front. Math. China 7(2), 365–384 (2012)

47. 47.

Yang, L., Pong, T.K., Chen, X.: Alternating direction method of multipliers for nonconvex background/foreground extraction. arXiv:1506.07029 (2015)

48. 48.

Yin, W., Osher, S., Goldfarb, D., Darbon, J.: Bregman iterative algorithms for $$l_1$$ minimization with applications to compressed sensing. SIAM J. Imaging Sci. 1(1), 143–168 (2008)

## Acknowledgements

We sincerely thank Dr. Gary Hewer and his colleagues (China Lake Naval Center) for providing help and guidance in practical optimal control and differential game problems. We also thank Prof. Jianliang Qian for reminding us of the literature [35] and the technical condition for the generalized Hopf formula.

## Author information

Authors

### Corresponding author

Correspondence to Yat Tin Chow.

Research supported by ONR Grant N000141410683, N000141210838, N000141712162, DOE Grant DE-SC00183838, and NSF Grant ECCS-1462398.

## Rights and permissions

Reprints and Permissions

Chow, Y.T., Darbon, J., Osher, S. et al. Algorithm for Overcoming the Curse of Dimensionality For Time-Dependent Non-convex Hamilton–Jacobi Equations Arising From Optimal Control and Differential Games Problems. J Sci Comput 73, 617–643 (2017). https://doi.org/10.1007/s10915-017-0436-5

• Revised:

• Accepted:

• Published:

• Issue Date:

### Keywords

• Hamilton–Jacobi equations
• Hopf–Lax formula
• Optimal control
• Differential game