Computational Optimization and Applications

, Volume 68, Issue 2, pp 289–315 | Cite as

Mitigating the curse of dimensionality: sparse grid characteristics method for optimal feedback control and HJB equations

  • Wei KangEmail author
  • Lucas C. Wilcox


We address finding the semi-global solutions to optimal feedback control and the Hamilton–Jacobi–Bellman (HJB) equation. Using the solution of an HJB equation, a feedback optimal control law can be implemented in real-time with minimum computational load. However, except for systems with two or three state variables, using traditional techniques for numerically finding a semi-global solution to an HJB equation for general nonlinear systems is infeasible due to the curse of dimensionality. Here we present a new computational method for finding feedback optimal control and solving HJB equations which is able to mitigate the curse of dimensionality. We do not discretize the HJB equation directly, instead we introduce a sparse grid in the state space and use the Pontryagin’s maximum principle to derive a set of necessary conditions in the form of a boundary value problem, also known as the characteristic equations, for each grid point. Using this approach, the method is spatially causality free, which enjoys the advantage of perfect parallelism on a sparse grid. Compared with dense grids, a sparse grid has a significantly reduced size which is feasible for systems with relatively high dimensions, such as the 6-D system shown in the examples. Once the solution obtained at each grid point, high-order accurate polynomial interpolation is used to approximate the feedback control at arbitrary points. We prove an upper bound for the approximation error and approximate it numerically. This sparse grid characteristics method is demonstrated with three examples of rigid body attitude control using momentum wheels.


Optimal feedback control Hamilton–Jacobi–Bellman equation Sparse grid Method of characteristics Rigid body attitude control 



This work was supported in part by AFOSR, NRL, DARPA, and CRUSER of Naval Postgraduate School. Thanks to Carlos F. Borges for his time discussing terminology and the naming of the method. Thanks to Arthur J. Krener for his insights on HJB solutions. Thanks to Lars Grüne for his comments on the parallel computation of reachable set [18].


  1. 1.
    Al’brekht, E.G.: On the optimal stabilization of nonlinear systems. J. Appl. Math. Mech. 25(5), 1254–1266 (1961). doi: 10.1016/0021-8928(61)90005-3 MathSciNetCrossRefzbMATHGoogle Scholar
  2. 2.
    Assellaou, M., Bokanowski, O., Desilles, A., Zidani, H.: A Hamilton–Jacobi–Bellman approach for the optimal control of an abort landing problem. In: Proceedings of the 55th IEEE Conference on Decision and Control (2016). doi: 10.1109/CDC.2016.7798815
  3. 3.
    Bardi, M., Capuzzo-Dolcetta, I.: Optimal control and viscosity solutions of Hamilton–Jacobi–Bellman equations. Systems and Control: Incollection and Applications. Birkhäuser, Boston (1997). doi: 10.1007/978-0-8176-4755-1_1 CrossRefzbMATHGoogle Scholar
  4. 4.
    Barron, E.N., Jensen, R.: The Pontryagin maximum principle from dynamic programming and viscosity solutions to first-order partial differential equations. Trans. Am. Math. Soc. 298(2), 635–635 (1986). doi: 10.1090/s0002-9947-1986-0860384-4 MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    Barthelmann, V., Novak, E., Ritter, K.: High dimensional polynomial interpolation on sparse grids. Adv. Comput. Math. 12(4), 273–288 (2000). doi: 10.1023/A:1018977404843 MathSciNetCrossRefzbMATHGoogle Scholar
  6. 6.
    Bokanowski, O., Garcke, J., Griebel, M., Klompmaker, I.: An adaptive sparse grid semi-Lagrangian scheme for first order Hamilton–Jacobi–Bellman equations. J. Sci. Comput. 55(3), 575–605 (2013). doi: 10.1007/s10915-012-9648-x MathSciNetCrossRefzbMATHGoogle Scholar
  7. 7.
    Bungartz, H.J., Griebel, M.: Sparse grids. Acta Numer. 13, 147–269 (2004). doi: 10.1017/S0962492904000182 MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Cacace, S., Cristiani, E., Falcone, M., Picarelli, A.: A patchy dynamic programming scheme for a class of Hamilton–Jacobi–Bellman equations. SIAM J. Sci. Comput. 34(5), A2625–A2649 (2012). doi: 10.1137/110841576 MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    Crouch, P.E.: Spacecraft attitude control and stabilization: applications of geometric control theory to rigid body models. IEEE Trans. Autom. Control 29(4), 321–331 (1984). doi: 10.1109/TAC.1984.1103519 CrossRefzbMATHGoogle Scholar
  10. 10.
    Delvos, F.-J.: \(d\)-variate Boolean interpolation. J. Approx. Theory 34(2), 99–114 (1982). doi: 10.1016/0021-9045(82)90085-5 MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Diebel, J.: Representing attitude: Euler angles, unit quaternions, and rotation vectors (2006).
  12. 12.
    Falcone, M., Ferretti, R.: Semi-Lagrangian Approximation Schemes for Linear and Hamilton–Jacobi Equations. Society for Industrial and Applied Mathematics, Philadelphia (2013). doi: 10.1137/1.9781611973051 CrossRefzbMATHGoogle Scholar
  13. 13.
    Fleming, W.H., Soner, H.M.: Controlled Markov Processes and Viscosity Solutions. Stochastic Modelling and Applied Probability, vol. 25. Springer, New York (2006). doi: 10.1007/0-387-31071-1_1 zbMATHGoogle Scholar
  14. 14.
    Garcke, J.: Sparse grids in a nutshell. In: Garcke J., Griebel M. (eds) Sparse Grids and Applications, Lecture Notes in Computational Science and Engineering, vol 88. Springer, Berlin, Heidelberg (2012)Google Scholar
  15. 15.
    Gui, H., Jin, L., Xu, S.: Attitude maneuver control of a two-wheeled spacecraft with bounded wheel speeds. Acta Astronaut. 88, 98–107 (2013). doi: 10.1016/j.actaastro.2013.03.006 CrossRefGoogle Scholar
  16. 16.
    Hesthaven, J.S., Gottlieb, S., Gottlieb, D.: Spectral Methods for Time-Dependent Problems. Cambridge Monographs on Applied and Computational Mathematics. Cambridge University Press, Cambridge (2007). doi: 10.1017/CBO9780511618352 zbMATHGoogle Scholar
  17. 17.
    Horri, N.M., Hodgart, S.: Attitude stabilization of an underactuated satellite using two wheels. In: Aerospace Conference, 2003. Proceedings. 2003 IEEE, vol. 6, pp. 6_2629–6_2635 (2003). doi: 10.1109/AERO.2003.1235188
  18. 18.
    Jahn, T.U.: A feasibility problem approach for reachable set approximation. Ph.D. thesis, University of Bayreuth, Bayreuth, Germany (2015).
  19. 19.
    Kang, W., De, P.K., Isidori, A.: Flight control in a windshear via nonlinear \(h_\infty \) methods. In: Proceedings of the 31st IEEE Conference on Decision and Control, pp. 1135–1142 (1992). doi: 10.1109/CDC.1992.371539
  20. 20.
    Kang, W., Wilcox, L.: A causality free computational method for HJB equations with application to rigid body satellites. In: AIAA Guidance, Navigation, and Control Conference, AIAA 2015-2009. American Institute of Aeronautics and Astronautics (2015). doi: 10.2514/6.2015-2009
  21. 21.
    Kierzenka, J., Shampine, L.F.: A BVP solver that controls residual and error. J. Numer. Anal. Ind. Appl. Math. 3(1–2), 27–41 (2008).
  22. 22.
    Kim, S., Kim, Y.: Spin-axis stabilization of a rigid spacecraft using two reaction wheels. J. Guid. Control Dyn. 24(5), 1046–1049 (2001). doi: 10.2514/2.4818 CrossRefGoogle Scholar
  23. 23.
    Klimke, A.: Uncertainty modeling using fuzzy arithmetic and sparse grids. Ph.D. thesis, Universität Stuttgart, Shaker Verlag, Aachen (2006)Google Scholar
  24. 24.
    Krishnan, H., McClamroch, N.H., Reyhanoglu, M.: Attitude stabilization of a rigid spacecraft using two momentum wheel actuators. J. Guid. Control Dyn. 18(2), 256–263 (1995). doi: 10.2514/3.21378 CrossRefzbMATHGoogle Scholar
  25. 25.
    Lukes, D.L.: Optimal regulation of nonlinear dynamical systems. SIAM J. Control 7(1), 75–100 (1969). doi: 10.1137/0307007 MathSciNetCrossRefzbMATHGoogle Scholar
  26. 26.
    Navasca, C., Krener, A.J.: Patchy solutions of Hamilton–Jacobi–Bellman partial differential equations. In: Chiuso, A., Pinzoni, S., Ferrante, A. (eds.) Modeling, Estimation and Control. Lecture Notes in Control and Information Sciences, vol. 364, pp. 251–270. Springer, Berlin (2007). doi: 10.1007/978-3-540-73570-0_20 Google Scholar
  27. 27.
    Pontryagin, L.S.: The Mathematical Theory of Optimal Processes. Interscience, New York (1962)zbMATHGoogle Scholar
  28. 28.
    Smolyak, S.A.: Quadrature and interpolation formulas for tensor products of certain classes of functions. Dokl. Akad. Nauk SSSR 4, 240–243 (1963)zbMATHGoogle Scholar
  29. 29.
    Terui, F., Kawamoto, S., Fujiwara, T., Noda, A., Sako, N., Nakasuka, S.: Target tracking attitude maneuver of a bias momentum micro satellite using two wheels. In: AIAA Guidance, Navigation, and Control Conference and Exhibit, AIAA 2000-4114. American Institute of Aeronautics and Astronautics (2000). doi: 10.2514/6.2000-4144
  30. 30.
    Tsiotras, P., Luo, J.: Control of underactuated spacecraft with bounded inputs. Automatica 36(8), 1153–1169 (2000). doi: 10.1016/S0005-1098(00)00025-X MathSciNetCrossRefzbMATHGoogle Scholar
  31. 31.
    Wasilkowski, G.W., Wozniakowski, H.: Explicit cost bounds of algorithms for multivariate tensor product problems. J. Complex. 11(1), 1–56 (1995). doi: 10.1006/jcom.1995.1001 MathSciNetCrossRefzbMATHGoogle Scholar
  32. 32.
    Xiu, D., Hesthaven, J.S.: High-order collocation methods for differential equations with random inputs. SIAM J. Sci. Comput. 27(3), 1118–1139 (2005). doi: 10.1137/040615201 MathSciNetCrossRefzbMATHGoogle Scholar
  33. 33.
    Zenger, C.: Sparse grids. In: Hackbusch, W., Gamm, G.f.A.M.u.M. (eds.) Parallel Algorithms for Partial Differential Equations—Proceedings of the Sixth GAMM-Seminar-Kiel, 19–21 Jan 1990, vol. 31. Vieweg, Braunschweig (1991)Google Scholar

Copyright information

© Springer Science+Business Media New York (outside the USA) 2017

Authors and Affiliations

  1. 1.Department of Applied MathematicsNaval Postgraduate SchoolMontereyUSA

Personalised recommendations