Computational Optimization and Applications

, Volume 68, Issue 2, pp 289–315

# Mitigating the curse of dimensionality: sparse grid characteristics method for optimal feedback control and HJB equations

Article

## Abstract

We address finding the semi-global solutions to optimal feedback control and the Hamilton–Jacobi–Bellman (HJB) equation. Using the solution of an HJB equation, a feedback optimal control law can be implemented in real-time with minimum computational load. However, except for systems with two or three state variables, using traditional techniques for numerically finding a semi-global solution to an HJB equation for general nonlinear systems is infeasible due to the curse of dimensionality. Here we present a new computational method for finding feedback optimal control and solving HJB equations which is able to mitigate the curse of dimensionality. We do not discretize the HJB equation directly, instead we introduce a sparse grid in the state space and use the Pontryagin’s maximum principle to derive a set of necessary conditions in the form of a boundary value problem, also known as the characteristic equations, for each grid point. Using this approach, the method is spatially causality free, which enjoys the advantage of perfect parallelism on a sparse grid. Compared with dense grids, a sparse grid has a significantly reduced size which is feasible for systems with relatively high dimensions, such as the 6-D system shown in the examples. Once the solution obtained at each grid point, high-order accurate polynomial interpolation is used to approximate the feedback control at arbitrary points. We prove an upper bound for the approximation error and approximate it numerically. This sparse grid characteristics method is demonstrated with three examples of rigid body attitude control using momentum wheels.

## Keywords

Optimal feedback control Hamilton–Jacobi–Bellman equation Sparse grid Method of characteristics Rigid body attitude control

## References

1. 1.
Al’brekht, E.G.: On the optimal stabilization of nonlinear systems. J. Appl. Math. Mech. 25(5), 1254–1266 (1961). doi:
2. 2.
Assellaou, M., Bokanowski, O., Desilles, A., Zidani, H.: A Hamilton–Jacobi–Bellman approach for the optimal control of an abort landing problem. In: Proceedings of the 55th IEEE Conference on Decision and Control (2016). doi:
3. 3.
Bardi, M., Capuzzo-Dolcetta, I.: Optimal control and viscosity solutions of Hamilton–Jacobi–Bellman equations. Systems and Control: Incollection and Applications. Birkhäuser, Boston (1997). doi:
4. 4.
Barron, E.N., Jensen, R.: The Pontryagin maximum principle from dynamic programming and viscosity solutions to first-order partial differential equations. Trans. Am. Math. Soc. 298(2), 635–635 (1986). doi:
5. 5.
Barthelmann, V., Novak, E., Ritter, K.: High dimensional polynomial interpolation on sparse grids. Adv. Comput. Math. 12(4), 273–288 (2000). doi:
6. 6.
Bokanowski, O., Garcke, J., Griebel, M., Klompmaker, I.: An adaptive sparse grid semi-Lagrangian scheme for first order Hamilton–Jacobi–Bellman equations. J. Sci. Comput. 55(3), 575–605 (2013). doi:
7. 7.
Bungartz, H.J., Griebel, M.: Sparse grids. Acta Numer. 13, 147–269 (2004). doi:
8. 8.
Cacace, S., Cristiani, E., Falcone, M., Picarelli, A.: A patchy dynamic programming scheme for a class of Hamilton–Jacobi–Bellman equations. SIAM J. Sci. Comput. 34(5), A2625–A2649 (2012). doi:
9. 9.
Crouch, P.E.: Spacecraft attitude control and stabilization: applications of geometric control theory to rigid body models. IEEE Trans. Autom. Control 29(4), 321–331 (1984). doi:
10. 10.
Delvos, F.-J.: $$d$$-variate Boolean interpolation. J. Approx. Theory 34(2), 99–114 (1982). doi:
11. 11.
Diebel, J.: Representing attitude: Euler angles, unit quaternions, and rotation vectors (2006). http://www.astro.rug.nl/software/kapteyn//_downloads/attitude.pdf
12. 12.
Falcone, M., Ferretti, R.: Semi-Lagrangian Approximation Schemes for Linear and Hamilton–Jacobi Equations. Society for Industrial and Applied Mathematics, Philadelphia (2013). doi:
13. 13.
Fleming, W.H., Soner, H.M.: Controlled Markov Processes and Viscosity Solutions. Stochastic Modelling and Applied Probability, vol. 25. Springer, New York (2006). doi:
14. 14.
Garcke, J.: Sparse grids in a nutshell. In: Garcke J., Griebel M. (eds) Sparse Grids and Applications, Lecture Notes in Computational Science and Engineering, vol 88. Springer, Berlin, Heidelberg (2012)Google Scholar
15. 15.
Gui, H., Jin, L., Xu, S.: Attitude maneuver control of a two-wheeled spacecraft with bounded wheel speeds. Acta Astronaut. 88, 98–107 (2013). doi:
16. 16.
Hesthaven, J.S., Gottlieb, S., Gottlieb, D.: Spectral Methods for Time-Dependent Problems. Cambridge Monographs on Applied and Computational Mathematics. Cambridge University Press, Cambridge (2007). doi:
17. 17.
Horri, N.M., Hodgart, S.: Attitude stabilization of an underactuated satellite using two wheels. In: Aerospace Conference, 2003. Proceedings. 2003 IEEE, vol. 6, pp. 6_2629–6_2635 (2003). doi:
18. 18.
Jahn, T.U.: A feasibility problem approach for reachable set approximation. Ph.D. thesis, University of Bayreuth, Bayreuth, Germany (2015). https://epub.uni-bayreuth.de/2087/
19. 19.
Kang, W., De, P.K., Isidori, A.: Flight control in a windshear via nonlinear $$h_\infty$$ methods. In: Proceedings of the 31st IEEE Conference on Decision and Control, pp. 1135–1142 (1992). doi:
20. 20.
Kang, W., Wilcox, L.: A causality free computational method for HJB equations with application to rigid body satellites. In: AIAA Guidance, Navigation, and Control Conference, AIAA 2015-2009. American Institute of Aeronautics and Astronautics (2015). doi:
21. 21.
Kierzenka, J., Shampine, L.F.: A BVP solver that controls residual and error. J. Numer. Anal. Ind. Appl. Math. 3(1–2), 27–41 (2008). http://jnaiam.org/index.php?/archives/57-A-BVP-Solver-that-Controls-Residual-and-Error.html
22. 22.
Kim, S., Kim, Y.: Spin-axis stabilization of a rigid spacecraft using two reaction wheels. J. Guid. Control Dyn. 24(5), 1046–1049 (2001). doi:
23. 23.
Klimke, A.: Uncertainty modeling using fuzzy arithmetic and sparse grids. Ph.D. thesis, Universität Stuttgart, Shaker Verlag, Aachen (2006)Google Scholar
24. 24.
Krishnan, H., McClamroch, N.H., Reyhanoglu, M.: Attitude stabilization of a rigid spacecraft using two momentum wheel actuators. J. Guid. Control Dyn. 18(2), 256–263 (1995). doi:
25. 25.
Lukes, D.L.: Optimal regulation of nonlinear dynamical systems. SIAM J. Control 7(1), 75–100 (1969). doi:
26. 26.
Navasca, C., Krener, A.J.: Patchy solutions of Hamilton–Jacobi–Bellman partial differential equations. In: Chiuso, A., Pinzoni, S., Ferrante, A. (eds.) Modeling, Estimation and Control. Lecture Notes in Control and Information Sciences, vol. 364, pp. 251–270. Springer, Berlin (2007). doi: Google Scholar
27. 27.
Pontryagin, L.S.: The Mathematical Theory of Optimal Processes. Interscience, New York (1962)
28. 28.
Smolyak, S.A.: Quadrature and interpolation formulas for tensor products of certain classes of functions. Dokl. Akad. Nauk SSSR 4, 240–243 (1963)
29. 29.
Terui, F., Kawamoto, S., Fujiwara, T., Noda, A., Sako, N., Nakasuka, S.: Target tracking attitude maneuver of a bias momentum micro satellite using two wheels. In: AIAA Guidance, Navigation, and Control Conference and Exhibit, AIAA 2000-4114. American Institute of Aeronautics and Astronautics (2000). doi:
30. 30.
Tsiotras, P., Luo, J.: Control of underactuated spacecraft with bounded inputs. Automatica 36(8), 1153–1169 (2000). doi:
31. 31.
Wasilkowski, G.W., Wozniakowski, H.: Explicit cost bounds of algorithms for multivariate tensor product problems. J. Complex. 11(1), 1–56 (1995). doi:
32. 32.
Xiu, D., Hesthaven, J.S.: High-order collocation methods for differential equations with random inputs. SIAM J. Sci. Comput. 27(3), 1118–1139 (2005). doi:
33. 33.
Zenger, C.: Sparse grids. In: Hackbusch, W., Gamm, G.f.A.M.u.M. (eds.) Parallel Algorithms for Partial Differential Equations—Proceedings of the Sixth GAMM-Seminar-Kiel, 19–21 Jan 1990, vol. 31. Vieweg, Braunschweig (1991)Google Scholar

© Springer Science+Business Media New York (outside the USA) 2017

## Authors and Affiliations

• Wei Kang
• 1
• Lucas C. Wilcox
• 1
1. 1.Department of Applied MathematicsNaval Postgraduate SchoolMontereyUSA