A data-driven indirect method for nonlinear optimal control

Abstract

Nonlinear optimal control problems are challenging to solve due to the prevalence of local minima that prevent convergence and/or optimality. This paper describes nearest-neighbors optimal control (NNOC), a data-driven framework for nonlinear optimal control using indirect methods. It determines initial guesses for new problems with the help of precomputed solutions to similar problems, retrieved using k-nearest neighbors. A sensitivity analysis technique is introduced to linearly approximate the variation of solutions between new and precomputed problems based on their variation of parameters. Experiments show that NNOC can obtain the global optimal solution orders of magnitude faster than standard random restart methods, and sensitivity analysis can further reduce the solving time almost by half. Examples are shown on optimal control problems in vehicle control and agile satellite reorientation demonstrating that global optima can be determined with more than 99% reliability within time at the order of 10–100 milliseconds.

This is a preview of subscription content, access via your institution.

References

  1. [1]

    Betts, J. T. Survey of numerical methods for trajectory optimization. Journal of Guidance, Control, and Dynamics, 1998, 21(2): 193–207.

    Article  Google Scholar 

  2. [2]

    Bryson Jr, A. E., Ho Y. C. Applied Optimal Control: Optimization, Estimation and Control. CRC Press, 1975.

    Google Scholar 

  3. [3]

    Jiang, F. H., Baoyin, H. X., Li, J. F. Practical techniques for low-thrust trajectory optimization with homotopic approach. Journal of Guidance, Control, and Dynamics, 2012, 35(1): 245–258.

    Article  Google Scholar 

  4. [4]

    Jetchev, N., Toussaint, M. Fast motion planning from experience: trajectory prediction for speeding up movement generation. Autonomous Robots, 2013, 34(1–2): 111–127.

    Article  Google Scholar 

  5. [5]

    Hauser, K. Learning the problem-optimum map: analysis and application to global optimization in robotics. IEEE Transactions on Robotics, 2017, 33(1): 141–152.

    Article  Google Scholar 

  6. [6]

    Bertrand, R., Epenoy, R. New smoothing techniques for solving bang-bang optimal control problems—Numerical results and statistical interpretation. Optimal Control Applications and Methods, 2002, 23(4): 171–197.

    MathSciNet  Article  Google Scholar 

  7. [7]

    Russell, R. P. Primer vector theory applied to global low-thrust trade studies. Journal of Guidance, Control, and Dynamics, 2007, 30(2): 460–472.

    Article  Google Scholar 

  8. [8]

    Tang, G., Jiang, F. H., Li, J. F. Fuel-optimal low-thrust trajectory optimization using indirect method and successive convex programming. IEEE Transactions on Aerospace and Electronic Systems, 2018, 54(4): 2053–2066.

    Article  Google Scholar 

  9. [9]

    Jiang, F. H., Tang, G., Li, J. F. Improving low-thrust trajectory optimization by adjoint estimation with shape-based path. Journal of Guidance, Control, and Dynamics, 2017, 40(12): 3282–3289.

    Article  Google Scholar 

  10. [10]

    Cassioli, A., Di Lorenzo, D., Locatelli, M., Schoen, F., Sciandrone, M. Machine learning for global optimization. Computational Optimization and Applications, 2012, 51(1): 279–303.

    MathSciNet  Article  Google Scholar 

  11. [11]

    Pan, J., Chen, Z., Abbeel, P. Predicting initialization effectiveness for trajectory optimization. In: Proceedings of 2014 IEEE International Conference on Robotics and Automation, 2014, 5183–5190.

    Google Scholar 

  12. [12]

    Bohg, J., Morales, A., Asfour, T., Kragic, D. Data-driven grasp synthesis—A survey. IEEE Transactions on Robotics, 2014, 30(2): 289–309.

    Article  Google Scholar 

  13. [13]

    Lampariello, R., Nguyen-Tuong, D., Castellini, C., Hirzinger, G., Peters, J. Trajectory planning for optimal robot catching in real-time. In: Proceedings of 2011 IEEE International Conference on Robotics and Automation, 2011, 3719–3726.

    Google Scholar 

  14. [14]

    Sutton, R. S., Barto, A. G. Reinforcement Learning: An Introduction. MIT Press, 1998.

    Google Scholar 

  15. [15]

    Sánchez-Sánchez, C., Izzo, D. Real-time optimal control via Deep Neural Networks: study on landing problems. Journal of Guidance, Control, and Dynamics, 2018, 41(5): 1122–1135.

    Article  Google Scholar 

  16. [16]

    Tang, G., Sun, W. D., Hauser, K. Learning trajectories for real-time optimal control of quadrotors. In: Proceedings of 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2018, 3620–3625.

    Google Scholar 

  17. [17]

    Tang, G., Hauser, K. Discontinuity-sensitive optimal control learning by mixture of experts. arXiv preprint arXiv:1803.02493, 2018.

    Google Scholar 

  18. [18]

    Bemporad, A., Morari, M., Dua, V., Pistikopoulos, E. N. The explicit solution of model predictive control via multiparametric quadratic programming. In: Proceedings of 2000 American Control Conference, 2000, 872–876.

    Google Scholar 

  19. [19]

    Furfaro, R., Bloise, I., Orlandelli, M., Di Lizia, P., Topputo, F., Linares, R. Deep learning for autonomous lunar landing. In: Proceedings of 2018 AAS/AIAA Astrodynamics Specialist Conference, 2018, 1–22.

    Google Scholar 

  20. [20]

    Ampatzis, C., Izzo, D. Machine learning techniques for approximation of objective functions in trajectory optimisation. In: Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI) 2009, Workshop on Artificial Intelligence in Space, 2009, 1–6.

    Google Scholar 

  21. [21]

    Mereta, A., Izzo, D., Wittig, A. Machine learning of optimal low-thrust transfers between near-earth objects. In: Proceedings of the 12th International Conference on Hybrid Artificial Intelligence Systems, 2017, 543–553.

    Google Scholar 

  22. [22]

    Izzo, D., Sprague, C., Tailor, D. Machine learning and evolutionary techniques in interplanetary trajectory design. arXiv preprint arXiv:1802.00180, 2018.

    Google Scholar 

  23. [23]

    Tang, G., Hauser, K. A data-driven indirect method for nonlinear optimal control. In: Proceedings of 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2017, 4854–4861.

    Google Scholar 

  24. [24]

    Moré, J. J., Garbow, B. S., Hillstrom, K. E. User guide for MINPACK-1. Argonne National Laboratory Report ANL-80-74, Argonne National Laboratory, 1980.

    Google Scholar 

  25. [25]

    Maurer, H., Augustin, D. Sensitivity analysis and real-time control of parametric optimal control problems using boundary value methods. Online Optimization of Large Scale Systems, 2001, 17–55.

    Google Scholar 

  26. [26]

    Xie, Z. M., Liu, C. K., Hauser, K. K. Differential dynamic programming with nonlinear constraints. In: Proceedings of 2017 IEEE International Conference on Robotics and Automation, 2017, 695–702.

    Google Scholar 

  27. [27]

    Ritz, R., Hehn, M., Lupashin, S., D’Andrea, R. Quadrocopter performance benchmarking using optimal control. In: Proceedings of 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2011, 5179–5186.

    Google Scholar 

  28. [28]

    Tomić T., Maier, M., Haddadin, S. Learning quadrotor maneuvers from optimal control and generalizing in real-time. In: Proceedings of 2014 IEEE International Conference on Robotics and Automation, 2014, 1747–1754.

    Google Scholar 

  29. [29]

    Tang, G., Jiang, F. H. Capture of near-Earth objects with low-thrust propulsion and invariant manifolds. Astrophysics and Space Science, 2016, 361(1): 10.

    MathSciNet  Article  Google Scholar 

  30. [30]

    Schaub, H., Junkins, J. L. Analytical Mechanics of Space Systems, 2nd edn. AIAA Education Series, 2009.

    Google Scholar 

  31. [31]

    Li, J., Xi, X. N. Time-optimal reorientation of the rigid spacecraft using a pseudospectral method integrated homotopic approach. Optimal Control Applications and Methods, 2015, 36(6): 889–918.

    MathSciNet  Article  Google Scholar 

  32. [32]

    Bai, X. L., Junkins, J. L. New results for time-optimal three-axis reorientation of a rigid spacecraft. Journal of Guidance, Control, and Dynamics, 2009, 32(4): 1071–1076.

    Article  Google Scholar 

  33. [33]

    Yershova, A., Jain, S., Lavalle, S. M., Mitchell, J. C. Generating uniform incremental grids on SO(3) using the Hopf fibration. The International Journal of Robotics Research, 2010, 29(7): 801–812.

    Article  Google Scholar 

Download references

Acknowledgements

This work was partially supported by NSF (Grant No. IIS-1816540).

Author information

Affiliations

Authors

Corresponding author

Correspondence to Gao Tang.

Additional information

Gao Tang is a Ph.D. student in the Department of Mechanical Engineering and Materials Science at Duke University. He received his B.S. and M.S. degrees in the Department of Aerospace Engineering at Tsinghua University. He is interested in trajectory optimization with an ambitious goal to achieve them in real time.

Kris Hauser is an associate professor at Duke University with joint appointments at the Departments of Electrical & Computer Engineering and Mechanical Engineering and Materials Science. He received his Ph.D. degree in computer science from Stanford University in 2008, B.S. degrees in computer science and mathematics from UC Berkeley in 2003, and was a postdoc at UC Berkeley. He joined the faculty of Indiana University from 2009 to 2014, where he started the Intelligent Motion Lab, and began his current position at Duke in 2014. He is a recipient of a Stanford Graduate Fellowship, Siebel Scholar Fellowship, Best Paper Award at IEEE Humanoids 2015, and an NSF CAREER Award.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Tang, G., Hauser, K. A data-driven indirect method for nonlinear optimal control. Astrodyn 3, 345–359 (2019). https://doi.org/10.1007/s42064-019-0051-3

Download citation

Keywords

  • data-driven approach
  • indirect method
  • optimal control
  • sensitivity analysis