Skip to main content
SpringerLink
Log in

A control Hamiltonian-preserving discretisation for optimal control

  • Published:
Multibody System Dynamics Aims and scope Submit manuscript

Abstract

Optimal control theory allows finding the optimal input of a mechanical system modelled as an initial value problem. The resulting minimisation problem may be solved with known direct and indirect methods. We propose time discretisations for both methods, direct midpoint (DMP) and indirect midpoint (IMP) algorithms, which despite their similarities, result in different convergence orders for the adjoint (or co-state) variables. We additionally propose a third time-integration scheme, Indirect Hamiltonian-preserving (IHP) algorithm, which preserves the control Hamiltonian, an integral of the analytical Euler–Lagrange equations of the optimal control problem.

We test the resulting algorithms to linear and nonlinear problems with and without dissipative forces: a propelled falling mass subjected to gravity and a drag force, an elastic inverted pendulum, and the locomotion of a worm-like organism on a frictional substrate. To improve the convergence of the solution process of the discretised equations in nonlinear problems, we also propose a computational simple suboptimal initial guess and apply a forward–backward sweep method, which computes each set of variables (state, adjoint and control) in a staggered manner. We demonstrate in our examples their practical advantage for computing optimal solutions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Box 1
Box 2
Box 3
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Bayón, L., Otero, J., Suárez, P., Tasis, C.: New developments in the application of optimal control theory to therapeutic protocols. Math. Biosci. 272, 34–43 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  2. Benzi, M., Golub, G., Liesen, J.: Numerical solution of saddle point problems. Acta Numer. 14, 1–137 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  3. Betsch, P., Becker, C.: Conservation of generalized momentum maps in mechanical optimal control problems with symmetry. Int. J. Numer. Methods Eng. 111, 144–155 (2017)

    Article  MathSciNet  Google Scholar 

  4. Betts, J.: Practical Methods for Optimal Control and Estimation Using Nonlinear Programming, 2nd edn. SIAM, Philadelphia, USA (2010)

    Book  MATH  Google Scholar 

  5. Bijalwan, A., Muñoz, J.J.: On the numerical stability of discretised Optimal Control Problems. IUTAM Bookseries, Springer. Under review. Available at http://arxiv.org/abs/2302.02464

  6. Bottasso, C., Croce, A.: Optimal control of multibody systems using an energy preserving direct transcription method. Multibody Syst. Dyn. 12(4), 17–45 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  7. Bryson, A., Ho, Y.: Applied Optimal Control. Optimization, Estimation and Control. Taylor & Francis, New York, U.S.A. (1975)

    Google Scholar 

  8. Djukić, D.S.: Noether’s theorem for optimum control systems. Int. J. Control 18(3), 667–672 (1973)

    Article  MathSciNet  MATH  Google Scholar 

  9. Flaßkamp, K., Murphey, T.: Structure-preserving local optimal control of mechanical systems. Optim. Control Appl. Methods 40(2), 310–329 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  10. Fletcher, R., Reeves, C.: Function minimization by conjugate gradients. Comput. J. 7(2), 149–154 (1964)

    Article  MathSciNet  MATH  Google Scholar 

  11. Gonzalez, O.: Time integration and discrete Hamiltonian systems. J. Nonlinear Sci. 8, 449–467 (1996)

    Article  MathSciNet  MATH  Google Scholar 

  12. Gray, J., Lissmann, H.: The locomotion of nematodes. J. Exp. Biol. 41(1), 135–154 (1964)

    Article  Google Scholar 

  13. Huang, K.-M., Cosman, P., Schafer, W.R.: Machine vision based detection of omega bends and reversals in C. elegans. J. Neurosci. Methods 158(2), 323–336 (2006)

    Article  Google Scholar 

  14. Hull, D.G.: Optimal Control Theory for Applications. Mechanical Engineering Series. Springer, New York (2003)

    Book  MATH  Google Scholar 

  15. Karrakchou, J., Rachik, M., Mostafa, S.: Optimal control and infectiology: application to an HIV/AIDS model. Appl. Math. Comput. 177(2), 807–818 (2006)

    MathSciNet  MATH  Google Scholar 

  16. Kelley, C.T.: Iterative Methods for Optimization. SIAM, Philadelphia (1999)

    Book  MATH  Google Scholar 

  17. Koch, M., Leyendecker, S.: Energy momentum consistent force formulation for the optimal control of multibody systems. Multibody Syst. Dyn. 29, 381–401 (2013)

    Article  MathSciNet  Google Scholar 

  18. Krieg, M., Stühmer, J., Cueva, J., Fetter, R., Spilker, K., Cremers, D., Shen, K., Dunn, A., Goodman, M.: Genetic defects in ß-spectrin and tau sensitize C. elegans axons to movement-induced damage via torque-tension coupling. eLife 6, e20172 (2017)

    Article  Google Scholar 

  19. Lasdon, L., Mitter, S., Waren, A.: The conjugate gradient method for optimal control problems. IEEE Trans. Autom. Control 12(2), 132–138 (1967)

    Article  MathSciNet  Google Scholar 

  20. Lauß, T., Oberpeilsteiner, S., Steiner, W., Nachbagauer, K.: The discrete adjoint gradient computation for optimization problems in multibody dynamics. J. Comput. Nonlinear Dyn. 12(3), 031016 (2017)

    Article  MATH  Google Scholar 

  21. Leyendecker, S., Ober-Blöbaum, S., Marsden, J.E., Ortiz, M.: Discrete mechanics and optimal control for constrained systems. Optim. Control Appl. Methods 39(6), 505–528 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  22. McAsey, M., Mou, L., Han, W.: Convergence of the forward-backward sweep method in optimal control. Comput. Optim. Appl. 53, 207–226 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  23. Miller, M., Trouvé, A., Younes, L.: Hamiltonian systems and optimal control in computational anatomy: 100 years since D’Arcy Thompson. Ann. Biomed. Eng. 17, 447–509 (2015)

    Article  Google Scholar 

  24. Muñoz, J.J., Condamin, L., Doste, D.: On the net displacement of contact surface centroid in contractile bodies. Mech. Res. Commun. 119, 103809 (2022)

    Article  Google Scholar 

  25. Nachbagauer, K., Oberpeilsteiner, S., Sherif, K., Steiner, W.: The use of the adjoint method for solving typical optimization problems in multibody dynamics. J. Comput. Nonlinear Dyn. 10(6), 061011 (2015)

    Article  Google Scholar 

  26. Nocedal, J., Wright, S.: Numerical Optimization, 2nd edn. Springer, New York (2006)

    MATH  Google Scholar 

  27. Ober-Blöbaum, S., Junge, O., Marsden, J.: Discrete mechanics and optimal control: an analysis. ESAIM Control Optim. Calc. Var. 17(2), 322–352 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  28. Pesch, H., Plail, M.: The cold war and the maximum principle of optimal control. Optimization Stories. Documenta Mathematica (2012)

  29. Pikuliński, M., Malczyk, P.: Adjoint method for optimal control of multibody systems in the Hamiltonian setting. Mech. Mach. Theory 166, 104473 (2021)

    Article  Google Scholar 

  30. Purcell, E.: Live at low Reynolds number. Am. J. Phys. 45(1), 3–11 (1977)

    Article  Google Scholar 

  31. Rao, A.V.: A survey of numerical methods for optimal control. Adv. Astronaut. Sci. 135(1), 497–528 (2009)

    Google Scholar 

  32. Saad, Y.: Iterative Methods for Sparse Linear Systems, 2nd edn. SIAM, Philadelphia (2003)

    Book  MATH  Google Scholar 

  33. Schmitthenner, D., Martin, A.: Comparing system identification techniques for identifying human-like walking controllers. R. Soc. Open Sci. 8(12), 211031 (2021)

    Article  Google Scholar 

  34. Schwind, W., Koditschek, D.: Approximating the stance map of a 2-DOF monoped runner. J. Nonlinear Sci. 10(5), 533–568 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  35. Sharma, H., Patil, M., Woolsey, C.: A review of structure-preserving numerical methods for engineering applications. Comput. Methods Appl. Mech. Eng. 366, 113067 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  36. Sharp, J., Burrage, K., Simpson, M.: Implementation and acceleration of optimal control for systems biology. J. R. Soc. Interface 18, 20210241 (2021)

    Article  Google Scholar 

  37. Simo, J., Tarnow, N.: The discrete energy-momentum method. Conserving algorithms for nonlinear elastodynamics. Z. Angew. Math. Phys. 43, 757–792 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  38. Stengel, R.: Optimal Control and Estimation. Dover, New York, USA (1994)

    MATH  Google Scholar 

  39. Stephens, G., Johnson-Kerner, B., Bialek, W., Ryu, W.: Dimensionality and dynamics in the behavior of C. elegans. PLoS Comput. Biol. 4, e1000028 (2008)

    Article  MathSciNet  Google Scholar 

  40. Sussmann, H., Willems, J.: 300 years of optimal control: from the brachystochrone to the maximum principle. IEEE Control Syst. Mag. 17(3), 32–44 (1997)

    Article  Google Scholar 

  41. Timmerman, P., van der Weele, J.P.: On the rise and fall of a ball with linear or quadratic drag. Am. J. Phys. 67(6), 538–546 (1999)

    Article  Google Scholar 

Download references

Acknowledgements

The authors acknowledge Dr. Michael Krieg and Dr. Ravi Das from the Institute of Photonic Sciences (ICFO, Spain) for their helpful discussions. This work is financially supported by the Spanish Ministry of Science and Innovation under grants CEX2018-000797-S and PID2020-116141GB-I00, and by the Generalitat de Catalunya local government, under grant 2021 SGR 01049.

Author information

Authors and Affiliations

  1. Centre Internacional de Mètodes Numèrics en Enginyeria (CIMNE), Escola d’Enginyeria Barcelona Est (EEBE), Universitat Poltiècnica de Catalunya (UPC), Av. Eduard Maristany 16, 08019, Barcelona, Spain

    Ashutosh Bijalwan

  2. Centre Internacional de Mètodes Numèrics en Enginyeria (CIMNE), Escola d’Enginyeria Barcelona Est (EEBE), Insitut de Matemàtiques de la UPC - BarcelonaTech, Universitat Poltiècnica de Catalunya (UPC), Av. Eduard Maristany 16, 08019, Barcelona, Spain

    José J. Muñoz

Authors
  1. Ashutosh Bijalwan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. José J. Muñoz
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to José J. Muñoz.

Ethics declarations

Competing Interests

The authors declare that they have no conflict of interest.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bijalwan, A., Muñoz, J.J. A control Hamiltonian-preserving discretisation for optimal control. Multibody Syst Dyn 59, 19–43 (2023). https://doi.org/10.1007/s11044-023-09902-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11044-023-09902-y

Keywords

Search

Navigation