Skip to main content
Book cover

Climbing and Walking Robots Conference

CLAWAR 2022: Robotics in Natural Settings pp 204–215Cite as

Learning Energy-Efficient Trotting for Legged Robots

  • 580 Accesses

Part of the Lecture Notes in Networks and Systems book series (LNNS,volume 530)

Abstract

Quadrupedal locomotion skills are challenging to develop. In recent years, Deep Reinforcement Learning (DRL) promises to automate the development of locomotion controllers and map sensory observations to low-level actions. However, legged locomotion still is a challenging task for DRL algorithms, especially when energy efficiency is taken into consideration. In this paper, we propose a DRL scheme for efficient trotting applied on Laelaps II quadruped in MuJoCo. First, an accurate model of the robot is created by revealing the necessary parameters to be imported in the simulation, while special focus is given to the quadruped’s drivetrain. Concerning, the reward function and the action space, we investigate the best way to integrate in the reward, the terms necessary to minimize the Cost of Transport (CoT) while maintaining a trotting locomotion pattern. Last, we present how our solution increased the energy efficiency for a simple task of trotting on level terrain similar to the treadmill-robot environment at the Control Systems Lab [1] of NTUA.

Keywords

  • Legged robots
  • Learning locomotion
  • Energy efficiency
  • Deep reinforcement learning

This is a preview of subscription content, access via your institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-031-15226-9_21
  • Chapter length: 12 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   219.00
Price excludes VAT (USA)
  • ISBN: 978-3-031-15226-9
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   279.99
Price excludes VAT (USA)
Learn about institutional subscriptions
Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.
Fig. 5.
Fig. 6.
Fig. 7.
Fig. 8.

Notes

  1. 1.

    https://www.fluke.com/en/product/electrical-testing/digital-multimeters/fluke-289.

  2. 2.

    https://docs.broadcom.com/doc/AV02-0993EN.

References

  1. https://nereus.mech.ntua.gr/legged/laelaps-2-2/. Accessed 15 Apr 2022

  2. Peng, X.B., Coumans, E., Zhang, T., Lee, T.-W.E., Tan, J., Levine, S.: Learning agile robotic locomotion skills by imitating animals. Robot. Sci. Syst., July 2020

    Google Scholar 

  3. Haarnoja, T., Ha, S., Zhou, A., Tan, J., Tucker, G., Levine, S.: Learning to walk via deep reinforcement learning. Robot. Sci. Syst. (2019)

    Google Scholar 

  4. Xie, Z., Clary, P., Dao, J., Morais, P., Hurst, J., van de Panne, M.: Iterative reinforcement learning based design of dynamic locomotion skills for cassie. arXiv, preprint arXiv:1903.09537 (2019)

  5. Lee, J., Hwangbo, J., Wellhausen, L., Koltun, V., Hutter, M.: Learning quadrupedal locomotion over challenging terrain. Sci. Robot. 5(47), eabc5986 (2020)

    CrossRef  Google Scholar 

  6. Mastrogeorgiou, A., ElBahrawy, Y., Kecskeméthy, A., Papadopoulos, E.: Slope handling for quadruped robots using deep reinforcement learning and trajectory planning. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2020), Las Vegas, NV, USA, 25–29 October 2020 (2020)

    Google Scholar 

  7. Iscen, A., Yu, G., Escontrela, A., Jain, D., Tan, J., Caluwaerts, K.: Learning agile locomotion skills with a mentor. In: 2021 International Conference on Robotics and Automation (ICRA) (2021)

    Google Scholar 

  8. Tan, J., et al.: Sim-to-real: learning agile locomotion for quadruped robots (2018). arXiv, doi: https://doi.org/10.48550/arxiv.1804.10332

  9. Koutsoukis, K., Papadopoulos, E.: On the effect of robotic leg design on energy efficiency. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA 2021), Xi’an, China, 30 May–5 June 2021 (2021)

    Google Scholar 

  10. Iscen, A., et al.: Policies modulating trajectory generators (2019). arXiv, doi: https://doi.org/10.48550/arxiv.1910.02812

  11. Machairas, K., Papadopoulos, E.: An active compliance controller for quadruped trotting. In: 24th Mediterranean Conference on Control and Automation (MED 2016), Athens, Greece, 21–24 June 2016 (2016)

    Google Scholar 

  12. Papatheodorou, A.: Design & implementation of a real-time distributed EtherCAT-based motion control system for a multi-DoF quadruped robot. Diploma thesis, CSL, NTUA, Athens (2021). https://dspace.lib.ntua.gr/xmlui/handle/123456789/53738

  13. Todorov, E., Erez, T., Tassa, Y.: MuJoCo: a physics engine for model-based control. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, vol. 2012, pp. 5026–5033 (2012). https://doi.org/10.1109/IROS.2012.6386109

  14. Collins, J., Chand, S., Vanderkop, A., Howard, D.: A review of physics simulators for robotic applications. IEEE Access 9, 51416–51431 (2021). https://doi.org/10.1109/ACCESS.2021.3068769

    CrossRef  Google Scholar 

  15. Virgala, I., Kelemen, M.: Experimental friction identification of a DC motor. Int. J. Mech. Appl. 3(1), 26–30 (2013)

    Google Scholar 

  16. Machairas, K., Papadopoulos, E.: An analytical study on trotting at constant velocity and height. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Madrid, Spain, October 2018, pp. 3279–3284 (2018)

    Google Scholar 

  17. Brockman, G., et al.: OpenAI Gym (2016)

    Google Scholar 

  18. Raffin, A., Hill, A., Gleave, A., Kanervisto, A., Ernestus, M., Dormann, N.: Stable-Baselines3: reliable reinforcement learning implementations. J. Mach. Learn. Res. 22, 1–8 (2021). https://jmlr.org/papers/v22/20-1364.html

  19. Mastrogeorgiou, A., ElBahrawy, Y., Machairas, K., Kecskeméthy, A., Papadopoulos, E.: Evaluating deep reinforcement learning algorithms for quadrupedal slope handling. In: 23rd International Conference on Climbing and Walking Robots and the Support Technologies for Mobile Machines (CLAWAR 2020), Moscow, Russian Federation, 24–26 August 2020 (2020)

    Google Scholar 

  20. Haarnoja, T., Zhou, A., Abbeel, P., Levine, S.: Soft actor-critic: off-policy maximum entropy deep reinforcement learning with a stochastic actor. In: International Conference on Machine Learning (2018)

    Google Scholar 

  21. Horwitz, A.: A version of Simpson’s rule for multiple integrals. J. Comput. Appl. Math. 134, 1–11 (2001)

    CrossRef  MathSciNet  Google Scholar 

  22. https://www.xsens.com/products/mti-100-series

  23. Gangapurwala, S., Geisert, M., Orsolino, R., Fallon, M., Havoutis, I.: RLOC: terrain-aware legged locomotion using reinforcement learning and optimal control (2020). arXiv, doi: https://doi.org/10.48550/arxiv.2012.03094

  24. https://ocw.mit.edu/courses/2-161-signal-processing-continuous-and-discrete-fall-2008

  25. https://bitbucket.org/csl_legged/laelaps_rl_mujoco/. Accessed 15 Apr 2022

Download references

Acknowledgements

The authors wish to thank K. Machairas for co-developing the quadruped robot Laelaps II. This work was supported by the Hellenic Foundation for Research and Innovation (H.F.R.I.) under the “First Call for H.F.R.I. Research Projects to support Faculty members and Researchers and the procurement of high-cost research equipment grant” (Project Number: 2182).

Author information

Authors and Affiliations

  1. School of Mechanical Engineering, National Technical University of Athens, Athens, 15780, Greece

    Athanasios Mastrogeorgiou, Aristotelis Papatheodorou, Konstantinos Koutsoukis & Evangelos Papadopoulos

Authors
  1. Athanasios Mastrogeorgiou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Aristotelis Papatheodorou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Konstantinos Koutsoukis
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Evangelos Papadopoulos
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Athanasios Mastrogeorgiou .

Editor information

Editors and Affiliations

  1. Faculty of Sciences and Technology, University of the Azores, Ponta Delgada, Portugal

    José M. Cascalho

  2. School of Engineering, London South Bank University, London, UK

    Mohammad Osman Tokhi

  3. School of Engineering, Polytechnic Institute of Porto, Porto, Portugal

    Manuel F. Silva

  4. Faculty of Sciences and Technology, University of the Azores, Ponta Delgada, Portugal

    Armando Mendes

  5. Mechanical, Materials and Manufacturing Engineering (M3), Faculty of Engineering, University of Nottingham, Nottingham, UK

    Khaled Goher

  6. Faculty of Sciences and Technology, University of the Azores, Ponta Delgada, Portugal

    Matthias Funk

Rights and permissions

Reprints and Permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

Mastrogeorgiou, A., Papatheodorou, A., Koutsoukis, K., Papadopoulos, E. (2023). Learning Energy-Efficient Trotting for Legged Robots. In: Cascalho, J.M., Tokhi, M.O., Silva, M.F., Mendes, A., Goher, K., Funk, M. (eds) Robotics in Natural Settings. CLAWAR 2022. Lecture Notes in Networks and Systems, vol 530. Springer, Cham. https://doi.org/10.1007/978-3-031-15226-9_21

Download citation