Skip to main content

Learning Energy-Efficient Trotting for Legged Robots

  • 580 Accesses

Part of the Lecture Notes in Networks and Systems book series (LNNS,volume 530)


Quadrupedal locomotion skills are challenging to develop. In recent years, Deep Reinforcement Learning (DRL) promises to automate the development of locomotion controllers and map sensory observations to low-level actions. However, legged locomotion still is a challenging task for DRL algorithms, especially when energy efficiency is taken into consideration. In this paper, we propose a DRL scheme for efficient trotting applied on Laelaps II quadruped in MuJoCo. First, an accurate model of the robot is created by revealing the necessary parameters to be imported in the simulation, while special focus is given to the quadruped’s drivetrain. Concerning, the reward function and the action space, we investigate the best way to integrate in the reward, the terms necessary to minimize the Cost of Transport (CoT) while maintaining a trotting locomotion pattern. Last, we present how our solution increased the energy efficiency for a simple task of trotting on level terrain similar to the treadmill-robot environment at the Control Systems Lab [1] of NTUA.


  • Legged robots
  • Learning locomotion
  • Energy efficiency
  • Deep reinforcement learning

This is a preview of subscription content, access via your institution.

Buying options

USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-031-15226-9_21
  • Chapter length: 12 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
USD   219.00
Price excludes VAT (USA)
  • ISBN: 978-3-031-15226-9
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   279.99
Price excludes VAT (USA)
Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.
Fig. 5.
Fig. 6.
Fig. 7.
Fig. 8.


  1. 1.

  2. 2.


  1. Accessed 15 Apr 2022

  2. Peng, X.B., Coumans, E., Zhang, T., Lee, T.-W.E., Tan, J., Levine, S.: Learning agile robotic locomotion skills by imitating animals. Robot. Sci. Syst., July 2020

    Google Scholar 

  3. Haarnoja, T., Ha, S., Zhou, A., Tan, J., Tucker, G., Levine, S.: Learning to walk via deep reinforcement learning. Robot. Sci. Syst. (2019)

    Google Scholar 

  4. Xie, Z., Clary, P., Dao, J., Morais, P., Hurst, J., van de Panne, M.: Iterative reinforcement learning based design of dynamic locomotion skills for cassie. arXiv, preprint arXiv:1903.09537 (2019)

  5. Lee, J., Hwangbo, J., Wellhausen, L., Koltun, V., Hutter, M.: Learning quadrupedal locomotion over challenging terrain. Sci. Robot. 5(47), eabc5986 (2020)

    CrossRef  Google Scholar 

  6. Mastrogeorgiou, A., ElBahrawy, Y., Kecskeméthy, A., Papadopoulos, E.: Slope handling for quadruped robots using deep reinforcement learning and trajectory planning. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2020), Las Vegas, NV, USA, 25–29 October 2020 (2020)

    Google Scholar 

  7. Iscen, A., Yu, G., Escontrela, A., Jain, D., Tan, J., Caluwaerts, K.: Learning agile locomotion skills with a mentor. In: 2021 International Conference on Robotics and Automation (ICRA) (2021)

    Google Scholar 

  8. Tan, J., et al.: Sim-to-real: learning agile locomotion for quadruped robots (2018). arXiv, doi:

  9. Koutsoukis, K., Papadopoulos, E.: On the effect of robotic leg design on energy efficiency. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA 2021), Xi’an, China, 30 May–5 June 2021 (2021)

    Google Scholar 

  10. Iscen, A., et al.: Policies modulating trajectory generators (2019). arXiv, doi:

  11. Machairas, K., Papadopoulos, E.: An active compliance controller for quadruped trotting. In: 24th Mediterranean Conference on Control and Automation (MED 2016), Athens, Greece, 21–24 June 2016 (2016)

    Google Scholar 

  12. Papatheodorou, A.: Design & implementation of a real-time distributed EtherCAT-based motion control system for a multi-DoF quadruped robot. Diploma thesis, CSL, NTUA, Athens (2021).

  13. Todorov, E., Erez, T., Tassa, Y.: MuJoCo: a physics engine for model-based control. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, vol. 2012, pp. 5026–5033 (2012).

  14. Collins, J., Chand, S., Vanderkop, A., Howard, D.: A review of physics simulators for robotic applications. IEEE Access 9, 51416–51431 (2021).

    CrossRef  Google Scholar 

  15. Virgala, I., Kelemen, M.: Experimental friction identification of a DC motor. Int. J. Mech. Appl. 3(1), 26–30 (2013)

    Google Scholar 

  16. Machairas, K., Papadopoulos, E.: An analytical study on trotting at constant velocity and height. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Madrid, Spain, October 2018, pp. 3279–3284 (2018)

    Google Scholar 

  17. Brockman, G., et al.: OpenAI Gym (2016)

    Google Scholar 

  18. Raffin, A., Hill, A., Gleave, A., Kanervisto, A., Ernestus, M., Dormann, N.: Stable-Baselines3: reliable reinforcement learning implementations. J. Mach. Learn. Res. 22, 1–8 (2021).

  19. Mastrogeorgiou, A., ElBahrawy, Y., Machairas, K., Kecskeméthy, A., Papadopoulos, E.: Evaluating deep reinforcement learning algorithms for quadrupedal slope handling. In: 23rd International Conference on Climbing and Walking Robots and the Support Technologies for Mobile Machines (CLAWAR 2020), Moscow, Russian Federation, 24–26 August 2020 (2020)

    Google Scholar 

  20. Haarnoja, T., Zhou, A., Abbeel, P., Levine, S.: Soft actor-critic: off-policy maximum entropy deep reinforcement learning with a stochastic actor. In: International Conference on Machine Learning (2018)

    Google Scholar 

  21. Horwitz, A.: A version of Simpson’s rule for multiple integrals. J. Comput. Appl. Math. 134, 1–11 (2001)

    CrossRef  MathSciNet  Google Scholar 


  23. Gangapurwala, S., Geisert, M., Orsolino, R., Fallon, M., Havoutis, I.: RLOC: terrain-aware legged locomotion using reinforcement learning and optimal control (2020). arXiv, doi:


  25. Accessed 15 Apr 2022

Download references


The authors wish to thank K. Machairas for co-developing the quadruped robot Laelaps II. This work was supported by the Hellenic Foundation for Research and Innovation (H.F.R.I.) under the “First Call for H.F.R.I. Research Projects to support Faculty members and Researchers and the procurement of high-cost research equipment grant” (Project Number: 2182).

Author information

Authors and Affiliations


Corresponding author

Correspondence to Athanasios Mastrogeorgiou .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

Mastrogeorgiou, A., Papatheodorou, A., Koutsoukis, K., Papadopoulos, E. (2023). Learning Energy-Efficient Trotting for Legged Robots. In: Cascalho, J.M., Tokhi, M.O., Silva, M.F., Mendes, A., Goher, K., Funk, M. (eds) Robotics in Natural Settings. CLAWAR 2022. Lecture Notes in Networks and Systems, vol 530. Springer, Cham.

Download citation