The Study on Optimal Gait for Five-Legged Robot with Reinforcement Learning

  • Adnan Rachmat Anom Besari
  • Ruzaidi Zamri
  • Anton Satria Prabuwono
  • Son Kuswadi
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5928)


The research of legged robot was rapidly developed. It can be seen from recent ideas about new systems of robot movement that take ideas from nature, called biology inspired. This type of robot begins replacing wheeled robot with various functions and interesting maneuvers ability. However, designers should decide how many legs are required to realize the ideas. One of the ideas that are rarely developed is odd number of legs. This research focused on five legs robot that inspired from starfish. To realize the intelligent system in robot that does not depend on the model, this research used reinforcement learning algorithm to find the optimal gait when robot is walking. In order to achieve this goal, trial and error have been used to provide learning through an interaction between robot and environment based on a policy of reward and punishment. The algorithm is successfully implemented to get the optimal gait on a five-legged robot.


Five-legged robot gait optimizing reinforcement learning 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Wang, Z., Ding, X., Rovetta, A.: Structure Design and Locomotion Analysis of a Novel Robot for Lunar Exploration. In: 12th IFToMM World Congress, Besançon, France, June18-21 (2007)Google Scholar
  2. 2.
    Inagaki, K.: Gait study for hexapod walking with disabled leg. In: Proceedings of the 1997 IEEE/RSJ International Conference on Intelligent Robot and Systems, 1997. IROS 1997, September 7-11, vol. 1, pp. 408–413 (1997)Google Scholar
  3. 3.
    Inagaki, K., Kobayashi, H.: Adaptive wave gait for hexapod synchronized walking. In: Proceedings IEEE International Conference on Robotics and Automation, May 8-13, vol. 2, pp. 1326–1331 (1994)Google Scholar
  4. 4.
    Makoto, T., Koji, Y., Satoshi, E.: Studies on Forward Motion of Five Legs Robot (Nippon Kikai Gakkai Robotikusu, Mekatoronikusu Koenkai Koen Ronbunshu). Journal Code: L0318B 2005, 2P1-S-065 (2005)Google Scholar
  5. 5.
    Kimura, H., Yamashita, T., Kobayashi, S.: Reinforcement Learning of Walking Behavior for a Four-Legged Robot. In: Proceedings of the 40th IEEE, Conference on Decision and Control, Orlando, Florida USA (December 2001)Google Scholar
  6. 6.
    Lewis, M.A., Bekey, G.A.: Gait Adaptation in a Quadruped Robot. Journal Autonomous Robot 12(3), 301–312 (2002)MATHCrossRefGoogle Scholar
  7. 7.
    Chernova, S., Veloso, M.: An Evolutionary Approach to Gait Learning For Four-Legged Robot. In: Proceedings of International Conference on Intelligent Robot and Systems 2004 (IROS 2004), Sendai, Japan (September 2004)Google Scholar
  8. 8.
    Kohl, N., Stone, P.: Policy Gradient Reinforcement Learning for Fast Quadruped Locomotion. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), New Orleans, LA, May 2004, pp. 2619–2624 (2004)Google Scholar
  9. 9.
    Zennir, Y., Couturier, P., Bétemps, M.: Distributed Reinforcement Learning of Six-Legged Robot to Walk. In: The Fourth International Conference on Control and Automation (ICCA 2003), Montreal, Canada, June 10-12 (2003)Google Scholar
  10. 10.
    Zennir, Y., Couturier, P.: Multiactor approach and hexapod robot learning. In: Proceedings 2005 IEEE International Symposium on Computational Intelligence in Journal of Robotics and Automation, Espoo, Finland, June 27-30, pp. 665–671 (2005)Google Scholar
  11. 11.
    Barfoot, T.D., Earon, E.J.P., D’Eleuterio, G.M.T.: Experiments in learning distributed control for a hexapod robot. Journal of Robotics and Autonomous Systems 54, 864–872 (2006)CrossRefGoogle Scholar
  12. 12.
    Haynes, G.C., Rizzi, A.A.: Gait Regulation and Feedback on a Robotic Climbing Hexapod. In: Proceedings of Robotics: Science and Systems (August 2006)Google Scholar
  13. 13.
    Erden, M.S., Leblebicioglu, K.: Free gait generation with reinforcement learning for a six-legged robot. Robotics and Autonomous Systems 56, 199–212 (2008)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Adnan Rachmat Anom Besari
    • 1
  • Ruzaidi Zamri
    • 1
  • Anton Satria Prabuwono
    • 2
  • Son Kuswadi
    • 3
  1. 1.Universiti Teknikal Malaysia MelakaAyer KerohMalaysia
  2. 2.Universiti Kebangsaan MalaysiaMalaysia
  3. 3.Electronic Engineering Polytechnic Institute of Surabaya (EEPIS)Institut Teknologi Sepuluh NopemberEast JavaIndonesia

Personalised recommendations