Intelligent Fuzzy Q-Learning Control of Humanoid Robots

  • Meng Joo Er
  • Yi Zhou
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3498)


In this paper, a design methodology for enhancing the stability of humanoid robots is presented. Fuzzy Q-Learning (FQL) is applied to improve the Zero Moment Point (ZMP) performance by intelligent control of the trunk of a humanoid robot. With the fuzzy evaluation signal and the neural networks of FQL, biped robots are dynamically balanced in situations of uneven terrains. At the mean time, expert knowledge can be embedded to reduce the training time. Simulation studies show that the FQL controller is able to improve the stability as the actual ZMP trajectories become close to the ideal case.


Humanoid Robot Biped Robot Zero Moment Point Negative Small Positive Medium 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Huang, Q., Yokoi, K., Kajita, S., Kaneko, S., Rrai, H., Koyachi, N., Tanie, K.: Planning Walking Patterns for A Biped Robot. IEEE Trans. Robotics and Automation 17, 280–289 (2001)CrossRefGoogle Scholar
  2. 2.
    Vukobratovic, M.: Zero-Moment Point-Thirty Five Yeas of Its Life. International Jounal of Humanoid Robotics 1, 157–173 (2001)CrossRefGoogle Scholar
  3. 3.
    Juang, J.G.: Fuzzy Neural Network Approaches for Robotic Gait Synthesis. IEEE Trans. on Systems, Man and Cybernetics, Part B: Cybernetics 30, 594–601 (2000)CrossRefGoogle Scholar
  4. 4.
    Ogino, M., Katoh, Y., Aono, M., Asada, M., Hosoda, K.: Reinforcement Learning of Humanoid Rhythmic Walking Parameters Based on Visual Information. Advanced Robotics 18, 677–697 (2004)CrossRefGoogle Scholar
  5. 5.
    Zhou, C.: Robot Learning with GA-based Fuzzy Reinforcement Learning Agents. Information Science 145, 45–68 (2002)zbMATHCrossRefGoogle Scholar
  6. 6.
    Watkins, C.J.C.H.: Learning from Delayed Rewards.PhD Thesis, Cambridge University (1989)Google Scholar
  7. 7.
    Jang, J.S.R.: ANFIS: Adaptive-network-based Fuzzy Inference System. IEEE Trans. System, Man and Cybernetics 23, 665–684 (1993)CrossRefGoogle Scholar
  8. 8.
    Wang, L.X.: A Course in Fuzzy Systems and Control. Prentice-Hall, New Jersey (1997)zbMATHGoogle Scholar
  9. 9.
    Er, M.J., Deng, C.: Online Tuning of Fuzzy Inference Systems Using Dynamic Fuzzy QLearning. IEEE Trans on Systems, Man and Cybernetics, Part B 34, 1478–1489 (2004)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  • Meng Joo Er
    • 1
  • Yi Zhou
    • 1
  1. 1.Intelligent Systems CenterSingapore

Personalised recommendations