Advertisement

Journal of Intelligent & Robotic Systems

, Volume 82, Issue 2, pp 189–205 | Cite as

A Learning Behavior Based Controller for Maintaining Balance in Robotic Locomotion

  • Richard BeranekEmail author
  • Mojtaba Ahmadi
Article
  • 299 Downloads

Abstract

The Behavior Based Locomotion Controller (BBLC) extends the applicability of the behavior based control (BBC) architecture to redundant systems with multiple task-space motions. A set of control behaviors are attributed to each task-space motion individually and a reinforcement learning algorithm is used to select the combination of behaviors which can achieve the control objective. The resulting behavior combination is an emergent control behavior robust to unknown environments due to the added learning capability. Hence, the BBLC is applicable to complex redundant systems operating in unknown environments, where the emergent control behaviors can satisfy higher level control objectives such as balance in locomotion. The balance control problem of two robotic systems, a bipedal robot walker and a mobile manipulator, are used to study the performance of this controller. Results show that the BBLC strategy can generate emergent balancing strategies capable of adapting to new unknown disturbances from the environment, using only a small fixed library of balancing behaviors.

Keywords

Balance control Behavior based control Reinforcement learning Mobile manipulator Biped robot 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Arkin, R.C.: Behavior-Based Robotics. MIT press (1998)Google Scholar
  2. 2.
    Antonelli, G., Arrichiello, F., Chiaverini, S.: The Null-Space-based Behavioral control for autonomous robotic systems. Intel. Serv. Robotics. 1(1), 27–39 (2008)CrossRefGoogle Scholar
  3. 3.
    Arrichiello, F., Chiaverini, S., Indiveri, G., Pedone, P.: The Null-Space-based Behavioral Control for Mobile Robots with Velocity Actuator Saturations. Int. J. Robot. Res. 29(10), 1317–1337 (2010)CrossRefGoogle Scholar
  4. 4.
    Jayasiri, A., Mann, G.K.I., Gosine, R.G.: Behavior coordination of mobile robotics using supervisory control of fuzzy discrete event systems. EEE Trans. Syst. Man Cybern. B 41(5), 1224–38 (2011)CrossRefGoogle Scholar
  5. 5.
    Pettersson, O., Karlsson, L., Saffiotti, A.: Model-free execution monitoring in behavior-based robotics. IEEE Trans. Syst. Man Cybern. B 37, 890–901 (2007)CrossRefGoogle Scholar
  6. 6.
    Nakamura, Y: Advanced Robotics: Redundancy and Optimization. Addison-Wesley (1990)Google Scholar
  7. 7.
    Vukobratovic, M., Juricic, D.: Contribution to the synthesis of biped gait. IEEE Trans. Biomed. Eng. 1, 1–6 (1969)CrossRefGoogle Scholar
  8. 8.
    Sugihara, T, Nakamura, Y.: Whole-body cooperative balancing of humanoid robot using COG Jacobian. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp 2575–2580 (2002)Google Scholar
  9. 9.
    Sugihara, T., Nakamura, Y., Inoue, H.: Real-time humanoid motion generation through ZMP manipulation based on inverted pendulum control. IEEE International Conference on Robotics and Automation, 1404–1409 (2002)Google Scholar
  10. 10.
    Popovic, M.B., Goswami, A., Herr, H.: Ground Reference Points in Legged Locomotion : Definitions, Biological Trajectories and Control Implications. Int. J. Robot. Res. 24(12), 1013–1032 (2005)CrossRefGoogle Scholar
  11. 11.
    Qing, T., Rong, X., Jian, C.: Tip over avoidance control for biped robot. Robotica 27(06), 884–889 (2009)CrossRefGoogle Scholar
  12. 12.
    Pratt, J., Chew, C.-M., Torres, A., Dilworth, P., Pratt, G.: Virtual Model Control : An Intuitive Approach. Int. J. Robot. Res. 129(2) (2001)Google Scholar
  13. 13.
    Hirai, K., Hirose, M., Haikawa, Y., Takenaka, T.: The development of Honda humanoid robot. IEEE International Conference on Robotics and Automation, 1321–1326 (1998)Google Scholar
  14. 14.
    Hirose, M., Ogawa, K.: Honda humanoid robots development. Philosophical transactions. Series A, Mathematical, physical, and engineering sciences 365(1850), 11–9 (2007)CrossRefGoogle Scholar
  15. 15.
    Shih, C.L., Chiou, C.J.: The motion control of a statically stable biped robot on an uneven floor. IEEE Trans. Syst. Man Cybern. B 28(2), 244–9 (1998)CrossRefGoogle Scholar
  16. 16.
    Park, J.H., Kim, E.S.: Foot and body control of biped robots to walk on irregularly protruded uneven surfaces. IEEE Trans. Syst. Man Cybern. B 39(1), 289–97 (2009)CrossRefGoogle Scholar
  17. 17.
    Rebula, J., Canas, F., Pratt, J., Goswami, A.: Learning Capture Points for humanoid push recovery. IEEE-RAS International Conference on Humanoid Robots, 65–72 (2007)Google Scholar
  18. 18.
    Stephens, B.: Humanoid push recovery. IEEE-RAS International Conference on Humanoid Robots, 589–595 (2007)Google Scholar
  19. 19.
    Stephens, B.J., Atkeson, C.G.: Push Recovery by Stepping for Humanoid Robots with Force Controlled Joints. Primus, 52–59 (2010)Google Scholar
  20. 20.
    Bortolami, S.B., DiZio, P., Rabin, E., Lackner, J.R.: Analysis of human postural responses to recoverable falls. Exp. Brain Res. 151(3), 387–404 (2003)CrossRefGoogle Scholar
  21. 21.
    Rebula, J.R., Canas, F., Pratt, J.E., Goswami, A.: Learning Capture Points for Bipedal Push Recovery. IEEE International Conference on Robotics and Automation, 1774–1774 (2008)Google Scholar
  22. 22.
    Ferreira, J.P., Crisostomo, M., Coimbra, A.P.: Sagittal stability PD controllers for a biped robot using a neurofuzzy network and an SVR. Robotica 29(05), 717–731 (2010)CrossRefGoogle Scholar
  23. 23.
    Beranek, R., Fung, H., Ahmadi, M.: A Walking Stability Controller with Disturbance Rejection Based on CMP Criterion and Ground Reaction Force Feedback. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (2011)Google Scholar
  24. 24.
    Goswami, A., Kallem, V.: Rate of change of angular momentum and balance maintenance of biped robots. IEEE International Conference on Robotics and Automation, 3785–3790 (2004)Google Scholar
  25. 25.
    Kajita, S., Kanehiro, F., Kaneko, K., Fujiwara, K., Harada, K., Yokoi, K., Hirukawa, H.: Resolved momentum control: Humanoid motion planning based on the linear and angular momentum. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp 1644–1650 (2003)Google Scholar
  26. 26.
    Popovic, M., Englehart: Angular Momentum Primitives for Human Walking: Biomechanics and Control. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp 1685–1691 (2004)Google Scholar
  27. 27.
    Zaier, R., Nagashima, F.: Motion Pattern Generator and Reflex System for Humanoid Robots. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp 840–845 (2006)Google Scholar
  28. 28.
    Zaier, R., Kanda, S.: Piecewise-Linear Pattern Generator and Reflex System for Humanoid Robots. In: IEEE International Conference on Robotics and Automation, pp 2188–2195 (2007)Google Scholar
  29. 29.
    Zaier, R., Kanda, S.: Adaptive locomotion controller and reflex system for humanoid robots. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp 2492–2497 (2008)Google Scholar
  30. 30.
    Khoukhi, A.: Neural based RSPN multi-agent strategy for biped motion control. Robotica 19(06), 611–612 (2001)CrossRefGoogle Scholar
  31. 31.
    Sugano, S., Huang, Q., Kato, I.: Stability criteria in controlling mobile robotic systems. In: IEEE/RSJ International Conference on Intelligent Robots and Systems Proceedings, vol. 2, pp 832–838 (1993)Google Scholar
  32. 32.
    Lee, B.: Real-time ZMP compensation method using null motion for mobile manipulators. IEEE International Conference on Robotics and Automation 2, 1967–1972 (2002)Google Scholar
  33. 33.
    Dubowsky, S., Vance, E.: Planning mobile manipulator motions considering vehicle dynamic stability constraints, pp 1271–1276 (1989)Google Scholar
  34. 34.
    Hootsmans, N.A.M., Dubowsky, S.: Control of Mobile Manipulators Including Vehicle Dynamic Characteristics. In: Proceedings of the IV Topical Meeting on Robotics and Remote Systems (1991)Google Scholar
  35. 35.
    Huang, Q.: Manipulator Motion Planning for Stabilizing a Mobile-Manipulator. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp 467–472 (1995)Google Scholar
  36. 36.
    Huang, Q., Sugano, S., Tanie, K.: Motion planning for a mobile manipulator considering stability and task constraints. In: IEEE International Conference on Robotics and Automation, vol. 3, pp 2192–2198 (1998)Google Scholar
  37. 37.
    Antoska, V., Jovanović, K., Petrović, V.M., Baṡċarević, N., Stankovski, M.: Balance analysis of the mobile anthropomimetic robot under disturbances-zmp approach. International Journal of Advanced Robotic Systems 10(206) (2013)Google Scholar
  38. 38.
    Rey, D., Papadoupoulos, E.: Online automatic tipover prevention for mobile manipulators. In: IEEE/RSJ International Conference on Intelligent Robot and Systems., vol. 3, pp 1273–1278 (1997)Google Scholar
  39. 39.
    Alipour, K.: Moment-Height Tip-Over Measure for Stability Analysis of Mobile Robotic Systems. In: IEEE/RSJ International Conference on Intelligent Robots and Systems., pp 5546–5551 (2006)Google Scholar
  40. 40.
    Ghaffari, A., Meghdari, A., Naderi, D., Eslami, S.: Tipover Stability Enhancement of Wheeled Mobile Manipulators Using an Adaptive Neuro- Fuzzy Inference Controller System. Eng. Technol., 241–247 (2008)Google Scholar
  41. 41.
    Meghdari, a., Naderi, D., Alam, M.: Neural-network-based observer for real-time tipover estimation. Mechatronics 15(8), 989–1004 (2005)CrossRefGoogle Scholar
  42. 42.
    Papadopoulos, E.G., Rey, D.A.: A New Measure of Tipover Stability Margin for Mobile Manipulators. In: IEEE International Conference on Robotics and Automation (1996)Google Scholar
  43. 43.
    Roan, P.R., Burmeister, A., Rahimi, A., Holz, K., Hooper, D.: Real-World Validation of Three Tipover Algorithms for Mobile Robots. In: IEEE International Conference on Robotics and Automation (2010)Google Scholar
  44. 44.
    Hohn, O., Gerth, W.: Probabilistic Balance Monitoring for Bipedal Robots. Int. J. Robot. Res. 28(2), 245–256 (2009)CrossRefGoogle Scholar
  45. 45.
    Kalyanakrishnan, S., Goswami, A.: Learning To Predict Humanoid Fall. International Journal of Humanoid Robotics 08(02), 245 (2011)CrossRefGoogle Scholar
  46. 46.
    Sutton, R.S., Barto, G.A.: Reinforcement Learning: An Introduction. MIT press (1998)Google Scholar
  47. 47.
    Watkins, C.J.C.H: Q-learning. Mach. Learn. 22(3–4), 25–292 (1992)Google Scholar
  48. 48.
    Peng, J., Williams, R.: Incremental multi-step Q-learning. Mach. Learn. 22(1), 283–290 (1996)Google Scholar
  49. 49.
    Luca, A.D., Oriolo, G., Samson, C.: Feedback Control of a Nonholonomic Car-like Robot, tech. rep., Laboratoire d’Analyse et d’Architecture des Systems, CNRS (1998)Google Scholar
  50. 50.
    Barker, O., Beranek, R., Ahmadi, M.: Design of a 13 Degree-of-Freedom Biped Robot with a CAN-Based Distributed Digital Control System. In: IEEE International Conference on Advanced Intelligent Mechatronics, pp 836–841 (2010)Google Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2015

Authors and Affiliations

  1. 1.Department of Mechanical and Aerospace EngineeringOttawaCanada

Personalised recommendations