The Chin Pinch: A Case Study in Skill Learning on a Legged Robot

  • Peggy Fidelman
  • Peter Stone
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4434)


When developing skills on a physical robot, it is appealing to turn to modern machine learning methods in order to automate the process. However, when no accurate simulator exists for the type of motion in question, all learning must occur on the physical robot itself. In such a case, there is a high premium on quick, efficient learning (specifically, learning with low sample complexity). Recent results in learning locomotion have demonstrated the feasibility of learning fast walks directly on quadrupedal robots. This paper demonstrates that it is also possible to learn a higher-level skill requiring more fine motor coordination, again with all learning occurring directly on the robot. In particular, the paper presents a learned ball-grasping skill on a commercially available Sony Aibo robot, with no human intervention other than battery changes. The learned skill significantly outperforms our best hand-tuned solution. As the learned grasping skill relies on a learned walk, we characterize our learning implementation within the layered learning formalism. To our knowledge, the two learned layers represent the first use of layered learning on a physical robot.


learning and adaptive systems sensor-motor control 


  1. 1.
    Bicchi, A., Kumar, V.: Robotic grasping and contact: A review. In: Proceedings of the IEEE International Conference on Robotics and Automation (April 2000)Google Scholar
  2. 2.
    Chernova, S., Veloso, M.: An evolutionary approach to gait learning for four-legged robots. In: Proceedings of IROS 2004 (September 2004)Google Scholar
  3. 3.
    Chernova, S., Veloso, M.: Learning and using models of kicking motions for legged robots. In: ICRA 2004. Proceedings of International Conference on Robotics and Automation (May 2004)Google Scholar
  4. 4.
    Cohen, D., Ooi, Y.H., Vernaza, P., Lee, D.D.: The University of Pennsylvania RoboCup 2004 legged soccer team. Available at URL
  5. 5.
    Gat, E.: On the role of simulation in the study of autonomous mobile robots. In: AAAI-95 Spring Symposium on Lessons Learned from Implemented Software Architectures for Physical Agents. Stanford, CA (March 1995)Google Scholar
  6. 6.
    Gustafson, S.M.: Layered learning for a cooperative robot soccer problem. Master’s thesis, Kansas State University (2000)Google Scholar
  7. 7.
    Hsu, W.H., Gustafson, S.M.: Genetic programming and multi-agent layered learning by reinforcements. In: Genetic and Evolutionary Computation Conference, New York,NY, pp. 764–771. Morgan Kaufmann, San Francisco (2002)Google Scholar
  8. 8.
    Kamon, I., Flash, T., Edelman, S.: Learning to grasp using visual information. Technical report, The Weizmann Institute of Science, Revhovot, Israel (March 1994)Google Scholar
  9. 9.
    Kim, M.S., Uther, W.: Automatic gait optimisation for quadruped robots. In: Australasian Conference on Robotics and Automation, Brisbane (December 2003)Google Scholar
  10. 10.
    Kohl, N., Stone, P.: Machine learning for fast quadrupedal locomotion. In: The Nineteenth National Conference on Artificial Intelligence, pp. 611–616 (July 2004)Google Scholar
  11. 11.
    Kohl, N., Stone, P.: Policy gradient reinforcement learning for fast quadrupedal locomotion. In: Proceedings of the IEEE International Conference on Robotics and Automation (May 2004)Google Scholar
  12. 12.
    Quinlan, M.J., Chalup, S.K., Middleton, R.H.: Techniques for improving vision and locomotion on the sony aibo robot. In: Proceedings of the 2003 Australasian Conference on Robotics and Automation (December 2003)Google Scholar
  13. 13.
    Rofer, T.: Evolutionary gait-optimization using a fitness function based on proprioception. In: Nardi, D., Riedmiller, M., Sammut, C., Santos-Victor, J. (eds.) RoboCup 2004. LNCS (LNAI), vol. 3276, Springer, Heidelberg (2005)Google Scholar
  14. 14.
    Rofer, T., Burkhard, H.-D., Duffert, U., Hoffman, J., Gohring, D., Jungel, M., Lotzach, M., Stryk, O.v., Brunn, R., Kallnik, M., Kunz, M., Petters, S., Risler, M., Stelzer, M., Dahm, I., Wachter, M., Engel, K., Osterhues, A., Schumann, C., Ziegler, J.: Germanteam robocup 2003. Technical report (2003)Google Scholar
  15. 15.
    Sony: Aibo robot (2004),
  16. 16.
    Stone, P., Dresner, K., Fidelman, P., Jong, N.K., Kohl, N., Kuhlmann, G., Sridharan, M., Stronger, D.: The UT Austin Villa 2004 RoboCup four-legged team: Coming of age. Technical Report UT-AI-TR-04-313, The University of Texas at Austin, Department of Computer Sciences, AI Laboratory (October 2004)Google Scholar
  17. 17.
    Stone, P., Veloso, M.: Layered learning. In: López de Mántaras, R., Plaza, E. (eds.) ECML 2000. LNCS (LNAI), vol. 1810, pp. 369–381. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  18. 18.
    Whiteson, S., Kohl, N., Miikkulainen, R., Stone, P.: Evolving keepaway soccer players through task decomposition. Machine Learning 59(1), 5–30 (2005)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  • Peggy Fidelman
    • 1
  • Peter Stone
    • 1
  1. 1.Department of Computer Sciences, The University of Texas at Austin, 1 University Station C0500, Austin, Texas 78712-0233 

Personalised recommendations