Abstract
When developing skills on a physical robot, it is appealing to turn to modern machine learning methods in order to automate the process. However, when no accurate simulator exists for the type of motion in question, all learning must occur on the physical robot itself. In such a case, there is a high premium on quick, efficient learning (specifically, learning with low sample complexity). Recent results in learning locomotion have demonstrated the feasibility of learning fast walks directly on quadrupedal robots. This paper demonstrates that it is also possible to learn a higher-level skill requiring more fine motor coordination, again with all learning occurring directly on the robot. In particular, the paper presents a learned ball-grasping skill on a commercially available Sony Aibo robot, with no human intervention other than battery changes. The learned skill significantly outperforms our best hand-tuned solution. As the learned grasping skill relies on a learned walk, we characterize our learning implementation within the layered learning formalism. To our knowledge, the two learned layers represent the first use of layered learning on a physical robot.
Chapter PDF
Similar content being viewed by others
References
Bicchi, A., Kumar, V.: Robotic grasping and contact: A review. In: Proceedings of the IEEE International Conference on Robotics and Automation (April 2000)
Chernova, S., Veloso, M.: An evolutionary approach to gait learning for four-legged robots. In: Proceedings of IROS 2004 (September 2004)
Chernova, S., Veloso, M.: Learning and using models of kicking motions for legged robots. In: ICRA 2004. Proceedings of International Conference on Robotics and Automation (May 2004)
Cohen, D., Ooi, Y.H., Vernaza, P., Lee, D.D.: The University of Pennsylvania RoboCup 2004 legged soccer team. Available at URL http://www.cis.upenn.edu/robocup/UPenn04.pdf
Gat, E.: On the role of simulation in the study of autonomous mobile robots. In: AAAI-95 Spring Symposium on Lessons Learned from Implemented Software Architectures for Physical Agents. Stanford, CA (March 1995)
Gustafson, S.M.: Layered learning for a cooperative robot soccer problem. Master’s thesis, Kansas State University (2000)
Hsu, W.H., Gustafson, S.M.: Genetic programming and multi-agent layered learning by reinforcements. In: Genetic and Evolutionary Computation Conference, New York,NY, pp. 764–771. Morgan Kaufmann, San Francisco (2002)
Kamon, I., Flash, T., Edelman, S.: Learning to grasp using visual information. Technical report, The Weizmann Institute of Science, Revhovot, Israel (March 1994)
Kim, M.S., Uther, W.: Automatic gait optimisation for quadruped robots. In: Australasian Conference on Robotics and Automation, Brisbane (December 2003)
Kohl, N., Stone, P.: Machine learning for fast quadrupedal locomotion. In: The Nineteenth National Conference on Artificial Intelligence, pp. 611–616 (July 2004)
Kohl, N., Stone, P.: Policy gradient reinforcement learning for fast quadrupedal locomotion. In: Proceedings of the IEEE International Conference on Robotics and Automation (May 2004)
Quinlan, M.J., Chalup, S.K., Middleton, R.H.: Techniques for improving vision and locomotion on the sony aibo robot. In: Proceedings of the 2003 Australasian Conference on Robotics and Automation (December 2003)
Rofer, T.: Evolutionary gait-optimization using a fitness function based on proprioception. In: Nardi, D., Riedmiller, M., Sammut, C., Santos-Victor, J. (eds.) RoboCup 2004. LNCS (LNAI), vol. 3276, Springer, Heidelberg (2005)
Rofer, T., Burkhard, H.-D., Duffert, U., Hoffman, J., Gohring, D., Jungel, M., Lotzach, M., Stryk, O.v., Brunn, R., Kallnik, M., Kunz, M., Petters, S., Risler, M., Stelzer, M., Dahm, I., Wachter, M., Engel, K., Osterhues, A., Schumann, C., Ziegler, J.: Germanteam robocup 2003. Technical report (2003)
Sony: Aibo robot (2004), http://www.sony.net/Products/aibo
Stone, P., Dresner, K., Fidelman, P., Jong, N.K., Kohl, N., Kuhlmann, G., Sridharan, M., Stronger, D.: The UT Austin Villa 2004 RoboCup four-legged team: Coming of age. Technical Report UT-AI-TR-04-313, The University of Texas at Austin, Department of Computer Sciences, AI Laboratory (October 2004)
Stone, P., Veloso, M.: Layered learning. In: López de Mántaras, R., Plaza, E. (eds.) ECML 2000. LNCS (LNAI), vol. 1810, pp. 369–381. Springer, Heidelberg (2000)
Whiteson, S., Kohl, N., Miikkulainen, R., Stone, P.: Evolving keepaway soccer players through task decomposition. Machine Learning 59(1), 5–30 (2005)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Fidelman, P., Stone, P. (2007). The Chin Pinch: A Case Study in Skill Learning on a Legged Robot. In: Lakemeyer, G., Sklar, E., Sorrenti, D.G., Takahashi, T. (eds) RoboCup 2006: Robot Soccer World Cup X. RoboCup 2006. Lecture Notes in Computer Science(), vol 4434. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-74024-7_6
Download citation
DOI: https://doi.org/10.1007/978-3-540-74024-7_6
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-74023-0
Online ISBN: 978-3-540-74024-7
eBook Packages: Computer ScienceComputer Science (R0)