Curriculum Learning for Motor Skills
Humans and animals acquire their wide repertoire of motor skills through an incremental learning process, during which progressively more complex skills are acquired and subsequently integrated with prior abilities. Inspired by this general idea, we develop an approach for learning motor skills based on a two-level curriculum. At the high level, the curriculum specifies an order in which different skills should be learned. At the low level, the curriculum defines a process for learning within a skill. We develop a set of integrated motor skills for a planar articulated figure capable of doing parameterized hops, flips, rolls, and acrobatic sequences. The same curriculum can be applied to yield individualized motor skill sets for articulated figures of varying proportions.
KeywordsMotor Skill Reinforcement Learning Task Parameter Learn Motor Skill Curriculum Learn
Unable to display preview. Download preview PDF.
- 1.Asada, M., Noda, S., Tawaratsumida, S., Hosoda, K.: Purposive behavior acquisition for a real robot by vision-based reinforcement learning. Machine Learning 23(2), 279–303 (1996)Google Scholar
- 2.Bengio, Y., Louradour, J., Collobert, R., Weston, J.: Curriculum learning. In: Proc. Intl. Conf. on Machine Learning, pp. 41–48. ACM (2009)Google Scholar
- 3.Boone, G.: Minimum-time control of the acrobot. In: IEEE Intl. Conf. on Robotics and Automation, pp. 3281–3287 (1997)Google Scholar
- 4.Hart, S.W.: The development of hierarchical knowledge in robot systems. PhD thesis, University of Massachusetts Amherst (2009)Google Scholar
- 5.Hauser, J., Murray, R.M.: Nonlinear controllers for non-integrable systems: the acrobot example. In: American Control Conf., pp. 669–671 (1990)Google Scholar
- 6.Konidaris, G., Barto, A.G.: Skill discovery in continuous reinforcement learning domains using skill chaining. In: Proc. NIPS, vol. 22, pp. 1015–1023 (2009)Google Scholar
- 7.Neumann, G., Maass, W., Peters, J.: Learning complex motions by sequencing simpler motion templates. In: Proceedings of the 26th Annual International Conference on Machine Learning, pp. 753–760. ACM (2009)Google Scholar
- 8.Ng, A.Y.: Shaping and policy search in reinforcement learning. PhD thesis, University of California, Berkeley (2003)Google Scholar
- 9.Pickett, M., Barto, A.G.: Policyblocks: An algorithm for creating useful macro-actions in reinforcement learning. In: Proc. ICML, pp. 506–513 (2002)Google Scholar
- 11.Schmidt, R.A., Lee, T.D.: Motor control and learning: A behavioral emphasis. Human Kinetics Publishers (2005)Google Scholar
- 14.Stout, A., Barto, A.G.: Competence progress intrinsic motivation. In: IEEE Intl. Conf. on Development and Learning, pp. 257–262. IEEE (2010)Google Scholar
- 15.Yin, K.K., Coros, S., Beaudoin, P., van de Panne, M.: Continuation methods for adapting simulated skills. In: ACM SIGGRAPH 2008 Papers, pp. 1–7. ACM (2008)Google Scholar