Machine Learning

, Volume 59, Issue 1, pp 5–30

Evolving Soccer Keepaway Players Through Task Decomposition

Authors

    • Department of Computer SciencesThe University of Texas at Austin
  • Nate Kohl
    • Department of Computer SciencesThe University of Texas at Austin
  • Risto Miikkulainen
    • Department of Computer SciencesThe University of Texas at Austin
  • Peter Stone
    • Department of Computer SciencesThe University of Texas at Austin
Article

DOI: 10.1007/s10994-005-0460-9

Cite this article as:
Whiteson, S., Kohl, N., Miikkulainen, R. et al. Mach Learn (2005) 59: 5. doi:10.1007/s10994-005-0460-9

Abstract

Complex control tasks can often be solved by decomposing them into hierarchies of manageable subtasks. Such decompositions require designers to decide how much human knowledge should be used to help learn the resulting components. On one hand, encoding human knowledge requires manual effort and may incorrectly constrain the learner’s hypothesis space or guide it away from the best solutions. On the other hand, it may make learning easier and enable the learner to tackle more complex tasks. This article examines the impact of this trade-off in tasks of varying difficulty. A space laid out by two dimensions is explored: (1) how much human assistance is given and (2) how difficult the task is. In particular, the neuroevolution learning algorithm is enhanced with three different methods for learning the components that result from a task decomposition. The first method, coevolution, is mostly unassisted by human knowledge. The second method, layered learning, is highly assisted. The third method, concurrent layered learning, is a novel combination of the first two that attempts to exploit human knowledge while retaining some of coevolution’s flexibility. Detailed empirical results are presented comparing and contrasting these three approaches on two versions of a complex task, namely robot soccer keepaway, that differ in difficulty of learning. These results confirm that, given a suitable task decomposition, neuroevolution can master difficult tasks. Furthermore, they demonstrate that the appropriate level of human assistance depends critically on the difficulty of the problem.

Keywords

coevolutionneural networksgenetic algorithmsRobot soccer
Download to read the full article text

Copyright information

© Springer Science + Business Media, Inc. 2005