Learning Goal Hierarchies from Structured Observations and Expert Annotations

  • Tolga Könik
  • John Laird
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3194)


We describe a framework for generating agent programs that model expert task performance in complex dynamic domains, using expert behavior observations and goal annotations as the primary source. We map the problem of learning an agent program on to multiple learning problems that can be represented in a “supervised concept learning” setting. The acquired procedural knowledge is partitioned into a hierarchy of goals and it is represented with first order rules. Using an inductive logic programming (ILP) learning component allows us to use structured goal annotations, structured background knowledge and structured behavior observations. We have developed an efficient mechanism for storing and retrieving structured behavior data. We have tested our system using artificially created examples and behavior observation traces generated by AI agents. We evaluate the learned rules by comparing them to hand-coded rules.


Agent Program Inductive Logic Programming Agent Architecture Decision Concept Inductive Logic Programming System 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Benson, S., Nilsson, N.: Inductive Learning of Reactive Action Models. In: Prieditis, A., Russell, S. (eds.) Machine Learning: Proceedings of the Twelfth International Conference, pp. 47–54. Morgan Kaufmann, San Fransisco (1995)Google Scholar
  2. 2.
    Camacho, R.: Inducing Models of Human Control Skills. In: Nédellec, C., Rouveirol, C. (eds.) ECML 1998. LNCS, vol. 1398, Springer, Heidelberg (1998)Google Scholar
  3. 3.
    Driessens, K., Dzeroski, S.: Integrating Experimentation and Guidance in Relational Reinforcement Learning. In: Sammut, C., Hoffmann, C. (eds.) Proceedings of the Nineteenth International Conference on Machine Learning (ICML 2002), pp. 115–122. Morgan Kaufmann Publishers, San Francisco (2002)Google Scholar
  4. 4.
    Dzeroski, S., Raedt, L.D., Driessens, K.: Relational Reinforcement Learning. Machine Learning 43, 5–52 (2001)CrossRefGoogle Scholar
  5. 5.
    Isaac, A., Sammut, C.: Goal-directed Learning to Fly. In: Fawcett, T., Mishra, N. (eds.) Proceedings of the Twentieth International Conference (ICML 2003), AAAI Press, Menlo Park (2003)Google Scholar
  6. 6.
    Jones, R.M., Laird, J.E., Nielsen, P.E., Coulter, K.J., Kenny, P.G., Koss, F.V.: Automated Intelligent Pilots for Combat Flight Simulation. AI Magazine 20, 27–42 (1999)Google Scholar
  7. 7.
    Khardon, R.: Learning to Take Actions. Machine Learning 35, 57–90 (1999)zbMATHCrossRefGoogle Scholar
  8. 8.
    Laird, J.E., Newell, A., Rosenbloom, P.S.: Soar: An Architecture for General Intelligence. Artificial Intelligence 33, 1–64 (1987)CrossRefGoogle Scholar
  9. 9.
    Magerko, B., Laird, J.E., Assanie, M., Kerfoot, A., Stokes, D.: AI Characters and Directors for Interactive Computer Games. In: The 16th Innovative Applications of Artificial Intelligence (IAAI 2004), San Jose, CA (2004)Google Scholar
  10. 10.
    Michie, D., Bain, M., Hayes-Michie, J.: Cognitive Models from Subcognitive Skills. In: Grimble, M., McGhee, J., Mowforth, P. (eds.) Knowledge-Based Systems in Industrial Control, pp. 71–90. Peter Peregrinus, Stevenage (1990)Google Scholar
  11. 11.
    Michie, D., Camacho, R.: Building Symbolic Representations of Intuitive Real-Time Skills From Performance Data. In: Machine Intelligence 13 Workshop (MI 1992), Glasgow, U.K (1992)Google Scholar
  12. 12.
    Moyle, S.: Using Theory Completion to Learn a Robot Navigation Control Program. In: Matwin, S., Sammut, C. (eds.) ILP 2002. LNCS (LNAI), vol. 2583, pp. 182–197. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  13. 13.
    Muggleton, S.: Inverse Entailment and Progol. New Generation Computing 13, 245–286 (1995)CrossRefGoogle Scholar
  14. 14.
    Newell, A.: Unified Theories of Cognition. Harvard Univ. Press, Cambridge (1990)Google Scholar
  15. 15.
    Otero, R.P.: Induction of the Effects of Actions by Monotonic Methods. In: Horváth, T., Yamamoto, A. (eds.) ILP 2003. LNCS (LNAI), vol. 2835, pp. 299–310. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  16. 16.
    Sammut, C., Hurst, S., Kedzier, D., Michie, D.: Learning to fly. In: Sleeman, D., Edwards, P. (eds.) Proceedings of the 9th International Conference on Machine Learning (ICML 1992), pp. 385–393. Morgan Kaufmann, San Francisco (1992)Google Scholar
  17. 17.
    Srinivasan, A.: The Aleph 5 Manual. machlearn/Aleph (2003),
  18. 18.
    Struyf, J., Ramon, J., Blockeel, H.: Compact Representation of Knowledge Bases in ILP. In: Matwin, S., Sammut, C. (eds.) ILP 2002. LNCS (LNAI), vol. 2583, pp. 254–269. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  19. 19.
    Suc, D., Bratko, I.: Problem decomposition for behavioural cloning. In: Lopez de Mantaras, R., Plaza, E. (eds.) ECML 2000. LNCS (LNAI), vol. 1810, pp. 382–391. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  20. 20.
    Urbancic, T., Bratko, I.: Reconstructing Human Skill with Machine Learning. In: Cohn, A.G. (ed.) Proceedings of the 11th European Conference on Artificial Intelligence (ECAI 1994), pp. 498–502. John Wiley and Sons, Chichester (1994)Google Scholar
  21. 21.
    van Lent, M., Laird, J.: Learning procedural knowledge through observation. In: Proceedings of the International Conference on Knowledge Capture (KCAP 2001), pp. 179–186. ACM Press, New York (2001)CrossRefGoogle Scholar
  22. 22.
    van Lent, M.: Learning Task-Performance Knowledge through Observation. Ph.D. Thesis. Univ. of Michigan (2000)Google Scholar
  23. 23.
    Wang, X.: Learning Planning Operators by Observation and Practice. Ph.D. Thesis (Tech. Report CMU-CS-96-154). Computer Science Department, Carnegie Mellon University (1996)Google Scholar
  24. 24.
    Wray, R.E., Laird, J.E., Nuxoll, A., Stokes, D., Kerfoot, A.: Synthetic Adversaries for Urban Combat Training. In: Proceedings of Innovative Applications of Artificial Intelligence (IAAI-2004), AAAI Press, Menlo Park (2004) (in press)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2004

Authors and Affiliations

  • Tolga Könik
    • 1
  • John Laird
    • 1
  1. 1.Artificial Intelligence Lab.University of MichiganUSA

Personalised recommendations