SMART (Stochastic Model Acquisition with ReinforcemenT) Learning Agents: A Preliminary Report

  • Christopher Child
  • Kostas Stathis
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3394)

Abstract

We present a framework for building agents that learn using SMART, a system that combines stochastic model acquisition with reinforcement learning to enable an agent to model its environment through experience and subsequently form action selection policies using the acquired model. We extend an existing algorithm for automatic creation of stochastic strips operators [9] as a preliminary method of environment modelling. We then define the process of generation of future states using these operators and an initial state and finally show the process by which the agent can use the generated states to form a policy with a standard reinforcement learning algorithm. The potential of SMART is exemplified using the well-known predator prey scenario. Results of applying SMART to this environment and directions for future work are discussed.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Boutilier, C., Dean, T., Hanks, S.: Decision-Theoretic Planning: Structural Assumptions and Computational Leverage. Journal of Artificial Intelligence Research 11, 1–94 (1999)MATHMathSciNetGoogle Scholar
  2. 2.
    Dehaspe, L.: Maximum Entropy Modeling with Clausal Constraints. In: Džeroski, S., Lavrač, N. (eds.) ILP 1997. LNCS, vol. 1297, pp. 109–125. Springer, Heidelberg (1997)Google Scholar
  3. 3.
    Dzeroski, S., De Raedt, L., Blockeel, H.: Relational Reinforcement Learning. In: International Workshop on Inductive Logic Programming (1998)Google Scholar
  4. 4.
    Drescher, G.L.: Made-Up Minds, A Constructivist Approach to Artificial Intelligence. MIT Press, Cambridge (1991)MATHGoogle Scholar
  5. 5.
    Fikes, R.E., Nilsson, N.J.: STRIPS: a new approach to the application of theorem proving to problem-solving. Artificial Intelligence 2(3-4), 189–208 (1971)MATHCrossRefGoogle Scholar
  6. 6.
    Hanks, S.: Projecting plans for uncertain worlds. Ph.D. thesis, Yale University, Department of Computer Science (1990)Google Scholar
  7. 7.
    Kaelbling, L.P., Littman, H.L., Moore, A.P.: Reinforcement Learning: A Survey. Journal of Artificial Intelligence Research 4, 237–285 (1996)Google Scholar
  8. 8.
    Muggleton, S.H.: Learning Stochastic Logic Programs. In: Getoor, L., Jensen, D. (eds.) Proceedings of the AAAI2000 Workshop on Learning Statistical Models from Relational Data. AAAI, Menlo Park (2000)Google Scholar
  9. 9.
    Oates, T., Schmill, M.D., Gregory, D.E., Cohen, P.R.: Detecting complex dependencies in categorical data. In: Chap. in Finding Structure in Data: Artificial Intelligence and Statistics V. Springer, Heidelberg (1995)Google Scholar
  10. 10.
    Oates, T., Cohen, P.R.: Learning Planning Operators with Conditional and Probabilistic Effects. In: AAAI 1996 Spring Symposium on Planning with Incomplete Information for Robot Problems. AAAI, Menlo Park (1996)Google Scholar
  11. 11.
    Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. A Bradford Book. MIT Press, Cambridge (1998)Google Scholar
  12. 12.
    Shen, W.: Discovery as Autonomous Learning from the Environment. Machine Learning 12, 143–165 (1993)Google Scholar
  13. 13.
    Tesauro, G.J.: TD-Gammon, a self-teaching backgammon program, achieves master-level play. Neural Computation 6(2), 215–219 (1994)CrossRefGoogle Scholar
  14. 14.
    Varsy, R.: Extending Planning and Learning Through Reinterpretation of World Model. M.Sc. thesis, City Univesity (2002)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  • Christopher Child
    • 1
  • Kostas Stathis
    • 1
  1. 1.Department of Computing, School of InformaticsCity UniversityLondon

Personalised recommendations