SMART (Stochastic Model Acquisition with ReinforcemenT) Learning Agents: A Preliminary Report
We present a framework for building agents that learn using SMART, a system that combines stochastic model acquisition with reinforcement learning to enable an agent to model its environment through experience and subsequently form action selection policies using the acquired model. We extend an existing algorithm for automatic creation of stochastic strips operators  as a preliminary method of environment modelling. We then define the process of generation of future states using these operators and an initial state and finally show the process by which the agent can use the generated states to form a policy with a standard reinforcement learning algorithm. The potential of SMART is exemplified using the well-known predator prey scenario. Results of applying SMART to this environment and directions for future work are discussed.
KeywordsWorld State Inductive Logic Programming Learn Agent Reinforcement Learning Algorithm State Action Pair
Unable to display preview. Download preview PDF.
- 2.Dehaspe, L.: Maximum Entropy Modeling with Clausal Constraints. In: Džeroski, S., Lavrač, N. (eds.) ILP 1997. LNCS, vol. 1297, pp. 109–125. Springer, Heidelberg (1997)Google Scholar
- 3.Dzeroski, S., De Raedt, L., Blockeel, H.: Relational Reinforcement Learning. In: International Workshop on Inductive Logic Programming (1998)Google Scholar
- 6.Hanks, S.: Projecting plans for uncertain worlds. Ph.D. thesis, Yale University, Department of Computer Science (1990)Google Scholar
- 7.Kaelbling, L.P., Littman, H.L., Moore, A.P.: Reinforcement Learning: A Survey. Journal of Artificial Intelligence Research 4, 237–285 (1996)Google Scholar
- 8.Muggleton, S.H.: Learning Stochastic Logic Programs. In: Getoor, L., Jensen, D. (eds.) Proceedings of the AAAI2000 Workshop on Learning Statistical Models from Relational Data. AAAI, Menlo Park (2000)Google Scholar
- 9.Oates, T., Schmill, M.D., Gregory, D.E., Cohen, P.R.: Detecting complex dependencies in categorical data. In: Chap. in Finding Structure in Data: Artificial Intelligence and Statistics V. Springer, Heidelberg (1995)Google Scholar
- 10.Oates, T., Cohen, P.R.: Learning Planning Operators with Conditional and Probabilistic Effects. In: AAAI 1996 Spring Symposium on Planning with Incomplete Information for Robot Problems. AAAI, Menlo Park (1996)Google Scholar
- 11.Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. A Bradford Book. MIT Press, Cambridge (1998)Google Scholar
- 12.Shen, W.: Discovery as Autonomous Learning from the Environment. Machine Learning 12, 143–165 (1993)Google Scholar
- 14.Varsy, R.: Extending Planning and Learning Through Reinterpretation of World Model. M.Sc. thesis, City Univesity (2002)Google Scholar