Abstract
Consider a robot wandering around an unfamiliar environment, performing actions and sensing the resulting environmental states. The robot's task is to construct an internal model of its environment, a model that will allow it to predict the consequences of its actions and to determine what sequences of actions to take to reach particular goal states. Rivest and Schapire (1987a, 1987b; Schapire, 1988) have studied this problem and have designed a symbolic algorithm to strategically explore and infer the structure of “finite state” environments. The heart of this algorithm is a clever representation of the environment called an update graph. We have developed a connectionist implementation of the update graph using a highly-specialized network architecture. With back propagation learning and a trivial exploration strategy--choosing random actions--the network can outperform the Rivest and Schapire algorithm on simple problems. Perhaps the most interesting consequence of the connectionist approach is that, by relaxing the constraints imposed by a symbolic description, it suggests a more general representation of the update graph, thus allowing for greater flexibility in expressing potential solutions.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Cohn, D., Atlas, L., Ladner, R., Marks II, R., El-sharkawi, M., Aggoune, M., & Park, D.(1990). Training connectionist networks with queries and selective sampling.In D.S.Touretzky (Ed.), Advances in neural infor-mation processing systems 2 (pp. 566-573). San Mateo, CA: Morgan Kaufmann.
Elman, J.L.(1988). Finding structure in time (CRL Technical Report 8801). La Jolla: University of California, San Diego, Center for Research in Language.
Mozer, M.C.(1989). A focused back-propagation algorithm for temporal pattern recognition. Complex Systems, 3, 349-381.
Rivest, R.L., & Schapire, R.E.(1987). Diversity-based inference of finite automata.In Proceedings of the Twenty-Eighth Annual Symposium on Foundations of Computer Science (pp. 78-87).
Rivest, R.L., & Schapire, R.E.(1987). A new approach to unsupervised learning in deterministic environments. In P.Langley (Ed.), Proceedings of the Fourth International Workshop on Machine Learning (pp. 364-375).
Rumelhart, D.E., Hinton, G.E., & Williams, R.J.(1986). Learning internal representations by error propagation. In D.E.Rumelhart & J.L.McClelland (Eds.), Parallel distributed processing:Explorations in the microstructure of cognition.Volume I:Foundations (pp. 318-362). Cambridge, MA: MIT Press/Bradford Books.
Schapire, R.E.(1988). Diversity-based inference of finite automata. Unpublished master's thesis, Massachusetts Institute of Technology, Cambridge, MA.
Servan-Schreiber, D., Cleeremans, A., & McClelland, J.L.(1988). Encoding sequential structure in simple recurrent networks (Technical Report CMU-CS-88-183). Pittsburgh, PA: Carnegie-Mellon University, Department of Computer Science.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Mozer, M.C., Bachrach, J. SLUG: A Connectionist Architecture for Inferring the Structure of Finite-State Environments. Machine Learning 7, 139–160 (1991). https://doi.org/10.1023/A:1022694928327
Issue Date:
DOI: https://doi.org/10.1023/A:1022694928327