Abstract
Connectionist learning models have had considerable empirical success, but it is hard to characterize exactly what they learn. The learning of finite-state languages (FSL) from example strings is a domain which has been extensively studied and might provide an opportunity to help understand connectionist learning. A major problem is that traditional FSL learning assumes the storage of all examples and thus violates connectionist principles. This paper presents a provably correct algorithm for inferring any minimum-state deterministic finite-state automata (FSA) from a complete ordered sample using limited total storage and without storing example strings. The algorithm is an iterative strategy that uses at each stage a current encoding of the data considered so far, and one single sample string. One of the crucial advantages of our algorithm is that the total amount of space used in the course of learning for encoding any finite prefix of the sample is polynomial in the size of the inferred minimum state deterministic FSA. The algorithm is also relatively efficient in time and has been implemented. More importantly, there is a connectionist version of the algorithm that preserves these properties. The connectionist version requires much more structure than the usual models and has been implemented using the Rochester Connectionist Simulator. We also show that no machine with finite working storage can iteratively identify the FSL from arbitrary presentations.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Angluin, D.(1987). Learning regular sets from queries and counterexamples. Information and Computation, 75, 87-106.
Angluin, D.(1981). A note on the number of queries needed to identify regular languages. Information and Control, 51, 76-87.
Angluin, D.(1978). On the complexity of minimum inference of regular sets. Information and Control, 39, 337-350.
Angluin, D.(1976). An application of the theory of computational complexity to the study of inductive inference. Ph.D.dissertation, Department of Electrical Engineering & Computer Science, Univ. California, Berkeley.
Angluin, D., & Smith, C.H.(1983). Inductive inference:Theory and methods. Computing Surveys, 15, 237-269.
Biermann, A.W., & Feldman, J.A.(1978). On the synthesis of finite-state machines from samples of their behavior. IEEE Trans, on Computers, C-21, 592-597.
Brooks, R.A.(1987). Intelligence without representation. Proceedings of the Conf.on foundations of AI. Cambridge, MA: MIT.
Feldman, J.A., & Ballard, D.H.(1982). Connectionist models and their properties. Cognitive Science, 6, 205-254.
Gold, E.M.(1978). Complexity of automaton identification from given data. Information and Control, 37, 302-320.
Gold, E.M.(1972). System identification via state characterization. Automatica, 8, 621-636.
Gold, E.M.(1967). Language identification in the limit. Information and Control, 10, 447-474.
Hinton, G.E.(1987). Connectionist learning procedures (TR CMU-CS-87-115). Pittsburgh, PA: Carnegie Mellon University, Computer Science Department.
Hopcroft, J.E., & Ullman, J.D.(1979). Introduction to automata and formal languages. Reading, MA: Addison-Wesley.
Horning, J.K.(1969). A study of grammatical inference.Ph.D.thesis, Stanford University.
Ibarra, O.H., & Jiang, T.(1988). Learning regular languages from counterexamples. Proceedings of the 1988 Workshop on Computational Learning Theory (pp. 371-385). Boston, MA.
Jantke, K.P., & Beick, H-R.(1981). Combining postulates of naturalness in inductive inference. Journal of Infor-mation Processing and Cybernetics, 17, 465-484.
Kearns, M., Li, J., Pitt, L., & Valiant, L.(1987). On the learnability of boolean formulae. Proceedings of the 9th Annual ACM Symp.on Theory of Computing (pp. 284-295), New York, NY.
Natarajan, B.K.(1987). On learning boolean functions. Proceedings of the 9th Annual ACM Symp.on Theory of Computing (pp. 296-394).New York, NY.
Osherson, D.N., Stob, M., & Weinstein, S.(1986). Systems that learn:An introduction to learning theory for conginitive and computer scientists. Cambridge, MA: MIT Press.
Rivest, R.L., & Schapire, R.E.(1987). A new approach to unsupervised learning in deterministic environments. Proceedings of the 4th International Workshop on Machine Learning.Irvine, CA.
Rivest, R.L., & Schapire, R.E.(1987). Diversity-based inference of finite automata. Proceedings of the 28th Annual Symp.on Foundations of Computer Science.Los Angeles, CA.
Rumelhart, D.E., & McClelland, J.L.(Eds.).(1986). Parallel distributed processing, explorations in the microstructure of cognition. Cambridge, MA: Bradford Books/MIT Press.
Sharon, M.(1990). Learning automata. M.Sc.Thesis in Computer Science, Technion, Haifa, Israel, (in Hebrew).
Trakhtenbrot, B.A., & Barzdin, Ya.M.(1973). Finite automata.Amsterdam:North-Holland.
Valiant, L.G.(1985). Learning disjunctions of conjunctions. Proceedings of the 9th IJCAI (pp. 560-566).Los Angeles, CA.
Valiant, L.G.(1984). A theory of the learnable. CACM, 27, 1134-1142.
Waltz, D., & Feldman, J.A.(Eds.).(1987). Connectionist models and their implications.Ablex Publishing Corp.
Weisberg, Y.(1990). Iterative learning finite automata-Application by neural net. M.Sc.thesis in Electrical Engi-neering, Technion, Haifa, Israel, (in Hebrew).
Wiehagen, R.(1976). Limeserkennung rekursiver funktionen durch spezielle strategien. Elektronische Informa-tionsverarbeitung und Kybernetik, 12, 93-99.
Williams, R.J.(1987). Reinforcement-learning connectionst systems (TR NU-CCS-87-3). 34
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Porat, S., Feldman, J.A. Learning Automata from Ordered Examples. Machine Learning 7, 109–138 (1991). https://doi.org/10.1023/A:1022642911489
Issue Date:
DOI: https://doi.org/10.1023/A:1022642911489