Learning DFA from Simple Examples
 Rajesh Parekh,
 Vasant Honavar
 … show all 2 hide
Abstract
Efficient learning of DFA is a challenging research problem in grammatical inference. It is known that both exact and approximate (in the PAC sense) identifiability of DFA is hard. Pitt has posed the following open research problem: “Are DFA PACidentifiable if examples are drawn from the uniform distribution, or some other known simple distribution?” (Pitt, in Lecture Notes in Artificial Intelligence, 397, pp. 18–44, SpringerVerlag, 1989). We demonstrate that the class of DFA whose canonical representations have logarithmic Kolmogorov complexity is efficiently PAC learnable under the Solomonoff Levin universal distribution (m). We prove that the class of DFA is efficiently learnable under the PACS (PAC learning with simple examples) model (Denis, D'Halluin & Gilleron, STACS'96—Proceedings of the 13th Annual Symposium on the Theoretical Aspects of Computer Science, pp. 231–242, 1996) wherein positive and negative examples are sampled according to the universal distribution conditional on a description of the target concept. Further, we show that any concept that is learnable under Gold's model of learning from characteristic samples, Goldman and Mathias' polynomial teachability model, and the model of learning from example based queries is also learnable under the PACS model.
 Angluin, D. (1981). A note on the number of queries needed to identify regular languages. Information and Control, 51, 76–87.
 Angluin, D. (1987). Learning regular sets from queries and counterexamples. Information and Computation, 75, 87–106.
 Angluin, D. (1988). Queries and concept learning. Machine Learning, 2:4, 319–342.
 Castro, J., & Guijarro, D. (1998). Query, pacs and simplepac learning. Technical Report LSI982R, Universitat Polytéctica de Catalunya, Spain.
 Chomsky, N. (1956). Three models for the description of language. PGIT, 2:3, 113–124.
 Denis, F., D'Halluin, C., & Gilleron, R. (1996). Pac learning with simple examples. STACS'96—Proceedings of the 13^{th} Annual Symposium on the Theoretical Aspects of Computer Science (pp. 231–242).
 Denis, F.,& Gilleron, R. (1997). Pac learning under helpful distributions. In Proceedings of the Eighth International Workshop on Algorithmic Learning Theory (ALT'97), Lecture Notes in Artificial Intelligence 1316 (pp. 132–145), Sendai, Japan.
 Dupont, P. (1996). Incremental regular inference. In L. Miclet, & C. Higuera, (Eds.), Proceedings of the Third ICGI96, Lecture Notes in Artificial Intelligence 1147 (pp. 222–237), Montpellier,France, Springer.
 Dupont, P. (1996). Utilisation et apprentissage de modèles de language pour la reconnaissance de la parole continue. PhD thesis, Ecole Normale Supérieure des Télécommunications, Paris, France.
 Dupont, P., Miclet, L., & Vidal, E. (1994). What is the search space of the regular inference? In Proceedings of the Second International Colloquium on Grammatical Inference (ICGI'94) (pp. 25–37). Alicante, Spain.
 Gold, E. (1978). Complexity of automaton identification from given data. Information and Control, 37:3, 302–320.
 Goldman, S., & Mathias, H. (1993). Teaching a smarter learner. In Proceedings of theWorkshop on Computational Learning Theory (COLT'93) (pp. 67–76). ACM Press.
 Goldman, S., & Mathias, H (1996). Teaching a smarter learner. Journal of Computer and System Sciences, 52, 255–267.
 Colin de la Higuera (1996). Characteristic sets for polynomial grammatical inference. In L. Miclet, & C. Higuera, (Eds.), Proceedings of the Third ICGI96, Lecture Notes in Artificial Intelligence 1147 (pp. 59–71). Montpellier, France, Springer.
 Hopcroft, J., & Ullman, J. (1979). Introduction to automata theory, languages, and computation. Reading, MA: Addison Wesley.
 Jackson, J., & Tomkins, A. (1992). A computational model of teaching. In Proceedings of the Workshop on Computational Learning Theory (COLT'92) (pp. 319–326). ACM Press.
 Kearns, M., & Valiant, L. G. (1989). Cryptographic limitations on learning boolean formulae and finite automata. In Proceedings of the 21st Annual ACM Symposium on Theory of Computing (pp. 433–444). New York: ACM.
 Lang, K. (1992). Random DFAs can be approximately learned from sparse uniform sample. In Proceedings of the 5th ACM workshop on Computational Learning Theory (pp. 45–52).
 Li, M., & Vitányi, P. (1991). Learning simple concepts under simple distributions. SIAM Journal of Computing, 20:5, 911–935.
 Li, M., & Vitányi, P. (1997). An introduction to Kolmogorov complexity and its applications, (2nd ed.) New York: Springer Verlag.
 Oncina, J., & Garcia, P. (1992). Inferring regular languages in polynomial update time. In N. Pérez et al. (eds.), Pattern recognition and image analysis (pp. 49–61). Singapore: World Scientific.
 Pao, T., & Carr, J. (1978). A solution of the syntactic inductioninference problem for regular languages. Computer Languages, 3, 53–64.
 Parekh, R., & Honavar, V. (1993). Efficient learning of regular languages using teacher supplied positive examples and learner generated queries. In Proceedings of the Fifth UNB Conference on AI (pp. 195–203). Fredricton, Canada.
 Parekh, R., & Honavar, V. (1997). Learning DFA from simple examples. In Proceedings of the Eighth International Workshop on Algorithmic Learning Theory (ALT'97), Lecture Notes in Artificial Intelligence 1316 (pp. 116–131). Sendai, Japan, Springer. Also presented at theWorkshop on Grammar Inference, Automata Induction, and Language Acquisition (ICML'97), Nashville, TN, July 12, 1997.
 Parekh, R & Honavar, V. (1999). Simple DFA are polynomially probably exactly learnable from simple examples. In Proceedings of the Sixteenth International Conference on Machine Learning (ICML'99) (pp. 298–306). Bled, Slovenia.
 Pitt, L. (1989). Inductive inference, DFAs and computational complexity. In Analogical and Inductive Inference, Lecture Notes in Artificial Intelligence, 397 (pp. 18–44). SpringerVerlag.
 Pitt, L., & Warmuth, M. K. (1988). Reductions among prediction problems: on the difficulty of predicting automata. In Proceedings of the 3rd IEEE Conference on Structure in Complexity Theory (pp. 60–69).
 Pitt, L., & Warmuth, M. K. (1989). The minimum consistency DFA problem cannot be approximated within any polynomial. In Proceedings of the 21st ACM Symposium on the Theory of Computing (pp. 421–432). ACM.
 Rivest, R. L. & Schapire, R. E. (1993). Inference of finite automata using homing sequences. Information and Computation, 103:2, 299–347.
 Trakhtenbrot, B., & Barzdin, Ya. (1973). Finite Automata: Behavior and Synthesis. Amsterdam, North Holland.
 Valiant, L. (1984). A theory of the learnable. Communications of the ACM, 27, 1134–1142.
 Title
 Learning DFA from Simple Examples
 Journal

Machine Learning
Volume 44, Issue 12 , pp 935
 Cover Date
 20010701
 DOI
 10.1023/A:1010822518073
 Print ISSN
 08856125
 Online ISSN
 15730565
 Publisher
 Kluwer Academic Publishers
 Additional Links
 Topics
 Keywords

 DFA inference
 exact identification
 characteristic sets
 PAC learning
 collusion
 Industry Sectors
 Authors

 Rajesh Parekh ^{(1)}
 Vasant Honavar ^{(2)}
 Author Affiliations

 1. Blue Martini Software, 2600 Campus Drive, San Mateo, CA, 94403, USA
 2. Department of Computer Science, Iowa State University, Ames, IA, 50011, USA