Topics in Grammatical Inference pp 25-46 | Cite as

# Efficiency in the Identification in the Limit Learning Paradigm

Chapter

First Online:

## Abstract

The most widely used learning paradigm in Grammatical Inference was introduced in 1967 and is known as *identification in the limit*. An important issue that has been raised with respect to the original definition is the absence of efficiency bounds. Nearly fifty years after its introduction, it remains an open problem how to best incorporate a notion of efficiency and tractability into this framework. This chapter surveys the different refinements that have been developed and studied, and the challenges they face. Main results for each formalization, along with comparisons, are provided.

## Keywords

Target Language Regular Language Finite State Automaton Target Representation Membership Query
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

## References

- 1.A. Ambainis, S. Jain, and A. Sharma. Ordinal mind change complexity of language identification.
*Theoretical Computer Science*, pages 323–343, 1999.Google Scholar - 2.D. Angluin. Finding patterns common to a set of strings.
*Journal of Computer and System Sciences*, 21:46–62, 1980.MathSciNetCrossRefMATHGoogle Scholar - 3.D. Angluin. Queries and concept learning.
*Machine Learning*, 2(4):319–342, 1987.Google Scholar - 4.D. Angluin, J. Aspnes, and A. Kontorovich. On the learnability of shuffle ideals. In
*Proceedings of the Algorithmic Learning Theory Conference*, pages 111–123, 2012.Google Scholar - 5.Dana Angluin. Inductive inference of formal languages from positive data.
*Information and Control*, 45:117–135, 1980.MathSciNetCrossRefMATHGoogle Scholar - 6.L. Becerra-Bonache, A. Dediu, and C. Tirnăucă. Learning DFA from correction and equivalence queries. In
*Proceedings of the International Colloquium on Grammatical Inference*, pages 281–292, 2006.Google Scholar - 7.L. E. Blum and M. Blum. Toward a mathematical theory of inductive inference.
*Information and Control*, 28(2):125–155, 1975.MathSciNetCrossRefMATHGoogle Scholar - 8.A. Blumer, A. Ehrenfeucht, D. Haussler, and M. Warmuth. Learnability and the Vapnik-Chervonenkis dimension.
*Journal of the ACM*, 36(4):929–965, 1989.MathSciNetCrossRefMATHGoogle Scholar - 9.R. Book and F. Otto.
*String-Rewriting Systems*. Springer Verlag, 1993.Google Scholar - 10.J. Case and T. Kötzing. Difficulties in forcing fairness of polynomial time inductive inference. In
*Proceedings of the Algorithmic Learning Theory Conference*, pages 263–277, 2009.Google Scholar - 11.N. Chomsky. Three models for the description of language.
*IRE Transactions on Information Theory*, 2:113–124, 1956.CrossRefMATHGoogle Scholar - 12.A. Clark. Learning trees from strings: A strong learning algorithm for some context-free grammars.
*Journal of Machine Learning Research*, 14:3537–3559, 2014.MathSciNetMATHGoogle Scholar - 13.A. Clark and R. Eyraud. Polynomial identification in the limit of substitutable context-free languages.
*Journal of Machine Learning Research*, 8:1725–1745, 2007.MathSciNetMATHGoogle Scholar - 14.A. Clark and S. Lappin.
*Linguistic Nativism and the Poverty of the Stimulus*. Wiley-Blackwell, 2011.Google Scholar - 15.A. Clark and F. Thollard. PAC-learnability of probabilistic deterministic finite state automata.
*Journal of Machine Learning Research*, 5:473–497, 2004.MathSciNetMATHGoogle Scholar - 16.A. Clark and R. Yoshinaka. Distributional learning of parallel multiple context-free grammars.
*Machine Learning*, 96:5–31, 2014.MathSciNetCrossRefMATHGoogle Scholar - 17.H. Comon, M. Dauchet, R. Gilleron, C. Löding, F. Jacquemard, D. Lugiez, S. Tison, and M. Tommasi. Tree automata techniques and applications. Available on: http://tata.gforge.inria.fr/, 2007.
- 18.C. de la Higuera. Characteristic sets for polynomial grammatical inference.
*Machine Learning*, 27:125–138, 1997.CrossRefMATHGoogle Scholar - 19.C. de la Higuera.
*Grammatical inference: learning automata and grammars*. Cambridge University Press, 2010.Google Scholar - 20.C. de la Higuera and J. Oncina. Learning deterministic linear languages. In
*Proceedings of Conference on Learning Theory*, pages 185–200, 2002.Google Scholar - 21.P. Dupont, L. Miclet, and E. Vidal. What is the search space of the regular inference? In
*Proceedings of the International Colloquium on Grammatical Inference*, pages 25–37, 1994.Google Scholar - 22.R. Eyraud, C. de la Higuera, and J.-C. Janodet. LARS: A learning algorithm for rewriting systems.
*Machine Learning*, 66(1):7–31, 2007.CrossRefGoogle Scholar - 23.F. Girosi. An equivalence between sparse approximation and support vector machines.
*Neural Comput.*, 10(6):1455–1480, 1998.CrossRefGoogle Scholar - 24.E. M. Gold. Language identification in the limit.
*Information and Control*, 10(5):447–474, 1967.CrossRefMATHGoogle Scholar - 25.J. Heinz. Computational theories of learning and developmental psycholinguistics. In J. Lidz, W. Synder, and J. Pater, editors,
*The Oxford Handbook of Developmental Linguistics*. Cambridge University Press, in pressGoogle Scholar - 26.D. Hsu, S. M. Kakade, and P. Liang. Identifiability and unmixing of latent parse trees. In
*Advances in Neural Information Processing Systems (NIPS)*, pages 1520–1528, 2013.Google Scholar - 27.M. Isberner, F. Howar, and B. Steffen. Learning register automata: from languages to program structures.
*Machine Learning*, 96:65–98, 2014.MathSciNetCrossRefMATHGoogle Scholar - 28.Y. Ishigami and S. Tani. VC-dimensions of finite automata and commutative finite automata with \(k\) letters and \(n\) states.
*Discrete Applied Mathematics*, 74:123–134, 1997.MathSciNetCrossRefMATHGoogle Scholar - 29.J. Langford. Tutorial on practical prediction theory for classification.
*Journal of Machine Learning Research*, 6:273–306, December 2005.MathSciNetMATHGoogle Scholar - 30.M. Li and P. Vitanyi. Learning simple concepts under simple distributions.
*SIAM Journal of Computing*, 20:911–935, 1991.MathSciNetCrossRefMATHGoogle Scholar - 31.E. Moore. Gedanken-experiments on sequential machines. In Claude Shannon and John McCarthy, editors,
*Automata Studies*, pages 129–153. Princeton University Press, 1956.Google Scholar - 32.T. Oates, D. Desai, and V. Bhat. Learning k-reversible context-free grammars from positive structural examples. In
*Proceedings of the International Conference in Machine Learning*, pages 459–465, 2002.Google Scholar - 33.J. Oncina and P. García. Identifying regular languages in polynomial time. In
*Advances in Structural and Syntactic Pattern Recognition*, volume 5 of*Series in Machine Perception and Artificial Intelligence*, pages 99–108. 1992.Google Scholar - 34.T.-W. Pao and J. Carr III. A solution of the syntactical induction-inference problem for regular languages.
*Computer Languages*, 3(1):53 – 64, 1978.CrossRefMATHGoogle Scholar - 35.L. Pitt. Inductive inference, DFA’s, and computational complexity. In
*Analogical and Inductive Inference*, number 397 in LNAI, pages 18–44. Springer-Verlag, 1989.Google Scholar - 36.D. Ron, Y. Singer, and N. Tishby. On the learnability and usage of acyclic probabilistic finite automata. In
*Proceedings of the Conference on Learning Theory*, pages 31–40, 1995.Google Scholar - 37.G. Rozenberg, editor.
*Handbook of Graph Grammars and Computing by Graph Transformation: Volume I. Foundations*. World Scientific, 1997.Google Scholar - 38.Y. Sakakibara. Efficient learning of context-free grammars from positive structural examples.
*Information and Computation*, 97:23–60, 1992.MathSciNetCrossRefMATHGoogle Scholar - 39.Hiroyuki Seki, Takashi Matsumura, Mamoru Fujii, and Tadao Kasami. On multiple context-free grammars.
*Theoretical Computer Science*, 88(2):191–229, 1991.MathSciNetCrossRefMATHGoogle Scholar - 40.J. M. Sempere and P. García. A characterization of even linear languages and its application to the learning problem. In
*Proceedings of the International Colloquium in Grammatical Inference*, pages 38–44, 1994.Google Scholar - 41.C. Shibata and R. Yoshinaka. PAC-learning of some subclasses of context-free grammars with basic distributional properties from positive data. In
*Proceedings of the Algorithmic Learning Theory conference*, pages 143–157, 2013.Google Scholar - 42.Y. Tajima, E. Tomita, M. Wakatsuki, and M. Terada. Polynomial time learning of simple deterministic languages via queries and a representative sample.
*Theoretical Computer Science*, 329(1-3):203 – 221, 2004.MathSciNetCrossRefMATHGoogle Scholar - 43.L. G. Valiant. A theory of the learnable.
*Communications of the Association for Computing Machinery*, 27(11):1134–1142, 1984.CrossRefMATHGoogle Scholar - 44.V. Vapnik.
*The nature of statistical learning theory*. Springer, 1995.Google Scholar - 45.M. Wakatsuki and E. Tomita. A fast algorithm for checking the inclusion for very simple deterministic pushdown automata.
*IEICE TRANSACTIONS on Information and Systems*, VE76-D(10):1224–1233, 1993.Google Scholar - 46.T. Yokomori. On polynomial-time learnability in the limit of strictly deterministic automata.
*Machine Learning*, 19:153–179, 1995.MATHGoogle Scholar - 47.T. Yokomori. Polynomial-time identification of very simple grammars from positive data.
*Theoretical Computer Science*, 1(298):179–206, 2003.MathSciNetCrossRefMATHGoogle Scholar - 48.R. Yoshinaka. Identification in the limit of \(k, l\)-substitutable context-free languages. In
*Proceedings of the International Colloquium in Grammatical Inference*, pages 266–279, 2008.Google Scholar - 49.R. Yoshinaka. Learning efficiency of very simple grammars from positive data.
*Theoretical Computer Science*, 410(19):1807–1825, 2009.MathSciNetCrossRefMATHGoogle Scholar - 50.R. Yoshinaka. Efficient learning of multiple context-free languages with multidimensional substitutability from positive data.
*Theoretical Computer Science*, 412:1821–1831, 2011.MathSciNetCrossRefMATHGoogle Scholar - 51.T. Zeugmann. Can learning in the limit be done efficiently? In
*Proceedings of the Algorithmic Learning Theory conference*, pages 17–38, 2003.Google Scholar - 52.T. Zeugmann. From learning in the limit to stochastic finite learning.
*Theoretical Computer Science*, 364(1):77–97, 2006.MathSciNetCrossRefMATHGoogle Scholar

## Copyright information

© Springer-Verlag Berlin Heidelberg 2016