Efficiency in the Identification in the Limit Learning Paradigm

  • Rémi EyraudEmail author
  • Jeffrey Heinz
  • Ryo Yoshinaka


The most widely used learning paradigm in Grammatical Inference was introduced in 1967 and is known as identification in the limit. An important issue that has been raised with respect to the original definition is the absence of efficiency bounds. Nearly fifty years after its introduction, it remains an open problem how to best incorporate a notion of efficiency and tractability into this framework. This chapter surveys the different refinements that have been developed and studied, and the challenges they face. Main results for each formalization, along with comparisons, are provided.


Target Language Regular Language Finite State Automaton Target Representation Membership Query 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    A. Ambainis, S. Jain, and A. Sharma. Ordinal mind change complexity of language identification. Theoretical Computer Science, pages 323–343, 1999.Google Scholar
  2. 2.
    D. Angluin. Finding patterns common to a set of strings. Journal of Computer and System Sciences, 21:46–62, 1980.MathSciNetCrossRefzbMATHGoogle Scholar
  3. 3.
    D. Angluin. Queries and concept learning. Machine Learning, 2(4):319–342, 1987.Google Scholar
  4. 4.
    D. Angluin, J. Aspnes, and A. Kontorovich. On the learnability of shuffle ideals. In Proceedings of the Algorithmic Learning Theory Conference, pages 111–123, 2012.Google Scholar
  5. 5.
    Dana Angluin. Inductive inference of formal languages from positive data. Information and Control, 45:117–135, 1980.MathSciNetCrossRefzbMATHGoogle Scholar
  6. 6.
    L. Becerra-Bonache, A. Dediu, and C. Tirnăucă. Learning DFA from correction and equivalence queries. In Proceedings of the International Colloquium on Grammatical Inference, pages 281–292, 2006.Google Scholar
  7. 7.
    L. E. Blum and M. Blum. Toward a mathematical theory of inductive inference. Information and Control, 28(2):125–155, 1975.MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    A. Blumer, A. Ehrenfeucht, D. Haussler, and M. Warmuth. Learnability and the Vapnik-Chervonenkis dimension. Journal of the ACM, 36(4):929–965, 1989.MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    R. Book and F. Otto. String-Rewriting Systems. Springer Verlag, 1993.Google Scholar
  10. 10.
    J. Case and T. Kötzing. Difficulties in forcing fairness of polynomial time inductive inference. In Proceedings of the Algorithmic Learning Theory Conference, pages 263–277, 2009.Google Scholar
  11. 11.
    N. Chomsky. Three models for the description of language. IRE Transactions on Information Theory, 2:113–124, 1956.CrossRefzbMATHGoogle Scholar
  12. 12.
    A. Clark. Learning trees from strings: A strong learning algorithm for some context-free grammars. Journal of Machine Learning Research, 14:3537–3559, 2014.MathSciNetzbMATHGoogle Scholar
  13. 13.
    A. Clark and R. Eyraud. Polynomial identification in the limit of substitutable context-free languages. Journal of Machine Learning Research, 8:1725–1745, 2007.MathSciNetzbMATHGoogle Scholar
  14. 14.
    A. Clark and S. Lappin. Linguistic Nativism and the Poverty of the Stimulus. Wiley-Blackwell, 2011.Google Scholar
  15. 15.
    A. Clark and F. Thollard. PAC-learnability of probabilistic deterministic finite state automata. Journal of Machine Learning Research, 5:473–497, 2004.MathSciNetzbMATHGoogle Scholar
  16. 16.
    A. Clark and R. Yoshinaka. Distributional learning of parallel multiple context-free grammars. Machine Learning, 96:5–31, 2014.MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    H. Comon, M. Dauchet, R. Gilleron, C. Löding, F. Jacquemard, D. Lugiez, S. Tison, and M. Tommasi. Tree automata techniques and applications. Available on:, 2007.
  18. 18.
    C. de la Higuera. Characteristic sets for polynomial grammatical inference. Machine Learning, 27:125–138, 1997.CrossRefzbMATHGoogle Scholar
  19. 19.
    C. de la Higuera. Grammatical inference: learning automata and grammars. Cambridge University Press, 2010.Google Scholar
  20. 20.
    C. de la Higuera and J. Oncina. Learning deterministic linear languages. In Proceedings of Conference on Learning Theory, pages 185–200, 2002.Google Scholar
  21. 21.
    P. Dupont, L. Miclet, and E. Vidal. What is the search space of the regular inference? In Proceedings of the International Colloquium on Grammatical Inference, pages 25–37, 1994.Google Scholar
  22. 22.
    R. Eyraud, C. de la Higuera, and J.-C. Janodet. LARS: A learning algorithm for rewriting systems. Machine Learning, 66(1):7–31, 2007.CrossRefGoogle Scholar
  23. 23.
    F. Girosi. An equivalence between sparse approximation and support vector machines. Neural Comput., 10(6):1455–1480, 1998.CrossRefGoogle Scholar
  24. 24.
    E. M. Gold. Language identification in the limit. Information and Control, 10(5):447–474, 1967.CrossRefzbMATHGoogle Scholar
  25. 25.
    J. Heinz. Computational theories of learning and developmental psycholinguistics. In J. Lidz, W. Synder, and J. Pater, editors, The Oxford Handbook of Developmental Linguistics. Cambridge University Press, in pressGoogle Scholar
  26. 26.
    D. Hsu, S. M. Kakade, and P. Liang. Identifiability and unmixing of latent parse trees. In Advances in Neural Information Processing Systems (NIPS), pages 1520–1528, 2013.Google Scholar
  27. 27.
    M. Isberner, F. Howar, and B. Steffen. Learning register automata: from languages to program structures. Machine Learning, 96:65–98, 2014.MathSciNetCrossRefzbMATHGoogle Scholar
  28. 28.
    Y. Ishigami and S. Tani. VC-dimensions of finite automata and commutative finite automata with \(k\) letters and \(n\) states. Discrete Applied Mathematics, 74:123–134, 1997.MathSciNetCrossRefzbMATHGoogle Scholar
  29. 29.
    J. Langford. Tutorial on practical prediction theory for classification. Journal of Machine Learning Research, 6:273–306, December 2005.MathSciNetzbMATHGoogle Scholar
  30. 30.
    M. Li and P. Vitanyi. Learning simple concepts under simple distributions. SIAM Journal of Computing, 20:911–935, 1991.MathSciNetCrossRefzbMATHGoogle Scholar
  31. 31.
    E. Moore. Gedanken-experiments on sequential machines. In Claude Shannon and John McCarthy, editors, Automata Studies, pages 129–153. Princeton University Press, 1956.Google Scholar
  32. 32.
    T. Oates, D. Desai, and V. Bhat. Learning k-reversible context-free grammars from positive structural examples. In Proceedings of the International Conference in Machine Learning, pages 459–465, 2002.Google Scholar
  33. 33.
    J. Oncina and P. García. Identifying regular languages in polynomial time. In Advances in Structural and Syntactic Pattern Recognition, volume 5 of Series in Machine Perception and Artificial Intelligence, pages 99–108. 1992.Google Scholar
  34. 34.
    T.-W. Pao and J. Carr III. A solution of the syntactical induction-inference problem for regular languages. Computer Languages, 3(1):53 – 64, 1978.CrossRefzbMATHGoogle Scholar
  35. 35.
    L. Pitt. Inductive inference, DFA’s, and computational complexity. In Analogical and Inductive Inference, number 397 in LNAI, pages 18–44. Springer-Verlag, 1989.Google Scholar
  36. 36.
    D. Ron, Y. Singer, and N. Tishby. On the learnability and usage of acyclic probabilistic finite automata. In Proceedings of the Conference on Learning Theory, pages 31–40, 1995.Google Scholar
  37. 37.
    G. Rozenberg, editor. Handbook of Graph Grammars and Computing by Graph Transformation: Volume I. Foundations. World Scientific, 1997.Google Scholar
  38. 38.
    Y. Sakakibara. Efficient learning of context-free grammars from positive structural examples. Information and Computation, 97:23–60, 1992.MathSciNetCrossRefzbMATHGoogle Scholar
  39. 39.
    Hiroyuki Seki, Takashi Matsumura, Mamoru Fujii, and Tadao Kasami. On multiple context-free grammars. Theoretical Computer Science, 88(2):191–229, 1991.MathSciNetCrossRefzbMATHGoogle Scholar
  40. 40.
    J. M. Sempere and P. García. A characterization of even linear languages and its application to the learning problem. In Proceedings of the International Colloquium in Grammatical Inference, pages 38–44, 1994.Google Scholar
  41. 41.
    C. Shibata and R. Yoshinaka. PAC-learning of some subclasses of context-free grammars with basic distributional properties from positive data. In Proceedings of the Algorithmic Learning Theory conference, pages 143–157, 2013.Google Scholar
  42. 42.
    Y. Tajima, E. Tomita, M. Wakatsuki, and M. Terada. Polynomial time learning of simple deterministic languages via queries and a representative sample. Theoretical Computer Science, 329(1-3):203 – 221, 2004.MathSciNetCrossRefzbMATHGoogle Scholar
  43. 43.
    L. G. Valiant. A theory of the learnable. Communications of the Association for Computing Machinery, 27(11):1134–1142, 1984.CrossRefzbMATHGoogle Scholar
  44. 44.
    V. Vapnik. The nature of statistical learning theory. Springer, 1995.Google Scholar
  45. 45.
    M. Wakatsuki and E. Tomita. A fast algorithm for checking the inclusion for very simple deterministic pushdown automata. IEICE TRANSACTIONS on Information and Systems, VE76-D(10):1224–1233, 1993.Google Scholar
  46. 46.
    T. Yokomori. On polynomial-time learnability in the limit of strictly deterministic automata. Machine Learning, 19:153–179, 1995.zbMATHGoogle Scholar
  47. 47.
    T. Yokomori. Polynomial-time identification of very simple grammars from positive data. Theoretical Computer Science, 1(298):179–206, 2003.MathSciNetCrossRefzbMATHGoogle Scholar
  48. 48.
    R. Yoshinaka. Identification in the limit of \(k, l\)-substitutable context-free languages. In Proceedings of the International Colloquium in Grammatical Inference, pages 266–279, 2008.Google Scholar
  49. 49.
    R. Yoshinaka. Learning efficiency of very simple grammars from positive data. Theoretical Computer Science, 410(19):1807–1825, 2009.MathSciNetCrossRefzbMATHGoogle Scholar
  50. 50.
    R. Yoshinaka. Efficient learning of multiple context-free languages with multidimensional substitutability from positive data. Theoretical Computer Science, 412:1821–1831, 2011.MathSciNetCrossRefzbMATHGoogle Scholar
  51. 51.
    T. Zeugmann. Can learning in the limit be done efficiently? In Proceedings of the Algorithmic Learning Theory conference, pages 17–38, 2003.Google Scholar
  52. 52.
    T. Zeugmann. From learning in the limit to stochastic finite learning. Theoretical Computer Science, 364(1):77–97, 2006.MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2016

Authors and Affiliations

  1. 1.QARMA TeamLaboratoire d’Informatique FondamentaleMarseilleFrance
  2. 2.University of DelawareNewarkUSA
  3. 3.Graduate School of InformaticsKyoto UniversityKyotoJapan

Personalised recommendations