Using Pseudo-stochastic Rational Languages in Probabilistic Grammatical Inference
In probabilistic grammatical inference, a usual goal is to infer a good approximation of an unknown distribution P called a stochastic language. The estimate of P stands in some class of probabilistic models such as probabilistic automata (PA). In this paper, we focus on probabilistic models based on multiplicity automata (MA). The stochastic languages generated by MA are called rational stochastic languages; they strictly include stochastic languages generated by PA and admit a very concise canonical representation. Despite the fact that this class is not recursively enumerable, it is efficiently identifiable in the limit by using the algorithm DEES, introduced by the authors in a previous paper. However, the identification is not proper and before the convergence of the algorithm, DEES can produce MA that do not define stochastic languages. Nevertheless, it is possible to use these MA to define stochastic languages. We show that they belong to a broader class of rational series, that we call pseudo-stochastic rational languages. The aim of this paper is twofold. First we provide a theoretical study of pseudo-stochastic rational languages, the languages output by DEES, showing for example that this class is decidable within polynomial time. Second, we have carried out experiments to compare DEES to classical inference algorithms (ALERGIA and MDI). They show that DEES outperforms them in most cases.
Keywordspseudo-stochastic rational languages multiplicity automata probabilistic grammatical inference
Unable to display preview. Download preview PDF.
- 1.Berstel, J., Reutenauer, C.: Les séries rationnelles et leurs langages. Masson (1984)Google Scholar
- 3.Carrasco, R.C., Oncina, J.: Learning stochastic regular grammars by means of a state merging method. In: Proceedings of ICGI 1994. LNCS (LNAI), pp. 139–150. Springer, Heidelberg (1994)Google Scholar
- 5.Denis, F., Esposito, Y.: Rational stochastic language. Technical report, LIF - Université de Provence (2006), http://fr.arxiv.org/abs/cs.LG/0602093
- 7.Gantmacher, F.R.: Théorie des matrices, tomes 1 et 2. Dunod (1966)Google Scholar
- 8.Habrard, A., Denis, F., Esposito, Y.: Using pseudo-stochastic rational languages in probabilistic grammatical inference (2006) (extended version),http://fr.arxiv.org/
- 10.Thollard, F., Dupont, P., de la Higuera, C.: Probabilistic dfa inference using kullback–leibler divergence and minimality. In: Proceedings of ICML 2000, pp. 975–982 (June 2000)Google Scholar