Using Pseudo-stochastic Rational Languages in Probabilistic Grammatical Inference

  • Amaury Habrard
  • François Denis
  • Yann Esposito
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4201)

Abstract

In probabilistic grammatical inference, a usual goal is to infer a good approximation of an unknown distribution P called a stochastic language. The estimate of P stands in some class of probabilistic models such as probabilistic automata (PA). In this paper, we focus on probabilistic models based on multiplicity automata (MA). The stochastic languages generated by MA are called rational stochastic languages; they strictly include stochastic languages generated by PA and admit a very concise canonical representation. Despite the fact that this class is not recursively enumerable, it is efficiently identifiable in the limit by using the algorithm DEES, introduced by the authors in a previous paper. However, the identification is not proper and before the convergence of the algorithm, DEES can produce MA that do not define stochastic languages. Nevertheless, it is possible to use these MA to define stochastic languages. We show that they belong to a broader class of rational series, that we call pseudo-stochastic rational languages. The aim of this paper is twofold. First we provide a theoretical study of pseudo-stochastic rational languages, the languages output by DEES, showing for example that this class is decidable within polynomial time. Second, we have carried out experiments to compare DEES to classical inference algorithms (ALERGIA and MDI). They show that DEES outperforms them in most cases.

Keywords

pseudo-stochastic rational languages multiplicity automata probabilistic grammatical inference 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Berstel, J., Reutenauer, C.: Les séries rationnelles et leurs langages. Masson (1984)Google Scholar
  2. 2.
    Blondel, V.D., Tsitsiklis, J.N.: A survey of computational complexity results in systems and control. Automatica 36(9), 1249–1274 (2000)MATHCrossRefMathSciNetGoogle Scholar
  3. 3.
    Carrasco, R.C., Oncina, J.: Learning stochastic regular grammars by means of a state merging method. In: Proceedings of ICGI 1994. LNCS (LNAI), pp. 139–150. Springer, Heidelberg (1994)Google Scholar
  4. 4.
    Denis, F., Esposito, Y.: Learning classes of probabilistic automata. In: Shawe-Taylor, J., Singer, Y. (eds.) COLT 2004. LNCS (LNAI), vol. 3120, pp. 124–139. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  5. 5.
    Denis, F., Esposito, Y.: Rational stochastic language. Technical report, LIF - Université de Provence (2006), http://fr.arxiv.org/abs/cs.LG/0602093
  6. 6.
    Denis, F., Esposito, Y., Habrard, A.: Learning rational stochastic languages. In: Lugosi, G., Simon, H.U. (eds.) COLT 2006. LNCS (LNAI), vol. 4005, pp. 274–288. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  7. 7.
    Gantmacher, F.R.: Théorie des matrices, tomes 1 et 2. Dunod (1966)Google Scholar
  8. 8.
    Habrard, A., Denis, F., Esposito, Y.: Using pseudo-stochastic rational languages in probabilistic grammatical inference (2006) (extended version),http://fr.arxiv.org/
  9. 9.
    Salomaa, A., Soittola, M.: Automata: Theoretic Aspects of Formal Power Series. Springer, Heidelberg (1978)MATHGoogle Scholar
  10. 10.
    Thollard, F., Dupont, P., de la Higuera, C.: Probabilistic dfa inference using kullback–leibler divergence and minimality. In: Proceedings of ICML 2000, pp. 975–982 (June 2000)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Amaury Habrard
    • 1
  • François Denis
    • 1
  • Yann Esposito
    • 1
  1. 1.Laboratoire d’Informatique Fondamentale de Marseille (L.I.F.) UMR CNRS 6166 

Personalised recommendations