Improving Probabilistic Automata Learning with Additional Knowledge

  • Christopher Kermorvant
  • Colin de la Higuera
  • Pierre Dupont
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3138)


In this paper, we propose a way of incorporating additional knowledge in probabilistic automata inference, by using typed automata. We compare two kinds of knowledge that are introduced into the learning algorithms. A statistical clustering algorithm and a part-of-speech tagger are used to label the data according to statistical or syntactic information automatically obtained from the data. The labeled data is then used to infer correctly typed automata. The inference of typed automata with statistically labeled data provides language models competitive with state-of-the-art n-grams on the Air Travel Information System (ATIS) task.


Statistical Cluster Typing Function Regular Language Inference Algorithm Additional Knowledge 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. [Bri92]
    Brill, E.: A simple rule-based part-of-speech tagger. In: Proc. of the Conf. on Applied Natural Language Processing, pp. 152–155 (1992)Google Scholar
  2. [CO94]
    Carrasco, R.C., Oncina, J.: Learning stochastic regular grammars by means of a state merging method. In: Carrasco, R.C., Oncina, J. (eds.) ICGI 1994. LNCS, vol. 862, pp. 139–152. Springer, Heidelberg (1994)Google Scholar
  3. [DC98]
    Dupont, P., Chase, L.: Using symbol clustering to improve probabilistic automaton inference. In: Honavar, V.G., Slutzki, G. (eds.) ICGI 1998. LNCS (LNAI), vol. 1433, pp. 232–243. Springer, Heidelberg (1998)CrossRefGoogle Scholar
  4. [DMV94]
    Dupont, P., Miclet, L., Vidal, E.: What is the search space of the regular inference? In: Carrasco, R.C., Oncina, J. (eds.) ICGI 1994. LNCS, vol. 862, pp. 25–37. Springer, Heidelberg (1994)Google Scholar
  5. [Gol67]
    Gold, E.M.: Language identification in the limit. Information and Control 10(5), 447–474 (1967)zbMATHCrossRefGoogle Scholar
  6. [KdlH02]
    Kermorvant, C., de la Higuera, C.: Learning languages with help. In: Adriaans, P.W., Fernau, H., van Zaanen, M. (eds.) ICGI 2002. LNCS (LNAI), vol. 2484, pp. 161–173. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  7. [KN95]
    Kneser, R., Ney, H.: Improved backing-off for N-gram language modeling. In: Proc. Int. Conf. on Acoustics, Speech and Signal Processing, pp. 181–184 (1995)Google Scholar
  8. [LVC02]
    Llorens, D., Vilar, J.M., Casacuberta, F.: Finite state language models smoothed using N-grams. Int. J. of Pattern Recognition and Artificial Intelligence 16(3), 275–289 (2002)CrossRefGoogle Scholar
  9. [OG92]
    Oncina, J., García, P.: Identifying regular languages in polynomial time. In: Bunke, H. (ed.) Adv. in Structural and Syntactic Pattern Recognition, pp. 99–108. World Scientific, Singapore (1992)Google Scholar
  10. [RST95]
    Ron, D., Singer, Y., Tishby, N.: On the learnability and usage of acyclic probabilistic finite automata. In: Proc. of the Annual Conference on Computational Learning Theory, pp. 31–40. ACM Press, New York (1995)CrossRefGoogle Scholar
  11. [SO94]
    Stolcke, A., Omohundro, S.: Inducing probabilistic grammars by bayesian model merging. In: Proc. Int. Coll. on Grammatical Inference. LNCS (LNAI), pp. 106–118. Springer, Heidelberg (1994)Google Scholar
  12. [TDdlH00]
    Thollard, F., Dupont, P., de la Higuera, C.: Probabilistic DFA inference using Kullback-Leibler divergence and minimality. In: Proc. Int. Conf. on Machine Learning, pp. 975–982. Morgan Kaufmann, San Francisco (2000)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2004

Authors and Affiliations

  • Christopher Kermorvant
    • 1
  • Colin de la Higuera
    • 2
  • Pierre Dupont
    • 3
  1. 1.Dept. IROUniversité de MontréalCanada
  2. 2.EURISEUniversité Jean MonnetSaint-EtienneFrance
  3. 3.INGIUniversité de LouvainLouvain-la-NeuveBelgique

Personalised recommendations