Advertisement

Smoothing Probabilistic Automata: An Error-Correcting Approach

  • Pierre Dupont
  • Juan-Carlos Amengual
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1891)

Abstract

In this paper we address the issue of smoothing the probability distribution defined by a probabilistic automaton. As inferring a probabilistic automaton is a statistical estimation problem, the usual data sparseness problem arises. We propose here the use of an error correcting technique for smoothing automata. This technique is based on a symbol dependent error model which guarantees that any possible string can be predicted with a non-zero probability. We detail how to define a consistent distribution after extending the original probabilistic automaton with error transitions. We show how to estimate the error model’s free parameters from independent data. Experiments on the ATIS travel information task show a 48 % test set perplexity reduction on new data with respect to a simply smoothed version of the original automaton.

Keywords

Error Model Viterbi Algorithm Editing Operation Error Transition Levenshtein Distance 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Amengual, J.-C., Vidal, E.: Efficient error-correcting viterbi parsing. IEEE Transactions on Pattern Analysis and Machine Intelligence, PAMI 20(10) (October 1998)Google Scholar
  2. 2.
    Amengual, J.-C., Vidal, E., Benedí, J.-M.: Simplifying language through error correcting techniques. In: International Conference on Spoken Language Processing, pp. 841–844 (1996)Google Scholar
  3. 3.
    Carrasco, R., Oncina, J.: Learning stochastic regular grammars by means of a state merging method. In: Carrasco, R.C., Oncina, J. (eds.) ICGI 1994. LNCS (LNAI), vol. 862, pp. 139–150. Springer, Heidelberg (1994)Google Scholar
  4. 4.
    Carrasco, R., Oncina, J.: Learning deterministic regular gramars from stochastic samples in polynomial time. Theoretical Informatics and Applications 33(1), 1–19 (1999)zbMATHCrossRefMathSciNetGoogle Scholar
  5. 5.
    Dupont, P., Chase, L.: Using symbol clustering to improve probabilistic automaton inference. In: Honavar, V.G., Slutzki, G. (eds.) ICGI 1998. LNCS (LNAI), vol. 1433, pp. 232–243. Springer, Heidelberg (1998)CrossRefGoogle Scholar
  6. 6.
    Forney, G.D.: The Viterbi algorithm. IEEE Proceedings 3, 268–278 (1973)CrossRefMathSciNetGoogle Scholar
  7. 7.
    Freitag, D.: Using grammatical inference to improve precision in information extraction. In: Workshop on Automata Induction, Grammatical Inference, and Language Acquisition, Fourteenth International Conference on Machine Learning, Nashville, Tennessee (1997)Google Scholar
  8. 8.
    Hart, G.W., Bouloutas, A.: Correcting dependent errors in sequences generated by finite-state processes. IEEE Trans. on Information Theory 39(4), 1249–1260 (1993)zbMATHCrossRefGoogle Scholar
  9. 9.
    Hirschman, L.: Multi-site data collection for a spoken language corpus. In: Proceedings of DARPA Speech and Natural Language Workshop, pp. 7–14. Arden House, NY (1992)CrossRefGoogle Scholar
  10. 10.
    Kita, K., Fukui, Y., Nagata, M., Morimoto, T.: Automatic acquisition of probabilistic dialogue models. In: Proceedings of ISSD 1996, workshop of the International Conference on Spoken Language Processing, Philadelphia, October 1996, pp. 196–199 (1996)Google Scholar
  11. 11.
    Kruskal, J.B.: An overview of sequence comparison. In: Sankoff, D., Kruskal, J.B. (eds.) Time Warps, String Edits, and Macromolecules: the Theory and Practice of Sequence Comparison, pp. 1–44. Addison-Wesley, Reading (1983)Google Scholar
  12. 12.
    Lang, K.J., Pearlmutter, B.A., Price, R.A.: Results of the abbadingo one DFA learning competition and a new evidence-driven state merging algorithm. In: Honavar, V.G., Slutzki, G. (eds.) ICGI 1998. LNCS (LNAI), vol. 1433, pp. 1–12. Springer, Heidelberg (1998)CrossRefGoogle Scholar
  13. 13.
    Ney, H., Essen, U., Kneser, R.: On structuring probabilistic dependences in stochastic language modelling. Computer Speech and Language 8, 1–38 (1994)CrossRefGoogle Scholar
  14. 14.
    Oncina, J., García, P.: Inferring regular languages in polynomial update time. In: Pérez de la Blanca, N., Sanfeliu, A., Vidal, E. (eds.) Pattern Recognition and Image Analysis. Series in Machine Perception and Artificial Intelligence, vol. 1, pp. 49–61. World Scientific, Singapore (1992)CrossRefGoogle Scholar
  15. 15.
    Ron, D., Singer, Y., Tishby, N.: On the learnability and usage of acyclic probabilistic automata. In: Proceedings of the Eighth Annual Conference on Computational Learning Theory, Santa Cruz, CA, pp. 31–40. ACM Press, New York (1995)CrossRefGoogle Scholar
  16. 16.
    Rulot, H., Vidal, E.: An efficient algorithm for the inference of circuit-free automata. In: Ferraté, G., Pavlidis, T., Sanfeliu, A., Bunke, H. (eds.) Advances in Structural and Syntactic Pattern Recognition. NATO ASI, pp. 173–184. Springer, Heidelberg (1988)Google Scholar
  17. 17.
    Young-Lai, M., Tompa, F.: Stochastic grammatical inference of text database structure. To appear in Machine Learning (2000)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2000

Authors and Affiliations

  • Pierre Dupont
    • 1
  • Juan-Carlos Amengual
    • 2
  1. 1.EURISEUniversité Jean MonnetSaint-Etienne CedexFrance
  2. 2.Campus de Riu SecUniversidad Jaume I de CastellónCastellónSpain

Personalised recommendations