3-Way Composition of Weighted Finite-State Transducers

  • Cyril Allauzen
  • Mehryar Mohri
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5148)

Abstract

Composition of weighted transducers is a fundamental algorithm used in many applications, including for computing complex edit-distances between automata, or string kernels in machine learning, or to combine different components of a speech recognition, speech synthesis, or information extraction system. We present a generalization of the composition of weighted transducers, 3-way composition, which is dramatically faster in practice than the standard composition algorithm when combining more than two transducers. The worst-case complexity of our algorithm for composing three transducers T1, T2, and T3 resulting in T, is O(|T|Q min (d(T1) d(T3), d(T2)) + |T|E), where |·|Q denotes the number of states, |·|E the number of transitions, and d(·) the maximum out-degree. As in regular composition, the use of perfect hashing requires a pre-processing step with linear-time expected complexity in the size of the input transducers. In many cases, this approach significantly improves on the complexity of standard composition. Our algorithm also leads to a dramatically faster composition in practice. Furthermore, standard composition can be obtained as a special case of our algorithm. We report the results of several experiments demonstrating this improvement. These theoretical and empirical improvements significantly enhance performance in the applications already mentioned.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Berstel, J.: Transductions and Context-Free Languages. Teubner (1979)Google Scholar
  2. 2.
    Chen, S., Goodman, J.: An empirical study of smoothing techniques for language modeling. Technical Report, TR-10-98, Harvard University (1998)Google Scholar
  3. 3.
    Cortes, C., Haffner, P., Mohri, M.: Rational Kernels: Theory and Algorithms. Journal of Machine Learning Research 5, 1035–1062 (2004)MathSciNetGoogle Scholar
  4. 4.
    Culik II, K., Kari, J.: Digital Images and Formal Languages. In: Rozenberg, G., Salomaa, A. (eds.) Handbook of Formal Languages, vol. 3, pp. 599–616. Springer, Heidelberg (1997)Google Scholar
  5. 5.
    Eilenberg, S.: Automata, Languages and Machines. Academic Press, London (1974–1976)Google Scholar
  6. 6.
    Katz, S.M.: Estimation of probabilities from sparse data for the language model component of a speech recogniser. IEEE Transactions on Acoustic, Speech, and Signal Processing 35(3), 400–401 (1987)CrossRefGoogle Scholar
  7. 7.
    Kuich, W., Salomaa, A.: Semirings, Automata, Languages. Springer, Heidelberg (1986)MATHGoogle Scholar
  8. 8.
    Mohri, M.: Finite-State Transducers in Language and Speech Processing. Computational Linguistics 23(2) (1997)Google Scholar
  9. 9.
    Mohri, M.: Edit-Distance of Weighted Automata: General Definitions and Algorithms. Int. J. Found. Comput. Sci. 14(6), 957–982 (2003)MATHCrossRefMathSciNetGoogle Scholar
  10. 10.
    Mohri, M.: Statistical Natural Language Processing. In: Lothaire, M. (ed.) Applied Combinatorics on Words. Cambridge University Press, Cambridge (2005)Google Scholar
  11. 11.
    Mohri, M., Pereira, F.C.N., Riley, M.: Weighted Automata in Text and Speech Processing. In: Proceedings of the 12th biennial European Conference on Artificial Intelligence (ECAI 1996). John Wiley and Sons, Chichester (1996)Google Scholar
  12. 12.
    Pereira, F., Riley, M.: Finite State Language Processing. In: Speech Recognition by Composition of Weighted Finite Automata. The MIT Press, Cambridge (1997)Google Scholar
  13. 13.
    Perrin, D.: Words. In: Lothaire, M. (ed.) Combinatorics on words, Cambridge Mathematical Library. Cambridge University Press, Cambridge (1997)Google Scholar
  14. 14.
    Salomaa, A., Soittola, M.: Automata-Theoretic Aspects of Formal Power Series. Springer, Heidelberg (1978)MATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Cyril Allauzen
    • 1
  • Mehryar Mohri
    • 1
    • 2
  1. 1.Courant Institute of Mathematical SciencesNew YorkUSA
  2. 2.Google ResearchNew YorkUSA

Personalised recommendations