Learning Weighted Automata

  • Borja Balle
  • Mehryar MohriEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9270)


Weighted finite automata (WFA) are finite automata whose transitions and states are augmented with some weights, elements of a semiring. A WFA induces a function over strings. The value it assigns to an input string is the semiring sum of the weights of all paths labeled with that string, where the weight of a path is obtained by taking the semiring product of the weights of its constituent transitions, as well as those of its origin and destination states.


Hide Markov Model Singular Value Decomposition Transition Weight Hankel Matrix Nuclear Norm 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Allauzen, C., Mohri, M., Riley, M.: Statistical modeling for unit selection in speech synthesis. In: Proceedings of ACL (2004)Google Scholar
  2. 2.
    Allauzen, C., Mohri, M., Talwalkar, A.: Sequence kernels for predicting protein essentiality. In: Proceedings of ICML (2008)Google Scholar
  3. 3.
    Aminof, B., Kupferman, O., Lampert, R.: Formal analysis of online algorithms. In: Bultan, T., Hsiung, P.-A. (eds.) ATVA 2011. LNCS, vol. 6996, pp. 213–227. Springer, Heidelberg (2011) CrossRefGoogle Scholar
  4. 4.
    Angluin, D.: On the complexity of minimum inference of regular sets. Information and Control 3(39) (1978)Google Scholar
  5. 5.
    Angluin, D.: Learning regular sets from queries and counterexamples. Information and Computation 75(2) (1987)Google Scholar
  6. 6.
    Bailly, R.: Méthodes spectrales pour l’inférence grammaticale probabiliste de langages stochastiques rationnels. Ph.D. thesis, Aix-Marseille Université (2011)Google Scholar
  7. 7.
    Bailly, R., Denis, F., Ralaivola, L.: Grammatical inference as a principal component analysis problem. In: Proceedings of ICML (2009)Google Scholar
  8. 8.
    Balle, B., Mohri, M.: On the Rademacher complexity of weighted automata. In: Proceedings of ALT (2015)Google Scholar
  9. 9.
    Balle, B.: Learning Finite-State Machines: Statistical and Algorithmic Aspects. Ph.D. thesis, Universitat Politecnica de Catalunya (2013)Google Scholar
  10. 10.
    Balle, B., Carreras, X., Luque, F.M., Quattoni, A.: Spectral learning of weighted automata. Machine Learning 96(1–2) (2014)Google Scholar
  11. 11.
    Balle, B., Mohri, M.: Spectral learning of general weighted automata via constrained matrix completion. In: Proceedings of NIPS (2012)Google Scholar
  12. 12.
    Beimel, A., Bergadano, F., Bshouty, N.H., Kushilevitz, E., Varricchio, S.: On the applications of multiplicity automata in learning. In: Proceeding FOCS (1996)Google Scholar
  13. 13.
    Beimel, A., Bergadano, F., Bshouty, N.H., Kushilevitz, E., Varricchio, S.: Learning functions represented as multiplicity automata. Journal of the ACM 47(3) (2000)Google Scholar
  14. 14.
    Bergadano, F., Varricchio, S.: Learning behaviors of automata from multiplicity and equivalence queries. In: Bonuccelli, M.A., Crescenzi, P., Petreschi, R. (eds.) CIAC 1994. LNCS, vol. 778, pp. 54–62. Springer, Heidelberg (1994)Google Scholar
  15. 15.
    Bergadano, F., Varricchio, S.: Learning behaviors of automata from multiplicity and equivalence queries. SIAM Journal on Computing 25(6) (1996)Google Scholar
  16. 16.
    Berstel, J., Reutenauer, C.: Rational Series and Their Languages. Springer (1988)Google Scholar
  17. 17.
    Bisht, L., Bshouty, N.H., Mazzawi, H.: On optimal learning algorithms for multiplicity automata. In: Lugosi, G., Simon, H.U. (eds.) COLT 2006. LNCS (LNAI), vol. 4005, pp. 184–198. Springer, Heidelberg (2006) CrossRefGoogle Scholar
  18. 18.
    Breuel, T.M.: The OCRopus open source OCR system. In: Proceedings of IS&T/SPIE (2008)Google Scholar
  19. 19.
    Cardon, A., Crochemore, M.: Détermination de la représentation standard d’une série reconnaissable. ITA 14(4), 371–379 (1980)zbMATHGoogle Scholar
  20. 20.
    Carlyle, J.W., Paz, A.: Realizations by stochastic finite automata. J. Comput. Syst. Sci. 5(1) (1971)Google Scholar
  21. 21.
    Chalermsook, P., Laekhanukit, B., Nanongkai, D.: Pre-reduction graph products: Hardnesses of properly learning dfas and approximating edp on dags. In: Proceedings of FOCS (2014)Google Scholar
  22. 22.
    Cortes, C., Haffner, P., Mohri, M.: Rational kernels: Theory and algorithms. Journal of Machine Learning Research 5 (2004)Google Scholar
  23. 23.
    Droste, M., Kuich, W. (eds.): Handbook of weighted automata. EATCS Monographs on Theoretical Computer Science. Springer (2009)Google Scholar
  24. 24.
    Dupont, P., Denis, F., Esposito, Y.: Links between probabilistic automata and hidden markov models: probability distributions, learning models and induction algorithms. Pattern Recognition (2005)Google Scholar
  25. 25.
    Durbin, R., Eddy, S.R., Krogh, A., Mitchison, G.J.: Biological Sequence Analysis: Probabilistic Models of Proteins and Nucleic Acids. Cambridge University Press (1998)Google Scholar
  26. 26.
    Eilenberg, S.: Automata, Languages and Machines. Academic Press (1974)Google Scholar
  27. 27.
    Fazel, M.: Matrix rank minimization with applications. Ph.D. thesis, Stanford University (2002)Google Scholar
  28. 28.
    Fliess, M.: Matrices de Hankel. Journal de Mathématiques Pures et Appliquées 53 (1974)Google Scholar
  29. 29.
    Gold, E.M.: Complexity of automaton identification from given data. Information and Control 3(37) (1978)Google Scholar
  30. 30.
    Golub, G., Loan, C.V.: Matrix Computations. Johns Hopkins University Press (1983)Google Scholar
  31. 31.
    Haussler, D., Littlestone, N., Warmuth, M.K.: Predicting \(\{0,1\}\)-functions on randomly drawn points. In: Proceedings of COLT (1988)Google Scholar
  32. 32.
    Hsu, D., Kakade, S.M., Zhang, T.: A spectral algorithm for learning hidden markov models. In: Proceedings of COLT (2009)Google Scholar
  33. 33.
    Hsu, D., Kakade, S.M., Zhang, T.: A spectral algorithm for learning hidden markov models. Journal of Computer and System Sciences 78(5) (2012)Google Scholar
  34. 34.
    II, K.C., Kari, J.: Image compression using weighted finite automata. Computers & Graphics 17(3) (1993)Google Scholar
  35. 35.
    Kaplan, R.M., Kay, M.: Regular models of phonological rule systems. Computational Linguistics 20(3) (1994)Google Scholar
  36. 36.
    Karttunen, L.: The replace operator. In: Proceedings of ACL (1995)Google Scholar
  37. 37.
    Kearns, M.J., Valiant, L.G.: Cryptographic limitations on learning boolean formulae and finite automata. Journal of ACM 41(1) (1994)Google Scholar
  38. 38.
    Kearns, M.J., Vazirani, U.V.: An Introduction to Computational Learning Theory. MIT Press (1994)Google Scholar
  39. 39.
    Kiefer, S., Marusic, I., Worrell, J.: Minimisation of multiplicity tree automata. In: Pitts, A. (ed.) FOSSACS 2015. LNCS, vol. 9034, pp. 297–311. Springer, Heidelberg (2015) CrossRefGoogle Scholar
  40. 40.
    Kuich, W., Salomaa, A.: Semirings, Automata. Springer, Languages (1986)Google Scholar
  41. 41.
    Mohri, M.: Finite-state transducers in language and speech processing. Computational Linguistics 23(2) (1997)Google Scholar
  42. 42.
    Mohri, M.: Weighted automata algorithms. In: Handbook of Weighted Automata. Springer (2009)Google Scholar
  43. 43.
    Mohri, M., Pereira, F., Riley, M.: Weighted automata in text and speech processing. In: Proceedings of ECAI 1996 Workshop on Extended finite state models of language (1996)Google Scholar
  44. 44.
    Mohri, M., Pereira, F., Riley, M.: Speech recognition with weighted finite-state transducers. In: Handbook on Speech Processing and Speech Comm. Springer (2008)Google Scholar
  45. 45.
    Mohri, M., Pereira, F.C.N.: Dynamic compilation of weighted context-free grammars. In: Proceedings of COLING-ACL (1998)Google Scholar
  46. 46.
    Mohri, M., Rostamizadeh, A., Talwalkar, A.: Foundations of Machine Learning. MIT Press (2012)Google Scholar
  47. 47.
    Mohri, M., Sproat, R.: An efficient compiler for weighted rewrite rules. In: Proceedings of ACL (1996)Google Scholar
  48. 48.
    Mornhinweg, D., Shapiro, D.B., Valente, K.: The principal axis theorem over arbitrary fields. American Mathematical Monthly (1993)Google Scholar
  49. 49.
    Pereira, F., Riley, M.: Speech recognition by composition of weighted finite automata. In: Finite-State Language Processing. MIT Press (1997)Google Scholar
  50. 50.
    Pitt, L., Warmuth, M.K.: The minimum consistent DFA problem cannot be approximated within any polynomial. J. ACM 40(1) (1993)Google Scholar
  51. 51.
    Quattoni, A., Balle, B., Carreras, X., Globerson, A.: Spectral regularization for max-margin sequence tagging. In: Proceedings of ICML (2014)Google Scholar
  52. 52.
    Salomaa, A., Soittola, M.: Automata-Theoretic Aspects of Formal Power Series. Springer (1978)Google Scholar
  53. 53.
    Schützenberger, M.P.: On a special class of recurrent events. The Annals of Mathematical Statistics 32(4) (1961)Google Scholar
  54. 54.
    Schützenberger, M.P.: On the definition of a family of automata. Information and Control 4 (1961)Google Scholar
  55. 55.
    Sproat, R.: A finite-state architecture for tokenization and grapheme-to-phoneme conversion in multilingual text analysis. In: Proceedings of the ACL SIGDAT Workshop. ACL (1995)Google Scholar
  56. 56.
    Valiant, L.G.: A theory of the learnable. Commun. ACM 27(11) (1984)Google Scholar
  57. 57.
    Vidal, E., Thollard, F., de la Higuera, C., Casacuberta, F., Carrasco, R.C.: Probabilistic finite-state machines - part I. PAMI (2005)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  1. 1.School of Computer ScienceMcGill UniversityMontréalCanada
  2. 2.Courant Institute of Mathematical SciencesNew YorkUSA
  3. 3.Google ResearchNew YorkUSA

Personalised recommendations