Learning Stochastic Finite Automata

  • Colin de la Higuera
  • Jose Oncina
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3264)


Stochastic deterministic finite automata have been introduced and are used in a variety of settings. We report here a number of results concerning the learnability of these finite state machines. In the setting of identification in the limit with probability one, we prove that stochastic deterministic finite automata cannot be identified from only a polynomial quantity of data. If concerned with approximation results, they become Pac-learnable if the L  ∞  norm is used. We also investigate queries that are sufficient for the class to be learnable.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Paz, A.: Introduction to probabilistic automata. Academic Press, NY (1971)Google Scholar
  2. 2.
    Lucas, S., Vidal, E., Amari, A., Hanlon, S., Amengual, J.C.: A comparison of syntactic and statistical techniques for off-line OCR.In: [18] ,pp.168–179 Google Scholar
  3. 3.
    Lyngsø, R.B., Pedersen, C.N.S., Nielsen, H.: Metrics and similarity measures for hidden Markov models. In: Blaschke, C., Shatkay, H. (eds.) Linking Literature, Information, and Knowledge for Biology. LNCS, vol. 6004, pp. 178–186. Springer, Heidelberg (2010)Google Scholar
  4. 4.
    Mohri, M.: Finite-state transducers in language and speech processing. Computational Linguistics 23, 269–311 (1997)MathSciNetGoogle Scholar
  5. 5.
    Kearns, M.J., Mansour, Y., Ron, D., Rubinfeld, R., Schapire, R.E., Sellie, L.: On the learnability of discrete distributions. In: Proc. of the 25th Annual ACM Symposium on Theory of Computing, pp. 273–282 (1994)Google Scholar
  6. 6.
    Carrasco, R.C., Oncina, J.: Learning stochastic regular grammars by means of a state merging method. In: [18] ,pp.139–150Google Scholar
  7. 7.
    Ron, D., Singer, Y., Tishby, N.: On the learnability and usage of acyclic probabilistic finite automata. In: Proceedings of COLT, pp. 31–40 (1995)Google Scholar
  8. 8.
    Harrison, M.H.: Introduction to Formal Language Theory. Addison-Wesley Publishing Company, Inc., Reading (1978)zbMATHGoogle Scholar
  9. 9.
    Angluin, D.: Identifying languages from stochastic examples. Technical Report YALEU/DCS/RR-614, Yale University (1988)Google Scholar
  10. 10.
    de la Higuera, C., Oncina, J.: Identification with probability one of stochastic deterministic linear languages. In: Gavaldá, R., Jantke, K.P., Takimoto, E. (eds.) ALT 2003. LNCS (LNAI), vol. 2842, pp. 134–148. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  11. 11.
    Pitt, L.: Inductive inference, DFA’s, and computational complexity. In: Jantke, K.P. (ed.) AII 1989. LNCS, vol. 397, pp. 18–44. Springer, Heidelberg (1989)Google Scholar
  12. 12.
    Valiant, L.G.: A theory of the learnable. Communications of the Association for Computing Machinery 27, 1134–1142 (1984)zbMATHGoogle Scholar
  13. 13.
    Angluin, D.: Queries and concept learning. Machine Learning Journal 2, 319–342 (1987)Google Scholar
  14. 14.
    Bergadano, F., Varricchio, S.: Learning behaviors of automata from multiplicity and equivalence queries. SIAM Journal of Computation 25, 1268–1280 (1996)zbMATHCrossRefMathSciNetGoogle Scholar
  15. 15.
    Angluin, D.: A note on the number of queries needed to identify regular languages. Information and Control 51, 76–87 (1981)zbMATHCrossRefMathSciNetGoogle Scholar
  16. 16.
    Carrasco, R.C., Oncina, J.: Learning deterministic regular grammars from stochastic samples in polynomial time. RAIRO (Theoretical Informatics and Applications) 33, 1–20 (1999)zbMATHCrossRefMathSciNetGoogle Scholar
  17. 17.
    Carlyle, J.W., Paz, A.: Realizations by stochastic finite automata. Journal of Computation and System Sciences, 26–40 (1971)Google Scholar
  18. 18.
    Carrasco, R.C., Oncina, J.: Grammatical Inference and Applications. In: Carrasco, R.C., Oncina, J. (eds.) ICGI 1994. LNCS, vol. 862, Springer, Heidelberg (1994)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2004

Authors and Affiliations

  • Colin de la Higuera
    • 1
  • Jose Oncina
    • 2
  1. 1.EURISEUniversité de Saint-EtienneSaint-EtienneFrance
  2. 2.Departamento de Lenguajes y Sistemas InformaticosUniversidad de AlicanteAlicanteSpain

Personalised recommendations