Advertisement

Learning Classes of Probabilistic Automata

  • François Denis
  • Yann Esposito
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3120)

Abstract

Probabilistic finite automata (PFA) model stochastic languages, i.e. probability distributions over strings. Inferring PFA from stochastic data is an open field of research. We show that PFA are identifiable in the limit with probability one. Multiplicity automata (MA) is another device to represent stochastic languages. We show that a MA may generate a stochastic language that cannot be generated by a PFA, but we show also that it is undecidable whether a MA generates a stochastic language. Finally, we propose a learning algorithm for a subclass of PFA, called PRFA.

Keywords

Convex Generator Learn Class Membership Query Complete Presentation Return Transition 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
This is a preview of subscription content, log in to check access.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Paz, A.: Introduction to probabilistic automata. Academic Press, London (1971)zbMATHGoogle Scholar
  2. 2.
    Abe, N., Warmuth, M.: On the computational complexity of approximating distributions by probabilistic automata. Machine Learning 9, 205–260 (1992)zbMATHGoogle Scholar
  3. 3.
    Dempster, A., Laird, N.M., Rubin, D.B.: Maximum likelyhood from incomplete data via the em algorithm. Journal of the Royal Statistical Society 39, 1–38 (1977)zbMATHMathSciNetGoogle Scholar
  4. 4.
    Baldi, P., Brunak, S.: Bioinformatics: The Machine Learning Approach. MIT Press, Cambridge (1998)Google Scholar
  5. 5.
    Freitag, D., McCallum, A.: Information extraction with HMM structures learned by stochastic optimization. In: AAAI/IAAI, pp. 584–589 (2000)Google Scholar
  6. 6.
    Gold, E.: Language identification in the limit. Inform. Control 10, 447–474 (1967)zbMATHCrossRefGoogle Scholar
  7. 7.
    Angluin, D.: Identifying languages from stochastic examples. Technical Report YALEU/DCS/RR-614, Yale University, New Haven, CT (1988) Google Scholar
  8. 8.
    Carrasco, R., Oncina, J.: Learning stochastic regular grammars by means of a state merging method. In: ICGI, pp. 139–152. Springer, Heidelberg (1994)Google Scholar
  9. 9.
    Carrasco, R.C., Oncina, J.: Learning deterministic regular grammars from stochastic samples in polynomial time. RAIRO 33, 1–20 (1999)zbMATHCrossRefMathSciNetGoogle Scholar
  10. 10.
    de la Higuera, C., Thollard, F.: Identification in the limit with probability one of stochastic deterministic finite automata. In: Oliveira, A.L. (ed.) ICGI 2000. LNCS (LNAI), vol. 1891, pp. 141–156. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  11. 11.
    Bergadano, F., Varricchio, S.: Learning behaviors of automata from multiplicity and equivalence queries. In: Italian Conf. on Algorithms and Complexity (1994)Google Scholar
  12. 12.
    Beimel, A., Bergadano, F., Bshouty, N.H., Kushilevitz, E., Varricchio, S.: On the applications of multiplicity automata in learning. In: IEEE Symposium on Foundations of Computer Science, pp. 349–358 (1996)Google Scholar
  13. 13.
    Beimel, A., Bergadano, F., Bshouty, N.H., Kushilevitz, E., Varricchio, S.: Learning functions represented as multiplicity automata. Journal of the ACM 47, 506–530 (2000)zbMATHCrossRefMathSciNetGoogle Scholar
  14. 14.
    Thollard, F., Dupont, P., de la Higuera, C.: In: Proc. 17th ICML in titleGoogle Scholar
  15. 15.
    Kearns, M., Mansour, Y., Ron, D., Rubinfeld, R., Schapire, R.E., Sellie, L.: On the learnability of discrete distributions, 273–282 (1994)Google Scholar
  16. 16.
    Esposito, Y., Lemay, A., Denis, F., Dupont, P.: Learning probabilistic residual finite state automata. In: Adriaans, P.W., Fernau, H., van Zaanen, M. (eds.) ICGI 2002. LNCS (LNAI), vol. 2484, p. 77. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  17. 17.
    Denis, F., Esposito, Y.: Residual languages and probabilistic automata. In: ICALP 2003, Springer, Heidelberg (2003)Google Scholar
  18. 18.
    Angluin, D.: Queries and concept learning. Machine Learning 2, 319–342 (1988)Google Scholar
  19. 19.
    Vapnik, V.N.: Statistical Learning Theory. John Wiley, Chichester (1998)zbMATHGoogle Scholar
  20. 20.
    Lugosi, G.: Pattern classification and learning theory. In: Principles of Nonparametric Learning., pp. 1–56. Springer, Heidelberg (2002)Google Scholar
  21. 21.
    Hardy, G.H., Wright, E.M.: An introduction to the theory of numbers. Oxford University Press, Oxford (1979)zbMATHGoogle Scholar
  22. 22.
    Blondel, V.D., Canterini, V.: Undecidable problems for probabilistic automata of fixed dimension. Theory of Computing Systems 36, 231–245 (2003)zbMATHCrossRefMathSciNetGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2004

Authors and Affiliations

  • François Denis
    • 1
  • Yann Esposito
    • 1
  1. 1.LIF-CMIMarseille Cedex 13France

Personalised recommendations

Cite paper

Buy options