Learning Classes of Probabilistic Automata
Conference paper
- 8 Citations
- 1.5k Downloads
Abstract
Probabilistic finite automata (PFA) model stochastic languages, i.e. probability distributions over strings. Inferring PFA from stochastic data is an open field of research. We show that PFA are identifiable in the limit with probability one. Multiplicity automata (MA) is another device to represent stochastic languages. We show that a MA may generate a stochastic language that cannot be generated by a PFA, but we show also that it is undecidable whether a MA generates a stochastic language. Finally, we propose a learning algorithm for a subclass of PFA, called PRFA.
Keywords
Convex Generator Learn Class Membership Query Complete Presentation Return Transition
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
Preview
Unable to display preview. Download preview PDF.
References
- 1.Paz, A.: Introduction to probabilistic automata. Academic Press, London (1971)zbMATHGoogle Scholar
- 2.Abe, N., Warmuth, M.: On the computational complexity of approximating distributions by probabilistic automata. Machine Learning 9, 205–260 (1992)zbMATHGoogle Scholar
- 3.Dempster, A., Laird, N.M., Rubin, D.B.: Maximum likelyhood from incomplete data via the em algorithm. Journal of the Royal Statistical Society 39, 1–38 (1977)zbMATHMathSciNetGoogle Scholar
- 4.Baldi, P., Brunak, S.: Bioinformatics: The Machine Learning Approach. MIT Press, Cambridge (1998)Google Scholar
- 5.Freitag, D., McCallum, A.: Information extraction with HMM structures learned by stochastic optimization. In: AAAI/IAAI, pp. 584–589 (2000)Google Scholar
- 6.Gold, E.: Language identification in the limit. Inform. Control 10, 447–474 (1967)zbMATHCrossRefGoogle Scholar
- 7.Angluin, D.: Identifying languages from stochastic examples. Technical Report YALEU/DCS/RR-614, Yale University, New Haven, CT (1988) Google Scholar
- 8.Carrasco, R., Oncina, J.: Learning stochastic regular grammars by means of a state merging method. In: ICGI, pp. 139–152. Springer, Heidelberg (1994)Google Scholar
- 9.Carrasco, R.C., Oncina, J.: Learning deterministic regular grammars from stochastic samples in polynomial time. RAIRO 33, 1–20 (1999)zbMATHCrossRefMathSciNetGoogle Scholar
- 10.de la Higuera, C., Thollard, F.: Identification in the limit with probability one of stochastic deterministic finite automata. In: Oliveira, A.L. (ed.) ICGI 2000. LNCS (LNAI), vol. 1891, pp. 141–156. Springer, Heidelberg (2000)CrossRefGoogle Scholar
- 11.Bergadano, F., Varricchio, S.: Learning behaviors of automata from multiplicity and equivalence queries. In: Italian Conf. on Algorithms and Complexity (1994)Google Scholar
- 12.Beimel, A., Bergadano, F., Bshouty, N.H., Kushilevitz, E., Varricchio, S.: On the applications of multiplicity automata in learning. In: IEEE Symposium on Foundations of Computer Science, pp. 349–358 (1996)Google Scholar
- 13.Beimel, A., Bergadano, F., Bshouty, N.H., Kushilevitz, E., Varricchio, S.: Learning functions represented as multiplicity automata. Journal of the ACM 47, 506–530 (2000)zbMATHCrossRefMathSciNetGoogle Scholar
- 14.Thollard, F., Dupont, P., de la Higuera, C.: In: Proc. 17th ICML in titleGoogle Scholar
- 15.Kearns, M., Mansour, Y., Ron, D., Rubinfeld, R., Schapire, R.E., Sellie, L.: On the learnability of discrete distributions, 273–282 (1994)Google Scholar
- 16.Esposito, Y., Lemay, A., Denis, F., Dupont, P.: Learning probabilistic residual finite state automata. In: Adriaans, P.W., Fernau, H., van Zaanen, M. (eds.) ICGI 2002. LNCS (LNAI), vol. 2484, p. 77. Springer, Heidelberg (2002)CrossRefGoogle Scholar
- 17.Denis, F., Esposito, Y.: Residual languages and probabilistic automata. In: ICALP 2003, Springer, Heidelberg (2003)Google Scholar
- 18.Angluin, D.: Queries and concept learning. Machine Learning 2, 319–342 (1988)Google Scholar
- 19.Vapnik, V.N.: Statistical Learning Theory. John Wiley, Chichester (1998)zbMATHGoogle Scholar
- 20.Lugosi, G.: Pattern classification and learning theory. In: Principles of Nonparametric Learning., pp. 1–56. Springer, Heidelberg (2002)Google Scholar
- 21.Hardy, G.H., Wright, E.M.: An introduction to the theory of numbers. Oxford University Press, Oxford (1979)zbMATHGoogle Scholar
- 22.Blondel, V.D., Canterini, V.: Undecidable problems for probabilistic automata of fixed dimension. Theory of Computing Systems 36, 231–245 (2003)zbMATHCrossRefMathSciNetGoogle Scholar
Copyright information
© Springer-Verlag Berlin Heidelberg 2004