PAC-Learnability of Probabilistic Deterministic Finite State Automata in Terms of Variation Distance

  • Nick Palmer
  • Paul W. Goldberg
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3734)


We consider the problem of PAC-learning distributions over strings, represented by probabilistic deterministic finite automata (PDFAs). PDFAs are a probabilistic model for the generation of strings of symbols, that have been used in the context of speech and handwriting recognition, and bioinformatics. Recent work on learning PDFAs from random examples has used the KL-divergence as the error measure; here we use the variation distance. We build on recent work by Clark and Thollard, and show that the use of the variation distance allows simplifications to be made to the algorithms, and also a strengthening of the results; in particular that using the variation distance, we obtain polynomial sample size bounds that are independent of the expected length of strings.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Abe, N., Takeuchi, J., Warmuth, M.: Polynomial Learnability of Stochastic Rules with respect to the KL-divergence and Quadratic Distance. IEICE Trans. Inf. and Syst. E84-D(3), 299–315 (2001)Google Scholar
  2. 2.
    Anthony, M., Bartlett, P.L.: Neural Network Learning: Theoretical Foundations. Cambridge University Press, Cambridge (1999)zbMATHCrossRefGoogle Scholar
  3. 3.
    Clark, A., Thollard, F.: PAC-learnability of Probabilistic Deterministic Finite State Automata. Journal of Machine Learning Research 5, 473–497 (2004)MathSciNetGoogle Scholar
  4. 4.
    Cover, T.M., Thomas, J.A.: Elements of Information Theory. Wiley Series in Telecommunications. John Wiley & Sons, Chichester (1991)zbMATHCrossRefGoogle Scholar
  5. 5.
    Cryan, M., Goldberg, L.A., Goldberg, P.W.: Evolutionary Trees can be Learnt in Polynomial Time in the Two-State General Markov Model. SIAM Journal on Computing 31(2), 375–397 (2001)zbMATHCrossRefMathSciNetGoogle Scholar
  6. 6.
    Goldberg, P.W.: When can two unsupervised learners achieve PAC separation? In: Helmbold, D.P., Williamson, B. (eds.) COLT 2001 and EuroCOLT 2001. LNCS (LNAI), vol. 2111, pp. 303–319. Springer, Heidelberg (2001)CrossRefGoogle Scholar
  7. 7.
    de la Higuera, C., Oncina, J.: Learning Probabilistic Finite Automata. tech. rept. EURISE, Université de Saint-Etienne and Departamento de Lenguajes y Sistemas Informaticos (2002)Google Scholar
  8. 8.
    Kearns, M., Mansour, Y., Ron, D., Rubinfeld, R., Schapire, R.E., Sellie, L.: On the Learnability of Discrete Distributions. In: Procs. of STOC, pp. 273–282 (1994)Google Scholar
  9. 9.
    Palmer, N., Goldberg, P.W.: PAC Classification via PAC Estimates of Label Class Distributions. Tech rept. 411, Dept. of Computer Science, University of Warwick (2004)Google Scholar
  10. 10.
    Ron, D., Singer, Y., Tishby, N.: On the Learnability and Usage of Acyclic Probabilistic Finite Automata. Journal of Computer and System Sciences 56(2), 133–152 (1998)zbMATHCrossRefMathSciNetGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  • Nick Palmer
    • 1
  • Paul W. Goldberg
    • 1
  1. 1.Dept. of Computer ScienceUniversity of WarwickCoventryU.K.

Personalised recommendations