Learning k-variable pattern languages efficiently stochastically finite on average from positive data

  • Peter Rossmanith
  • Thomas Zeugmann
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1433)

Abstract

The present paper presents a new approach of how to convert Gold-style [4] learning in the limit into stochastically finite learning with high confidence. We illustrate this approach on the concept class of all pattern languages. The transformation of learning in the limit into stochastically finite learning with high confidence is achieved by first analyzing the Lange-Wiehagen [7] algorithm with respect to its average-case time behavior until convergence. This algorithm learns the class of all pattern languages in the limit from positive data. The expectation of the total learning time is analyzed and exponentially small tail bounds are established for a large class of probability distributions. For patterns containing k different variables Lange and Wiehagen's algorithm possesses an expected total learning time of {ie13-01}, where α and Β are two easily computable parameters from the underlying probability distribution, and E[λ] is the expected example string length.

Finally, we show how to arrive at stochastically finite learning with high confidence.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    D. Angluin. Finding patterns common to a set of strings. Journal of Computer and System Sciences, 21(1):46–62, 1980.MATHMathSciNetCrossRefGoogle Scholar
  2. 2.
    D. Angluin. Inductive inference of formal languages from positive data. Information and Control, 45:117–135, 1980.MATHMathSciNetCrossRefGoogle Scholar
  3. 3.
    R. Daley and C.H. Smith. On the complexity of inductive inference. Information and Control, 69:12–40, 1986.MATHMathSciNetCrossRefGoogle Scholar
  4. 4.
    E. M. Gold. Language identification in the limit. Information and Control, 10:447–474, 1967.MATHCrossRefGoogle Scholar
  5. 5.
    M. Kearns and L. Pitt. A polynomial-time algorithm for learning k-variable pattern languages from examples. In R. Rivest, D. Haussler and M.K. Warmuth, editors, Proc. 2nd Annual ACM Workshop on Computational Learning Theory pp. 57–71, 1991, Morgan Kaufmann Publishers Inc., San Mateo.Google Scholar
  6. 6.
    Ker-I Ko, A. Marron and W.G. Tzeng. Learning string patterns and tree patterns from examples. In B.W. Porter and R.J. Mooney, editors, Proc. 7th International Conference on Machine Learning, pp. 384–391, 1990, Morgan-Kaufmann Publishers Inc., San Mateo.Google Scholar
  7. 7.
    S. Lange and R. Wiehagen. Polynomial-time inference of arbitrary pattern languages. New Generation Computing, 8:361–370, 1991.MATHCrossRefGoogle Scholar
  8. 8.
    L. Pitt. Inductive inference, DFAs and computational complexity. In K.P. Jantke, editor, Proc. Analogical and Inductive Inference, Lecture Notes in Artificial Intelligence 397, pp. 18–44, Berlin, 1989, Springer-Verlag.Google Scholar
  9. 9.
    P. Rossmanith and T. Zeugmann. Learning k-variable pattern languages efficiently stochastically finite on average from positive data, DOI Technical Report DOI-TR-145, Department of Informatics, Kyushu University, January 1998.Google Scholar
  10. 10.
    A. Salomaa. Patterns & Return to patterns. (The Formal Language Theory Column). EATCS Bulletin, 54:46–62 and 55:144–157, 1994.Google Scholar
  11. 11.
    R.E. Schapire. Pattern languages are not learnable. In M.A. Fulk and J. Case, editors, Proc. 3rd Annual ACM Workshop on Computational Learning Theory, pp. 122–129, 1990. Morgan Kaufmann Publishers Inc., San Mateo.Google Scholar
  12. 12.
    T. Shinohara and S. Arikawa. Pattern inference. In K. P. Jantke and S. Lange, editors, Algorithmic Learning for Knowledge-Based Systems, Lecture Notes in Artificial Intelligence 961, pp. 259–291, Berlin, 1995. Springer-Verlag.Google Scholar
  13. 13.
    K. Wexler and P. Culicover. Formal Principles of Language Acquisition. MIT Press, Cambridge, MA, 1980.Google Scholar
  14. 14.
    T. Zeugmann. Lange and Wiehagen's pattern learning algorithm: An average-case analysis with respect to its total learning time. Annals of Mathematics and Artificial Intelligence, 1998. To appear.Google Scholar

Copyright information

© Springer-Verlag 1998

Authors and Affiliations

  • Peter Rossmanith
    • 1
  • Thomas Zeugmann
    • 2
  1. 1.Institut für InformatikTechnische Universität MünchenMünchenGermany
  2. 2.Department of InformaticsKyushu UniversityKasugaJapan

Personalised recommendations