# Learning *k*-variable pattern languages efficiently stochastically finite on average from positive data

## Abstract

The present paper presents a new approach of how to convert Gold-style [4] learning in the limit into *stochastically finite* learning with *high confidence*. We illustrate this approach on the concept class of all pattern languages. The transformation of learning in the limit into stochastically finite learning with high confidence is achieved by first analyzing the Lange-Wiehagen [7] algorithm with respect to its average-case time behavior until convergence. This algorithm learns the class of all pattern languages in the limit from positive data. The expectation of the total learning time is analyzed and *exponentially* small tail bounds are established for a large class of probability distributions. For patterns containing *k* different variables Lange and Wiehagen's algorithm possesses an expected total learning time of {ie13-01}, where *α* and *Β* are two easily computable parameters from the underlying probability distribution, and *E*[*λ*] is the expected example string length.

Finally, we show how to arrive at stochastically finite learning with high confidence.

## Preview

Unable to display preview. Download preview PDF.

## References

- 1.D. Angluin. Finding patterns common to a set of strings.
*Journal of Computer and System Sciences*, 21(1):46–62, 1980.zbMATHMathSciNetCrossRefGoogle Scholar - 2.D. Angluin. Inductive inference of formal languages from positive data.
*Information and Control*, 45:117–135, 1980.zbMATHMathSciNetCrossRefGoogle Scholar - 3.R. Daley and C.H. Smith. On the complexity of inductive inference.
*Information and Control*, 69:12–40, 1986.zbMATHMathSciNetCrossRefGoogle Scholar - 4.E. M. Gold. Language identification in the limit.
*Information and Control*, 10:447–474, 1967.zbMATHCrossRefGoogle Scholar - 5.M. Kearns and L. Pitt. A polynomial-time algorithm for learning
*k*-variable pattern languages from examples. In R. Rivest, D. Haussler and M.K. Warmuth, editors,*Proc. 2nd Annual ACM Workshop on Computational Learning Theory*pp. 57–71, 1991, Morgan Kaufmann Publishers Inc., San Mateo.Google Scholar - 6.Ker-I Ko, A. Marron and W.G. Tzeng. Learning string patterns and tree patterns from examples. In B.W. Porter and R.J. Mooney, editors,
*Proc. 7th International Conference on Machine Learning*, pp. 384–391, 1990, Morgan-Kaufmann Publishers Inc., San Mateo.Google Scholar - 7.S. Lange and R. Wiehagen. Polynomial-time inference of arbitrary pattern languages.
*New Generation Computing*, 8:361–370, 1991.zbMATHCrossRefGoogle Scholar - 8.L. Pitt. Inductive inference, DFAs and computational complexity. In K.P. Jantke, editor,
*Proc. Analogical and Inductive Inference*, Lecture Notes in Artificial Intelligence 397, pp. 18–44, Berlin, 1989, Springer-Verlag.Google Scholar - 9.P. Rossmanith and T. Zeugmann. Learning
*k*-variable pattern languages efficiently stochastically finite on average from positive data, DOI Technical Report DOI-TR-145, Department of Informatics, Kyushu University, January 1998.Google Scholar - 10.A. Salomaa. Patterns & Return to patterns. (The Formal Language Theory Column). EATCS Bulletin, 54:46–62 and 55:144–157, 1994.Google Scholar
- 11.R.E. Schapire. Pattern languages are not learnable. In M.A. Fulk and J. Case, editors,
*Proc. 3rd Annual ACM Workshop on Computational Learning Theory*, pp. 122–129, 1990. Morgan Kaufmann Publishers Inc., San Mateo.Google Scholar - 12.T. Shinohara and S. Arikawa. Pattern inference. In K. P. Jantke and S. Lange, editors,
*Algorithmic Learning for Knowledge-Based Systems*, Lecture Notes in Artificial Intelligence 961, pp. 259–291, Berlin, 1995. Springer-Verlag.Google Scholar - 13.K. Wexler and P. Culicover.
*Formal Principles of Language Acquisition*. MIT Press, Cambridge, MA, 1980.Google Scholar - 14.T. Zeugmann. Lange and Wiehagen's pattern learning algorithm: An average-case analysis with respect to its total learning time.
*Annals of Mathematics and Artificial Intelligence*, 1998. To appear.Google Scholar