# Lange and Wiehagen's pattern language learning algorithm: An average-case analysis with respect to its total learning time

## Abstract

The present paper deals with the best-case, worst-case and average-case behavior of Lange and Wiehagen's (1991) pattern language learning algorithm with respect to its total learning time. Pattern languages have been introduced by Angluin (1980) and are defined as follows: Let \(\mathcal{A} = \{ 0,1,...\} \) be any non-empty finite alphabet containing at least two elements. Furthermore, let \(X = \{ x|i \in \mathbb{N}\} \) be an infinite set of variables such that \(\mathcal{A} \cap X = \emptyset \). Patterns are non-empty strings over \(\mathcal{A} \cap X\). *L*(π), the language generated by pattern π, is the set of strings which can be obtained by substituting non-null strings from \(\mathcal{A}^ * \) for the variables of the pattern π. Lange and Wiehagen's (1991) algorithm learns the class of all pattern languages in the limit from text. We analyze this algorithm with respect to its total learning time behavior, i.e., the overall time taken by the algorithm until convergence. For every pattern π containing *k* different variables it is shown that the total learning time is \(O(\left| \pi \right|^2 \log _{\left| \mathcal{A} \right|} (\left| \mathcal{A} \right| + k))\) in the best-case and unbounded in the worst-case. Furthermore, we estimate the expectation of the total learning time. In particular, it is shown that Lange and Wiehagen's algorithm possesses an expected total learning time of \(O(2^k k^2 \left| \pi \right|^2 \log _{\left| \mathcal{A} \right|} (k\left| \mathcal{A} \right|))\) with respect to the uniform distribution.

## Preview

Unable to display preview. Download preview PDF.

### References

- [1]D. Angluin, Finding patterns common to a set of strings, Journal of Computer and System Sciences 21 (1980) 46–62.MATHMathSciNetCrossRefGoogle Scholar
- [2]D. Angluin, Queries and concept learning, Machine Learning 2 (1988) 319–342.Google Scholar
- [3]H. Arimura, H. Ishizaka and T. Shinohara, Learning unions of tree patterns using queries, Theoretical Computer Science 185 (1997) 47–62.MATHMathSciNetCrossRefGoogle Scholar
- [4]R. Daley and C.H. Smith, On the complexity of inductive inference, Information and Control 69 (1986) 12–40.MATHMathSciNetCrossRefGoogle Scholar
- [5]E.M. Gold, Language identification in the limit, Information and Control 10 (1967) 447–474.MATHCrossRefGoogle Scholar
- [6]R.L. Graham, D.E. Knuth and O. Patashnik,
*Concrete Mathematics*(Addison-Wesley, Reading, MA, 1989).MATHGoogle Scholar - [7]J.E. Hopcroft and J.D. Ullman,
*Formal Languages and Their Relation to Automata*(Addison-Wesley, Reading, MA, 1969).MATHGoogle Scholar - [8]T. Jiang, A. Salomaa, K. Salomaa and S. Yu, Inclusion is undecidable for pattern languages, in:
*Proc. 20th International Colloquium on Automata, Languages and Programming*, eds. A. Lingas, R. Karlsson and S. Carlsson, Lecture Notes in Computer Science 700 (Springer, 1993) pp. 301–312.Google Scholar - [9]M. Kearns and L. Pitt, A polynomial-time algorithm for learning
*k*-variable pattern languages from examples, in:*Proc. 2nd Annual ACM Workshop on Computational Learning Theory*, eds. R. Rivest, D. Haussler and M.K. Warmuth (Morgan Kaufmann, San Mateo, CA, 1989) pp. 57–71.Google Scholar - [10]P. Kilpeläinen, H. Mannila and E. Ukkonen, MDL learning of unions of simple pattern languages from positive examples, in:
*Proc. 2nd European Conference on Computational Learning Theory – EuroCOLT '95*, ed. P. Vitanyi, Lecture Notes in Artificial Intelligence 904 (Springer, 1995) pp. 252–260.Google Scholar - [11]Ker-I Ko, A. Marron and W.G. Tzeng, Learning string patterns and tree patterns from examples, in:
*Proc. 7th Conference on Machine Learning*, eds. B.W. Porter and R.J. Mooney (Morgan Kaufmann, San Mateo, CA, 1990) pp. 384–391.Google Scholar - [12]S. Lange and R. Wiehagen, Polynomial-time inference of arbitrary pattern languages, New Generation Computing 8 (1991) 361–370.MATHCrossRefGoogle Scholar
- [13]S. Lange and T. Zeugmann, Types of monotonic language learning and their characterization, in:
*Proc. 5th Annual ACM Workshop on Computational Learning Theory*, ed. D. Haussler (ACM Press, New York, 1992) pp. 377–390.Google Scholar - [14]S. Lange and T. Zeugmann, Monotonic versus non-monotonic language learning, in:
*Proc. 2nd International Workshop on Nonmonotonic and Inductive Logic*, eds. G. Brewka, K.P. Jantke and P.H. Schmitt, Lecture Notes in Artificial Intelligence 659 (Springer, 1993) pp. 254–269.Google Scholar - [15]S. Lange and T. Zeugmann, Set-driven and rearrangement-independent learning of recursive languages, Mathematical Systems Theory 29(6) (1996) 599–634.MATHMathSciNetGoogle Scholar
- [16]S. Lange and T. Zeugmann, Incremental learning from positive data, Journal of Computer and System Sciences 53(1) (1996) 88–103.MATHMathSciNetCrossRefGoogle Scholar
- [17]A. Marron, Learning pattern languages from a single initial example and from queries, in:
*Proc. 1st Annual ACM Workshop on Computational Learning Theory*, eds. D. Haussler and L. Pitt (Morgan Kaufmann, San Mateo, CA, 1988) pp. 345–358.Google Scholar - [18]R.P. Nix, Editing by examples, Technical Report 280, Department of Computer Science, Yale University, New Haven, USA (1983).Google Scholar
- [19]G. Pólya, R.E. Tarjan and D.R. Woods,
*Notes on Introductory Combinatorics*(Birkhäuser, Basel, 1983).MATHGoogle Scholar - [20]A. Salomaa, Patterns (The Formal Language Theory Column), EATCS Bulletin 54 (1994) 46–62.Google Scholar
- [21]A. Salomaa, Return to patterns (The Formal Language Theory Column), EATCS Bulletin 55 (1994) 144–157.Google Scholar
- [22]R.E. Schapire, Pattern languages are not learnable, in:
*Proc. 3rd Annual ACM Workshop on Computational Learning Theory*, eds. M.A. Fulk and J. Case (Morgan Kaufmann, San Mateo, CA, 1990) pp. 122–129.Google Scholar - [23]S. Shimozono, A. Shinohara, T. Shinohara, S. Miyano, S. Kuhara and S. Arikawa, Knowledge acquisition from amino acid sequences by machine learning system BONSAI,
*Transactions of Information Processing Society of Japan*35 (1994) 2009–2018.Google Scholar - [24]T. Shinohara, Polynomial time inference of extended regular pattern languages, in:
*Proc. RIMS Symposia on Software Science and Engineering*, eds. E. Goto, K. Furukawa, R. Nakajima, I. Nakata, and A. Yonezawa, Lecture Notes in Computer Science 147 (Springer, 1983) pp. 115–127.Google Scholar - [25]T. Shinohara and S. Arikawa, Learning data entry systems: An application of inductive inference of pattern languages, Research Report 102, Research Institute of Fundamental Information Science, Kyushu University, Fukuoka, Japan (1983).Google Scholar
- [26]T. Shinohara and S. Arikawa, Pattern inference, in:
*Algorithmic Learning for Knowledge-Based Systems*, eds. K.P. Jantke and S. Lange, Lecture Notes in Artificial Intelligence 961 (Springer, 1995) pp. 259–291.Google Scholar - [27]K. Wexler and P. Culicover,
*Formal Principles of Language Acquisition*(MIT Press, Cambridge, MA, 1980).Google Scholar - [28]R. Wiehagen and T. Zeugmann, Ignoring data may be the only way to learn efficiently, Journal of Experimental and Theoretical Artificial Intelligence 6 (1994) 131–144.MATHGoogle Scholar
- [29]T. Zeugmann, S. Lange and S. Kapur, Characterizations of monotonic and dual monotonic language learning, Information and Computation 120 (1995) 155–173.MATHMathSciNetCrossRefGoogle Scholar