Annals of Mathematics and Artificial Intelligence

, Volume 23, Issue 1–2, pp 117–145

# Lange and Wiehagen's pattern language learning algorithm: An average-case analysis with respect to its total learning time

• Thomas Zeugmann
Article

## Abstract

The present paper deals with the best-case, worst-case and average-case behavior of Lange and Wiehagen's (1991) pattern language learning algorithm with respect to its total learning time. Pattern languages have been introduced by Angluin (1980) and are defined as follows: Let $$\mathcal{A} = \{ 0,1,...\}$$ be any non-empty finite alphabet containing at least two elements. Furthermore, let $$X = \{ x|i \in \mathbb{N}\}$$ be an infinite set of variables such that $$\mathcal{A} \cap X = \emptyset$$. Patterns are non-empty strings over $$\mathcal{A} \cap X$$. L(π), the language generated by pattern π, is the set of strings which can be obtained by substituting non-null strings from $$\mathcal{A}^ *$$ for the variables of the pattern π. Lange and Wiehagen's (1991) algorithm learns the class of all pattern languages in the limit from text. We analyze this algorithm with respect to its total learning time behavior, i.e., the overall time taken by the algorithm until convergence. For every pattern π containing k different variables it is shown that the total learning time is $$O(\left| \pi \right|^2 \log _{\left| \mathcal{A} \right|} (\left| \mathcal{A} \right| + k))$$ in the best-case and unbounded in the worst-case. Furthermore, we estimate the expectation of the total learning time. In particular, it is shown that Lange and Wiehagen's algorithm possesses an expected total learning time of $$O(2^k k^2 \left| \pi \right|^2 \log _{\left| \mathcal{A} \right|} (k\left| \mathcal{A} \right|))$$ with respect to the uniform distribution.

### Keywords

Positive Data Target Pattern Pattern Language Alphabet Size Union Operation

## Preview

Unable to display preview. Download preview PDF.

### References

1. [1]
D. Angluin, Finding patterns common to a set of strings, Journal of Computer and System Sciences 21 (1980) 46–62.
2. [2]
D. Angluin, Queries and concept learning, Machine Learning 2 (1988) 319–342.Google Scholar
3. [3]
H. Arimura, H. Ishizaka and T. Shinohara, Learning unions of tree patterns using queries, Theoretical Computer Science 185 (1997) 47–62.
4. [4]
R. Daley and C.H. Smith, On the complexity of inductive inference, Information and Control 69 (1986) 12–40.
5. [5]
E.M. Gold, Language identification in the limit, Information and Control 10 (1967) 447–474.
6. [6]
R.L. Graham, D.E. Knuth and O. Patashnik, Concrete Mathematics (Addison-Wesley, Reading, MA, 1989).
7. [7]
J.E. Hopcroft and J.D. Ullman, Formal Languages and Their Relation to Automata (Addison-Wesley, Reading, MA, 1969).
8. [8]
T. Jiang, A. Salomaa, K. Salomaa and S. Yu, Inclusion is undecidable for pattern languages, in: Proc. 20th International Colloquium on Automata, Languages and Programming, eds. A. Lingas, R. Karlsson and S. Carlsson, Lecture Notes in Computer Science 700 (Springer, 1993) pp. 301–312.Google Scholar
9. [9]
M. Kearns and L. Pitt, A polynomial-time algorithm for learning k-variable pattern languages from examples, in: Proc. 2nd Annual ACM Workshop on Computational Learning Theory, eds. R. Rivest, D. Haussler and M.K. Warmuth (Morgan Kaufmann, San Mateo, CA, 1989) pp. 57–71.Google Scholar
10. [10]
P. Kilpeläinen, H. Mannila and E. Ukkonen, MDL learning of unions of simple pattern languages from positive examples, in: Proc. 2nd European Conference on Computational Learning Theory – EuroCOLT '95, ed. P. Vitanyi, Lecture Notes in Artificial Intelligence 904 (Springer, 1995) pp. 252–260.Google Scholar
11. [11]
Ker-I Ko, A. Marron and W.G. Tzeng, Learning string patterns and tree patterns from examples, in: Proc. 7th Conference on Machine Learning, eds. B.W. Porter and R.J. Mooney (Morgan Kaufmann, San Mateo, CA, 1990) pp. 384–391.Google Scholar
12. [12]
S. Lange and R. Wiehagen, Polynomial-time inference of arbitrary pattern languages, New Generation Computing 8 (1991) 361–370.
13. [13]
S. Lange and T. Zeugmann, Types of monotonic language learning and their characterization, in: Proc. 5th Annual ACM Workshop on Computational Learning Theory, ed. D. Haussler (ACM Press, New York, 1992) pp. 377–390.Google Scholar
14. [14]
S. Lange and T. Zeugmann, Monotonic versus non-monotonic language learning, in: Proc. 2nd International Workshop on Nonmonotonic and Inductive Logic, eds. G. Brewka, K.P. Jantke and P.H. Schmitt, Lecture Notes in Artificial Intelligence 659 (Springer, 1993) pp. 254–269.Google Scholar
15. [15]
S. Lange and T. Zeugmann, Set-driven and rearrangement-independent learning of recursive languages, Mathematical Systems Theory 29(6) (1996) 599–634.
16. [16]
S. Lange and T. Zeugmann, Incremental learning from positive data, Journal of Computer and System Sciences 53(1) (1996) 88–103.
17. [17]
A. Marron, Learning pattern languages from a single initial example and from queries, in: Proc. 1st Annual ACM Workshop on Computational Learning Theory, eds. D. Haussler and L. Pitt (Morgan Kaufmann, San Mateo, CA, 1988) pp. 345–358.Google Scholar
18. [18]
R.P. Nix, Editing by examples, Technical Report 280, Department of Computer Science, Yale University, New Haven, USA (1983).Google Scholar
19. [19]
G. Pólya, R.E. Tarjan and D.R. Woods, Notes on Introductory Combinatorics (Birkhäuser, Basel, 1983).
20. [20]
A. Salomaa, Patterns (The Formal Language Theory Column), EATCS Bulletin 54 (1994) 46–62.Google Scholar
21. [21]
A. Salomaa, Return to patterns (The Formal Language Theory Column), EATCS Bulletin 55 (1994) 144–157.Google Scholar
22. [22]
R.E. Schapire, Pattern languages are not learnable, in: Proc. 3rd Annual ACM Workshop on Computational Learning Theory, eds. M.A. Fulk and J. Case (Morgan Kaufmann, San Mateo, CA, 1990) pp. 122–129.Google Scholar
23. [23]
S. Shimozono, A. Shinohara, T. Shinohara, S. Miyano, S. Kuhara and S. Arikawa, Knowledge acquisition from amino acid sequences by machine learning system BONSAI, Transactions of Information Processing Society of Japan 35 (1994) 2009–2018.Google Scholar
24. [24]
T. Shinohara, Polynomial time inference of extended regular pattern languages, in: Proc. RIMS Symposia on Software Science and Engineering, eds. E. Goto, K. Furukawa, R. Nakajima, I. Nakata, and A. Yonezawa, Lecture Notes in Computer Science 147 (Springer, 1983) pp. 115–127.Google Scholar
25. [25]
T. Shinohara and S. Arikawa, Learning data entry systems: An application of inductive inference of pattern languages, Research Report 102, Research Institute of Fundamental Information Science, Kyushu University, Fukuoka, Japan (1983).Google Scholar
26. [26]
T. Shinohara and S. Arikawa, Pattern inference, in: Algorithmic Learning for Knowledge-Based Systems, eds. K.P. Jantke and S. Lange, Lecture Notes in Artificial Intelligence 961 (Springer, 1995) pp. 259–291.Google Scholar
27. [27]
K. Wexler and P. Culicover, Formal Principles of Language Acquisition (MIT Press, Cambridge, MA, 1980).Google Scholar
28. [28]
R. Wiehagen and T. Zeugmann, Ignoring data may be the only way to learn efficiently, Journal of Experimental and Theoretical Artificial Intelligence 6 (1994) 131–144.
29. [29]
T. Zeugmann, S. Lange and S. Kapur, Characterizations of monotonic and dual monotonic language learning, Information and Computation 120 (1995) 155–173.

## Copyright information

© Kluwer Academic Publishers 1998

## Authors and Affiliations

• Thomas Zeugmann
• 1
1. 1.Department of InformaticsKyushu UniversityKasugaJapan