Lange and Wiehagen's pattern language learning algorithm: An average-case analysis with respect to its total learning time

  • Thomas Zeugmann

Abstract

The present paper deals with the best-case, worst-case and average-case behavior of Lange and Wiehagen's (1991) pattern language learning algorithm with respect to its total learning time. Pattern languages have been introduced by Angluin (1980) and are defined as follows: Let \(\mathcal{A} = \{ 0,1,...\} \) be any non-empty finite alphabet containing at least two elements. Furthermore, let \(X = \{ x|i \in \mathbb{N}\} \) be an infinite set of variables such that \(\mathcal{A} \cap X = \emptyset \). Patterns are non-empty strings over \(\mathcal{A} \cap X\). L(π), the language generated by pattern π, is the set of strings which can be obtained by substituting non-null strings from \(\mathcal{A}^ * \) for the variables of the pattern π. Lange and Wiehagen's (1991) algorithm learns the class of all pattern languages in the limit from text. We analyze this algorithm with respect to its total learning time behavior, i.e., the overall time taken by the algorithm until convergence. For every pattern π containing k different variables it is shown that the total learning time is \(O(\left| \pi \right|^2 \log _{\left| \mathcal{A} \right|} (\left| \mathcal{A} \right| + k))\) in the best-case and unbounded in the worst-case. Furthermore, we estimate the expectation of the total learning time. In particular, it is shown that Lange and Wiehagen's algorithm possesses an expected total learning time of \(O(2^k k^2 \left| \pi \right|^2 \log _{\left| \mathcal{A} \right|} (k\left| \mathcal{A} \right|))\) with respect to the uniform distribution.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. [1]
    D. Angluin, Finding patterns common to a set of strings, Journal of Computer and System Sciences 21 (1980) 46–62.MATHMathSciNetCrossRefGoogle Scholar
  2. [2]
    D. Angluin, Queries and concept learning, Machine Learning 2 (1988) 319–342.Google Scholar
  3. [3]
    H. Arimura, H. Ishizaka and T. Shinohara, Learning unions of tree patterns using queries, Theoretical Computer Science 185 (1997) 47–62.MATHMathSciNetCrossRefGoogle Scholar
  4. [4]
    R. Daley and C.H. Smith, On the complexity of inductive inference, Information and Control 69 (1986) 12–40.MATHMathSciNetCrossRefGoogle Scholar
  5. [5]
    E.M. Gold, Language identification in the limit, Information and Control 10 (1967) 447–474.MATHCrossRefGoogle Scholar
  6. [6]
    R.L. Graham, D.E. Knuth and O. Patashnik, Concrete Mathematics (Addison-Wesley, Reading, MA, 1989).MATHGoogle Scholar
  7. [7]
    J.E. Hopcroft and J.D. Ullman, Formal Languages and Their Relation to Automata (Addison-Wesley, Reading, MA, 1969).MATHGoogle Scholar
  8. [8]
    T. Jiang, A. Salomaa, K. Salomaa and S. Yu, Inclusion is undecidable for pattern languages, in: Proc. 20th International Colloquium on Automata, Languages and Programming, eds. A. Lingas, R. Karlsson and S. Carlsson, Lecture Notes in Computer Science 700 (Springer, 1993) pp. 301–312.Google Scholar
  9. [9]
    M. Kearns and L. Pitt, A polynomial-time algorithm for learning k-variable pattern languages from examples, in: Proc. 2nd Annual ACM Workshop on Computational Learning Theory, eds. R. Rivest, D. Haussler and M.K. Warmuth (Morgan Kaufmann, San Mateo, CA, 1989) pp. 57–71.Google Scholar
  10. [10]
    P. Kilpeläinen, H. Mannila and E. Ukkonen, MDL learning of unions of simple pattern languages from positive examples, in: Proc. 2nd European Conference on Computational Learning Theory – EuroCOLT '95, ed. P. Vitanyi, Lecture Notes in Artificial Intelligence 904 (Springer, 1995) pp. 252–260.Google Scholar
  11. [11]
    Ker-I Ko, A. Marron and W.G. Tzeng, Learning string patterns and tree patterns from examples, in: Proc. 7th Conference on Machine Learning, eds. B.W. Porter and R.J. Mooney (Morgan Kaufmann, San Mateo, CA, 1990) pp. 384–391.Google Scholar
  12. [12]
    S. Lange and R. Wiehagen, Polynomial-time inference of arbitrary pattern languages, New Generation Computing 8 (1991) 361–370.MATHCrossRefGoogle Scholar
  13. [13]
    S. Lange and T. Zeugmann, Types of monotonic language learning and their characterization, in: Proc. 5th Annual ACM Workshop on Computational Learning Theory, ed. D. Haussler (ACM Press, New York, 1992) pp. 377–390.Google Scholar
  14. [14]
    S. Lange and T. Zeugmann, Monotonic versus non-monotonic language learning, in: Proc. 2nd International Workshop on Nonmonotonic and Inductive Logic, eds. G. Brewka, K.P. Jantke and P.H. Schmitt, Lecture Notes in Artificial Intelligence 659 (Springer, 1993) pp. 254–269.Google Scholar
  15. [15]
    S. Lange and T. Zeugmann, Set-driven and rearrangement-independent learning of recursive languages, Mathematical Systems Theory 29(6) (1996) 599–634.MATHMathSciNetGoogle Scholar
  16. [16]
    S. Lange and T. Zeugmann, Incremental learning from positive data, Journal of Computer and System Sciences 53(1) (1996) 88–103.MATHMathSciNetCrossRefGoogle Scholar
  17. [17]
    A. Marron, Learning pattern languages from a single initial example and from queries, in: Proc. 1st Annual ACM Workshop on Computational Learning Theory, eds. D. Haussler and L. Pitt (Morgan Kaufmann, San Mateo, CA, 1988) pp. 345–358.Google Scholar
  18. [18]
    R.P. Nix, Editing by examples, Technical Report 280, Department of Computer Science, Yale University, New Haven, USA (1983).Google Scholar
  19. [19]
    G. Pólya, R.E. Tarjan and D.R. Woods, Notes on Introductory Combinatorics (Birkhäuser, Basel, 1983).MATHGoogle Scholar
  20. [20]
    A. Salomaa, Patterns (The Formal Language Theory Column), EATCS Bulletin 54 (1994) 46–62.Google Scholar
  21. [21]
    A. Salomaa, Return to patterns (The Formal Language Theory Column), EATCS Bulletin 55 (1994) 144–157.Google Scholar
  22. [22]
    R.E. Schapire, Pattern languages are not learnable, in: Proc. 3rd Annual ACM Workshop on Computational Learning Theory, eds. M.A. Fulk and J. Case (Morgan Kaufmann, San Mateo, CA, 1990) pp. 122–129.Google Scholar
  23. [23]
    S. Shimozono, A. Shinohara, T. Shinohara, S. Miyano, S. Kuhara and S. Arikawa, Knowledge acquisition from amino acid sequences by machine learning system BONSAI, Transactions of Information Processing Society of Japan 35 (1994) 2009–2018.Google Scholar
  24. [24]
    T. Shinohara, Polynomial time inference of extended regular pattern languages, in: Proc. RIMS Symposia on Software Science and Engineering, eds. E. Goto, K. Furukawa, R. Nakajima, I. Nakata, and A. Yonezawa, Lecture Notes in Computer Science 147 (Springer, 1983) pp. 115–127.Google Scholar
  25. [25]
    T. Shinohara and S. Arikawa, Learning data entry systems: An application of inductive inference of pattern languages, Research Report 102, Research Institute of Fundamental Information Science, Kyushu University, Fukuoka, Japan (1983).Google Scholar
  26. [26]
    T. Shinohara and S. Arikawa, Pattern inference, in: Algorithmic Learning for Knowledge-Based Systems, eds. K.P. Jantke and S. Lange, Lecture Notes in Artificial Intelligence 961 (Springer, 1995) pp. 259–291.Google Scholar
  27. [27]
    K. Wexler and P. Culicover, Formal Principles of Language Acquisition (MIT Press, Cambridge, MA, 1980).Google Scholar
  28. [28]
    R. Wiehagen and T. Zeugmann, Ignoring data may be the only way to learn efficiently, Journal of Experimental and Theoretical Artificial Intelligence 6 (1994) 131–144.MATHGoogle Scholar
  29. [29]
    T. Zeugmann, S. Lange and S. Kapur, Characterizations of monotonic and dual monotonic language learning, Information and Computation 120 (1995) 155–173.MATHMathSciNetCrossRefGoogle Scholar

Copyright information

© Kluwer Academic Publishers 1998

Authors and Affiliations

  • Thomas Zeugmann
    • 1
  1. 1.Department of InformaticsKyushu UniversityKasugaJapan

Personalised recommendations