ALT 1997: Algorithmic Learning Theory pp 260-276

Learning one-variable pattern languages very efficiently on average, in parallel, and by asking queries

• Thomas Erlebach
• Peter Rossmanith
• Angelika Steger
• Thomas Zeugmann
Session 8
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1316)

Abstract

A pattern is a string of constant and variable symbols. The language generated by a pattern π is the set of all strings of constant symbols which can be obtained from π by substituting non-empty strings for variables. We study the learnability of one-variable pattern languages in the limit with respect to the update time needed for computing a new single guess and the expected total learning time taken until convergence to a correct hypothesis. The results obtained are threefold. First, we design a consistent and set-driven learner that, using the concept of descriptive patterns, achieves update time O(n2 log n), where n is the size of the input sample. The best previously known algorithm to compute descriptive one-variable patterns requires time O(n4 log n) (cf. Angluin [1]). Second, we give a parallel version of this algorithm requiring time O(log n) and O(n3/log n) processors on an EREW-PRAM. Third, we devise a one-variable pattern learner whose expected total learning time is O(l2 log l) provided the sample strings are drawn from the target language according to a probability distribution D with expected string length l. The distribution D must be such that strings of equal length have equal probability, but can be arbitrary otherwise. Thus, we establish the first one-variable pattern learner having an expected total learning time that provably differs from the update time by a constant factor only.

Finally, we apply the algorithm for finding descriptive one-variable patterns to learn one-variable patterns with a polynomial number of superset queries with respect to the one-variable patterns as query language.

Preview

References

1. 1.
D. Angluin. Finding patterns common to a set of strings. Journal of Computer and System Sciences, 21:46–62, 1980.
2. 2.
D. Angluin. Inductive inference of formal languages from positive data. Information and Control, 45:117–135, 1980.
3. 3.
D. Angluin. Queries and concept learning. Machine Learning, 2:319–342, 1988.Google Scholar
4. 4.
T. Erlebach, P. Rossmanith, H. Stadtherr, A. Steger, and T. Zeugmann. Efficient Learning of One-Variable Pattern Languages from Positive Data. DOI-TR-128, Department of Informatics, Kyushu University, Dec. 12, 1996; http://www.i.kyushuu.ac.jp/ thomas/treport.html.Google Scholar
5. 5.
G. Filé. The relation of two patterns with comparable languages. In Proc. 5th Ann. Symp. Theoretical Aspects of Computer Science, LNCS 294, pp. 184–192, Berlin, 1988. Springer-Verlag.Google Scholar
6. 6.
S. Fortune and J. Wyllie. Parallelism in random access machines. In Proc. 10th Ann. ACM Symp. Theory of Computing, pp. 114–118, New York, 1978, ACM Press.Google Scholar
7. 7.
E. M. Gold. Language identification in the limit. Inf. Control, 10:447–474, 1967.Google Scholar
8. 8.
9. 9.
K. P. Jantke. Polynomial time inference of general pattern languages. In Proc. Symp. Theoretical Aspects of Computer Science, LNCS 166, pp. 314–325, Berlin, 1984. Springer-Verlag.Google Scholar
10. 10.
T. Jiang, A. Salomaa, K. Salomaa, and S. Yu. Inclusion is undecidable for pattern languages. In Proc. 20th Int. Colloquium on Automata, Languages and Programming, LNCS 700, pp. 301–312, Berlin, 1993. Springer-Verlag.Google Scholar
11. 11.
M. Kearns and L. Pitt. A polynomial-time algorithm for learning k-variable pattern languages from examples. In Proc. 2nd Ann. ACM Workshop on Computational Learning Theory, pp. 57–71, Morgan Kaufmann Publ., San Mateo, 1989.Google Scholar
12. 12.
K.-I. Ko and C.-M. Hua. A note on the two-variable pattern-finding problem. J. Comp. Syst. Sci., 34:75–86, 1987.Google Scholar
13. 13.
S. Lange and R. Wiehagen. Polynomial-time inference of arbitrary pattern languages. New Generation Computing, 8:361–370, 1991.Google Scholar
14. 14.
S. Lange and T. Zeugmann. Set-driven and rearrangement-independent learning of recursive languages. Mathematical Systems Theory, 29:599–634, 1996.Google Scholar
15. 15.
D. Osherson, M. Stob and S. Weinstein. Systems that learn: An introduction to learning theory for cognitive and computer scientists. MIT Press, 1986Google Scholar
16. 16.
L. Pitt. Inductive inference, DFAs and computational complexity. In Proc. Analogical and Inductive Inference, LNAI 397, pp. 18–44, Berlin, 1989, Springer-Verlag.Google Scholar
17. 17.
A. Salomaa. Patterns. EATCS Bulletin 54:46–62, 1994.Google Scholar
18. 18.
19. 19.
T. Shinohara and S. Arikawa. Pattern inference. In Algorithmic Learning for Knowledge-Based Systems, LNAI 961, pp. 259–291, Berlin, 1995. Springer-Verlag.Google Scholar
20. 20.
K. Wexler and P. Culicover. Formal Principles of Language Acquisition. MIT Press, Cambridge, MA, 1980.Google Scholar
21. 21.
T. Zeugmann. Lange and Wiehagen's pattern language learning algorithm: An average-case analysis with respect to its total learning time. Annals of Mathematics and Artificial Intelligence, 1997, to appear.Google Scholar
22. 22.
T. Zeugmann and S. Lange. A guided tour across the boundaries of learning recursive languages. In Algorithmic Learning for Knowledge-Based Systems, LNAI 961, pp. 190–258, Berlin, 1995. Springer-Verlag.Google Scholar

Authors and Affiliations

• Thomas Erlebach
• 1
• Peter Rossmanith
• 1