Learning one-variable pattern languages very efficiently on average, in parallel, and by asking queries
A pattern is a string of constant and variable symbols. The language generated by a pattern π is the set of all strings of constant symbols which can be obtained from π by substituting non-empty strings for variables. We study the learnability of one-variable pattern languages in the limit with respect to the update time needed for computing a new single guess and the expected total learning time taken until convergence to a correct hypothesis. The results obtained are threefold. First, we design a consistent and set-driven learner that, using the concept of descriptive patterns, achieves update time O(n2 log n), where n is the size of the input sample. The best previously known algorithm to compute descriptive one-variable patterns requires time O(n4 log n) (cf. Angluin ). Second, we give a parallel version of this algorithm requiring time O(log n) and O(n3/log n) processors on an EREW-PRAM. Third, we devise a one-variable pattern learner whose expected total learning time is O(l2 log l) provided the sample strings are drawn from the target language according to a probability distribution D with expected string length l. The distribution D must be such that strings of equal length have equal probability, but can be arbitrary otherwise. Thus, we establish the first one-variable pattern learner having an expected total learning time that provably differs from the update time by a constant factor only.
Finally, we apply the algorithm for finding descriptive one-variable patterns to learn one-variable patterns with a polynomial number of superset queries with respect to the one-variable patterns as query language.
Unable to display preview. Download preview PDF.
- 3.D. Angluin. Queries and concept learning. Machine Learning, 2:319–342, 1988.Google Scholar
- 4.T. Erlebach, P. Rossmanith, H. Stadtherr, A. Steger, and T. Zeugmann. Efficient Learning of One-Variable Pattern Languages from Positive Data. DOI-TR-128, Department of Informatics, Kyushu University, Dec. 12, 1996; http://www.i.kyushuu.ac.jp/ thomas/treport.html.Google Scholar
- 5.G. Filé. The relation of two patterns with comparable languages. In Proc. 5th Ann. Symp. Theoretical Aspects of Computer Science, LNCS 294, pp. 184–192, Berlin, 1988. Springer-Verlag.Google Scholar
- 6.S. Fortune and J. Wyllie. Parallelism in random access machines. In Proc. 10th Ann. ACM Symp. Theory of Computing, pp. 114–118, New York, 1978, ACM Press.Google Scholar
- 7.E. M. Gold. Language identification in the limit. Inf. Control, 10:447–474, 1967.Google Scholar
- 8.J. Jálá. An introduction to parallel algorithms. Addison-Wesley, 1992.Google Scholar
- 9.K. P. Jantke. Polynomial time inference of general pattern languages. In Proc. Symp. Theoretical Aspects of Computer Science, LNCS 166, pp. 314–325, Berlin, 1984. Springer-Verlag.Google Scholar
- 10.T. Jiang, A. Salomaa, K. Salomaa, and S. Yu. Inclusion is undecidable for pattern languages. In Proc. 20th Int. Colloquium on Automata, Languages and Programming, LNCS 700, pp. 301–312, Berlin, 1993. Springer-Verlag.Google Scholar
- 11.M. Kearns and L. Pitt. A polynomial-time algorithm for learning k-variable pattern languages from examples. In Proc. 2nd Ann. ACM Workshop on Computational Learning Theory, pp. 57–71, Morgan Kaufmann Publ., San Mateo, 1989.Google Scholar
- 12.K.-I. Ko and C.-M. Hua. A note on the two-variable pattern-finding problem. J. Comp. Syst. Sci., 34:75–86, 1987.Google Scholar
- 13.S. Lange and R. Wiehagen. Polynomial-time inference of arbitrary pattern languages. New Generation Computing, 8:361–370, 1991.Google Scholar
- 14.S. Lange and T. Zeugmann. Set-driven and rearrangement-independent learning of recursive languages. Mathematical Systems Theory, 29:599–634, 1996.Google Scholar
- 15.D. Osherson, M. Stob and S. Weinstein. Systems that learn: An introduction to learning theory for cognitive and computer scientists. MIT Press, 1986Google Scholar
- 16.L. Pitt. Inductive inference, DFAs and computational complexity. In Proc. Analogical and Inductive Inference, LNAI 397, pp. 18–44, Berlin, 1989, Springer-Verlag.Google Scholar
- 17.A. Salomaa. Patterns. EATCS Bulletin 54:46–62, 1994.Google Scholar
- 18.A. Salomaa. Return to patterns. EATCS Bulletin 55:144–157, 1994.Google Scholar
- 19.T. Shinohara and S. Arikawa. Pattern inference. In Algorithmic Learning for Knowledge-Based Systems, LNAI 961, pp. 259–291, Berlin, 1995. Springer-Verlag.Google Scholar
- 20.K. Wexler and P. Culicover. Formal Principles of Language Acquisition. MIT Press, Cambridge, MA, 1980.Google Scholar
- 21.T. Zeugmann. Lange and Wiehagen's pattern language learning algorithm: An average-case analysis with respect to its total learning time. Annals of Mathematics and Artificial Intelligence, 1997, to appear.Google Scholar
- 22.T. Zeugmann and S. Lange. A guided tour across the boundaries of learning recursive languages. In Algorithmic Learning for Knowledge-Based Systems, LNAI 961, pp. 190–258, Berlin, 1995. Springer-Verlag.Google Scholar