Abstract
Iterative learning is a model of language learning from positive data, due to Wiehagen. When compared to a learner in Gold’s original model of language learning from positive data, an iterative learner can be thought of as memory-limited. However, an iterative learner can memorize some input elements by coding them into the syntax of its hypotheses. A main concern of this paper is: to what extent are such coding tricks necessary?
One means of preventing some such coding tricks is to require that the hypothesis space used be free of redundancy, i.e., that it be 1-1. By extending a result of Lange & Zeugmann, we show that many interesting and non-trivial classes of languages can be iteratively identified in this manner. On the other hand, we show that there exists a class of languages that cannot be iteratively identified using any 1-1 effective numbering as the hypothesis space.
We also consider an iterative-like learning model in which the computational component of the learner is modeled as an enumeration operator, as opposed to a partial computable function. In this new model, there are no hypotheses, and, thus, no syntax in which the learner can encode what elements it has or has not yet seen. We show that there exists a class of languages that can be identified under this new model, but that cannot be iteratively identified. On the other hand, we show that there exists a class of languages that cannot be identified under this new model, but that can be iteratively identified using a Friedberg numbering as the hypothesis space.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Angluin, D.: Finding patterns common to a set of strings. J. Comput. Syst. Sci. 21(1), 46–62 (1980)
Blum, L., Blum, M.: Toward a mathematical theory of inductive inference. Inform. Control 28(2), 125–155 (1975)
Becerra-Bonache, L., Case, J., Jain, S., Stephan, F.: Iterative learning of simple external contextual languages. Theor. Comput. Sci. 411(29-30), 2741–2756 (2010)
Beick, H.-R.: Induktive Inferenz mit höchster Konvergenzgeschwindigkeit. PhD thesis, Sektion Mathematik, Humboldt-Universität Berlin (1984)
Case, J.: Periodicity in generations of automata. Math. Syst. Theory 8(1), 15–32 (1974)
Case, J.: Infinitary self-reference in learning theory. J. Exp. Theor. Artif. In. 6(1), 3–16 (1994)
Carlucci, L., Case, J., Jain, S., Stephan, F.: Results on memory-limited U-shaped learning. Inform. Comput. 205(10), 1551–1573 (2007)
Case, J., Kötzing, T.: Strongly non-U-shaped learning results by general techniques. In: Proc. of COLT 2010, pp. 181–193 (2010)
Case, J., Moelius, S.: Optimal language learning. In: Freund, Y., Györfi, L., Turán, G., Zeugmann, T. (eds.) ALT 2008. LNCS (LNAI), vol. 5254, pp. 419–433. Springer, Heidelberg (2008)
de Brecht, M., Yamamoto, A.: Topological properties of concept spaces (full version). Inform. Comput. 208(4), 327–340 (2010)
Freivalds, R., Kinber, E., Wiehagen, R.: Inductive inference and computable one-one numberings. Z. Math. Logik 28(27), 463–479 (1982)
Friedberg, R.: Three theorems on recursive enumeration. I. Decomposition. II. Maximal set. III. Enumeration without duplication. J. Symbolic Logic 23(3), 309–316 (1958)
Fulk, M.: Prudence and other conditions on formal language learning. Inform. Comput. 85(1), 1–11 (1990)
Mark Gold, E.: Language identification in the limit. Inform. Control 10(5), 447–474 (1967)
Jain, S.: Private communcation (2010)
Jain, S., Lange, S., Moelius, S., Zilles, S.: Incremental learning with temporary memory. Theor. Comput. Sci. 411(29-30), 2757–2772 (2010)
Jain, S., Stephan, F.: Learning in Friedberg numberings. Inform. Comput. 206(6), 776–790 (2008)
Kummer, M.: An easy priority-free proof of a theorem of Friedberg. Theor. Comput. Sci. 74(2), 249–251 (1990)
Lange, S., Wiehagen, R.: Polynomial time inference of arbitrary pattern languages. New Generat. Comput. 8(4), 361–370 (1991)
Lange, S., Zeugmann, T.: Incremental learning from positive data. J. Comput. Syst. Sci. 53(1), 88–103 (1996)
Lange, S., Zeugmann, T., Zilles, S.: Learning indexed families of recursive languages from positive data: A survey. Theor. Comput. Sci. 397(1-3), 194–232 (2008)
Moelius, S., Zilles, S.: Learning without coding (2010),(unpublished manuscript), http://www2.cs.uregina.ca/~zilles/moeliusZ10TR.pdf
Rogers, H.: Theory of Recursive Functions and Effective Computability. McGraw Hill, New York (1967); Reprinted, MIT Press (1987)
Shimozono, S., Shinohara, A., Shinohara, T., Miyano, S., Kuhara, S., Arikawa, S.: Knowledge acquisition from amino acid sequences by machine learning system BONSAI. Trans. Inform. Process. Soc. Jpn. 35(10), 2009–2018 (1994)
Wiehagen, R.: Limes-Erkennung rekursiver Funktionen durch spezielle Strategien. J. Inform. Process. Cybern. (EIK) 12(1/2), 93–99 (1976)
Wiehagen, R.: A thesis in inductive inference. In: Dix, J., Schmitt, P.H., Jantke, K.P. (eds.) NIL 1990. LNCS (LNAI), vol. 543, pp. 184–207. Springer, Heidelberg (1991)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Moelius, S.E., Zilles, S. (2010). Learning without Coding. In: Hutter, M., Stephan, F., Vovk, V., Zeugmann, T. (eds) Algorithmic Learning Theory. ALT 2010. Lecture Notes in Computer Science(), vol 6331. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-16108-7_25
Download citation
DOI: https://doi.org/10.1007/978-3-642-16108-7_25
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-16107-0
Online ISBN: 978-3-642-16108-7
eBook Packages: Computer ScienceComputer Science (R0)