COLT 2005: Learning Theory pp 382-397 | Cite as
Variations on U-Shaped Learning
Abstract
The paper deals with the following problem: is returning to wrong conjectures necessary to achieve full power of learning? Returning to wrong conjectures complements the paradigm of U-shaped learning [2,6,8,20,24] when a learner returns to old correct conjectures. We explore our problem for classical models of learning in the limit: TxtEx-learning – when a learner stabilizes on a correct conjecture, and TxtBc-learning – when a learner stabilizes on a sequence of grammars representing the target concept. In all cases, we show that, surprisingly, returning to wrong conjectures is sometimes necessary to achieve full power of learning. On the other hand it is not necessary to return to old “overgeneralizing” conjectures containing elements not belonging to the target language. We also consider our problem in the context of so-called vacillatory learning when a learner stabilizes to a finite number of correct grammars. In this case we show that both returning to old wrong conjectures and returning to old “overgeneralizing” conjectures is necessary for full learning power. We also show that, surprisingly, learners consistent with the input seen so far can be made decisive [2,21] – they do not have to return to any old conjectures – wrong or right.
Keywords
Language Learning Target Language Recursive Function Inductive Inference Full PowerPreview
Unable to display preview. Download preview PDF.
References
- 1.Angluin, D.: Inductive inference of formal languages from positive data. Information and Control 45, 117–135 (1980)MATHCrossRefMathSciNetGoogle Scholar
- 2.Baliga, G., Case, J., Merkle, W., Stephan, F., Wiehagen, R.: When unlearning helps (Manuscript 2005); Preliminary version of the paper appeared in ICALP (2000), http://www.cis.udel.edu/~case/papers/decisive.ps
- 3.Bārzdiņš, J.: Inductive inference of automata, functions and programs. In: Int. Math. Congress, Vancouver, pp. 771–776 (1974)Google Scholar
- 4.Blum, L., Blum, M.: Toward a mathematical theory of inductive inference. Information and Control 28, 125–155 (1975)MATHCrossRefMathSciNetGoogle Scholar
- 5.Blum, M.: A machine-independent theory of the complexity of recursive functions. Journal of the ACM 14, 322–336 (1967)MATHCrossRefGoogle Scholar
- 6.Bowerman, M.: Starting to talk worse: Clues to language acquisition from children’s late speech errors. In: Strauss, S., Stavy, R. (eds.) U-Shaped Behavioral Growth. Developmental Psychology Series, Academic Press, New York (1982)Google Scholar
- 7.Carey, S.: An analysis of a learning paradigm. In: Strauss, S., Stavy, R. (eds.) U-Shaped Behavioral Growth. Developmental Psychology Series, Academic Press, New York (1982)Google Scholar
- 8.Carlucci, L., Case, J., Jain, S., Stephan, F.: U-shaped learning may be necessary. Technical Report TRA11/04, School of Computing, National University of Singapore (November 2004)Google Scholar
- 9.Case, J.: The power of vacillation in language learning. SIAM Journal on Computing 28(6), 1941–1969 (1999)MATHCrossRefMathSciNetGoogle Scholar
- 10.Case, J., Lynes, C.: Machine inductive inference and language identification. In: Nielsen, M., Schmidt, E.M. (eds.) ICALP 1982. LNCS, vol. 140, pp. 107–115. Springer, Heidelberg (1982)CrossRefGoogle Scholar
- 11.Case, J., Smith, C.: Comparison of identification criteria for machine inductive inference. Theoretical Computer Science 25, 193–220 (1983)MATHCrossRefMathSciNetGoogle Scholar
- 12.Fulk, M.: Prudence and other conditions on formal language learning. Information and Computation 85, 1–11 (1990)MATHCrossRefMathSciNetGoogle Scholar
- 13.Fulk, M., Jain, S., Osherson, D.: Open problems in systems that learn. Journal of Computer and System Sciences 49(3), 589–604 (1994)CrossRefMathSciNetGoogle Scholar
- 14.Gold, E.M.: Language identification in the limit. Information and Control 10, 447–474 (1967)MATHCrossRefGoogle Scholar
- 15.Hopcroft, J., Ullman, J.: Introduction to Automata Theory, Languages, and Computation. Addison-Wesley, Reading (1979)MATHGoogle Scholar
- 16.Jantke, K., Beick, H.: Combining postulates of naturalness in inductive inference. Journal of Information Processing and Cybernetics (EIK) 17, 465–484 (1981)MATHMathSciNetGoogle Scholar
- 17.Kurtz, S., Royer, J.: Prudence in language learning. In: Haussler, D., Pitt, L. (eds.) Proceedings of the Workshop on Computational Learning Theory, pp. 143–156. Morgan Kaufmann, San Francisco (1988)Google Scholar
- 18.Lange, S., Wiehagen, R.: Polynomial time inference of arbitrary pattern languages. New Generation Computing 8, 361–370 (1991)MATHCrossRefGoogle Scholar
- 19.Machtey, M., Young, P.: An Introduction to the General Theory of Algorithms. North-Holland, New York (1978)MATHGoogle Scholar
- 20.Marcus, G., Pinker, S., Ullman, M., Hollander, M., Rosen, T., Xu, F.: Overregularization in Language Acquisition. In: Monographs of the Society for Research in Child Development, vol. 57(4), University of Chicago Press, Chicago (1992); Includes commentary by Harold ClahsenGoogle Scholar
- 21.Osherson, D., Stob, M., Weinstein, S.: Systems that Learn: An Introduction to Learning Theory for Cognitive and Computer Scientists. MIT Press, Cambridge (1986)Google Scholar
- 22.Plunkett, K., Marchman, V.: U-shaped learning and frequency effects in a multi-layered perceptron: implications for child language acquisition. Cognition 38(1), 43–102 (1991)CrossRefGoogle Scholar
- 23.Rogers, H.: Theory of Recursive Functions and Effective Computability. MIT Press, Cambridge (1987); Reprinted by MIT Press in 1987Google Scholar
- 24.Strauss, S., Stavy, R.: U-Shaped Behavioral Growth. In: Developmental Psychology Series, Academic Press, New York (1982)Google Scholar
- 25.Strauss, S., Stavy, R., Orpaz, N.: The child’s development of the concept of temperature. Manuscript, Tel-Aviv University (1977)Google Scholar
- 26.Taatgen, N.A., Anderson, J.R.: Why do children learn to say broke? a model of learning the past tense without feedback. Cognition 86(2), 123–155 (2002)CrossRefGoogle Scholar
- 27.Wiehagen, R., Liepe, W.: Charakteristische Eigenschaften von erkennbaren Klassen rekursiver Funktionen. Journal of Information Processing and Cybernetics (EIK) 12, 421–438 (1976)MATHMathSciNetGoogle Scholar