Advertisement

Memory-Limited U-Shaped Learning

  • Lorenzo Carlucci
  • John Case
  • Sanjay Jain
  • Frank Stephan
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4005)

Abstract

U-shaped learning is a learning behaviour in which the learner first learns something, then unlearns it and finally relearns it. Such a behaviour, observed by psychologists, for example, in the learning of past-tenses of English verbs, has been widely discussed among psychologists and cognitive scientists as a fundamental example of the non-monotonicity of learning. Previous theory literature has studied whether or not U-shaped learning, in the context of Gold’s formal model of learning languages from positive data, is necessary for learning some tasks.

It is clear that human learning involves memory limitations. In the present paper we consider, then, this question of the necessity of U-shaped learning for some learning models featuring memory limitations. Our results show that the question of the necessity of U-shaped learning in this memory-limited setting depends on delicate tradeoffs between the learner’s ability to remember its own previous conjecture, to store some values in its long-term memory, to make queries about whether or not items occur in previously seen data and on the learner’s choice of hypothesis space.

Keywords

Data Item Inductive Inference Iterative Learning Hypothesis Space Semantic Constraint 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Angluin, D.: Inductive inference of formal languages from positive data. Information and Control 45, 117–135 (1980)CrossRefMathSciNetMATHGoogle Scholar
  2. 2.
    Baliga, G., Case, J., Merkle, W., Stephan, F., Wiehagen, R.: When unlearning helps (manuscript, 2005). In: Welzl, E., Montanari, U., Rolim, J.D.P. (eds.) ICALP 2000. LNCS, vol. 1853, pp. 844–855. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  3. 3.
    Bārzdiņš, J.: Inductive Inference of automata, functions and programs. In: International Mathematical Congress, Vancouver, pp. 771–776 (1974)Google Scholar
  4. 4.
    Blum, M.: A machine independent theory of the complexity of the recursive functions. Journal of the Association for Computing Machinery 14, 322–336 (1967)MathSciNetMATHGoogle Scholar
  5. 5.
    Bower, T.G.R.: Concepts of development. In: Proceedings of the 21st International Congress of Psychology, pp. 79–97. Presses Universitaires de France (1978)Google Scholar
  6. 6.
    Bowerman, M.: Starting to talk worse: Clues to language acquisition from children’s late speech errors. In: Strauss, S., Stavy, R. (eds.) U-Shaped Behavioral Growth, Academic Press, New York (1982)Google Scholar
  7. 7.
    Carey, S.: Face perception: Anomalies of development. In: Strauss, S., Stavy, R. (eds.) U-Shaped Behavioral Growth. Developmental Psychology Series, pp. 169–190. Academic Press, London (1982)Google Scholar
  8. 8.
    Carlucci, L., Case, J., Jain, S., Stephan, F.: Non U-Shaped Vacillatory and Team Learning. In: Jain, S., Simon, H.U., Tomita, E. (eds.) ALT 2005. LNCS (LNAI), vol. 3734, pp. 241–255. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  9. 9.
    Carlucci, L., Case, J., Jain, S., Stephan, F.: Memory-limited U-shaped learning (Long version of the present paper). TR51/05, School of Computing, National University of Singapore (2005)Google Scholar
  10. 10.
    Carlucci, L., Jain, S., Kinber, E., Stephan, F.: Variations on U-shaped learning. In: Auer, P., Meir, R. (eds.) COLT 2005. LNCS, vol. 3559, pp. 382–397. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  11. 11.
    Case, J., Jain, S., Lange, S., Zeugmann, T.: Incremental Concept Learning for Bounded Data Mining. Information and Computation 152(1), 74–110 (1999)CrossRefMathSciNetMATHGoogle Scholar
  12. 12.
    Freivalds, R., Kinber, E.B., Smith, C.H.: On the impact of forgetting on learning machines. Journal of the ACM 42, 1146–1168 (1995)CrossRefMathSciNetMATHGoogle Scholar
  13. 13.
    Mark Gold, E.: Language identification in the limit. Information and Control 10, 447–474 (1967)CrossRefMATHGoogle Scholar
  14. 14.
    Jain, S., Osherson, D., Royer, J., Sharma, A.: Systems that Learn: An Introduction to Learning Theory, 2nd edn. MIT Press, Cambridge (1999)Google Scholar
  15. 15.
    Jantke, K.-P.: Monotonic and non-monotonic Inductive Inference. New Generation Computing 8, 349–360 (1991)CrossRefMATHGoogle Scholar
  16. 16.
    Kinber, E., Stephan, F.: Language learning from texts: mind changes, limited memory and monotonicity. Information and Computation 123, 224–241 (1995)CrossRefMathSciNetMATHGoogle Scholar
  17. 17.
    Lange, S., Zeugmann, T.: The learnability of recursive languages in dependence on the space of hypotheses. GOSLER-Report, 20/93. Fachbereich Mathematik und Informatik, TH Leipzig (1993)Google Scholar
  18. 18.
    Lange, S., Zeugmann, T.: Incremental Learning from Positive Data. Journal of Computer and System Sciences 53, 88–103 (1996)CrossRefMathSciNetMATHGoogle Scholar
  19. 19.
    Marcus, G., Pinker, S., Ullman, M., Hollander, M., John Rosen, T., Xu, F.: Overregularization in Language Acquisition. In: Monographs of the Society for Research in Child Development, vol. 57(4), University of Chicago Press, Chicago (1992); Includes commentary by Harold ClahsenGoogle Scholar
  20. 20.
    Odifreddi, P.: Classical Recursion Theory. North Holland, Amsterdam (1989)MATHGoogle Scholar
  21. 21.
    Osherson, D., Stob, M., Weinstein, S.: Systems that Learn: An Introduction to Learning Theory for Cognitive and Computer Scientists. MIT Press, Cambridge (1986)Google Scholar
  22. 22.
    Plunkett, K., Marchman, V.: U-shaped learning and frequency effects in a multi-layered perceptron: implications for child language acquisition. Cognition 38(1), 43–102 (1991)CrossRefGoogle Scholar
  23. 23.
    Royer, J.: A Connotational Theory of Program Structure. In: Royer, J.S. (ed.) A Connotational Theory of Program Structure. LNCS, vol. 273. Springer, Heidelberg (1987)Google Scholar
  24. 24.
    Strauss, S., Stavy, R.: U-Shaped Behavioral Growth. Developmental Psychology Series. Academic Press, London (1982)Google Scholar
  25. 25.
    Strauss, S., Stavy, R., Orpaz, N.: The child’s development of the concept of temperature. Manuscript, Tel-Aviv University (1977)Google Scholar
  26. 26.
    Taatgen, N.A., Anderson, J.R.: Why do children learn to say broke? A model of learning the past tense without feedback. Cognition 86(2), 123–155 (2002)CrossRefGoogle Scholar
  27. 27.
    Wexler, K., Culicover, P.W.: Formal Principles of Language Acquisition. MIT Press, Cambridge (1980)Google Scholar
  28. 28.
    Wiehagen, R.: Limes-Erkennung rekursiver Funktionen durch spezielle Strategien. Journal of Information Processing and Cybernetics 12, 93–99 (1976)MathSciNetMATHGoogle Scholar
  29. 29.
    Wiehagen, R.: A thesis in Inductive Inference. In: Dix, J., Schmitt, P.H., Jantke, K.P. (eds.) NIL 1990. LNCS (LNAI), vol. 543, pp. 184–207. Springer, Heidelberg (1991)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Lorenzo Carlucci
    • 1
    • 2
  • John Case
    • 1
  • Sanjay Jain
    • 3
  • Frank Stephan
    • 4
  1. 1.Department of Computer and Information SciencesUniversity of DelawareNewarkUSA
  2. 2.Dipartimento di MatematicaUniversità di SienaSienaItaly
  3. 3.School of ComputingNational University of SingaporeSingaporeRepublic of Singapore
  4. 4.School of Computing and Department of MathematicsNational University of SingaporeSingaporeRepublic of Singapore

Personalised recommendations