Iterative Learning from Positive Data and Negative Counterexamples

  • Sanjay Jain
  • Efim Kinber
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4264)


A model for learning in the limit is defined where a (so-called iterative) learner gets all positive examples from the target language, tests every new conjecture with a teacher (oracle) if it is a subset of the target language (and if it is not, then it receives a negative counterexample), and uses only limited long-term memory (incorporated in conjectures). Three variants of this model are compared: when a learner receives least negative counterexamples, the ones whose size is bounded by the maximum size of input seen so far, and arbitrary ones. We also compare our learnability model with other relevant models of learnability in the limit, study how our model works for indexed classes of recursive languages, and show that learners in our model can work in non-U-shaped way — never abandoning the first right conjecture.


Target Language Recursive Function Regular Language Positive Data Target Concept 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [Ang80]
    Angluin, D.: Finding patterns common to a set of strings. Journal of Computer and System Sciences 21, 46–62 (1980)MATHCrossRefMathSciNetGoogle Scholar
  2. [Ang88]
    Angluin, D.: Queries and concept learning. Machine Learning 2, 319–342 (1988)Google Scholar
  3. [BB75]
    Blum, L., Blum, M.: Toward a mathematical theory of inductive inference. Information and Control 28, 125–155 (1975)MATHCrossRefMathSciNetGoogle Scholar
  4. [BCJ95]
    Baliga, G., Case, J., Jain, S.: Language learning with some negative information. Journal of Computer and System Sciences 51(5), 273–285 (1995)MATHCrossRefMathSciNetGoogle Scholar
  5. [BCM+05]
    Baliga, G., Case, J., Merkle, W., Stephan, F., Wiehagen, R.: When unlearning helps (manuscript, 2005),
  6. [Bow82]
    Bowerman, M.: Starting to talk worse: Clues to language acquisition from children’s late speech errors. In: Strauss, S., Stavy, R. (eds.) U-Shaped Behavioral Growth. Developmental Psychology Series. Academic Press, New York (1982)Google Scholar
  7. [CL82]
    Case, J., Lynes, C.: Machine inductive inference and language identification. In: Nielsen, M., Schmidt, E.M. (eds.) ICALP 1982. LNCS, vol. 140, pp. 107–115. Springer, Heidelberg (1982)CrossRefGoogle Scholar
  8. [CS83]
    Case, J., Smith, C.: Comparison of identification criteria for machine inductive inference. Theoretical Computer Science 25, 193–220 (1983)MATHCrossRefMathSciNetGoogle Scholar
  9. [Ful90]
    Fulk, M.: Prudence and other conditions on formal language learning. Information and Computation 85, 1–11 (1990)MATHCrossRefMathSciNetGoogle Scholar
  10. [Gol67]
    Gold, E.M.: Language identification in the limit. Information and Control 10, 447–474 (1967)MATHCrossRefGoogle Scholar
  11. [JK04]
    Jain, S., Kinber, E.: Learning languages from positive data and negative counterexamples. In: Ben-David, S., Case, J., Maruoka, A. (eds.) ALT 2004. LNCS (LNAI), vol. 3244, pp. 54–68. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  12. [JK06a]
    Jain, S., Kinber, E.: Iterative learning from positive data and negative counterexamples. Technical Report TRA3/06, School of Computing, National University of Singapore (2006)Google Scholar
  13. [JK06b]
    Jain, S., Kinber, E.: Learning languages from positive data and negative counterexamples. Journal of Computer and System Sciences (to appear, 2006)Google Scholar
  14. [JORS99]
    Jain, S., Osherson, D., Royer, J., Sharma, A.: Systems that Learn: An Introduction to Learning Theory, 2nd edn. MIT Press, Cambridge (1999)Google Scholar
  15. [LZ96]
    Lange, S., Zeugmann, T.: Incremental learning from positive data. Journal of Computer and System Sciences 53, 88–103 (1996)MATHCrossRefMathSciNetGoogle Scholar
  16. [LZ04]
    Lange, S., Zilles, S.: Comparison of query learning and gold-style learning in dependence of the hypothesis space. In: Ben-David, S., Case, J., Maruoka, A. (eds.) ALT 2004. LNCS (LNAI), vol. 3244, pp. 99–113. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  17. [Mot91]
    Motoki, T.: Inductive inference from all positive and some negative data. Information Processing Letters 39(4), 177–182 (1991)MATHCrossRefMathSciNetGoogle Scholar
  18. [Pin79]
    Pinker, S.: Formal models of language learning. Cognition 7, 217–283 (1979)CrossRefGoogle Scholar
  19. [Pop68]
    Popper, K.: The Logic of Scientific Discovery, 2nd edn. Harper Torch Books, New York (1968)Google Scholar
  20. [Rog67]
    Rogers, H.: Theory of Recursive Functions and Effective Computability. McGraw-Hill, New York (1967); Reprinted by MIT Press in 1987.Google Scholar
  21. [Wie76]
    Wiehagen, R.: Limes-Erkennung rekursiver Funktionen durch spezielle Strategien. Journal of Information Processing and Cybernetics (EIK) 12, 93–99 (1976)MATHMathSciNetGoogle Scholar
  22. [ZL95]
    Zeugmann, T., Lange, S.: A guided tour across the boundaries of learning recursive languages. In: Lange, S., Jantke, K.P. (eds.) GOSLER 1994. LNCS (LNAI), vol. 961, pp. 190–258. Springer, Heidelberg (1995)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Sanjay Jain
    • 1
  • Efim Kinber
    • 2
  1. 1.School of ComputingNational University of SingaporeSingapore
  2. 2.Department of Computer ScienceSacred Heart UniversityFairfieldU.S.A.

Personalised recommendations