Parallelism Increases Iterative Learning Power

  • John Case
  • Samuel E. MoeliusIII
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4754)


Iterative learning (\(\textbf{It}\)-learning) is a Gold-style learning model in which each of a learner’s output conjectures may depend only upon the learner’s current conjecture and the current input element. Two extensions of the \(\textbf{It}\)-learning model are considered, each of which involves parallelism. The first is to run, in parallel, distinct instantiations of a single learner on each input element. The second is to run, in parallel, n individual learners incorporating the first extension, and to allow the n learners to communicate their results. In most contexts, parallelism is only a means of improving efficiency. However, as shown herein, learners incorporating the first extension are more powerful than \(\textbf{It}\)-learners, and, collective learners resulting from the second extension increase in learning power as n increases. Attention is paid to how one would actually implement a learner incorporating each extension. Parallelism is the underlying mechanism employed.


Input Sequence Initial Segment Individual Learner Computable Function Total Function 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Arikawa, S., Miyano, S., Shinohara, A., Kuhara, S., Mukouchi, Y., Shinohara, T.: A machine discovery from amino-acid-sequences by decision trees over regular patterns. New Generation Computing 11, 361–375 (1993)CrossRefzbMATHGoogle Scholar
  2. Angluin, D.: Finding patterns common to a set of strings. Journal of Computer and System Sciences 21, 46–62 (1980)MathSciNetCrossRefzbMATHGoogle Scholar
  3. Brazma, A., Ukkonen, E., Vilo, J.: Discovering unbounded unions of regular pattern languages from positive examples. In: Nagamochi, H., Suri, S., Igarashi, Y., Miyano, S., Asano, T. (eds.) ISAAC 1996. LNCS, vol. 1178, Springer, Heidelberg (1996)Google Scholar
  4. Case, J.: Periodicity in generations of automata. Mathematical Systems Theory 8, 15–32 (1974)MathSciNetCrossRefzbMATHGoogle Scholar
  5. Case, J.: Infinitary self-reference in learning theory. Journal of Experimental and Theoretical Artificial Intelligence 6, 3–16 (1994)CrossRefzbMATHGoogle Scholar
  6. Carlucci, L., Case, J., Jain, S., Stephan, F.: Memory-limited U-shaped learning. In: Lugosi, G., Simon, H.U. (eds.) COLT 2006. LNCS (LNAI), vol. 4005, pp. 244–258. Springer, Heidelberg (2006)Google Scholar
  7. Case, J., Jain, S., Kaufmann, S., Sharma, A., Stephan, F.: Predictive learning models for concept drift. Theoretical Computer Science, Special Issue for ALT 1998, 268, 323–349 (2001)Google Scholar
  8. Case, J., Jain, S., Lange, S., Zeugmann, T.: Incremental concept learning for bounded data mining. Information and Computation 152, 74–110 (1999)MathSciNetCrossRefzbMATHGoogle Scholar
  9. Case, J., Lynes, C.: Machine inductive inference and language identification. In: Nielsen, M., Schmidt, E.M. (eds.) Automata, Languages, and Programming. LNCS, vol. 140, pp. 107–115. Springer, Heidelberg (1982)CrossRefGoogle Scholar
  10. Case, J., Moelius, S.E.: U-shaped, iterative, and iterative-with-counter learning. In: COLT 2007. LNCS(LNAI), vol. 4539, pp. 172–186. Springer, Berlin (2007)Google Scholar
  11. Case, J., Moelius, S.E.: U-shaped, iterative, and iterative-with-counter learning (expanded version). Technical report, University of Delaware (2007), Available at
  12. Davis, M., Sigal, R., Weyuker, E.: Computability, Complexity, and Languages, 2nd edn. Academic Press, London (1994)Google Scholar
  13. Fulk, M., Jain, S., Osherson, D.: Open problems in Systems That Learn. Journal of Computer and System Sciences 49(3), 589–604 (1994)MathSciNetCrossRefGoogle Scholar
  14. Gold, E.: Language identification in the limit. Information and Control 10, 447–474 (1967)MathSciNetCrossRefzbMATHGoogle Scholar
  15. Jain, S., Osherson, D., Royer, J., Sharma, A.: Systems that Learn: An Introduction to Learning Theory, 2nd edn. MIT Press, Cambridge (1999)Google Scholar
  16. Kilpeläinen, P., Mannila, H., Ukkonen, E.: MDL learning of unions of simple pattern languages from positive examples. In: Vitányi, P.M.B. (ed.) EuroCOLT 1995. LNCS, vol. 904, pp. 252–260. Springer, Heidelberg (1995)CrossRefGoogle Scholar
  17. Lange, S., Wiehagen, R.: Polynomial time inference of arbitrary pattern languages. New Generation Computing 8, 361–370 (1991)CrossRefzbMATHGoogle Scholar
  18. Lange, S., Zeugmann, T.: Incremental learning from positive data. Journal of Computer and System Sciences 53, 88–103 (1996)MathSciNetCrossRefzbMATHGoogle Scholar
  19. Nix, R.: Editing by examples. Technical Report 280, Department of Computer Science, Yale University, New Haven, CT, USA (1983)Google Scholar
  20. Osherson, D., Stob, M., Weinstein, S.: Systems that Learn: An Introduction to Learning Theory for Cognitive and Computer Scientists. MIT Press, Cambridge (1986)Google Scholar
  21. Rogers, H.: Theory of Recursive Functions and Effective Computability. MIT Press, Cambridge (1967) (reprinted, MIT Press, 1987)Google Scholar
  22. Shinohara, T., Arikawa, A.: Pattern inference. In: Lange, S., Jantke, K.P. (eds.) Algorithmic Learning for Knowledge-Based Systems. LNCS, vol. 961, pp. 259–291. Springer, Heidelberg (1995)CrossRefGoogle Scholar
  23. Salomaa, A.: Patterns (The Formal Language Theory Column). EATCS Bulletin 54, 46–62 (1994)Google Scholar
  24. Salomaa, A.: Return to patterns (The Formal Language Theory Column). EATCS Bulletin 55, 144–157 (1994)Google Scholar
  25. Shinohara, T.: Inferring unions of two pattern languages. Bulletin of Informatics and Cybernetics 20, 83–88 (1983)MathSciNetzbMATHGoogle Scholar
  26. Shimozono, S., Shinohara, A., Shinohara, T., Miyano, S., Kuhara, S., Arikawa, S.: Knowledge acquisition from amino acid sequences by machine learning system BONSAI. Trans. Information Processing Society of Japan 35, 2009–2018 (1994)Google Scholar
  27. Wiehagen, R.: Limes-erkennung rekursiver funktionen durch spezielle strategien. Electronische Informationverarbeitung und Kybernetik 12, 93–99 (1976)MathSciNetzbMATHGoogle Scholar
  28. Winskel, G.: The Formal Semantics of Programming Languages: An Introduction. In: Foundations of Computing Series, MIT Press, Cambridge (1993)Google Scholar
  29. Wright, K.: Identification of unions of languages drawn from an identifiable class. In: Rivest, R., Haussler, D., Warmuth, M. (eds.) Proceedings of the Second Annual Workshop on Computational Learning Theory, Santa Cruz, California, pp. 328–333. Morgan Kaufmann, San Francisco (1989)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  • John Case
    • 1
  • Samuel E. MoeliusIII
    • 1
  1. 1.Department of Computer & Information Sciences, University of Delaware, 103 Smith Hall, Newark, DE 19716USA

Personalised recommendations