Skip to main content
SpringerLink
Log in
Book cover

International Conference on Algorithmic Learning Theory

ALT 2003: Algorithmic Learning Theory pp 17–38Cite as

Can Learning in the Limit Be Done Efficiently?

  • Conference paper

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 2842))

Abstract

Inductive inference can be considered as one of the fundamental paradigms of algorithmic learning theory. We survey results recently obtained and show their impact to potential applications.

Since the main focus is put on the efficiency of learning, we also deal with postulates of naturalness and their impact to the efficiency of limit learners. In particular, we look at the learnability of the class of all pattern languages and ask whether or not one can design a learner within the paradigm of learning in the limit that is nevertheless efficient.

For achieving this goal, we deal with iterative learning and its interplay with the hypothesis spaces allowed. This interplay has also a severe impact to postulates of naturalness satisfiable by any learner.

Finally, since a limit learner is only supposed to converge in the limit, one never knows at any particular learning stage whether or not the learner did already succeed. The resulting uncertainty may be prohibitive in many applications. We survey results to resolve this problem by outlining a new learning model, called stochastic finite learning. Though pattern languages can neither be finitely inferred from positive data nor PAC-learned, our approach can be extended to a stochastic finite learner that exactly infers all pattern languages from positive data with high confidence.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Angluin, D.: Finding Patterns common to a Set of Strings. Journal of Computer and System Sciences 21, 46–62 (1980)

    Article  MATH  MathSciNet  Google Scholar 

  2. Angluin, D.: Inductive inference of formal languages from positive data. Information and Control 45, 117–135 (1980)

    Article  MATH  MathSciNet  Google Scholar 

  3. Angluin, D., Smith, C.H.: Inductive inference: Theory and methods. Computing Surveys 15(3), 237–269 (1983)

    Article  MathSciNet  Google Scholar 

  4. Angluin, D., Smith, C.H.: Formal inductive inference. In: Shapiro, S.C. (ed.) Encyclopedia of Artificial Intelligence, vol. 1, pp. 409–418. Wiley-Interscience Publication, New York

    Google Scholar 

  5. Arikawa, S., Shinohara, T., Yamamoto, A.: Learning elementary formal systems. Theoretical Computer Science 95, 97–113 (1992)

    Article  MATH  MathSciNet  Google Scholar 

  6. Blum, L., Blum, M.: Toward a mathematical theory of inductive inference. Information and Control 28, 125–155 (1975)

    Article  MATH  MathSciNet  Google Scholar 

  7. Blumer, A., Ehrenfeucht, A., Haussler, D., Warmuth, M.: Learnability and the Vapnik-Chervonenkis Dimension. Journal of the ACM 36, 929–965 (1989)

    Article  MATH  MathSciNet  Google Scholar 

  8. Bratko, I., Muggleton, S.: Applications of inductive logic programming. Communications of the ACM (1995)

    Google Scholar 

  9. Case, J., Jain, S., Lange, S., Zeugmann, T.: Incremental Concept Learning for Bounded Data Mining. Information and Computation 152(1), 74–110 (1999)

    Article  MATH  MathSciNet  Google Scholar 

  10. Daley, R., Smith, C.H.: On the Complexity of Inductive Inference. Information and Control 69, 12–40 (1986)

    Article  MATH  MathSciNet  Google Scholar 

  11. Erlebach, T., Rossmanith, P., Stadtherr, H., Steger, A., Zeugmann, T.: Learning one-variable pattern languages very efficiently on average, in parallel, and by asking queries. Theoretical Computer Science 261(1-2), 119–156 (2001)

    Article  MATH  MathSciNet  Google Scholar 

  12. Gold, E.M.: Language identification in the limit. Information and Control 10, 447–474 (1967)

    Article  MATH  Google Scholar 

  13. Goldman, S.A., Kearns, M.J., Schapire, R.E.: Exact identification of circuits using fixed points of amplification functions. SIAM Journal of Computing 22, 705–726 (1993)

    Article  MATH  MathSciNet  Google Scholar 

  14. Haussler, D.: Bias, version spaces and Valiant’s learning framework. In: Proc. 8th National Conference on Artificial Intelligence, pp. 564–569. Morgan Kaufmann, San Mateo (1987)

    Google Scholar 

  15. Haussler, D., Kearns, M., Littlestone, N., Warmuth, M.K.: Equivalence of models for polynomial learnability. Information and Computation 95, 129–161 (1991)

    Article  MATH  MathSciNet  Google Scholar 

  16. Jain, S., Osherson, D., Royer, J.S., Sharma, A.: Systems That Learn: An Introduction to Learning Theory. MIT-Press, Boston (1999)

    Google Scholar 

  17. Jiang, T., Salomaa, A., Salomaa, K., Yu, S.: Inclusion is undecidable for pattern languages. In: Lingas, A., Carlsson, S., Karlsson, R. (eds.) ICALP 1993. LNCS, vol. 700, pp. 301–312. Springer, Heidelberg (1993)

    Google Scholar 

  18. Kearns, M., Pitt, L.: A polynomial-time algorithm for learning k-variable pattern languages from examples. In: Proc. Second Annual ACM Workshop on Computational Learning Theory, pp. 57–71. Morgan Kaufmann, San Mateo (1989)

    Google Scholar 

  19. Lange, S., Wiehagen, R.: Polynomial-time inference of arbitrary pattern languages. New Generation Computing 8, 361–370 (1991)

    Article  MATH  Google Scholar 

  20. Lange, S., Zeugmann, T.: Language learning in dependence on the space of hypotheses. In: Pitt, L. (ed.) Proc. of the 6th Annual ACM Conference on Computational Learning Theory, pp. 127–136. ACM Press, New York (1993)

    Chapter  Google Scholar 

  21. Lange, S., Zeugmann, T.: Set-driven and Rearrangement-independent Learning of Recursive Languages. Mathematical Systems Theory 29, 599–634 (1996)

    MATH  MathSciNet  Google Scholar 

  22. Lange, S., Zeugmann, T.: Incremental Learning from Positive Data. Journal of Computer and System Sciences 53, 88–103 (1996)

    Article  MATH  MathSciNet  Google Scholar 

  23. Lavrač, N., Džeroski, S.: Inductive Logic Programming: Techniques and Applications. Ellis Horwood (1994)

    Google Scholar 

  24. Mitchell, T.: Machine Learning. McGraw Hill, New York (1997)

    MATH  Google Scholar 

  25. Mitchell, A., Sharma, A., Scheffer, T., Stephan, F.: The VC-dimension of Subclasses of Pattern Languages. In: Watanabe, O., Yokomori, T. (eds.) ALT 1999. LNCS (LNAI), vol. 1720, pp. 93–105. Springer, Heidelberg (1999)

    Chapter  Google Scholar 

  26. Miyano, S., Shinohara, A., Shinohara, T.: Polynomial-time learning of elementary formal systems. New Generation Computing 18, 217–242 (2000)

    Article  Google Scholar 

  27. Muggleton, S.: Bayesian Inductive Logic Programming. In: Warmuth, M. (ed.) Proc. 7th Annual ACM Conference on Computational Learning Theory, pp. 3–11. ACM Press, New York (1994)

    Chapter  Google Scholar 

  28. Muggleton, S., De Raedt, L.: Inductive logic programming: Theory and methods. Journal of Logic Programming 19/20, 669–679 (1994)

    Article  Google Scholar 

  29. Nix, R.P.: Editing by examples, Yale University, Dept. Computer Science, Technical Report 280 (1983)

    Google Scholar 

  30. Osherson, D.N., Stob, M., Weinstein, S.: Systems that Learn, An Introduction to Learning Theory for Cognitive and Computer Scientists. MIT-Press, Cambridge (1986)

    Google Scholar 

  31. Pitt, L.: Inductive Inference, DFAs and Computational Complexity. In: Jantke, K.P. (ed.) AII 1989. LNCS, vol. 397, pp. 18–44. Springer, Heidelberg (1989)

    Google Scholar 

  32. Reischuk, R., Zeugmann, T.: Learning One- Variable Pattern Languages in Linear Average Time. In: Proc. 11th Annual Conference on Computational Learning Theory - COLT 1998, Madison, July 24th - 26th, pp. 198–208. ACM Press, New York (1998)

    Chapter  Google Scholar 

  33. Reischuk, R., Zeugmann, T.: A Complete and Tight Average-Case Analysis of Learning Monomials. In: Meinel, C., Tison, S. (eds.) STACS 1999. LNCS, vol. 1563, pp. 414–423. Springer, Heidelberg (1999)

    Chapter  Google Scholar 

  34. Reischuk, R., Zeugmann, T.: An Average-Case Optimal One-Variable Pattern Language Learner. Journal of Computer and System Sciences 60(2), 302–335 (2000)

    Article  MATH  MathSciNet  Google Scholar 

  35. Rogers Jr., H.: Theory of Recursive Functions and Effective Computability. McGraw–Hill, New York (1967)

    MATH  Google Scholar 

  36. Rossmanith, P., Zeugmann, T.: Stochastic Finite Learning of the Pattern Languages. Machine Learning 44(1-2), 67–91 (2001)

    Article  MATH  Google Scholar 

  37. Patterns (The Formal Language Theory Column). EATCS Bulletin 54, 46–62 (1994)

    Google Scholar 

  38. Return to patterns (The Formal Language Theory Column). EATCS Bulletin 55, 144–157 (1994)

    Google Scholar 

  39. Shimozono, S., Shinohara, A., Shinohara, T., Miyano, S., Kuhara, S., Arikawa, S.: Knowledge acquisition from amino acid sequences by machine learning system BONSAI. Trans. Information Processing Society of Japan 35, 2009–2018 (1994)

    Google Scholar 

  40. Schapire, R.E.: Pattern languages are not learnable. In: Fulk, M.A., Case, J. (eds.) Proceedings of the Third Annual ACM Workshop on Computational Learning Theory, pp. 122–129. Morgan Kaufmann, San Mateo (1990)

    Google Scholar 

  41. Shinohara, T.: Inferring unions of two pattern languages. Bulletin of Informatics and Cybernetics 20, 83–88 (1983)

    MATH  MathSciNet  Google Scholar 

  42. Shinohara, T.: Inductive inference of monotonic formal systems from positive data. New Generation Computing 8, 371–384 (1991)

    Article  MATH  Google Scholar 

  43. Shinohara, T., Arikawa, S.: Pattern inference. In: Lange, S., Jantke, K.P. (eds.) GOSLER 1994. LNCS (LNAI), vol. 961, pp. 259–291. Springer, Heidelberg (1995)

    Google Scholar 

  44. Shinohara, T., Arimura, H.: Inductive inference of unbounded unions of pattern languages from positive data. In: Arikawa, S., Sharma, A.K. (eds.) ALT 1996. LNCS (LNAI), vol. 1160, pp. 256–271. Springer, Heidelberg (1996)

    Google Scholar 

  45. Smullyan, R.: Theory of Formal Systems. Annals of Mathematical Studies, Princeton, NJ, vol. (47) (1961)

    Google Scholar 

  46. Valiant, L.G.: A Theory of the Learnable. Communications of the ACM 27, 1134–1142 (1984)

    Article  MATH  Google Scholar 

  47. Wiehagen, R.: Limes-Erkennung rekursiver Funktionen durch spezielle Strategien. Journal of Information Processing and Cybernetics (EIK) 12, 93–99 (1976)

    MATH  MathSciNet  Google Scholar 

  48. Wiehagen, R., Zeugmann, T.: Ignoring Data may be the only Way to Learn Efficiently. Journal of Experimental and Theoretical Artificial Intelligence 6, 131–144 (1994)

    Article  MATH  Google Scholar 

  49. Wright, K.: Identification of unions of languages drawn from an identifiable class. In: Rivest, R., Haussler, D., Warmuth, M. (eds.) Proceedings of the 2nd Workshop on Computational Learning Theory, pp. 328–333. Morgan Kaufmann, San Mateo (1989)

    Google Scholar 

  50. Zeugmann, T.: Lange and Wiehagen’s Pattern Language Learning Algorithm: An Average-case Analysis with respect to its Total Learning Time. Annals of Mathematics and Artificial Intelligence 23(1-2), 117–145 (1998)

    Article  MATH  MathSciNet  Google Scholar 

  51. Zeugmann, T., Lange, S.: A guided tour across the boundaries of learning recursive languages. In: Lange, S., Jantke, K.P. (eds.) GOSLER 1994. LNCS, vol. 961, pp. 190–258. Springer, Heidelberg (1995)

    Google Scholar 

  52. Zeugmann, T., Lange, S., Kapur, S.: Characterizations of monotonic and dual monotonic language learning. Information and Computation 120, 155–173 (1995)

    Article  MATH  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institut für Theoretische Informatik, Universität zu Lübeck, Wallstraße 40, 23560, Lübeck, Germany

    Thomas Zeugmann

Authors
  1. Thomas Zeugmann
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Universitat Politècnica de Catalunya, Barcelona, Spain

    Ricard Gavaldá

  2. Meme Media Laboratory, Hokkaido University Sapporo, Kita 13, Nishi 8, Kita-ku, 060-8628, Sapporo, Japan

    Klaus P. Jantke

  3.  ,  

    Eiji Takimoto

Rights and permissions

Reprints and permissions

Copyright information

© 2003 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Zeugmann, T. (2003). Can Learning in the Limit Be Done Efficiently?. In: Gavaldá, R., Jantke, K.P., Takimoto, E. (eds) Algorithmic Learning Theory. ALT 2003. Lecture Notes in Computer Science(), vol 2842. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-39624-6_5

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-39624-6_5

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-20291-2

  • Online ISBN: 978-3-540-39624-6

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation