Encyclopedia of Machine Learning

2010 Edition
| Editors: Claude Sammut, Geoffrey I. Webb

Complexity of Inductive Inference

  • Sanjay Jain
  • Frank Stephan
Reference work entry
DOI: https://doi.org/10.1007/978-0-387-30164-8_149


In  inductive inference, the complexity of learning can be measured in various ways: by the number of hypotheses issued in the worst case until the correct hypothesis is found; by the number of data items to be consumed or to be memorized in order to learn in the worst case; by the Turing degree of oracles needed to learn the class under a certain criterion; by the intrinsic complexity which is – like the Turing degrees in recursion theory – a way to measure the complexity of classes by using reducibilities between them.


We refer the reader to the article  Inductive Inference for basic definitions in inductive inference and the notations used below. Let \(\mathbb{N}\)

This is a preview of subscription content, log in to check access.

Recommended Reading

  1. Adleman, L., & Blum, M. (1991). Inductive inference and unsolvability. Journal of Symbolic Logic, 56, 891–900.MathSciNetMATHGoogle Scholar
  2. Angluin, D. (1980). Finding patterns common to a set of strings. Journal of Computer and System Sciences, 21, 46–62.MathSciNetMATHGoogle Scholar
  3. Case, J., Jain, S., & Lange, S., & Zeugmann, T. (1999). Incremental concept learning for bounded data mining. Information and Computation, 152(1), 74–110.MathSciNetMATHGoogle Scholar
  4. Case, J., & Smith, C. H. (1983). Comparison of identification criteria for machine inductive inference. Theoretical Computer Science, 25, 193–220.MathSciNetMATHGoogle Scholar
  5. Daley, R. P., & Smith, C. H. (1986). On the complexity of inductive inference. Information and Control, 69, 12–40.MathSciNetMATHGoogle Scholar
  6. Chen, K.-J. (1982). Tradeoffs in inductive inference of nearly minimal sized programs. Information and Control, 52, 68–86.MathSciNetMATHGoogle Scholar
  7. Freivalds, R. (1975). Minimal Gödel numbers and their identification in the limit. Lecture Notes in Computer Science, 32, 219–225.MathSciNetGoogle Scholar
  8. Freivalds, R., Kinber, E., & Smith, C. H. (1995). On the intrinsic complexity of learning. Information and Computation, 123, 64–71.MathSciNetMATHGoogle Scholar
  9. Freivalds, R., & Smith, C. H. (1993). On the role of procrastination in machine learning. Information and Computation, 107(2), 237–271.MathSciNetMATHGoogle Scholar
  10. Jain, S., Kinber, E., & Wiehagen, R. (2001). Language learning from texts: Degrees of intrinsic complexity and their characterizations. Journal of Computer and System Sciences, 63, 305–354.MathSciNetMATHGoogle Scholar
  11. Jain, S., & Sharma, A. (1993). On the non-existence of maximal inference degrees for language identification. Information Processing Letters, 47, 81–88.MathSciNetMATHGoogle Scholar
  12. Jain, S., & Sharma, A. (1994). Program size restrictions in computational learning. Theoretical Computer Science, 127, 351–386.MathSciNetMATHGoogle Scholar
  13. Jain, S., & Sharma, A. (1996). The intrinsic complexity of language identification. Journal of Computer and System Sciences, 52, 393–402.MathSciNetMATHGoogle Scholar
  14. Jain, S., & Sharma, A. (1997). The structure of intrinsic complexity of learning. Journal of Symbolic Logic, 62, 1187–1201.MathSciNetMATHGoogle Scholar
  15. Jain, S., & Sharma, A. (1997). Elementary formal systems, intrinsic complexity and procrastination. Information and Computation, 132, 65–84.MathSciNetMATHGoogle Scholar
  16. Lange, S., & Zeugmann, T. (1996). Incremental learning from positive data. Journal of Computer and System Sciences, 53, 88–103.MathSciNetMATHGoogle Scholar
  17. Pitt, L. (1989). Inductive inference, DFAs, and computational complexity. Analogical and inductive inference, second international workshop, AII 1989, LNAI. (Vol. 397, pp. 18–44) Heidelberg: Springer.Google Scholar
  18. Rogers, H. (1967). Theory of recursive functions and effective computability. New York: McGraw-Hill (Reprinted, MIT Press 1987).Google Scholar
  19. Shinohara, T. (1994). Rich classes inferable from positive data: Length–bounded elementary formal systems. Information and Computation, 108, 175–186.MathSciNetMATHGoogle Scholar
  20. Slaman, T. A. & Solovay, R. (1991). When oracles do not help. Proceedings of the Fourth Annual Workshop on Computational Learning Theory, (pp. 379–383), Morgan Kaufmann.Google Scholar
  21. Wiehagen, R. (1976). Limes-Erkennung rekursiver Funktionen durch spezielle Strategien. Journal of Information Processing and Cybernetics (EIK), 12, 93–99.Google Scholar
  22. Wiehagen, R. (1986). On the complexity of effective program synthesis. In: K. Jantke (Ed.). Analogical and Inductive Inference. Proceedings of the International Workshop, Springer LNCS, (Vol. 265, pp. 209–219).Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2011

Authors and Affiliations

  • Sanjay Jain
  • Frank Stephan

There are no affiliations available