Not-so-nearly-minimal-size program inference (preliminary report)

  • John Case
  • Mandayam Suraj
  • Sanjay Jain
1 Inductive Inference Theory 1.1 Inductive Inference of Recursive Functions
Part of the Lecture Notes in Computer Science book series (LNCS, volume 961)


Freivalds defined an acceptable programming system independent criterion for learning programs for functions in which the final programs were required to be both correct and “nearly” minimal size, i.e, within a computable function of being purely minimal size. Kinber showed that this parsimony requirement on final programs severely limits learning power. Nonetheless, in, for example, scientific inference, parsimony is considered highly desirable. A limcomputable function is (by definition) one computable by a procedure allowed to change its mind finitely many times about its output. Investigated is the possibility of assuaging somewhat the limitation on learning power resulting from requiring parsimonious final programs by use of criteria which require the final, correct programs to be “not-so-nearly” minimal size, e.g.; to be within a lim-computable function of actual minimal size. It is interestingly shown that some parsimony in the final program is thereby retained, yet learning power strictly increases. Also considered are lim-computable functions as above but for which notations for constructive ordinals are used to bound the number of mind changes allowed regarding the output. This is a variant of an idea introduced by Freivalds and Smith. For this ordinal complexity bounded version of lim-computability, the power of the resultant learning criteria form strict infinite hierarchies intermediate between the computable and the lim-computable cases. Many open questions are also presented.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [Bar74]
    J. M. Barzdin. Two theorems on the limiting synthesis of functions. Theory of Algorithms and Programs, Latvian State University, Riga, 88:82–88, 1974.Google Scholar
  2. [BB75]
    L. Blum and M. Blum. Toward a mathematical theory of inductive inference. Information and Control, 28:125–155, 1975.Google Scholar
  3. [Blu67]
    M. Blum. A machine independent theory of the complexity of recursive functions. Journal of the ACM, 14:322–336, 1967.Google Scholar
  4. [Che81]
    K. Chen. Tradeoffs in Machine Inductive Inference. PhD thesis, Computer Science Department, SUNY at Buffalo, 1981.Google Scholar
  5. [Che82]
    K. Chen. Tradeoffs in inductive inference of nearly minimal sized programs. Information and Control, 52:68–86, 1982.Google Scholar
  6. [CJS89]
    J. Case, S. Jain, and A. Sharma. Convergence to nearly minimal size grammars by vacillating learning machines. In R. Rivest, D. Haussler, and M.K. Warmuth, editors, Proceedings of the Second Annual Workshop on Computational Learning Theory, Santa Cruz, California, pages 189–199. Morgan Kaufmann Publishers, Inc., August 1989. Journal version in press for Journal of Computer and System Sciences.Google Scholar
  7. [CK37]
    A Church and S. C. Kleene. Formal definitions in the theory of ordinal numbers. Fundamenta Mathematicae, 28:11–21, 1937.Google Scholar
  8. [CS83]
    J. Case and C. Smith. Comparison of identification criteria for machine inductive inference. Theoretical Computer Science, 25:193–220, 1983.Google Scholar
  9. [Ers68]
    Y.L. Ershov. A hierarchy of sets II. Algebra and Logic, 7:212–232, 1968.Google Scholar
  10. [FK77]
    R. Freivalds and E. B. Kinber. Limit identification of minimal Gödel numbers. Theory of Algorithms and Programs 3;Riga 1977, pages 3–34, 1977.Google Scholar
  11. [Fre75]
    R. Freivalds. Minimal Gödel numbers and their identification in the limit. Lecture Notes in Computer Science, 32:219–225, 1975.Google Scholar
  12. [Fre90]
    R. Freivalds. Inductive inference of minimal programs. In M. Fulk and J. Case, editors, Proceedings of the Third Annual Workshop on Computational Learni ng Theory, pages 3–20. Morgan Kaufmann Publishers, Inc., August 1990.Google Scholar
  13. [FS91]
    R Freivalds and C. H. Smith. On the role of procrastination for machine learning. Technical Report TR-91-27, UMIACS, 1991. To appear in Information and Computation.Google Scholar
  14. [Gol67]
    E. M. Gold. Language identification in the limit. Information and Control, 10:447–474, 1967.Google Scholar
  15. [JS91]
    S. Jain and A. Sharma. Program size restrictions in inductive learning. In AAAI Symposium Series, Symposium: Machine Learning of Natural Language and Ontology, March 1991. Journal version accepted by Theoretical Computer Science.Google Scholar
  16. [Kin74]
    E. B. Kinber. On the synthesis in the limit of almost minimal Gödel numbers. Theory Of Algorithms and Programs, LSU, Riga, 1:221–223, 1974.Google Scholar
  17. [Kin77a]
    E. B. Kinber. On a theory of inductive inference. Lecture Notes in Computer Science, 56:435–440, 1977.Google Scholar
  18. [Kin77b]
    E. B. Kinber. On limit identification of minimal Gödel numbers for functions from enumerable classes. Theory of Algorithms and Programs 3;Riga 1977, pages 35–56, 1977.Google Scholar
  19. [Kin83]
    E. B. Kinber. A note on limit identification of c-minimal indices. Electronische Informationverarbeitung und Kybernetik, 19:459–463, 1983.Google Scholar
  20. [Kle38]
    S. C. Kleene. Notations for ordinal numbers. Journal of Symbolic Logic, 3:150–155, 1938.Google Scholar
  21. [Kle55]
    S. C. Kleene. On the forms of predicates in the theory of constructive ordinals, II. American Journal of Mathematics, 77:405–428, 1955.Google Scholar
  22. [Mar89]
    Y. Marcoux. Composition is almost as good as s-1-1. In Proceedings, Structure in Complexity Theory-Fourth Annual Conference. IEEE Computer Society Press, 1989.Google Scholar
  23. [MY78]
    M. Machtey and P. Young. An Introduction to the General Theory of Algorithms. North Holland, New York, 1978.Google Scholar
  24. [Ric80]
    G. Riccardi. The Independence of Control Structures in Abstract Programming Systems. PhD thesis, SUNY Buffalo, 1980.Google Scholar
  25. [Ric81]
    G. Riccardi. The independence of control structures in abstract programming systems. Journal of Computer and System Sciences, 22:107–143, 1981.Google Scholar
  26. [Rog58]
    H. Rogers. Gödel numberings of partial recursive functions. Journal of Symbolic Logic, 23:331–341, 1958.Google Scholar
  27. [Rog67]
    H. Rogers. Theory of Recursive Functions and Effective Computability. Mc-Graw Hill, New York, 1967. Reprinted, MIT Press, 1987.Google Scholar
  28. [Roy87]
    J. Royer. A Connotational Theory of Program Structure. Lecture Notes in Computer Science 273. Springer Verlag, 1987.Google Scholar
  29. [Sac90]
    G. E. Sacks. Higher Recursion Theory. Springer-Verlag, 1990.Google Scholar
  30. [Sha71]
    N. Shapiro. Review of “Limiting recursion” by E.M. Gold and “Trial and error predicates and the solution to a problem of Mostowski” by H. Putnam. Journal of Symbolic Logic, 36:342, 1971.Google Scholar
  31. [Sie65]
    W. Sierpinski. Cardinal and ordinal numbers. PWN-Polish Scientific Publishers, 1965. Second revised edition.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1995

Authors and Affiliations

  • John Case
    • 1
  • Mandayam Suraj
    • 1
  • Sanjay Jain
    • 2
  1. 1.Department of Computer and Information SciencesUniversity of DelawareNewarkUSA
  2. 2.Institute of Systems ScienceNational University of SingaporeSingaporeRepublic of Singapore

Personalised recommendations