Inductive programming systems characteristically exhibit an exponential explosion in search time as one increases the size of the programs to be generated. As a way of overcoming this, we introduce incremental learning, a process in which an inductive programming system automatically modifies its inductive bias towards some domain through solving a sequence of gradually more difficult problems in that domain. We demonstrate a simple form of incremental learning in which a system incorporates solution programs into its background knowledge as it progresses through a sequence of problems. Using a search-based inductive functional programming system modelled on the MagicHaskeller system of Katayama [4], we perform a set of experiments comparing the performance of inductive programming with and without incremental learning. Incremental learning is shown to produce a performance improvement of at least a factor of thirty on each of the four problem sequences tested. We describe how, given some assumptions, inductive programming with incremental learning can be shown to have a polynomial, rather than exponential, time complexity with respect to the size of the program to be generated. We discuss the difficulties involved in constructing suitable problem sequences for our incremental learning system, and consider what improvements can be made to overcome these difficulties.


Inductive programming inductive functional programming incremental learning 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Augusteijn, L.: Sorting morphisms. In: Swierstra, S.D., Oliveira, J.N. (eds.) AFP 1998. LNCS, vol. 1608, pp. 1–27. Springer, Heidelberg (1999)CrossRefGoogle Scholar
  2. 2.
    De Raedt, L., Kersting, K.: Probabilistic inductive logic programming. In: Ben-David, S., Case, J., Maruoka, A. (eds.) ALT 2004. LNCS (LNAI), vol. 3244, pp. 19–36. Springer, Heidelberg (2004)Google Scholar
  3. 3.
    Hofmann, M., Kitzelmann, E., Schmid, U.: A unifying framework for analysis and evaluation of inductive programming systems. In: Proceedings of the Second Conference on Artificial General Intelligence, AGI 2009 (2009)Google Scholar
  4. 4.
    Katayama, S.: Systematic search for lambda expressions. In: Revised Selected Papers from the Sixth Symposium on Trends in Functional Programming, TFP 2005, vol. 6, pp. 111–126. Intellect (2007)Google Scholar
  5. 5.
    Khan, K., Muggleton, S., Parson, R.: Repeat learning using predicate invention. In: Page, D.L. (ed.) ILP 1998. LNCS, vol. 1446, pp. 165–174. Springer, Heidelberg (1998)CrossRefGoogle Scholar
  6. 6.
    Kitzelmann, E.: Data-driven induction of recursive functions from input/output-examples. In: Proceedings of the ECML/PKDD 2007 Workshop on Approaches and Applications of Inductive Programming (AAIP 2007), pp. 15–26 (2007)Google Scholar
  7. 7.
    Muggleton, S.: Duce, an oracle based approach to constructive induction. In: IJCAI 1987, pp. 287–292 (1987)Google Scholar
  8. 8.
    Olsson, J.R.: Inductive functional programming using incremental program transformation. Artificial Intelligence 74(1), 55–83 (1995)CrossRefGoogle Scholar
  9. 9.
    Quinlan, J.R., Cameron-Jones, R.M.: FOIL: A midterm report. In: Brazdil, P.B. (ed.) ECML 1993. LNCS, vol. 667, pp. 3–20. Springer, Heidelberg (1993)Google Scholar
  10. 10.
    Schmidhuber, J.: Optimal ordered problem solver. Machine Learning 54(3), 211–254 (2004)zbMATHCrossRefGoogle Scholar
  11. 11.
    Schmidhuber, J., Zhao, J., Wiering, M.: Shifting inductive bias with success-story algorithm, adaptive levin search, and incremental self-improvement. Machine Learning 28(1), 105–130 (1997)CrossRefGoogle Scholar
  12. 12.
    Solomonoff, R.J.: Progress in incremental machine learning. Given at: NIPS Workshop on Universal Learning Algorithms and Optimal Search. Whistler, B.C., Canada, December 14 (2002),

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Robert Henderson
    • 1
  1. 1.Department of ComputingImperial College LondonUnited Kingdom

Personalised recommendations