Incremental Learning in Inductive Programming
Inductive programming systems characteristically exhibit an exponential explosion in search time as one increases the size of the programs to be generated. As a way of overcoming this, we introduce incremental learning, a process in which an inductive programming system automatically modifies its inductive bias towards some domain through solving a sequence of gradually more difficult problems in that domain. We demonstrate a simple form of incremental learning in which a system incorporates solution programs into its background knowledge as it progresses through a sequence of problems. Using a search-based inductive functional programming system modelled on the MagicHaskeller system of Katayama , we perform a set of experiments comparing the performance of inductive programming with and without incremental learning. Incremental learning is shown to produce a performance improvement of at least a factor of thirty on each of the four problem sequences tested. We describe how, given some assumptions, inductive programming with incremental learning can be shown to have a polynomial, rather than exponential, time complexity with respect to the size of the program to be generated. We discuss the difficulties involved in constructing suitable problem sequences for our incremental learning system, and consider what improvements can be made to overcome these difficulties.
KeywordsInductive programming inductive functional programming incremental learning
Unable to display preview. Download preview PDF.
- 2.De Raedt, L., Kersting, K.: Probabilistic inductive logic programming. In: Ben-David, S., Case, J., Maruoka, A. (eds.) ALT 2004. LNCS (LNAI), vol. 3244, pp. 19–36. Springer, Heidelberg (2004)Google Scholar
- 3.Hofmann, M., Kitzelmann, E., Schmid, U.: A unifying framework for analysis and evaluation of inductive programming systems. In: Proceedings of the Second Conference on Artificial General Intelligence, AGI 2009 (2009)Google Scholar
- 4.Katayama, S.: Systematic search for lambda expressions. In: Revised Selected Papers from the Sixth Symposium on Trends in Functional Programming, TFP 2005, vol. 6, pp. 111–126. Intellect (2007)Google Scholar
- 6.Kitzelmann, E.: Data-driven induction of recursive functions from input/output-examples. In: Proceedings of the ECML/PKDD 2007 Workshop on Approaches and Applications of Inductive Programming (AAIP 2007), pp. 15–26 (2007)Google Scholar
- 7.Muggleton, S.: Duce, an oracle based approach to constructive induction. In: IJCAI 1987, pp. 287–292 (1987)Google Scholar
- 9.Quinlan, J.R., Cameron-Jones, R.M.: FOIL: A midterm report. In: Brazdil, P.B. (ed.) ECML 1993. LNCS, vol. 667, pp. 3–20. Springer, Heidelberg (1993)Google Scholar
- 12.Solomonoff, R.J.: Progress in incremental machine learning. Given at: NIPS Workshop on Universal Learning Algorithms and Optimal Search. Whistler, B.C., Canada, December 14 (2002), http://world.std.com/~rjs/pubs.html