Learning an Approximation to Inductive Logic Programming Clause Evaluation

  • Frank DiMaio
  • Jude Shavlik
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3194)


One challenge faced by many Inductive Logic Programming (ILP) systems is poor scalability to problems with large search spaces and many examples. Randomized search methods such as stochastic clause selection (SCS) and rapid random restarts (RRR) have proven somewhat successful at addressing this weakness. However, on datasets where hypothesis evaluation is computationally expensive, even these algorithms may take unreasonably long to discover a good solution. We attempt to improve the performance of these algorithms on datasets by learning an approximation to ILP hypothesis evaluation. We generate a small set of hypotheses, uniformly sampled from the space of candidate hypotheses, and evaluate this set on actual data. These hypotheses and their corresponding evaluation scores serve as training data for learning an approximate hypothesis evaluator. We outline three techniques that make use of the trained evaluation-function approximator in order to reduce the computation required during an ILP hypothesis search. We test our approximate clause evaluation algorithm using the popular ILP system Aleph. Empirical results are provided on several benchmark datasets. We show that the clause evaluation function can be accurately approximated.


Local Search Inductive Logic Programming Trained Neural Network Candidate Clause Inductive Logic Programming System 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Lavrac, N., Dzeroski, S.: Inductive Logic Programming. Ellis Horwood (1994)Google Scholar
  2. 2.
    King, R., Muggleton, S., Sternberg, M.: Predicting protein secondary structure using inductive logic programming. Protein Engineering 5, 647–657 (1992)CrossRefGoogle Scholar
  3. 3.
    Srinivasan, A., King, R., Muggleton, S., Sternberg, M.: The predictive toxicology evaluation challenge. In: Proc. 15th Intl. Joint Conf. on Artificial Intelligence, pp. 1–6 (1997)Google Scholar
  4. 4.
    Dolsak, B., Muggleton, S.: The application of ILP to finite element mesh design. In: Proc. 1st Intl. Workshop on ILP, pp. 225–242 (1991)Google Scholar
  5. 5.
    Zelle, J., Mooney, R.: Learning semantic grammars with constructive inductive logic programming. In: Proc. 11th Natl. Conf. on Artificial Intelligence, pp. 817–822 (1993)Google Scholar
  6. 6.
    Bratko, I., Grobelnik, M.: Inductive learning applied to program construction and verification. In: Proc. 3rd Intl. Workshop on Inductive Logic Programming, pp. 169–182 (1993)Google Scholar
  7. 7.
    Nienhuys-Cheng, S., de Wolf, R.: Foundations of Inductive Logic Programming. Springer, Heidelberg (1997)Google Scholar
  8. 8.
    Schmidt-Schauss, M.: Implication of clauses is undecidable. Theoretical Computer Science 59, 287–296 (1988)zbMATHCrossRefMathSciNetGoogle Scholar
  9. 9.
    Quinlan, J.: Learning logical definitions from relations. Machine Learning, 239–266 (1990)Google Scholar
  10. 10.
    Muggleton, S., Feng, C.: Efficient induction of logic programs. In: Proc. 1st Conf. on Algorithmic Learning Theory, pp. 368–381 (1990)Google Scholar
  11. 11.
    Muggleton, S.: Inverse Entailment and Progol. New Generation Computing 13, 245–286 (1995)CrossRefGoogle Scholar
  12. 12.
    Srinivasan, A.: A study of two probabilistic methods for searching large spaces with ILP. Tech. Report PRG-TR-16-00. Oxford Univ. Computing Lab (2000)Google Scholar
  13. 13.
    Zelezny, F., Srinivasan, A., Page, D.: Lattice-search runtime distributions may be heavy-tailed. In: Proc. 12th Intl. Conf. on Inductive Logic Programming,pp. 333-345 (2002)Google Scholar
  14. 14.
    Giordana, L.: Saitta & F. Zini, Learning disjunctive concepts by means of genetic algorithms. In: Proc. 11th Intl. Conf. on Machine Learning ,pp. 96-104 (1994)Google Scholar
  15. 15.
    Hanschke, P., Wurtz, J.: Satisfiability of the smallest binary program. Info. Proc. Letters 496, 237–241 (1993)CrossRefMathSciNetGoogle Scholar
  16. 16.
    Dantsin, E., Eiter, T., Gottlob, G., Voronkov, A.: Complexity and expressive power of logic programming. ACM Computing Surveys 33, 374–425 (2001)CrossRefGoogle Scholar
  17. 17.
    Rückert, U., Kramer, S.: Stochastic local search in k-term DNF learning. In: Proc. 20th Intl. Conf. on Machine Learning,pp. 648-655 (2003)Google Scholar
  18. 18.
    Blockeel, H., Dehasp, L., Demoen, B., Janssens, G., Ramon, J., Vandecasteele, H.: Improving the efficiency of inductive logic programming through the use of query packs. J. AI Research 16, 135–166 (2002)zbMATHGoogle Scholar
  19. 19.
    Santos Costa, V., Srinivasan, A., Camacho, R., Blockeel, H., Demoen, B., Janssens, G., Struyf, J., Van de casteele, H., Van Laer, W.: Query transformations for improving the efficiency of ILP systems. J. Machine Learning Research 4, 465–491 (2003)CrossRefGoogle Scholar
  20. 20.
    Srinivasan, A.: A study of two sampling methods for analysing large datasets with ILP. Data Mining and Knowledge Discovery 3, 95–123 (1999)CrossRefGoogle Scholar
  21. 21.
    Sebag, M., Rouveirol, C.: Resource-bounded relational reasoning: induction and deduction through stochastic matching. Machine Learning 38, 41–62 (2000)zbMATHCrossRefGoogle Scholar
  22. 22.
    Maloberti, J., Sebag, M.: Theta-subsumption in a constraint satisfaction perspective. In: Proc. 11th Intl. Conf. on Inductive Logic Programming, pp. 164–178 (2001)Google Scholar
  23. 23.
    Boyan, J., Moore, A.: Learning evaluation functions to improve optimization by local search. J. Machine Learning Research 1, 77–112 (2000)CrossRefGoogle Scholar
  24. 24.
    Hornik, K., Stinchcombe, M., White, H.: Multilayer feedforward networks are universal approximators. Neural Networks 2, 359–366 (1989)CrossRefGoogle Scholar
  25. 25.
    Nix, D., Weigend, A.: Learning local error bars for nonlinear regression. In: Advances in Neural Information Processing Systems, MIT Press, Cambridge (1995)Google Scholar
  26. 26.
    Craven, M., Shavlik, J.: Extracting tree-structured representations of trained networks. In: Advances in Neural Information Processing Systems, MIT Press, Cambridge (1995)Google Scholar
  27. 27.
    King, R., Muggleton, S., Srinivasan, A., Sternberg, M.: Structure-activity relationships derived by machine learning. PNAS 93, 438–442 (1996)CrossRefGoogle Scholar
  28. 28.
    Srinivasan, A., King, R., Muggleton, S., Sternberg, M.: Carcinogenesis predictions using ILP. In: Proc. 7th Intl. Workshop on Inductive Logic Programming, pp. 273–287 (1997)Google Scholar
  29. 29.
    Witten, I., Frank, E.: Data Mining. Morgan Kaufmann Publishers, San Francisco (1999)Google Scholar
  30. 30.
    Goadrich, M., Oliphant, L., Shavlik, J.: Learning ensembles of first-order clauses for recall-precision curves: a case study in biomedical information extraction. In: Proc. 14th Intl. Conf. on Inductive Logic Programming (2004)Google Scholar
  31. 31.
    Botta, M., Giordana, A., Saitta, L., Sebag, M.: Relational learning as search in a critical region. J. Machine Learning Research 4, 431–463 (2003)CrossRefMathSciNetGoogle Scholar
  32. 32.
    Caruana, R., Baluja, S.: Using the future to ’sort out’ the present. In: Advances in Neural Information Processing Systems, MIT Press, Cambridge (1996)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2004

Authors and Affiliations

  • Frank DiMaio
    • 1
  • Jude Shavlik
    • 1
  1. 1.Computer Sciences DepartmentUniversity of Wisconsin MadisonMadisonUSA

Personalised recommendations