Advertisement

Derivation Reduction of Metarules in Meta-interpretive Learning

  • Andrew CropperEmail author
  • Sophie Tourret
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11105)

Abstract

Meta-interpretive learning (MIL) is a form of inductive logic programming. MIL uses second-order Horn clauses, called metarules, as a form of declarative bias. Metarules define the structures of learnable programs and thus the hypothesis space. Deciding which metarules to use is a trade-off between efficiency and expressivity. The hypothesis space increases given more metarules, so we wish to use fewer metarules, but if we use too few metarules then we lose expressivity. A recent paper used Progol’s entailment reduction algorithm to identify irreducible, or minimal, sets of metarules. In some cases, as few as two metarules were shown to be sufficient to entail all hypotheses in an infinite language. Moreover, it was shown that compared to non-minimal sets, learning with minimal sets of metarules improves predictive accuracies and lowers learning times. In this paper, we show that entailment reduction can be too strong and can remove metarules necessary to make a hypothesis more specific. We describe a new reduction technique based on derivations. Specifically, we introduce the derivation reduction problem, the problem of finding a finite subset of a Horn theory from which the whole theory can be derived using SLD-resolution. We describe a derivation reduction algorithm which we use to reduce sets of metarules. We also theoretically study whether certain sets of metarules can be derivationally reduced to minimal finite subsets. Our experiments compare learning with entailment and derivation reduced sets of metarules. In general, using derivation reduced sets of metarules outperforms using entailment reduced sets of metarules, both in terms of predictive accuracies and learning times.

Notes

Acknowledgements

The authors thank Stephen Muggleton and Katsumi Inoue for helpful discussions on this topic.

References

  1. 1.
    Albarghouthi, A., Koutris, P., Naik, M., Smith, C.: Constraint-based synthesis of datalog programs. In: Beck, J.C. (ed.) CP 2017. LNCS, vol. 10416, pp. 689–706. Springer, Cham (2017).  https://doi.org/10.1007/978-3-319-66158-2_44CrossRefGoogle Scholar
  2. 2.
    Bradley, A.R., Manna, Z.: The Calculus of Computation: Decision Procedures with Applications to Verification. Springer, Heidelberg (2007).  https://doi.org/10.1007/978-3-540-74113-8CrossRefzbMATHGoogle Scholar
  3. 3.
    Cropper, A., Muggleton, S.H.: Logical minimisation of meta-rules within meta-interpretive learning. In: Davis, J., Ramon, J. (eds.) ILP 2014. LNCS (LNAI), vol. 9046, pp. 62–75. Springer, Cham (2015).  https://doi.org/10.1007/978-3-319-23708-4_5CrossRefGoogle Scholar
  4. 4.
    Cropper, A., Muggleton, S.H.: Learning higher-order logic programs through abstraction and invention. In: Kambhampati, S. (ed.) Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, IJCAI 2016, 9–15 July 2016, pp. 1418–1424. IJCAI/AAAI Press, New York (2016)Google Scholar
  5. 5.
    Cropper, A., Muggleton, S.H.: Metagol system (2016). https://github.com/metagol/metagol
  6. 6.
    Cropper, A., Muggleton, S.H.: Learning efficient logic programs. Mach. Learn., 1–21 (2018)Google Scholar
  7. 7.
    Emde, W., Habel, C., Rollinger, C.-R.: The discovery of the equator or concept driven learning. In: Bundy, A. (ed.) Proceedings of the 8th International Joint Conference on Artificial Intelligence, August 1983, pp. 455–458. William Kaufmann, Karlsruhe (1983)Google Scholar
  8. 8.
    Evans, R., Grefenstette, E.: Learning explanatory rules from noisy data. J. Artif. Intell. Res. 61, 1–64 (2018)MathSciNetCrossRefGoogle Scholar
  9. 9.
    Flener, P.: Inductive logic program synthesis with DIALOGS. In: Muggleton, S. (ed.) ILP 1996. LNCS, vol. 1314, pp. 175–198. Springer, Heidelberg (1997).  https://doi.org/10.1007/3-540-63494-0_55CrossRefGoogle Scholar
  10. 10.
    Fonseca, N., Costa, V.S., Silva, F., Camacho, R.: On avoiding redundancy in inductive logic programming. In: Camacho, R., King, R., Srinivasan, A. (eds.) ILP 2004. LNCS (LNAI), vol. 3194, pp. 132–146. Springer, Heidelberg (2004).  https://doi.org/10.1007/978-3-540-30109-7_13CrossRefzbMATHGoogle Scholar
  11. 11.
    Garey, M.R., Johnson, D.S.: Computers and Intractability: A Guide to the Theory of NP-Completeness. W. H. Freeman, New York (1979)zbMATHGoogle Scholar
  12. 12.
    Heule, M., Järvisalo, M., Lonsing, F., Seidl, M., Biere, A.: Clause elimination for SAT and QSAT. J. Artif. Intell. Res. 53, 127–168 (2015)MathSciNetCrossRefGoogle Scholar
  13. 13.
    Kaminski, T., Eiter, T., Inoue, K.: Exploiting answer set programming with external sources for meta-interpretive learning. In: 34th International Conference on Logic Programming (2018)Google Scholar
  14. 14.
    Kietz, J.-U., Wrobel, S.: Controlling the complexity of learning in logic through syntactic and task-oriented models. In: Inductive Logic Programming. Citeseer (1992)Google Scholar
  15. 15.
    Kowalski, R.A.: Predicate logic as programming language. In: IFIP Congress, pp. 569–574 (1974)Google Scholar
  16. 16.
    Langlois, M., Mubayi, D., Sloan, R.H., Turán, G.: Combinatorial problems for horn clauses. In: Lipshteyn, M., Levit, V.E., McConnell, R.M. (eds.) Graph Theory, Computational Intelligence and Thought. LNCS, vol. 5420, pp. 54–65. Springer, Heidelberg (2009).  https://doi.org/10.1007/978-3-642-02029-2_6CrossRefGoogle Scholar
  17. 17.
    Larson, J., Michalski, R.S.: Inductive inference of VL decision rules. SIGART Newslett. 63, 38–44 (1977)Google Scholar
  18. 18.
    Liberatore, P.: Redundancy in logic I: CNF propositional formulae. Artif. Intell. 163(2), 203–232 (2005)MathSciNetCrossRefGoogle Scholar
  19. 19.
    Liberatore, P.: Redundancy in logic II: 2CNF and horn propositional formulae. Artif. Intell. 172(2–3), 265–299 (2008)MathSciNetCrossRefGoogle Scholar
  20. 20.
    Lin, D., Dechter, E., Ellis, K., Tenenbaum, J.B., Muggleton, S.: Bias reformulation for one-shot function induction. In: ECAI 2014–21st European Conference on Artificial Intelligence, 18–22 August 2014, Prague, Czech Republic - Including Prestigious Applications of Intelligent Systems (PAIS 2014), pp. 525–530 (2014)Google Scholar
  21. 21.
    Lloyd, J.W.: Foundations of Logic Programming. Springer, Heidelberg (2012).  https://doi.org/10.1007/978-3-642-83189-8CrossRefGoogle Scholar
  22. 22.
    Marcinkowski, J., Pacholski, L.: Undecidability of the horn-clause implication problem. In: 33rd Annual Symposium on Foundations of Computer Science, Pittsburgh, Pennsylvania, USA, 24–27 October 1992, pp. 354–362 (1992)Google Scholar
  23. 23.
    Muggleton, S.: Inverse entailment and progol. New Gener. Comput. 13(3&4), 245–286 (1995)CrossRefGoogle Scholar
  24. 24.
    Muggleton, S., Feng, C.: Efficient induction of logic programs. In: Algorithmic Learning Theory, First International Workshop, ALT 1990, Tokyo, Japan, 8–10 October 1990, Proceedings, pp. 368–381 (1990)Google Scholar
  25. 25.
    Muggleton, S.H., Lin, D., Tamaddoni-Nezhad, A.: Meta-interpretive learning of higher-order dyadic datalog: predicate invention revisited. Mach. Learn. 100(1), 49–73 (2015)MathSciNetCrossRefGoogle Scholar
  26. 26.
    Nienhuys-Cheng, S.-H., de Wolf, R.: Foundations of Inductive Logic Programming. LNCS, vol. 1228. Springer, Heidelberg (1997).  https://doi.org/10.1007/3-540-62927-0CrossRefzbMATHGoogle Scholar
  27. 27.
    Plotkin, G.D.: Automatic methods of inductive inference. Ph.D. thesis, Edinburgh University, August 1971Google Scholar
  28. 28.
    Raedt, L.: Declarative modeling for machine learning and data mining. In: Bshouty, N.H., Stoltz, G., Vayatis, N., Zeugmann, T. (eds.) ALT 2012. LNCS (LNAI), vol. 7568, pp. 12–12. Springer, Heidelberg (2012).  https://doi.org/10.1007/978-3-642-34106-9_2CrossRefGoogle Scholar
  29. 29.
    De Raedt, L., Bruynooghe, M.: Interactive concept-learning and constructive induction by analogy. Mach. Learn. 8, 107–150 (1992)zbMATHGoogle Scholar
  30. 30.
    Schmidt-Schauß, M.: Implication of clauses is undecidable. Theor. Comput. Sci. 59, 287–296 (1988)MathSciNetCrossRefGoogle Scholar
  31. 31.
    Shapiro, E.Y.: Algorithmic Program Debugging. MIT Press, Cambridge (1983)zbMATHGoogle Scholar
  32. 32.
    Wang, W.Y., Mazaitis, K., Cohen, W.W.: Structure learning via parameter learning. In: Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management, pp. 1199–1208. ACM (2014)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.University of OxfordOxfordUK
  2. 2.Max Planck Institute for InformaticsSaarbrückenGermany

Personalised recommendations