# Derivation Reduction of Metarules in Meta-interpretive Learning

## Abstract

Meta-interpretive learning (MIL) is a form of inductive logic programming. MIL uses second-order Horn clauses, called metarules, as a form of declarative bias. Metarules define the structures of learnable programs and thus the hypothesis space. Deciding which metarules to use is a trade-off between efficiency and expressivity. The hypothesis space increases given more metarules, so we wish to use fewer metarules, but if we use too few metarules then we lose expressivity. A recent paper used Progol’s entailment reduction algorithm to identify irreducible, or minimal, sets of metarules. In some cases, as few as two metarules were shown to be sufficient to entail all hypotheses in an infinite language. Moreover, it was shown that compared to non-minimal sets, learning with minimal sets of metarules improves predictive accuracies and lowers learning times. In this paper, we show that entailment reduction can be too strong and can remove metarules necessary to make a hypothesis more specific. We describe a new reduction technique based on derivations. Specifically, we introduce the *derivation reduction* problem, the problem of finding a finite subset of a Horn theory from which the whole theory can be derived using SLD-resolution. We describe a derivation reduction algorithm which we use to reduce sets of metarules. We also theoretically study whether certain sets of metarules can be derivationally reduced to minimal finite subsets. Our experiments compare learning with entailment and derivation reduced sets of metarules. In general, using derivation reduced sets of metarules outperforms using entailment reduced sets of metarules, both in terms of predictive accuracies and learning times.

## Notes

### Acknowledgements

The authors thank Stephen Muggleton and Katsumi Inoue for helpful discussions on this topic.

## References

- 1.Albarghouthi, A., Koutris, P., Naik, M., Smith, C.: Constraint-based synthesis of datalog programs. In: Beck, J.C. (ed.) CP 2017. LNCS, vol. 10416, pp. 689–706. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-66158-2_44CrossRefGoogle Scholar
- 2.Bradley, A.R., Manna, Z.: The Calculus of Computation: Decision Procedures with Applications to Verification. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-74113-8CrossRefzbMATHGoogle Scholar
- 3.Cropper, A., Muggleton, S.H.: Logical minimisation of meta-rules within meta-interpretive learning. In: Davis, J., Ramon, J. (eds.) ILP 2014. LNCS (LNAI), vol. 9046, pp. 62–75. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-23708-4_5CrossRefGoogle Scholar
- 4.Cropper, A., Muggleton, S.H.: Learning higher-order logic programs through abstraction and invention. In: Kambhampati, S. (ed.) Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, IJCAI 2016, 9–15 July 2016, pp. 1418–1424. IJCAI/AAAI Press, New York (2016)Google Scholar
- 5.Cropper, A., Muggleton, S.H.: Metagol system (2016). https://github.com/metagol/metagol
- 6.Cropper, A., Muggleton, S.H.: Learning efficient logic programs. Mach. Learn., 1–21 (2018)Google Scholar
- 7.Emde, W., Habel, C., Rollinger, C.-R.: The discovery of the equator or concept driven learning. In: Bundy, A. (ed.) Proceedings of the 8th International Joint Conference on Artificial Intelligence, August 1983, pp. 455–458. William Kaufmann, Karlsruhe (1983)Google Scholar
- 8.Evans, R., Grefenstette, E.: Learning explanatory rules from noisy data. J. Artif. Intell. Res.
**61**, 1–64 (2018)MathSciNetCrossRefGoogle Scholar - 9.Flener, P.: Inductive logic program synthesis with DIALOGS. In: Muggleton, S. (ed.) ILP 1996. LNCS, vol. 1314, pp. 175–198. Springer, Heidelberg (1997). https://doi.org/10.1007/3-540-63494-0_55CrossRefGoogle Scholar
- 10.Fonseca, N., Costa, V.S., Silva, F., Camacho, R.: On avoiding redundancy in inductive logic programming. In: Camacho, R., King, R., Srinivasan, A. (eds.) ILP 2004. LNCS (LNAI), vol. 3194, pp. 132–146. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-30109-7_13CrossRefzbMATHGoogle Scholar
- 11.Garey, M.R., Johnson, D.S.: Computers and Intractability: A Guide to the Theory of NP-Completeness. W. H. Freeman, New York (1979)zbMATHGoogle Scholar
- 12.Heule, M., Järvisalo, M., Lonsing, F., Seidl, M., Biere, A.: Clause elimination for SAT and QSAT. J. Artif. Intell. Res.
**53**, 127–168 (2015)MathSciNetCrossRefGoogle Scholar - 13.Kaminski, T., Eiter, T., Inoue, K.: Exploiting answer set programming with external sources for meta-interpretive learning. In: 34th International Conference on Logic Programming (2018)Google Scholar
- 14.Kietz, J.-U., Wrobel, S.: Controlling the complexity of learning in logic through syntactic and task-oriented models. In: Inductive Logic Programming. Citeseer (1992)Google Scholar
- 15.Kowalski, R.A.: Predicate logic as programming language. In: IFIP Congress, pp. 569–574 (1974)Google Scholar
- 16.Langlois, M., Mubayi, D., Sloan, R.H., Turán, G.: Combinatorial problems for horn clauses. In: Lipshteyn, M., Levit, V.E., McConnell, R.M. (eds.) Graph Theory, Computational Intelligence and Thought. LNCS, vol. 5420, pp. 54–65. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-02029-2_6CrossRefGoogle Scholar
- 17.Larson, J., Michalski, R.S.: Inductive inference of VL decision rules. SIGART Newslett.
**63**, 38–44 (1977)Google Scholar - 18.Liberatore, P.: Redundancy in logic I: CNF propositional formulae. Artif. Intell.
**163**(2), 203–232 (2005)MathSciNetCrossRefGoogle Scholar - 19.Liberatore, P.: Redundancy in logic II: 2CNF and horn propositional formulae. Artif. Intell.
**172**(2–3), 265–299 (2008)MathSciNetCrossRefGoogle Scholar - 20.Lin, D., Dechter, E., Ellis, K., Tenenbaum, J.B., Muggleton, S.: Bias reformulation for one-shot function induction. In: ECAI 2014–21st European Conference on Artificial Intelligence, 18–22 August 2014, Prague, Czech Republic - Including Prestigious Applications of Intelligent Systems (PAIS 2014), pp. 525–530 (2014)Google Scholar
- 21.Lloyd, J.W.: Foundations of Logic Programming. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-83189-8CrossRefGoogle Scholar
- 22.Marcinkowski, J., Pacholski, L.: Undecidability of the horn-clause implication problem. In: 33rd Annual Symposium on Foundations of Computer Science, Pittsburgh, Pennsylvania, USA, 24–27 October 1992, pp. 354–362 (1992)Google Scholar
- 23.Muggleton, S.: Inverse entailment and progol. New Gener. Comput.
**13**(3&4), 245–286 (1995)CrossRefGoogle Scholar - 24.Muggleton, S., Feng, C.: Efficient induction of logic programs. In: Algorithmic Learning Theory, First International Workshop, ALT 1990, Tokyo, Japan, 8–10 October 1990, Proceedings, pp. 368–381 (1990)Google Scholar
- 25.Muggleton, S.H., Lin, D., Tamaddoni-Nezhad, A.: Meta-interpretive learning of higher-order dyadic datalog: predicate invention revisited. Mach. Learn.
**100**(1), 49–73 (2015)MathSciNetCrossRefGoogle Scholar - 26.Nienhuys-Cheng, S.-H., de Wolf, R.: Foundations of Inductive Logic Programming. LNCS, vol. 1228. Springer, Heidelberg (1997). https://doi.org/10.1007/3-540-62927-0CrossRefzbMATHGoogle Scholar
- 27.Plotkin, G.D.: Automatic methods of inductive inference. Ph.D. thesis, Edinburgh University, August 1971Google Scholar
- 28.Raedt, L.: Declarative modeling for machine learning and data mining. In: Bshouty, N.H., Stoltz, G., Vayatis, N., Zeugmann, T. (eds.) ALT 2012. LNCS (LNAI), vol. 7568, pp. 12–12. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-34106-9_2CrossRefGoogle Scholar
- 29.De Raedt, L., Bruynooghe, M.: Interactive concept-learning and constructive induction by analogy. Mach. Learn.
**8**, 107–150 (1992)zbMATHGoogle Scholar - 30.Schmidt-Schauß, M.: Implication of clauses is undecidable. Theor. Comput. Sci.
**59**, 287–296 (1988)MathSciNetCrossRefGoogle Scholar - 31.Shapiro, E.Y.: Algorithmic Program Debugging. MIT Press, Cambridge (1983)zbMATHGoogle Scholar
- 32.Wang, W.Y., Mazaitis, K., Cohen, W.W.: Structure learning via parameter learning. In: Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management, pp. 1199–1208. ACM (2014)Google Scholar