Abstract
This paper deals with learning first-order logic rules from data lacking an explicit classification predicate. Consequently, the learned rules are not restricted to predicate definitions as in supervised inductive logic programming. First-order logic offers the ability to deal with structured, multi-relational knowledge. Possible applications include first-order knowledge discovery, induction of integrity constraints in databases, multiple predicate learning, and learning mixed theories of predicate definitions and integrity constraints. One of the contributions of our work is a heuristic measure of confirmation, trading off novelty and satisfaction of the rule. The approach has been implemented in the Tertius system. The system performs an optimal best-first search, finding the k most confirmed hypotheses, and includes a non-redundant refinement operator to avoid duplicates in the search. Tertius can be adapted to many different domains by tuning its parameters, and it can deal either with individual-based representations by upgrading propositional representations to first-order, or with general logical rules. We describe a number of experiments demonstrating the feasibility and flexibility of our approach.
Article PDF
Similar content being viewed by others
References
Badea, L. & Stanciu, M. (1999). Refinement operators can be (weakly) perfect. In S. Džeroski and P. Flach (Eds.), Proceedings of the 9th InternationalWorkshop on Inductive Logic Programming, volume 1634 of Lecture Notes in Artificial Intelligence, (pp. 21–32). Springer-Verlag.
Dahl, T. S. (1999). Background knowledge in the Tertius first-order knowledge discovery tool. Technical Report CSTR-99-006, Department of Computer Science, University of Bristol, March 1999.
De Raedt, L. & Lavrač, N. (1996). Multiple predicate learning in two inductive logic programming settings. Journal on Pure and Applied Logic, 4(2), 227–254.
De Raedt, L. (1997). Logical settings for concept learning. Artificial Intelligence 95(1), 187–201.
De Raedt, L. & Dehaspe, L. (1997). Clausal discovery. Machine Learning, 26(2/3), 99–146.
Dehaspe L. & De Raedt, L. (1997). Mining association rules in multiple relations. In S. Džeroski & N. Lavrač (Eds.), In Proceedings of the 7th International Workshop on Inductive Logic Programming, volume 1297 of Lecture Notes in Artificial Intelligence (pp. 125–132). Springer-Verlag.
Dehaspe L. & Toivonen H. (1999). Discovery of frequent datalog patterns. Data Mining and Knowledge Discovery, 3(1), 7–36.
Dimopoulos, Y., Dzeroski, S., & Kakas, A. C. (1997). Integrating explanatory and descriptive learning in ILP. In M. E. Pollack (Ed.), Proceedings of the 15th International Joint Conference on Artificial Intelligence (pp. 900–906), Morgan Kaufmann.
Flach, P. (1993). Predicate invention in inductive data engineering. In Brazdil, P. (Ed.), Proceedings of the 6th European Conference on Machine Learning, volume 667 of Lecture Notes in Artificial Intelligence (pp. 83–94). Springer-Verlag.
Flach, P., Giraud-Carrier, C., & Lloyd, J. (1998). Strongly typed inductive concept learning. In Page, D. (Ed.), Proceedings of the 8th International Conference on Inductive Logic Programming, volume 1446 of Lecture Notes in Artificial Intelligence (pp. 185–194). Springer-Verlag.
Flach, P. & Lachiche, N. (1999a). 1BC: A first-order Bayesian classifier. In Džeroski, S. & Flach P. (Eds.), Proceedings of the 9th International Workshop on Inductive Logic Programming (pp. 92–103). volume 1634 of Lecture Notes in Artificial Intelligence, Springer-Verlag.
Flach, P. & Savnik, I. (1999). Database dependency discovery: a machine learning approach. AI Communications, 12(3), 139–160.
Flach, P. A. & Lachiche, N. (1999b). The Tertius system. http://www.cs.bris.ac.uk/Research/MachineLearning/Tertius/.
Goodman, L. A. & Kruskal, W. H. (1979). Measures of association for cross classifications. Springer-Verlag.
Klösgen, W. (1996). Explora: A multipattern and multistrategy discovery assistant. In Fayyad, U., Piatetsky-Shapiro, G., Smyth, P., & Uthurusamy, R. (Eds.), Advances in Knowledge Discovery and Data Mining (pp. 249–271). AAAI Press.
Lavrač, N., Flach, P., & Zupan, B. (1999). Rule evaluation measures: A unifying view. In Džeroski, S. & Flach, P. (Eds.), Proceedings of the 9th International Workshop on Inductive Logic Programming, volume 1634 of Lecture Notes in Artificial Intelligence (pp. 174–185). Springer-Verlag.
Muggleton, S. (1995). Inverse entailment and Progol. New Generation Computing, Special issue on Inductive Logic Programming, 13(3/4), 245–286.
Oates, T. & Cohen, P. (1996). Searching for structure in multiple streams of data. In Saitta, L. (Ed.), Proceedings of the 13th International Conference on Machine Learning (pp. 346–354). Morgan Kaufmann.
Piatetsky-Shapiro, G. (1991). Discovery, analysis and presentation of strong rules. In Piatetsky-Shapiro, G. and Frawley, W. (Eds.), Knowledge Discovery in Databases (pp. 229–249). AAAI Press.
Plotkin, G. (1970). A note on inductive generalization. In Machine Intelligence (vol. 5, pp. 153–163). Edinburgh University Press.
Plotkin, G. (1971). A further note on inductive generalization. In Machine Intelligence (vol. 6, pp. 101–124). Edinburgh University Press.
Rymon, R. (1992). Search through systematic set enumeration. In Proceedings of the 3d International Conference on Knowledge Representation and Reasoning (pp. 539–550). Morgan Kaufmann.
Srinivasan, A., King, R. D., & Muggleton, S. H. (1996). The role of background knowledge: Using a problem from chemistry to examine the performance of an ILP program. Unpublished manuscript, available from the first author.
Srinivasan, A., Muggleton, S., King, R., & Sternberg, M. (1994). Mutagenesis: ILP experiments in a nondeterminate biological domain. In Wrobel, S. (Ed.), Proceedings of the 4th InternationalWorkshop on Inductive Logic Programming (vol. 237 of GMD-Studien, pp. 217–232). Gesellschaft für Mathematik und Datenverarbeitung MBH.
Vere, S. A. (1978). Inductive learning of relational productions. In Waterman, D. & Hayes-Roth, F. (Eds.), Pattern-Directed Inference Systems (pp. 281–295). Academic Press.
Wickens, T. (1989). Multiway contingency tables analysis for the social sciences. Lawrence Erlbaum.
Wrobel, S. (1997). An algorithm for multi-relational discovery of subgroups. In Komorowski, J. & Zytkow, J. (Eds.), Proceedings of the 1st European Symposium on Principles of Data Mining and Knowledge Discovery. Springer-Verlag.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Flach, P.A., Lachiche, N. Confirmation-Guided Discovery of First-Order Rules with Tertius. Machine Learning 42, 61–95 (2001). https://doi.org/10.1023/A:1007656703224
Issue Date:
DOI: https://doi.org/10.1023/A:1007656703224