# Filtering Multi-Instance Problems to Reduce Dimensionality in Relational Learning

- 75 Downloads
- 9 Citations

## Abstract

Attribute-value based representations, standard in today's data mining systems, have a limited expressiveness. Inductive Logic Programming provides an interesting alternative, particularly for learning from structured examples whose parts, each with its own attributes, are related to each other by means of first-order predicates. Several subsets of first-order logic (FOL) with different expressive power have been proposed in Inductive Logic Programming (ILP). The challenge lies in the fact that the more expressive the subset of FOL the learner works with, the more critical the dimensionality of the learning task. The Datalog language is expressive enough to represent realistic learning problems when data is given directly in a relational database, making it a suitable tool for data mining. Consequently, it is important to elaborate techniques that will dynamically decrease the dimensionality of learning tasks expressed in Datalog, just as Feature Subset Selection (FSS) techniques do it in attribute-value learning. The idea of re-using these techniques in ILP runs immediately into a problem as ILP examples have variable size and do not share the same set of literals. We propose here the first paradigm that brings Feature Subset Selection to the level of ILP, in languages at least as expressive as Datalog. The main idea is to first perform a change of representation, which approximates the original relational problem by a multi-instance problem. The representation obtained as the result is suitable for FSS techniques which we adapted from attribute-value learning by taking into account some of the characteristics of the data due to the change of representation. We present the simple FSS proposed for the task, the requisite change of representation, and the entire method combining those two algorithms. The method acts as a filter, preprocessing the relational data, prior to the model building, which outputs relational examples with empirically relevant literals. We discuss experiments in which the method was successfully applied to two real-world domains.

## Preview

Unable to display preview. Download preview PDF.

## References

- Almuallim, H. and Dietterich, T.G. (1994). Learning Boolean Concepts in the Presence of Many Irrelevant Features.
*Artificial Intelligence*, 69(1/2), 279-305.Google Scholar - Alphonse, E. and Rouveirol, C. (2000). Lazy Propositionalization for Relational Learning. In W. Horn (Ed.),
*Proc. of the 14th European Conference on Artificial Intelligence (ECAI*'2000) (pp. 256-260). IOS Press.Google Scholar - Bisson, G. (1991). KBG: A Generator of Knowledge Bases. In Y. Kodratoff (Ed.),
*Proceedings of the European Working Session on Learning: Machine Learning (EWSL-91)*, Vol. 482 of LNAI (pp. 137-137). Porto, Portugal: Springer Verlag. ISBN 3-540-53816-X.Google Scholar - Blum, A. and Kalai, A. (1998). A Note on Learning from Multiple-Instance Examples.
*Machine Learning*, 30, 23-29.Google Scholar - Chevaleyre, Y. and Zucker, J. (2000). Noise-Tolerant Rule Induction for Multi-Instance Data. In
*ICML 2000, Workshop on Attribute-Value and Relational Learning*.Google Scholar - Cohen, W.W. (1993). Learnability of Restricted Logic Programs. In S. Muggleton (Ed.),
*Proceedings of the 3rd International Workshop on Inductive Logic Programming*(pp. 41-72). J. Stefan Institute.Google Scholar - Cohen, W.W. (1995). Learning to Classify English Text with ILP Methods. In L.D. Raedt (Ed.),
*Advances in Inductive Logic Programming*(pp. 124-143). Amsterdam, NL: IOS Press.Google Scholar - Conklin, D. (1995). Machine Discovery of Protein Motifs.
*Machine Learning*, 21, 125-150.Google Scholar - de Raedt, L. (1997). Logical Settings for Concept-Learning.
*Artificial Intelligence*, 95(1), 187-201.Google Scholar - Dietterich, T.G., Lathrop, R.H., and Lozano-Pérez, T. (1997). Solving the Multiple Instance Problem with Axis-Parallel Rectangles.
*Artificial Intelligence*, 89(1/2), 31-71.Google Scholar - Esposito, F., Malerba, D., and Semeraro, G. (1994). Multistrategy Learning for Document Recognition.
*Applied Artificial Intelligence*, 8, 33-84.Google Scholar - Feo, T.A. and Resende, M.G.C. (1995). Greedy Randomized Adaptive Search Procedures.
*Journal of Global Optimization*, 6, 109-133.Google Scholar - Fürnkranz, J. (1997). Dimensionality Reduction in ILP: A Call to Arms. In L. de Raedt and S. Muggleton (Eds.),
*Proceedings of the IJCAI-97 Workshop on Frontiers of Inductive Logic Programming*(pp. 81-86).Google Scholar - Gottlob, G. (1987). Subsumption and Implication.
*Information Processing Letters*, 24(2), 109-111.Google Scholar - Jagota, A. (1993). Constraint Satisfaction and Maximum Clique. In
*Working Notes, AAAI Spring Symposium on AI and NP-Hard Problems*(pp. 92-97). Stanford University.Google Scholar - Kietz, J.-U., Reimer, U., and Staudt, M. (1997). Mining Insurance Data at Swiss Life.
*The VLDB Journal*, 562-566.Google Scholar - Kira, K. and Rendell, L.A. (1992). A Practical Approach to Feature Selection. In
*Proc. of the Ninth Int. Conference on Machine Learning*(pp. 249-256). MK.Google Scholar - Kohavi, R. and John, G.H. (1997). Wrappers for Feature Subset Selection.
*Artificial Intelligence*, 97(1/2), 273-323.Google Scholar - Lavra?, N. and Džeroski, S. (1994).
*Inductive Logic Programming: Techniques and Applications*. Ellis Horwood.Google Scholar - Lee, Y., Buchanan, B.G., and Aronis, J.M. (1998). Knowledge-Based Learning in Exploratory Science: Learning Rules to Predict Rodent Carcinogenicity.
*Machine Learning*, 30, 217-240.Google Scholar - Liu, H. and Motoda, H. (1998).
*Feature Extraction, Construction and Selection: A Data Mining Perspective*. Kluwer Academic Publisher.Google Scholar - Nédellec, C., Rouveirol, C., Ade, H., Bergadano, F., and Tausend, B. (1996). Declarative Bias in Inductive Logic Programming. In L. de Raedt (Ed.),
*Advances in Inductive Logic Programming*(pp. 82-103). IOS Press.Google Scholar - Plotkin, G. (1970). A Note on Inductive Generalization. In
*Machine Intelligence*, Vol. 5. Edinburgh University Press.Google Scholar - Pompe, U. and Kononenko, I. (1995). Linear Space Induction in First Order Logic with RELIEFF. In R. Kruse, R. Viertl, and G. Della Ricci (Eds.),
*Mathematical and Statistical Methods in Artificial Intelligence, CISM Course and Lecture Notes 363*(pp. 185-220). Springer-Verlag.Google Scholar - Quinlan, J.R. (1990). Learning Logical Definitions from Relations.
*Machine Learning*, 5(3), 239-266.Google Scholar - Rouveirol, C. (2000). Expressiveness/Efficiency: The Dilemma of the Inductive Logic Programming. LRI, Université Paris. HDR, in French.Google Scholar
- Sebag, M. (1998). Resource Bounded Induction and Deduction in FOL. In
*Proceedings of 4th International Workshop on Multistrategy Learning*(pp. 95-105).Google Scholar - Sebag, M. and Rouveirol, C. (1996). Constraint Inductive Logic Programming. In L. De Raedt (Ed.),
*Advances in Inductive Logic Programming*(pp. 277-294). IOS Press.Google Scholar - Sebag, M. and Rouveirol, C. (1997). Tractable Induction and Classification in First Order Logic via Stochastic Matching. In
*15th Int. Join Conf. on Artificial Intelligence (IJCAI*'97) (pp. 888-893). Morgan Kaufmann.Google Scholar - Silverstein, G. and Pazzani, M.J. (1991). Relational Cliches: Constraining Constructive Induction During Relational Learning. In L. Birnbaum and G. Collins (Eds.),
*Proceedings of the 8th International Workshop on Machine Learning*(pp. 203-207). Morgan Kaufmann.Google Scholar - Srinivasan, A., Muggleton, S., King, R., and Sternberg, M. (1994). Mutagenesis: ILP Experiments in a Non-Determinate Biological Domain. In S. Wrobel (Ed.),
*Proceedings of the 4th International Workshop on Inductive Logic Programming*, Vol. 237 of*GMD-Studien*(pp. 217-232). Gesellschaft für Mathematik und Datenverarbeitung MBH.Google Scholar - Zucker, J.-D. and Ganascia, J.-G. (1996). Changes of Representation for Efficient Learning in Structural Domains. In
*Proc. of 13th International Conference on Machine Learning*. Morgan Kaufmann.Google Scholar - Zucker, J.-D. and Ganascia, J.-G. (1998). Learning Structural Indeterminate Clauses. In D. Page (Ed.),
*Proc. of the 8th International Workshop on Inductive Logic Programming*(pp. 235-244). Springer Verlag.Google Scholar