Journal of Intelligent Information Systems

, Volume 22, Issue 1, pp 23–40 | Cite as

Filtering Multi-Instance Problems to Reduce Dimensionality in Relational Learning

  • Érick Alphonse
  • Stan Matwin


Attribute-value based representations, standard in today's data mining systems, have a limited expressiveness. Inductive Logic Programming provides an interesting alternative, particularly for learning from structured examples whose parts, each with its own attributes, are related to each other by means of first-order predicates. Several subsets of first-order logic (FOL) with different expressive power have been proposed in Inductive Logic Programming (ILP). The challenge lies in the fact that the more expressive the subset of FOL the learner works with, the more critical the dimensionality of the learning task. The Datalog language is expressive enough to represent realistic learning problems when data is given directly in a relational database, making it a suitable tool for data mining. Consequently, it is important to elaborate techniques that will dynamically decrease the dimensionality of learning tasks expressed in Datalog, just as Feature Subset Selection (FSS) techniques do it in attribute-value learning. The idea of re-using these techniques in ILP runs immediately into a problem as ILP examples have variable size and do not share the same set of literals. We propose here the first paradigm that brings Feature Subset Selection to the level of ILP, in languages at least as expressive as Datalog. The main idea is to first perform a change of representation, which approximates the original relational problem by a multi-instance problem. The representation obtained as the result is suitable for FSS techniques which we adapted from attribute-value learning by taking into account some of the characteristics of the data due to the change of representation. We present the simple FSS proposed for the task, the requisite change of representation, and the entire method combining those two algorithms. The method acts as a filter, preprocessing the relational data, prior to the model building, which outputs relational examples with empirically relevant literals. We discuss experiments in which the method was successfully applied to two real-world domains.

relational learning feature subset selection propositionalization 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Almuallim, H. and Dietterich, T.G. (1994). Learning Boolean Concepts in the Presence of Many Irrelevant Features. Artificial Intelligence, 69(1/2), 279-305.Google Scholar
  2. Alphonse, E. and Rouveirol, C. (2000). Lazy Propositionalization for Relational Learning. In W. Horn (Ed.), Proc. of the 14th European Conference on Artificial Intelligence (ECAI'2000) (pp. 256-260). IOS Press.Google Scholar
  3. Bisson, G. (1991). KBG: A Generator of Knowledge Bases. In Y. Kodratoff (Ed.), Proceedings of the European Working Session on Learning: Machine Learning (EWSL-91), Vol. 482 of LNAI (pp. 137-137). Porto, Portugal: Springer Verlag. ISBN 3-540-53816-X.Google Scholar
  4. Blum, A. and Kalai, A. (1998). A Note on Learning from Multiple-Instance Examples. Machine Learning, 30, 23-29.Google Scholar
  5. Chevaleyre, Y. and Zucker, J. (2000). Noise-Tolerant Rule Induction for Multi-Instance Data. In ICML 2000, Workshop on Attribute-Value and Relational Learning.Google Scholar
  6. Cohen, W.W. (1993). Learnability of Restricted Logic Programs. In S. Muggleton (Ed.), Proceedings of the 3rd International Workshop on Inductive Logic Programming (pp. 41-72). J. Stefan Institute.Google Scholar
  7. Cohen, W.W. (1995). Learning to Classify English Text with ILP Methods. In L.D. Raedt (Ed.), Advances in Inductive Logic Programming (pp. 124-143). Amsterdam, NL: IOS Press.Google Scholar
  8. Conklin, D. (1995). Machine Discovery of Protein Motifs. Machine Learning, 21, 125-150.Google Scholar
  9. de Raedt, L. (1997). Logical Settings for Concept-Learning. Artificial Intelligence, 95(1), 187-201.Google Scholar
  10. Dietterich, T.G., Lathrop, R.H., and Lozano-Pérez, T. (1997). Solving the Multiple Instance Problem with Axis-Parallel Rectangles. Artificial Intelligence, 89(1/2), 31-71.Google Scholar
  11. Esposito, F., Malerba, D., and Semeraro, G. (1994). Multistrategy Learning for Document Recognition. Applied Artificial Intelligence, 8, 33-84.Google Scholar
  12. Feo, T.A. and Resende, M.G.C. (1995). Greedy Randomized Adaptive Search Procedures. Journal of Global Optimization, 6, 109-133.Google Scholar
  13. Fürnkranz, J. (1997). Dimensionality Reduction in ILP: A Call to Arms. In L. de Raedt and S. Muggleton (Eds.), Proceedings of the IJCAI-97 Workshop on Frontiers of Inductive Logic Programming (pp. 81-86).Google Scholar
  14. Gottlob, G. (1987). Subsumption and Implication. Information Processing Letters, 24(2), 109-111.Google Scholar
  15. Jagota, A. (1993). Constraint Satisfaction and Maximum Clique. In Working Notes, AAAI Spring Symposium on AI and NP-Hard Problems (pp. 92-97). Stanford University.Google Scholar
  16. Kietz, J.-U., Reimer, U., and Staudt, M. (1997). Mining Insurance Data at Swiss Life. The VLDB Journal, 562-566.Google Scholar
  17. Kira, K. and Rendell, L.A. (1992). A Practical Approach to Feature Selection. In Proc. of the Ninth Int. Conference on Machine Learning (pp. 249-256). MK.Google Scholar
  18. Kohavi, R. and John, G.H. (1997). Wrappers for Feature Subset Selection. Artificial Intelligence, 97(1/2), 273-323.Google Scholar
  19. Lavra?, N. and Džeroski, S. (1994). Inductive Logic Programming: Techniques and Applications. Ellis Horwood.Google Scholar
  20. Lee, Y., Buchanan, B.G., and Aronis, J.M. (1998). Knowledge-Based Learning in Exploratory Science: Learning Rules to Predict Rodent Carcinogenicity. Machine Learning, 30, 217-240.Google Scholar
  21. Liu, H. and Motoda, H. (1998). Feature Extraction, Construction and Selection: A Data Mining Perspective. Kluwer Academic Publisher.Google Scholar
  22. Nédellec, C., Rouveirol, C., Ade, H., Bergadano, F., and Tausend, B. (1996). Declarative Bias in Inductive Logic Programming. In L. de Raedt (Ed.), Advances in Inductive Logic Programming (pp. 82-103). IOS Press.Google Scholar
  23. Plotkin, G. (1970). A Note on Inductive Generalization. In Machine Intelligence, Vol. 5. Edinburgh University Press.Google Scholar
  24. Pompe, U. and Kononenko, I. (1995). Linear Space Induction in First Order Logic with RELIEFF. In R. Kruse, R. Viertl, and G. Della Ricci (Eds.), Mathematical and Statistical Methods in Artificial Intelligence, CISM Course and Lecture Notes 363 (pp. 185-220). Springer-Verlag.Google Scholar
  25. Quinlan, J.R. (1990). Learning Logical Definitions from Relations. Machine Learning, 5(3), 239-266.Google Scholar
  26. Rouveirol, C. (2000). Expressiveness/Efficiency: The Dilemma of the Inductive Logic Programming. LRI, Université Paris. HDR, in French.Google Scholar
  27. Sebag, M. (1998). Resource Bounded Induction and Deduction in FOL. In Proceedings of 4th International Workshop on Multistrategy Learning (pp. 95-105).Google Scholar
  28. Sebag, M. and Rouveirol, C. (1996). Constraint Inductive Logic Programming. In L. De Raedt (Ed.), Advances in Inductive Logic Programming (pp. 277-294). IOS Press.Google Scholar
  29. Sebag, M. and Rouveirol, C. (1997). Tractable Induction and Classification in First Order Logic via Stochastic Matching. In 15th Int. Join Conf. on Artificial Intelligence (IJCAI'97) (pp. 888-893). Morgan Kaufmann.Google Scholar
  30. Silverstein, G. and Pazzani, M.J. (1991). Relational Cliches: Constraining Constructive Induction During Relational Learning. In L. Birnbaum and G. Collins (Eds.), Proceedings of the 8th International Workshop on Machine Learning (pp. 203-207). Morgan Kaufmann.Google Scholar
  31. Srinivasan, A., Muggleton, S., King, R., and Sternberg, M. (1994). Mutagenesis: ILP Experiments in a Non-Determinate Biological Domain. In S. Wrobel (Ed.), Proceedings of the 4th International Workshop on Inductive Logic Programming, Vol. 237 of GMD-Studien (pp. 217-232). Gesellschaft für Mathematik und Datenverarbeitung MBH.Google Scholar
  32. Zucker, J.-D. and Ganascia, J.-G. (1996). Changes of Representation for Efficient Learning in Structural Domains. In Proc. of 13th International Conference on Machine Learning. Morgan Kaufmann.Google Scholar
  33. Zucker, J.-D. and Ganascia, J.-G. (1998). Learning Structural Indeterminate Clauses. In D. Page (Ed.), Proc. of the 8th International Workshop on Inductive Logic Programming (pp. 235-244). Springer Verlag.Google Scholar

Copyright information

© Kluwer Academic Publishers 2004

Authors and Affiliations

  • Érick Alphonse
    • 1
  • Stan Matwin
    • 2
  1. 1.LRI–UMR 8623 CNRSUniversit Paris-SudOrsay CedexFrance
  2. 2.SITEUniversity of OttawaOttawaCanada

Personalised recommendations