Machine Learning

, Volume 55, Issue 2, pp 137–174 | Cite as

Fast Theta-Subsumption with Constraint Satisfaction Algorithms

  • Jérôme Maloberti
  • Michèle Sebag


Relational learning and Inductive Logic Programming (ILP) commonly use as covering test the θ-subsumption test defined by Plotkin. Based on a reformulation of θ-subsumption as a binary constraint satisfaction problem, this paper describes a novel θ-subsumption algorithm named Django,1 which combines well-known CSP procedures and θ-subsumption-specific data structures. Django is validated using the stochastic complexity framework developed in CSPs, and imported in ILP by Giordana et Saitta. Principled and extensive experiments within this framework show that Django improves on earlier θ-subsumption algorithms by several orders of magnitude, and that different procedures are better at different regions of the stochastic complexity landscape. These experiments allow for building a control layer over Django, termed Meta-Django, which determines the best procedures to use depending on the order parameters of the θ-subsumption problem instance. The performance gains and good scalability of Django and Meta-Django are finally demonstrated on a real-world ILP task (emulating the search for frequent clauses in the mutagenesis domain) though the smaller size of the problems results in smaller gain factors (ranging from 2.5 to 30).

relational learning constraint satisfaction phase transition meta-learning k-locality 


  1. Alphonse, E., & Rouveirol, C. (2000). Lazy propositionalisation for relational learning. In W. Horn (Ed.), Proceedings of the 14th European Conference on Artificial Intelligence, ECAI’OO (pp. 256–260). IOS Press.Google Scholar
  2. Agrawal, R., & Srikant, R. (1994). Fast algorithms for mining association rules. In J. B. Bocca, M. Jarke, & C. Zaniolo (Eds.), Proceedings of the 20th Int. Conf Very Large Data Bases, VLDB (pp. 487–499). Morgan Kaufmann.Google Scholar
  3. Augier, S. (2000). Apprentissage Supervise relationnel par algorithmes dvolution. PhD thesis, Universit6 de Paris 11-Orsay.Google Scholar
  4. Blockeel, H., Dehaspe, L., Demoen, B., Janssens, G., Ramon, J., & Vandecasteele, H. (2000). Executing query packs in ILP. In Proceedings of the o10th International Conference on Inductive Logic Programming, LNAI 1866 (pp. 60–77). Berlin: Springer-Verlag.Google Scholar
  5. Brazdil, P., Gama, J., & Henery, B. (1994). Characterizing the applicability of classification algorithms using meta-level learning. In Proceedings of the 7th European Conference on Machine Learning:pp. 83–102). Lecture Notes in Artificial Intelligence, Springer.Google Scholar
  6. Botta, M., Giordana, A., Saitta, L., & Sebag, M. (2003). Relational learning as searh in aritical region. Journal of Machine Learning Research, 4, 431–463.CrossRefGoogle Scholar
  7. Bensusan, H., & Kalousis, A. (2001). Estimating the predictive accuracy ofa:classifier. In P. Flach, & L. De Raedt (Eds.), Proceedings of the 12th European conference on machine learning (pp. 25–36). Springer Verlag.Google Scholar
  8. Blake, C., Keogh, E., & Merz, C.J. (1998). UCI Repository of-machine learning databases. http://www.ics.’mlearn/MLRepository.html.Google Scholar
  9. Bessiere, C., & Rtgin, J.-C. (1996). Mac and combined heuristics: Two reasons to forsake FC (and CBJ ?) on hard problems. In Proceedings of the 2nd Int. Conf on Principles and Practice of Constraint Programming (pp. 61–75).Google Scholar
  10. Blockeel, H., & DeRaedt, L. (1998). Top-downinduction of first order logical decision trees. Artificial Intelligence, 101:1/2, 285–297.CrossRefMathSciNetGoogle Scholar
  11. Chen, X. (2000). A theoretical comparison of selected CSP solving and modeling techniques. PhD thesis, University of Alberta, Canada.Google Scholar
  12. Cheeseman, P., Kanefsky, B., & TaylorW.M. (1991). Where the really hard problems are. In Proceedings of lnt. Joint Conf on Artificial Inteligce, IJCAI’91 (pp. 331–337).Google Scholar
  13. Costa, V., Srinivasan, A., Camacho, R., Blockeel, H., Demoen, B., Janssens, G., Struyf, J., Vandecasteele, H., & Van Laer, W. (2004). Query transformations for improving the efficiency of ILP systems. Journal of Machine Learning Research, 4, 465–491. CrossRefGoogle Scholar
  14. Dzeroski, S., & Lavrac N. (Eds.). (2001). Relational data mining. Springer Verlag.Google Scholar
  15. Dolsak, D. & Muggleton, S. (1991). The application of ILP to finite element mesh design. In S. Muggleton (Ed.), Proceedingsof the First International Workshop on Inductive Logic Programming (pp. 453–472).Google Scholar
  16. Dechter, R., & Pearl, J. (1989). Tree clustering for constraint networks. Artificial Intelligence, 38, 353–366.CrossRefGoogle Scholar
  17. Dehaspe, L., & De Raedt, L. (1997). Mining association rules in multiple relations. In Proceedings of the 7th International Workshop on Inductive Logic Programming, LNCS 1297 (pp. 125–132). Berlin: Springer-Verlag.Google Scholar
  18. Esposito, F., Fanizzi, N., Ferilli, S., & Semeraro, G. (2000). Ideal theory refinement under object identity. In P. Langley (Ed.), Proceedings of the 17th International Conference on Machine Learning (pp. 263–270). Morgan Kaufmann.Google Scholar
  19. Furnkranz, J., & Petrak, J. (2001). An evaluation of landmarking variants. Technical report, Austrian Research Institute for Artificial Intelligence.Google Scholar
  20. Gomes, C.P., Selman, B., Crato, N., & Kautz, H.A. (2000). Heavy-tailed phenomena in satisfiability and constraint satisfaction problems. Journal ofAutomated Reasoning, 24:1/2, 67–100.CrossRefGoogle Scholar
  21. Gyssens, M., Jeavons, PG., & Cohen, D.A. (1994). Decomposing constraint satisfaction problems using database techniques. Artificial Intelligence, 66, 57–89.CrossRefGoogle Scholar
  22. Gottlob, G., & Leitsch, A. (1985). On the efficiency of subsumption algorithms. Journal of the Association for Computing Machinery, 32:2, 280–295. Google Scholar
  23. Gottlob, G., Leone, N., & Scarcello, F. (2000). A comparison of structural CSP decomposition methods. Artificial Intelligence, 124:2, 243–282.CrossRefGoogle Scholar
  24. Gent, IP., MacIntyre, E., Prosser, P., & Walsh, T. (1996). The constrainedness of search. In AAAI/IAAI, vol. 1 (pp. 246–252).Google Scholar
  25. Giordana, A., & Saitta, L. (2000). Phase transitions in relational learning. Machine Learning, 2, 217–251.CrossRefGoogle Scholar
  26. Hogg, T., Huberman, B.A., & Williams, C.P. (Eds.). (1996). Artificial intelligence: Special issue on frontiers in problem solving: Phase transitions and complexity, vol. 81(1/2), Elsevier.Google Scholar
  27. Hirata, K. (2003). On condensation of a clause. In T. Horvath, & A. Yamamoto (Eds.), Proceedings of the 15th Int. Conf on Inductive Logic Programming, LNAI (pp. 164–179). Springer-Verlag.Google Scholar
  28. Hogg, T. (1996). Refining the phase transition in combinatorial search. Artificial Intelligence, 81, 127–154.CrossRefGoogle Scholar
  29. Kalousis, A. (2002). Algorithm Selection via Meta-Learning. PhD thesis, Universit6 de Genve, Centre Universitaire d’Informatique.Google Scholar
  30. Kietz, J.-U., & Ltibbe, M. (1994). An efficient subsumption algorithm for inductive logic programming. In W. Cohen, & H. Hirsh (Eds.), Proceedings of the 11th International Conference on Machine Learning (pp. 130–137). Morgan Kaufmann.Google Scholar
  31. Kramer, S., Lavrac, N., & Flach, P. (2001). Propositionalization approaches torelational data mini. In S. Dzeroski, & N. Lavrac (Eds.), Relational data mining (pp. 262–291). Springer-Verlag.Google Scholar
  32. King, R.D., Srinivasan, A., & Steinberg, M.J.E. (1995). Relating chemical activity to structure: An examination of ILP successes. New Gen. Comput., 13. Google Scholar
  33. Lachiche, N. & Flach, P.A. (2003). A true first-order bayesian classifier. In Si. Matwin (Ed.), Proceedings of the 14th Int. Conf on Inductive Logic Programming, LNAI (pp. 133–148). Springer.Google Scholar
  34. Lloyd, J. (2003). Logic for learning: Learning comprehensive theories frostructured data. Springer-Verlag.Google Scholar
  35. Mackworth, A.K. (1977). Consistency in networks of relations. Arficial Intelligence, 8, 99–118.CrossRefGoogle Scholar
  36. Di Mauro, N., Basile, T.M.A., Ferilli, S., Esposito, F., & Fanizzi:N. (2003). An exhaustive matching procedure for the improvement of learning efficiency. In T. Horvath & A. Yamamoto (Eds.), Proceedings ofthe 15th Int. Conf on Inductive Logic Programming, LNAI (pp. 112–129): Springer-Verlag.Google Scholar
  37. Muggleton, S., & De Raedt, L. (1994). Inductive itlgic programming: Theory and methods. Journal of Logic Programming, 19, 629–679.CrossRefGoogle Scholar
  38. Maloberti, J., & Sebag, M. (2001). Theta-subsumptlon in a constraint satisfaction perspective. In C. Rouveirol, & M. Sebag (Eds.), Proceedings of thel3th Inductive Logic Programming, LNAI 2157 (pp. 164–178). Springer-Verlag.Google Scholar
  39. Maloberti, J., & Suzuki, E. (2003). Improving efficiency of frequent query discovery by eliminating non-relevant candidates. In Proceeding of t:he6th International Conference on Discovery Science (DS 2003), LNAI (pp. 219–231), Berlin, Hidelberg, New York, Springer-Verlag.Google Scholar
  40. Muggleton, S. (1995). Inverse entailment and PROGOL. New Gen. Comput., 13,245–286. Google Scholar
  41. Nienhuys-Cheng, S., & de Wolf, R. (1997). Foundations of inductive logic programming. Springer-Verlag.Google Scholar
  42. Nedellec, C., Roueiol, C., Ad, H., Bergadano, F., & Tausend, B. (1996). Declarative bias in ILP. In L. de Raedt (Ed.), AdvancsI ILP (pp. 82–103). IOS Press.Google Scholar
  43. Pfahringer, B. Bensusan, H., & Giraud-Carrier, C. (2000). Meta-learning by landmarking various learning algorithms. In Proceedings of the 17th International Conference on Machine Learning (pp. 743–750). Morgan Kaufmann.Google Scholar
  44. Pefia-Castillo, L., & Wrobel, S. (2002). Macro-operators in multirelational learning: A search-space reduction technique. In H. Mannila et al. (Eds.), Proceedings of the 13th European Conference on Machine Learning, LNCS 2430 (pp. 357–368). Berlin: Springer-Verlag.Google Scholar
  45. Plotkin, G. (1970). A note on inductive generalization. In B. Meltzer, & D. Michie (Eds.), Machine Intelligence, vol. 5 (pp. 153–163). Edinburgh University Press.Google Scholar
  46. Prosser, P. (1993). Hybrid algorithms for the constraint satisfaction problem. Computational Intelligence, 9, 268–299.Google Scholar
  47. Quinlan, J.R. (1990). Learning logical definitions from relations. Machine Learning, 5:3, 239–266.Google Scholar
  48. De Raedt, L. (1997). Logical settings for concept learning. Artificial Intelligence, 95:1, 187–201.CrossRefGoogle Scholar
  49. De Raedt, L., & Bruynooghe, M. (1993). A theory of clausal discovery. In Proceedings of Int. Joint Conf on Artificial Intelligence, IJCAI’93 (pp. 1058–1063). Morgan Kaufmann.Google Scholar
  50. Read, R.C., & Corneil, D.G. (1977). The graph isomorph disease. Journal of Graph Theory, 1, 339–363. Google Scholar
  51. Robinson, J. (1965). A machine-oriented logic based on the resolution principle. Journal of the ACM, 12:1, 23–41.CrossRefGoogle Scholar
  52. Scheffer, T., Herbrich, R., & Wysotzki, F. (1997). Efficient 0-subsumption based on graph algorithms. In S. Muggleton (Ed.), Proceedings Int. Workshop on Inductive Logic Programming (pp. 212–228). Springer-Verlag.Google Scholar
  53. Sebag. M., & Rouveirol, C. (2000). Resource-bounded relational reasoning: Induction and deduction through stochastic matching. Machine Learning, 38, 41–62.CrossRefGoogle Scholar
  54. Srinivasan, A. (1997). The predictive toxicology evaluation challenge. In Proceedings of Int. Joint Conf on Artificial Intelligence, IJCAI’97 (pp. 4–8). Morgan Kaufmann.Google Scholar
  55. Srinivasan, A. (2001). The Aleph manual, http://we Scholar
  56. Tsang, E. (1993). Foundations of constraint satisfaction. Academic Press.Google Scholar

Copyright information

© Kluwer Academic Publishers 2004

Authors and Affiliations

  • Jérôme Maloberti
    • 1
  • Michèle Sebag
    • 1
  1. 1.Laboratoire de Recherche en InformatiqueCNRS UMR 8623

Personalised recommendations