Traps and pitfalls when learning logical definitions from relations

  • Floriana Esposito
  • Donato Malerba
  • Giovanni Semeraro
  • Clifford Brunk
  • Michael Pazzani
Communications Learning and Adaptive Systems
Part of the Lecture Notes in Computer Science book series (LNCS, volume 869)


In the paper, we present some learning tasks that cannot be solved by two wellknown systems, FOIL and FOCL. Two kinds of explanations can be provided for these failures. For some tasks, the failures can be ascribed to a wrong definition of the space in which these systems perform the search for logical definitions. By moving from θ-subsumption to a weaker, but more mechanizable and manageable, model of generalization, called θOI-subsumption, a new search space is defined in which such tasks can be solved. Such a solution has been implemented in a new version of FOCL, called FOCL-OI. However, other learning tasks cannot be solved by changing the search space. For these tasks, the conceptual problem detected both in FOIL and in FOCL concerns the generation of meaningless rules, which do not mirror at all the structure of the training instances. We claim that, whenever possible, the training/test examples should be represented as ground Horn clauses, rather than as tuples of a relational database or facts of a Prolog database.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bell, S., and Weber, S., On the close logical relationship between FOIL and the frameworks of Helft and Plotkin, Proceedings of The Third International Workshop on Inductive Logic Programming ILP'93, Bled, Slovenia, 1–10, 1993.Google Scholar
  2. 2.
    Bergadano, F., Giordana, A., and Saitta, L., Automated concept acquisition in noisy environments, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. PAMI-10, 555–578, 1988.Google Scholar
  3. 3.
    Datta, P., and Kibler, D., Concept Sharing: A Means to Improve Multi-Concept Learning, Proceedings of the 10th International Conference on Machine Learning, Amherst, MA, 89–96, 1993.Google Scholar
  4. 4.
    Esposito, F., Automated acquisition of production rules by empirical supervised learning methods, in Data, Expert Knowledge and Decisions, (Vol. II), M. Schader (Ed.), Springer-Verlag, Heidelberg, Germany, 1990.Google Scholar
  5. 5.
    Esposito, F., Malerba, D., Semeraro, G., and Pazzani, M., A Machine Learning Approach To Document Understanding, Proceedings of the 2nd International Workshop on Multistrategy Learning MSL-93, Harpers Ferry, West Virginia, 276–292, 1993.Google Scholar
  6. 6.
    Esposito, F., Malerba, D., and Semeraro, G., Multistrategy Learning for Document Recognition, Applied Artificial Intelligence 8, 33–88, 1994.Google Scholar
  7. 7.
    Esposito F., Malerba, D., Semeraro, G., Brunk, C., and Pazzani, M., Avoiding Non-Termination when Learning Logical Programs: A Case Study with FOIL and FOCL, in Pre-proceedings of LOPSTR 94 — Fourth Workshop on Logic Program Synthesis and Transformation, Pisa, Italy, June 20–21, 1994.Google Scholar
  8. 8.
    Garey, M.R., and Johnson, D.S., Computers and Intractability, Freeman, San Francisco, CA, 1979.Google Scholar
  9. 9.
    Genesereth, M.R., and Nilsson, N.J., Logical Foundations of Artificial Intelligence, Morgan Kaufmann, Palo Alto, CA, 1987.Google Scholar
  10. 10.
    Hayes-Roth, F., Schematic classification problems and their solution, Pattern Recognition, 105–113, 1974.Google Scholar
  11. 11.
    Helft, N., Inductive Generalization: A Logical Framework, in Progress in Machine Learning-Proceedings of EWSL 87, I. Bratko & N. Lavrac (Eds.), Sigma Press, Bled, Yugoslavia, 149–157, 1987.Google Scholar
  12. 12.
    Knight, K., Unification: A Multidisciplinary Survey, ACM Computing Surveys, Vol.21, No.1, 1989.Google Scholar
  13. 13.
    Lloyd, J.W., Foundations of Logic Programming, Second Edition, Springer-Verlag, New York, 1987.Google Scholar
  14. 14.
    Malerba, D., Document Understanding: A Machine Learning Approach, Technical Report, Esprit Project 5203 INTREPID, March 1993.Google Scholar
  15. 15.
    Michalski, R.S., Pattern Recognition as Rule-Guided Inductive Inference, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. PAMI-2, 349–361, 1980.Google Scholar
  16. 16.
    Muggleton, S., Inductive Logic Programming, New Generation Computing, 8(4), 295–318, 1991.Google Scholar
  17. 17.
    Pazzani, M.J., and Brunk, C., FOCL-1-2-3 Version 1.1, Department of Information and Computer Science, University of California, Irvine, California, March 1993.Google Scholar
  18. 18.
    Pazzani, M., and Kibler, D., The utility of knowledge in inductive learning, Machine Learning 9, 1, 57–94, 1992.Google Scholar
  19. 19.
    Plotkin, G.D., A Note on Inductive Generalization, in Machine Intelligence 5, B. Meltzer and D. Michie (Eds.), 153–163, Edinburgh University Press, 1970.Google Scholar
  20. 20.
    Quinlan, J. R., Learning Logical Definitions from Relations, Machine Learning 5, 3, 239–266, 1990.Google Scholar
  21. 21.
    Quinlan, J.R., Determinate Literals in Inductive Logic Programming, Proceedings of the 11th International Joint Conference on Artificial Intelligence, Sydney, Australia, 746–750, 1991.Google Scholar
  22. 22.
    Quinlan, J.R., Cameron-Jones, R.M., FOIL:amidterm report, in Machine Learning:ECML-93-Proceedings of the Sixth European Conference on Machine Learning, Lecture Notes in Artificial Intelligence 667, Pavel B. Brazdil (Ed.), Springer-Verlag, Vienna, Austria, 3–20, 1993.Google Scholar
  23. 23.
    Rissanen, J., A universal prior for integers and estimation by minimum description length, Annals of Statistics, 11, 1, 416–431, 1983.Google Scholar
  24. 24.
    Semeraro, G., Brunk, C.A., and Pazzani M.J., Traps and Pitfalls when Learning Logical Theories: A Case Study with FOIL and FOCL, Technical Report 93-33, Department of Information and Computer Science, University of California, Irvine, California, July 26, 1993.Google Scholar
  25. 25.
    Semeraro, G., Esposito, F., and Malerba, D., Learning Contextual Rules for Document Understanding, Proceedings of the Tenth IEEE Conference on Artificial Intelligence for Applications, San Antonio, Texas, 108–115, 1994.Google Scholar
  26. 26.
    Silverstein, G., and Pazzani, M., Relational clichés: constraining constructive induction during relational learning, Proceedings of the Eighth International Workshop on Machine Learning, Evanston, Illinois, 203–207, 1991.Google Scholar
  27. 27.
    Stepp, R.E., Conjunctive Conceptual Clustering: A Methodology and Experimentation, Ph.D. dissertation, University of Illinois at Urbana-Champaign, Urbana, Illinois, UIUCDCS-R-84-1189, 1984.Google Scholar
  28. 28.
    Vere, S.A., Induction of relational productions in the presence of background information, Proceedings of the 5th International Joint Conference on Artificial Intelligence, Cambridge, MA, 349–355, 1977.Google Scholar
  29. 29.
    Winston, P.H., Learning Structural Descriptions from Examples, Ph.D. dissertation, Dept. of Electrical Engineering, Massachusetts Institute of Technology, Cambridge, MA, January 1970.Google Scholar
  30. 30.
    Wirth, R., and O'Rorke, P., Constraints on Predicate Invention, Proceedings of the Eighth International Workshop on Machine Learning, Evanston, Illinois, 457–461, 1991.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1994

Authors and Affiliations

  • Floriana Esposito
    • 1
  • Donato Malerba
    • 1
  • Giovanni Semeraro
    • 1
  • Clifford Brunk
    • 2
  • Michael Pazzani
    • 2
  1. 1.Laboratorio di Acquisizione della Conoscenza e Apprendimento nelle Macchine Dipartimento di InformaticaUniversità degli Studi di BariBariItaly
  2. 2.Department of Information and Computer ScienceUniversity of California, IrvineIrvineUSA

Personalised recommendations