Machine Learning

, Volume 38, Issue 1–2, pp 63–87 | Cite as

Strategies in Combined Learning via Logic Programs

  • Evelina Lamma
  • Fabrizio Riguzzi
  • Luís Moniz Pereira


We discuss the adoption of a three-valued setting for inductive concept learning. Distinguishing between what is true, what is false and what is unknown can be useful in situations where decisions have to be taken on the basis of scarce, ambiguous, or downright contradictory information. In a three-valued setting, we learn a definition for both the target concept and its opposite, considering positive and negative examples as instances of two disjoint classes. To this purpose, we adopt Extended Logic Programs (ELP) under a Well-Founded Semantics with explicit negation (WFSX) as the representation formalism for learning, and show how ELPs can be used to specify combinations of strategies in a declarative way also coping with contradiction and exceptions.

Explicit negation is used to represent the opposite concept, while default negation is used to ensure consistency and to handle exceptions to general rules. Exceptions are represented by examples covered by the definition for a concept that belong to the training set for the opposite concept.

Standard Inductive Logic Programming techniques are employed to learn the concept and its opposite. Depending on the adopted technique, we can learn the most general or the least general definition. Thus, four epistemological varieties occur, resulting from the combination of most general and least general solutions for the positive and negative concept. We discuss the factors that should be taken into account when choosing and strategically combining the generality levels for positive and negative concepts.

In the paper, we also handle the issue of strategic combination of possibly contradictory learnt definitions of a predicate and its explicit negation.

All in all, we show that extended logic programs under well-founded semantics with explicit negation add expressivity to learning tasks, and allow the tackling of a number of representation and strategic issues in a principled way.

Our techniques have been implemented and examples run on a state-of-the-art logic programming system with tabling which implements WFSX.

inductive logic programming non-monotonic learning multi-strategy learning explicit negation contradiction handling 


  1. Alferes, J. J. Damásio, C. V., & Pereira, L. M. (1994). SLX-A top-down derivation procedure for programs with explicit negation. In M. Bruynooghe, (Ed.), Proc. Int. Symp. on Logic Programming. The MIT Press.Google Scholar
  2. Alferes, J. J. & Pereira, L. M. (1996). Reasoning with logic programming, LNAI (Vol. 1111). Springer-Verlag.Google Scholar
  3. Alferes, J. J., Pereira, L. M., & Przymusinski, T. C. (1998). “Classical” negation in non-monotonic reasoning and logic programming. Journal of Automated Reasoning, 20, 107–142.Google Scholar
  4. Bain, M. & Muggleton, S. (1992). Non-monotonic learning. In S. Muggleton, (Ed.), Inductive logic programming (pp. 145–161). Academic Press.Google Scholar
  5. Baral, C. & Gelfond, M. (1994). Logic programming and knowledge representation. Journal of logic programming, 19(20), 73–148.Google Scholar
  6. Chan, P. & Stolfo, S. (1993). Meta-learning for multistrategy and parallel learning. Proceedings of the 2nd International Workshop on Multistrategy Learning (pp. 150–165).Google Scholar
  7. Damásio, C. V., Nejdl, W., & Pereira, L. M. (1994). REVISE: An extended logic programming system for revising knowledge bases. In J. Doyle, E. Sandewall & P. Torasso, (Eds.), Knowledge representation and reasoning (pp. 607–618). Morgan Kaufmann.Google Scholar
  8. Damásio, C. V. & Pereira, L. M. (1997). Abduction on 3-valued extended logic programs. In V. W. Marek, A. Nerode & M. Trusczynski, (Eds.), Logic Programming and Non-Monotonic Reasoning-Proc. of 3rd International Conference LPNMR'95. LNAI (Vol. 925, pp. 29–42). Germany: Springer-Verlag.Google Scholar
  9. Damásio, C. V. & Pereira, L. M. (1998). A survey on paraconsistent semantics for extended logic programs. In D. Gabbay & P. Smets, (Eds.), Handbook of defeasible reasoning and uncertainty management systems (Vol. 2, pp. 241–320). Kluwer Academic Publishers.Google Scholar
  10. De Raedt, L. (1992). Interactive theory revision: an inductive logic programming approach. Academic Press.Google Scholar
  11. De Raedt, L., Bleken, E., Coget, V., Ghil, C., Swennen, B., & Bruynooghe, M. (1993). Learning to survive. Proceedings of the 2nd International Workshop on Multistrategy Learning (pp. 92–106).Google Scholar
  12. De Raedt, L. & Bruynooghe, M. (1989). Towards friendly concept-learners. Proceedings of the 11th International Joint Conference on Artificial Intelligence (pp. 849–856). Morgan Kaufmann.Google Scholar
  13. De Raedt, L. & Bruynooghe, M. (1990). On negation and three-valued logic in interactive concept learning. Proceedings of the 9th European Conference on Artificial Intelligence.Google Scholar
  14. De Raedt, L. & Bruynooghe, M. (1992). Interactive concept learning and constructive induction by analogy. Machine Learning, 8(2), 107–150.Google Scholar
  15. Dix, J. (1995). A classification-theory of semantics of normal logic programs: I & II. Fundamenta Informaticae, XXII(3), 227–255, 257–288.Google Scholar
  16. Dix, J., Pereira, L. M., & Przymusinski, T. (1997). Prolegomena to logic programming and non-monotonic reasoning. In J. Dix, L. M. Pereira & T. Przymusinski, (Eds.), Non-Monotonic Extensions of Logic Programming-Selected papers from NMELP'96. LNAI (Vol. 1216, pp. 1–36). Germany: Springer-Verlag.Google Scholar
  17. Drobnic, M. & Gams, M. (1993). Multistrategy learning: An analytical approach. Proceedings of the 2nd International Workshop on Multistrategy Learning (pp. 31–41).Google Scholar
  18. Džeroski, S. (1991). Handling noise in inductive logic programming. Master's Thesis, Faculty of Electrical Engineering and Computer Science, University of Ljubljana.Google Scholar
  19. Esposito, F., Ferilli, S., Lamma, E., Mello, P., Milano, M., Riguzzi, F., & Semeraro, G. (submitted for publication). Cooperation of abduction and induction in logic programming. In P. A. Flach & A. C. Kakas (Eds.), Abductive and inductive reasoning, pure and applied logic. Kluwer.Google Scholar
  20. Gelfond, M. & Lifschitz, V. (1988). The stable model semantics for logic programming. In R. Kowalski & K. A. Bowen (Eds.), Proceedings of the 5th Int. Conf. on Logic Programming (pp. 1070–1080) MIT Press.Google Scholar
  21. Gelfond, M. & Lifschitz, V. (1990). Logic programs with classical negation. Proceedings of the 7th International Conference on Logic Programming ICLP90 (pp. 579–597). The MIT Press.Google Scholar
  22. Gordon, D. & Perlis, D. (1989). Explicitly biased generalization. Computational Intelligence, 5(2), 67–81.Google Scholar
  23. Greiner, R., Grove, A. J.,& Roth, D. (1996). Learning active classifiers. Proceedings of the Thirteenth International Conference on Machine Learning (ICML96).Google Scholar
  24. Inoue, K. (submitted for publication). Learning abductive and nonmonotonic logic programs. In P. A. Flach & A. C. Kakas, (Eds.), Abductive and inductive reasoning, pure and applied logic. Kluwer.Google Scholar
  25. Inoue, K. & Kudoh, Y. (1997). Learning extended logic programs. Proceedings of the 15th International Joint Conference on Artificial Intelligence (pp. 176–181). Morgan Kaufmann.Google Scholar
  26. Jenkins, W. (1993). Intelog: A framework for multistrategy learning. Proceedings of the 2nd International Workshop on Multistrategy Learning (pp. 58–65).Google Scholar
  27. Lamma, E., Riguzzi, F., & Pereira, L. M. (1988). Learning in a three-valued setting. Proceedings of the Fourth International Workshop on Multistrategy Learning.Google Scholar
  28. Lamma, E., Riguzzi, F., & Pereira, L. M. (1999). Agents learning in a three-valued setting. Technical Report, DEIS-University of Bologna.Google Scholar
  29. Lamma, E., Riguzzi, F., & Pereira, L. M. (to appear). Strategies in combined learning via logic programs. Machine Learning.Google Scholar
  30. Lapointe, S. & Matwin, S. (1992). Sub-unification: A tool for efficient induction of recursive programs. In D. Sleeman & P. Edwards, (Eds.), Proceedings of the 9th International Workshop on Machine Learning (pp. 273–281). Morgan Kaufmann.Google Scholar
  31. Lavrač, N. & Džeroski, S. (1994). Inductive Logic Programming: Techniques and Applications. Ellis Horwood.Google Scholar
  32. Leite, J. A. & Pereira, L. M. (1998). Generalizing updates: from models to programs. In J. Dix, L. M. Pereira, & T. C. Przymusinski, (Eds.), Collected Papers from Workshop on Logic Programming and Knowledge Representation LPKR'97 LNAI (Vol. 1471) Springer-Verlag.Google Scholar
  33. Michalski, R. (1973). Discovery classification rules using variable-valued logic system VL1. Proceedings of the Third International Conference on Artificial Intelligence (pp. 162–172). Stanford University.Google Scholar
  34. Michalski, R. (1984). A theory and methodology of inductive learning. In R. Michalski, J. Carbonell & T. Mitchell, (Eds.), Machine Learning-an artificial intelligence approach (Vol. 1, pp. 83–134). Springer-Verlag.Google Scholar
  35. Muggleton, S. (1995). Inverse entailment and Progol. New Generation Computing, Special issue on Inductive Logic Programming, 13(3/4), 245–286.Google Scholar
  36. Muggleton, S. & Buntine, W. (1992). Machine invention of first-order predicates by inverting resolution. In S. Muggleton, (Ed.), Inductive Logic programming (pp. 261–280). Academic Press.Google Scholar
  37. Muggleton, S. & Feng, C. (1990). Efficient induction of logic programs. Proceedings of the 1st Conference on Algorithmic Learning Theory, Ohmsma, Tokyo, Japan (pp. 368–381).Google Scholar
  38. Pazzani, M. J., Merz, C., Murphy, P., Ali, K., Hume, T., & Brunk, C. (1994). Reducing misclassification costs. Proceedings of the Eleventh International Conference on Machine Learning (ML94) (pp. 217–225).Google Scholar
  39. Pereira, L. M. & Alferes, J. J. (1992).Well founded semantics for logic programs with explicit negation. Proceedings of the European Conference on Artificial Intelligenece ECAI92 (pp. 102–106). John Wiley and Sons.Google Scholar
  40. Plotkin, G. (1970). A note on inductive generalization. Machine Intelligence (Vol. 5, pp. 153–163). Edinburgh University Press.Google Scholar
  41. Provost, F. J. & Fawcett, T. (1997). Analysis and visualization of classifier performance: Comparison under imprecise class and cost distribution. Proceedings of the Third International Conference on Knowledge Discovery and Data Mining (KDD97). AAAI Press.Google Scholar
  42. Quinlan, J. (1990). Learning logical definitions from relations. Machine Learning, 5, 239–266.Google Scholar
  43. Quinlan, J. R. (1993). C4.5: Programs for Machine Learning. San Mateo, CA: Morgan Kaufmann.Google Scholar
  44. Reiter, R. (1978). On closed-word data bases. In H. Gallaire & J. Minker, (Eds.), Logic and data bases (pp. 55–76). Plenum Press.Google Scholar
  45. Sagonas, K. F., Swift, T., Warren, D. S., Freire, J., & Rao, P. (1997). The XSB Programmer's Manual Version 1.7.1.Google Scholar
  46. Van Gelder, A., Ross, K. A., & Schlipf, J. S. (1991). The well-founded semantics for general logic programs. Journal of the ACM, 38(3), 620–650.Google Scholar
  47. Vere, S. A. (1975). Induction of concepts in the predicate calculus. Proceedings of the Fourth International Joint Conference on Artificial Intelligence (IJCAI75) (pp. 281–287).Google Scholar

Copyright information

© Kluwer Academic Publishers 2000

Authors and Affiliations

  • Evelina Lamma
    • 1
  • Fabrizio Riguzzi
    • 2
  • Luís Moniz Pereira
    • 3
  1. 1.Università di BolognaBolognaItaly
  2. 2.Università di BolognaBolognaItaly
  3. 3.Centro de Inteligência Artificial (CENTRIA), Departamento de Informática, Faculdade de Ciências e TecnologiaUniversidade Nova de LisboaMonte da CaparicaPortugal

Personalised recommendations