# Strategies in Combined Learning via Logic Programs

- 224 Downloads
- 17 Citations

## Abstract

We discuss the adoption of a three-valued setting for inductive concept learning. Distinguishing between what is true, what is false and what is unknown can be useful in situations where decisions have to be taken on the basis of scarce, ambiguous, or downright contradictory information. In a three-valued setting, we learn a definition for both the target concept and its opposite, considering positive and negative examples as instances of two disjoint classes. To this purpose, we adopt Extended Logic Programs (ELP) under a Well-Founded Semantics with explicit negation (WFSX) as the representation formalism for learning, and show how ELPs can be used to specify combinations of strategies in a declarative way also coping with contradiction and exceptions.

Explicit negation is used to represent the opposite concept, while default negation is used to ensure consistency and to handle exceptions to general rules. Exceptions are represented by examples covered by the definition for a concept that belong to the training set for the opposite concept.

Standard Inductive Logic Programming techniques are employed to learn the concept and its opposite. Depending on the adopted technique, we can learn the most general or the least general definition. Thus, four epistemological varieties occur, resulting from the combination of most general and least general solutions for the positive and negative concept. We discuss the factors that should be taken into account when choosing and strategically combining the generality levels for positive and negative concepts.

In the paper, we also handle the issue of strategic combination of possibly contradictory learnt definitions of a predicate and its explicit negation.

All in all, we show that extended logic programs under well-founded semantics with explicit negation add expressivity to learning tasks, and allow the tackling of a number of representation and strategic issues in a principled way.

Our techniques have been implemented and examples run on a state-of-the-art logic programming system with tabling which implements WFSX.

## References

- Alferes, J. J. Damásio, C. V., & Pereira, L. M. (1994). SLX-A top-down derivation procedure for programs with explicit negation. In M. Bruynooghe, (Ed.),
*Proc. Int. Symp. on Logic Programming*. The MIT Press.Google Scholar - Alferes, J. J. & Pereira, L. M. (1996).
*Reasoning with logic programming*,*LNAI*(Vol. 1111). Springer-Verlag.Google Scholar - Alferes, J. J., Pereira, L. M., & Przymusinski, T. C. (1998). “Classical” negation in non-monotonic reasoning and logic programming.
*Journal of Automated Reasoning, 20*, 107–142.Google Scholar - Bain, M. & Muggleton, S. (1992). Non-monotonic learning. In S. Muggleton, (Ed.),
*Inductive logic programming*(pp. 145–161). Academic Press.Google Scholar - Baral, C. & Gelfond, M. (1994). Logic programming and knowledge representation.
*Journal of logic programming, 19*(20), 73–148.Google Scholar - Chan, P. & Stolfo, S. (1993). Meta-learning for multistrategy and parallel learning.
*Proceedings of the 2nd International Workshop on Multistrategy Learning*(pp. 150–165).Google Scholar - Damásio, C. V., Nejdl, W., & Pereira, L. M. (1994). REVISE: An extended logic programming system for revising knowledge bases. In J. Doyle, E. Sandewall & P. Torasso, (Eds.),
*Knowledge representation and reasoning*(pp. 607–618). Morgan Kaufmann.Google Scholar - Damásio, C. V. & Pereira, L. M. (1997). Abduction on 3-valued extended logic programs. In V. W. Marek, A. Nerode & M. Trusczynski, (Eds.),
*Logic Programming and Non-Monotonic Reasoning-Proc. of 3rd International Conference LPNMR'95*. LNAI (Vol. 925, pp. 29–42). Germany: Springer-Verlag.Google Scholar - Damásio, C. V. & Pereira, L. M. (1998). A survey on paraconsistent semantics for extended logic programs. In D. Gabbay & P. Smets, (Eds.),
*Handbook of defeasible reasoning and uncertainty management systems*(Vol. 2, pp. 241–320). Kluwer Academic Publishers.Google Scholar - De Raedt, L. (1992).
*Interactive theory revision: an inductive logic programming approach*. Academic Press.Google Scholar - De Raedt, L., Bleken, E., Coget, V., Ghil, C., Swennen, B., & Bruynooghe, M. (1993). Learning to survive.
*Proceedings of the 2nd International Workshop on Multistrategy Learning*(pp. 92–106).Google Scholar - De Raedt, L. & Bruynooghe, M. (1989). Towards friendly concept-learners.
*Proceedings of the 11th International Joint Conference on Artificial Intelligence*(pp. 849–856). Morgan Kaufmann.Google Scholar - De Raedt, L. & Bruynooghe, M. (1990). On negation and three-valued logic in interactive concept learning.
*Proceedings of the 9th European Conference on Artificial Intelligence*.Google Scholar - De Raedt, L. & Bruynooghe, M. (1992). Interactive concept learning and constructive induction by analogy.
*Machine Learning, 8*(2), 107–150.Google Scholar - Dix, J. (1995). A classification-theory of semantics of normal logic programs: I & II.
*Fundamenta Informaticae, XXII*(3), 227–255, 257–288.Google Scholar - Dix, J., Pereira, L. M., & Przymusinski, T. (1997). Prolegomena to logic programming and non-monotonic reasoning. In J. Dix, L. M. Pereira & T. Przymusinski, (Eds.),
*Non-Monotonic Extensions of Logic Programming-Selected papers from NMELP'96*. LNAI (Vol. 1216, pp. 1–36). Germany: Springer-Verlag.Google Scholar - Drobnic, M. & Gams, M. (1993). Multistrategy learning: An analytical approach.
*Proceedings of the 2nd International Workshop on Multistrategy Learning*(pp. 31–41).Google Scholar - Džeroski, S. (1991).
*Handling noise in inductive logic programming*. Master's Thesis, Faculty of Electrical Engineering and Computer Science, University of Ljubljana.Google Scholar - Esposito, F., Ferilli, S., Lamma, E., Mello, P., Milano, M., Riguzzi, F., & Semeraro, G. (submitted for publication). Cooperation of abduction and induction in logic programming. In P. A. Flach & A. C. Kakas (Eds.),
*Abductive and inductive reasoning*,*pure and applied logic.*Kluwer.Google Scholar - Gelfond, M. & Lifschitz, V. (1988). The stable model semantics for logic programming. In R. Kowalski & K. A. Bowen (Eds.),
*Proceedings of the 5th Int. Conf. on Logic Programming*(pp. 1070–1080) MIT Press.Google Scholar - Gelfond, M. & Lifschitz, V. (1990). Logic programs with classical negation.
*Proceedings of the 7th International Conference on Logic Programming ICLP90*(pp. 579–597). The MIT Press.Google Scholar - Gordon, D. & Perlis, D. (1989). Explicitly biased generalization.
*Computational Intelligence, 5*(2), 67–81.Google Scholar - Greiner, R., Grove, A. J.,& Roth, D. (1996). Learning active classifiers.
*Proceedings of the Thirteenth International Conference on Machine Learning (ICML96)*.Google Scholar - Inoue, K. (submitted for publication). Learning abductive and nonmonotonic logic programs. In P. A. Flach & A. C. Kakas, (Eds.),
*Abductive and inductive reasoning, pure and applied logic*. Kluwer.Google Scholar - Inoue, K. & Kudoh, Y. (1997). Learning extended logic programs.
*Proceedings of the 15th International Joint Conference on Artificial Intelligence*(pp. 176–181). Morgan Kaufmann.Google Scholar - Jenkins, W. (1993). Intelog: A framework for multistrategy learning.
*Proceedings of the 2nd International Workshop on Multistrategy Learning*(pp. 58–65).Google Scholar - Lamma, E., Riguzzi, F., & Pereira, L. M. (1988). Learning in a three-valued setting.
*Proceedings of the Fourth International Workshop on Multistrategy Learning*.Google Scholar - Lamma, E., Riguzzi, F., & Pereira, L. M. (1999).
*Agents learning in a three-valued setting*. Technical Report, DEIS-University of Bologna.Google Scholar - Lamma, E., Riguzzi, F., & Pereira, L. M. (to appear). Strategies in combined learning via logic programs.
*Machine Learning*.Google Scholar - Lapointe, S. & Matwin, S. (1992). Sub-unification: A tool for efficient induction of recursive programs. In D. Sleeman & P. Edwards, (Eds.),
*Proceedings of the 9th International Workshop on Machine Learning*(pp. 273–281). Morgan Kaufmann.Google Scholar - Lavrač, N. & Džeroski, S. (1994).
*Inductive Logic Programming: Techniques and Applications*. Ellis Horwood.Google Scholar - Leite, J. A. & Pereira, L. M. (1998). Generalizing updates: from models to programs. In J. Dix, L. M. Pereira, & T. C. Przymusinski, (Eds.),
*Collected Papers from Workshop on Logic Programming and Knowledge Representation LPKR'97*LNAI (Vol. 1471) Springer-Verlag.Google Scholar - Michalski, R. (1973). Discovery classification rules using variable-valued logic system VL1.
*Proceedings of the Third International Conference on Artificial Intelligence*(pp. 162–172). Stanford University.Google Scholar - Michalski, R. (1984). A theory and methodology of inductive learning. In R. Michalski, J. Carbonell & T. Mitchell, (Eds.),
*Machine Learning-an artificial intelligence approach*(Vol. 1, pp. 83–134). Springer-Verlag.Google Scholar - Muggleton, S. (1995). Inverse entailment and Progol.
*New Generation Computing, Special issue on Inductive Logic Programming, 13*(3/4), 245–286.Google Scholar - Muggleton, S. & Buntine, W. (1992). Machine invention of first-order predicates by inverting resolution. In S. Muggleton, (Ed.),
*Inductive Logic programming*(pp. 261–280). Academic Press.Google Scholar - Muggleton, S. & Feng, C. (1990). Efficient induction of logic programs.
*Proceedings of the 1st Conference on Algorithmic Learning Theory*, Ohmsma, Tokyo, Japan (pp. 368–381).Google Scholar - Pazzani, M. J., Merz, C., Murphy, P., Ali, K., Hume, T., & Brunk, C. (1994). Reducing misclassification costs.
*Proceedings of the Eleventh International Conference on Machine Learning (ML94)*(pp. 217–225).Google Scholar - Pereira, L. M. & Alferes, J. J. (1992).Well founded semantics for logic programs with explicit negation.
*Proceedings of the European Conference on Artificial Intelligenece ECAI92*(pp. 102–106). John Wiley and Sons.Google Scholar - Plotkin, G. (1970). A note on inductive generalization.
*Machine Intelligence*(Vol. 5, pp. 153–163). Edinburgh University Press.Google Scholar - Provost, F. J. & Fawcett, T. (1997). Analysis and visualization of classifier performance: Comparison under imprecise class and cost distribution.
*Proceedings of the Third International Conference on Knowledge Discovery and Data Mining (KDD97)*. AAAI Press.Google Scholar - Quinlan, J. (1990). Learning logical definitions from relations.
*Machine Learning, 5*, 239–266.Google Scholar - Quinlan, J. R. (1993).
*C4.5: Programs for Machine Learning*. San Mateo, CA: Morgan Kaufmann.Google Scholar - Reiter, R. (1978). On closed-word data bases. In H. Gallaire & J. Minker, (Eds.),
*Logic and data bases*(pp. 55–76). Plenum Press.Google Scholar - Sagonas, K. F., Swift, T., Warren, D. S., Freire, J., & Rao, P. (1997).
*The XSB Programmer's Manual Version 1.7.1*.Google Scholar - Van Gelder, A., Ross, K. A., & Schlipf, J. S. (1991). The well-founded semantics for general logic programs.
*Journal of the ACM, 38*(3), 620–650.Google Scholar - Vere, S. A. (1975). Induction of concepts in the predicate calculus.
*Proceedings of the Fourth International Joint Conference on Artificial Intelligence (IJCAI75)*(pp. 281–287).Google Scholar