## Abstract

We investigate how abduction and induction can be integrated into a common learning framework. In particular, we consider an extension of Inductive Logic Programming (ILP) for the case in which both the background and the target theories are abductive logic programs and where an abductive notion of entailment is used as the basic coverage relation for learning. This extended learning framework has been called Abductive Concept Learning (ACL). In this framework, it is possible to learn with incomplete background information about the training examples by exploiting the hypothetical reasoning of abduction. We also study how the ACL framework can be used as a basis for multiple predicate learning.

An algorithm for ACL is developed by suitably extending the top-down ILP method: the deductive proof procedure of Logic Programming is replaced by an abductive proof procedure for Abductive Logic Programming. This algorithm also incorporates a phase for learning integrity constraints by suitably employing a system that learns from interpretations like ICL. The framework of ACL thus integrates the two ILP settings of explanatory (predictive) learning and confirmatory (descriptive) learning. The above algorithm has been implemented into a system also called ACL Several experiments have been performed that show the effectiveness of the ACL framework in learning from incomplete data and its appropriate use for multiple predicate learning.

## Keywords

Machine Learning Inductive Logic Programming Abductive Logic Programming Non-Monotonic Reasoning## Preview

Unable to display preview. Download preview PDF.

## References

- 1).Abe., A., “The Relation between Abductive Hypotheses and Inductive Hypotheses,”
*Flach and Kakas.*, Kluwer, 1998.Google Scholar - 2).Adé, A., and Denecker, M., “RUTH: An ILP Theory Revision System,” in
*Proc. of the 8th International Symposium on Methodologies for Intelligent Systems*, 1994.Google Scholar - 3).Adé, H., and Denecker, M., “AILP: Abductive Inductive Logic Programming,” in
*Proc. of the 14th International Joint Conference on Artificial Intelligence*, 1995.Google Scholar - 4).Apt, K. R., and Bezem, M., “Acyclic Programs,” in
*Proc. of the 7th International Conference on Logic Programming*, pp. 579–597, Jerusalem, 1990.Google Scholar - 5).Bain, M., and Muggleton, S., “Non-monotonic Learning,” Michie, J. E. H., and Tyugu, E., ed.,
*Machine Intelligence, 12*, Oxford University Press, 1991.Google Scholar - 6).Blockeel, H., and De Raedt, L., “Inductive Database Design,” in
*Proc. of the 10th International Symposium on Methodologies for Intelligent Systems, 1079*of*Lecture Notes in Artificial Intelligence*, pp. 376–385, Springer-Verlag, 1996.Google Scholar - 7).Keogh, E., Blake, C., and Merz, C. J., “UCI Repository of Machine Learning Databases,” 1998.Google Scholar
- 8).Cestnik, B., Knononenko, I., and Bratko, I., “ASSISTANT 86: A Knowledge Elicitation Tool for Sophisticated Users.” Bratko, I., and Lavrač, N., eds.,
*Progress in Machine Learning*, pp. 31–45, Sigma Press, Wilmslow, UK, 1987.Google Scholar - 9).Clark, P., and Boswell, R., “The CN2 Induction Algorithm,”
*Machine Learning, 3, 4*, pp. 261–283, 1989.Google Scholar - 10).Cohen, W. W., “Abductive Explanation-based Learning: A Solution to the Multiple Inconsistent Explanation Problem,”
*Machine Learning, 8*, pp. 167–219, 1992.zbMATHGoogle Scholar - 11).Cohen, W. W., “Pac-learning a Restricted Class of Recursive Logic Programs,” in
*Proc. of the 11th National Conference on Artificial Intelligence*, pp. 86–92, Washington, D.C., 1993.Google Scholar - 12).De Raedt, L.,
*Interactive Theory Revision: An Inductive Logic Programming Approach*, Academic Press, 1992.Google Scholar - 13).De Raedt, L., and Bruynooghe, M., “Belief Updating from Integrity Constraints and Queries,”
*Artificial Intelligence, 53*, pp. 291–307, 1992.CrossRefGoogle Scholar - 14).De Raedt, L., and Bruynooghe, M., “A theory of Clausal Discovery,” in
*Proc. of the 13th International Joint Conference on Artificial Intelligence*, 1993.Google Scholar - 15).De Raedt, L., and Dehaspe, L., “Learning from Satisfiability,”
*Technical report*, Katholieke Universiteit Leuven, 1996.Google Scholar - 16).De Raedt, L., and Džeroski, S., “First Order
*jk*-clausal Theories Are PAC-learnable”*Artificial Intelligence, 70*, pp. 375–392, 1994.zbMATHCrossRefMathSciNetGoogle Scholar - 17).De Raedt, L., Lavrač, N., and Džeroski, S., “Multiple Predicate Learning,” in Muggleton, S., ed.,
*Proc. of the 3rd International Workshop on Inductive Logic Programming*, pp. 221–240. J. Stefan Institute, 1993.Google Scholar - 18).De Raedt, L., and Lear, W. V., “Inductive Constraint Logic,” in
*Proc. of the 5th International Workshop on Algorithmic Learning Theory*, 1995.Google Scholar - 19).Denecker, M., and Schreye, D. D., “SLDNFA: an Abductive Procedure for Normal Abductive Programs,” in Apt, K. R., ed., in
*Proc. of the International Joint Conference and Symposium on Logic Programming*, pp. 686–700, 1992.Google Scholar - 20).Dimopoulos, Y., Džeroski, S., and Kakas, A. C., “Integrating Explanatory and Descriptive Learning in ILP,” in
*Proc. of the 15th International Joint Conference on Artificial Intelligence*, 1997.Google Scholar - 21).Dimopoulos, Y., and Kakas, A. C., “Learning Non-monotonic Logic Programs: Learning Exceptions,” in
*Proc. of the 8th European Conference on Machine Learning*, 1995.Google Scholar - 22).Dimopoulos, Y., and Kakas, A. C., “Abduction and Inductive Learning,” in
*Advances in Inductive Logic Programming*, IOS Press, 1996.Google Scholar - 23).Duval, B., Abduction for Explanation Based Learning,” in
*Proc. of the European Working Session on Learning*, 482 in LNCS, pp. 348–360, 1991.Google Scholar - 24).Džeroski, S., and Bratko, I., “Handling Noise in Inductive Logic Programming,” Muggleton, S., ed.,
*Proc. of the 2nd International Workshop on Inductive Logic Programming*, Report ICOT TM-1182, 1992.Google Scholar - 25).Eshghi, K., and Kowalski, R. A., “Abduction Compared with Negation by Failure,” in
*Proc. of the 6th International Conference on Logic Programming*, 1989.Google Scholar - 26).Esposito, F., Lamma, E., Malerba, D., Mello, P., Milano, M., Riguzzi, F., and Semeraro, G., “Learning Abductive Logic Programs,” in Flach and Kakas. 1996. Available on-line at http://www.cs.bris.ac.uk/~flach/ECAI96/.Google Scholar
- 27).Flach, P. A.,
*Conjectures: An Inquiry Concerning the Logic of Induction*, PhD Thesis, Katholieke Universiteit Brabant, 1995.Google Scholar - 28).Flach, P. A., and Kakas, A. C., eds., in
*Proc. of the ECAI’96 Workshop on Abductive and Inductive Reasoning*, 1996. Available on-line at http://www.cs.bris.ac.uk/~flach/ECAI96/.Google Scholar - 29).Flach, P. A., and Kakas, A. C., eds.,
*Proc. of the IJCAI’97 Workshop on Abductive and Inductive Reasoning*, 1997. Available on-line at http://www.cs.bris.ac.uk/~flach/IJCAI97/.Google Scholar - 30).Flach, P. A., and Kakas, A. C., eds., “Abductive and Inductive Reasoning,”
*Pure and Applied Logic*, Kluwer, 1998.Google Scholar - 31).Flener, P., “Inductive Logic Program Synthesis with Dialogs,” in Muggleton, S., ed.,
*Inductive Logic Programming: Selected Papers from the 6th International Workshop*, pp. 175–198. Springer-Verlag, 1997.Google Scholar - 32).Giordano, L., and Martelli, A., “Generalized Stable Model Semantics, Truth Maintenance and Conflict Resolution”, in
*Proc. of the 7th International Conf. on Logic Programming*, pp. 427–411, Jerusalem, 1990. MIT Press.Google Scholar - 33).Helft, N., “Induction as Nonmonotonic Inference,” in
*Proc. of the 1st International Conference on Principles of Knowledge Representation and Reasoning*, pp. 149–156. Morgan Kaufmann, 1989.Google Scholar - 34).Inoue, K., “Hypothetical Reasoning in Logic Programs,”
*Journal of Logic Programming, 18*, pp. 191–227, 1994.zbMATHCrossRefMathSciNetGoogle Scholar - 35).Inoue, K., and Haneda, H., “Learning Abductive and Nonmonotonic Logic Programsg,” in Flach, P. A., and Kakas, A. C., eds.,
*Abductive and Inductive Reasoning*, Pure and Applied Logic. Kluwer, 1999.Google Scholar - 36).Inoue, K., and Sakama, C., “On the Equivalence between Disjunctive and Abductive Logic Programs,” in
*Proc. of ICLP94*, pp. 489–503, 1994.Google Scholar - 37).Inoue, K., and Sakama, C., “Abductive Framework for Nonmonotonic Theory Change,” in
*Proc. of the 14th International Joint Conference on Artificial Intelligence*, pp. 204–210, 1995.Google Scholar - 38).Inuzuka, N., Kamo, M., Ishii, N., Seki, H., and Itoh, H., “Top-down Induction of Logic Programs from Incomplete Samples,” in Muggleton, S., ed.,
*Proc. of the 6th International Workshop on Inductive Logic Programming*, pp. 119–136. Stockholm University, Royal Institute of Technology, 1996.Google Scholar - 39).Jorge, A., and Brazdil, P., “Architecture for Iterative Learning of Recursive Definitions,” in De Raedt, L., ed.,
*Advances in Inductive Logic Programming*, pp. 206–218. IOS Press, 1996.Google Scholar - 40).Kakas, A. C., Kowalski, R. A., and Toni, F., “Abductive Logic Programming,”
*Journal of Logic and Computation, 2*, pp. 719–770, 1993.CrossRefMathSciNetGoogle Scholar - 41).Kakas, A. C., Kowalski, R. A., and Toni, F., “The Role of Abduction in Logic Programming,” in Gabbay, D., Hogger, C., and Robinson, J., eds.,
*Handbook of Logic in AI and Logic Programming, 5*, pp 233–306. Oxford University Press, 1997.Google Scholar - 42).Kakas, A. C., and Mancarella, P., “Database Updates through Abduction,” in Davis, R. S., McLeod, D., and Schek, H., eds.,
*Proc. of the 16th International Conference on Very Large Databases, VLDB-90*, pp. 650–661. Morgan Kaufmann, 1990.Google Scholar - 43).Kakas, A. C., and Mancarella, P., “Generalized Stable Models: a Semantics for Abduction,” in
*Proc. of the 9th European Conference on Artificial Intelligence*, 1990.Google Scholar - 44).Kakas, A. C., and Mancarella, P., “On the Relation between Truth Maintenance and Abduction,” in
*Proc. of the 2nd Pacific Rim International Conference on Artificial Intelligence*, 1990.Google Scholar - 45).Kakas, A. C., and Riguzzi, F., “Learning with Abduction,” in
*Proc. of the 7th International Workshop on ILP*, 1997.Google Scholar - 46).Kalbfleish, J.,
*Probability and Statistical Inference, II*, Springer, New York, 1979.Google Scholar - 47).Kanai, T., and Kunifuji, S., “Extending Inductive Generalization with Abduction,” Flach and Kakas, Kluwer, 1998.Google Scholar
- 48).Khan, K., Muggleton, S., and Parson, R., “Repeat Learning Using Predicate Invention,” in
*Proc. of the 8th International Workshop on Inductive Logic Programming*, 1998.Google Scholar - 49).Kijsirikul, B., Numao, M., and Shimura, M., “Discrimination-based Constructive Induction of Logic Programs,” in
*Proc. of the 10th National Conference on Artificial Intelligence*, Morgan Kaufmann, 1992.Google Scholar - 50).Lamma, E., Mello, P., Milano, M., and Riguzzi, F., “Integrating Induction and Abduction in Logic Programming,” Wang, P. P., ed.,
*Proc. of the Third Joint Conference on Information Sciences, 2*, pp 203–206, 1997.Google Scholar - 51).Lamma, E., Mello, P., Milano, M., and Riguzzi, F., “Integrating Induction and Abduction in Logic Programming,”
*Information Sciences*, 1998.Google Scholar - 52).Lavrač, N., and Džeroski, S.,
*Inductive Logic Programming: Techniques and Applications*, Ellis Horwood, 1994.Google Scholar - 53).Lavrač, N., Džeroski, S., and Bratko, I., “Handling Imperfect Data in Inductive Logic Programming,” De Raedt, L., ed.,
*Advances in Inductive Logic Programming*, pp. 48–64. IOS Press, 1996.Google Scholar - 54).Lavrač, N., Džeroski, S., and Grobelnik, M., “Learning Nonrecursive Definitions of Relations with LINUS,” in Kodratoff, Y., ed.,
*Proc. of the 5th European Working Session on Learning, Lecture Notes in Artificial Intelligence, 482*, pp. 265–281. Springer-Verlag, 1991.Google Scholar - 55).Lloyd, J.,
*Foundations of Logic Programming*, Springer Verlag, Berlin, second ed., 1987.zbMATHGoogle Scholar - 56).Mooney, R., “Integrating Abduction and Induction in Machine Learning,” Flach and Kakas. Kluwer, 1998.Google Scholar
- 57).Muggleton, S., “Inductive Logic Programming,”
*New Generation Computing, 8, 4*, pp. 295–317, 1991.zbMATHCrossRefGoogle Scholar - 58).Muggleton, S., “Inverse Entailment and Progol,”
*New Generation Computing, Special issue on Inductive Logic Programming, 13, 3–4*, pp. 245–286, 1995.Google Scholar - 59).Muggleton, S., and De Raedt, L., “Inductive Logic Programming: Theory and methods,”
*Journal of Logic Programming, 19, 20*, pp. 629–679, 1994.CrossRefMathSciNetGoogle Scholar - 60).O’Rourke, P., “Abduction and Explanation-based Learning: Case Studies in Diverse Domains,”
*Computational Intelligence, 10*, pp. 295–330, 1994.CrossRefGoogle Scholar - 61).Plotkin, G. D., “A Note on Inductive Generalization,” in
*Machine Intelligence, 5*, pp. 153–163. Edinburgh University Press, 1970.Google Scholar - 62).Plotkin, G. D., “A Further Note on Inductive Generalization,” in
*Machine Intelligence, 6*, pp. 101–124. Edinburgh University Press, 1971.Google Scholar - 63).Poole, D., Goebel, R. G., and Aleliunas, “Theorist: a Logical Reasoning System for Default and Diagnosis,” Cercone and McCalla, eds.,
*The Knowledge Fronteer: Essays in the Representation of Knowledge*, Lecture Notes in Computer Science, pp. 331–352. Springer-Verlag, 1987.Google Scholar - 64).Quinlan, J. R., “Learning Logical Definitions from Relations,”
*Machine Learning, 5*, pp. 239–266, 1990.Google Scholar - 65).Quinlan, J. R., “Unknown Attribute Values in Induction,” in
*Proc. of the Sixth International Machine Learning Workshop*, pp. 164–168, Morgan Kaufmann, San Mateo, CA, 1991.Google Scholar - 66).Quinlan, J. R.,
*C4.5: Programs for Machine Learning*, Morgan Kaufmann, San Mateo, CA, 1993.Google Scholar - 67).Richards, B. L., and Mooney, R. J., “Refinement of first-order Horn-clause Domain Theories,”
*Machine Learning, 19, 2*, pp. 95–131, 1995.Google Scholar - 68).Sakama, C., “Computing Induction through Abduction,” Flach and Kakas, Kluwer, 1998.Google Scholar
- 69).Satoh, K., and Iwayama, N., “A Query Evaluation Method for Abductive Logic Programming,”
*Proc. of the 1992 Joint International Conference and Symposium on Logic Programming*, pp. 671–685, 1992.Google Scholar - 70).Stahl, I., “Predicate Invention in Inductive Logic Programming,” De Raedt, L., ed.,
*Advances in Inductive Logic Programming*, pp. 34–47. IOS Press, 1996.Google Scholar - 71).Thompson, C., and Mooney, R., “Inductive Learning for Abductive Diagnosis,” in
*Proc. of the 12th National Conference on Artificial Intelligence*, 1994.Google Scholar - 72).Wirth, R., and O’Rorke, P., “Constraints on Predicate Invention,” in Birnbaum, L., and Collins, G., eds.,
*Proc. of the 8th International Workshop on Machine Learning*, pp. 457–461. Morgan Kaufmann, 1991.Google Scholar - 73).Wrobel, S., and Džeroski, S., “The ILP Description Learning Problem: Towards a Genearl Model-leve Definition of Data Mining in ILP,” in
*Proc. of the Fachgruppentreffen Maschinelles Lernen*, 1995.Google Scholar - 74).Zelle, J. M., Mooney, R. J., and Konvisser, J. B., “Combining Top-down and Bottom-up Techniques in Inductive Logic Programming,” Cohen, W. W., and Hirsh, H., eds., in
*Proc. of the 11th International Conference on Machine Learning*, pp. 343–351. Morgan Kaufmann, 1994.Google Scholar