Abstract
This paper presents a method for approximate match of first-order rules with unseen data. The method is useful especially in case of a multi-class problem or a noisy domain where unseen data are often not covered by the rules. Our method employs the Backpropagation Neural Network for the approximation. To build the network, we propose a technique for generating features from the rules to be used as inputs to the network. Our method has been evaluated on four domains of first-order learning problems. The experimental results show improvements of our method over the use of the original rules. We also applied our method to approximate match of propositional rules converted from an unpruned decision tree. In this case, our method can be thought of as soft-pruning of the decision tree. The results on multi-class learning domains in the UCI repository of machine learning databases show that our method performs better than standard C4.5's pruned and unpruned trees.
Article PDF
Similar content being viewed by others
References
Baroglio, C., & Botta, M. (1995). Multiple predicate learning with RTL. In Proceedings of the Fourth Congress of the Italian Association for Artificial Intelligence, (AI*IA95) (pp. 44–55). Florence, Italy.
Bergadano, F., & Gunetti, D. (1995). Inductive logic programming:Frommachine learning to software engineering. Cambridge, MA: The MIT Press.
Blockeel, H., & Raedt, L. D. (1997). Experiments with top-down induction of logical decision trees. Technical Report CW247, Department of Computer Science, K.U. Leuven.
Botta, M., Giordana, A., & Piola, R. (1997). FONN: Combining first order logic with connectionist learning. In Proceedings of the Fourteenth International Conference on Machine Learning (pp. 48–56). San Francisco, CA: Morgan Kaufmann.
Brazdil, P., & Jorge, A. (1993). Exploiting algorithm sketches in ILP. In Proceedings of the Third International Workshop on Inductive Logic Programming. Ljubljana, Slovenia: Jozef Stefan Institute.
Clark, P., & Niblett, T. (1989). The CN2 induction algorithm. Machine Learning, 3(4), 261–283.
Dolsak, B., Jezernik, A., & Bratko, I. (1994). A knowledge base for finite element mesh design. Artificial Intelligence in Engineering, 9, 19–27.
Dolsak, B., & Muggleton, S. (1992). The application of inductive logic programming to finite element mesh design. In S. Muggleton (Ed.), Inductive logic programming. London: Academic Press.
Džeroski, S., Schulze-Kremer, S., Heidtke, K. R., Siems, K., & Wettschereck, D. (1996). Applying ILP to diterpene structure elucidation from 13C NMR spectra. In Proceedings of the Sixth International Workshop on Inductive Logic Programming (pp. 14–27). Lecture notes in artificial intelligence (Vol. 1314). Berlin: Springer-Verlag.
Flach, P., & Lachiche, N. (1999). 1BC: A first-order Bayesian classifier. In Proceedings of the Ninth International Workshop on Inductive Logic Programming (pp. 92–103). Lecture notes in artificial intelligence (Vol. 1634). Berlin: Springer-Verlag.
Fürnkranz, J. (1994). Fossil: A robust relational learner. In Proceedings of the Seventh European Conference on Machine Learning, ECML-94 (pp. 122–137). Berlin: Springer-Verlag.
Kietz, J. U., & Wrobel, S. (1992). Controlling the complexity of learning in logic through syntactic and task-oriented models. In S. Muggleton (Ed.), Inductive logic programming. London: Academic Press.
Kijsirikul, B., & Sinthupinyo, S. (1999). Approximate ILP rules by backpropagation neural network: A result on Thai character recognition. In Proceedings of the Ninth International Workshop on Inductive Logic Programming (pp. 162–173). Berlin: Springer-Verlag.
Lavrač, N., & Džeroski, S. (1994). Inductive logic programming: Techniques and applications. New York: Ellis Horwood.
Lavrač, N., Gamberger, D., & Jovanoski, V. (1999). A study of relevance for learning in deductive databases. Journal of Logic Programming, 40, 215–249.
Mahoney, J., & Mooney, R. (1994). Comparing methods for refining certainty-factor rule-bases. In Proceedings of the Eleventh International Workshop on Machine Learning, (pp. 173–180). New Brunswick, NJ.
Martin, L., & Vrain, C. (1995). MULT ICN: An empirical multiple predicate learner. In Proceedings of the Fifth International Workshop on Inductive Logic Programming (pp. 129–144). Department of Computer Science, Katholieke, Universiteit Leuven.
Merz, C. J., Murphy, P. M., & Aha, D.W. (1997). UCI repository of machine learning databases. Department of Information and Computer Science, University of California, Irvine, CA. http://www.ics.uci.edu/~mlearn/ MLRepository.html.
Muggleton, S. (1991). Inductive logic programming. New Generation Computing, 8(4), 295–318.
Muggleton, S. (1995). Inverse entailment and PROGOL. New Generation Computing, 13, 245–286.
Muggleton, S., Bain, M., Hayes-Michie, J., & Michie, D. (1989). An experimental comparison of human and machine learning algorithms. In Proceedings of the Sixth International Workshop on Machine Learning (pp. 113–118). Los Altos, CA: Morgan Kaufmann.
Muggleton, S., & Feng, C. (1990). Efficient induction of logic programs. In Proceedings of the First Conference on Algorithmic Learning Theory (pp. 368–381). Tokyo: Ohmsha.
Muggleton, S., & Raedt, L. D. (1994). Inductive logic programming: Theory and methods. Journal of Logic Programming, 19(20), 629–679.
Nilsson, N. J. (1980). Principles of artificial intelligence. Palo Alto, CA: Tioga.
Plotkin, G. D. (1970). A note on inductive generalization. In B. Meltzer & D. Michie (Eds.), Machine intelligence (Vol. 5). New York: Elsevier North-Holland.
Quinlan, J. R. (1990). Learning logical definitions from relations. Machine Learning, 5(3), 239–266.
Quinlan, J. R. (1993). C4.5:Programs for machine learning. San Mateo, CA: Morgan Kaufmann.
Raedt, L. D., & Laer, W. V. (1995). Inductive constraint logic. In Proceedings of the Fifth Workshop on Algorithmic Learning Theory (pp. 80–94). Lecture notes in artificial intelligence (Vol. 997). Berlin: Springer-Verlag.
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986). Learning internal representations by error propagation. In D. E. Rumelhart & J. L. McClelland (Eds.), Parallel distributed processing (Vol. 1). Cambridge, MA: MIT Press.
Srinivasan, A., Muggleton, S., Sternberg, M. J. E., & King, R. D. (1996). Theories for mutagenicity: A study in first-order and feature-based induction. Artificial Intelligence, 85, 277–299.
Tausend, B. (1994). Representing biases for inductive logic programming. In Proceedings of European Conferences on Machine Learning, ECML-94 (pp. 427–430). Berlin: Springer-Verlag.
Towell, G. G., & Shavlik, J.W. (1994). Knowledge-based artificial neural networks. Artificial Intelligence, 70(4), 119–116.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Kijsirikul, B., Sinthupinyo, S. & Chongkasemwongse, K. Approximate Match of Rules Using Backpropagation Neural Networks. Machine Learning 44, 273–299 (2001). https://doi.org/10.1023/A:1010976022223
Issue Date:
DOI: https://doi.org/10.1023/A:1010976022223