Margin-based first-order rule learning
We present a new margin-based approach to first-order rule learning. The approach addresses many of the prominent challenges in first-order rule learning, such as the computational complexity of optimization and capacity control. Optimizing the mean of the margin minus its variance, we obtain an algorithm linear in the number of examples and a handle for capacity control based on error bounds. A useful parameter in the optimization problem tunes how evenly the weights are spread among the rules. Moreover, the search strategy for including new rules can be adjusted flexibly, to perform variants of propositionalization or relational learning. The implementation of the system includes plugins for logical queries, graphs and mathematical terms. In extensive experiments, we found that, at least on the most commonly used toxicological datasets, overfitting is hardly an issue. In another batch of experiments, a comparison with margin-based ILP approaches using kernels turns out to be favorable. Finally, an experiment shows how many features are needed by propositionalization and relational learning approaches to reach a certain predictive performance.
- Ben-David, S., Eiron, N., Long, P. M. (2003) On the difficulty of approximately maximizing agreements. Journal of Computer and System Sciences 66: pp. 496-514 CrossRef
- Cohen, W., Singer, Y. (1999) A simple, fast, and effective rule learner. Proceedings of the sixteenth national conference on artificial intelligence (AAAI-99). AAAI Press, Menlo Park, pp. 335-342
- Raedt, L. D. (1997) Logical settings for concept-learning. Artificial Intelligence 95: pp. 187-201 CrossRef
- Dias, A. M. (2006). CxProlog. http://ctp.di.fct.unl.pt/~amd/cxprolog/.
- Friedman, J. H., & Popescu, B. E. (2005). Predictive learning via rule ensembles (Technical report). Stanford University.
- Hoeffding, W. (1963) Probability inequalities for sums of bounded random variables. Journal of the American Statistical Association 58: pp. 13-30 CrossRef
- Kearns, M. J., Vazirani, U. V. (1994) An introduction to computational learning theory. MIT Press, Cambridge
- King, R., Srinivasan, A. (1995) Relating chemical activity to structure: an examination of ILP successes. New Generation Computing, Special issue on Inductive Logic Programming 13: pp. 411-434
- Kramer, S., Lavrac, N., Flach, P. Propositionalization approaches to relational data mining. In: Dzeroski, S., Lavrac, N. eds. (2001) Relational Data Mining. Springer, Berlin, pp. 262-291
- Landwehr, N., Passerini, A., Raedt, L., Frasconi, P. (2006) kFOIL: Learning simple relational kernels. Proceedings of the twenty-first national conference on artificial intelligence and the eighteenth innovative applications of artificial intelligence conference. AAAI Press, Menlo Park
- Li, H., Yap, C. W., Ung, C. Y., Xue, Y., Cao, Z. W., Chen, Y. Z. (2005) Effect of selection of molecular descriptors on the prediction of blood-brain barrier penetrating and nonpenetrating agents by statistical learning methods. Journal of Chemical Information and Modeling 45: pp. 1376-1384 CrossRef
- McDiarmid, C. (1989) On the method of bounded differences. Surveys in combinatorics. Cambridge Univ. Press, Cambridge, pp. 148-188
- Muggleton, S., Lodhi, H., Amini, A., Sternberg, M. J. E. Support vector inductive logic programming. In: Hoffmann, A. G., Motoda, H., Scheffer, T. eds. (2005) Discovery science. Springer, New York, pp. 163-175 CrossRef
- Popescul, A., & Ungar, L. (2003). Statistical relational learning for link prediction. In IJCAI workshop on learning statistical models from relational data.
- Rückert, U., Kramer, S. Frequent free tree discovery in graph data. In: Haddad, H., Omicini, A. eds. (2004) Proceedings of the ACM symposium on applied computing. ACM, New York, pp. 564-570
- Rückert, U., Kramer, S. (2006) A statistical approach to rule learning. Machine learning, proceedings of the twenty-third international conference (ICML 2006). ACM, New York, pp. 785-792
- Srinivasan, A., Muggleton, S., Sternberg, M. J. E., King, R. D. (1996) Theories for mutagenicity: A study in first-order and feature-based induction. Artificial Intelligence 85: pp. 277-299 CrossRef
- Woźnica, A., Kalousis, A., Hilario, M. Kernels over relational algebra structures. In: Ho, T. B., Cheung, D., Liu, H. eds. (2005) PAKDD. Springer, Berlin, pp. 588-598
- Yoshida, F., Topliss, J. (2000) QSAR model for drug human oral bioavailability. Journal of Medicinal Chemistry 43: pp. 2575-2585 CrossRef
- Margin-based first-order rule learning
Volume 70, Issue 2-3 , pp 189-206
- Cover Date
- Print ISSN
- Online ISSN
- Springer US
- Additional Links
- First-order learning
- Relational learning
- Rule learning
- Capacity control
- Industry Sectors