Abstract
Boosting is a powerful and thoroughly investigated learning technique that improves the accuracy of any given learning algorithm by weighting training examples and hypotheses. Several authors contributed to the general boosting learning framework with theoretical and experimental results, mainly in the propositional learning framework. In a previous paper, we investigated the applicability of Freund and Schapire’s AdaBoost.M1 algorithm to a first order logic weak learner. In this paper, we extend the weak learner in order to directly deal with weighted instances and compare two ways to apply boosting to such a weak learner: resampling instances at each round and using weighted instances.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
E. L. Allwein, R. E. Schapire and Y. Singer (2000). Reducing multiclass to binary: A unifying approach for margin classifiers. Journal of Machine Learning Research, 1, 113–141.
C. Anglano, A. Giordana, G. Lo Bello and L. Saitta (1998). An Experimental Evaluation of Coevolutive Concept Learning. Proc. 15th Int. Conf. on Machine Learning (Madison, WI, 1998), pp. 19–27.
F. Bergadano, A. Giordana and L. Saitta (1988). Learning Concepts in Noisy Environment. IEEE Transaction on Pattern Analysis and Machine Intelligence, PAMI-10, 555–578.
F. Bergadano, A. Giordana and L. Saitta (1991). Machine Learning: An Integrated Framework and its Applications. Hellis Horwood, Chichester, UK.
M. Botta and A. Giordana (1993). Smart+: A MultiStrategy Learning Tool. Proc. of the 13th Int. Joint Conf. on Artificial Intelligence, (Chambery, France, 1993), pp. 937–943.
M. Botta and R. Piola (2000). Refining Numerical Constants in Structured First Order Logic Theories. Machine Learning Journal, 38, 109–131.
M. Botta (2001). WIL: a First Order Logic Weak Learner for Boosting. Technical Report RT 60/01, Dipartimento di Informatica, Università di Torino, <http://www.di.unito.it/~botta/rt60-01.pdf>.
L. Breiman, J. H. Friedman, R. A. Olshen and C. J. Stone (1984). Classification and Regression Trees. Wadsworth & Brooks.
L. Breiman (1996). Bagging Predictors. Machine Learning, 24, 123–140.
L. Breiman (1998). Arcing Classifiers. The Annals of Statistics, 26, 801–849.
L. Dehaspe, H. Toivonen and R. King (1998). Finding frequent substructures in chemical compounds. Proc. of the 4th Int. Conf. On Knowledge Disciovery and Data Mining (New York, NY, 1998), pp. 30–66.
R. DeMori, A. Giordana, P. Laface and L. Saitta (1984). An Expert System for Mapping Acoustic Cues into Phoenetic Features. Information Sciences, 33, 115–155.
T. G. Dietterich and G Bakiri (1995). Solving Multiclass Learning Problems via Error-Correcting Output Codes. Journal of AI Research, 2, 263–286.
F. Esposito, D. Malerba and G. Semeraro (1992). Classification in noisy environments using a distance measure between structural symbolic descriptions. IEEE Transactions on Pattern Analisys and Machine Intelligence, PAMI-14(3), 390–402.
F. Esposito, D. Malerba, G. Semeraro, and M. Pazzani (1993). A Machine Learning Approach to Document Understanding. Proc. of the 2nd International Workshop on Multistrategy Learning, (Harpers Ferry, WV, 1993), pp. 276–292.
Y. Freund and R. E. Schapire (1996). Experiments with a New Boosting Algorithm. Proc. of the 13th Int. Conf. on Machine Learning, (Bari, Italy, 1996), pp. 148–156.
Y. Freund and R. E. Schapire (1997). A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences, 55, 119–139.
A. Giordana and F. Neri (1996). Search-intensive Concept Induction. Evolutionary Computation, 3, 375–416.
A. Giordana, F. Neri, L. Saitta and M. Botta (1997). Integrating Multiple Learning Strategies in First Order Logics. Machine Learning, 27, 209–240.
A. Giordana, L. Saitta and M. Botta (1999). An Experimental Study of Phase Transitions in Matching. Proc. 16th Int. Joint Conf. on Artificial Intelligence, (Stockholm, Sweden, 1999), pp. 1198–1203.
R. King, A. Srinivasan and M. Stenberg (1995). Relating chemical activity to structure: an examination of ILP successes. New Generation Computing, 13.
R. S. Michalski (1983). A Theory and Methodology of Inductive Learning. Artificial Intelligence, 20, 111–161.
S. Muggleton (Ed.). (1992). Inductive Logic Programming. Academic Press, London. UK.
S. Muggleton (1995). Inverse Entailment and Progol. New Generation Computing, 13, 245–286.
J. R. Quinlan (1990). Learning Logical Definitions from Relations. Machine Learning, 5, 239–266.
J. R. Quinlan (1993). C4.5: Programs for Machine Learning. Morgan Kaufmann.
J. R. Quinlan (1996). Boosting First-Order Learning. LNAI, 1160, 143–155.
J. R. Quinlan (1996). Bagging, Boosting, and C4.5. Proc. of the 14th AAAI, (Portland, OR, 1996), pp. 725–730.
M. Sebag and C. Rouveirol (1997). Tractable Induction and Classification in First Order Logic via Stochastic Matching. Proc. of the 15th Int. Joint Conf. On Artificial Intelligence, (Nagoya, Japan, 1997), pp. 888–893.
G. Towell and J. Shavlik (1994). Knowledge Based Artificial Neural Networks. Artificial Intelligence, 70, 119–166.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2001 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Botta, M. (2001). Resampling vs Reweighting in Boosting a Relational Weak Learner. In: Esposito, F. (eds) AI*IA 2001: Advances in Artificial Intelligence. AI*IA 2001. Lecture Notes in Computer Science(), vol 2175. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-45411-X_9
Download citation
DOI: https://doi.org/10.1007/3-540-45411-X_9
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-42601-1
Online ISBN: 978-3-540-45411-3
eBook Packages: Springer Book Archive