Abstract
Parametric models such as linear regression can provide useful, interpretable descriptions of simple structure in data. However, sometimes such simple structure does not extend across an entire data set and may instead be confined more locally within subsets of the data. Nonparametric regression typically involves local averaging. In this study, local averaging estimator is coupled with a machine learning technique – boosting. In more detail, we propose a technique of local boosting of decision stumps. We performed a comparison with other well known methods and ensembles, on standard benchmark datasets and the performance of the proposed technique was greater in most cases.
The Project is Co-Funded by the European Social Fund & National Resources – EPEAEK II.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Aha, D.: Lazy Learning. Kluwer Academic Publishers, Dordrecht (1997)
Atkeson, C.G., Moore, A.W., Schaal, S.: Locally weighted learning for control. Artificial Intelligence Review 11, 75–113 (1997)
Atkeson, C.G., Moore, A.W., Schaal, S.: Locally weighted learning. Artificial Intelligence Review 11, 11–73 (1997)
Blake, C., Merz, C.: UCI Repository of machine learning databases. University of California, Department of Information and Computer Science, Irvine, CA (1998)
Dietterich, T.G.: Ensemble Methods in Machine Learning. In: Kittler, J., Roli, F. (eds.) MCS 2000. LNCS, vol. 1857, pp. 1–10. Springer, Heidelberg (2000)
Brighton, H., Mellish, C.: Advances in Instance Selection for Instance-Based Learning Algorithms. Data Mining and Knowledge Discovery 6, 153–172 (2002)
Bottou, L., Vapnik, V.: Local learning algorithm. Neural Computation 4(6), 888–901 (1992)
Breiman, L.: Bagging Predictors. Machine Learning 24, 123–140 (1996)
Cohen, S., Intrator, N.: Automatic model selection in a hybrid perceptron/Radial network. In: Kittler, J., Roli, F. (eds.) MCS 2001. LNCS, vol. 2096, pp. 349–358. Springer, Heidelberg (2001)
Duffy, N., Helmbold, D.: Boosting Methods for Regression. Machine Learning 47, 153–200 (2002)
John, C., Trigg, L.: K*: An Instance- based Learner Using an Entropic Distance Measure. In: Proc. of the 12th International Conference on ML, pp. 108–114 (1995)
Frank, E., Hall, M., Pfahringer, B.: Locally weighted naive Bayes. In: Proc. of the 19th Conference on Uncertainty in Artificial Intelligence, Acapulco, Mexico, pp. 249–256. Morgan Kaufmann, San Francisco (2003)
Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences 55(1), 119–139 (1997)
Friedman, J.: Stochastic Gradient Boosting. Computational Statistics and Data Analysis 38, 367–378 (2002)
Iba, W., Langley, P.: Induction of one-level decision trees. In: Proc. of the Ninth International Machine Learning Conference, pp. 233–240. Morgan Kaufmann, Aberdeen, Scotland (1992)
Kleinberg, E.M.: A mathematically rigorous foundation for supervised learning. In: Kittler, J., Roli, F. (eds.) MCS 2000. LNCS, vol. 1857, pp. 67–76. Springer, Heidelberg (2000)
Kohavi, R.: The Power of Decision Tables. In: Proc European Conference on Machine Learning, pp. 174–189 (1995)
Loader, C.: Local Regression and Likelihood. Springer, New York (1999)
Nadeau, C., Bengio, Y.: Inference for the Generalization Error. Machine Learning 52, 239–281 (2003)
Vapnik, V.N.: Statistical Learning Theory, ch. 2 & 3. Wiley, New York (1998)
Wilson, D., Martinez, T.: Reduction Techniques for Instance-Based Learning Algorithms. Machine Learning 38, 257–286 (2000)
Witten, I., Frank, E.: Data Mining: Practical Machine Learning Tools and Techniques with Java Implementations. Morgan Kaufmann, San Mateo (2000)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Kotsiantis, S.B., Kanellopoulos, D., Pintelas, P.E. (2006). Local Additive Regression of Decision Stumps . In: Antoniou, G., Potamias, G., Spyropoulos, C., Plexousakis, D. (eds) Advances in Artificial Intelligence. SETN 2006. Lecture Notes in Computer Science(), vol 3955. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11752912_17
Download citation
DOI: https://doi.org/10.1007/11752912_17
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-34117-8
Online ISBN: 978-3-540-34118-5
eBook Packages: Computer ScienceComputer Science (R0)