Mining Tolerance Regions with Model Trees
Many problems encountered in practice involve the prediction of a continuous attribute associated with an example. This problem, known as regression, requires that samples of past experience with known continuous answers are examined and generalized in a regression model to be used in predicting future examples. Regression algorithms deeply investigated in statistics, machine learning and data mining usually lack measures to give an indication of how “good” the predictions are. Tolerance regions, i.e., a range of possible predictive values, can provide a measure of reliability for every bare prediction. In this paper, we focus on tree-based prediction models, i.e., model trees, and resort to the inductive inference to output tolerance regions in addition to bare prediction. In particular, we consider model trees mined by SMOTI (Stepwise Model Tree Induction) that is a system for data-driven stepwise construction of model trees with regression and splitting nodes and we extend the definition of trees to build tolerance regions to be associated with each leaf. Experiments evaluate validity and quality of output tolerance regions.
KeywordsModel Tree Leaf Node Regression Tree Inductive Inference Splitting Node
Unable to display preview. Download preview PDF.
- 1.Breiman, L., Friedman, J., Olshen, R., Stone, J.: Classification and regression tree. Wadsworth & Brooks (1984)Google Scholar
- 3.Cowan, G.: Statistical Data Analysis. Oxford University Press, USA (1998)Google Scholar
- 4.Dobra, A., Gehrke, J.E.: SECRET: A scalable linear regression tree algorithm. In: Proceedings of the Eighth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Edmonton, Alberta, Canada (2002)Google Scholar
- 5.Draper, N.R., Smith, H.: Applied regression analysis. John Wiley, Chichester (1982)Google Scholar
- 6.Fraser, D.A.S.: Non-parametric Methods in Statistics. Wiley, New York (1957)Google Scholar
- 7.Gammerman, A., Vovk, V., Vapnik, V.: Learning by transduction. In: 14th Conference on Uncertainty in Artificial Intelligence, pp. 148–155 (1998)Google Scholar
- 10.Karalic, A.: Linear regression in regression tree leaves. In: Proceedings of Int. School for Synthesis of Expert Knowledge, ISSEK 1992, pp. 151–163 (1992)Google Scholar
- 15.Proedrou, K., Nouretdinov, I., Vovk, V., Gammerman, A.: Transductive confidence machines for pattern recognition (2001)Google Scholar
- 16.Quinlan, J.R.: Learning with continuous classes. In: Adams, Sterling (eds.) Proceedings of the 5th Australian Joint Conference on Artificial Intelligence, pp. 343–348. World Scientific, Singapore (1992)Google Scholar
- 17.Torgo, L.: Functional models for regression tree leaves. In: Fisher, D. (ed.) Proceedings of the Fourteenth International Conference on Machine Learning, ICML 1997, Nashville, Tennessee, pp. 385–393 (1997)Google Scholar
- 18.Torgo, L.: Inductive Learning of Tree-based Regression Models. PhD thesis, Department of Computer Science, University of Porto, Porto, Portugal (1999)Google Scholar
- 19.Wang, Y., Witten, I.H.: Inducing model trees for continuous classes. In: van Someren, M., Widmer, G. (eds.) ECML 1997. LNCS, vol. 1224, pp. 128–137. Springer, Heidelberg (1997)Google Scholar