Mining Tolerance Regions with Model Trees

  • Annalisa Appice
  • Michelangelo Ceci
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4203)


Many problems encountered in practice involve the prediction of a continuous attribute associated with an example. This problem, known as regression, requires that samples of past experience with known continuous answers are examined and generalized in a regression model to be used in predicting future examples. Regression algorithms deeply investigated in statistics, machine learning and data mining usually lack measures to give an indication of how “good” the predictions are. Tolerance regions, i.e., a range of possible predictive values, can provide a measure of reliability for every bare prediction. In this paper, we focus on tree-based prediction models, i.e., model trees, and resort to the inductive inference to output tolerance regions in addition to bare prediction. In particular, we consider model trees mined by SMOTI (Stepwise Model Tree Induction) that is a system for data-driven stepwise construction of model trees with regression and splitting nodes and we extend the definition of trees to build tolerance regions to be associated with each leaf. Experiments evaluate validity and quality of output tolerance regions.


Model Tree Leaf Node Regression Tree Inductive Inference Splitting Node 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Breiman, L., Friedman, J., Olshen, R., Stone, J.: Classification and regression tree. Wadsworth & Brooks (1984)Google Scholar
  2. 2.
    Chaudhuri, P., Huang, M.C., Loh, W.Y., Yao, R.: Piecewise-polynomial regression trees. Statistica Sinica 4, 143–167 (1994)MATHGoogle Scholar
  3. 3.
    Cowan, G.: Statistical Data Analysis. Oxford University Press, USA (1998)Google Scholar
  4. 4.
    Dobra, A., Gehrke, J.E.: SECRET: A scalable linear regression tree algorithm. In: Proceedings of the Eighth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Edmonton, Alberta, Canada (2002)Google Scholar
  5. 5.
    Draper, N.R., Smith, H.: Applied regression analysis. John Wiley, Chichester (1982)Google Scholar
  6. 6.
    Fraser, D.A.S.: Non-parametric Methods in Statistics. Wiley, New York (1957)Google Scholar
  7. 7.
    Gammerman, A., Vovk, V., Vapnik, V.: Learning by transduction. In: 14th Conference on Uncertainty in Artificial Intelligence, pp. 148–155 (1998)Google Scholar
  8. 8.
    Hahn, G.J., Meeker, W.Q.: Statistical Intervals: A Guide for Practitioners. Series in Probability and Statistics. John Wiley & Sons, Chichester (1991)MATHGoogle Scholar
  9. 9.
    Nouretdinov, V.V.I., Gammerman, A.: Transductive confidence machine is universal. In: Gavaldá, R., Jantke, K.P., Takimoto, E. (eds.) ALT 2003. LNCS (LNAI), vol. 2842, p. 283. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  10. 10.
    Karalic, A.: Linear regression in regression tree leaves. In: Proceedings of Int. School for Synthesis of Expert Knowledge, ISSEK 1992, pp. 151–163 (1992)Google Scholar
  11. 11.
    Loh, W.Y.: Regression trees with unbiased variable selection and interaction detection. Statistica Sinica 12, 361–386 (2002)MATHMathSciNetGoogle Scholar
  12. 12.
    Malerba, D., Esposito, F., Ceci, M., Appice, A.: Top down induction of model trees with regression and splitting nodes. IEEE Transactions on Pattern Analysis and Machine Intelligence 26(5), 612–625 (2004)CrossRefGoogle Scholar
  13. 13.
    Melluish, T., Saunders, C., Nouretdinov, I., Vovk, V.: Comparing the bayes and typicalness frameworks. In: Flach, P.A., De Raedt, L. (eds.) ECML 2001. LNCS (LNAI), vol. 2167, Springer, Heidelberg (2001)CrossRefGoogle Scholar
  14. 14.
    Papadopoulos, H., Proedrou, K., Vovk, V., Gammerman, A.: Inductive confidence machines for regression. In: Elomaa, T., Mannila, H., Toivonen, H. (eds.) ECML 2002. LNCS (LNAI), vol. 2430, pp. 345–356. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  15. 15.
    Proedrou, K., Nouretdinov, I., Vovk, V., Gammerman, A.: Transductive confidence machines for pattern recognition (2001)Google Scholar
  16. 16.
    Quinlan, J.R.: Learning with continuous classes. In: Adams, Sterling (eds.) Proceedings of the 5th Australian Joint Conference on Artificial Intelligence, pp. 343–348. World Scientific, Singapore (1992)Google Scholar
  17. 17.
    Torgo, L.: Functional models for regression tree leaves. In: Fisher, D. (ed.) Proceedings of the Fourteenth International Conference on Machine Learning, ICML 1997, Nashville, Tennessee, pp. 385–393 (1997)Google Scholar
  18. 18.
    Torgo, L.: Inductive Learning of Tree-based Regression Models. PhD thesis, Department of Computer Science, University of Porto, Porto, Portugal (1999)Google Scholar
  19. 19.
    Wang, Y., Witten, I.H.: Inducing model trees for continuous classes. In: van Someren, M., Widmer, G. (eds.) ECML 1997. LNCS, vol. 1224, pp. 128–137. Springer, Heidelberg (1997)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Annalisa Appice
    • 1
  • Michelangelo Ceci
    • 1
  1. 1.Dipartimento di InformaticaUniversità degli Studi di BariBariItaly

Personalised recommendations