Comparing Simplification Methods for Model Trees with Regression and Splitting Nodes

  • Michelangelo Ceci
  • Annalisa Appice
  • Donato Malerba
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2871)


In this paper we tackle the problem of simplifying tree-based regression models, called model trees, which are characterized by two types of internal nodes, namely regression nodes and splitting nodes. We propose two methods which are based on two distinct simplification operators, namely pruning and grafting. Theoretical properties of the methods are reported and the effect of the simplification on several data sets is empirically investigated. Results are in favor of simplified trees in most cases.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Ceci, M., Appice, A., Malerba, D.: Simplification Methods for Model Trees with Regression and Splitting Nodes. In: Perner, P., Rosenfeld, A. (eds.) MLDM 2003. LNCS, vol. 2734, Springer, Heidelberg (2003) (in press)Google Scholar
  2. 2.
    Cestnik, B., Bratko, I.: On estimating probabilities in tree pruning. In: Proc. of the Fifth European Working Session on Learning, pp. 151–163. Springer, Heidelberg (1991)Google Scholar
  3. 3.
    Draper, N.R., Smith, H.: Applied regression analysis. John Wiley & Sons, Chichester (1982)Google Scholar
  4. 4.
    Esposito, F., Malerba, D., Semeraro, G.: A comparative analysis of methods for pruning decision trees. IEEE Trans. PAMI 19(5), 476–491 (1997)Google Scholar
  5. 5.
    Karalic, A.: Linear regression in regression tree leaves. In: Proceedings of ISSEK 1992 (International School for Synthesis of Expert Knowledge), Bled, Slovenia (1992)Google Scholar
  6. 6.
    Lubinsky, D.: Tree Structured Interpretable Regression. In: Fisher, D., Lenz, H.J. (eds.) Learning from Data. Lecture Notes in Statistics, vol. 112, pp. 387–398. Springer, Heidelberg (1996)Google Scholar
  7. 7.
    Malerba, D., Appice, A., Ceci, M., Monopoli, M.: Trading-off Local versus Global Effects of Regression Nodes in Model Trees. In: Hacid, M.-S., Raś, Z.W., Zighed, D.A., Kodratoff, Y. (eds.) ISMIS 2002. LNCS (LNAI), vol. 2366, pp. 393–402. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  8. 8.
    Niblett, T., Bratko, I.: Learning decision rules in noisy domains. In: Bramer, M.A. (ed.) Research and Development in Expert Systems III, pp. 25–34. Cambridge University Press, Cambridge (1986)Google Scholar
  9. 9.
    Quinlan, J.R.: Simplifying decision trees. International Journal of Man-Machine Studies 27, 221–234 (1987)CrossRefGoogle Scholar
  10. 10.
    Quinlan, J.R.: Learning with continuous classes. In: Adams, Sterling (eds.) Proceedings AI 1992, pp. 343–348. World Scientific, Singapore (1992)Google Scholar
  11. 11.
    Robnik-Šikonja, M., Kononenko, I.: Pruning Regression Trees with MDL. In: Prade, H. (ed.) Proceedings of the 13th European Conference on Artificial Intelligence, pp. 455–459. John Wiley & Sons, Chichester (1998)Google Scholar
  12. 12.
    Torgo, L.: Functional Models for Regression Tree Leaves. In: Fisher, D. (ed.) Proceedings of the Fourteenth International Conference (ICML 1997), Nashville, Tennessee (1997)Google Scholar
  13. 13.
    Torgo, L.: Inductive Learning of Tree-based Regression Models, Ph.D. Thesis, Department of Computer Science, Faculty of Sciences, University of Porto (1999)Google Scholar
  14. 14.
    Wang, Y., Witten, I.H.: Inducing Model Trees for Continuous Classes. In: van Someren, M., Widmer, G. (eds.) ECML 1997. LNCS, vol. 1224, pp. 128–137. Springer, Heidelberg (1997)Google Scholar
  15. 15.
    Weisberg, S.: Applied regression analysis, 2nd edn. Wiley, New York (1985)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2003

Authors and Affiliations

  • Michelangelo Ceci
    • 1
  • Annalisa Appice
    • 1
  • Donato Malerba
    • 1
  1. 1.Dipartimento di InformaticaUniversità degli StudiBariItaly

Personalised recommendations