Machine Learning

, Volume 32, Issue 1, pp 63–76 | Cite as

Using Model Trees for Classification

  • Eibe Frank
  • Yong Wang
  • Stuart Inglis
  • Geoffrey Holmes
  • Ian H. Witten

Abstract

Model trees, which are a type of decision tree with linear regression functions at the leaves, form the basis of a recent successful technique for predicting continuous numeric values. They can be applied to classification problems by employing a standard method of transforming a classification problem into a problem of function approximation. Surprisingly, using this simple transformation the model tree inducer M5′, based on Quinlan's M5, generates more accurate classifiers than the state-of-the-art decision tree learner C5.0, particularly when most of the attributes are numeric.

Model trees classification algorithms M5 C5.0 decision trees 

References

  1. Breiman, L., Friedman, J.H., Olshen, R.A. & Stone, C.J. (1984). Classification and regression trees. Belmont CA: Wadsworth.Google Scholar
  2. Devroye, L., Gyoerfi, L. & Lugosi, G. (1996). A probabilistic theory of pattern recognition. New York: Springer-Verlag.Google Scholar
  3. Dietterich, T.G. & Bakiri G. (1995). “Solving multiclass learning problems via error-correcting output codes,” Journal of AI research, 2, 263-286.Google Scholar
  4. Holte, R.C. (1993). “Very simple classification rules perform well on most commonly used datasets,” Machine Learning, 11, 63-91.CrossRefGoogle Scholar
  5. Merz, C.J. & Murphy, P.M. (1996). UCI Repository of machine learning data-bases[http://www. ics.uci.edu/~mlearn/MLRepository.html]. Irvine, CA: University of California, Department of Information and Computer Science.Google Scholar
  6. Quinlan, J.R. (1992). “Learning with continuous classes,” Proceedings Australian Joint Conference on Artificial Intelligence(pp. 343-348). World Scientific, Singapore.Google Scholar
  7. Quinlan, J.R. (1993). C4.5: Programs for machine learning. Morgan Kaufmann.Google Scholar
  8. Smyth, P., Gray, A. & Fayyad, U. (1995). “Retrofitting Decision tree classifiers using Kernel Density Estimation,” Proceedings International Conference on Machine Learning(pp. 506-514). San Francisco, CA: Morgan Kaufmann.Google Scholar
  9. Torgo, L. (1997). “Kernel Regression Trees,” Proceedings of the poster papers of the European Conference on Machine Learning. University of Economics, Faculty of Informatics and Statistics, Prague.Google Scholar
  10. Wang, Y. & Witten, I.H. (1997). “Induction of model trees for predicting continuous classes,” Proceedings of the poster papers of the European Conference on Machine Learning, University of Economics, Faculty of Informatics and Statistics, Prague.Google Scholar

Copyright information

© Kluwer Academic Publishers 1998

Authors and Affiliations

  • Eibe Frank
    • 1
  • Yong Wang
    • 1
  • Stuart Inglis
    • 1
  • Geoffrey Holmes
    • 1
  • Ian H. Witten
    • 1
  1. 1.Department of Computer ScienceUniversity of WaikatoHamiltonNew Zealand. E-mail

Personalised recommendations