Machine Learning

, Volume 59, Issue 1, pp 161–205

Logistic Model Trees

Article

DOI: 10.1007/s10994-005-0466-3

Cite this article as:
Landwehr, N., Hall, M. & Frank, E. Mach Learn (2005) 59: 161. doi:10.1007/s10994-005-0466-3

Abstract

Tree induction methods and linear models are popular techniques for supervised learning tasks, both for the prediction of nominal classes and numeric values. For predicting numeric quantities, there has been work on combining these two schemes into ‘model trees’, i.e. trees that contain linear regression functions at the leaves. In this paper, we present an algorithm that adapts this idea for classification problems, using logistic regression instead of linear regression. We use a stagewise fitting process to construct the logistic regression models that can select relevant attributes in the data in a natural way, and show how this approach can be used to build the logistic regression models at the leaves by incrementally refining those constructed at higher levels in the tree. We compare the performance of our algorithm to several other state-of-the-art learning schemes on 36 benchmark UCI datasets, and show that it produces accurate and compact classifiers.

Keywords

model treeslogistic regressionclassification
Download to read the full article text

Copyright information

© Springer Science + Business Media, Inc. 2005

Authors and Affiliations

  1. 1.Institute for Computer ScienceUniversity of FreiburgFreiburgGermany
  2. 2.Department of Computer ScienceUniversity of WaikatoHamiltonNew Zealand