Machine Learning

, Volume 40, Issue 3, pp 203–228

A Comparison of Prediction Accuracy, Complexity, and Training Time of Thirty-Three Old and New Classification Algorithms

  • Tjen-Sien Lim
  • Wei-Yin Loh
  • Yu-Shan Shih

DOI: 10.1023/A:1007608224229

Cite this article as:
Lim, TS., Loh, WY. & Shih, YS. Machine Learning (2000) 40: 203. doi:10.1023/A:1007608224229


Twenty-two decision tree, nine statistical, and two neural network algorithms are compared on thirty-two datasets in terms of classification accuracy, training time, and (in the case of trees) number of leaves. Classification accuracy is measured by mean error rate and mean rank of error rate. Both criteria place a statistical, spline-based, algorithm called POLYCLSSS at the top, although it is not statistically significantly different from twenty other algorithms. Another statistical algorithm, logistic regression, is second with respect to the two accuracy criteria. The most accurate decision tree algorithm is QUEST with linear splits, which ranks fourth and fifth, respectively. Although spline-based statistical algorithms tend to have good accuracy, they also require relatively long training times. POLYCLASS, for example, is third last in terms of median training time. It often requires hours of training compared to seconds for other algorithms. The QUEST and logistic regression algorithms are substantially faster. Among decision tree algorithms with univariate splits, C4.5, IND-CART, and QUEST have the best combinations of error rate and speed. But C4.5 tends to produce trees with twice as many leaves as those from IND-CART and QUEST.

classification treedecision treeneural netstatistical classifier
Download to read the full article text

Copyright information

© Kluwer Academic Publishers 2000

Authors and Affiliations

  • Tjen-Sien Lim
    • 1
  • Wei-Yin Loh
    • 2
  • Yu-Shan Shih
    • 3
  1. 1.Department of StatisticsUniversity of WisconsinMadisonUSA
  2. 2.Department of StatisticsUniversity of WisconsinMadisonUSA
  3. 3.Department of MathematicsNational Chung Cheng UniversityChiayiTaiwan, R.O.C.