Dynamic Recursive Model Class Selection for Classifier Construction

  • Carla E. Brodley
  • Paul E. Utgoff
Conference paper
Part of the Lecture Notes in Statistics book series (LNS, volume 89)


Before applying an automated model selection procedure, one must first choose the class (or family) of models from which the model will be selected. If there is no prior knowledge about the data that indicates the best class of models, then the choice is difficult at best. In this chapter, we present an approach to automating this step in classifier construction. In addition to searching for the best model, our approach searches for the best model class using a heuristic search strategy that finds the best model class for each recursive call of a divide-and-conquer tree induction algorithm. The end result is a hybrid tree-structured classifier, which allows different subspaces of a data set to be fit by models from different model classes. During search for the best model, the method considers whether and why a model class is a poor choice, and selects a better class on that basis. We describe an implementation of the approach, the MCS system, and present experimental results illustrating the system’s ability to identify the best model (and model class) efficiently.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [Box, 1990]
    Box, D. R. (1990). “Role of models in statistical analysis.” Statistical Science, 5, 169–174.MathSciNetCrossRefGoogle Scholar
  2. [Breiman, Friedman, Olshen & Stone, 1984]
    Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (1984). Classification and regression trees. Belmont, CA: Wadsworth International Group.zbMATHGoogle Scholar
  3. [Breiman, Friedman, Olshen & Stone, 1984]
    Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (1984). Classification and regression trees. Belmont, CA: Wadsworth International Group.zbMATHGoogle Scholar
  4. [Cook & Weisberg, 1982]
    Cook, R. D., & Weisberg, S. (1982). Residuals and influence in regression. Chapman and Hall.zbMATHGoogle Scholar
  5. [Detrano, et al.]
    Detrano, R., Janosi, A., Steinbrunn, W., Pfisterer, M., Schmid, J., Sandhu, S., Guppy, K., Lee, S., & Froelicher, V. (1989). “International application of a new probability algorithm for the diagnosis of coronary artery disese.” American Journal of Cardiology, 64, 304–310.CrossRefGoogle Scholar
  6. [Frean, 1990]
    Frean, M. (1990). Small nets and short paths: Optimising neural computation. Doctoral dissertation, Center for Cognitive Science, University of Edinburgh.Google Scholar
  7. [Kittler, 1986]
    Kittler, J. (1986). “Feature selection and extraction.” Handbook of pattern recognition and image processing.Google Scholar
  8. [Lehmann, 1990]
    Lehmann, E. L. (1990). “Model specification: The views of Fisher and Neyman, and later developments.” Statistical Science, 5, 160–168.MathSciNetzbMATHCrossRefGoogle Scholar
  9. [Linhart & Zucchini, 1986]
    Linhart, H., & Zucchini, W. (1986). Model selection. NY: Wiley.zbMATHGoogle Scholar
  10. [Matheus, 1990]
    Matheus, C. J. (1990). Feature construction: An analytic framework and an application to decision trees. Doctoral dissertation, Department of Computer Science, University of Illinois, Urbana-Champaign, IL.Google Scholar
  11. [Nilsson, 1965]
    Nilsson, N. J. (1965). Learning machines. McGraw-HillzbMATHGoogle Scholar
  12. [Pagallo, 1990]
    Pagallo, G. M. (1990). Adaptive decision tree algorithms for learning from examples. Doctoral dissertation, University of California at Santa Cruz.Google Scholar
  13. [Quinlan, 1986]
    Quinlan, J. R. (1986). “Induction of decision trees.” Machine Learning, 1, 81–106.Google Scholar
  14. [Quinlan, 1987]
    Quinlan, J. R. (1987). “Simplifying decision trees” Internation Journal of Man-Machine Studies, 27, 221–234.CrossRefGoogle Scholar
  15. [Rissanen, 1989]
    Rissanen, J. (1989). Stochastic complexity in statistical inquiry. New Jersey: World Scientific.zbMATHGoogle Scholar
  16. [Safavian & Langrebe, 1991]
    Safavian, S. R., & Langrebe, D. (1991). “A survey of decision tree classifier methodology.” IEEE Transactions on Systems, Man and Cybernetics, 21, 660–674.CrossRefGoogle Scholar
  17. [Young, 1984]
    Young, R (1984). Recursive estimation and time-series analysis. New York: Springer- Verlag.zbMATHGoogle Scholar

Copyright information

© Springer-Verlag New York, Inc. 1994

Authors and Affiliations

  • Carla E. Brodley
    • 1
  • Paul E. Utgoff
    • 1
  1. 1.Department of Computer ScienceUniversity of MassachusettsAmherstUSA

Personalised recommendations