Proposed is a histogram approach for feature selection and classification. The axes are divided into equally-spaced intervals, but the division numbers are different among axes. The main difference from similar approaches is that feature selection mechanism is embedded in the method. The optimal division is determined by an MDL criterion, so that the classifier is guaranteed to converge to the Bayes optimal classifier. We also introduce the concept of “soft feature selection” that is carried out by this method as an extension of traditional “feature selection.”


Histogram classifier MDL criterion Soft feature selection 


  1. 1.
    Rissanen, J., Yu, B.: MDL Learning. In: Kueker, D.W., Smith, C.H. (eds.) Learning and Geometry: Computational Approaches, pp. 3–19. Birkhauser, Basel (1998)Google Scholar
  2. 2.
    Yamanishi, K.: Learning Non-Parametric Smooth Rules Using Stochastic Rules with Finite Partitioning. In: Proceedings of the Computational Learning Theory: EuroCOLT 1993, pp. 217–227 (1993)Google Scholar
  3. 3.
    Tsuchiya, H., Itoh, S., Mashimoto, T.: An Algorithm for Designing a Pattern Classifier by Using MDL Criterion. IEICE Transactions on Fundamentals, E79-A 6, 910–920 (1996)Google Scholar
  4. 4.
    Kudo, M., Sklansky, J.: Comparison of Algorithms that Select Features for Pattern Classifiers. Pattern Recognition 33(1), 25–41 (2000)CrossRefGoogle Scholar
  5. 5.
    Rissanen, J.: Stochastic Complexity in Statistical Inquiry. Series in Computer Science, vol. 15. World Scientific, Singapore (1989)zbMATHGoogle Scholar
  6. 6.
    Kudo, M., et al.: Construction of Class Regions by a Randomized Algorithm: A Randomized Subclass Method. Pattern Recognition 29, 581–588 (1996)CrossRefGoogle Scholar
  7. 7.
    Park, Y., Sklansky, J.: Automated Design of Multiple-Class Piecewise Linear Classifiers. Journal of Classification 6, 195–222 (1989)MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Hiroshi Tenmoto
    • 1
  • Mineichi Kudo
    • 2
  1. 1.Kushiro National College of TechnologyKushiroJapan
  2. 2.Division of Computer Science, Graduate School of Information Science and TechnologyHokkaido UniversitySapporoJapan

Personalised recommendations