Comparison of Classifier-Specific Feature Selection Algorithms

  • Mineichi Kudo
  • Petr Somol
  • Pavel Pudil
  • Masaru Shimbo
  • Jack Sklansky
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1876)


The performance and speed of three classifier-specific feature selection algorithms, the sequential forward (backward) floating search (SFFS (SBFS)) algorithm, the ASFFS (ASBFS) algorithm (its adaptive version), and the genetic algorithm (GA) for large-scale problems are compared. The experimental results showed that 1) ASFFS (ASBFS) has better performance than does SFFS (SBFS) but requires much computation time, 2) much training in GA with a larger number of generations or with a larger population size, or both, is effective, 3) the performance of SFFS (SBFS) is comparable to that of GA with less training, and the performance of ASFFS (ASBFS) is comparable to that of GA with much training, but in terms of speed GA is better than ASFFS (ASBFS) for large-scale problems.


Genetic Algorithm Feature Selection Recognition Rate Feature Subset Feature Selection Algorithm 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Kittler, J.: Feature set search algorithms. In: Chen, C. H. (ed.): Pattern Recognition and Signal Processing, Sijthoff and Noordhoff, Alphen aan den Rijn, Netherlands (1978) 41–60Google Scholar
  2. 2.
    Ferri, F. J., Pudil, P., Hatef, M. and Kittler, J.: Comparative study of techniques for large-scale feature selection. In: Gelsema, E. S. and Kanal, L. N. (eds.): Pattern Recognition in Practice IV, Elsevier Science B. V. (1994) 403–413Google Scholar
  3. 3.
    Kudo, M. and Shimbo, M.: Feature selection based on the structural indices of categories. Pattern Recognition 26 (1993) 891–901CrossRefGoogle Scholar
  4. 4.
    Pudil, P., Novovičová, J. and Kittler, J.: Floating search methods in feature selection. Pattern Recognition Letters 15 (1994) 1119–1125CrossRefGoogle Scholar
  5. 5.
    Pudil, P., Ferri, F. J., Novovičová, J. and Kittler, J.: Floating search methods for feature selection with nonmonotonic criterion functions. In: 12th International Conference on Pattern Recognition (1994) 279–283Google Scholar
  6. 6.
    Novovičová, J., Pudil, P. and Kittler, J.: Divergence based feature selection for mulimodal class densities. IEEE Transactions on Pattern Analysis and Machine Intelligence 18 (1996) 218–223CrossRefGoogle Scholar
  7. 7.
    Holz, H.J. and Loew, M. H.: Relative feature importance: A classifier-independent approach to feature selection. In: Gelsema, E. S. and Kanal, L. N. (eds.), Pattern Recognition in Practice IV, Amsterdam: Elsevier (1994) 473–487Google Scholar
  8. 8.
    Jain, A. and Zongkerm, D.: Feature selection: Evaluation, application, and small sample performance. IEEE Trans. Pattern Anal. Machine Intell 19 (1997) 153–157CrossRefGoogle Scholar
  9. 9.
    Kudo, M. and Sklansky, J.: Comparison of algorithms that select features for pattern classifiers. Pattern Recognition, 33 (2000) 25–41CrossRefGoogle Scholar
  10. 10.
    Somol, P., Pudil, P., Novovičová, J. and Paclík, P.: Adaptive floating search methods in feature selection. Pattern Recognition Letters 20 (1999) 1157–1163CrossRefGoogle Scholar
  11. 11.
    Vriesenga, M.R.: Genetic Selection and Neureal Modeling for Designing Pattern Classifier. Doctor thesis, University of California, Irvine (1995)Google Scholar
  12. 12.
    Siedlecki, W. and Sklansky, J.: A note on genetic algorithms for large-scale feature selection. Pattern Recognition Letters 10 (1989) 335–347zbMATHCrossRefGoogle Scholar
  13. 13.
    Murphy, P.M. and Aha, D. W.: UCI Repository of machine learning databases [Machine-readable data repository]. University of California, Irivne, Department of Information amd Computation Science (1996)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2000

Authors and Affiliations

  • Mineichi Kudo
    • 1
  • Petr Somol
    • 2
  • Pavel Pudil
    • 2
  • Masaru Shimbo
    • 1
  • Jack Sklansky
    • 3
  1. 1.Division of Systems and Information Engineering Graduate School of EngineeringHokkaido UniversitySapporoJapan
  2. 2.Department of Pattern Recognition Inst. of Information Theory and AutomationAcademy of Science of the Czech RepublicPragueCzech Republic
  3. 3.Department of Electrical EngineeringUniversity of CaliforniaIrvineUSA

Personalised recommendations