Improving Statistical Measures of Feature Subsets by Conventional and Evolutionary Approaches

  • Helmut A. Mayer
  • Petr Somol
  • Reinhold Huber
  • Pavel Pudil
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1876)


In this paper we compare recently developed and highly effective sequential feature selection algorithms with approaches based on evolutionary algorithms enabling parallel feature subset selection. We introduce the oscillating search method, employ permutation encoding offering some advantages over the more traditional bitmap encoding for the evolutionary search, and compare these algorithms to the often studied and well-performing sequential forward floating search. For the empirical analysis of these algorithms we utilize three well-known benchmark problems, and assess the quality of feature subsets by means of the statistical Bhattacharyya distance measure.


Feature Selection Evolutionary Algorithm Feature Subset Feature Selection Method Feature Selection Algorithm 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. [Blake and Merz, 1998]
    Blake, C. and Merz, C. (1998). WWW Repository, University of California, Irvine, Dept. of Information and Computer Sciences.
  2. [Chaikla and Qi, 1999]
    Chaikla, N. and Qi, Y. (1999). Genetic Algorithms in Feature Selection. In IEEE International Conference on Systems, Man, and Cybernetics, pages V-538–540. IEEE.Google Scholar
  3. [Devijver and Kittler, 1982]
    Devijver, P. and Kittler, J. (1982). Pattern Recognition: A Statistical Approach. Prentice.Google Scholar
  4. [Fukunaga, 1990]
    Fukunaga, K. (1990). Introduction to Statistical Pattern Recognition. Academic Press.Google Scholar
  5. [Goldberg and Lingle, 1985]
    Goldberg, D. E. and Lingle, R. (1985). Alleles, Loci, and the Traveling Salesman Problem. In Grefenstette, J. J., editor, Proceedings of the First International Conference on Genetic Algorithms and their Applications, pages 154–159. Texas Instruments, Inc. and Naval Research Laboratory, Lawrence Erlbaum Associates.Google Scholar
  6. [Jain and Zongker, 1997]
    Jain, A. and Zongker, D. (1997). Feature Selection: Evaluation, Application and Small Sample Performance. IEEE Transactions on PAMI, 19(2):153–158.Google Scholar
  7. [John et al., 1994]
    John, G., Kohavi, R., and Pfleger, K. (1994). Irrelevant Features and the Subset Selection Problem. In Proceedings of the Eleventh International Conference on Machine Learning, pages 121–129, San Mateo, CA. Morgan Kaufmann.Google Scholar
  8. [Kailath, 1967]
    Kailath, T. (1967). The divergence and bhattacharyya distance measures in signal selection. IEEE Transactions on Communications Technology, 15(1):52–60.CrossRefGoogle Scholar
  9. [Mitchell, 1996]
    Mitchell, M. (1996). An Introduction to Genetic Algorithms. Complex Adaptive Systems. MIT Press, Cambridge, MA.Google Scholar
  10. [Pudil et al., 1994]
    Pudil, P., Novovičová, J., and Kittler, J. (1994). Floating search methods in feature selection. Pattern Recognition Letters, 15:1119–1125.CrossRefGoogle Scholar
  11. [Punch et al., 1993]
    Punch, W. F., Goodman, E. D., Pei, M., Chia-Shun, L., Hovland, P., and Enbody, R. (1993). Further Research on Feature Selection and Classification Using Genetic Algorithms. In Forrest, S., editor, Fifth International Conference on Genetic Algorithms, pages 557–564, San Mateo, CA. Morgan Kaufmann.Google Scholar
  12. [Schwefel, 1995]
    Schwefel, H.-P. (1995). Evolution and Optimum Seeking Sixth-Generation Computer Technology Series. Wiley, New York.Google Scholar
  13. [Siedlecki and Sklansky, 1988]
    Siedlecki, W. and Sklansky, J. (1988). On automatic feature selection. International Journal of Pattern Recognition and Artificial Intelligence, 2(2):197–220.CrossRefGoogle Scholar
  14. [Siedlecki and Sklansky, 1989]
    Siedlecki, W. and Sklansky, J. (1989). A Note on Genetic Algorithms for Large-Scale Feature Selection. Pattern Recognition Letters, 10:335–347.zbMATHCrossRefGoogle Scholar
  15. [Somol and Pudil, 2000]
    Somol, P. and Pudil, P. (2000). Oscillating Search Algorithms for Feature Selection. In Submission to the 15th International Conference on Pattern Recognition, Barcelona.Google Scholar
  16. [Yang and Honavar, 1997]
    Yang, J. and Honavar, V. (1997). Feature Subset Selection Using a Genetic Algorithm. In Genetic Programming, pages 380–385.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2000

Authors and Affiliations

  • Helmut A. Mayer
    • 2
  • Petr Somol
    • 1
  • Reinhold Huber
    • 3
  • Pavel Pudil
    • 1
  1. 1.Department of Pattern RecognitionUTIA PraguePragueCzech Republic
  2. 2.Department of Computer ScienceUniversity of SalzburgSalzburgAustria
  3. 3.Department of MathematicsUniversity of SalzburgSalzburgAustria

Personalised recommendations