Novel Methods for Feature Subset Selection with Respect to Problem Knowledge

  • Pavel Pudil
  • Jana Novovičová
Part of the The Springer International Series in Engineering and Computer Science book series (SECS, volume 453)


Recent advances in the statistical methodology for selecting optimal subsets of features for data representation and classification are presented. This chapter attempts to provide a guideline of which approach to choose with respect to the extent of a priori knowledge of the problem. Two basic approaches are reviewed and the conditions under which they should be used are specified. One approach involves the use of the computationally effective Floating search methods. The alternative approach trades off the requirement for a priori information for the requirement of sufficient data to represent the distributions involved. Owing to its nature it is particularly suitable for cases when the underlying probability distributions are not unimodal. The approach attempts to achieve simultaneous feature selection and decision rule inference. According to the criterion adopted there are two variants allowing the selection of features either for optimal representation or discrimination.


Feature Selection Feature Extraction Feature Subset Feature Selection Method Subset Selection 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Boekee, D. E. and Lubbe, J. C. A. V. D. (1979). Some aspects of error bounds in feature selection. Pattern Recognition, 11:353–360.MathSciNetzbMATHCrossRefGoogle Scholar
  2. Devijver, P. A. and Kittler, J. (1982). Pattern Recognition: A Statistical Approach. Prentice-Hall.zbMATHGoogle Scholar
  3. F.J. Ferri, P. Pudil, M. Hatef, and J. Kittler (1994). Comparative study of technique for large-scale features selection. In Pattern Recognition in Practice IV: Multiple Paradigms,Comparative Studies and Hybrid Systems; edited by E.S. Gelsema and L.N.Kanal,pages 403–413, North-Holland Elsevier.Google Scholar
  4. Jain, A. K. and Zongker, D. (1997). Feature selection: Evaluation, application and small sample performance. IEEE Transactions on PAMI,19:153–158.CrossRefGoogle Scholar
  5. Novovičová, J. and Pudil, P. (1997). Dealing With Complexity: Neural Network Approach, chapter Feature Selection and Classification by Modified Model with Latent Structure. Springer Verlag.Google Scholar
  6. Novovičová, J., Pudil, P., and Kittler, J. (1996). Divergence based feature selection for multimodal class densities. IEEE Transactions on Pattern Analysis and Machine Intelligence, 18:218–223.CrossRefGoogle Scholar
  7. Pudil, P. and Novovičová, J. (1998). Novel methods for subset selection with respect to problem knowledge. IEEE Intelligent Systems - Special Issue on Feature Transformation and Subset Selection,in print.Google Scholar
  8. Pudil, P., Novovičová, J., Choakjarernwanit, N., and Kittler, J. (1993). An analysis of the max-min approach to feature selection. Pattern Recognition Letters, 14(11):841–847.zbMATHCrossRefGoogle Scholar
  9. Pudil, P. Novovičová, J. Choakjarernwanit, N., and Kittler, J. (1995). Feature selection based on the approximation of class densities by finite mixtures of special type. Pattern Recognition,28(9):1389–98. CrossRefGoogle Scholar
  10. Pudil, P., Novovičová, J., and Kittler, J. (1994). Floating search methods in feature selection. Pattern Recognition Letters, 15:1119–1125.CrossRefGoogle Scholar
  11. Redner, R. A. and Walker, H. F. (1984). Mixture densities, maximum likelihood and the EM algorithm. SIAM J. Appl. Math.,26(2):195–239.MathSciNetzbMATHGoogle Scholar
  12. Siedlecki, W. and Sklansky, J. (1988). On automatic feature selection. International Journal of Pattern Recognition and Artificial Intelligence, 2(2):197–220.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 1998

Authors and Affiliations

  • Pavel Pudil
    • 1
  • Jana Novovičová
    • 1
  1. 1.Institute of Information Theory and AutomationAcademy of Sciences of the Czech RepublicPrague 8Czech Republic

Personalised recommendations