Current Feature Selection Techniques in Statistical Pattern Recognition

  • Pavel Pudil
  • Petr Somol
Part of the Advances in Soft Computing book series (AINSC, volume 30)


The paper addresses the problem of feature selection (abbreviated FS in the sequel) in statistical pattern recognition with particular emphasis to recent knowledge. Besides over-viewing advances in methodology it attempts to put them into a taxonomical framework. The methods discussed include the latest variants of the Branch & Bound algorithm, enhanced sub-optimal techniques and the simultaneous semi-parametric probability density function modeling and feature space selection method.


Feature Selection Feature Subset Feature Selection Method Optimal Search Feature Subset Selection 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Devijver PA, Kittler J (1982) Pattern Recognition: A Statistical Approach, Prentice-HallGoogle Scholar
  2. 2.
    Duda RO, Hart PE, Stork DG (2000) Pattern Classification, 2nd Ed., Wiley-InterscienceGoogle Scholar
  3. 3.
    Ferri FJ, Pudil P, Hatef M, Kittler J (1994) Comparative Study of Techniques for Large-Scale Feature Selection, Gelsema ES, Kanal LN (eds.) Pattern Recognition in Practice IV, Elsevier Science B.V., 403–413Google Scholar
  4. 4.
    Fukunaga K (1990) Introduction to Statistical Pattern Recognition, Academic PressGoogle Scholar
  5. 5.
    Chen X (2003) An Improved Branch and Bound Algorithm for Feature Selection, Pattern Recognition Letters 24(12):1925–1933CrossRefGoogle Scholar
  6. 6.
    Jain AK, Zongker D (1997) Feature Selection: Evaluation, Application and Small Sample Performance, IEEE Transactions on Pattern Analysis and Machine Intelligence 19(2):153–158CrossRefGoogle Scholar
  7. 7.
    Kohavi R, John GH (1997) Wrappers for Feature Subset Selection. Artificial Intelligence 97(1–2):273–324zbMATHCrossRefGoogle Scholar
  8. 8.
    Kudo M, Sklansky J (2000) Comparison of Algorithms that Select Features for Pattern Classifiers, Pattern Recognition 33(1):25–41CrossRefGoogle Scholar
  9. 9.
    McLachlan GJ (1992) Discriminant Analysis and Statistical Pattern Recognition, John Wiley & Sons, New YorkGoogle Scholar
  10. 10.
    Murphy PM, Aha DW (1994) UCI Repository of Machine Learning Databases []. University of California, Department of Information and Computer Science, Irvine, CAGoogle Scholar
  11. 11.
    Narendra PM, Fukunaga K (1977) A Branch and Bound Algorithm for Feature Subset Selection. IEEE Transactions on Computers 26:917–922zbMATHCrossRefGoogle Scholar
  12. 12.
    Novovičová J, Pudil P, Kittler J (1996) Divergence Based Feature Selection for Multimodal Class Densities, IEEE Transactions on Pattern Analysis and Machine Intelligence 18(2):218–223CrossRefGoogle Scholar
  13. 13.
    Pudil P, Novovičová J, Kittler J (1994) Floating Search Methods in Feature Selection, Pattern Recognition Letters 15(11):1119–1125CrossRefGoogle Scholar
  14. 14.
    Pudil P, Novovičová J, Kittler J (1994) Simultaneous Learning of Decision Rules and Important Attributes for Classification Problems in Image Analysis, Image and Vision Computing 12:193–198CrossRefGoogle Scholar
  15. 15.
    Ripley B (1996) Pattern Recognition and Neural Networks, Cambridge University Press, Cambridge, MassachusettszbMATHGoogle Scholar
  16. 16.
    Siedlecki W, Sklansky J (1988) On Automatic Feature Selection, International Journal of Pattern Recognition and Artificial Intelligence 2(2):197–220CrossRefGoogle Scholar
  17. 17.
    Somol P, Pudil P, Novovičová J, Paclík P (1999) Adaptive Floating Search Methods in Feature Selection, Pattern Recognition Letters 20(11,12,13):1157–1163CrossRefGoogle Scholar
  18. 18.
    Somol P, Pudil P (2000) Oscillating Search Algorithms For Feature Selection, In: Proc 15th IAPR International Conference on Pattern Recognition, Barcelona, Spain, 406–409Google Scholar
  19. 19.
    Somol P, Pudil P (2002) Feature Selection Toolbox. Pattern Recognition 35(12):2749–2759zbMATHCrossRefGoogle Scholar
  20. 20.
    Somol P, Pudil P, Kittler J (2004) Fast Branch & Bound Algorithms for Optimal Feature Selection. IEEE Transactions on Pattern Analysis and Machine Intelligence 26(7):900–912CrossRefGoogle Scholar
  21. 21.
    Somol P, Pudil P, Grim J (2004) On Prediction Mechanisms in Fast Branch & Bound Algorithms. Lecture Notes in Computer Science 3138, Springer, Berlin, 716–724Google Scholar
  22. 22.
    Theodoridis S, Koutroumbas K (2003) Pattern Recognition, 2nd Ed., Academic PressGoogle Scholar
  23. 23.
    Wang Z, Yang J, Li G (2003) An Improved Branch & Bound Algorithm in Feature Selection, Lecture Notes in Computer Science LNCS 2639, Springer, 549–556.Google Scholar
  24. 24.
    Webb A (2002) Statistical Pattern Recognition, 2nd Ed., John Wiley & SonsGoogle Scholar
  25. 25.
    Yu B, Yuan B (1993) A More Efficient Branch and Bound Algorithm for Feature Selection. Pattern Recognition 26:883–889CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  • Pavel Pudil
    • 1
  • Petr Somol
    • 1
  1. 1.Dept. of Pattern Recognition, Inst. of Information Theory and AutomationAcademy of Sciences of the Czech RepublicPrague 8Czech Republic

Personalised recommendations