Evaluating the Stability of Feature Selectors That Optimize Feature Subset Cardinality

  • Petr Somol
  • Jana Novovičová
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5342)


Stability (robustness) of feature selection methods is a topic of recent interest. Unlike other known stability criteria, the new consistency measures proposed in this paper evaluate the overall occurrence of individual features in selected subsets of possibly varying cardinality. The new measures are compared to the generalized Kalousis measure which evaluates pairwise similarities between subsets. The new measures are computationally very effective and offer more than one type of insight into the stability problem. All considered measures have been used to compare two standard feature selection methods on a set of examples.


Feature selection stability relative weighted consistency measure sequential search floating search 


  1. 1.
    Dunne, K., Cunningham, P., Azuaje, F.: Solutions to instability problems with sequential wrapper-based approaches to feature selection. Technical Report TCD-CD-2002-28, Dept. of Computer Science, Trinity College, Dublin, Ireland (2002)Google Scholar
  2. 2.
    Kalousis, A., Prados, J., Hilario, M.: Stability of feature selection algorithms: a study on high-dimensional spaces. Knowledge and Inf. Syst. 12(1), 95–116 (2007)CrossRefGoogle Scholar
  3. 3.
    Kuncheva, L.I.: A stability index for feature selection. In: Proc. 25th IASTED Int. Multi-Conf. Artificial Intelligence and Applications, pp. 421–427 (2007)Google Scholar
  4. 4.
    Křížek, P., Kittler, J., Hlaváč, V.: Improving stability of feature selection methods. In: Kropatsch, W.G., Kampel, M., Hanbury, A. (eds.) CAIP 2007. LNCS, vol. 4673, pp. 929–936. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  5. 5.
    Saeys, Y., Abeel, T., de Peer, Y.V.: Towards robust feature selection techniques. In: Proceedings of Benelearn, pp. 45–46 (2008)Google Scholar
  6. 6.
    Raudys, Š.: Feature over-selection. In: Yeung, D.-Y., Kwok, J.T., Fred, A., Roli, F., de Ridder, D. (eds.) SSPR 2006 and SPR 2006. LNCS, vol. 4109, pp. 622–631. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  7. 7.
    Devijver, P.A., Kittler, J.: Pattern Recognition: A Statistical Approach. Prentice-Hall International, London (1982)zbMATHGoogle Scholar
  8. 8.
    Pudil, P., Novovičová, J., Kittler, J.: Floating search methods in feature selection. Pattern Recognition Letters 15, 1119–1125 (1994)CrossRefGoogle Scholar
  9. 9.
    Asuncion, A., Newman, D.: UCI machine learning repository (2007),
  10. 10.
    Kohavi, R., John, G.: Wrappers for feature subset selection. Artificial Intelligence 97, 273–324 (1997)CrossRefzbMATHGoogle Scholar
  11. 11.
    Chang, C.C., Lin, C.J.: LIBSVM: a library for support vector machines (2001),

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Petr Somol
    • 1
    • 2
  • Jana Novovičová
    • 1
    • 2
  1. 1.Dept. of Pattern Recognition, Institute of Information Theory and AutomationAcademy of Sciences of the Czech RepublicPragueCzech Republic
  2. 2.Faculty of ManagementPrague University of EconomicsCzech Republic

Personalised recommendations