Evaluating the Stability of Feature Selectors That Optimize Feature Subset Cardinality
Stability (robustness) of feature selection methods is a topic of recent interest. Unlike other known stability criteria, the new consistency measures proposed in this paper evaluate the overall occurrence of individual features in selected subsets of possibly varying cardinality. The new measures are compared to the generalized Kalousis measure which evaluates pairwise similarities between subsets. The new measures are computationally very effective and offer more than one type of insight into the stability problem. All considered measures have been used to compare two standard feature selection methods on a set of examples.
KeywordsFeature selection stability relative weighted consistency measure sequential search floating search
- 1.Dunne, K., Cunningham, P., Azuaje, F.: Solutions to instability problems with sequential wrapper-based approaches to feature selection. Technical Report TCD-CD-2002-28, Dept. of Computer Science, Trinity College, Dublin, Ireland (2002)Google Scholar
- 3.Kuncheva, L.I.: A stability index for feature selection. In: Proc. 25th IASTED Int. Multi-Conf. Artificial Intelligence and Applications, pp. 421–427 (2007)Google Scholar
- 5.Saeys, Y., Abeel, T., de Peer, Y.V.: Towards robust feature selection techniques. In: Proceedings of Benelearn, pp. 45–46 (2008)Google Scholar
- 9.Asuncion, A., Newman, D.: UCI machine learning repository (2007), http://www.ics.uci.edu/~mlearn/MLRepository.html
- 11.Chang, C.C., Lin, C.J.: LIBSVM: a library for support vector machines (2001), http://www.csie.ntu.edu.tw/~cjlin/libsvm