Flexible-Hybrid Sequential Floating Search in Statistical Feature Selection

  • Petr Somol
  • Jana Novovičová
  • Pavel Pudil
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4109)


Among recent topics studied in context of feature selection the hybrid algorithms seem to receive particular attention. In this paper we propose a new hybrid algorithm, the flexible hybrid floating sequential search algorithm, that combines both the filter and wrapper search principles. The main benefit of the proposed algorithm is its ability to deal flexibly with the quality-of-result versus computational time trade-off and to enable wrapper based feature selection in problems of higher dimensionality than before. We show that it is possible to trade significant reduction of search time for negligible decrease of the classification accuracy. Experimental results are reported on two data sets, WAVEFORM data from the UCI repository and SPEECH data from British Telecom.


Feature Selection Hybrid Algorithm Feature Subset Subset Size Feature Subset Selection 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Liu, H., Yu, L.: Toward Integrating Feature Selection Algorithms for Classification and Clustering. IEEE Trans. on Knowledge and Data Engineering 17, 491–502 (2005)CrossRefGoogle Scholar
  2. 2.
    Yu, L., Liu, H.: Feature Selection for High-Dimensional Data: A Fast Correlation-Based Filter Solution. In: Proc. 20th Intl. Conf. Machine Learning, pp. 856–863 (2003)Google Scholar
  3. 3.
    Dash, M., Choi, K., Scheuermann, P., Liu, H.: Feature Selection for Clustering - a Filter Solution. In: Proc. Second Int. Conf. Data Mining, pp. 15–122 (2002)Google Scholar
  4. 4.
    Kohavi, R., John, G.H.: Wrappers for Feature Subset Selection. Artificial Intelligence 97, 273–324 (1997)zbMATHCrossRefGoogle Scholar
  5. 5.
    Das, S.: Filters, Wrappers and a Boosting-Based Hybrid for Feature Selection. In: Proc. 18th Intl. Conf. Machine Learning, pp. 74–81 (2001)Google Scholar
  6. 6.
    Sebban, M., Nock, R.: A Hybrid Filter/Wrapper Approach of Feature Selection using Information Theory. Pattern Recognition 35, 835–846 (2002)zbMATHCrossRefGoogle Scholar
  7. 7.
    Pudil, P., Novovicova, J., Kittler, J.: Floating Search Methods in Feature Selection. Pattern Recognition Letters 15, 1119–1125 (1994)CrossRefGoogle Scholar
  8. 8.
    Pudil, P., Novovicova, J., Somol, P.: Recent Feature Selection Methods in Statistical Pattern Recognition. In: Pattern Recognition and String Matching. Springer, Berlin (2003)Google Scholar
  9. 9.
    Jain, A.K., Zongker, D.: Feature selection: evaluation, application and small sample performance. IEEE Trans. PAMI 19, 153–158 (1997)Google Scholar
  10. 10.
    Kudo, M., Sklansky, J.: Comparison of algorithms that select features for pattern classifiers. Pattern Recognition 33, 25–41 (2000)CrossRefGoogle Scholar
  11. 11.
    Devijver, P.A., Kittler, J.: Pattern Recognition: A Statistical Approach. Prentice-Hall, Englewood Cliffs (1982)zbMATHGoogle Scholar
  12. 12.
    Murphy, P.M., Aha, D.W.: UCI Repository of Machine Learning Databases [Machine-readable data repository]. University of California, Department of Information and Computer Science, Irvine, CA (1994)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Petr Somol
    • 1
    • 2
  • Jana Novovičová
    • 1
    • 2
  • Pavel Pudil
    • 1
    • 2
  1. 1.Dept. of Pattern Recognition, Institute of Information Theory and AutomationAcademy of Sciences of the Czech RepublicPragueCzech Republic
  2. 2.Faculty of ManagementPrague University of EconomicsCzech Republic

Personalised recommendations