Sequential Instance Based Feature Subset Selection for High Dimensional Data

Abstract

Feature subset selection is a key problem in the data-mining classification task that helps to obtain more compact and understandable models without degrading their performance. This paper deals with the problem of supervised wrapper based feature subset selection in data sets with a very large number of attributes and a low sample size. In this case, standard wrapper algorithms cannot be applied because of their complexity. In this work we propose a new hybrid -filter wrapper- approach based on instance learning with the main goal of accelerating the feature subset selection process by reducing the number of wrapper evaluations. In our hybrid feature selection method, named Hybrid Instance Based Sequential Backward Search (HIB-SBS), instance learning is used to weight features and generate candidate feature subsets, then SBS and K-nearest neighbours (KNN) compose an evaluation system of wrappers. Our method is experimentally tested and compared with state-of-the-art algorithms over four high-dimensional low sample size datasets. The results show an impressive reduction in the execution time compared to the wrapper approach and that our proposal outperforms other methods in terms of accuracy and cardinality of the selected subset.