Feature selection for high-dimensional classification using a competitive swarm optimizer
When solving many machine learning problems such as classification, there exists a large number of input features. However, not all features are relevant for solving the problem, and sometimes, including irrelevant features may deteriorate the learning performance.Please check the edit made in the article title Therefore, it is essential to select the most relevant features, which is known as feature selection. Many feature selection algorithms have been developed, including evolutionary algorithms or particle swarm optimization (PSO) algorithms, to find a subset of the most important features for accomplishing a particular machine learning task. However, the traditional PSO does not perform well for large-scale optimization problems, which degrades the effectiveness of PSO for feature selection when the number of features dramatically increases. In this paper, we propose to use a very recent PSO variant, known as competitive swarm optimizer (CSO) that was dedicated to large-scale optimization, for solving high-dimensional feature selection problems. In addition, the CSO, which was originally developed for continuous optimization, is adapted to perform feature selection that can be considered as a combinatorial optimization problem. An archive technique is also introduced to reduce computational cost. Experiments on six benchmark datasets demonstrate that compared to the canonical PSO-based and a state-of-the-art PSO variant for feature selection, the proposed CSO-based feature selection algorithm not only selects a much smaller number of features, but result in better classification performance as well.
KeywordsFeature selection High dimensionality Large-scale optimization Classification Competitive swarm optimization
This work was supported in part by National Natural Science Foundation of China (No. 71533001), the Joint Research Fund for Overseas Chinese, Hong Kong and Macao Scholars of the National Natural Science Foundation of China (No. 61428302), and an EPSRC Grant (No. EP/M017869/1).
Compliance with ethical standards
Conflict of interest
S. Gu declares that he has no conflict of interest. R. Cheng declares that he has no conflict of interest. Y. Jin declares that he has no conflict of interest.
- Aha D, Kibler D, Albert M (1991) Instance-based learning algorithms. Mach Learn 6(1):37–66Google Scholar
- Bache K, Lichman M (2013) UCI machine learning repository. http://archive.ics.uci.edu/ml
- Cheng R, Jin Y (2014) Demonstrator selection in a social learning particle swarm optimizer. In: 2014 IEEE congress on evolutionary computation, pp 3103–3110Google Scholar
- Fei H, Huan J (2010) Boosting with structure information in the functional space: an application to graph classification. In: Proceedings of the 16th ACM SIGKDD international conference on knowledge discovery and data mining, ACM, New York, pp 643–652Google Scholar
- Fong S, Wong R, Vasilakos AV (2016) Accelerated PSO swarm search feature selection for data stream mining big data. IEEE Trans Serv Comput 9(1):33–45Google Scholar
- Kennedy J, Eberhart R (1995) Particle swarm optimization. In: IEEE international conference on neural networks: proceedings, IS - SN -, vol 4, pp 1942–1948Google Scholar
- Kira K, Rendell LA(1992) A practical approach to feature selection. In: Proceedings of the international workshop on machine learning, pp 249–256Google Scholar
- Liu Z, Jiang F, Tian G, Wang S, Sato F, Meltzer SJ, Tan M (2007) Sparse logistic regression with Lp penalty for biomarker identification. Stat Appl Gen Mol Biol 6(1):6. doi: 10.2202/1544-6115.1248
- Neshatian K, Zhang M(2009) Pareto front feature selection: using genetic programming to explore feature space. In: Proceedings of the annual conference on genetic and evolutionary computation, ACM, pp 1027–1034Google Scholar
- Shi Y, Eberhart R (1998) A modified particle swarm optimizer. In: Evolutionary computation proceedings, 1998. IEEE world congress on computational intelligence., The 1998 IEEE international conference on IS - SN - VO -, pp 69–73Google Scholar
- Tran B, Xue B, Zhang M (2016) Bare-bone particle swarm optimisation for simultaneously discretising and selecting features for high-dimensional classification. In: Squillero G, Burelli P (eds) Applications of evolutionary computation: 19th European conference, evoapplications 2016, Porto, Portugal, March 30–April 1, 2016, Proceedings, Part I, Springer International Publishing, pp 701–718Google Scholar
- Xue B, Zhang M, Browne W, Yao X (2016) A survey on evolutionary computation approaches to feature selection. IEEE Trans Evol Comput PP(99):1–1Google Scholar