Advertisement

Overview of Particle Swarm Optimisation for Feature Selection in Classification

  • Binh Tran
  • Bing Xue
  • Mengjie Zhang
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8886)

Abstract

Feature selection is a process of selecting a subset of relevant features from a large number of original features to achieve similar or better classification performance and improve the computation efficiency. As an important data pre-processing technique, research into feature selection has been carried out over the past four decades. Determining an optimal feature subset is a complicated problem. Due to the limitations of conventional methods, evolutionary computation (EC) has been proposed to solve feature selection problems. Particle swarm optimisation (PSO) is an EC technique which recently has caught much interest from researchers in the field. This paper presents a review of PSO for feature selection in classification. After describing the background of feature selection and PSO, recent work involving PSO for feature selection is reviewed. Current issues and challenges are also presented for future research.

Keywords

Particle swarm optimisation feature selection evolutionary computation classification 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Alba, E., Garcia-Nieto, J., Jourdan, L., Talbi, E.G.: Gene selection in cancer classification using PSO/SVM and GA/SVM hybrid algorithms. In: IEEE Congress on Evolutionary Computation (CEC 2007), pp. 284–290 (2007)Google Scholar
  2. 2.
    Aneesh, M., Masand, A.A., Manikantan, K.: Optimal feature selection based on image pre-processing using accelerated binary particle swarm optimization for enhanced face recognition. Procedia Engineering 30(5), 750–758 (2012)CrossRefGoogle Scholar
  3. 3.
    Azevedo, G., Cavalcanti, G., Filho, E.: An approach to feature selection for keystroke dynamics systems based on PSO and feature weighting. In: IEEE Congress on Evolutionary Computation (CEC 2007), pp. 3577–3584 (2007)Google Scholar
  4. 4.
    Cervante, L., Xue, B., Shang, L., Zhang, M.: Binary particle swarm optimisation and rough set theory for dimension reduction in classification. In: IEEE Congress on Evolutionary Computation, pp. 2428–2435 (2013)Google Scholar
  5. 5.
    Cervante, L., Xue, B., Shang, L., Zhang, M.: A multi-objective feature selection approach based on binary pso and rough set theory. In: Middendorf, M., Blum, C. (eds.) EvoCOP 2013. LNCS, vol. 7832, pp. 25–36. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  6. 6.
    Cervante, L., Xue, B., Zhang, M., Shang, L.: Binary particle swarm optimisation for feature selection: A filter based approach. In: IEEE Congress on Evolutionary Computation (CEC 2012), pp. 881–888 (2012)Google Scholar
  7. 7.
    Chakraborty, B.: Feature subset selection by particle swarm optimization with fuzzy fitness function. In: 3rd International Conference on Intelligent System and Knowledge Engineering (ISKE 2008), pp. 1038–1042 (2008)Google Scholar
  8. 8.
    Chakraborty, B., Chakraborty, G.: Fuzzy consistency measure with particle swarm optimization for feature selection. In: 2013 IEEE International Conference on Systems, Man, and Cybernetics (SMC 2013), pp. 4311–4315 (2013)Google Scholar
  9. 9.
    Chuang, L.Y., Chang, H.W., Tu, C.J., Yang, C.H.: Improved binary PSO for feature selection using gene expression data. Computational Biology and Chemistry 32(29), 29–38 (2008)CrossRefzbMATHGoogle Scholar
  10. 10.
    Chuang, L.Y., Tsai, S.W., Yang, C.H.: Improved binary particle swarm optimization using catfish effect for feature selection. Expert Syst. Appl. 38(10), 12699–12707 (2011)CrossRefGoogle Scholar
  11. 11.
    Dash, M., Liu, H.: Feature selection for classification. Intelligent Data Analysis 1(4), 131–156 (1997)CrossRefGoogle Scholar
  12. 12.
    Eberhart, R., Kennedy, J.: A new optimizer using particle swarm theory. In: Proceedings of the Sixth International Symposium on Micro Machine and Human Science, MHS 1995, pp. 39–43 (October 1995)Google Scholar
  13. 13.
    Eberhart, R., Shi, Y.: Particle swarm optimization: developments, applications and resources. In: IEEE Congress on Evolutionary Computation, vol. 1, pp. 81–86 (2001)Google Scholar
  14. 14.
    Guan, J., Han, F., Yang, S.: A new gene selection method for microarray data based on PSO and informativeness metric. In: Huang, D.-S., Jo, K.-H., Zhou, Y.-Q., Han, K. (eds.) ICIC 2013. LNCS, vol. 7996, pp. 145–154. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  15. 15.
    Huang, C.L., Dun, J.F.: A distributed PSO-SVM hybrid system with feature selection and parameter optimization. Appl. Soft Comput. 8, 1381–1391 (2008)CrossRefGoogle Scholar
  16. 16.
    Ke, L., Feng, Z., Xu, Z., Shang, K., Wang, Y.: A multiobjective ACO algorithm for rough feature selection. In: Second Pacific-Asia Conference on Circuits,Communications and System (PACCS 2010), vol. 1, pp. 207–210 (2010)Google Scholar
  17. 17.
    Kennedy, J., Eberhart, R.: Particle swarm optimization. In: IEEE International Conference on Neural Networks, vol. 4, pp. 1942–1948 (1995)Google Scholar
  18. 18.
    Kennedy, J., Eberhart, R.: A discrete binary version of the particle swarm algorithm. In: IEEE International Conference on Systems, Man, and Cybernetics, vol. 5, pp. 4104–4108 (1997)Google Scholar
  19. 19.
    Kennedy, J., Spears, W.: Matching algorithms to problems: an experimental test of the particle swarm and some genetic algorithms on the multimodal problem generator. In: IEEE World Congress on Computational Intelligence, pp. 78–83 (1998)Google Scholar
  20. 20.
    Lane, M., Xue, B., Liu, I., Zhang, M.: Particle swarm optimisation and statistical clustering for feature selection. In: Cranefield, S., Nayak, A. (eds.) AI 2013. LNCS, vol. 8272, pp. 214–220. Springer, Heidelberg (2013)Google Scholar
  21. 21.
    Lane, M., Xue, B., Liu, I., Zhang, M.: Gaussian based particle swarm optimisation and statistical clustering for feature selection. In: Blum, C., Ochoa, G. (eds.) EvoCOP 2014. LNCS, vol. 8600, pp. 133–144. Springer, Heidelberg (2014)CrossRefGoogle Scholar
  22. 22.
    Langley, P.: Selection of relevant features in machine learning. In: AAAI Technique Report FS-94-02, pp. 127–131 (October 1994)Google Scholar
  23. 23.
    Lin, S.W., Ying, K.C., Chen, S.C., Lee, Z.J.: Particle swarm optimization for parameter determination and feature selection of support vector machines. Expert Systems with Applications 35(4), 1817–1824 (2008)CrossRefGoogle Scholar
  24. 24.
    Liu, H., Dougherty, E., Dy, J., Torkkola, K., Tuv, E., Peng, H., Ding, C., Long, F., Berens, M., Parsons, L., Zhao, Z., Yu, L., Forman, G.: Evolving feature selection. IEEE Intelligent Systems 20(6), 64–76 (2005)CrossRefGoogle Scholar
  25. 25.
    Mohamad, M., Omatu, S., Deris, S., Yoshioka, M., Abdullah, A., Ibrahim, Z.: An enhancement of binary particle swarm optimization for gene selection in classifying cancer classes. Algorithms for Molecular Biology 8(1), 15 (2013)CrossRefGoogle Scholar
  26. 26.
    Mohamad, M., Omatu, S., Deris, S., Yoshioka, M.: A modified binary particle swarm optimization for selecting the small subset of informative genes from gene expression data. Information Technology in Biomedicine 15(6), 813–822 (2011)CrossRefGoogle Scholar
  27. 27.
    Mohemmed, A., Zhang, M., Johnston, M.: Particle Swarm Optimization based Adaboost for face detection. In: IEEE Congress on Evolutionary Computation, pp. 2494–2501 (2009)Google Scholar
  28. 28.
    Narendra, P., Fukunaga, K.: A Branch and Bound Algorithm for Feature Subset Selection. IEEE Transactions on Computers C-26(9), 917–922 (1977)CrossRefGoogle Scholar
  29. 29.
    Neshatian, K., Zhang, M., Andreae, P.: A filter approach to multiple feature construction for symbolic learning classifiers using genetic programming. IEEE Transactions on Evolutionary Computation 16(5), 645–661 (2012)CrossRefGoogle Scholar
  30. 30.
    Oliveira, L., Sabourin, R., Bortolozzi, F., Suen, C.: Feature selection using multi-objective genetic algorithms for handwritten digit recognition. In: 16th International Conference on Pattern Recognition, vol. 1, pp. 568–571 (2002)Google Scholar
  31. 31.
    Pal, S., Chakraborty, B.: Fuzzy Set Theoretic Measure for Automatic Feature Evaluation. IEEE Transactions on Systems, Man and Cybernetics 16(5), 754–760 (1986)CrossRefGoogle Scholar
  32. 32.
    Purohit, A., Chaudhari, N., Tiwari, A.: Construction of classifier with feature selection based on genetic programming. In: IEEE Congress on Evolutionary Computation, pp. 1–5 (2010)Google Scholar
  33. 33.
    Sahu, B., Mishra, D.: A Novel Feature Selection Algorithm using Particle Swarm Optimization for Cancer Microarray Data. Procedia Engineering 38(0), 27–31 (2012)CrossRefGoogle Scholar
  34. 34.
    Shi, Y., Eberhart, R.: A modified particle swarm optimizer. In: IEEE Congress on Evolutionary Computation, pp. 69–73 (1998)Google Scholar
  35. 35.
    Sivagaminathan, R.K., Ramakrishnan, S.: A hybrid approach for feature subset selection using neural networks and ant colony optimization. Expert Systems with Applications 33(1), 49–60 (2007)CrossRefGoogle Scholar
  36. 36.
    Stevanovic, A., Xue, B., Zhang, M.: Feature selection based on pso and decision-theoretic rough set model. In: IEEE Congress on Evolutionary Computation, pp. 2840–2847 (2013)Google Scholar
  37. 37.
    Subbotin, S., Oleynik, A.: The multi objective evolutionary feature selection. In: International Conference on Modern Problems of Radio Engineering, Telecommunications and Computer Science, pp. 115–116 (2008)Google Scholar
  38. 38.
    Unler, A., Alper Murat, R.B.C.: mr2pso: A maximum relevance minimum redundancy feature selection method based on swarm intelligence for support vector machine classification. Information Sciences 20, 4625–4641 (2011)CrossRefGoogle Scholar
  39. 39.
    Unler, A., Murat, A.: A discrete particle swarm optimization method for feature selection in binary classification problems. European Journal of Operational Research 206(3), 528–539 (2010)CrossRefzbMATHGoogle Scholar
  40. 40.
    Wang, X., Yang, J., Teng, X., Xia, W., Jensen, R.: Feature selection based on rough sets and particle swarm optimization. Pattern Recognition Letters 28(4), 459–471 (2007)CrossRefGoogle Scholar
  41. 41.
    Waqas, K., Baig, R., Ali, S.: Feature subset selection using multi-objective genetic algorithms. In: 13th International Conference on INMIC, pp. 1–6 (2009)Google Scholar
  42. 42.
    Xue, B., Cervante, L., Shang, L., Browne, W.N., Zhang, M.: Binary PSO and rough set theory for feature selection: A multi-objective filter based approach. International Journal of Computational Intelligence and Applications 13(02), 1450009 (2014)CrossRefGoogle Scholar
  43. 43.
    Xue, B., Cervante, L., Shang, L., Browne, W., Zhang, M.: A multi-objective particle swarm optimisation for filter-based feature selection in classification problems. Connect. Sci. 24(2-3), 91–116 (2012)CrossRefGoogle Scholar
  44. 44.
    Xue, B., Zhang, M., Browne, W.N.: Multi-objective particle swarm optimisation for feature selection. In: Genetic and Evolutionary Computation Conference (GECCO 2012), Philadelphia, PA, USA, pp. 81–88. ACM (2012)Google Scholar
  45. 45.
    Xue, B., Zhang, M., Browne, W.N.: Particle swarm optimisation for feature selection in classification: Novel initialisation and updating mechanisms. Applied Soft Computing 18(0), 261–276 (2014)CrossRefGoogle Scholar
  46. 46.
    Xue, B., Zhang, M., Browne, W.N.: Novel initialisation and updating mechanisms in PSO for feature selection in classification. In: Esparcia-Alcázar, A.I. (ed.) EvoApplications 2013. LNCS, vol. 7835, pp. 428–438. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  47. 47.
    Xue, B., Zhang, M., Browne, W.: New fitness functions in binary particle swarm optimisation for feature selection. In: IEEE Congress on Evolutionary Computation, pp. 1–8 (2012)Google Scholar
  48. 48.
    Xue, B., Zhang, M., Browne, W.: Particle swarm optimization for feature selection in classification: A multi-objective approach. IEEE Transactions on Cybernetics 43(6), 1656–1671 (2013)CrossRefGoogle Scholar
  49. 49.
    Yang, C.S., Chuang, L.Y., Ke, C.H., Yang, C.H.: Boolean binary particle swarm optimization for feature selection. In: IEEE Congress on Evolutionary Computation (CEC 2008), pp. 2093–2098 (2008)Google Scholar
  50. 50.
    Yang, C.S., Chuang, L.Y., Li, J.C., Yang, C.H.: Chaotic maps in binary particle swarm optimization for feature selection. In: IEEE Conference on Soft Computing in Industrial Applications, pp. 107–112 (2008)Google Scholar
  51. 51.
    Yong, Z., Dunwei, G., Ying, H., Wanqiu, Z.: Feature selection algorithm based on bare bones particle swarm optimization. Neurocomputing 148(0), 150–157 (2015)Google Scholar
  52. 52.
    Yu, X.M., Xiong, X.Y., Wu, Y.W.: A PSO-based approach to optimal capacitor placement with harmonic distortion consideration. Electric Power Systems Research 71(1), 27–33 (2004)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Binh Tran
    • 1
  • Bing Xue
    • 1
  • Mengjie Zhang
    • 1
  1. 1.Victoria University of WellingtonWellingtonNew Zealand

Personalised recommendations