Skip to main content

Overview of Particle Swarm Optimisation for Feature Selection in Classification

  • Conference paper

Part of the Lecture Notes in Computer Science book series (LNTCS,volume 8886)

Abstract

Feature selection is a process of selecting a subset of relevant features from a large number of original features to achieve similar or better classification performance and improve the computation efficiency. As an important data pre-processing technique, research into feature selection has been carried out over the past four decades. Determining an optimal feature subset is a complicated problem. Due to the limitations of conventional methods, evolutionary computation (EC) has been proposed to solve feature selection problems. Particle swarm optimisation (PSO) is an EC technique which recently has caught much interest from researchers in the field. This paper presents a review of PSO for feature selection in classification. After describing the background of feature selection and PSO, recent work involving PSO for feature selection is reviewed. Current issues and challenges are also presented for future research.

Keywords

  • Particle swarm optimisation
  • feature selection
  • evolutionary computation
  • classification

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-319-13563-2_51
  • Chapter length: 13 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   89.00
Price excludes VAT (USA)
  • ISBN: 978-3-319-13563-2
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   119.99
Price excludes VAT (USA)

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Alba, E., Garcia-Nieto, J., Jourdan, L., Talbi, E.G.: Gene selection in cancer classification using PSO/SVM and GA/SVM hybrid algorithms. In: IEEE Congress on Evolutionary Computation (CEC 2007), pp. 284–290 (2007)

    Google Scholar 

  2. Aneesh, M., Masand, A.A., Manikantan, K.: Optimal feature selection based on image pre-processing using accelerated binary particle swarm optimization for enhanced face recognition. Procedia Engineering 30(5), 750–758 (2012)

    CrossRef  Google Scholar 

  3. Azevedo, G., Cavalcanti, G., Filho, E.: An approach to feature selection for keystroke dynamics systems based on PSO and feature weighting. In: IEEE Congress on Evolutionary Computation (CEC 2007), pp. 3577–3584 (2007)

    Google Scholar 

  4. Cervante, L., Xue, B., Shang, L., Zhang, M.: Binary particle swarm optimisation and rough set theory for dimension reduction in classification. In: IEEE Congress on Evolutionary Computation, pp. 2428–2435 (2013)

    Google Scholar 

  5. Cervante, L., Xue, B., Shang, L., Zhang, M.: A multi-objective feature selection approach based on binary pso and rough set theory. In: Middendorf, M., Blum, C. (eds.) EvoCOP 2013. LNCS, vol. 7832, pp. 25–36. Springer, Heidelberg (2013)

    CrossRef  Google Scholar 

  6. Cervante, L., Xue, B., Zhang, M., Shang, L.: Binary particle swarm optimisation for feature selection: A filter based approach. In: IEEE Congress on Evolutionary Computation (CEC 2012), pp. 881–888 (2012)

    Google Scholar 

  7. Chakraborty, B.: Feature subset selection by particle swarm optimization with fuzzy fitness function. In: 3rd International Conference on Intelligent System and Knowledge Engineering (ISKE 2008), pp. 1038–1042 (2008)

    Google Scholar 

  8. Chakraborty, B., Chakraborty, G.: Fuzzy consistency measure with particle swarm optimization for feature selection. In: 2013 IEEE International Conference on Systems, Man, and Cybernetics (SMC 2013), pp. 4311–4315 (2013)

    Google Scholar 

  9. Chuang, L.Y., Chang, H.W., Tu, C.J., Yang, C.H.: Improved binary PSO for feature selection using gene expression data. Computational Biology and Chemistry 32(29), 29–38 (2008)

    CrossRef  MATH  Google Scholar 

  10. Chuang, L.Y., Tsai, S.W., Yang, C.H.: Improved binary particle swarm optimization using catfish effect for feature selection. Expert Syst. Appl. 38(10), 12699–12707 (2011)

    CrossRef  Google Scholar 

  11. Dash, M., Liu, H.: Feature selection for classification. Intelligent Data Analysis 1(4), 131–156 (1997)

    CrossRef  Google Scholar 

  12. Eberhart, R., Kennedy, J.: A new optimizer using particle swarm theory. In: Proceedings of the Sixth International Symposium on Micro Machine and Human Science, MHS 1995, pp. 39–43 (October 1995)

    Google Scholar 

  13. Eberhart, R., Shi, Y.: Particle swarm optimization: developments, applications and resources. In: IEEE Congress on Evolutionary Computation, vol. 1, pp. 81–86 (2001)

    Google Scholar 

  14. Guan, J., Han, F., Yang, S.: A new gene selection method for microarray data based on PSO and informativeness metric. In: Huang, D.-S., Jo, K.-H., Zhou, Y.-Q., Han, K. (eds.) ICIC 2013. LNCS, vol. 7996, pp. 145–154. Springer, Heidelberg (2013)

    CrossRef  Google Scholar 

  15. Huang, C.L., Dun, J.F.: A distributed PSO-SVM hybrid system with feature selection and parameter optimization. Appl. Soft Comput. 8, 1381–1391 (2008)

    CrossRef  Google Scholar 

  16. Ke, L., Feng, Z., Xu, Z., Shang, K., Wang, Y.: A multiobjective ACO algorithm for rough feature selection. In: Second Pacific-Asia Conference on Circuits,Communications and System (PACCS 2010), vol. 1, pp. 207–210 (2010)

    Google Scholar 

  17. Kennedy, J., Eberhart, R.: Particle swarm optimization. In: IEEE International Conference on Neural Networks, vol. 4, pp. 1942–1948 (1995)

    Google Scholar 

  18. Kennedy, J., Eberhart, R.: A discrete binary version of the particle swarm algorithm. In: IEEE International Conference on Systems, Man, and Cybernetics, vol. 5, pp. 4104–4108 (1997)

    Google Scholar 

  19. Kennedy, J., Spears, W.: Matching algorithms to problems: an experimental test of the particle swarm and some genetic algorithms on the multimodal problem generator. In: IEEE World Congress on Computational Intelligence, pp. 78–83 (1998)

    Google Scholar 

  20. Lane, M., Xue, B., Liu, I., Zhang, M.: Particle swarm optimisation and statistical clustering for feature selection. In: Cranefield, S., Nayak, A. (eds.) AI 2013. LNCS, vol. 8272, pp. 214–220. Springer, Heidelberg (2013)

    Google Scholar 

  21. Lane, M., Xue, B., Liu, I., Zhang, M.: Gaussian based particle swarm optimisation and statistical clustering for feature selection. In: Blum, C., Ochoa, G. (eds.) EvoCOP 2014. LNCS, vol. 8600, pp. 133–144. Springer, Heidelberg (2014)

    CrossRef  Google Scholar 

  22. Langley, P.: Selection of relevant features in machine learning. In: AAAI Technique Report FS-94-02, pp. 127–131 (October 1994)

    Google Scholar 

  23. Lin, S.W., Ying, K.C., Chen, S.C., Lee, Z.J.: Particle swarm optimization for parameter determination and feature selection of support vector machines. Expert Systems with Applications 35(4), 1817–1824 (2008)

    CrossRef  Google Scholar 

  24. Liu, H., Dougherty, E., Dy, J., Torkkola, K., Tuv, E., Peng, H., Ding, C., Long, F., Berens, M., Parsons, L., Zhao, Z., Yu, L., Forman, G.: Evolving feature selection. IEEE Intelligent Systems 20(6), 64–76 (2005)

    CrossRef  Google Scholar 

  25. Mohamad, M., Omatu, S., Deris, S., Yoshioka, M., Abdullah, A., Ibrahim, Z.: An enhancement of binary particle swarm optimization for gene selection in classifying cancer classes. Algorithms for Molecular Biology 8(1), 15 (2013)

    CrossRef  Google Scholar 

  26. Mohamad, M., Omatu, S., Deris, S., Yoshioka, M.: A modified binary particle swarm optimization for selecting the small subset of informative genes from gene expression data. Information Technology in Biomedicine 15(6), 813–822 (2011)

    CrossRef  Google Scholar 

  27. Mohemmed, A., Zhang, M., Johnston, M.: Particle Swarm Optimization based Adaboost for face detection. In: IEEE Congress on Evolutionary Computation, pp. 2494–2501 (2009)

    Google Scholar 

  28. Narendra, P., Fukunaga, K.: A Branch and Bound Algorithm for Feature Subset Selection. IEEE Transactions on Computers C-26(9), 917–922 (1977)

    CrossRef  Google Scholar 

  29. Neshatian, K., Zhang, M., Andreae, P.: A filter approach to multiple feature construction for symbolic learning classifiers using genetic programming. IEEE Transactions on Evolutionary Computation 16(5), 645–661 (2012)

    CrossRef  Google Scholar 

  30. Oliveira, L., Sabourin, R., Bortolozzi, F., Suen, C.: Feature selection using multi-objective genetic algorithms for handwritten digit recognition. In: 16th International Conference on Pattern Recognition, vol. 1, pp. 568–571 (2002)

    Google Scholar 

  31. Pal, S., Chakraborty, B.: Fuzzy Set Theoretic Measure for Automatic Feature Evaluation. IEEE Transactions on Systems, Man and Cybernetics 16(5), 754–760 (1986)

    CrossRef  Google Scholar 

  32. Purohit, A., Chaudhari, N., Tiwari, A.: Construction of classifier with feature selection based on genetic programming. In: IEEE Congress on Evolutionary Computation, pp. 1–5 (2010)

    Google Scholar 

  33. Sahu, B., Mishra, D.: A Novel Feature Selection Algorithm using Particle Swarm Optimization for Cancer Microarray Data. Procedia Engineering 38(0), 27–31 (2012)

    CrossRef  Google Scholar 

  34. Shi, Y., Eberhart, R.: A modified particle swarm optimizer. In: IEEE Congress on Evolutionary Computation, pp. 69–73 (1998)

    Google Scholar 

  35. Sivagaminathan, R.K., Ramakrishnan, S.: A hybrid approach for feature subset selection using neural networks and ant colony optimization. Expert Systems with Applications 33(1), 49–60 (2007)

    CrossRef  Google Scholar 

  36. Stevanovic, A., Xue, B., Zhang, M.: Feature selection based on pso and decision-theoretic rough set model. In: IEEE Congress on Evolutionary Computation, pp. 2840–2847 (2013)

    Google Scholar 

  37. Subbotin, S., Oleynik, A.: The multi objective evolutionary feature selection. In: International Conference on Modern Problems of Radio Engineering, Telecommunications and Computer Science, pp. 115–116 (2008)

    Google Scholar 

  38. Unler, A., Alper Murat, R.B.C.: mr2pso: A maximum relevance minimum redundancy feature selection method based on swarm intelligence for support vector machine classification. Information Sciences 20, 4625–4641 (2011)

    CrossRef  Google Scholar 

  39. Unler, A., Murat, A.: A discrete particle swarm optimization method for feature selection in binary classification problems. European Journal of Operational Research 206(3), 528–539 (2010)

    CrossRef  MATH  Google Scholar 

  40. Wang, X., Yang, J., Teng, X., Xia, W., Jensen, R.: Feature selection based on rough sets and particle swarm optimization. Pattern Recognition Letters 28(4), 459–471 (2007)

    CrossRef  Google Scholar 

  41. Waqas, K., Baig, R., Ali, S.: Feature subset selection using multi-objective genetic algorithms. In: 13th International Conference on INMIC, pp. 1–6 (2009)

    Google Scholar 

  42. Xue, B., Cervante, L., Shang, L., Browne, W.N., Zhang, M.: Binary PSO and rough set theory for feature selection: A multi-objective filter based approach. International Journal of Computational Intelligence and Applications 13(02), 1450009 (2014)

    CrossRef  Google Scholar 

  43. Xue, B., Cervante, L., Shang, L., Browne, W., Zhang, M.: A multi-objective particle swarm optimisation for filter-based feature selection in classification problems. Connect. Sci. 24(2-3), 91–116 (2012)

    CrossRef  Google Scholar 

  44. Xue, B., Zhang, M., Browne, W.N.: Multi-objective particle swarm optimisation for feature selection. In: Genetic and Evolutionary Computation Conference (GECCO 2012), Philadelphia, PA, USA, pp. 81–88. ACM (2012)

    Google Scholar 

  45. Xue, B., Zhang, M., Browne, W.N.: Particle swarm optimisation for feature selection in classification: Novel initialisation and updating mechanisms. Applied Soft Computing 18(0), 261–276 (2014)

    CrossRef  Google Scholar 

  46. Xue, B., Zhang, M., Browne, W.N.: Novel initialisation and updating mechanisms in PSO for feature selection in classification. In: Esparcia-Alcázar, A.I. (ed.) EvoApplications 2013. LNCS, vol. 7835, pp. 428–438. Springer, Heidelberg (2013)

    CrossRef  Google Scholar 

  47. Xue, B., Zhang, M., Browne, W.: New fitness functions in binary particle swarm optimisation for feature selection. In: IEEE Congress on Evolutionary Computation, pp. 1–8 (2012)

    Google Scholar 

  48. Xue, B., Zhang, M., Browne, W.: Particle swarm optimization for feature selection in classification: A multi-objective approach. IEEE Transactions on Cybernetics 43(6), 1656–1671 (2013)

    CrossRef  Google Scholar 

  49. Yang, C.S., Chuang, L.Y., Ke, C.H., Yang, C.H.: Boolean binary particle swarm optimization for feature selection. In: IEEE Congress on Evolutionary Computation (CEC 2008), pp. 2093–2098 (2008)

    Google Scholar 

  50. Yang, C.S., Chuang, L.Y., Li, J.C., Yang, C.H.: Chaotic maps in binary particle swarm optimization for feature selection. In: IEEE Conference on Soft Computing in Industrial Applications, pp. 107–112 (2008)

    Google Scholar 

  51. Yong, Z., Dunwei, G., Ying, H., Wanqiu, Z.: Feature selection algorithm based on bare bones particle swarm optimization. Neurocomputing 148(0), 150–157 (2015)

    Google Scholar 

  52. Yu, X.M., Xiong, X.Y., Wu, Y.W.: A PSO-based approach to optimal capacitor placement with harmonic distortion consideration. Electric Power Systems Research 71(1), 27–33 (2004)

    CrossRef  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2014 Springer International Publishing Switzerland

About this paper

Cite this paper

Tran, B., Xue, B., Zhang, M. (2014). Overview of Particle Swarm Optimisation for Feature Selection in Classification. In: , et al. Simulated Evolution and Learning. SEAL 2014. Lecture Notes in Computer Science, vol 8886. Springer, Cham. https://doi.org/10.1007/978-3-319-13563-2_51

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-13563-2_51

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-13562-5

  • Online ISBN: 978-3-319-13563-2

  • eBook Packages: Computer ScienceComputer Science (R0)