Advertisement

Gaussian Transformation Based Representation in Particle Swarm Optimisation for Feature Selection

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9028)

Abstract

In classification, feature selection is an important but challenging task, which requires a powerful search technique. Particle swarm optimisation (PSO) has recently gained much attention for solving feature selection problems, but the current representation typically forms a high-dimensional search space. A new representation based on feature clusters was recently proposed to reduce the dimensionality and improve the performance, but it does not form a smooth fitness landscape, which may limit the performance of PSO. This paper proposes a new Gaussian based transformation rule for interpreting a particle as a feature subset, which is combined with the feature cluster based representation to develop a new PSO-based feature selection algorithm. The proposed algorithm is examined and compared with two recent PSO-based algorithms, where the first uses a Gaussian based updating mechanism and the conventional representation, and the second uses the feature cluster representation without using Gaussian distribution. Experiments on commonly used datasets of varying difficulty show that the proposed algorithm achieves better performance than the other two algorithms in terms of the classification performance and the number of features in both the training sets and the test sets. Further analyses show that the Gaussian transformation rule improves the stability, i.e. selecting similar features in different independent runs and almost always selects the most important features.

Keywords

Particle swarm optimisation Feature selection Representation Gaussian distribution Classification 

References

  1. 1.
    Guyon, I., Elisseeff, A.: An introduction to variable and feature selection. J. Mach. Learn. Res. 3, 1157–1182 (2003)zbMATHGoogle Scholar
  2. 2.
    Liu, H., Motoda, H., Setiono, R., Zhao, Z.: Feature selection: an ever evolving frontier in data mining. In: FSDM. JMLR Proceedings, vol. 10, pp. 4–13 (2010)Google Scholar
  3. 3.
    Matechou, E., Liu, I., Pledger, S., Arnold, R.: Biclustering models for ordinal data. Presentation at the NZ Statistical Assn. Annual Conference, University of Auckland (2011)Google Scholar
  4. 4.
    Pledger, S., Arnold, R.: Multivariate methods using mixtures: correspondence analysis, scaling and pattern-detection. Comput. Stat. Data Anal. 71, 241–261 (2014)CrossRefMathSciNetGoogle Scholar
  5. 5.
    Lane, M.C., Xue, B., Liu, I., Zhang, M.: Gaussian based particle swarm optimisation and statistical clustering for feature selection. In: Blum, C., Ochoa, G. (eds.) EvoCOP 2014. LNCS, vol. 8600, pp. 133–144. Springer, Heidelberg (2014) CrossRefGoogle Scholar
  6. 6.
    Nguyen, H.B., Xue, B., Liu, I., Zhang, M.: PSO and statistical clustering for feature selection: a new representation. In: Dick, G., et al. (eds.) SEAL 2014. LNCS, vol. 8886, pp. 569–581. Springer, Heidelberg (2014) CrossRefGoogle Scholar
  7. 7.
    Zhu, Z., Ong, Y.S., Dash, M.: Markov blanket-embedded genetic algorithm for gene selection. Pattern Recogn. 40(11), 3236–3248 (2007)CrossRefzbMATHGoogle Scholar
  8. 8.
    Xue, B., Zhang, M., Browne, W.N.: Novel initialisation and updating mechanisms in PSO for feature selection in classification. In: Esparcia-Alcázar, A.I. (ed.) EvoApplications 2013. LNCS, vol. 7835, pp. 428–438. Springer, Heidelberg (2013) CrossRefGoogle Scholar
  9. 9.
    Boubezoul, A., Paris, S.: Application of global optimization methods to model and feature selection. Pattern Recogn. 45(10), 3676–3686 (2012)CrossRefzbMATHGoogle Scholar
  10. 10.
    Xue, B., Zhang, M., Browne, W.N.: Particle swarm optimization for feature selection in classification: a multi-objective approach. IEEE Trans. Cybern. 43(6), 1656–1671 (2013)CrossRefGoogle Scholar
  11. 11.
    Vieira, S.M., Mendonça, L.F., Farinha, G.J., Sousa, J.M.: Modified binary PSO for feature selection using SVM applied to mortality prediction of septic patients. Appl. Soft Comput. 13(5), 3494–3504 (2013)CrossRefGoogle Scholar
  12. 12.
    Xue, B., Fu, W., Zhang, M.: Multi-objective feature selection in classification: a differential evolution approach. In: Dick, G., et al. (eds.) SEAL 2014. LNCS, vol. 8886, pp. 516–528. Springer, Heidelberg (2014) CrossRefGoogle Scholar
  13. 13.
    Kennedy, J., Eberhart, R.: Particle swarm optimization. In: IEEE International Conference on Neural Networks, vol. 4, pp. 1942–1948 (1995)Google Scholar
  14. 14.
    Lin, S.W., Ying, K.C., Chen, S.C., Lee, Z.J.: Particle swarm optimization for parameter determination and feature selection of support vector machines. Expert Syst. Appl. 35(4), 1817–1824 (2008)CrossRefGoogle Scholar
  15. 15.
    Bache, K., Lichman, M.: Uci machine learning repository (2013)Google Scholar
  16. 16.
    Clerc, M., Kennedy, J.: The particle swarm- explosion, stability, and convergence in a multidimensional complex space. IEEE Trans. Evol. Comput. 6(1), 58–73 (2002)CrossRefGoogle Scholar
  17. 17.
    Xue, B., Zhang, M., Browne, W.N.: Single feature ranking and binary PSO based feature subset ranking for feature selection. In: Australasian Computer Science Conference (ACSC 2012), vol. 122, pp. 27–36 (2012)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  1. 1.School of Engineering and Computer ScienceVictoria University of WellingtonWellingtonNew Zealand
  2. 2.School of Mathematics, Statistics and Operations ResearchVictoria University of WellingtonWellingtonNew Zealand

Personalised recommendations