A Subset Similarity Guided Method for Multi-objective Feature Selection

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9592)


This paper presents a particle swarm optimisation (PSO) based multi-objective feature selection method for evolving a set of non-dominated feature subsets and achieving high classification performance. Firstly, a multi-objective PSO (named MOPSO-SRD) algorithm, is applied to solve feature selection problems. The results of this algorithm are then used to compare with the proposed multi-objective PSO algorithm, called MOPSO-SiD. MOPSO-SiD is specifically designed for feature selection problems, in which a subset similarity distance measure (distance in the solution space) is used to select a leader for each particle in the swarm. This distance measure is also used to update the archive set, which will be the final solutions returned by the MOPSO-SiD algorithm. The results show that both algorithms successfully evolve a set of non-dominated solutions, which include a small number of features while achieving similar or better performance than using all features. In addition, in most case MOPSO-SiD selects smaller feature subsets than MOPSO-SRD, and outperforms single objective PSO for feature selection and a traditional feature selection method.


Feature selection Classification Multi-objective optimisation Particle swarm optimisation 


  1. 1.
    Bache, K., Lichman, M.: Uci machine learning repository (2013).
  2. 2.
    Chuang, L.Y., Chang, H.W., Tu, C.J., Yang, C.H.: Improved binary PSO for feature selection using gene expression data. Comput. Biol. Chem. 32(1), 29–38 (2008)CrossRefzbMATHGoogle Scholar
  3. 3.
    Chuang, L.Y., Chang, H.W., Tu, C.J., Yang, C.H.: Improved binary PSO for feature selection using gene expression data. Comput. Biol. Chem. 32(29), 29–38 (2008)CrossRefzbMATHGoogle Scholar
  4. 4.
    Guyon, I., Elisseeff, A.: An introduction to variable and feature selection. J. Mach. Learn. Res. 3, 1157–1182 (2003)zbMATHGoogle Scholar
  5. 5.
    Kennedy, J., Eberhart, R., et al.: Particle swarm optimization. In: Proceedings of IEEE International Conference on Neural Networks, vol. 4, pp. 1942–1948 (1995)Google Scholar
  6. 6.
    Lane, M.C., Xue, B., Liu, I., Zhang, M.: Gaussian based particle swarm optimisation and statistical clustering for feature selection. In: Blum, C., Ochoa, G. (eds.) EvoCOP 2014. LNCS, vol. 8600, pp. 133–144. Springer, Heidelberg (2014) Google Scholar
  7. 7.
    Leung, M.F., Ng, S.C., Cheung, C.C., Lui, A.: A new strategy for finding good local guides in mopso. In: 2014 IEEE Congress on Evolutionary Computation (CEC), pp. 1990–1997, July 2014Google Scholar
  8. 8.
    Lin, F., Liang, D., Yeh, C.C., Huang, J.C.: Novel feature selection methods to financial distress prediction. Expert Syst. Appl. 41(5), 2472–2483 (2014)CrossRefGoogle Scholar
  9. 9.
    Liu, H., Motoda, H., Setiono, R., Zhao, Z.: Feature selection: an ever evolving frontier in data mining. In: JMLR Proceedings on Feature Selection for Data Mining, vol. 10, pp. 4–13 (2010).
  10. 10.
    Liu, Y., Wang, G., Chen, H., Dong, H.: An improved particle swarm optimization for feature selection. J. Bionic Eng. 8(2), 191–200 (2011)CrossRefGoogle Scholar
  11. 11.
    Marill, T., Green, D.M.: On the effectiveness of receptors in recognition systems. IEEE Trans. Inf. Theory 9(1), 11–17 (1963)CrossRefGoogle Scholar
  12. 12.
    Muni, D., Pal, N., Das, J.: Genetic programming for simultaneous feature selection and classifier design. IEEE Trans. Syst. Man Cybern. Part B Cybern. 36(1), 106–117 (2006)CrossRefGoogle Scholar
  13. 13.
    Purohit, A., Chaudhari, N., Tiwari, A.: Construction of classifier with feature selection based on genetic programming. In: IEEE Congress on Evolutionary Computation (CEC 2010), pp. 1–5 (2010)Google Scholar
  14. 14.
    Tran, B., Xue, B., Zhang, M.: Improved PSO for feature selection on high-dimensional datasets. In: Dick, G., et al. (eds.) SEAL 2014. LNCS, vol. 8886, pp. 503–515. Springer, Heidelberg (2014) Google Scholar
  15. 15.
    Van Den Bergh, F.: An analysis of particle swarm optimizers. Ph.D. thesis, University of Pretoria (2006)Google Scholar
  16. 16.
    Whitney, A.W.: A direct method of nonparametric measurement selection. IEEE Trans. Comput. 100(9), 1100–1103 (1971)CrossRefMathSciNetGoogle Scholar
  17. 17.
    Xue, B., Zhang, M., Browne, W.N.: Single feature ranking and binary particle swarm optimisation based feature subset ranking for feature selection. In: Australasian Computer Science Conference (ACSC 2012), vol. 122, pp. 27–36 (2012)Google Scholar
  18. 18.
    Xue, B., Cervante, L., Shang, L., Browne, W.N., Zhang, M.: Binary PSO and rough set theory for feature selection: a multi-objective filter based approach. Int. J. Comput. Intell. Appl. 13(02), 1450009:1–1450009:34 (2014)CrossRefGoogle Scholar
  19. 19.
    Xue, B., Cervante, L., Shang, L., Browne, W.N., Zhang, M.: A multi-objective particle swarm optimisation for filter-based feature selection in classification problems. Connect. Sci. 24(2–3), 91–116 (2012)CrossRefGoogle Scholar
  20. 20.
    Xue, B., Zhang, M., Browne, W.N.: A comprehensive comparison on evolutionary feature selection approaches to classification. Int. J. Comput. Intell. Appl. 14(02), 1550008 (2015)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  1. 1.School of Engineering and Computer ScienceVictoria University of WellingtonWellingtonNew Zealand

Personalised recommendations