Two-Stage Discriminative Feature Selection

  • Xiaobin ZhiEmail author
  • Shaoru WuEmail author
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 1075)


Recently, a filter supervised feature selection method, namely discriminative feature selection (DFS), was proposed, which combines linear discriminant analysis (LDA) and sparsity regularization effectively. However, DFS is computationally expensive due to the use of eigenvalue decomposition on large matrix. In this paper, we propose a two-stage DFS method, namely TSDFS, to improve the efficiency and keep the accuracy of classification. A direct LDA based feature selection is firstly performed to achieve dimension reduction preprocessing of the data. Then, a DFS procedure is performed efficiently on the reduced data in the second stage. The high efficiency of the whole TSDFS is credited with the high efficiency of dimension reduction preprocessing. In addition, we employ nested cross-validation technology to achieve automatic parameter selection for TSDFS. Extensive experimental results based on several benchmark data sets validate the effectiveness of TSDFS.


Feature selection Linear Discriminant Analysis Two-stage method 



This work is supported by the National Science Foundation of China (Grant nos. 61671377, 61102095, 61571361 and 11401045), and the Science Plan Foundation of the Education Bureau of Shanxi Province (No. 18JK0719), and New Star Team of Xi’ an University of Posts and Telecommunications (Grant no. xyt2016-01).


  1. 1.
    Chang, C., Lin, C.: LIBSVM: a library for support vector machines. ACM Trans. Intell. Syst. Technol. 2(3), 27 (2011)CrossRefGoogle Scholar
  2. 2.
    Eroglu, D.Y., Kilic, K.: A novel Hybrid Genetic Local Search Algorithm for feature selection and weighting with an application in strategic decision making in innovation management. Inf. Sci. 405, 18–32 (2017)CrossRefGoogle Scholar
  3. 3.
    He, X., Cai, D., Niyogi, P.: Laplacian score for feature selection. In: Proceedings of the 18th International Conference on Neural Information Processing Systems, NIPS 2005, pp. 507–517 (2005)Google Scholar
  4. 4.
    Masaeli, M., Fung, G., Dy, J.G.: From transformation-based dimensionality reduction to feature selection. In: Proceedings of 27th International Conference on Machine Learning, ICML 2010, pp. 751–758. Omnipress, Haifa, Israel (2010)Google Scholar
  5. 5.
    Nie, F., Xiang, S., Jia, Y., Zhang, C., Yan, S.: Trace ratio criterion for feature selection. In: Proceedings of 23rd AAAI 2008 Conference on Artificial Intelligence, vol. 2, pp. 671–676. AAAI Press, Chicago, Illinois (2008)Google Scholar
  6. 6.
    Peng, H., Long, F., Ding, C.: Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy. IEEE Trans. Pattern Anal. Mach. Intell. 27(8), 1226–1238 (2005)CrossRefGoogle Scholar
  7. 7.
    Robnik-Šikonja, M., Kononenko, I.: Theoretical and empirical analysis of ReliefF and RReliefF. Mach. Learn. 53(1–2), 23–69 (2003)CrossRefGoogle Scholar
  8. 8.
    Statnikov, A., Tsamardinos, I., Dosbayev, Y., Aliferis, C.F.: GEMS: a system for automated cancer diagnosis and biomarker discovery from microarray gene expression data. Int. J. Med. Inform. 74(7–8), 491–503 (2005)CrossRefGoogle Scholar
  9. 9.
    Tao, H., Hou, C., Nie, F., Jiao, Y., Yi, D.: Effective discriminative feature selection with nontrivial solution. IEEE Trans. Neural Netw. Learn. Syst. 27(4), 796–808 (2016)MathSciNetCrossRefGoogle Scholar
  10. 10.
    Wang, L., Chu, F., Xie, W.: Accurate cancer classification using expressions of very few genes. IEEE/ACM Trans. Comput. Biol. Bioinform. 4(1), 40–53 (2007)CrossRefGoogle Scholar
  11. 11.
    Wang, L., Wang, Y., Chang, Q.: Feature selection methods for big data bioinformatics: a survey from the search perspective. Methods 111, 21–31 (2016)CrossRefGoogle Scholar
  12. 12.
    Wang, L., Zhou, N., Chu, F.: A general wrapper approach to selection of class-dependent features. IEEE Trans. Neural Netw. 19(7), 1267–1278 (2008)CrossRefGoogle Scholar
  13. 13.
    Xue, B., Zhang, M., Browne, W.N., Yao, X.: A survey on evolutionary computation approaches to feature selection. IEEE Trans. Evol. Comput. 20(4), 606–626 (2016)CrossRefGoogle Scholar
  14. 14.
    Ye, J., Li, Q.: A two-stage linear discriminant analysis via QR-decomposition. IEEE Trans. Pattern Anal. Mach. Intell. 27(6), 929–941 (2005)CrossRefGoogle Scholar
  15. 15.
    Zhao, M., Lin, M., Chiu, B., Zhang, Z., Tang, X.: Trace ratio criterion based discriminative feature selection via \(l_{2, p}\)-norm regularization for supervised learning. Neurocomputing 321, 1–16 (2018)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.School of ScienceXi’an University of Posts and TelecommunicationsXi’anChina
  2. 2.School of Communication and Information EngineeringXi’an University of Posts and TelecommunicationsXi’anChina

Personalised recommendations