Two-Stage Discriminative Feature Selection
Recently, a filter supervised feature selection method, namely discriminative feature selection (DFS), was proposed, which combines linear discriminant analysis (LDA) and sparsity regularization effectively. However, DFS is computationally expensive due to the use of eigenvalue decomposition on large matrix. In this paper, we propose a two-stage DFS method, namely TSDFS, to improve the efficiency and keep the accuracy of classification. A direct LDA based feature selection is firstly performed to achieve dimension reduction preprocessing of the data. Then, a DFS procedure is performed efficiently on the reduced data in the second stage. The high efficiency of the whole TSDFS is credited with the high efficiency of dimension reduction preprocessing. In addition, we employ nested cross-validation technology to achieve automatic parameter selection for TSDFS. Extensive experimental results based on several benchmark data sets validate the effectiveness of TSDFS.
KeywordsFeature selection Linear Discriminant Analysis Two-stage method
This work is supported by the National Science Foundation of China (Grant nos. 61671377, 61102095, 61571361 and 11401045), and the Science Plan Foundation of the Education Bureau of Shanxi Province (No. 18JK0719), and New Star Team of Xi’ an University of Posts and Telecommunications (Grant no. xyt2016-01).
- 3.He, X., Cai, D., Niyogi, P.: Laplacian score for feature selection. In: Proceedings of the 18th International Conference on Neural Information Processing Systems, NIPS 2005, pp. 507–517 (2005)Google Scholar
- 4.Masaeli, M., Fung, G., Dy, J.G.: From transformation-based dimensionality reduction to feature selection. In: Proceedings of 27th International Conference on Machine Learning, ICML 2010, pp. 751–758. Omnipress, Haifa, Israel (2010)Google Scholar
- 5.Nie, F., Xiang, S., Jia, Y., Zhang, C., Yan, S.: Trace ratio criterion for feature selection. In: Proceedings of 23rd AAAI 2008 Conference on Artificial Intelligence, vol. 2, pp. 671–676. AAAI Press, Chicago, Illinois (2008)Google Scholar