Quadratic Programming Optimization with Feature Selection for Nonlinear Models
- 3 Downloads
The paper is devoted to the problem of constructing a predictive model in the high-dimensional feature space. The space is redundant, there is multicollinearity in the design matrix columns. In this case the model is unstable to changes in data or in parameter values. To build a stable model, the authors solve the dimensionality reduction problem for the feature space. It is proposed to use feature selection methods during parameter optimization process. The idea is to select the active set of model parameters which have to be optimized in the current optimization step. Quadratic programming feature selection is used to find the active set of parameters. The algorithm maximizes the relevance of model parameters to the residuals and makes them pairwise independent. Nonlinear regression and logistic regression models are investigated. We carried out the experiment to show how the proposed method works and compare it with other methods. The proposed algorithm achieves the less error and greater stability with comparison to the other methods.
Keywords and phrasesquadratic programming feature selection nonlinear regression logistic regression Newton method
Unable to display preview. Download preview PDF.
- 3.D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv:1412 6980 (2014).Google Scholar
- 7.A. Botev, H. Ritter, and D. Barber, “Practical gauss-newton optimisation for deep learning,” in Proceedings of the International Conference on Machine Learning, 2017, pp. 557–565.Google Scholar
- 8.J. Li, K. Cheng, S. Wang, F. Morstatter, R. P. Trevino, J. Tang, and H. Liu, “Feature selection: a data perspective,” ACMComputing Surveys 50 (6), 94 (2017).Google Scholar
- 10.M. Yamada, A. Saha, H. Ouyang, D. Yin, and Y. Chang, “N3LARS: minimum redundancy maximum relevance feature selection for large and high-dimensional data,” arXiv:1411. 2331 (2014).Google Scholar
- 15.D. Dheeru and E. Karra Taniskidou, “UCI machine learning repository” (2017).Google Scholar