Feature Selection Methods Comparison: Logistic Regression-Based Algorithm and Neural Network Tools

Abstract

Features selection of high-dimensional data is desirable, mainly when extensive data is used and generated more often. Currently considered research problems are related to the appropriate feature selection in a multidimensional space allowing the selection of only those relevant to the analyzed problem. The implemented and applied machine learning approach made it possible to recognize feature profiles to distinguish two classes of observations. Regarding methods based on logistic regression, 21 features were selected, and 10 features related to the examined problem were identified for neural networks. This made it possible to significantly reduce the dimensionality of the data from as many as 406 original dimensions. Moreover, the feature selection approaches allowed for consistent results; as many as eight features were common to both utilized methods. The application of the recognized profiles also made it possible to obtain very high classification quality metrics, which in the case of logistic regression both for feature selection and classification, amounted to almost 94% of the weighted classification accuracy while maintaining the F1score metric at a very high level of 93%. The achieved results indicate the high efficiency of the proposed feature selection solution. The presented results support the potential of the implemented logistic regression-based approach for solving feature selection and classification for two-class problems.