Feature Selection Using Ensemble Techniques

Abstract

Data used in Machine Learning tasks need to be pre-processed and prepared to improve its quality. Features or variables in data play a major role in the results obtained after applying Machine Learning models. The features which are irrelevant to the domain should be discarded with the objective of improving the accuracy and validity of results. For this purpose, Feature Selection is used. It is a way of reducing the size, the purpose of settling down to the right element from the original elements by removing the negative, redundant or noisy features. Feature selection can often result in better learning performance, such as, lower computational cost, and better model translation. It is very important to shed some light on the nature of feature selection for student performance measurement, because constructive educational approaches can be found in the appropriate set of features. Feature selection plays a major role in refining the quality of the data models. Increased data quality can produce better results and therefore based options for such quality data can increase the quality of education by predicting performance. In the light of the aforementioned fact, it is necessary to carefully stabilize the selection of the algorithm. Feature selection key can directly affect classification accuracy and simplify operation. Several data experiments were performed to demonstrate the effectiveness of the proposed method. Selective bias may also be the term used in data analytics. In this paper, two ensemble techniques namely, Random Forests and Gradient Boosting Machines have been applied on a dataset, for the purpose of Feature Selection. Experimental results show that Gradient Boosting Machines are better at Feature Selection.