Diverse Selection of Feature Subsets for Ensemble Regression

Abstract

Regression tasks such as forecasting of sensor values play a principal role in industrial applications. For instance, modern automobiles have hundreds of process variables which are used to predict target sensor values. Due to the complexity of these systems, each subset of features often shows different type of correlations with the target. Capturing such local interactions improve the regression models. Nevertheless, several existing feature selection algorithms focus on obtaining a single projection of the features and are not able to exploit the multiple local interactions from different subsets of variables. It is still an open challenge to efficiently select multiple subsets that not only contribute for the prediction quality, but are also diverse, i.e., subsets with complementary information. Such diverse subsets enrich the regression model with novel and essential knowledge by capturing the local interactions using multiple views of a high-dimensional feature space. In this work, we propose a framework to select multiple diverse subsets. First, our approach prunes the feature space by using the properties of multiple correlation measures. The pruned feature space is used to systematically generate new diverse combinations of feature subsets without decrease in the prediction quality. We show that our approach outperforms prevailing approaches on synthetic and several real world datasets from different application domains.