Dynamic Forest for Learning from Data Streams with Varying Feature Spaces

Abstract

In this paper, we propose a new ensemble method, which is called Dynamic Forest, for learning from data streams with varying feature spaces. Unlike traditional online learning where the feature space is static, in varying feature spaces, new features may emerge while others may vanish. This leads to several problems for which state-of-the-art online random forest algorithms are not equipped. We benchmark our proposed method against the state-of-the-art method OLVF on data streams with varying feature spaces and against OLVF and $OL_{SF}$ on trapezoidal data streams. These trapezoidal data streams can be considered as a sub-problem of varying feature spaces, where the only characteristic is that new features emerge over time. Our proposed approach dynamically learns and relearns decision stumps while applying a dynamic weighting strategy for the decision stumps. Furthermore, it employs a dynamic strategy for adding and removing weak learners. The proposed method is empirically evaluated by replicating the benchmark of the OLVF algorithm with nine UCI Machine Learning Repository datasets and one real-world dataset. In the experiments, we can show that Dynamic Forest proves to be a good addition to the current state-of-the-art for learning from data streams with varying feature spaces.