Hybrid Dynamic Learning Systems for Regression

Abstract

Methods of introducing diversity into ensemble learning predictors for regression problems are presented. Two methods are proposed in this paper, one involving pruning and the other a hybrid approach. In these ensemble learning approaches, diversity is introduced while simultaneously training, as part of the same learning process. Here not all members of the ensemble are trained in the same manner, but selectively trained, resulting in a diverse selection of ensemble members that have strengths in different parts of the training set. The result is that the prediction accuracy and generalization ability of the trained ensemble is enhanced. Pruning and hybrid heuristics attempt to combine accurate yet complementary members; therefore these methods enhance the performance by dynamically modifying the pruned aggregation through distributing the ensemble member selection over the entire dataset. A comparison is drawn with Negative Correlation Learning and a static ensemble pruning approach used in regression to highlight the performance improvement yielded by the dynamic methods. Experimental comparison is made using Multiple Layer Perceptron predictors on benchmark datasets, and on a signal calibration application.