Hybrid Dynamic Learning Systems for Regression
Methods of introducing diversity into ensemble learning predictors for regression problems are presented. Two methods are proposed in this paper, one involving pruning and the other a hybrid approach. In these ensemble learning approaches, diversity is introduced while simultaneously training, as part of the same learning process. Here not all members of the ensemble are trained in the same manner, but selectively trained, resulting in a diverse selection of ensemble members that have strengths in different parts of the training set. The result is that the prediction accuracy and generalization ability of the trained ensemble is enhanced. Pruning and hybrid heuristics attempt to combine accurate yet complementary members; therefore these methods enhance the performance by dynamically modifying the pruned aggregation through distributing the ensemble member selection over the entire dataset. A comparison is drawn with Negative Correlation Learning and a static ensemble pruning approach used in regression to highlight the performance improvement yielded by the dynamic methods. Experimental comparison is made using Multiple Layer Perceptron predictors on benchmark datasets, and on a signal calibration application.
KeywordsEnsemble methods Ensemble pruning Ensemble learning Neural networks
Unable to display preview. Download preview PDF.
- 4.Olvera-Lopez J., Carrasco-Ochao J., Martinez-Trinidad J., Kittler J.: A review of instance selection methods. Artificial Intelligence Review 34(2) 133–143 (2010)Google Scholar
- 6.Chen H., Yao X.: Multiobjective Neural Network Ensembles Based on Regularized Negative Correlation Learning. IEEE Trans. Knowledge and Data Engineering 22(12), 1738–1751 (2010)Google Scholar
- 9.Dias K., Windeatt T.: Dynamic Ensemble Selection and Instantaneous Pruning for Regression. In: European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning, ESANN 2014, pp. 643–648 (2014)Google Scholar