Ensemble Learning with Local Diversity
The concept of Diversity is now recognized as a key characteristic of successful ensembles of predictors. In this paper we investigate an algorithm to generate diversity locally in regression ensembles of neural networks, which is based on the idea of imposing a neighborhood relation over the set of learners. In this algorithm each predictor iteratively improves its state considering only information about the performance of the neighbors to generate a sort of local negative correlation. We will assess our technique on two real data sets and compare this with Negative Correlation Learning, an effective technique to get diverse ensembles. We will demonstrate that the local approach exhibits better or comparable results than this global one.
KeywordsNeighborhood Size Neighborhood Relation Neural Network Ensemble Neighborhood Order Negative Correlation Learn
Unable to display preview. Download preview PDF.
- 1.Vedelsby, J., Krogh, A.: Neural network ensembles, cross-validation and active learning. Neural Information Processing Systems 7, 231–238 (1995)Google Scholar
- 2.Blake, C.L., Merz, C.J.: UCI repository of machine learning databases (1998)Google Scholar
- 3.Brown, G.: Diversity in neural network ensembles, Ph.D. thesis, School of Computer Science, University of Birmingham (2003)Google Scholar
- 4.Fahlman, S., Lebiere, C.: The cascade-correlation learning architecture. In: Advances in Neural Information Processing Systems, vol. 2, pp. 524–532. Morgan Kaufmann, San Francisco (1990)Google Scholar
- 5.Harris, R., Brown, G., Wyatt, J., Yao, X.: Diversity creation methods: A survey and categorisation. Information Fusion Journal (Special issue on Diversity in Multiple Classifier Systems) 6(1), 5–20 (2004)Google Scholar
- 8.Druovec, T.W., Povalej, P., Kokol, P., Stiglic, B.: Machine-learning with cellular automata. In: Advances in Intelligent Data Analysis VI, vol. 1, pp. 305–315. Springer, Heidelberg (2005)Google Scholar
- 9.Rosen, B.: Ensemble learning using decorrelated neural networks. Connection Science 8(3-4), 373–384 (1999)Google Scholar
- 10.Vlachos, P.: StatLib datasets archive (2005)Google Scholar