Local Negative Correlation with Resampling
This paper deals with a learning algorithm which combines two well known methods to generate ensemble diversity – error negative correlation and resampling. In this algorithm, a set of learners iteratively and synchronously improve their state considering information about the performance of a fixed number of other learners in the ensemble, to generate a sort of local negative correlation. Resampling allows the base algorithm to control the impact of highly influential data points which in turns can improve its generalization error. The resulting algorithm can be viewed as a generalization of bagging, where each learner no longer is independent but can be locally coupled with other learners. We will demonstrate our technique on two real data sets using neural networks ensembles.
KeywordsTraining Pattern Generalization Error Ensemble Learning Neighborhood Function Individual Error
Unable to display preview. Download preview PDF.
- 1.Blake, C.L., Merz, C.J.: UCI repository of machine learning databases (1998)Google Scholar
- 3.Brown, G.: Diversity in neural network ensembles, Ph.D. thesis, School of Computer Science, University of Birmingham (2003)Google Scholar
- 4.Harris, R., Brown, G., Wyatt, J., Yao, X.: Diversity creation methods: A survey and categorisation. Information Fusion Journal (Special issue on Diversity in Multiple Classifier Systems) 6(1), 5–20 (2004)Google Scholar
- 5.Grandvalet, Y.: Bagging down-weights leverage points. In: IJCNN, vol. 4, pp. 505–510 (2000)Google Scholar
- 6.Goandvalet, Y.: Bagging equalizes influence. Machine Learning 55(3), 251–270 (2004)Google Scholar
- 8.Allende, H., Ñanculef, R., Valle, C., Moraga, C.: Ensemble learning with local diversity. In: ICANN 2006 (2006) (to appear)Google Scholar
- 9.Rosen, B.: Ensemble learning using decorrelated neural networks. Connection Science (Special Issue on Combining Artificial Neural Networks: Ensemble Approaches) 8(3-4), 373–384 (1999)Google Scholar
- 10.Rifkin, R., Poggio, T., Mukherjee, S.: Bagging regularizes, Tech. Report 214/AI Memo 2002-003, MIT CBCL (2002)Google Scholar
- 11.Tikhonov, A., Arsenin, V.: Solutions of ill-posed problems. Winston (1977)Google Scholar
- 13.Vlachos, P.: StatLib datasets archive (2005)Google Scholar