Ensemble Learning with Local Diversity

  • Ricardo Ñanculef
  • Carlos Valle
  • Héctor Allende
  • Claudio Moraga
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4131)


The concept of Diversity is now recognized as a key characteristic of successful ensembles of predictors. In this paper we investigate an algorithm to generate diversity locally in regression ensembles of neural networks, which is based on the idea of imposing a neighborhood relation over the set of learners. In this algorithm each predictor iteratively improves its state considering only information about the performance of the neighbors to generate a sort of local negative correlation. We will assess our technique on two real data sets and compare this with Negative Correlation Learning, an effective technique to get diverse ensembles. We will demonstrate that the local approach exhibits better or comparable results than this global one.


Neighborhood Size Neighborhood Relation Neural Network Ensemble Neighborhood Order Negative Correlation Learn 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Vedelsby, J., Krogh, A.: Neural network ensembles, cross-validation and active learning. Neural Information Processing Systems 7, 231–238 (1995)Google Scholar
  2. 2.
    Blake, C.L., Merz, C.J.: UCI repository of machine learning databases (1998)Google Scholar
  3. 3.
    Brown, G.: Diversity in neural network ensembles, Ph.D. thesis, School of Computer Science, University of Birmingham (2003)Google Scholar
  4. 4.
    Fahlman, S., Lebiere, C.: The cascade-correlation learning architecture. In: Advances in Neural Information Processing Systems, vol. 2, pp. 524–532. Morgan Kaufmann, San Francisco (1990)Google Scholar
  5. 5.
    Harris, R., Brown, G., Wyatt, J., Yao, X.: Diversity creation methods: A survey and categorisation. Information Fusion Journal (Special issue on Diversity in Multiple Classifier Systems) 6(1), 5–20 (2004)Google Scholar
  6. 6.
    Whitaker, C., Kuncheva, L.: Measures of diversity in classifier ensembles. Machine Learning 51, 181–207 (2003)MATHCrossRefGoogle Scholar
  7. 7.
    Kittler, J., Roli, F., Oza, N.C., Polikar, R. (eds.): MCS 2005. LNCS, vol. 3541. Springer, Heidelberg (2005)MATHGoogle Scholar
  8. 8.
    Druovec, T.W., Povalej, P., Kokol, P., Stiglic, B.: Machine-learning with cellular automata. In: Advances in Intelligent Data Analysis VI, vol. 1, pp. 305–315. Springer, Heidelberg (2005)Google Scholar
  9. 9.
    Rosen, B.: Ensemble learning using decorrelated neural networks. Connection Science 8(3-4), 373–384 (1999)Google Scholar
  10. 10.
    Vlachos, P.: StatLib datasets archive (2005)Google Scholar
  11. 11.
    Yao, X., Lui, Y.: Ensemble learning via negative correlation. Neural Networks 12(10), 1399–1404 (1999)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Ricardo Ñanculef
    • 1
  • Carlos Valle
    • 1
  • Héctor Allende
    • 1
  • Claudio Moraga
    • 2
    • 3
  1. 1.Departamento de InformáticaUniversidad Técnica Federico Santa MaríaValparaísoChile
  2. 2.European Centre for Soft ComputingMieres, AsturiasSpain
  3. 3.Dortmund UniversityDortmundGermany

Personalised recommendations