DIVACE: Diverse and Accurate Ensemble Learning Algorithm

  • Arjun Chandra
  • Xin Yao
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3177)


In order for a neural network ensemble to generalise properly, two factors are considered vital. One is the diversity and the other is the accuracy of the networks that comprise the ensemble. There exists a tradeoff as to what should be the optimal measures of diversity and accuracy. The aim of this paper is to address this issue. We propose the DIVACE algorithm which tries to produce an ensemble as it searches for the optimum point on the diversity-accuracy curve. The DIVACE algorithm formulates the ensemble learning problem as a multi-objective problem explicitly.


Mutual Information Pareto Front Ensemble Member Member Network Neural Computation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Abbass, H.A.: A memetic pareto evolutionary approach to artificial neural networks. In: Proceedings of the 14th Australian Joint Conference on Artificial Intelligence, Berlin, pp. 1–12. Springer, Heidelberg (2000)Google Scholar
  2. 2.
    Abbass, H.A.: Pareto neuro-evolution: Constructing ensemble of neural networks using multi-objective optimization. In: The IEEE 2003 Conference on Evolutionary Computation, vol. 3, pp. 2074–2080. IEEE Press, Los Alamitos (2003)Google Scholar
  3. 3.
    Abbass, H.A.: Pareto neuro-ensemble. In: Gedeon, T(T.) D., Fung, L.C.C. (eds.) AI 2003. LNCS (LNAI), vol. 2903, pp. 554–566. Springer, Heidelberg (2003a)CrossRefGoogle Scholar
  4. 4.
    Abbass, H.A., Sarker, R., Newton, C.: PDE: A pareto-frontier differential evolution approach for multi-objective optimization problems. In: Proceedings of the IEEE Congress on Evolutionary Computation (CEC 2001), vol. 2, pp. 971–978. IEEE Press, Los Alamitos (2001)CrossRefGoogle Scholar
  5. 5.
    Brown, G., Wyatt, J., Harris, R., Yao, X.: Diversity creation methods: A survey and categorisation. Journal of Information Fusion (2004) (to appear)Google Scholar
  6. 6.
    Geman, S., Bienenstock, E., Doursat, R.: Neural networks and the bias/variance dilemma. Neural Computation 4(1), 1–58 (1992)CrossRefGoogle Scholar
  7. 7.
    Liu, Y., Yao, X., Higuchi, T.: Evolutionary ensembles with negative correlation learning. IEEE Transactions on Evolutionary Computation 4(4), 380–387 (2000)CrossRefGoogle Scholar
  8. 8.
    Liu, Y., Yao, X.: Ensemble learning via negative correlation. Neural Networks 12(10), 1399–1404 (1999)CrossRefGoogle Scholar
  9. 9.
    Liu, Y., Yao, X.: Learning and evolution by minimization of mutual information. In: Guervós, J.J.M., Adamidis, P.A., Beyer, H.-G., Fernández-Villacañas, J.-L., Schwefel, H.-P. (eds.) PPSN 2002. LNCS, vol. 2439, pp. 495–504. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  10. 10.
    Srinivas, N., Deb, K.: Multi-objective function optimization using nondominated sorting genetic algorithms. Evolutionary Computation 2(3), 221–248 (1994)CrossRefGoogle Scholar
  11. 11.
    Storn, R., Price, K.: Differential evolution - a simple and efficient adaptive scheme for global optimization over continuous spaces. Technical Report TR-95-012, International Computer Science Institute, Berkeley, USA (1995)Google Scholar
  12. 12.
    Yao, X., Liu, Y.: Evolving neural network ensembles by minimization of mutual information. International Journal of Hybrid Intelligent Systems 1(1) (January 2004)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2004

Authors and Affiliations

  • Arjun Chandra
    • 1
  • Xin Yao
    • 1
  1. 1.The Centre of Excellence for Research in Computational Intelligence and Applications (CERCIA), School of Computer ScienceThe University of BirminghamEdgbaston, BirminghamUK

Personalised recommendations