A Novel Ensemble Approach for Improving Generalization Ability of Neural Networks

  • Lei Lu
  • Xiaoqin Zeng
  • Shengli Wu
  • Shuiming Zhong
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5326)


Ensemble learning is one of the main directions in machine learning and data mining, which allows learners to achieve higher training accuracy and better generalization ability. In this paper, with an aim at improving generalization performance, a novel approach to construct an ensemble of neural networks is proposed. The main contributions of the approach are its diversity measure for selecting diverse individual neural networks and weighted fusion technique for assigning proper weights to the selected individuals. Experimental results demonstrate that the proposed approach is effective.


Ensemble learning diversity sensitivity fusion clustering the second training 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Dietterich, T.G.: Ensemble Learning The Handbook of Brain Theory and Neural Networks, 2nd edn. The MIT Press, Cambridge (2002)Google Scholar
  2. 2.
    Hansen, L.K., Salamon, P.: Neural Network ensembles. IEEE Trans Pattern Analysis and Machine Intelligence 12(10), 993–1001 (1990)CrossRefGoogle Scholar
  3. 3.
    Sollich, P., Krogh, A.: Learning with ensembles: How over-fitting can be useful. In: Touretzky, D., Mozer, M., Hasselmo, M. (eds.) Advances in Neural Information Processing Systems, vol. 8, pp. 190–196. MIT Press, Cambridge (1996)Google Scholar
  4. 4.
    Sharkey, A.: Combining Artificial Neural Nets: Ensemble and Modular Multi-Nets Systems. Springer, London (1999)CrossRefzbMATHGoogle Scholar
  5. 5.
    Valiant, L.G.: A Theory of the learnable. Communications of the ACM 27(11), 1134–1142 (1984)CrossRefzbMATHGoogle Scholar
  6. 6.
    Breiman, L.: Bagging predictors. Machine Learning 24(2), 123–140 (1996)zbMATHGoogle Scholar
  7. 7.
    Schapire, R.E.: The strength of weak learnability. Machine Learning 5(2), 197–227 (1990)Google Scholar
  8. 8.
    Tang, J., Zeng, X., Lu, L.: Ensemble Learning Based on the Output Sensitivity of Multilayer Perceptrons. In: Proceedings of IEEE IJCNN, pp. 1067–1072 (2007)Google Scholar
  9. 9.
    Hagan, M.T., Demuth, H.B., Beale, M.H.: Neural Network Design. PWS Publishing Company (1996)Google Scholar
  10. 10.
    Zeng, X., Yeung, D.S.: A Quantified Sensitivity Measure for Multilayer Perceptron to Input Perturbation. Neural Computation 15, 183–211 (2003)CrossRefzbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Lei Lu
    • 1
  • Xiaoqin Zeng
    • 1
  • Shengli Wu
    • 2
  • Shuiming Zhong
    • 1
  1. 1.Department of Computer Science and EngineeringHohai UniversityNanjingChina
  2. 2.School of Computer and MathematicsUniversity of UlsterUK

Personalised recommendations