Introducing Reordering Algorithms to Classic Well-Known Ensembles to Improve Their Performance

  • Joaquín Torres-Sospedra
  • Carlos Hernández-Espinosa
  • Mercedes Fernández-Redondo
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7063)


Most of the well-known ensemble techniques use the same training algorithm and the same sequence of patterns from the learning set to adapt the trainable parameters (weights) of the neural networks in the ensemble. In this paper, we propose to replace the traditional training algorithm in which the sequence of patterns is kept unchanged during learning. With the new algorithms we want to add diversity to the ensemble and increase its accuracy by altering the sequence of patterns for each concrete network. Two new training set reordering strategies are proposed: Static reordering and Dynamic reordering. The new algorithms have been successfully tested with six different ensemble methods and the results show that reordering is a good alternative to traditional training.


Backpropagation Ensembles of ANN Reordering of Training set 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Asuncion, A., Newman, D.: UCI machine learning repository, University of California, Irvine, School of Information and Computer Sciences (2007)Google Scholar
  2. 2.
    Breiman, L.: Bagging predictors. Machine Learning 24(2), 123–140 (1996)zbMATHGoogle Scholar
  3. 3.
    Dietterich, T.G.: Ensemble Methods in Machine Learning. In: Kittler, J., Roli, F. (eds.) MCS 2000. LNCS, vol. 1857, pp. 1–15. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  4. 4.
    Fernández-Redondo, M., Hernández-Espinosa, C., Torres-Sospedra, J.: Multilayer Feedforward Ensembles for Classification Problems. In: Pal, N.R., Kasabov, N., Mudi, R.K., Pal, S., Parui, S.K. (eds.) ICONIP 2004. LNCS, vol. 3316, pp. 744–749. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  5. 5.
    Freund, Y., Schapire, R.E.: Experiments with a new boosting algorithm. In: International Conference on Machine Learning, pp. 148–156 (1996)Google Scholar
  6. 6.
    Hernández-Espinosa, C., Torres-Sospedra, J., Fernández-Redondo, M.: New experiments on ensembles of multilayer feedforward for classification problems. In: Proceedings of IJCNN 2005, pp. 1120–1124 (2005)Google Scholar
  7. 7.
    Kuncheva, L.I., Whitaker, C.J.: Using Diversity with Three Variants of Boosting: Aggressive, Conservative and Inverse. In: Roli, F., Kittler, J. (eds.) MCS 2002. LNCS, vol. 2364, pp. 81–90. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  8. 8.
    Liu, Y., Yao, X., Higuchi, T.: Evolutionary ensembles with negative correlation learning. IEEE Transactions on Evolutionary Computation 4(4), 380–387 (2000)CrossRefGoogle Scholar
  9. 9.
    Parmanto, B., Munro, P.W., Doyle, H.R.: Improving committee diagnosis with resampling techniques. In: Advances in Neural Information Processing Systems, pp. 882–888 (1996)Google Scholar
  10. 10.
    Rosen, B.E.: Ensemble learning using decorrelated neural networks. Connection Science 8(3-4), 373–384 (1996)CrossRefGoogle Scholar
  11. 11.
    Torres-Sospedra, J.: Ensembles of Artificial Neural Networks: Analysis and Development of Design Methods. Ph.D. thesis, Department of Computer Science and Engineering, Universitat Jaume I (2011)Google Scholar
  12. 12.
    Tumer, K., Ghosh, J.: Error correlation and error reduction in ensemble classifiers. Connection Science 8(3-4), 385–403 (1996)CrossRefGoogle Scholar
  13. 13.
    Verikas, A., Lipnickas, A., Malmqvist, K., Bacauskiene, M., Gelzinis, A.: Soft combination of neural classifiers: A comparative study. Pattern Recognition Letters 20(4), 429–444 (1999)CrossRefGoogle Scholar
  14. 14.
    Yildiz, O.T., Alpaydin, E.: Ordering and finding the best of k>2 supervised learning algorithms. IEEE T. Pattern. Anal. 28(3), 392–402 (2006)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Joaquín Torres-Sospedra
    • 1
  • Carlos Hernández-Espinosa
    • 1
  • Mercedes Fernández-Redondo
    • 1
  1. 1.Department of Computer Science and EngineeringUniversitat Jaume ICastellónSpain

Personalised recommendations