Researching on Multi-net Systems Based on Stacked Generalization

  • Carlos Hernández-Espinosa
  • Joaquín Torres-Sospedra
  • Mercedes Fernández-Redondo
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5064)


Among the approaches to build a Multi-Net system, Stacked Generalization is a well-known model. The classification system is divided into two steps. Firstly, the level-0 generalizers are built using the original input data and the class label. Secondly, the level-1 generalizers networks are built using the outputs of the level-0 generalizers and the class label. Then, the model is ready for pattern recognition. We have found two important adaptations of Stacked Generalization that can be applyied to artificial neural networks. Moreover, two combination methods, Stacked and Stacked+, based on the Stacked Generalization idea were successfully introduced by our research group. In this paper, we want to empirically compare the version of the original Stacked Generalization along with other traditional methodologies to build Multi-Net systems. Moreover, we have also compared the combiners we proposed. The best results are provided by the combiners Stacked and Stacked+ when they are applied to ensembles previously trained with Simple Ensemble.


  1. 1.
    Bishop, C.M.: Neural Networks for Pattern Recognition. Oxford University Press, Inc., New York (1995)Google Scholar
  2. 2.
    Fernndez-Redondo, M., Hernndez-Espinosa, C., Torres-Sospedra, J.: Multilayer feedforward ensembles for classification problems. In: Pal, N.R., Kasabov, N., Mudi, R.K., Pal, S., Parui, S.K. (eds.) ICONIP 2004. LNCS, vol. 3316, pp. 744–749. Springer, Heidelberg (2004)Google Scholar
  3. 3.
    Ghorbani, A.A., Owrangh, K.: Stacked generalization in neural networks: Generalization on statistically neutral problems. In: Proceedings of the International Joint conference on Neural Networks, IJCNN 2001, Washington D.C., USA, pp. 1715–1720. IEEE, Los Alamitos (2001)CrossRefGoogle Scholar
  4. 4.
    Hernndez-Espinosa, C., Torres-Sospedra, J., Fernndez-Redondo, M.: New experiments on ensembles of multilayer feedforward for classification problems. In: IJCNN 2005 proceedings, pp. 1120–1124 (2005)Google Scholar
  5. 5.
    Kuncheva, L.I.: Combining Pattern Classifiers: Methods and Algorithms. Wiley-Interscience, Chichester (2004)zbMATHGoogle Scholar
  6. 6.
    Newman, D.J., Hettich, S., Blake, C.L., Merz, C.J.: UCI repository of machine learning databases (1998),
  7. 7.
    Raviv, Y., Intratorr, N.: Bootstrapping with noise: An effective regularization technique. Connection Science, Special issue on Combining Estimators 8, 356–372 (1996)Google Scholar
  8. 8.
    Sharkey, A.J. (ed.): Combining Artificial Neural Nets: Ensemble and Modular Multi-Net Systems. Springer, Heidelberg (1999)zbMATHGoogle Scholar
  9. 9.
    Ting, K.M., Witten, I.H.: Stacked generalizations: When does it work? In: International Joint Conference on Artificial Intelligence proceedings, vol. 2, pp. 866–873 (1997)Google Scholar
  10. 10.
    Ting, K.M., Witten, I.H.: Issues in stacked generalization. Journal of Artificial Intelligence Research 10, 271–289 (1999)zbMATHGoogle Scholar
  11. 11.
    Torres-Sospedra, J., Hernndez-Espinosa, C., Fernndez-Redondo, M.: Combining MF networks: A comparison among statistical methods and stacked generalization. In: Schwenker, F., Marinai, S. (eds.) ANNPR 2006. LNCS (LNAI), vol. 4087, pp. 210–220. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  12. 12.
    Torres-Sospedra, J., Hernndez-Espinosa, C., Fernndez-Redondo, M.: Designing a new multilayer feedforward modular network for classification problems. In: WCCI 2006 proceedings, pp. 2263–2268 (2006)Google Scholar
  13. 13.
    Tumer, K., Ghosh, J.: Error correlation and error reduction in ensemble classifiers. Connection Science 8(3-4), 385–403 (1996)CrossRefGoogle Scholar
  14. 14.
    Wolpert, D.H.: Stacked generalization. Neural Networks 5(6), 1289–1301 (1994)MathSciNetGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Carlos Hernández-Espinosa
    • 1
  • Joaquín Torres-Sospedra
    • 1
  • Mercedes Fernández-Redondo
    • 1
  1. 1.Departamento de Ingenieria y Ciencia de los ComputadoresUniversitat Jaume ICastellonSpain

Personalised recommendations