Advertisement

Stacking MF Networks to Combine the Outputs Provided by RBF Networks

  • Joaquín Torres-Sospedra
  • Carlos Hernández-Espinosa
  • Mercedes Fernández-Redondo
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4668)

Abstract

The performance of a Radial Basis Functions network (RBF) can be increased with the use of an ensemble of RBF networks because the RBF networks are successfully applied to solve classification problems and they can be trained by gradient descent algorithms. Reviewing the bibliography we can see that the performance of ensembles of Multilayer Feedforward (MF) networks can be improved by the use of the two combination methods based on Stacked Generalization described in [1]. We think that we could get a better classification system if we applied these combiners to an RBF ensemble. In this paper we satisfactory apply these two new methods, Stacked and Stacked+, on ensembles of RBF networks. Increasing the number of networks used in the combination module is also successfully proposed in this paper. The results show that training 3 MF networks to combine an RBF ensemble is the best alternative.

Keywords

Radial Basis Function Minimum Mean Square Error Radial Basis Function Neural Network Combination Method Radial Basis Function Network 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Torres-Sospedra, J., Hernndez-Espinosa, C., Fernndez-Redondo, M.: Combining MF networks: A comparison among statistical methods and stacked generalization. In: Schwenker, F., Marinai, S. (eds.) ANNPR 2006. LNCS (LNAI), vol. 4087, pp. 302–9743. Springer, Heidelberg (2006)Google Scholar
  2. 2.
    Hernndez-Espinosa, C., Fernndez-Redondo, M., Torres-Sospedra, J.: First experiments on ensembles of radial basis functions. In: Roli, F., Kittler, J., Windeatt, T. (eds.) MCS 2004. LNCS, vol. 3077, pp. 253–262. Springer, Heidelberg (2004)Google Scholar
  3. 3.
    Torres-Sospedra, J., Hernndez-Espinosa, C., Fernndez-Redondo, M.: An experimental study on training radial basis functions by gradient descent. In: Schwenker, F., Marinai, S. (eds.) ANNPR 2006. LNCS (LNAI), vol. 4087, pp. 302–9743. Springer, Heidelberg (2006)Google Scholar
  4. 4.
    Drucker, H., Cortes, C., Jackel, L.D., LeCun, Y., Vapnik, V.: Boosting and other ensemble methods. Neural Computation 6, 1289–1301 (1994)zbMATHCrossRefGoogle Scholar
  5. 5.
    Karayiannis, N.B.: Reformulated radial basis neural networks trained by gradient descent. IEEE Transactions on Neural Networks 10, 657–671 (1999)CrossRefGoogle Scholar
  6. 6.
    Karayiannis, N.B., Randoph-Gips, M.M.: On the construction and training of reformulated radial basis function neural networks. IEEE Transactions on Neural Networks 14, 835–846 (2003)CrossRefGoogle Scholar
  7. 7.
    Bishop, C.M.: Neural Networks for Pattern Recognition. Oxford University Press, New York (1995)Google Scholar
  8. 8.
    Kuncheva, L.I.: Combining Pattern Classifiers: Methods and Algorithms. Wiley-Interscience, Chichester (2004)zbMATHGoogle Scholar
  9. 9.
    Wolpert, D.H.: Stacked generalization. Neural Networks 5, 1289–1301 (1994)Google Scholar
  10. 10.
    Ghorbani, A.A., Owrangh, K.: Stacked generalization in neural networks: Generalization on statistically neutral problems. In: IJCNN 2001. Proceedings of the International Joint conference on Neural Networks, Washington DC, USA, pp. 1715–1720. IEEE Computer Society Press, Los Alamitos (2001)Google Scholar
  11. 11.
    Ting, K.M., Witten, I.H.: Stacked generalizations: When does it work? In: International Joint Conference on Artificial Intelligence proceedings, vol. 2, pp. 866–873 (1997)Google Scholar
  12. 12.
    Ting, K.M., Witten, I.H.: Issues in stacked generalization. Journal of Artificial Intelligence Research 10, 271–289 (1999)zbMATHGoogle Scholar
  13. 13.
    Newman, D.J., Hettich, S., Blake, C.L., Merz, C.J.: UCI repository of machine learning databases (1998), http://www.ics.uci.edu/~mlearn/MLRepository.html
  14. 14.
    Torres-Sospedra, J., Hernandez-Espinosa, C., Fernandez-Redondo, M.: A comparison of combination methods for ensembles of RBF networks. In: IJCNN 2005. Proceedings of International Conference on Neural Networks, Montreal, Canada, vol. 2, pp. 1137–1141 (2005)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  • Joaquín Torres-Sospedra
    • 1
  • Carlos Hernández-Espinosa
    • 1
  • Mercedes Fernández-Redondo
    • 1
  1. 1.Departamento de Ingenieria y Ciencia de los Computadores, Universitat Jaume I, Avda. Sos Baynat s/n, C.P. 12071, CastellonSpain

Personalised recommendations