Cooperative Recurrent Neural Network for Multiclass Support Vector Machine Learning

  • Ying Yu
  • Youshen Xia
  • Mohamed Kamel
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5552)


Binary classification problem can be reformulated as one optimization problem based on support vector machines and thus is well solved by one recurrent neural network (RNN). Multi-category classification problem in one-step method is then decomposed into two sub-optimization problems.In this paper, we first modify the sub-optimization problem about the bias so that its computation is reduced and its testing accuracy of classification is improved. We then propose a cooperative recurrent neural network (CRNN) for multiclass support vector machine learning. The proposed CRNN consists of two recurrent neural networks (RNNs) and each optimization problem is solved by one of the two RNNs. The proposed CRNN combines adaptively the two RNN models so that the global optimal solutions of the two optimization problems can be obtained. Furthermore, the convergence speed of the proposed CRNN is enhanced by a scaling technique. Computed results show the computational advantages of the proposed CRNN for multiclass SVM learning.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bennett, K.P., Mangasarian, O.L.: Multicategory Discrimination via Linear Programming. Optimization Methods and Software 3, 27–39 (1994)CrossRefGoogle Scholar
  2. 2.
    Vapnik, V.: The Nature of Statistica Learning Theory. Springer, Heidelberg (1995)CrossRefzbMATHGoogle Scholar
  3. 3.
    Cortes, C., Vapnik, V.: Support-vector Networks. Machine Learning 20, 273–297 (1995)zbMATHGoogle Scholar
  4. 4.
    Sukens, J.A.K., Vandewalle, J.: Least Square Support Machine Classifiers. Neural Process Letter 9, 293–300 (1999)CrossRefGoogle Scholar
  5. 5.
    Hsu, C.W., Lin, C.J.: A Comparison of Methods for Multiclass Support Vector Machines. IEEE Transactions on Neural Networks 13, 415–425 (2002)CrossRefGoogle Scholar
  6. 6.
    Bredensteiner, E.J., Bennett, K.P.: Multicategory Classification by Support Vector Machines. Computational Optimization and Applications 12, 53–79 (1999)MathSciNetCrossRefzbMATHGoogle Scholar
  7. 7.
    Jiang, J., Wu, C., Liang, Y.C.: Multicategory Classification by Least Squares Support Vector Regression. In: Wang, J., Liao, X.-F., Yi, Z. (eds.) ISNN 2005. LNCS, vol. 3496, pp. 863–868. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  8. 8.
    Anguita, D., Boni, A.: Improved Neural Network for SVM Learning. IEEE Transactions on Neural Networks 13, 1243–1244 (2002)CrossRefGoogle Scholar
  9. 9.
    Anand, R., Mehrotra, K., Mohan, C.K., Ranka, S.: Efficient Classification for Multiclass Problems using Modular Neural Networks. IEEE Transactions on Neural Networks 6, 117–124 (1995)CrossRefGoogle Scholar
  10. 10.
    Zhang, G.P.: Neural Networks for Classification: A Survey. IEEE Transactions on Systems, Man and Cybernetics - Part B 30, 451–459 (2000)CrossRefGoogle Scholar
  11. 11.
    Xia, Y.S., Wang, J.: A One-layer Recurrent Neural Network for Support Vector Machine Learning. IEEE Transactions on Systems, Man and Cybernetics - Part B, 1261–1269 (2004)Google Scholar
  12. 12.
    Auda, G., Kamel, M.S.: CMNN: Cooperative Modular Neural Networks for Pattern Recognition. Pattern Recognition Letters 18, 1391–1398 (1997)CrossRefzbMATHGoogle Scholar
  13. 13.
    Auda, G., Kamel, M.S.: CMNN: Cooperative Modular Neural Network. Neurocomputing 20, 189–207 (1998)CrossRefzbMATHGoogle Scholar
  14. 14.
    Caelli, T., Guan, L., Wen, W.: Modularity in Neural Computing. Proceedings of the IEEE 87, 1497–1518 (1999)CrossRefGoogle Scholar
  15. 15.
    Yang, S., Browne, A.: Neural Network Ensembles: Combing Multiple Models for Enhanced Performance Using a Multistage Approach. Expert Systems 21, 279–288 (2001)CrossRefGoogle Scholar
  16. 16.
    Xia, Y.S.: A New Neural Network for Solving Linear Programming Problems and its Applications. IEEE Transactions on Neural Networks 7, 525–529 (1996)CrossRefGoogle Scholar
  17. 17.
    Xia, Y.S.: An Extended Projection Neural Network for Constrained Optimization. Neural Computation 16(4), 863–883 (2004)CrossRefzbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Ying Yu
    • 1
  • Youshen Xia
    • 1
  • Mohamed Kamel
    • 2
  1. 1.College of Mathematics and Computer ScienceFuzhou UniversityFuzhouChina
  2. 2.Department of Electrical and Computer EngineeringUniversity of WaterlooCanada

Personalised recommendations