Skip to main content

Simple Recurrent Neural Networks for Support Vector Machine Training

Part of the Lecture Notes in Computer Science book series (LNTCS,volume 11141)

Abstract

We show how to implement a simple procedure for support vector machine training as a recurrent neural network. Invoking the fact that support vector machines can be trained using Frank-Wolfe optimization which in turn can be seen as a form of reservoir computing, we obtain a model that is of simpler structure and can be implemented more easily than those proposed in previous contributions.

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-01424-7_2
  • Chapter length: 10 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   89.00
Price excludes VAT (USA)
  • ISBN: 978-3-030-01424-7
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   119.99
Price excludes VAT (USA)
Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.
Fig. 5.
Fig. 6.

References

  1. Anguita, D., Boni, A.: Improved neural network for SVM learning. IEEE Trans. Neural Netw. 13(2), 1243–1244 (2002)

    CrossRef  Google Scholar 

  2. Bauckhage, C.: A neural network implementation of Frank-Wolfe optimization. In: Lintas, A., Rovetta, S., Verschure, P., Villa, A. (eds.) ICANN 2017. LNCS, vol. 10613, pp. 219–226. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-68600-4_26

    CrossRef  Google Scholar 

  3. Bauckhage, C.: The dual problem of \(L_2\) SVM training. Technical report, ResearchGate (2018)

    Google Scholar 

  4. Bishop, C.: Neural Networks for Pattern Recognition. Oxford University Press, Oxford (1995)

    MATH  Google Scholar 

  5. Clarkson, K.: Coresets, sparse greedy approximation, and the Frank-Wolfe algorithm. ACM Trans. Algorithms 6(4), 63:1–63:30 (2010)

    MathSciNet  CrossRef  Google Scholar 

  6. Cortes, C., Vapnik, V.: Support vector networks. Mach. Learn. 20(3), 273–297 (1995)

    MATH  Google Scholar 

  7. Duch, W.: Support vector neural training. In: Duch, W., Kacprzyk, J., Oja, E., Zadrożny, S. (eds.) ICANN 2005. LNCS, vol. 3697, pp. 67–72. Springer, Heidelberg (2005). https://doi.org/10.1007/11550907_11

    CrossRef  Google Scholar 

  8. Frank, M., Wolfe, P.: An algorithm for quadratic programming. Nav. Res. Logist. Q. 3(1–2), 95–110 (1956)

    MathSciNet  CrossRef  Google Scholar 

  9. Jäger, H., Haas, H.: Harnessing nonlinearity: predicting chaotic systems and saving energy in wireless communication. Science 304(5667), 78–80 (2004)

    CrossRef  Google Scholar 

  10. Jaggi, M.: Revisiting Frank-Wolfe: projection-free sparse convex optimization. J. Mach. Learn. Res. 28(1), 427–435 (2013)

    Google Scholar 

  11. Jändel, M.: Biologically relevant neural network architectures for support vector machines. Neural Netw. 49, 39–50 (2014)

    CrossRef  Google Scholar 

  12. Koshiba, Y., Abe, S.: Comparison of L1 and L2 support vector machines. In: Proceedings IJCNN (2003)

    Google Scholar 

  13. Perfetti, R., Ricci, E.: Analogue neural network for support vector machine learning. IEEE Trans. Neural Netw. 17(4), 1085–1091 (2006)

    CrossRef  Google Scholar 

  14. Sifa, R.: An overview of Frank-Wolfe optimization for stochasticity constrained interpretable matrix and tensor factorization. In: ICANN 2018 (2018)

    CrossRef  Google Scholar 

  15. Tang, Y.: Deep Learning using Linear Support Vector Machines. arXiv:1306.0239 [cs.LG] (2013)

  16. Vincent, P., Bengio, Y.: A neural support vector network architecture with adaptive kernels. In: Proceedings IJCNN (2000)

    Google Scholar 

  17. Xia, Y.: A new neural network for solving linear and quadratic programming problems. IEEE Trans. Neural Netw. 7(6), 1544–1547 (1996)

    CrossRef  Google Scholar 

  18. Yang, Y., He, Q., Hu, X.: A compact neural network for training support vector machines. Neurocomputing 86, 193–198 (2012)

    CrossRef  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Rafet Sifa .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2018 Springer Nature Switzerland AG

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

Sifa, R., Paurat, D., Trabold, D., Bauckhage, C. (2018). Simple Recurrent Neural Networks for Support Vector Machine Training. In: Kůrková, V., Manolopoulos, Y., Hammer, B., Iliadis, L., Maglogiannis, I. (eds) Artificial Neural Networks and Machine Learning – ICANN 2018. ICANN 2018. Lecture Notes in Computer Science(), vol 11141. Springer, Cham. https://doi.org/10.1007/978-3-030-01424-7_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-01424-7_2

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-01423-0

  • Online ISBN: 978-3-030-01424-7

  • eBook Packages: Computer ScienceComputer Science (R0)