Skip to main content
SpringerLink
Log in

A Neural Network Implementation of Frank-Wolfe Optimization

  • Conference paper
  • First Online:
Artificial Neural Networks and Machine Learning – ICANN 2017 (ICANN 2017)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 10613))

Included in the following conference series:

Abstract

We revisit the Frank-Wolfe algorithm for constrained convex optimization and show that it can be implemented as a simple recurrent neural network with softmin activation functions. As an example for a practical application of this result, we discuss how to train such a network to act as an associative memory.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    http://www.ai.mit.edu/projects/cbcl.

References

  1. Bauckhage, C., Sifa, R.: k-Maxoids Clustering. In: Proceedings KDML-LWA (2015)

    Google Scholar 

  2. Bauckhage, C., Thurau, C.: Making archetypal analysis practical. In: Denzler, J., Notni, G., Süße, H. (eds.) DAGM 2009. LNCS, vol. 5748, pp. 272–281. Springer, Heidelberg (2009). doi:10.1007/978-3-642-03798-6_28

    Chapter  Google Scholar 

  3. Civril, A., Magdon-Ismail, M.: On selecting a maximum volume sub-matrix of a matrix and related problems. Theore. Comput. Sci. 410(47–49), 4801–4811 (2009)

    Article  MATH  MathSciNet  Google Scholar 

  4. Clarkson, K.: Coresets, sparse greedy approximation, and the Frank-Wolf algorithm. ACM Trans. Algorithms 6(4) (2010)

    Google Scholar 

  5. de Berg, M., van Kreveld, M., Overmars, M., Schwarzkopf, O.: Computational Geometry. Springer, Heidelberg (2000)

    Book  MATH  Google Scholar 

  6. Faloutsos, C., Lin, K.I.: FastMap: A fast algorithm for indexing, data-mining and visualization of traditional and multimedia datasets. In: Proceedings SIGMOD (1995)

    Google Scholar 

  7. Frank, M., Wolfe, P.: An algorithm for quadratic programming. Naval Research Logistics Quarterly 3(1–2), 95–110 (1956)

    Article  MathSciNet  Google Scholar 

  8. Jäger, H., Haas, H.: Harnessing nonlinearity: predicting chaotic systems and saving energy in wireless communication. Science 304(5667), 78–80 (2004)

    Article  Google Scholar 

  9. Jaggi, M.: Revisiting frank-wolfe: projection-free sparse convex optimization. J. Mach. Learn. Res. 28(1), I-427–I-435 (2013)

    Google Scholar 

  10. Kohonen, T., Oja, E., Lehtiö, P.: Storage and processing of information in distributed associative memory systems. In: Hinton, G., Anderson, J. (eds.) Parallel Models of Associative Memory. Laurence Earbaum Associates (1989). Chap. 4

    Google Scholar 

  11. Kosko, B.: Bidirectional associative memory. IEEE Trans. Systems Man Cybern. 18(1) (1988)

    Google Scholar 

  12. Lacoste-Julien, S., Jaggi, M., Schmidt, M., Pletscher, P.: Block-coordinate frank-wolfe optimization for structural SVMs. In: Proceedings ICML (2013)

    Google Scholar 

  13. McClelland, J., Rummelhart, D.: Distributed memory and the representation of general and specific information. J. Exp. Psychol. Gen. 114(2), 159–188 (1985)

    Article  Google Scholar 

  14. Ouyang, H., Gray, A.: Fast stochastic frank-wolfe algorithms for nonlinear SVMs. In: Proceedings SDM (2010)

    Google Scholar 

  15. Ziegler, G.: Lectures on Polytopes. Springer, New York (1995)

    Book  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. B-IT, University of Bonn, Bonn, Germany

    Christian Bauckhage

  2. Germany Fraunhofer IAIS, Sankt Augustin, Germany

    Christian Bauckhage

Authors
  1. Christian Bauckhage
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Christian Bauckhage .

Editor information

Editors and Affiliations

  1. University of Lausanne, Lausanne, Switzerland

    Alessandra Lintas

  2. University of Genoa, Genoa, Italy

    Stefano Rovetta

  3. Universitat Pompeu Fabra, Barcelona, Spain

    Paul F.M.J. Verschure

  4. University of Lausanne, Lausanne, Switzerland

    Alessandro E.P. Villa

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Bauckhage, C. (2017). A Neural Network Implementation of Frank-Wolfe Optimization. In: Lintas, A., Rovetta, S., Verschure, P., Villa, A. (eds) Artificial Neural Networks and Machine Learning – ICANN 2017. ICANN 2017. Lecture Notes in Computer Science(), vol 10613. Springer, Cham. https://doi.org/10.1007/978-3-319-68600-4_26

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-68600-4_26

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-68599-1

  • Online ISBN: 978-3-319-68600-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation