Abstract
We revisit the Frank-Wolfe algorithm for constrained convex optimization and show that it can be implemented as a simple recurrent neural network with softmin activation functions. As an example for a practical application of this result, we discuss how to train such a network to act as an associative memory.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
References
Bauckhage, C., Sifa, R.: k-Maxoids Clustering. In: Proceedings KDML-LWA (2015)
Bauckhage, C., Thurau, C.: Making archetypal analysis practical. In: Denzler, J., Notni, G., Süße, H. (eds.) DAGM 2009. LNCS, vol. 5748, pp. 272–281. Springer, Heidelberg (2009). doi:10.1007/978-3-642-03798-6_28
Civril, A., Magdon-Ismail, M.: On selecting a maximum volume sub-matrix of a matrix and related problems. Theore. Comput. Sci. 410(47–49), 4801–4811 (2009)
Clarkson, K.: Coresets, sparse greedy approximation, and the Frank-Wolf algorithm. ACM Trans. Algorithms 6(4) (2010)
de Berg, M., van Kreveld, M., Overmars, M., Schwarzkopf, O.: Computational Geometry. Springer, Heidelberg (2000)
Faloutsos, C., Lin, K.I.: FastMap: A fast algorithm for indexing, data-mining and visualization of traditional and multimedia datasets. In: Proceedings SIGMOD (1995)
Frank, M., Wolfe, P.: An algorithm for quadratic programming. Naval Research Logistics Quarterly 3(1–2), 95–110 (1956)
Jäger, H., Haas, H.: Harnessing nonlinearity: predicting chaotic systems and saving energy in wireless communication. Science 304(5667), 78–80 (2004)
Jaggi, M.: Revisiting frank-wolfe: projection-free sparse convex optimization. J. Mach. Learn. Res. 28(1), I-427–I-435 (2013)
Kohonen, T., Oja, E., Lehtiö, P.: Storage and processing of information in distributed associative memory systems. In: Hinton, G., Anderson, J. (eds.) Parallel Models of Associative Memory. Laurence Earbaum Associates (1989). Chap. 4
Kosko, B.: Bidirectional associative memory. IEEE Trans. Systems Man Cybern. 18(1) (1988)
Lacoste-Julien, S., Jaggi, M., Schmidt, M., Pletscher, P.: Block-coordinate frank-wolfe optimization for structural SVMs. In: Proceedings ICML (2013)
McClelland, J., Rummelhart, D.: Distributed memory and the representation of general and specific information. J. Exp. Psychol. Gen. 114(2), 159–188 (1985)
Ouyang, H., Gray, A.: Fast stochastic frank-wolfe algorithms for nonlinear SVMs. In: Proceedings SDM (2010)
Ziegler, G.: Lectures on Polytopes. Springer, New York (1995)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Bauckhage, C. (2017). A Neural Network Implementation of Frank-Wolfe Optimization. In: Lintas, A., Rovetta, S., Verschure, P., Villa, A. (eds) Artificial Neural Networks and Machine Learning – ICANN 2017. ICANN 2017. Lecture Notes in Computer Science(), vol 10613. Springer, Cham. https://doi.org/10.1007/978-3-319-68600-4_26
Download citation
DOI: https://doi.org/10.1007/978-3-319-68600-4_26
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-68599-1
Online ISBN: 978-3-319-68600-4
eBook Packages: Computer ScienceComputer Science (R0)