Abstract
This work proposes a new representation learning technique called convolutional transform learning. In standard transform learning, a dense basis is learned that analyses the image to generate the representation from the image. Here, we learn a set of independent convolutional filters that operate on the images to produce representations (one corresponding to each filter). The major advantage of our proposed approach is that it is completely unsupervised; unlike CNNs where labeled images are required for training. Moreover, it relies on a well-sounded minimization technique with established convergence guarantees. We have compared the proposed method with dictionary learning and transform learning on standard image classification datasets. Results show that our method improves over the rest by a considerable margin.
This work was supported by the CNRS-CEFIPRA project under grant NextGenBP PRC2017.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
See also http://proximity-operator.net/.
References
Vincent, P., Larochelle, H., Lajoie, I., Bengio, Y., Manzagol, P.-A.: Stacked denoising autoencoders: learning useful representations in a deep network with a local denoising criterion. J. Mach. Learn. Res. 11, 3371–3408 (2010)
Kingma, D.P., Welling, M.: Auto-encoding variational bayes. In: Proceedings of ICLR, Banff, Canada, April 2014
Makhzani, A., Shlens, J., Jaitly, N., Goodfellow, I.: Adversarial autoencoders. In: Proceedings of ICLR, San Juan, Puerto Rico, May 2016
LeCun, Y., et al.: Backpropagation applied to handwritten zip code recognition. J. Neural Comput. 1(4), 541–551 (1989)
Lecun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86, 2278–2324 (1998)
Lee, D.D., Seung, H.S.: Learning the parts of objects by non-negative matrix factorization. J. Nature 401(6755), 788–791 (1999)
Aharon, M., Elad, M., Bruckstein, A.: K-SVD: an algorithm for designing overcomplete dictionaries for sparse representation. J. IEEE Trans. Signal Process. 54(11), 4311–4322 (2006)
Shekhar, M., Patel, S., Chellappa, R.: Analysis sparse coding models for image-based classification. In: Proceedings of ICIP, pp. 5207–5211, Paris, France (2014)
Guo, J., Guo, Y., Kong, X., Zhang, M., He, R.: Discriminative analysis dictionary learning. In: Proceedings of AAAI, pp. 1617–1623, Phoenix, AZ, USA (2016)
Huang, F., Anandkumar, A.: Convolutional dictionary learning through tensor factorization. In: NIPS Workshop: Feature Extraction, pp. 116–129, Montreal, Canada (2015)
Garcia-Cardona, C., Wohlberg, B.: Convolutional dictionary learning, Preprint arXiv:1709.02893 (2017)
Papyan, V., Romano, Y., Sulam, J., Elad, M.: Convolutional dictionary learning via local processing, Preprint arXiv:1705.03239 (2017)
Ravishankar, S., Bresler, Y.: Learning sparsifying transforms. J. IEEE Trans. Signal Process. 61(5), 1072–1086 (2013)
Ravishankar, S., Wen, B., Bresler, Y.: Online sparsifying transform learning - Part I. J. IEEE J. Sel. Topics Signal Process. 9(4), 625–636 (2015)
Ravishankar, S., Bresler, Y.: Online sparsifying transform learning - Part II. J. IEEE J. Sel. Topics Signal Process. 9(4), 637–646 (2015)
Chabiron, O., Malgouyres, F., Tourneret, J.Y., Dobigeon, N.: Toward fast transform learning. Int. J. Comput. Vis. 114, 195 (2015)
Maggu, J., Majumdar, A.: Kernel transform learning. J. Pattern Recognit. Lett. 98, 117–122 (2017)
Maggu, J., Majumdar, A.: Greedy deep transform learning. In: Proceedings of ICIP, Beijing, China (2017)
Maggu, J., Majumdar, A.: Unsupervised deep transform learning. In: Proceedings of ICASSP, Calgary, Canada (2018)
Fagot, D., Fevotte, C., Wendt, H.: Nonnegative Matrix Factorization with Transform Learning, Preprint arXiv:1705.04193, December 2017
Bauschke, H.H., Combettes, P.L.: Convex Analysis and Monotone Operator Theory in Hilbert Spaces, 2nd edn. Springer, New York (2017). https://doi.org/10.1007/978-3-319-48311-5
Chouzenoux, E., Pesquet, J.C., Repetti, A.: A block coordinate variable metric forward-backward algorithm. J. Global Optim. 66(3), 457–485 (2016)
Bolte, J., Sabach, S., Teboulle, M.: Proximal alternating linearized minimization for nonconvex and nonsmooth problems. J. Math. Program. 146(1–2), 459–494 (2014)
Attouch, H., Bolte, J., Svaiter, B.F.: Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward-backward splitting, and regularized Gauss-Seidel methods. J. Math. Program. 137(1), 91–129 (2013)
Moreau, J.J.: Proximité et dualité dans un espace hilbertien. J. Bull. Soc. Math. France 93, 273–299 (1965)
Combettes, P.L., Pesquet, J.C.: Proximal splitting methods in signal processing. In: Fixed-Point Algorithms for Inverse Problems in Science and Engineering, pp. 185–212. Springer, New York (2010). https://doi.org/10.1007/978-1-4419-9569-8_10
Bolte, J., Combettes, P.L., Pesquet, J.C.: Alternating proximal algorithm for blind image recovery. In: Proceedings of ICIP, pp. 1673–1676, Hong Kong, China (2010)
Chouzenoux, E., Benfenati, A., Pesquet, J.C.: A proximal approach for a class of matrix optimization problems, Tech. Rep. (2017). http://arxiv.org/abs/1801.07452
Bellhumer, P.N., Hespanha, J., Kriegman, D.: Eigenfaces vs. fisherfaces: recognition using class specific linear projection. J. IEEE Trans. Pattern Anal. Mach. Intell. 17(7), 711–720 (1997)
Lee, K.C., Ho, J., Kriegman, D.: Acquiring linear subspaces for face recognition under variable lighting. J. IEEE Trans. Pattern Anal. Mach. Intell. 27(5), 684–698 (2005)
Martinez, M., Benavente, R.: The AR face database, Tech. Rep., CVC 24 (1998)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Maggu, J., Chouzenoux, E., Chierchia, G., Majumdar, A. (2018). Convolutional Transform Learning. In: Cheng, L., Leung, A., Ozawa, S. (eds) Neural Information Processing. ICONIP 2018. Lecture Notes in Computer Science(), vol 11303. Springer, Cham. https://doi.org/10.1007/978-3-030-04182-3_15
Download citation
DOI: https://doi.org/10.1007/978-3-030-04182-3_15
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-04181-6
Online ISBN: 978-3-030-04182-3
eBook Packages: Computer ScienceComputer Science (R0)