Abstract
Non-negative tensor factorization (NTF) has recently been proposed as sparse and efficient image representation (Welling and Weber, Patt. Rec. Let., 2001). Until now, sparsity of the tensor factorization has been empirically observed in many cases, but there was no systematic way to control it. In this work, we show that a sparsity measure recently proposed for non-negative matrix factorization (Hoyer, J. Mach. Learn. Res., 2004) applies to NTF and allows precise control over sparseness of the resulting factorization. We devise an algorithm based on sequential conic programming and show improved performance over classical NTF codes on artificial and on real-world data sets.
Chapter PDF
Similar content being viewed by others
Keywords
- Nonnegative Matrix Factorization
- Positive Matrix Factorization
- Tensor Factorization
- Sparsity Constraint
- Alternate Minimization
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Welling, M., Weber, M.: Positive tensor factorization. Pattern Recog. Letters 22(12), 1255–1261 (2001)
Shashua, A., Hazan, T.: Non-negative tensor factorization with applications to statistics and computer vision. In: Proc. of ICML (2005)
Hazan, T., Polak, S., Shashua, A.: Sparse image coding using a 3D non-negative tensor factorization. In: Proc. of ICCV (2005)
Shen, J., Israël, G.W.: A receptor model using a specific non-negative transformation technique for ambient aerosol. Atmospheric Environment 23(10), 2289–2298 (1989)
Paatero, P., Tapper, U.: Positive matrix factorization: A non-negative factor model with optimal utilization of error estimates of data values. Environmetrics 5, 111–126 (1994)
Lee, D.D., Seung, H.S.: Learning the parts of objects by non-negative matrix factorization. Nature 401, 788–791 (1999)
Shashua, A., Levin, A.: Linear image coding for regression and classification using the tensor-rank principle. In: Proc. of CVPR (2001)
Donoho, D., Stodden, V.: When does non-negative matrix factorization give a correct decomposition into parts? In: Adv. in NIPS, vol. 17 (2004)
Kruskal, J.B.: Three-way arrays: Rank and uniqueness of trilinear decompositions, with application to arithmetic complexity and statistics. Linear Algebra and its Applications 18, 95–138 (1977)
Sidiropoulos, N.D., Bro, R.: On the uniqueness of multilinear decompositions of N-way arrays. J. of Chemometrics 14, 229–239 (2000)
Hoyer, P.O.: Non-negative matrix factorization with sparseness constraints. J. of Mach. Learning Res. 5, 1457–1469 (2004)
Lee, D.D., Seung, H.S.: Algorithms for non-negative matrix factorization. In: Adv. in NIPS (2000)
Chu, M., Diele, F., Plemmons, R., Ragni, S.: Optimality, computations, and interpretation of nonnegative matrix factorizations. SIAM J. Mat. Anal. Appl. (submitted, 2004), www4.ncsu.edu:8030/~mtchu/Research/Papers/nnmf.pdf
Horst, R., Tuy, H.: Global Optimization. Springer, Berlin (1996)
Heiler, M., Schnörr, C.: Learning non-negative sparse image codes by convex programming. In: Proc. of ICCV (2005)
Littlestone, N., Warmuth, M.: Relating data compression, learnability, and the Vapnik-Chervonenkis dimension. Tech. Rep., Univ. of Calif. Santa Cruz (1986)
Herbrich, R., Williamson, R.C.: Algorithmic luckiness. J. of Mach. Learning Res. 3, 175–212 (2002)
Olshausen, B.A., Field, D.J.: Sparse coding with an overcomplete basis set: A strategy employed by V1? Vision Research 37, 3311–3325 (1997)
Lobo, M.S., Vandenberghe, L., Boyd, S., Lebret, H.: Applications of second-order cone programming. In: Linear Algebra and its Applications (1998)
Sturm, J.F.: Using SeDuMi 1.02, a Matlab toolbox for optimization over symmetric cones (updated version 1.05). Department of Econometrics, Tilburg University, Tilburg, The Netherlands (2001)
Mittelmann, H.: An independent benchmarking of SDP and SOCP solvers. Math. Programming, Series B 95(2), 407–430 (2003)
MOSEK ApS, Denmark, The MOSEK optimization tools version 3.2 (Revision 8) User’s manual and reference (2005)
Buchanan, A.M., Fitzgibbon, A.W.: Damped Newton algorithms for matrix factorization with missing data. In: CVPR 2005, vol. 2, pp. 316–322 (2005)
Tuy, H.: Convex programs with an additional reverse convex constraint. J. of Optim. Theory and Applic. 52, 463–486 (1987)
Rockafellar, R., Wets, R.-B.: Variational Analysis. Grundlehren der math. Wissenschaften, vol. 317. Springer, Heidelberg (1998)
CBCL: CBCL face database #1. MIT Center For Biological and Computational Learning (2000), http://cbcl.mit.edu/software-datasets
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Heiler, M., Schnörr, C. (2006). Controlling Sparseness in Non-negative Tensor Factorization. In: Leonardis, A., Bischof, H., Pinz, A. (eds) Computer Vision – ECCV 2006. ECCV 2006. Lecture Notes in Computer Science, vol 3951. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11744023_5
Download citation
DOI: https://doi.org/10.1007/11744023_5
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-33832-1
Online ISBN: 978-3-540-33833-8
eBook Packages: Computer ScienceComputer Science (R0)