A proximal ANLS algorithm for nonnegative tensor factorization with a periodic enhanced line search Authors Douglas Bunker Department of Mathematics University of Michigan-Flint Lixing Han Department of Mathematics University of Michigan-Flint Shuhua Zhang Research Center for Mathematics and Economics Tianjin University of Finance and Economics Article
First Online: 21 September 2013 Received: 27 June 2011 DOI :
10.1007/s10492-013-0026-2
Cite this article as: Bunker, D., Han, L. & Zhang, S. Appl Math (2013) 58: 493. doi:10.1007/s10492-013-0026-2 Abstract The Alternating Nonnegative Least Squares (ANLS) method is commonly used for solving nonnegative tensor factorization problems. In this paper, we focus on algorithmic improvement of this method. We present a Proximal ANLS (PANLS) algorithm to enforce convergence. To speed up the PANLS method, we propose to combine it with a periodic enhanced line search strategy. The resulting algorithm, PANLS/PELS, converges to a critical point of the nonnegative tensor factorization problem under mild conditions. We also provide some numerical results comparing the ANLS and PANLS/PELS methods.
Keywords nonnegative tensor factorization proximal method alternating least squares enhanced line search global convergence
References [1]
B. W. Bader, T. G. Kolda :
Tensor toolbox for Matlab, version 2.2 .
http://csmr.ca.sandia.gov/~tgkolda/TensorToolbox/ (2008).
[2]
R. Bro : PARAFAC. Tutorial and applications. Chemometrics and Intelligent Laboratory Systems
38 (1997), 149–171.
CrossRef [3]
R. Bro : Multi-way Analysis in the Food Industry: Models, Algorithms, and Applications. Ph.D. thesis. University of Amsterdam, Amsterdam, 1998.
[4]
R. Bro : Exploratory study of sugar production using fluorescence spectroscopy and multi-way analysis. Chemometrics and Intelligent Laboratory Systems
46 (1999), 133–147.
CrossRef [5]
J. D. Carroll, J.-J. Chang : Analysis of individual differences in multidimensional scaling via an N-way generalization of ‘Eckart-Young’ decomposition. Psychometrika
35 (1970), 283–319.
CrossRef MATH [6]
A. Cichocki, R. Zdunek, S. Amari : Nonnegative matrix and tensor factorization. IEEE Signal Processing Magazine
25 (2008), 142–145.
CrossRef [7]
A. Cichocki, R. Zdunek, S. Choi, R. Plemmons, S. Amari : Non-negative tensor factorization using alpha and beta divergences. Proc. of the 32nd International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Honolulu, April 2007.
[8]
A. Cichocki, R. Zdunek, S. Choi, R. Plemmons, S. Amari : Novel multi-layer non-negative tensor factorization with sparsity constraints. Adaptive and Natural Computing Algorithms. Lecture Notes in Computer Science 4432. Proc. of the 8th International Conference on Adaptive and Natural Computing Algorithms, Warsaw, Poland, April 2007, Springer, Berlin.
[9]
A. Cichocki, R. Zdunek, A. H. Phan, S. Amari : Nonnegative Matrix and Tensor Factorizations, Applications to Exploratory Multi-way Data Analysis and Blind Source Separation. Wiley, Chichester, 2009.
CrossRef [10]
P. Comon, X. Luciani, A. L. F. de Almeida : Tensor decompositions, alternating least squares and other tales.
Journal of Chemometrics 23 (2009), 393–405.
CrossRef [11]
I. S. Dhillon : Fast Newton-type Methods for Nonnegative Matrix and Tensor Approximation. Talk given at the NSF Workshop, Future Directions in Tensor-Based Computation and Modeling, February 2009.
[12]
M. P. Friedlander, K. Hatz : Computing non-negative tensor factorizations. Optim. Methods Softw.
23 (2008), 631–647.
MathSciNet CrossRef MATH [13]
L. Grippo, M. Sciandrone : On the convergence of the block nonlinear Gauss-Seidel method under convex constraints. Oper. Res. Lett.
26 (2000), 127–136.
MathSciNet CrossRef MATH [14]
L. Han, M. Neumann, U. Prasad : Alternating projected Barzilai-Borwein methods for nonnegative matrix factorization. ETNA, Electron. Trans. Numer. Anal.
36 (2009–2010), 54–82, electronic only.
MathSciNet MATH [15]
R. A. Harshman : Foundations of the PARAFAC procedure: Models and conditions for an “explanatory” multi-modal factor analysis. UCLA Working Papers in Phonetics
16 (1970), 1–84,
http://www.psychology.uwo.ca/faculty/harshman/wpppfac0.pdf .
[16]
H. Kim, H. Park, L. Elden : Non-negative tensor factorization based on alternating large-scale non-negativity-constrained least squares. Proceedings of IEEE 7th International Conference on Bioinformatics and Bioengineering (BIBE07), Vol. II. 2007, pp. 1147–1151.
[17]
T. G. Kolda, B. W. Bader : Tensor decompositions and applications. SIAM Rev.
51 (2009), 455–500.
MathSciNet CrossRef MATH [18]
D. D. Lee, H. S. Seung : Learning the parts of objects by non-negative matrix factorization. Nature
401 (1999), 788–791.
CrossRef [19]
L. Lim, P. Comon : Nonnegative approximations of nonnegative tensors. Journal of Chemometrics
23 (2009), 432–441.
CrossRef [20]
B. C. Mitchell, D. S. Burdick : Slowly converging PARAFAC sequences: Swamps and two-factor degeneracies. Journal of Chemometrics
8 (1994), 155–168.
CrossRef [21]
M. Mørup, L. K. Hansen, S. M. Arnfred : Algorithms for sparse nonnegative Tucker decompositions. Neural Comput.
20 (2008), 2112–2131.
CrossRef [22]
D. Nion, L. De Lathauwer : An enhanced line search scheme for complex-valued tensor decompositions. Application in DS-CDMA. Signal Process.
88 (2008), 749–755.
CrossRef MATH [23]
P. Paatero : A weighted non-negative least squares algorithm for three-way ‘PARAFAC’ factor analysis. Chemometrics and Intelligent Laboratory Systems
38 (1997), 223–242.
CrossRef [24]
P. Paatero, U. Tapper : Positive matrix factorization: A non-negative factor model with optimal utilization of error estimates of data values. Environmetrics
5 (1994), 111–126.
CrossRef [25]
M. Rajih, P. Comon, R. A. Harshman : Enhanced line search: a novel method to accelerate PARAFAC. SIAM J. Matrix Anal. Appl.
30 (2008), 1128–1147.
MathSciNet CrossRef [26]
W. S. Rayens, B. C. Mitchell : Two-factor degeneracies and a stabilization of PARAFAC. Chemometrics and Intelligent Laboratory Systems
38 (1997), 173–181.
CrossRef [27]
R. T. Rockafellar : Monotone operators and the proximal point algorithm. SIAM J. Control Optim.
14 (1976), 877–898.
MathSciNet CrossRef MATH [28]
J.-P. Royer, N. Thirion-Moreau, P. Comon : Computing the polyadic decomposition of nonnegative third order tensors. Signal Process.
91 (2011), 2159–2171.
CrossRef MATH [29]
A. Shashua, T. Hazan : Non-negative tensor factorization with applications to statistics and computer vision. ICML 2005: Proceedings of the 22nd International Conference on Machine Learning. ACM, New York, 2005, pp. 792–799.
[30]
A. Smilde, R. Bro, P. Geladi : Multi-way Analysis: Applications in the Chemical Sciences. Wiley, Chichester, 2004.
CrossRef [31]
G. Tomasi, R. Bro : A comparison of algorithms for fitting the PARAFAC model. Comput. Stat. Data Anal.
50 (2006), 1700–1734.
MathSciNet CrossRef MATH [32]
MATLAB 7.5.0. The Mathworks (2008).
[33]
© Institute of Mathematics of the Academy of Sciences of the Czech Republic, Praha, Czech Republic 2013