Skip to main content
SpringerLink
Log in

An alternating direction algorithm for matrix completion with nonnegative factors

  • Research Article
  • Published:
Frontiers of Mathematics in China Aims and scope Submit manuscript

Abstract

This paper introduces an algorithm for the nonnegative matrix factorization-and-completion problem, which aims to find nonnegative low-rank matrices X and Y so that the product XY approximates a nonnegative data matrix M whose elements are partially known (to a certain accuracy). This problem aggregates two existing problems: (i) nonnegative matrix factorization where all entries of M are given, and (ii) low-rank matrix completion where nonnegativity is not required. By taking the advantages of both nonnegativity and low-rankness, one can generally obtain superior results than those of just using one of the two properties. We propose to solve the non-convex constrained least-squares problem using an algorithm based on the classical alternating direction augmented Lagrangian method. Preliminary convergence properties of the algorithm and numerical simulation results are presented. Compared to a recent algorithm for nonnegative matrix factorization, the proposed algorithm produces factorizations of similar quality using only about half of the matrix entries. On tasks of recovering incomplete grayscale and hyperspectral images, the proposed algorithm yields overall better qualities than those produced by two recent matrix-completion algorithms that do not exploit nonnegativity.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Berry M W, Browne M, Langville A N, Pauca V P, Plemmons R J. Algorithms and applications for approximate nonnegative matrix factorization. Comput Statist Data Anal, 2007, 52(1): 155–173

    Article  MathSciNet  MATH  Google Scholar 

  2. Bertsekas D P, Tsitsiklis J N. Parallel and Distributed Computation: Numerical Methods. Upper Saddle River: Prentice-Hall, Inc, 1989

    MATH  Google Scholar 

  3. Biswas P, Lian T C, Wang T C, Ye Y. Semidefinite programming based algorithms for sensor network localization. ACM Trans Sensor Networks, 2006, 2(2): 188–220

    Article  Google Scholar 

  4. Cai J F, Candes E J, Shen Z. A singular value thresholding algorithm for matrix completion export. SIAM J Optim, 2010, 20: 1956–1982

    Article  MathSciNet  MATH  Google Scholar 

  5. Cand`es E J, Li X, Ma Y, Wright J. Robust principal component analysis? J ACM, 2011, 58(3): 11

    MathSciNet  Google Scholar 

  6. Cand`es E J, Recht B. Exact matrix completion via convex optimization. Found Comput Math, 2009, 9(6): 717–772

    Article  MathSciNet  Google Scholar 

  7. Cand`es E J, Tao T. The power of convex relaxation: Near-optimal matrix completion. IEEE Trans Inform Theory, 2010, 56(5): 2053–2080

    Article  MathSciNet  Google Scholar 

  8. Cichocki A, Morup M, Smaragdis P, Wang W, Zdunek R. Advances in Nonnegative Matrix and Tensor Factorization. Computational Intelligence Neuroscience. New York: Hindawi Publishing Corporation, 2008

    Google Scholar 

  9. Cichocki A, Zdunek R, Phan A H, Amari S. Nonnegative Matrix and Tensor Factorizations-Applications to Exploratory Multiway Data Analysis and Blind Source Separation. Hoboken: John Wiley & Sons, Ltd, 2009

    Google Scholar 

  10. Fazel M. Matrix Rank Minimization with Applications. PhD Thesis, Stanford University. 2002

  11. Gabay D, Mercier B. A dual algorithm for the solution of nonlinear variational problems via finite element approximation. Comput Math Appl, 1976, 2(1): 17–40

    Article  MATH  Google Scholar 

  12. Glowinski R, Marrocco A. Sur lapproximation par elements finis dordre un, et la resolution par penalisation-dualite dune classe de problemes de Dirichlet nonlineaires. Rev Francaise dAut Inf Rech Oper, 1975, 41–76

  13. Goldberg D, Nichols D, Oki B M, Terry D. Using collaborative filtering to weave an information tapestry. Commun ACM, 1992, 35(12): 61–70

    Article  Google Scholar 

  14. Goldfarb D, Ma S, Wen Z. Solving low-rank matrix completion problems efficiently. In: Proceedings of 47th Annual Allerton Conference on Communication, Control, and Computing, Monticello, Illinois. 2009

  15. Grippo L, Sciandrone M. On the convergence of the block nonlinear Gauss-Seidel method under convex constraints. Oper Res Lett, 2000, 26(3): 127–136

    Article  MathSciNet  MATH  Google Scholar 

  16. Hale E T, Yin W, Zhang Y. Fixed-point continuation for l 1-minimization: methodology and convergence. SIAM J Optim, 2008, 19(3): 1107–1130

    Article  MathSciNet  MATH  Google Scholar 

  17. Hestenes M R. Multiplier and gradient methods. J Optim Theory Appl, 1969, 4(5): 303–320

    Article  MathSciNet  MATH  Google Scholar 

  18. Lee D D, Seung H S. Learning the parts of objects by non-negative matrix factorization. Nature, 1999, 401(6755): 788–791

    Article  Google Scholar 

  19. Lee D D, Seung H S. Algorithms for non-negative matrix factorization. Adv Neural Inf Process Syst, 2001, 13: 556–562

    Google Scholar 

  20. Liu Z, Vandenberghe L. Interior-point method for nuclear norm approximation with application to system identification. SIAM J Matrix Anal Appl, 2009, 31(3): 1235–1256

    Article  MathSciNet  Google Scholar 

  21. Ma S, Goldfarb D, Chen L. Fixed point and Bregman iterative methods for matrix rank minimization. Math Program, Ser A, 2011, 128(1–2): 321–353

    Article  MathSciNet  MATH  Google Scholar 

  22. Paatero P. Least squares formulation of robust non-negative factor analysis. Chemometrics Intell Lab Syst, 1997, 37(1): 23–35

    Article  Google Scholar 

  23. Paatero P. The multilinear engine: A table-driven, least squares program for solving multilinear problems, including the n-way parallel factor analysis model. J Comput Graph Statist, 1999, 8(4): 854–888

    Article  MathSciNet  Google Scholar 

  24. Paatero P, Tapper U. Positive matrix factorization: A non-negative factor model with optimal utilization of error estimates of data values. Environmetrics, 1994, 5(2): 111–126

    Article  Google Scholar 

  25. Powell M J D. A method for nonlinear constraints in minimization problems. In: Fletcher R, ed. Optimization. New York: Academic Press, 1969, 283–298

    Google Scholar 

  26. Recht B, Fazel M, Parrilo P A. Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization. SIAM Review, 2010, 52(3): 471–501

    Article  MathSciNet  MATH  Google Scholar 

  27. Rockafellar R T. The multiplier method of Hestenes and Powell applied to convex programming. J Optim Theory Appl, 1973, 12(6): 555–562

    Article  MathSciNet  MATH  Google Scholar 

  28. Wang Y, Yang J, Yin W, Zhang Y. A new alternating minimization algorithm for total variation image reconstruction. SIAM J Imaging Sci, 2008, 1(3): 248–272

    Article  MathSciNet  MATH  Google Scholar 

  29. Wen Z, Goldfarb D, Yin W. Alternating direction augmented Lagrangian methods for semidefinite programming. Math Program Comput, 2010, 2(3–4): 203–230

    Article  MathSciNet  MATH  Google Scholar 

  30. Wen Z, Yin W, Zhang Y. Solving a low-rank factorization model for matrix completion by a non-linear successive over-relaxation algorithm. Rice Univ CAAM Technical Report TR 10-07, 2010

  31. Yang J, Yuan X. Linearized augmented Lagrangian and alternating direction methods for nuclear norm minimization. Math Comp (to appear)

  32. Yang J, Zhang Y, Yin W. An efficient TVL1 algorithm for deblurring multichannel images corrupted by impulsive noise. SIAM J Sci Comput, 2008, 31: 2842–2865

    Article  MathSciNet  Google Scholar 

  33. Yin W, Osher S, Goldfarb D, Darbon J. Bregman iterative algorithms for ℓ 1-minimization with applications to compressed sensing. SIAM J Imaging Sci, 2008, 1(1): 143–168

    Article  MathSciNet  MATH  Google Scholar 

  34. Zhang Y. An alternating direction algorithm for nonnegative matrix factorization. Rice Technical Report TR 10-03, 2010

Download references

Author information

Authors and Affiliations

  1. Department of Computational and Applied Mathematics, Rice University, Houston, TX, 77005, USA

    Yangyang Xu, Wotao Yin & Yin Zhang

  2. Department of Mathematics and Institute of Natural Sciences, Shanghai Jiao Tong University, Shanghai, 200240, China

    Zaiwen Wen

Authors
  1. Yangyang Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wotao Yin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zaiwen Wen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yin Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wotao Yin.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Xu, Y., Yin, W., Wen, Z. et al. An alternating direction algorithm for matrix completion with nonnegative factors. Front. Math. China 7, 365–384 (2012). https://doi.org/10.1007/s11464-012-0194-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11464-012-0194-5

Keywords

MSC

Search

Navigation