# A new nonconvex approach to low-rank matrix completion with application to image inpainting

- 410 Downloads
- 1 Citations

## Abstract

The problem of recovering a low-rank matrix from partial entries, known as low-rank matrix completion, has been extensively investigated in recent years. It can be viewed as a special case of the affine constrained rank minimization problem which is NP-hard in general and is computationally hard to solve in practice. One widely studied approach is to replace the matrix rank function by its nuclear-norm, which leads to the convex nuclear-norm minimization problem solved efficiently by many popular convex optimization algorithms. In this paper, we propose a new nonconvex approach to better approximate the rank function. The new approximation function is actually the Moreau envelope of the rank function (MER) which has an explicit expression. The new approximation problem of low-rank matrix completion based on MER can be converted to an optimization problem with two variables. We then adapt the proximal alternating minimization algorithm to solve it. The convergence (rate) of the proposed algorithm is proved and its accelerated version is also developed. Numerical experiments on completion of low-rank random matrices and standard image inpainting problems have shown that our algorithms have better performance than some state-of-art methods.

## Keywords

Low-rank matrix completion Moreau envelope Proximal alternating minimization Image Inpainting## Notes

### Acknowledgements

The authors would like to thank the editor-in chief and the anonymous reviewers for their insightful and constructive comments, which help to enrich the content and improve the presentation of this paper. This research was supported by the National Natural Science Foundation of China under the grants 11771347, 91730306, 41390454, and 11271297, and supported by the Nanhu Scholars Program for Young Scholars of XYNU.

## References

- Attouch, H., & Bolte, J. (2009). On the convergence of the proximal algorithm for nonsmooth functions involving analytic features.
*Mathematical Programming*,*116*(1), 5–16.MathSciNetzbMATHGoogle Scholar - Attouch, H., Bolte, J., Redont, P., & Soubeyran, A. (2010). Proximal alternating minimization and projection methods for nonconvex problems: An approach based on the Kurdyka–Łojasiewicz inequality.
*Mathematics of Operations Research*,*35*(2), 438–457.MathSciNetzbMATHGoogle Scholar - Auslender, A. (1992). Asymptotic properties of the Fenchel dual functional and applications to decomposition problems.
*Journal of Optimization Theory and Applications*,*73*(3), 427–449.MathSciNetzbMATHGoogle Scholar - Bauschke, H. H., & Combettes, P. L. (2011).
*Convex analysis and monotone operator theory in Hilbert spaces. CMS Books in Mathematics*. New York: Springer.zbMATHGoogle Scholar - Beck, A., & Teboulle, M. (2009). A fast iterative shrinkage-thresholding algorithm for linear inverse problems.
*SIAM Journal on Imaging Sciences*,*2*(1), 183–202.MathSciNetzbMATHGoogle Scholar - Bertsekas, D. (1999).
*Nonlinear optimisation*, 2nd ed., Athena, Belmont, Massachusetts.Google Scholar - Bolte, J., Sabach, S., & Teboulle, M. (2013). Proximal alternating linearized minimization for nonconvex and nonsmooth problems.
*Mathematical Programming*,*146*(1), 459–494.MathSciNetzbMATHGoogle Scholar - Cai, J. F., Candès, E. J., & Shen, Z. W. (2010). A singular value thresholding algorithm for matrix completion.
*SIAM Journal on Optimization*,*20*(4), 1956–1982.MathSciNetzbMATHGoogle Scholar - Candès, E. J., & Recht, B. (2009). Exact matrix completion via convex optimization.
*Foundations of Computational Mathematics*,*9*(6), 717–772.MathSciNetzbMATHGoogle Scholar - Candès, E. J., Romberg, J., & Tao, T. (2006). Stable signal recovery from incomplete and inaccurate measurements.
*Communications on Pure and Applied Mathematics*,*59*(8), 1207–1223.MathSciNetzbMATHGoogle Scholar - Candès, E. J., & Tao, T. (2010). The power of convex relaxation: Near-optimal matrix completion.
*IEEE Transactions on Information Theory*,*56*(5), 2053–2080.MathSciNetzbMATHGoogle Scholar - Candès, E. J., & Wakin, M. (2008). An introduction to compressive sampling.
*IEEE Signal Processing Magazine*,*25*(2), 21–30.Google Scholar - Cao, W. F., Sun, J., & Xu, Z. B. (2013). Fast image deconvolution using closed-form thresholding formulas of \(L_q\) \((q = 1/2, 2/3)\) regularization.
*Journal of Visual Communication and Image Representation*,*24*(1), 31–41.Google Scholar - Chen, P., & Suter, D. (2004). Recovering the missing components in a large noisy low-rank matrix: Application to SFM source.
*IEEE Transactions on Pattern Analysis and Machine Intelligence*,*26*(8), 1051–1063.Google Scholar - Fan, J. Q., & Li, R. Z. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties.
*Journal of the American Statistical Association*,*96*(456), 1348–1360.MathSciNetzbMATHGoogle Scholar - Fazel, M. (2002).
*Matrix rank minimization with applications*. Ph.D. thesis, Stanford University.Google Scholar - Geng, J., Wang, L. S., & Wang, Y. F. (2015). A non-convex algorithm framework based on DC programming and DCA for matrix completion.
*Numerical Algorithms*,*68*(4), 903–921.MathSciNetzbMATHGoogle Scholar - Goldberg, D., Nichols, D., Oki, B. M., & Terry, D. (1992). Using collaborative filtering to weave an information tapestry.
*Communications of the ACM*,*35*(12), 61–70.Google Scholar - Hu, Y., Zhang, D. B., Ye, J. P., Li, X. L., & He, X. F. (2013). Fast and accurate matrix completion via truncated nuclear norm regularization.
*IEEE Transactions on Pattern Analysis and Machine Intelligence*,*35*(9), 2117–2130.Google Scholar - Jin, Z. F., Wan, Z. P., Jiao, Y. L., & Lu, J. X. (2016). An alternating direction method with continuation for nonconvex low rank minimization.
*Journal of Scientific Computing*,*66*(2), 849–869.MathSciNetzbMATHGoogle Scholar - Lai, M. J., & Wang, J. Y. (2011). An unconstrained \(\ell _q\) minimization with \(0<q\le 1\) for sparse solution of underdetermined linear systems.
*SIAM Journal on Optimization*,*21*(1), 82–101.MathSciNetGoogle Scholar - Li, G., & Pong, T. (2016). Calculus of the exponent of Kurdyka–Łojasiewicz inequality and its applications to linear convergence of first-order methods. arXiv:1602.02915v1.
- Li, G., Mordukhovich, B. S., & Pham, T. S. (2015). New fractional error bounds for polynomial systems with applications to Hölderian stability in optimization and spectral theory of tensors.
*Mathematical Programming*,*153*(2), 333–362.MathSciNetzbMATHGoogle Scholar - Liu, Z., & Vandenberghe, L. (2009). Interior-point method for nuclear norm approximation with application to system identification.
*SIAM Journal on Matrix Analysis and Applications*,*31*(3), 1235–1256.MathSciNetzbMATHGoogle Scholar - Lu, Z. S., & Zhang, Y. (2015). Schatten-\(p\) quasi-norm regularized matrix optimization via iterative reweighted singular value minimization. http://www.optimization-online.org/DB_HTML/2015/11/5215.html.
- Lu, C. Y., Tang, J. H., Yan, S. C., & Lin, Z. C. (2015). Nonconvex nonsmooth low-rank minimization via iteratively reweighted nuclear norm. arXiv:1510.06895.
- Ma, S. Q., Goldfarb, D., & Chen, L. F. (2011). Fixed point and Bregman iterative methods for matrix rank minimization.
*Mathematical Programming*,*128*(1), 321–353.MathSciNetzbMATHGoogle Scholar - Mohammadi, M. M., Zadeh, M. B., Amini, A., & Jutten, C. (2013). Recovery of low-rank matrices under affine constraints via a smoothed rank function. arXiv:1308.2293.
- Mohammadi, M. M., Zadeh, M. B., & Jutten, C. (2009). A fast approach for overcomplete sparse decomposition based on smoothed \(\ell _0\)-norm.
*IEEE Transactions on Signal Processing*,*57*(1), 289–301.MathSciNetzbMATHGoogle Scholar - Recht, B., Fazel, M., & Parrilo, P. A. (2010). Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization.
*SIAM Review*,*52*(3), 471–501.MathSciNetzbMATHGoogle Scholar - Rockafellar, R. (1970).
*Convex analysis*. Princeton: Princeton University Press.zbMATHGoogle Scholar - Singer, A. (2008). A remark on global positioning from local distances.
*Proceedings of the National Academy of Sciences of the United States of America*,*105*(28), 9507–9511.MathSciNetzbMATHGoogle Scholar - Srebro, N. (2004).
*Learning with matrix factorizations*. Ph.D. thesis, Massachusetts Institute of Technology.Google Scholar - Toh, K. C., & Yun, S. W. (2012). An accelerated proximal gradient algorithm for nuclear norm regularized linear least squares problems.
*Pacific Journal of Optimization*,*6*(3), 615–640.MathSciNetzbMATHGoogle Scholar - Tomasi, C., & Kanade, T. (1992). Shape and motion from image streams under orthography: A factorization method.
*International Journal of Computer Vision*,*9*(2), 137C154.Google Scholar - Tseng, P. (2001). Convergence of a block coordinate descent method for nondifferentiable minimization.
*Journal of Optimization Theory and Applications*,*109*(3), 475–494.MathSciNetzbMATHGoogle Scholar - Tütüncü, R. H., Toh, K. C., & Todd, M. J. (2003). Solving semidefinite-quadratic-linear programs using SDPT3.
*Mathematical Programming*,*95*(2), 189–217.MathSciNetzbMATHGoogle Scholar - Yang, J. F., & Yuan, X. M. (2013). Linearized augmented Lagrangian and alternating direction methods for nuclear norm minimization.
*Mathematics of Computation*,*82*(281), 301–329.MathSciNetzbMATHGoogle Scholar - Zhang, S., & Xin, J. (2014). Minimization of transformed \(\ell _1\) penalty: Closed form representation and iterative thresholding algorithms. arXiv preprint arXiv:1412.5240.
- Zhang, S., Yin, P. H., & Xin, J. (2015). Transformed Schatten-1 iterative thresholding algorithms for matrix rank minimization and applications. arXiv:1506.04444.
- Zhang, C. H. (2010). Nearly unbiased variable selection under minimax concave penalty.
*The Annals of Statistics*,*38*(2), 894–942.MathSciNetzbMATHGoogle Scholar - Zhang, T. (2010). Analysis of multi-stage convex relaxation for sparse regularization.
*Journal of Machine Learning Research*,*11*(Mar), 1081–1107.MathSciNetzbMATHGoogle Scholar - Zhao, Y. B. (2012). An approximation theory of matrix rank minimization and its application to quadratic equations.
*Linear Algebra and Its Applications*,*437*(1), 77–93.MathSciNetzbMATHGoogle Scholar