Abstract
We provide the first positive result on the nonsmooth optimization landscape of robust principal component analysis, to the best of our knowledge. It is the object of several conjectures and remains mostly uncharted territory. We identify a necessary and sufficient condition for the absence of spurious local minima in the rank-one case. Our proof exploits the subdifferential regularity of the objective function in order to eliminate the existence quantifier from the first-order optimality condition known as Fermat’s rule.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Anderson, B. G., Sojoudi, S.: Global optimality guarantees for nonconvex unsupervised video segmentation. Annual Allerton Conference on Communication, Control, and Computing, (2019)
Baldi, P., Hornik, K.: neural networks and principal component analysis: learning from examples without local minima. Neural Netw. 2(1), 53–58 (1989)
Beck, A.: First-order methods in optimization. SIAM, New Delhi (2017)
Bertsimas, D., Cory-Wright, R., Pauphilet, J.: Solving large-scale sparse pca to certifiable (near) optimality. arXiv preprint arXiv:2005.05195, (2020)
Bouwmans, T., Sobral, A., Javed, S., Jung, S.K., Zahzah, E.-H.: Decomposition into low-rank plus additive matrices for background/foreground separation: A review for a comparative evaluation with a large-scale dataset. Comput. Sci. Rev. 23, 1–71 (2017)
Burke, J.V., Ferris, M.C.: Weak sharp minima in mathematical programming. SIAM J. Control Optim. 31(5), 1340–1359 (1993)
Candès, E.J., Li, X., Ma, Y., Wright, J.: Robust principal component analysis? J. ACM (JACM) 58(3), 1–37 (2011)
Candès, E.J., Tao, T.: The power of convex relaxation: Near-optimal matrix completion. IEEE Trans. Inf. Theory 56(5), 2053–2080 (2010)
Chandrasekaran, V., Sanghavi, S., Parrilo, P. A., Willsky, A. S.: Rank-sparsity incoherence for matrix decomposition. SIOPT, (2011)
Charisopoulos, V., Chen, Y., Davis, D., Díaz, M., Ding, L., Drusvyatskiy, D.: Low-rank matrix recovery with composite optimization: good conditioning and rapid convergence. Found. Comput. Math., pp. 1–89, 2021
Chen, Y., Chi, Y.: Harnessing structures in big data via guaranteed low-rank matrix estimation: Recent theory and fast algorithms via convex and nonconvex optimization. IEEE Signal Process. Mag. 35(4), 14–31 (2018)
Chi, Y., Lu, Y.M., Chen, Y.: Nonconvex optimization meets low-rank matrix factorization: an overview. IEEE Trans. Signal Process. 67(20), 5239–5269 (2019)
Clarke, F. H.: Optimization and nonsmooth analysis. SIAM Classics Appl. Math., 1990
Du, Simon S., Hu, Wei, Lee, Jason D.: Algorithmic regularization in learning deep homogeneous models: layers are automatically balanced. NeurIPS 2018, (2018)
Fattahi, S., Sojoudi, S.: Exact guarantees on the absence of spurious local minima for non-negative rank-1 robust principal component analysis. JMLR, 2020
Garcia-Garcia, B., Bouwmans, T., Silva, A.J.R.: Background subtraction in real applications: challenges, current models and future directions. Comput. Sci. Rev. 35, 100–204 (2020)
Gillis, Nicolas, Vavasis, Stephen A.: On the complexity of robust PCA and l1-norm low-rank matrix approximation. Math. Op. Res. 43(4), 1072–1084 (2018)
Gross, D.: Recovering low-rank matrices from few coefficients in any basis. IEEE Trans. Inf. Theory, (2011)
Ioffe, A.D.: An invitation to tame optimization. SIAM J. Optim. 19(4), 1894–1917 (2009)
Jain, P., Kar, P.: Non-convex optimization for machine learning. Found Trends Mach. Learn., (2017)
Josz, C., Ouyang, Y., Zhang, R. Y., Lavaei, J., Sojoudi, S.: A theory on the absence of spurious solutions for nonconvex and nonsmooth optimization. NeurIPS, (2018)
Koren, Y., Bell, R., Volinsky, C.: Matrix factorization techniques for recommender systems. Comput. (2009)
Levy, O., Goldberg, Y.: Neural word embedding as implicit matrix factorization. NeurIPS, (2014)
Li, X., Zhu, Z., Man-Cho So, A., Vidal, R.: Nonconvex robust low-rank matrix recovery. SIAM J. Optim. (2019)
Li, Xiao, Zhu, Zhihui, Man-Cho So, Anthony, Lee, Jason D.: Incremental methods for weakly convex optimization. 12th Annual Workshop on Optimization for Machine Learning, (2020)
Moitra, A.: Algorithmic aspects of machine learning. Cambridge University Press, Cambridge (2018)
Nesterov, Yu.: Smooth minimization of non-smooth functions. Math. Program. 103(1), 127–152 (2005)
Nguyen, L.T., Kim, J., Shim, B.: Low-rank matrix completion: a contemporary survey. IEEE Access 7, 94215–94237 (2019)
Recht, B.: A simpler approach to matrix completion. JMLR, (2011)
Seidenberg, A.: A new decision method for elementary algebra. Annals Math., 365–374, (1954)
Tarski, A.: A decision method for elementary algebra and geometry: prepared for publication with the assistance of JCC McKinsey. (1951)
Valavi, H., Liu, S., Ramadge, P. J.: The landscape of matrix factorization revisited. AISTATS, (2020)
Acknowledgements
The authors acknowledge support from DTRA grant 13-1-0021, DARPA grant Lagrange, NSF grant 2023032, and ONR grant N00014-21-1-2282. Data sharing not applicable to this article as no datasets were generated or analysed during the current study
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Josz, C., Lai, L. Nonsmooth rank-one matrix factorization landscape. Optim Lett 16, 1611–1631 (2022). https://doi.org/10.1007/s11590-021-01819-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11590-021-01819-9