Solving Nuclear Norm Regularized and Semidefinite Matrix Least Squares Problems with Linear Equality Constraints

Part of the Fields Institute Communications book series (FIC, volume 69)


We introduce a partial proximal point algorithm for solving nuclear norm regularized and semidefinite matrix least squares problems with linear equality constraints. For the inner subproblems, we show that the positive definiteness of the generalized Hessian of the objective function for the inner subproblems is equivalent to the constraint nondegeneracy of the corresponding primal problem, which is a key property for applying a semismooth Newton-CG method to solve the inner subproblems efficiently. Numerical experiments on large scale matrix least squares problems arising from low rank matrix approximation, as well as regularized kernel estimation and Euclidean distance matrix completion problems in molecular conformation, show that our algorithm is efficient and robust.

Key words

Partial proximal-point algorithm Semidefinite least squares problem Nuclear norm minimization Euclidean distance matrix completion 

Subject Classifications

90C06 90C22 90C25 65F10 


  1. 1.
    Alfakih, A.Y., Khandani, A., Wolkowicz, H.: Solving Euclidean distance matrix completion problems via semidefinite programming. Comput. Optim. Appl. 12, 13–30 (1999)MathSciNetzbMATHCrossRefGoogle Scholar
  2. 2.
    Bhatia, R.: Matrix Analysis. Springer, NewYork (1997)CrossRefGoogle Scholar
  3. 3.
    Cai, J.F., Candès, E.J., Shen, Z.: A singular value thresholding algorithm for matrix completion. SIAM J. Optim. 20, 1956–1982 (2010)MathSciNetzbMATHCrossRefGoogle Scholar
  4. 4.
    Candès, E.J., Recht, B.: Exact matrix completion via convex optimization. Found. Comput. Math. 9, 717–772 (2009)MathSciNetzbMATHCrossRefGoogle Scholar
  5. 5.
    Clarke, F.: Optimization and Nonsmooth Analysis. Wiley, New York (1983)zbMATHGoogle Scholar
  6. 6.
    Combettes, P.L., Pesquet, J.-C.: Primal-dual splitting algorithm for solving inclusions with mixtures of composite, Lipschitzian, and parallel-sum type monotone operators. (2011)
  7. 7.
    Condat, L.: A primal-dual splitting method for convex optimization involving Lipschitzian, proximable and linear composite terms (2011, preprint)Google Scholar
  8. 8.
    Eckstein, J., Bertsekas, D.: On the Douglas-Rachford splitting method and the proximal point algorithm for maximal monotone operators. Math. Program. 55, 293–318 (1992)MathSciNetzbMATHCrossRefGoogle Scholar
  9. 9.
    Fazel, M.: Matrix rank minimization with applications. Ph.D. thesis, Stanford University (2002)Google Scholar
  10. 10.
    Fazel, M., Hindi, H., Boyd, S.: A rank minimization heuristic with application to minimum order system approximation. In: Proceedings of the American Control Conference, Arlington (2001)Google Scholar
  11. 11.
    Gabay, D., Mercier, B.: A dual algorithm for the solution of nonlinear variational problems via finite element approximations. Comput. Math. Appl. 2, 17–40 (1976)zbMATHCrossRefGoogle Scholar
  12. 12.
    Gansner, E.R., Hu, Y., North, S.: A maxent-stress model for graph layout. In: Proceedings of IEEE Pacific Visualization Symposium, Songdo (2012)Google Scholar
  13. 13.
    Gao, Y., Sun, D.F.: Calibrating least squares semidefinite programming with equality and inequality constraints. SIAM J. Matrix Anal. Appl. 31, 1432–1457 (2009)MathSciNetCrossRefGoogle Scholar
  14. 14.
    Glowinski, R., Marroco, A.: Sur l’approximation, par éléments finis d’ordre un, et la résolution, par pénalisation-dualité, d’une classe de problèmes de Dirichlet non linéares. Revue Francaise d’Automatique, Informatique et Recherche Opérationelle 9(R-2), 41–76 (1975)Google Scholar
  15. 15.
    Golub, G.H., van Loan, C.F.: Matrix Computations, 3rd edn. The Johns Hopkins University Press, Baltimore (1996)zbMATHGoogle Scholar
  16. 16.
    Golub, G.H., Hoffman, A., Stewart, G.W.: A generalization of the Eckart-Young-Mirsky matrix approximation theorem. Linear Algebra Appl. 88, 317–327 (1987)MathSciNetCrossRefGoogle Scholar
  17. 17.
    Ha, C.: A generalization of the proximal point algorithm. SIAM J. Control Optim. 28, 503–512 (1990)MathSciNetzbMATHCrossRefGoogle Scholar
  18. 18.
    Hou, J., Jun, S.R., Zhang, C., Kim, S.H.: Global mapping of the protein structure space and application in structure-based inference of protein function. Proc. Natl. Acad. Sci. USA 102, 3651–3656 (2005)CrossRefGoogle Scholar
  19. 19.
    Jiang, K.F., Sun, D.F., Toh, K.C.: A partial proximal point algorithm for nuclear norm regularized matrix least squares problems with polyhedral constraints (2012, preprint)Google Scholar
  20. 20.
    Liu, Y.J., Sun, D.F., Toh, K.C.: An implementable proximal point algorithmic framework for nuclear norm minimization. Math. Program. 133, 399–436 (2012)MathSciNetzbMATHCrossRefGoogle Scholar
  21. 21.
    Löwner, K.: Über monotone matrixfunktionen. Math. Z. 38, 177–216 (1934)MathSciNetCrossRefGoogle Scholar
  22. 22.
    Lu, F., Keleş, S., Wright, S., Wahba, G.: Framework for kernel regularization with application to protein clustering. Proc. Natl. Acad. Sci. USA 102, 12332–12337 (2005)MathSciNetzbMATHCrossRefGoogle Scholar
  23. 23.
    Ma, S., Goldfarb, D., Chen, L.: Fixed point and Bregman iterative methods for matrix rank minimization. Math. Program. 128, 321–353 (2011)MathSciNetzbMATHCrossRefGoogle Scholar
  24. 24.
    Qi, L., Sun, J.: A nonsmooth version of Newton’s method. Math. Program. 58, 353–367 (1993)MathSciNetzbMATHCrossRefGoogle Scholar
  25. 25.
    Recht, B., Fazel, M., Parrilo, P.A.: Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization. SIAM Rev. 52, 471–501 (2010)MathSciNetzbMATHCrossRefGoogle Scholar
  26. 26.
    Rockafellar, R.T.: Convex Analysis. Princeton University Press, Princeton (1970)zbMATHGoogle Scholar
  27. 27.
    Rockafellar, R.T.: Monotone operators and the proximal point algorithm. SIAM J. Control Optim. 14, 877–898 (1976)MathSciNetzbMATHCrossRefGoogle Scholar
  28. 28.
    Rockafellar, R.T.: Augmented Lagrangains and applications of the proximal point algorithm in convex programming. Math. Oper. Res. 1, 97–116 (1976)MathSciNetzbMATHCrossRefGoogle Scholar
  29. 29.
    Toh, K.C., Yun, S.W.: An accelerated proximal gradient algorithm for nuclear norm regularized least squares problems. Pac. J. Optim. 6, 615–640 (2010)MathSciNetzbMATHGoogle Scholar
  30. 30.
    Weinberger, K.Q., Sha, F., Zhu, Q., Saul, L.K.: Graph Laplacian regularization for large-scale semidefinite programming. In: Schoelkopf, B., Platt, J., Hofmann, T. (eds.) Advances in Neural Information Processing Systems, pp. 1489–1496. MIT, Cambridge (2007)Google Scholar
  31. 31.
    Yang, Z.: A study on nonsymmetric matrix-valued functions. Master thesis, Department of Mathematics, National University of Singapore (2009)Google Scholar
  32. 32.
    Zhao, X.Y., Sun, D.F., Toh, K.C.: A Newton-CG augmented Lagrangian method for semidefinite programming. SIAM J. Optim. 20, 1737–1765 (2010)MathSciNetzbMATHCrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2013

Authors and Affiliations

  1. 1.Department of MathematicsNational University of SingaporeSingaporeSingapore
  2. 2.Department of Mathematics and Risk Management InstituteNational University of SingaporeSingaporeSingapore
  3. 3.Department of MathematicsNational University of SingaporeSingaporeSingapore
  4. 4.Singapore-MIT AllianceSingaporeSingapore

Personalised recommendations