Advertisement

PPA-Like Contraction Methods for Convex Optimization: A Framework Using Variational Inequality Approach

  • Bing-Sheng HeEmail author
Article

Abstract

Linearly constrained convex optimization has many applications. The first-order optimal condition of the linearly constrained convex optimization is a monotone variational inequality (VI). For solving VI, the proximal point algorithm (PPA) in Euclidean-norm is classical but abstract. Hence, the classical PPA only plays an important theoretical role and it is rarely used in the practical scientific computation. In this paper, we give a review on the recently developed customized PPA in H-norm (H is a positive definite matrix). In the frame of customized PPA, it is easy to construct the contraction-type methods for convex optimization with different linear constraints. In each iteration of the proposed methods, we need only to solve the proximal subproblems which have the closed form solutions or can be efficiently solved up to a high precision. Some novel applications and numerical experiments are reported. Additionally, the original primal-dual hybrid gradient method is modified to a convergent algorithm by using a prediction-correction uniform framework. Using the variational inequality approach, the contractive convergence and convergence rate proofs of the framework are more general and quite simple.

Keywords

Linearly constrained convex optimization Variational inequality Proximal point algorithm 

Mathematics Subject Classification

65K10 90C25 90C30 

Notes

Acknowledgments

The author is grateful to the two anonymous referees for their careful reading and valuable comments which have helped me substantially improve the presentation of this paper.

References

  1. 1.
    Higham, N.J.: Computing the nearest correlation matrix—a problem from finance. IMA J. Numer. Anal. 22, 329–343 (2002)Google Scholar
  2. 2.
    Cai, J.F., Candès, E.J., Shen, Z.W.: A singular value thresholding algorithm for matrix completion. SIAM J. Optim. 20, 1956–1982 (2010)Google Scholar
  3. 3.
    Chen, C.H., He, B.S., Yuan, X.M.: Matrix completion via alternating direction method. IMA J. Numer. Anal. 32, 227–245 (2012)Google Scholar
  4. 4.
    Martinet, B.: Regularisation, d’inéquations variationelles par approximations succesives. Rev. Francaise d’Inform. Recherche Oper. 4, 154–159 (1970) (In French)Google Scholar
  5. 5.
    Rockafellar, R.T.: Monotone operators and the proximal point algorithm. SIAM J. Control. Optim. 14, 877–898 (1976)Google Scholar
  6. 6.
    Zhu, M., Chan, T.F.: An efficient primal-dual hybrid gradient algorithm for total variation image restoration, CAM Report 08–34, UCLA, (2008)Google Scholar
  7. 7.
    He, B.S., You, Y.F., Yuan, X.M.: On the convergence of primal-dual hybrid gradient algorithm. SIAM J. Imaging Sci. 7, 2526–2537 (2014)Google Scholar
  8. 8.
    Chambolle, A., Pock, T.: A first-order primal-dual algorithms for convex problem with applications to imaging. J. Math. Imaging Vis. 40, 120–145 (2011)Google Scholar
  9. 9.
    Pock, T., Chambolle, A.: Diagonal preconditioning for first order primal-dual algorithms in convex optimization. In: 2011 IEEE International Conference on Computer Vision, pp. 1762–1769 (2011)Google Scholar
  10. 10.
    He, B.S., Yuan, X.M.: Convergence analysis of primal-dual algorithms for a saddle-point problem: from contraction perspective. SIAM J. Imaging Sci. 5, 119–149 (2012)Google Scholar
  11. 11.
    Gu, G.Y., He, B.S., Yuan, X.M.: Customized proximal point algorithms for linearly constrained convex minimization and saddle-point problems: a unified approach. Comput. Optim. Appl. 59, 135–161 (2014)Google Scholar
  12. 12.
    He, B.S., Yuan, X.M., Zhang, W.X.: A customized proximal point algorithm for convex minimization with linear constraints. Comput. Optim. Appl. 56, 559–572 (2013)Google Scholar
  13. 13.
    You, Y.F., Fu, X.L., He, B.S.: Lagrangian-PPA based contraction methods for linearly constrained convex optimization. Pacific. J. Opt. 10, 199–213 (2014)Google Scholar
  14. 14.
    Facchinei, F., Pang, J.S.: Finite-Dimensional Variational Inequalities and Complementarity Problems, Volume 1. Springer Series in Operations Research. Springer, New York (2003)Google Scholar
  15. 15.
    He, B.S., Yuan, X.M.: On the \(O(1/t)\) convergence rate of the alternating direction method. SIAM J. Numer. Anal. 50, 700–709 (2012)Google Scholar
  16. 16.
    Nesterov, Y.E.: Gradient methods for minimizing composite objective function. Math. Program. Ser. B. 140, 125–161 (2013)Google Scholar
  17. 17.
    He, B.S., Shen, Y.: On the convergence rate of customized proximal point algorithm for convex optimization and saddle-point problem. Sci. Sin. Math. 42, 515–525 (2012) (In Chinese)Google Scholar
  18. 18.
    Nemirovski, A.: Prox-method with rate of convergence \(O(1/t)\) for variational inequalities with Lipschitz continuous monotone operators and smooth convex-concave saddle point problems. SIAM J. Optim. 15, 229–251 (2004)Google Scholar

Copyright information

© Operations Research Society of China, Periodicals Agency of Shanghai University, Science Press, and Springer-Verlag Berlin Heidelberg 2015

Authors and Affiliations

  1. 1.Department of MathematicsNanjing UniversityNanjingChina

Personalised recommendations