Skip to main content
SpringerLink
Log in

A generalized alternating direction method of multipliers with semi-proximal terms for convex composite conic programming

  • Full Length Paper
  • Published:
Mathematical Programming Computation Aims and scope Submit manuscript

Abstract

In this paper, we propose a generalized alternating direction method of multipliers (ADMM) with semi-proximal terms for solving a class of convex composite conic optimization problems, of which some are high-dimensional, to moderate accuracy. Our primary motivation is that this method, together with properly chosen semi-proximal terms, such as those generated by the recent advance of block symmetric Gauss–Seidel technique, is capable of tackling these problems. Moreover, the proposed method, which relaxes both the primal and the dual variables in a natural way with a common relaxation factor in the interval of (0, 2), has the potential of enhancing the performance of the classic ADMM. Extensive numerical experiments on various doubly non-negative semidefinite programming problems, with or without inequality constraints, are conducted. The corresponding results showed that all these multi-block problems can be successively solved, and the advantage of using the relaxation step is apparent.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2

Similar content being viewed by others

Notes

  1. In [7], the authors considered the case that \({\mathcal B}^{*}\) is an identity operator and \(c=0\). However, it is easy to see that the scheme described in (10) is a direct application of their idea to problem (5).

  2. http://biqmac.uni-klu.ac.at/biqmaclib.html.

  3. In fact, we have applied the technique discussed in [3, 33] for using a step-length which can sometimes larger than 1.618 to further improve the computational efficiency. However, we still call the step-length is 1.618, just for convenience.

References

  1. Chen, C.H.: Numerical algorithms for a class of matrix norm approximation problems. Ph.D. Thesis, Department of Mathematics, Nanjing University, Nanjing, China. http://www.math.nus.edu.sg/~matsundf/Thesis_Caihua.pdf (2012)

  2. Chen, L., Sun, D.F., Toh, K.-C.: An efficient inexact symmetric Gauss–Seidel based majorized ADMM for high-dimensional convex composite conic programming. Math. Program. 161(1), 237–270 (2017)

    Article  MathSciNet  Google Scholar 

  3. Chen, L., Sun, D.F., Toh, K.-C.: A note on the convergence of ADMM for linearly constrained convex optimization Problems. Comput. Optim. Appl. 66(2), 327–343 (2017)

    Article  MathSciNet  Google Scholar 

  4. Cui, Y., Li, X.D., Sun, D.F., Toh, K.-C.: On the convergence properties of a majorized ADMM for linearly constrained convex optimization problems with coupled objective functions. J. Optim. Theory Appl. 169(3), 1013–1041 (2016)

    Article  MathSciNet  Google Scholar 

  5. Deng, W., Lai, M.-J., Peng, Z.: W, Yin: Parallel multi-block ADMM with \(o(1/k)\) convergence. J. Sci. Comput. 71(2), 712–736 (2017)

    Article  MathSciNet  Google Scholar 

  6. Eckstein, J.: Some saddle-function splitting methods for convex programming. Optim. Methods Softw. 4, 75–83 (1994)

    Article  Google Scholar 

  7. Eckstein, J., Bertsekas, D.P.: On the Douglas–Rachford splitting method and the proximal point algorithm for maximal monotone operators. Math. Program. 55, 293–318 (1992)

    Article  MathSciNet  Google Scholar 

  8. Eckstein, J., Yao, W.: Understanding the convergence of the alternating direction method of multipliers: theoretical and computational perspectives. Pac. J. Optim. 11(4), 619–644 (2014)

    MathSciNet  MATH  Google Scholar 

  9. Fazel, M., Pong, T.K., Sun, D.F., Tseng, P.: Hankel matrix rank minimization with applications in system identification and realization. SIAM J. Matrix Anal. Appl. 34, 946–977 (2013)

    Article  MathSciNet  Google Scholar 

  10. Fortin, M., Glowinski, R.: Augmented Lagrangian methods: applications to the numerical solution of boundary-value problems. Studies in mathematics and its applications, vol. 15. (translated from French by Hunt, B. and Spicer, D.C.) Elsevier Science Publishers B.V. (1983)

    MATH  Google Scholar 

  11. Gabay, D.: Studies in mathematics and its applications. In: Fortin, M., Glowinski, R. (eds.) Applications of the method of multipliers to variational inequalities in Augmented Lagrangian Methods: Applications to the Numerical Solution of Boundary-Value Problems, vol. 15, pp. 299–331. Elsevier, Amsterdam (1983)

    Google Scholar 

  12. Gabay, D., Mercier, B.: A dual algorithm for the solution of nonlinear variational problems via finite element approximation. Comput. Math. Appl. 2, 17–40 (1976)

    Article  Google Scholar 

  13. Glowinski, R.: Lectures on numerical methods for non-linear variational problems. Published for the Tata Institute of Fundamental Research, Bombay [by] Springer (1980)

  14. Glowinski, R. and Marroco, A.: Sur l’approximation, par éléments finis d’ordre un, et la résolution, par pénalisation-dualité d’une classe de problèmes de Dirichlet non linéaires. Revue française d’atomatique, Informatique Recherche Opérationelle. Analyse Numérique, 9(2), 41–76 (1975)

  15. Glowinski, R.: Modeling, simulation and optimization for science and technology. In: Fitzgibbon, W., Kuznetsov, Y.A., Neittaanmaki, P., Pironneau, O. (eds.) On alternating direction methods of multipliers: A historical perspective, pp. 59–82. Springer, Netherlands (2014)

    MATH  Google Scholar 

  16. He, B., Tao, M., Yuan, X.: Alternating direction method with Gaussian back substitution for separable convex programming. SIAM J. Optim. 22(2), 313–340 (2012)

    Article  MathSciNet  Google Scholar 

  17. Hestenes, M.: Multiplier and gradient methods. J. Optim. Theory Appl. 4(5), 303–320 (1969)

    Article  MathSciNet  Google Scholar 

  18. Hong, M., Chang, T.-H., Wang, X., Razaviyayn, M., Ma, S. and Luo, Z.-Q.: A block successive upper bound minimization method of multipliers for linearly constrained convex optimization. arXiv:1401.7079 (2014)

  19. Li, X.D., Sun, D.F., Toh. K.-C.: QSDPNAL: A two-phase Newton-CG proximal augmented Lagrangian method for convex quadratic semidefinite programming problems, arXiv:1512.08872 (2015)

  20. Li, X.D.: A two-phase augmented Lagrangian method for convex composite quadratic programming, PhD Thesis, Department of Mathematics, National University of Singapore (2015)

  21. Li, M., Sun, D.F., Toh, K.-C.: A majorized ADMM with indefinite proximal terms for linearly constrained convex composite optimization. SIAM J. Optim. 26, 922–950 (2016)

    Article  MathSciNet  Google Scholar 

  22. Li, X.D., Sun, D.F., Toh, K.-C.: A Schur complement based semi-proximal ADMM for convex quadratic conic programming and extensions. Math. Program. 155, 333–373 (2016)

    Article  MathSciNet  Google Scholar 

  23. Lin, M., Ma, S.Q., Zhang, S.Z.: On the global linear convergence of the ADMM with multi-block variables. SIAM J. Optim. 25(3), 1478–1497 (2015)

    Article  MathSciNet  Google Scholar 

  24. Lin, M., Ma, S.Q., Zhang, S.Z.: Iteration complexity analysis of multi-block ADMM for a family of convex minimization without strong convexity. J. Sci. Comput. 69, 52–81 (2016)

    Article  MathSciNet  Google Scholar 

  25. Monteiro, R.D.C., Ortiz, C., Svaiter, B.F.: A first-order block-decomposition method for solving two-easy-block structured semidefinite programs. Math. Program. Comput. 6, 103–150 (2014)

    Article  MathSciNet  Google Scholar 

  26. Monteiro, R.D.C., Ortiz, C., Svaiter, B.F.: Implementation of a block-decomposition algorithm for solving large-scale conic semidefinite programming problems. Comput. Optim. Appl. 57, 45–69 (2014)

    Article  MathSciNet  Google Scholar 

  27. Povh, J., Rendl, F., Wiegele, A.: A boundary point method to solve semidefinite programs. Computing 78, 277–286 (2006)

    Article  MathSciNet  Google Scholar 

  28. Powell, M.J.D.: Optimization. In: Fletcher, R. (ed.) A method for nonlinear constraints in minimization problems, pp. 283–298. Academic Press, London (1969)

    Google Scholar 

  29. Rockafellar, R.T.: Convex Analysis. Princeton University Press, Princeton (1970)

    Book  Google Scholar 

  30. Rockafellar, R.T.: Augmented Lagrangians and applications of the proximal point algorithm in convex programming. Math. Oper. Res. 1(2), 97–116 (1976)

    Article  MathSciNet  Google Scholar 

  31. Rockafellar, R.T.: Monotone operators and the proximal point algorithm. SIAM J. Control Optim. 14(5), 877–898 (1976)

    Article  MathSciNet  Google Scholar 

  32. Rockafellar, R.T.: Monotone operators and augmented lagrangian methods in nonlinear programming. In: Mangasarian, O.L., Meyer, R.M., Robinson, S.M. (eds.) Nonlinear Programming 3, pp. 1–25. Academic Press, New York (1977)

    Google Scholar 

  33. Sun, D.F., Toh, K.-C., Yang, L.: A convergent 3-block semi-proximal alternating direction method of multipliers for conic programming with 4-type constraints. SIAM J. Optim. 25, 882–915 (2015)

    Article  MathSciNet  Google Scholar 

  34. Wen, Z., Goldfarb, D., Yin, W.: Alternating direction augmented Lagrangian methods for semidefinite programming. Math. Program. Comput. 2, 203–230 (2010)

    Article  MathSciNet  Google Scholar 

  35. Zhao, X.Y., Sun, D.F., Toh, K.-C.: A Newton-CG augmented Lagrangian method for semidefinite programming. SIAM J. Optim. 20, 1737–1765 (2010)

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

We would like to thank the anonymous referees and the associate editor for their useful comments and suggestions which improved this paper greatly. We are very grateful to Professor Defeng Sun at the Hong Kong Polytechnic University for sharing his knowledge with us on topics covered in this paper and beyond. The research of Y. Xiao and L. Chen was supported by the China Scholarship Council while they were visiting the National University of Singapore. The research of Y. Xiao was supported by the Major State Basic Research Development Program of China (973 Program) (Grant No. 2015CB856003), and the National Natural Science Foundation of China (Grant No. 11471101). The research of L. Chen was supported by the Fundamental Research Funds for Central Universities and the National Natural Science Foundation of China (Grant No. 11271117). The research of D. Li was supported by the National Natural Science Foundation of China (Grant No. 11371154 and 11771157).

Author information

Authors and Affiliations

  1. Institute of Applied Mathematics, College of Mathematics and Statistics, Henan University, Kaifeng, 475000, China

    Yunhai Xiao

  2. College of Mathematics and Econometrics, Hunan University, Changsha, 410082, China

    Liang Chen

  3. School of Mathematical Sciences, South China Normal University, Guangzhou, 510631, China

    Donghui Li

Authors
  1. Yunhai Xiao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Liang Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Donghui Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yunhai Xiao.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 71 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xiao, Y., Chen, L. & Li, D. A generalized alternating direction method of multipliers with semi-proximal terms for convex composite conic programming. Math. Prog. Comp. 10, 533–555 (2018). https://doi.org/10.1007/s12532-018-0134-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12532-018-0134-9

Keywords

Mathematics Subject Classification

Search

Navigation