Advertisement

Mathematical Programming Computation

, Volume 10, Issue 4, pp 533–555 | Cite as

A generalized alternating direction method of multipliers with semi-proximal terms for convex composite conic programming

  • Yunhai Xiao
  • Liang Chen
  • Donghui Li
Full Length Paper

Abstract

In this paper, we propose a generalized alternating direction method of multipliers (ADMM) with semi-proximal terms for solving a class of convex composite conic optimization problems, of which some are high-dimensional, to moderate accuracy. Our primary motivation is that this method, together with properly chosen semi-proximal terms, such as those generated by the recent advance of block symmetric Gauss–Seidel technique, is capable of tackling these problems. Moreover, the proposed method, which relaxes both the primal and the dual variables in a natural way with a common relaxation factor in the interval of (0, 2), has the potential of enhancing the performance of the classic ADMM. Extensive numerical experiments on various doubly non-negative semidefinite programming problems, with or without inequality constraints, are conducted. The corresponding results showed that all these multi-block problems can be successively solved, and the advantage of using the relaxation step is apparent.

Keywords

Convex composite conic programming Alternating direction method of multipliers Doubly non-negative semidefinite programming Relaxation Semi-proximal terms 

Mathematics Subject Classification

90C22 90C25 90C06 65K05 

Notes

Acknowledgements

We would like to thank the anonymous referees and the associate editor for their useful comments and suggestions which improved this paper greatly. We are very grateful to Professor Defeng Sun at the Hong Kong Polytechnic University for sharing his knowledge with us on topics covered in this paper and beyond. The research of Y. Xiao and L. Chen was supported by the China Scholarship Council while they were visiting the National University of Singapore. The research of Y. Xiao was supported by the Major State Basic Research Development Program of China (973 Program) (Grant No. 2015CB856003), and the National Natural Science Foundation of China (Grant No. 11471101). The research of L. Chen was supported by the Fundamental Research Funds for Central Universities and the National Natural Science Foundation of China (Grant No. 11271117). The research of D. Li was supported by the National Natural Science Foundation of China (Grant No. 11371154 and 11771157).

Supplementary material

12532_2018_134_MOESM1_ESM.pdf (72 kb)
Supplementary material 1 (pdf 71 KB)

References

  1. 1.
    Chen, C.H.: Numerical algorithms for a class of matrix norm approximation problems. Ph.D. Thesis, Department of Mathematics, Nanjing University, Nanjing, China. http://www.math.nus.edu.sg/~matsundf/Thesis_Caihua.pdf (2012)
  2. 2.
    Chen, L., Sun, D.F., Toh, K.-C.: An efficient inexact symmetric Gauss–Seidel based majorized ADMM for high-dimensional convex composite conic programming. Math. Program. 161(1), 237–270 (2017)MathSciNetCrossRefGoogle Scholar
  3. 3.
    Chen, L., Sun, D.F., Toh, K.-C.: A note on the convergence of ADMM for linearly constrained convex optimization Problems. Comput. Optim. Appl. 66(2), 327–343 (2017)MathSciNetCrossRefGoogle Scholar
  4. 4.
    Cui, Y., Li, X.D., Sun, D.F., Toh, K.-C.: On the convergence properties of a majorized ADMM for linearly constrained convex optimization problems with coupled objective functions. J. Optim. Theory Appl. 169(3), 1013–1041 (2016)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Deng, W., Lai, M.-J., Peng, Z.: W, Yin: Parallel multi-block ADMM with \(o(1/k)\) convergence. J. Sci. Comput. 71(2), 712–736 (2017)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Eckstein, J.: Some saddle-function splitting methods for convex programming. Optim. Methods Softw. 4, 75–83 (1994)CrossRefGoogle Scholar
  7. 7.
    Eckstein, J., Bertsekas, D.P.: On the Douglas–Rachford splitting method and the proximal point algorithm for maximal monotone operators. Math. Program. 55, 293–318 (1992)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Eckstein, J., Yao, W.: Understanding the convergence of the alternating direction method of multipliers: theoretical and computational perspectives. Pac. J. Optim. 11(4), 619–644 (2014)MathSciNetzbMATHGoogle Scholar
  9. 9.
    Fazel, M., Pong, T.K., Sun, D.F., Tseng, P.: Hankel matrix rank minimization with applications in system identification and realization. SIAM J. Matrix Anal. Appl. 34, 946–977 (2013)MathSciNetCrossRefGoogle Scholar
  10. 10.
    Fortin, M., Glowinski, R.: Augmented Lagrangian methods: applications to the numerical solution of boundary-value problems. Studies in mathematics and its applications, vol. 15. (translated from French by Hunt, B. and Spicer, D.C.) Elsevier Science Publishers B.V. (1983)zbMATHGoogle Scholar
  11. 11.
    Gabay, D.: Studies in mathematics and its applications. In: Fortin, M., Glowinski, R. (eds.) Applications of the method of multipliers to variational inequalities in Augmented Lagrangian Methods: Applications to the Numerical Solution of Boundary-Value Problems, vol. 15, pp. 299–331. Elsevier, Amsterdam (1983)Google Scholar
  12. 12.
    Gabay, D., Mercier, B.: A dual algorithm for the solution of nonlinear variational problems via finite element approximation. Comput. Math. Appl. 2, 17–40 (1976)CrossRefGoogle Scholar
  13. 13.
    Glowinski, R.: Lectures on numerical methods for non-linear variational problems. Published for the Tata Institute of Fundamental Research, Bombay [by] Springer (1980)Google Scholar
  14. 14.
    Glowinski, R. and Marroco, A.: Sur l’approximation, par éléments finis d’ordre un, et la résolution, par pénalisation-dualité d’une classe de problèmes de Dirichlet non linéaires. Revue française d’atomatique, Informatique Recherche Opérationelle. Analyse Numérique, 9(2), 41–76 (1975)Google Scholar
  15. 15.
    Glowinski, R.: Modeling, simulation and optimization for science and technology. In: Fitzgibbon, W., Kuznetsov, Y.A., Neittaanmaki, P., Pironneau, O. (eds.) On alternating direction methods of multipliers: A historical perspective, pp. 59–82. Springer, Netherlands (2014)zbMATHGoogle Scholar
  16. 16.
    He, B., Tao, M., Yuan, X.: Alternating direction method with Gaussian back substitution for separable convex programming. SIAM J. Optim. 22(2), 313–340 (2012)MathSciNetCrossRefGoogle Scholar
  17. 17.
    Hestenes, M.: Multiplier and gradient methods. J. Optim. Theory Appl. 4(5), 303–320 (1969)MathSciNetCrossRefGoogle Scholar
  18. 18.
    Hong, M., Chang, T.-H., Wang, X., Razaviyayn, M., Ma, S. and Luo, Z.-Q.: A block successive upper bound minimization method of multipliers for linearly constrained convex optimization. arXiv:1401.7079 (2014)
  19. 19.
    Li, X.D., Sun, D.F., Toh. K.-C.: QSDPNAL: A two-phase Newton-CG proximal augmented Lagrangian method for convex quadratic semidefinite programming problems, arXiv:1512.08872 (2015)
  20. 20.
    Li, X.D.: A two-phase augmented Lagrangian method for convex composite quadratic programming, PhD Thesis, Department of Mathematics, National University of Singapore (2015)Google Scholar
  21. 21.
    Li, M., Sun, D.F., Toh, K.-C.: A majorized ADMM with indefinite proximal terms for linearly constrained convex composite optimization. SIAM J. Optim. 26, 922–950 (2016)MathSciNetCrossRefGoogle Scholar
  22. 22.
    Li, X.D., Sun, D.F., Toh, K.-C.: A Schur complement based semi-proximal ADMM for convex quadratic conic programming and extensions. Math. Program. 155, 333–373 (2016)MathSciNetCrossRefGoogle Scholar
  23. 23.
    Lin, M., Ma, S.Q., Zhang, S.Z.: On the global linear convergence of the ADMM with multi-block variables. SIAM J. Optim. 25(3), 1478–1497 (2015)MathSciNetCrossRefGoogle Scholar
  24. 24.
    Lin, M., Ma, S.Q., Zhang, S.Z.: Iteration complexity analysis of multi-block ADMM for a family of convex minimization without strong convexity. J. Sci. Comput. 69, 52–81 (2016)MathSciNetCrossRefGoogle Scholar
  25. 25.
    Monteiro, R.D.C., Ortiz, C., Svaiter, B.F.: A first-order block-decomposition method for solving two-easy-block structured semidefinite programs. Math. Program. Comput. 6, 103–150 (2014)MathSciNetCrossRefGoogle Scholar
  26. 26.
    Monteiro, R.D.C., Ortiz, C., Svaiter, B.F.: Implementation of a block-decomposition algorithm for solving large-scale conic semidefinite programming problems. Comput. Optim. Appl. 57, 45–69 (2014)MathSciNetCrossRefGoogle Scholar
  27. 27.
    Povh, J., Rendl, F., Wiegele, A.: A boundary point method to solve semidefinite programs. Computing 78, 277–286 (2006)MathSciNetCrossRefGoogle Scholar
  28. 28.
    Powell, M.J.D.: Optimization. In: Fletcher, R. (ed.) A method for nonlinear constraints in minimization problems, pp. 283–298. Academic Press, London (1969)Google Scholar
  29. 29.
    Rockafellar, R.T.: Convex Analysis. Princeton University Press, Princeton (1970)CrossRefGoogle Scholar
  30. 30.
    Rockafellar, R.T.: Augmented Lagrangians and applications of the proximal point algorithm in convex programming. Math. Oper. Res. 1(2), 97–116 (1976)MathSciNetCrossRefGoogle Scholar
  31. 31.
    Rockafellar, R.T.: Monotone operators and the proximal point algorithm. SIAM J. Control Optim. 14(5), 877–898 (1976)MathSciNetCrossRefGoogle Scholar
  32. 32.
    Rockafellar, R.T.: Monotone operators and augmented lagrangian methods in nonlinear programming. In: Mangasarian, O.L., Meyer, R.M., Robinson, S.M. (eds.) Nonlinear Programming 3, pp. 1–25. Academic Press, New York (1977)Google Scholar
  33. 33.
    Sun, D.F., Toh, K.-C., Yang, L.: A convergent 3-block semi-proximal alternating direction method of multipliers for conic programming with 4-type constraints. SIAM J. Optim. 25, 882–915 (2015)MathSciNetCrossRefGoogle Scholar
  34. 34.
    Wen, Z., Goldfarb, D., Yin, W.: Alternating direction augmented Lagrangian methods for semidefinite programming. Math. Program. Comput. 2, 203–230 (2010)MathSciNetCrossRefGoogle Scholar
  35. 35.
    Zhao, X.Y., Sun, D.F., Toh, K.-C.: A Newton-CG augmented Lagrangian method for semidefinite programming. SIAM J. Optim. 20, 1737–1765 (2010)MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature and The Mathematical Programming Society 2018

Authors and Affiliations

  1. 1.Institute of Applied Mathematics, College of Mathematics and StatisticsHenan UniversityKaifengChina
  2. 2.College of Mathematics and EconometricsHunan UniversityChangshaChina
  3. 3.School of Mathematical SciencesSouth China Normal UniversityGuangzhouChina

Personalised recommendations