Advertisement

Journal of Optimization Theory and Applications

, Volume 164, Issue 1, pp 218–233 | Cite as

Convergence Analysis of the Generalized Alternating Direction Method of Multipliers with Logarithmic–Quadratic Proximal Regularization

  • Min Li
  • Xinxin Li
  • Xiaoming YuanEmail author
Article

Abstract

We consider combining the generalized alternating direction method of multipliers, proposed by Eckstein and Bertsekas, with the logarithmic–quadratic proximal method proposed by Auslender, Teboulle, and Ben-Tiba for solving a variational inequality with separable structures. For the derived algorithm, we prove its global convergence and establish its worst-case convergence rate measured by the iteration complexity in both the ergodic and nonergodic senses.

Keywords

Generalized alternating direction method of multipliers Logarithmic–quadratic proximal method Convergence rate Variational inequality 

Mathematics Subject Classification

90C25 90C33 65K05 

Notes

Acknowledgments

The first author was supported by the National Natural Science Foundation of China Grant 11001053, the Program for New Century Excellent Talents in University Grant NCET-12-0111, and the Natural Science Foundation of Jiangsu Province grant BK2012662. The third author was partially supported by the FRG Grant from Hong Kong Baptist University: FRG2/13-14/061 and the General Research Fund from Hong Kong Research Grants Council: 203613.

References

  1. 1.
    Eckstein, J., Bertsekas, D.: On the Douglas–Rachford splitting method and the proximal point algorithm for maximal monotone operators. Math. Program. 55, 293–318 (1992)CrossRefzbMATHMathSciNetGoogle Scholar
  2. 2.
    Auslender, A., Teboulle, M., Ben-Tiba, S.: A logarithmic-quadratic proximal method for variational inequalities. Comput. Optim. Appl. 12, 31–40 (1999)CrossRefzbMATHMathSciNetGoogle Scholar
  3. 3.
    Bertsekas, D.P., Tsitsiklis, J.N.: Parallel and Distributed Computation: Numerical Methods. Athena Scientific, Belmont (1997)Google Scholar
  4. 4.
    Fukushima, M.: Application of the alternating directions method of multipliers to separable convex programming problems. Comput. Optim. Appl. 1(1), 93–111 (1992)CrossRefzbMATHMathSciNetGoogle Scholar
  5. 5.
    Glowinski, R.: Numerical Methods for Nonlinear Variational Problems. Springer, New York (1984)CrossRefzbMATHGoogle Scholar
  6. 6.
    Glowinski, R., Tallec, P.L.: Augmented Lagrangian and Operator-Splitting Methods in Nonlinear Mechanics. Society for Industrial and Applied Mathematics, Philadelphia (1989)CrossRefzbMATHGoogle Scholar
  7. 7.
    Tseng, P.: Alternating projection-proximal methods for convex programming and variational inequalities. SIAM J. Optim. 7(4), 951–965 (1997)CrossRefzbMATHMathSciNetGoogle Scholar
  8. 8.
    Glowinski, R., Marrocco, A.: Sur l’approximation par éléments finis d’ordre un et la résolution par pénalisation-dualité d’une classe de problémes de Dirichlet non linéaires. Revue Fr. Autom. Inform. Rech. Opér. Anal. Numér. 2, 41–76 (1975)MathSciNetGoogle Scholar
  9. 9.
    Gabay, D., Mercier, B.: A dual algorithm for the solution of nonlinear variational problems via finite element approximations. Comput. Math. Appl. 2(1), 17–40 (1976)CrossRefzbMATHGoogle Scholar
  10. 10.
    Chan, T.F., Glowinski, R.: Finite element approximation and iterative solution of a class of mildly non-linear elliptic equations. Stanford University, Technical Report (1978)Google Scholar
  11. 11.
    He, B.S., Liao, L.-Z., Han, D., Yang, H.: A new inexact alternating directions method for monontone variational inequalities. Math. Program. 92, 103–118 (2002)CrossRefzbMATHMathSciNetGoogle Scholar
  12. 12.
    Kontogiorgis, S., Meyer, R.R.: A variable-penalty alternating directions method for convex optimization. Math. Program. 83, 29–53 (1998)zbMATHMathSciNetGoogle Scholar
  13. 13.
    Tseng, P.: Applications of a splitting algorithm to decomposition in convex programming and variational inequalities. SIAM J. Con. Optim. 29, 119–138 (1991)CrossRefzbMATHGoogle Scholar
  14. 14.
    Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 3, 1–122 (2010)CrossRefzbMATHGoogle Scholar
  15. 15.
    Martinet, B.: Regularision d’inéquations variationnelles par approximations successive. Revue Fr. Autom. Inform. Rech. Opér. 126, 154–159 (1970)MathSciNetGoogle Scholar
  16. 16.
    Rockafellar, R.T.: Monotone operators and the proximal point algorithm. SIAM J. Control Optim. 14(5), 877–898 (1976)CrossRefzbMATHMathSciNetGoogle Scholar
  17. 17.
    Gol’shtein, E.G., Tret’yakov, N.V.: Modified Lagrangian in convex programming and their generalizations. Math. Program. Stud. 10, 86–97 (1979)CrossRefMathSciNetGoogle Scholar
  18. 18.
    Cai, X.J., Gu, G.Y., He, B.S., Yuan, X.M.: A proximal point algorithm revisit on the alternating direction method of multipliers. Sci. China Math. 56(10), 2179–2186 (2013)CrossRefzbMATHMathSciNetGoogle Scholar
  19. 19.
    Auslender, A., Teboulle, M.: Entropic proximal decomposition method for convex programs and variational inequalities. Math. Program. 91(1), 33–47 (2001)zbMATHMathSciNetGoogle Scholar
  20. 20.
    Yuan, X.M., Li, M.: An LQP-based decomposition method for solving a class of variational inequalities. SIAM J. Optim. 21(4), 1309–1318 (2011)CrossRefzbMATHMathSciNetGoogle Scholar
  21. 21.
    Nesterov, Y.E.: A method for solving the convex programming problem with convergence rate \(O(1/k^2)\). Dokl. Akad. Nauk SSSR 269, 543–547 (1983)MathSciNetGoogle Scholar
  22. 22.
    He, B.S., Yuan, X.M.: On the \(O(1/n)\) convergence rate of the Douglas-Rachford alternating direction method. SIAM J. Numer. Anal. 50(2), 700–709 (2012)CrossRefzbMATHMathSciNetGoogle Scholar
  23. 23.
    He, B.S., Yuan, X.M.: On non-ergodic convergence rate of Douglas-Rachford alternating direction method of multipliers. Submission, (2013)Google Scholar
  24. 24.
    Shefi, R., Teboulle, M.: Rate of convergence analysis of decomposition methods based on the proximal method of multipliers for convex minimization. SIAM J. Optim. 24(1), 269–297 (2014)CrossRefzbMATHMathSciNetGoogle Scholar
  25. 25.
    Tao, M., Yuan, X.M.: On the \(O(1/t)\) convergence rate of alternating direction method with logarithmic-quadratic proximal regularization. SIAM J. Optim. 22(4), 1431–1448 (2012)CrossRefzbMATHMathSciNetGoogle Scholar
  26. 26.
    Facchinei, F., Pang, J.S.: Finite-dimensional Variational Inequalities and Complementarity Problems, Vols. I and II. Springer, New York (2003)Google Scholar
  27. 27.
    Nesterov, Y.: Gradient methods for minimizing composite objective function. Math. Program. Ser. B 140, 125–161 (2013)CrossRefzbMATHMathSciNetGoogle Scholar
  28. 28.
    He, B.S., Liao, L.-Z., Yuan, X.M.: A LQP-based interior prediction-correction method for nonlinear complementarity problems. J. Comput. Math. 24(1), 33–44 (2006)zbMATHMathSciNetGoogle Scholar
  29. 29.
    Ng, M.K., Wang, F., Yuan, X.M.: Inexact alternating direction methods for image recovery. SIAM J. Sci. Comput. 33(4), 1643–1668 (2011)CrossRefzbMATHMathSciNetGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  1. 1.School of Economics and ManagementSoutheast UniversityNanjingChina
  2. 2.Department of MathematicsHong Kong Baptist UniversityHong KongChina

Personalised recommendations