Skip to main content
SpringerLink
Log in

Inexact Alternating-Direction-Based Contraction Methods for Separable Linearly Constrained Convex Optimization

  • Published:
Journal of Optimization Theory and Applications Aims and scope Submit manuscript

Abstract

Alternating direction method of multipliers has been well studied in the context of linearly constrained convex optimization. In the last few years, we have witnessed a number of novel applications arising from image processing, compressive sensing and statistics, etc., where the approach is surprisingly efficient. In the early applications, the objective function of the linearly constrained convex optimization problem is separable into two parts. Recently, the alternating direction method of multipliers has been extended to the case where the number of the separable parts in the objective function is finite. However, in each iteration, the subproblems are required to be solved exactly. In this paper, by introducing some reasonable inexactness criteria, we propose two inexact alternating-direction-based contraction methods, which substantially broaden the applicable scope of the approach. The convergence and complexity results for both methods are derived in the framework of variational inequalities.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Floudas, C.A., Pardalos, P.M. (eds.): Encyclopedia of Optimization, 2nd edn. Springer, Berlin (2009)

    MATH  Google Scholar 

  2. Pardalos, P.M., Resende, M.G.C. (eds.): Handbook of Applied Optimization. Oxford University Press, Oxford (2002)

    MATH  Google Scholar 

  3. Ng, M.K., Weiss, P., Yuan, X.: Solving constrained total-variation image restoration and reconstruction problems via alternating direction methods. SIAM J. Sci. Comput. 32(5), 2710–2736 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  4. Yang, J., Zhang, Y.: Alternating direction algorithms for ℓ 1-problems in compressive sensing. SIAM J. Sci. Comput. 33(1), 250–278 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  5. Wen, Z., Goldfarb, D., Yin, W.: Alternating direction augmented Lagrangian methods for semidefinite programming. Math. Program. Comput. 2(3–4), 203–230 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  6. He, B., Xu, M., Yuan, X.: Solving large-scale least squares semidefinite programming by alternating direction methods. SIAM J. Matrix Anal. Appl. 32(1), 136–152 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  7. Yuan, X.: Alternating direction method for covariance selection models. J. Sci. Comput. 51(2), 261–273 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  8. Yuan, X., Yang, J.: Sparse and low-rank matrix decomposition via alternating direction method. Pac. J. Optim. 9(1), 167–180 (2013)

    MathSciNet  MATH  Google Scholar 

  9. Bertsekas, D.P.: Constrained Optimization and Lagrange Multiplier Methods. Computer Science and Applied Mathematics. Academic Press/Harcourt Brace Jovanovich Publishers, New York (1982)

    MATH  Google Scholar 

  10. Nocedal, J., Wright, S.J.: Numerical Optimization, 2nd edn. Springer Series in Operations Research and Financial Engineering. Springer, New York (2006)

    MATH  Google Scholar 

  11. Candès, E.J., Romberg, J., Tao, T.: Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information. IEEE Trans. Inf. Theory 52(2), 489–509 (2006)

    Article  MATH  Google Scholar 

  12. Chan, R.H., Yang, J., Yuan, X.: Alternating direction method for image inpainting in wavelet domains. SIAM J. Imaging Sci. 4(3), 807–826 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  13. Goldstein, T., Osher, S.: The split Bregman method for L1-regularized problems. SIAM J. Imaging Sci. 2(2), 323–343 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  14. Gabay, D., Mercier, B.: A dual algorithm for the solution of nonlinear variational problems via finite element approximation. Comput. Math. Appl. 2(1), 17–40 (1976)

    Article  MATH  Google Scholar 

  15. Douglas, J. Jr., Rachford, H.H. Jr.: On the numerical solution of heat conduction problems in two and three space variables. Trans. Am. Math. Soc. 82, 421–439 (1956)

    Article  MathSciNet  MATH  Google Scholar 

  16. He, B., Tao, M., Xu, M., Yuan, X.: An alternating direction-based contraction method for linearly constrained separable convex programming problems. Optimization 62(4), 573–596 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  17. Blum, E., Oettli, W.: Grundlagen und Verfahren. Springer, Berlin (1975). Grundlagen und Verfahren, Mit einem Anhang “Bibliographie zur Nichtlinearer Programmierung”, Ökonometrie und Unternehmensforschung, No. XX

    MATH  Google Scholar 

  18. Bertsekas, D.P., Tsitsiklis, J.N.: Parallel and Distributed Computation: Numerical Methods. Prentice-Hall, Upper Saddle River (1989)

    MATH  Google Scholar 

  19. He, B., Liao, L., Qian, M.: Alternating projection based prediction-correction methods for structured variational inequalities. J. Comput. Math. 24(6), 693–710 (2006)

    MathSciNet  MATH  Google Scholar 

  20. He, B., Liao, L., Han, D., Yang, H.: A new inexact alternating directions method for monotone variational inequalities. Math. Program., Ser. A 92(1), 103–118 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  21. Chen, G., Teboulle, M.: A proximal-based decomposition method for convex minimization problems. Math. Program., Ser. A 64(1), 81–101 (1994)

    Article  MathSciNet  MATH  Google Scholar 

  22. Rockafellar, R.T.: Monotone operators and the proximal point algorithm. SIAM J. Control Optim. 14(5), 877–898 (1976)

    Article  MathSciNet  MATH  Google Scholar 

  23. Teboulle, M.: Convergence of proximal-like algorithms. SIAM J. Optim. 7(4), 1069–1083 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  24. Tseng, P.: Alternating projection-proximal methods for convex programming and variational inequalities. SIAM J. Optim. 7(4), 951–965 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  25. He, B., Xu, M.: A general framework of contraction methods for monotone variational inequalities. Pac. J. Optim. 4(2), 195–212 (2008)

    MathSciNet  MATH  Google Scholar 

  26. Nemirovski, A.: Prox-method with rate of convergence O(1/t) for variational inequalities with Lipschitz continuous monotone operators and smooth convex-concave saddle point problems. SIAM J. Optim. 15(1), 229–251 (2004) (Electronic)

    Article  MathSciNet  MATH  Google Scholar 

  27. Facchinei, F., Pang, J.S.: Finite-Dimensional Variational Inequalities and Complementarity Problems, vol. I. Springer Series in Operations Research. Springer, New York (2003)

    MATH  Google Scholar 

  28. Cai, X., Gu, G., He, B.: On the O(1/t) convergence rate of the projection and contraction methods for variational inequalities with Lipschitz continuous monotone operators. Comput. Optim. Appl. (2013). doi:10.1007/s10589-013-9599-7

    Google Scholar 

Download references

Acknowledgements

Authors wish to thank Prof. Panos M. Pardalos (the associate editor) and Prof. Cornelis Roos (Delft University of Technology) for useful comments and suggestions.

G. Gu was supported by the NSFC grant 11001124.

B. He was supported by the NSFC grant 91130007.

J. Yang was supported by the NSFC grant 11371192.

Author information

Authors and Affiliations

  1. Department of Mathematics, Nanjing University, Nanjing, 210093, China

    Guoyong Gu, Bingsheng He & Junfeng Yang

Authors
  1. Guoyong Gu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bingsheng He
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Junfeng Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Guoyong Gu.

Additional information

Communicated by Panos M. Pardalos.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Gu, G., He, B. & Yang, J. Inexact Alternating-Direction-Based Contraction Methods for Separable Linearly Constrained Convex Optimization. J Optim Theory Appl 163, 105–129 (2014). https://doi.org/10.1007/s10957-013-0489-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10957-013-0489-z

Keywords

Search

Navigation