Skip to main content
SpringerLink
Log in

A Partially Inexact Proximal Alternating Direction Method of Multipliers and Its Iteration-Complexity Analysis

  • Published:
Journal of Optimization Theory and Applications Aims and scope Submit manuscript

Abstract

This paper proposes a partially inexact proximal alternating direction method of multipliers for computing approximate solutions of a linearly constrained convex optimization problem. This method allows its first subproblem to be solved inexactly using a relative approximate criterion, whereas a proximal term is added to its second subproblem in order to simplify it. A stepsize parameter is included in the updating rule of the Lagrangian multiplier to improve its computational performance. Pointwise and ergodic iteration-complexity bounds for the proposed method are established. To the best of our knowledge, this is the first time that complexity results for an inexact alternating direction method of multipliers with relative error criteria have been analyzed. Some preliminary numerical experiments are reported to illustrate the advantages of the new method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Gabay, D., Mercier, B.: A dual algorithm for the solution of nonlinear variational problems via finite element approximation. Comput. Math. Appl. 2(1), 17–40 (1976)

    Article  MATH  Google Scholar 

  2. Glowinski, R., Marroco, A.: Sur l’approximation, par éléments finis d’ordre un, et la résolution, par penalisation-dualité, d’une classe de problèmes de Dirichlet non linéaires. R.A.I.R.O. Anal. Numér. 9(2), 41–76 (1975)

    MATH  Google Scholar 

  3. Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 3(1), 1–122 (2011)

    Article  MATH  Google Scholar 

  4. Attouch, H., Soueycatt, M.: Augmented Lagrangian and proximal alternating direction methods of multipliers in Hilbert spaces. Applications to games, PDE’s and control. Pac. J. Optim. 5(1), 17–37 (2008)

    MathSciNet  MATH  Google Scholar 

  5. Xu, M.H.: Proximal alternating directions method for structured variational inequalities. J. Optim. Theory Appl. 134(1), 107–117 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  6. He, B., Yuan, X.: On the \(\cal{O}(1/n)\) convergence rate of the Douglas-Rachford alternating direction method. SIAM J. Numer. Anal. 50(2), 700–709 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  7. He, B., Yuan, X.: On non-ergodic convergence rate of Douglas–Rachford alternating direction method of multipliers. Numer. Math. 130(3), 567–577 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  8. Deng, W., Yin, W.: On the global and linear convergence of the generalized alternating direction method of multipliers. J. Sci. Comput. 66(3), 889–916 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  9. Gonçalves, M.L.N., Melo, J.G., Monteiro, R.D.C.: Extending the ergodic convergence rate of the proximal ADMM. arXiv preprint arXiv:1611.02903 (2016)

  10. He, B., Liao, L.Z., Han, D., Yang, H.: A new inexact alternating directions method for monotone variational inequalities. Math. Program. 92(1), 103–118 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  11. Cui, Y., Li, X., Sun, D., Toh, K.C.: On the convergence properties of a majorized ADMM for linearly constrained convex optimization problems with coupled objective functions. J. Optim. Theory Appl. 169(3), 1013–1041 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  12. Gu, Y., Jiang, B., Deren, H.: A semi-proximal-based strictly contractive Peaceman–Rachford splitting method. arXiv preprint arXiv:1506.02221 (2015)

  13. Eckstein, J., Yao, W.: Approximate ADMM algorithms derived from Lagrangian splitting. Comput. Optim. Appl. 68(2), 363–405 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  14. Eckstein, J., Yao, W.: Relative-error approximate versions of Douglas–Rachford splitting and special cases of the ADMM. Math. Program. 170(2), 417–444 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  15. Xie, J., Liao, A., Yang, X.: An inexact alternating direction method of multipliers with relative error criteria. Optim. Lett. 11(3), 583–596 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  16. Eckstein, J., Bertsekas, D.P.: On the Douglas–Rachford splitting method and the proximal point algorithm for maximal monotone operators. Math. Program. 55(3, Ser. A), 293–318 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  17. Eckstein, J., Silva, P.J.S.: A practical relative error criterion for augmented Lagrangians. Math. Program. 141(1), 319–348 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  18. Solodov, M.V., Svaiter, B.F.: A hybrid approximate extragradient-proximal point algorithm using the enlargement of a maximal monotone operator. Set Valued Anal. 7(4), 323–345 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  19. Solodov, M.V., Svaiter, B.F.: A hybrid projection-proximal point algorithm. J. Convex Anal. 6(1), 59–70 (1999)

    MathSciNet  MATH  Google Scholar 

  20. Monteiro, R.D.C., Svaiter, B.F.: Iteration-complexity of block-decomposition algorithms and the alternating direction method of multipliers. SIAM J. Optim. 23(1), 475–507 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  21. Gonçalves, M.L.N.: On the pointwise iteration-complexity of a dynamic regularized ADMM with over-relaxation stepsize. Appl. Math. Comput. 336, 315–325 (2018)

    MathSciNet  MATH  Google Scholar 

  22. Bitterlich, S., Boţ, R.I., Csetnek, E.R., Wanka, G.: The proximal alternating minimization algorithm for two-block separable convex optimization problems with linear constraints. arXiv preprint arXiv:1806.00260 (2018)

  23. Adona, V.A., Gonçalves, M.L.N., Melo, J.G.: Iteration-complexity analysis of a generalized alternating direction method of multipliers. J. Glob. Optim. (2018). https://doi.org/10.1007/s10898-018-0697-z

    Article  MATH  Google Scholar 

  24. Gonçalves, M.L.N., Melo, J.G., Monteiro, R.D.C.: Improved pointwise iteration-complexity of a regularized ADMM and of a regularized non-euclidean HPE framework. SIAM J. Optim. 27(1), 379–407 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  25. Gonçalves, M.L.N., Alves, M.M., Melo, J.G.: Pointwise and ergodic convergence rates of a variable metric proximal alternating direction method of multipliers. J. Optim. Theory Appl. 177(2), 448–478 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  26. Boţ, R.I., Csetnek, E.R.: ADMM for monotone operators: convergence analysis and rates. Adv. Comput. Math. (2018). https://doi.org/10.1007/s10444-018-9619-3

    Article  MATH  Google Scholar 

  27. Hager, W.W., Yashtini, M., Zhang, H.: An \({O}(1/k)\) convergence rate for the variable stepsize Bregman operator splitting algorithm. SIAM J. Numer. Anal. 54(3), 1535–1556 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  28. Fang, E.X., Bingsheng, H., Liu, H., Xiaoming, Y.: Generalized alternating direction method of multipliers: new theoretical insights and applications. Math. Program. Comput. 7(2), 149–187 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  29. Shefi, R., Teboulle, M.: Rate of convergence analysis of decomposition methods based on the proximal method of multipliers for convex minimization. SIAM J. Optim. 24(1), 269–297 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  30. Rockafellar, R.T.: On the maximal monotonicity of subdifferential mappings. Pac. J. Math. 33, 209–216 (1970)

    Article  MathSciNet  MATH  Google Scholar 

  31. Burachik, R.S., Sagastizábal, C.A., Svaiter, B.F.: \(\epsilon \)-Enlargements of maximal monotone operators: theory and applications. In: Reformulation: Nonsmooth, Piecewise Smooth, Semismooth and Smoothing Methods (Lausanne, 1997). Applied Optimization, vol. 22, pp. 25–43. Kluwer Acad. Publ., Dordrecht (1999)

  32. Burachik, R.S., Iusem, A.N., Svaiter, B.F.: Enlargement of monotone operators with applications to variational inequalities. Set Valued Anal. 5(2), 159–180 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  33. Monteiro, R.D.C., Svaiter, B.F.: On the complexity of the hybrid proximal extragradient method for the iterates and the ergodic mean. SIAM J. Optim. 20(6), 2755–2787 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  34. Tibshirani, R.: Regression shrinkage and selection via the lasso. J. R. Stat. Soc. Ser. B 58(1), 267–288 (1996)

    MathSciNet  MATH  Google Scholar 

  35. Tibshirani, R.J.: The lasso problem and uniqueness. Electron. J. Stat. 7, 1456–1490 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  36. Koh, K., Kim, S.J., Boyd, S.: An interior-point method for large-scale \(l_{1}\)-regularized logistic regression. J. Mach. Learn. Res. 8, 1519–1555 (2007)

    MathSciNet  MATH  Google Scholar 

  37. Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Physica D 60(1), 259–268 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  38. Parikh, N., Boyd, S.: Proximal algorithms. Found. Trends Optim. 1(3), 127–239 (2014)

    Article  Google Scholar 

  39. Beck, A.: First-Order Methods in Optimization. SIAM, Philadelphia (2017)

    Book  MATH  Google Scholar 

  40. Nocedal, J., Wright, S.J.: Numerical Optimization, 2nd edn. Springer, New York (2006)

    MATH  Google Scholar 

  41. Cano, A., Masegosa, A., Moral, S.: ELVIRA biomedical data set repository. http://leo.ugr.es/elvira/DBCRepository/ (2005). Accessed 7 Jan 2018

  42. Alon, U., Barkai, N., Notterman, D.A., Gish, K., Ybarra, S., Mack, D., Levine, A.J.: Broad patterns of gene expression revealed by clustering analysis of tumor and normal colon tissues probed by oligonucleotide arrays. Proc. Natl. Acad. Sci. USA 96(12), 6745–6750 (1999)

    Article  Google Scholar 

  43. Pomeroy, S.L., Tamayo, P., Gaasenbeek, M., Sturla, L.M., Angelo, M., McLaughlin, M.E., Kim, J.Y.H., Goumnerova, L.C., Black, P.M., Lau, C., et al.: Prediction of central nervous system embryonal tumour outcome based on gene expression. Nature 415(6870), 436–442 (2002)

    Article  Google Scholar 

  44. Singh, D., Febbo, P.G., Ross, K., Jackson, D.G., Manola, J., Ladd, C., Tamayo, P., Renshaw, A.A., D’Amico, A.V., Richie, J.P., et al.: Gene expression correlates of clinical prostate cancer behavior. Cancer Cell 1(2), 203–209 (2002)

    Article  Google Scholar 

  45. Golub, T.R., Slonim, D.K., Tamayo, P., Huard, C., Gaasenbeek, M., Mesirov, J.P., Coller, H., Loh, M.L., Downing, J.R., Caligiuri, M.A., Bloomfield, C.D., Lander, E.S.: Molecular classification of cancer: class discovery and class prediction by gene expression monitoring. Science 286(5439), 531–537 (1999)

    Article  Google Scholar 

  46. Beer, D.G., Kardia, S.L.R., Huang, C., Giordano, T.J., Levin, A.M., Misek, D.E., Lin, L., Chen, G., Gharib, T.G., Thomas, D.G., et al.: Gene-expression profiles predict survival of patients with lung adenocarcinoma. Nat. Med. 8(8), 816 (2002)

    Article  Google Scholar 

  47. Dheeru, D., Taniskidou, E.K.: UCI machine learning repository. http://archive.ics.uci.edu/ml/datasets/madelon (2018). Accessed 7 Jan 2018

  48. Sigillito, V.G., Wing, S.P., Hutton, L.V., Baker, K.B.: Classification of radar returns from the ionosphere using neural networks. Johns Hopkins APL Tech. Dig. 10, 262–266 (1989)

    Google Scholar 

Download references

Acknowledgements

The work of these authors was supported in part by CAPES, CNPq Grants 302666/2017-6 and 406975/2016-7. We thank the reviewers for their careful reading and comments.

Author information

Authors and Affiliations

  1. IME, Universidade Federal de Goiás, Goiânia, GO, 74001-970, Brazil

    Vando A. Adona, Max L. N. Gonçalves & Jefferson G. Melo

Authors
  1. Vando A. Adona
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Max L. N. Gonçalves
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jefferson G. Melo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Max L. N. Gonçalves.

Additional information

Alfredo N. Iusem.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Adona, V.A., Gonçalves, M.L.N. & Melo, J.G. A Partially Inexact Proximal Alternating Direction Method of Multipliers and Its Iteration-Complexity Analysis. J Optim Theory Appl 182, 640–666 (2019). https://doi.org/10.1007/s10957-019-01525-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10957-019-01525-8

Keywords

Mathematics Subject Classification

Search

Navigation