Skip to main content

Advertisement

SpringerLink
Log in

Inertial accelerated primal-dual methods for linear equality constrained convex optimization problems

  • Original Paper
  • Published:
Numerical Algorithms Aims and scope Submit manuscript

Abstract

In this paper, we propose an inertial accelerated primal-dual method for the linear equality constrained convex optimization problem. When the objective function has a “nonsmooth + smooth” composite structure, we further propose an inexact inertial primal-dual method by linearizing the smooth individual function and solving the subproblem inexactly. Assuming merely convexity, we prove that the proposed methods enjoy \(\mathcal {O}(1/k^{2})\) convergence rate on the objective residual and the feasibility violation in the primal model. Numerical results are reported to demonstrate the validity of the proposed methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  1. Apidopoulos, V., Aujol, J. F., Dossal, C.: Convergence rate of inertial forward-backward algorithm beyond Nesterov’s rule. Math. Program. 180(1), 137–156 (2020)

    Article  MathSciNet  Google Scholar 

  2. Attouch, H., Chbani, Z., Peypouquet, J., Redont, P.: Fast convergence of inertial dynamics and algorithms with asymptotic vanishing viscosity. Math. Program. 168(1-2), 123–175 (2018)

    Article  MathSciNet  Google Scholar 

  3. Attouch, H., Peypouquet, J.: The rate of convergence of Nesterov’s accelerated forward-backward method is actually faster than 1/k2. SIAM J. Optim. 26(3), 1824–1834 (2016)

    Article  MathSciNet  Google Scholar 

  4. Attouch, H., Peypouquet, J., Redont, P.: A dynamical approach to an inertial forward-backward algorithm for convex minimization. SIAM J. Optim. 24(1), 232–256 (2014)

    Article  MathSciNet  Google Scholar 

  5. Beck, A., Teboulle, M.: A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Imaging Sci. 2(1), 183–202 (2009)

    Article  MathSciNet  Google Scholar 

  6. Bertsekas, D.: Constrained optimization and lagrange multiplier methods. Academic Press (1982)

  7. Boţ, R. I., Csetnek, E. R., Nguyen, D. K.: Fast augmented Lagrangian method in the convex regime with convergence guarantees for the iterates. arXiv:https://arxiv.org/abs/2111.09370 (2021)

  8. Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 3, 1–122 (2010)

    Article  Google Scholar 

  9. Candes, E. J., Wakin, M. B.: An introduction to compressive sampling. IEEE Signal. Proc. Mag. 25(2), 21–30 (2008)

    Article  Google Scholar 

  10. Chen, S. S., Donoho, D. L., Saunders, M. A.: Atomic decomposition by basis pursuit. SIAM Rev. 43(1), 129–159 (2001)

    Article  MathSciNet  Google Scholar 

  11. Chen, G., Teboulle, M.: A proximal-based decomposition method for convex minimization problems. Math. Program. 64(1-3), 81–101 (1994)

    Article  MathSciNet  Google Scholar 

  12. Goldfarb, D., Ma, S., Scheinberg, K.: Fast alternating linearization methods for minimizing the sum of two convex functions. Math. Program. 141 (1-2), 349–382 (2013)

    Article  MathSciNet  Google Scholar 

  13. Gu, G., He, B., Yuan, X.: Customized proximal point algorithms for linearly constrained convex minimization and saddle-point problems: a unified approach. Comput. Optim. Appl. 59(1-2), 135–161 (2014)

    Article  MathSciNet  Google Scholar 

  14. He, X., Hu, R., Fang, Y. P.: Convergence rates of inertial primal-dual dynamical methods for separable convex optimization problems. SIAM J. Control Optim. 59(5), 3278–3301 (2021)

  15. He, B., Yuan, X.: On the acceleration of augmented Lagrangian method for linearly constrained optimization. Optimization online http://www.optimization-online.org/DB_FILE/2010/10/2760.pdf (2010)

  16. Huang, B., Ma, S., Goldfarb, D.: Accelerated linearized Bregman method. J. Sci. Comput. 54(2), 428–453 (2013)

    Article  MathSciNet  Google Scholar 

  17. Kang, M., Yun, S., Woo, H., Kang, M.: Accelerated Bregman method for linearly constrained ℓ1 − ℓ2 minimization. J. Sci. Comput. 56(3), 515–534 (2013)

    Article  MathSciNet  Google Scholar 

  18. Kang, M., Kang, M., Jung, M.: Inexact accelerated augmented Lagrangian methods. Comput. Optim. Appl. 62(2), 373–404 (2015)

    Article  MathSciNet  Google Scholar 

  19. Lin, Z., Li, H., Fang, C.: Accelerated optimization for machine learning. Springer, Singapore (2019)

  20. Li, H., Lin, Z.: Accelerated alternating direction method of multipliers: An optimal \(\mathcal {O}(1/k)\) nonergodic analysis. J. Sci. Comput. 79(2), 671–699 (2019)

    Article  MathSciNet  Google Scholar 

  21. Ma, F., Ni, M.: A class of customized proximal point algorithms for linearly constrained convex optimization. Comput. Appl. Math. 37(2), 896–911 (2018)

    Article  MathSciNet  Google Scholar 

  22. Madan, R., Lall, S.: Distributed algorithms for maximum lifetime routing in wireless sensor networks. IEEE Trans Wirel. Commun. 5(8), 2185–2193 (2006)

    Article  Google Scholar 

  23. Nedic, A., Ozdaglar, A.: Cooperative Distributed Multi-agent. Convex Optimization in Signal Processing and Communications. Cambridge University Press (2010)

  24. Liu, Y. F., Liu, X., Ma, S.: On the nonergodic convergence rate of an inexact augmented Lagrangian framework for composite convex programming. Math. Oper. Res. 44(2), 632–650 (2019)

    Article  MathSciNet  Google Scholar 

  25. Nesterov, Y.: A method of solving a convex programming problem with convergence rate \(\mathcal {O}(1/k^{2})\). Insov. Math. Dokl. 27, 372–376 (1983)

  26. Nesterov, Y.: Gradient methods for minimizing composite functions. Math. Program. 140(1), 125–161 (2013)

    Article  MathSciNet  Google Scholar 

  27. Nocedal, J., Wright, S.: Numerical optimization. Springer Science and Business Media (2006)

  28. Shi, G., Johansson, K. H.: Randomized optimal consensus of multi-agent systems. Automatica 48(12), 3018–3030 (2012)

    Article  MathSciNet  Google Scholar 

  29. Su, W., Boyd, S., Candes, E. J.: A differential equation for modeling Nesterov’s accelerated gradient method: theory and insights. J. Mach. Learn. Res. 17(1), 5312–5354 (2016)

    MathSciNet  MATH  Google Scholar 

  30. Tran-Dinh, Q., Zhu, Y.: Non-stationary first-Order primal-Dual algorithms with fast convergence rates. SIAM J. Optim. 30(4), 2866–2896 (2020)

    Article  MathSciNet  Google Scholar 

  31. Van Den Berg, E., Friedlander, M. P.: Probing the Pareto frontier for basis pursuit solutions. SIAM J. Sci. Comput. 31(2), 890–912 (2009)

    Article  MathSciNet  Google Scholar 

  32. Xu, Y.: Accelerated first-order primal-dual proximal methods for linearly constrained composite convex programming. SIAM J. Optim. 27(3), 1459–1484 (2017)

    Article  MathSciNet  Google Scholar 

  33. Yin, W., Osher, S., Goldfarb, D., Darbon, J.: Bregman iterative algorithms for ℓ1-minimization with applications to compressed sensing. SIAM J. Imaging Sci. 1(1), 143–168 (2008)

    Article  MathSciNet  Google Scholar 

  34. Zeng, X., Lei, J., Chen, J.: Dynamical primal-dual accelerated method with applications to network optimization. arXiv:https://arxiv.org/abs/1912.03690 (2019)

Download references

Acknowledgements

The authors would like to thank the referees and the editor for their helpful comments and suggestions, which have led to the improvement of this paper.

Funding

This work was supported by the National Natural Science Foundation of China (11471230).

Author information

Authors and Affiliations

  1. Department of Mathematics, Sichuan University, Chengdu, Sichuan, People’s Republic of China

    Xin He & Ya-Ping Fang

  2. Department of Applied Mathematics, Chengdu University of Information Technology, Chengdu, Sichuan, People’s Republic of China

    Rong Hu

Authors
  1. Xin He
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rong Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ya-Ping Fang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ya-Ping Fang.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

He, X., Hu, R. & Fang, YP. Inertial accelerated primal-dual methods for linear equality constrained convex optimization problems. Numer Algor 90, 1669–1690 (2022). https://doi.org/10.1007/s11075-021-01246-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11075-021-01246-y

Keywords

Mathematics Subject Classification (2010)

Search

Navigation