Skip to main content
SpringerLink
Log in

Preconditioned golden ratio primal-dual algorithm with linesearch

  • Original Paper
  • Published:
Numerical Algorithms Aims and scope Submit manuscript

Abstract

The golden ratio primal-dual algorithm (GRPDA) was proposed for solving the saddle point problems which are being widely used in a variety of areas. Compared to the popular primal-dual algorithm (PDA), GRPDA allows for larger stepsizes by replacing the extrapolation step with a convex combination step for numerical acceleration. In this paper, we incorporate preconditioning techniques into GRPDA, resulting in the preconditioned GRPDA (PreGRPDA). PreGRPDA eliminates the need for calculating the operator norm of the linear operator and allows for larger stepsizes when the operator norm is large, thus accelerating convergence. We further improve the PreGRPDA by implementing a linesearch strategy, leading to the preconditioned GRPDA with linesearch (PreGRPDA-L). In many instances, the linesearch step can adjust the preconditioners in each iteration without incurring additional costly computations, thus improving algorithm performance. Moreover, PreGRPDA-L mitigates the issue of a slower convergence rate of PreGRPDA compared to GRPDA when the operator norm of the linear operator is small. Furthermore, we establish the global convergence of the proposed algorithms under general assumptions. Finally, numerical experiments on the LASSO, CT image reconstruction, and graph cuts are presented to verify the effectiveness of proposed algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Algorithm 1
Algorithm 2
Algorithm 3
Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Data and code availability

The data and code used in the numerical experiments are available at https://github.com/LiGitGo/paper-code.

References

  1. Chambolle, A., Pock, T.: A first-order primal-dual algorithm for convex problems with applications to imaging. J. Math. Imaging Vision 40, 120–145 (2011)

    Article  MathSciNet  Google Scholar 

  2. Estellers, V., Soatto, S., Bresson, X.: Adaptive regularization with the structure tensor. IEEE Trans. Image Process. 24(6), 1777–1790 (2015)

    Article  MathSciNet  Google Scholar 

  3. Kongskov, R.D., Dong, Y., Knudsen, K.: Directional total generalized variation regularization. BIT Numer. Math. 59(4), 903–928 (2019)

    Article  MathSciNet  Google Scholar 

  4. Wen, M., Zhang, Y., Li, H., Tang, Y., Peng, J.: A fast inertial primal-dual algorithm to composite optimization models with application to image restoration problems. J. Comput. Appl. Math. 425, 115043 (2023)

    Article  MathSciNet  Google Scholar 

  5. Knoll, F., Holler, M., Koesters, T., Otazo, R., Bredies, K., Sodickson, D.K.: Joint MR-PET reconstruction using a multi-channel image regularizer. IEEE Trans. Med. Imaging 36(1), 1–16 (2016)

    Article  Google Scholar 

  6. Feijer, D., Paganini, F.: Stability of primal-dual gradient dynamics and applications to network optimization. Automatica 46(12), 1974–1981 (2010)

    Article  MathSciNet  Google Scholar 

  7. Xu, J., Tian, Y., Sun, Y., Scutari, G.: Accelerated primal-dual algorithms for distributed smooth convex optimization over networks. In: International Conference on Artificial Intelligence and Statistics, pp. 2381–2391 (2020)

  8. Donoho, D.L.: Compressed sensing. IEEE Trans. Inf. Theory 52(4), 1289–1306 (2006)

    Article  MathSciNet  Google Scholar 

  9. He, B., Yuan, X.: Convergence analysis of primal-dual algorithms for a saddle-point problem: from contraction perspective. SIAM J. Imaging Sci. 5(1), 119–149 (2012)

    Article  MathSciNet  Google Scholar 

  10. Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 3(1), 1–122 (2011)

    Article  Google Scholar 

  11. Nishihara, R., Lessard, L., Recht, B., Packard, A., Jordan, M.: A general analysis of the convergence of ADMM. In: International Conference on Machine Learning, pp. 343–352 (2015)

  12. Hong, M., Luo, Z.-Q.: On the linear convergence of the alternating direction method of multipliers. Math. Program. 162(1–2), 165–199 (2017)

    Article  MathSciNet  Google Scholar 

  13. Chambolle, A., Ehrhardt, M.J., Richtárik, P., Schonlieb, C.-B.: Stochastic primal-dual hybrid gradient algorithm with arbitrary sampling and imaging applications. SIAM J. Optim. 28(4), 2783–2808 (2018)

    Article  MathSciNet  Google Scholar 

  14. He, B., Ma, F., Xu, S., Yuan, X.: A generalized primal-dual algorithm with improved convergence condition for saddle point problems. SIAM J. Imaging Sci. 15(3), 1157–1183 (2022)

    Article  MathSciNet  Google Scholar 

  15. Malitsky, Y., Pock, T.: A first-order primal-dual algorithm with linesearch. SIAM J. Optim. 28(1), 411–432 (2018)

    Article  MathSciNet  Google Scholar 

  16. Pock, T., Chambolle, A.: Diagonal preconditioning for first order primal-dual algorithms in convex optimization. In: 2011 International Conference on Computer Vision, pp. 1762–1769 (2011)

  17. Rasch, J., Chambolle, A.: Inexact first-order primal-dual algorithms. Comput. Optim. Appl. 76(2), 381–430 (2020)

    Article  MathSciNet  Google Scholar 

  18. He, B., Ma, F., Yuan, X.: An algorithmic framework of generalized primal-dual hybrid gradient methods for saddle point problems. J. Math. Imaging Vision 58, 279–293 (2017)

    Article  MathSciNet  Google Scholar 

  19. Chang, X., Yang, J.: A golden ratio primal-dual algorithm for structured convex optimization. J. Sci. Comput. 87, 1–26 (2021)

    Article  MathSciNet  Google Scholar 

  20. Arrow, K.J., Hurwicz, L., Uzawa, H.: Studies in linear and non-linear programming. Stanford University Press, Stanford (1958)

    Google Scholar 

  21. He, B., Xu, S., Yuan, X.: On convergence of the Arrow-Hurwicz method for saddle point problems. J. Math. Imaging Vision 64(6), 662–671 (2022)

    Article  MathSciNet  Google Scholar 

  22. Chambolle, A., Pock, T.: On the ergodic convergence rates of a first-order primal-dual algorithm. Math. Program. 159(1–2), 253–287 (2016)

    Article  MathSciNet  Google Scholar 

  23. Jiang, F., Zhang, Z., He, H.: Solving saddle point problems: a landscape of primal-dual algorithm with larger stepsizes. J. Global Optim. 85(4), 821–846 (2023)

    Article  MathSciNet  Google Scholar 

  24. Liu, Y., Xu, Y., Yin, W.: Acceleration of primal-dual methods by preconditioning and simple subproblem procedures. J. Sci. Comput. 86(2), 21 (2021)

    Article  MathSciNet  Google Scholar 

  25. Ma, Y., Cai, X., Jiang, B., Han, D.: Understanding the convergence of the preconditioned PDHG method: a view of indefinite proximal ADMM. J. Sci. Comput. 94(3), 60 (2023)

    Article  MathSciNet  Google Scholar 

  26. Goldstein, T., Li, M., Yuan, X.: Adaptive primal-dual splitting methods for statistical learning and image processing. Adv. Neural Inf. Process. Syst. 28, 2089–2097 (2015)

    Google Scholar 

  27. Yokota, T., Hontani, H.: An efficient method for adapting step-size parameters of primal-dual hybrid gradient method in application to total variation regularization. In: 2017 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC), pp. 973–979 (2017)

  28. Chang, X., Yang, J.: GRPDA revisited: relaxed condition and connection to Chambolle-Pock’s primal-dual algorithm. J. Sci. Comput. 93(3), 70 (2022)

    Article  MathSciNet  Google Scholar 

  29. Chang, X.-K., Yang, J., Zhang, H.: Golden ratio primal-dual algorithm with linesearch. SIAM J. Optim. 32(3), 1584–1613 (2022)

    Article  MathSciNet  Google Scholar 

  30. Beck, A.: First-order methods in optimization. SIAM, Philadelphia (2017)

    Book  Google Scholar 

  31. Tibshirani, R.: Regression shrinkage and selection via the lasso. J. Royal Stat. Soc. Ser. B: Stat. Methodol. 58(1), 267–288 (1996)

  32. Sidky, E.Y., Jørgensen, J.H., Pan, X.: Convex optimization problem prototyping for image reconstruction in computed tomography with the Chambolle-Pock algorithm. Phys. Med. Biol. 57(10), 3065 (2012)

    Article  Google Scholar 

  33. Hansen, P.C., Jørgensen, J.S.: Air tools II: algebraic iterative reconstruction methods, improved implementation. Numer. Algorithm. 79(1), 107–137 (2018)

    Article  MathSciNet  Google Scholar 

  34. Chambolle, A.: Total variation minimization and a class of binary MRF models. In: International Workshop on Energy Minimization Methods in Computer Vision and Pattern Recognition, pp. 136–152 (2005)

  35. Grant, M., Boyd, S.: CVX: Matlab software for disciplined convex programming, version 2.1 (2014)

Download references

Funding

Feng Ma was supported by the National Natural Science Foundation of China under Grant 12171481.

Author information

Authors and Affiliations

  1. School of Automation, Central South University, Changsha, 410083, China

    Shan Ma & Si Li

  2. High-Tech Institute of Xi’an, Xi’an, 710025, China

    Feng Ma

Authors
  1. Shan Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Si Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Feng Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

F. Ma conceived of the presented idea. S. Ma and S. Li developed the theory and performed the numerical computations. F. Ma supervised the findings of this work. All authors discussed the results and contributed to the final manuscript.

Corresponding author

Correspondence to Feng Ma.

Ethics declarations

Ethical approval and consent to participate

Not applicable

Consent for publication

Not applicable

Conflict of interest

The authors declare no competing interests.

Human and animal ethics

Not applicable

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ma, S., Li, S. & Ma, F. Preconditioned golden ratio primal-dual algorithm with linesearch. Numer Algor (2024). https://doi.org/10.1007/s11075-024-01834-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11075-024-01834-8

Keywords

Search

Navigation