Skip to main content
SpringerLink
Log in

Preconditioned Three-Operator Splitting Algorithm with Applications to Image Restoration

  • Published:
Journal of Scientific Computing Aims and scope Submit manuscript

Abstract

In this paper, we present primal-dual splitting algorithms for the convex minimization problem involving smooth functions with Lipschitzian gradient, finite sum of nonsmooth proximable functions, and linear composite functions. Many total variation-based image processing problems are special cases of such problems. The obtained primal-dual splitting algorithms are derived from a preconditioned three-operator splitting algorithm applied to primal-dual optimality conditions in a proper product space. The convergence of the proposed algorithms under appropriate assumptions on the parameters has been proved. Numerical experiments on a novel image restoration problem are presented to demonstrate the efficiency and effectiveness of the proposed algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Data availability statements

The datasets analyzed during the current study are available from the corresponding author upon reasonable request.

References

  1. Combettes, P.L., Pesquet, J.C.: Primal-dual splitting algorithm for solving inclusions with mixtures of composite, lipschitzian, and paralle-sum type monotone operators. Set-Valued Var. Anal. 20(2), 307–330 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  2. Combettes, P.L.: Systems of structured monotone inclusions: Duality, algorithms, and applications. SIAM J. Optim. 23(4), 2420–2447 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  3. Boţ, R.I., Hendrich, C.: Convergence analysis for a primal-dual monotone+skew splitting algorithm with applications to total variation minimization. J. Math. Imaging Vis. 49, 551–568 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  4. Boţ, R.I., Csetnek, E.R.: An inertial forward-backward-forward primal-dual splitting algorithm for solving monotone inclusion problems. Numer. Algorithms 71, 519–540 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  5. Vũ, B.C.: A splitting algorithm for dual monotone inclusions involving cocoercive operators. Adv. Comput. Math. 38, 667–681 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  6. Condat, L.: A primal-dual splitting method for convex optimization involving lipschitzian, proximable and linear composite terms. J. Optim. Theory Appl. 158, 460–479 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  7. Condat, L.: A generic proximal algorithm for convex optimization-application to total variation minimization. IEEE Signal Process. Lett. 21(8), 985–989 (2014)

    Article  Google Scholar 

  8. Lorenz, D.A., Pock, T.: An inertial forward-backward algorithm for monotone inclusions. J. Math. Imaging Vis. 51, 311–325 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  9. Chambolle, A., Pock, T.: On the ergodic convergence rates of a first-order primal-dual algorithm. Math. Program. 159, 253–287 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  10. Li, Q., Zhang, N.: Fast proximity-gradient algorithms for structured convex optimization problems. Appl. Comput. Harmon. Anal. 41, 491–517 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  11. Chen, P.J., Huang, J.G., Zhang, X.Q.: A primal-dual fixed point algorithm for minimization of the sum of three convex separable functions. Fixed Point Theory and Appl. 2016, 54 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  12. Wen, M., Peng, J.G., Tang, Y.C., Zhu, C.X., Yue, S.G.: A preconditioning technique for first-order primal-dual splitting method in convex optimization. Math. Probl. Eng. 2017, 3694525 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  13. Latafat, P., Patrinos, P.: Asymmetric forward-backward-adjoint splitting for solving monotone inclusions involving three operators. Comput. Optim. Appl. 68, 57–93 (2017)

  14. Yan, M.: A new primal-dual algorithm for minimizing the sum of three functions with a linear operator. J. Sci. Comput. 76(3), 1698–1717 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  15. Wen, M., Tang, Y.C., Xing, Z.W., Peng, J.G.: A two-step inertial primal-dual algorithm for minimizing the sum of three functions. IEEE Access 7, 161748–161753 (2019)

    Article  Google Scholar 

  16. Tang, Y.C., Wu, G.R., Zhu, C.X.: A first-order splitting method for solving a large-scale composite convex optimization problem. J. Comput. Math. 37, 668–690 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  17. Huang, W.L., Tang, Y.C.: Primal-dual fixed point algorithm based on adapted metric method for solving convex minimization problem with application. Appl. Numer. Math. 157, 236–254 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  18. Raguet, H., Fadili, J., Peyré, G.: A generalized forward-backward splitting. SIAM J. Imaging Sci. 6(3), 1199–1226 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  19. Raguet, H., Landrieu, L.: Preconditioning of a generalized forward-backward splitting and application to optimization on graphs. SIAM J. Imaging Sci. 8(4), 2706–2739 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  20. Briceño-Arias, L.M.: Forward-Douglas-Rachford splitting and forward-partial inverse method for solving monotone inclusions. Optim. 64, 1239–1261 (2015)

  21. Davis, D., Yin, W.T.: A three-operator splitting scheme and its optimization applications. Set-Valued Var. Anal. 25(4), 829–858 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  22. Yurtsever, A., Vu, B. C., Cevher, V.: Stochastic three-composite convex minimization. In: Advances in Neural Information Processing Systems (NIPS), vol. 29, pp. 4322–4330 (2016)

  23. Zong, C.X., Tang, Y.C., Cho, Y.J.: Convergence analysis of an inexact three-operator splitting algorithm. Symmetry 10(11), 563 (2018)

    Article  MATH  Google Scholar 

  24. Cui, F.Y., Tang, Y.C., Yang, Y.: An inertial three-operator splitting algorithm with applications to image inpainting. Appl. Set-Valued Anal. Optim. 1(2), 113–134 (2019)

    Google Scholar 

  25. Liu, Y.L., Yin, W.T.: An envelope for Davis-Yin splitting and strict saddle point avoidance. J. Optim. Theory Appl. 181(2), 567–587 (2019)

  26. Yang, Y.X., Tang, Y.C., Wen, M., Zeng, T.Y.: Preconditioned Douglas-Rachford type primal-dual method for solving composite monotone inclusion problems with applications. Inverse Probl. Imaging 15(4), 787–825 (2021)

  27. Boţ, R.I., Hendrich, C.: A Douglas-Rachford type primal-dual method for solving inclusions with mixtures of composite and parallel -sum type monotone operators. SIAM J. Optim. 4, 2541–2565 (2013)

  28. Chambolle, A., Pock, T.: A first-order primal-dual algorithm for convex problems with applications to imaging. J. Math. Imaging Vision 40(1), 120–145 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  29. Boţ, R.I., Csetnek, E.R., Heinrich, A.: A primal-dual splitting for finding zeros of sums of maximal monotone operators. SIAM J. Optim. 23(4), 2011–2036 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  30. Alotaibi, A., Combettes, P.L., Shahzad, N.: Solving coupled composite monotone inclusions by successive fejér approximations of their kukn-tucker set. SIAM J. Optim. 24(4), 2076–2095 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  31. Alghamdi, M.A., Alotaibi, A., Combettes, P.L., Shahzad, N.: A primal-dual method of partial inverses for composite inclusions. Optim. Lett. 8(8), 2271–2284 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  32. He, C., Hu, C.H., Li, X.L., Zhang, W.: A parallel primal-dual splitting method for image restoration. Inform. Sci. 358–359, 73–91 (2016)

    Article  MATH  Google Scholar 

  33. Tang, Y.C., Zhu, C.X., Wen, M., Peng, J.G.: A splitting primal-dual proximity algorithm for solving composite optimization problems. Acta. Math. Sin.-English Ser. 33(6), 868–886 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  34. Banert, S., Ringh, A., Adler, J., Karlsson, J., Öktem, O.: Data-driven nonsmooth optimization. SIAM J. Optim. 30(1), 102–131 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  35. Dong, Y.D.: Weak convergence of an extended splitting method for monotone inclusions. J. Global Optim. 79, 257–277 (2021)

    Article  MathSciNet  MATH  Google Scholar 

  36. Combettes, P. L., Condat, L., Pesquet, J.C., Vu, B.C.: A forward-backward view of some primal-dual optimization methods in image recovery. In: Proceedings of 2014 IEEE International Conference on Image Processing, pp. 4141–4145 (2014)

  37. Chen, P.J., Huang, J.G., Zhang, X.Q.: A primal-dual fixed point algorithm for convex separable minimization with applications to image restoration. Inverse Probl. 29, 025011 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  38. Drori, Y., Sabach, S., Teboulle, M.: A simple algorithm for a class of nonsmooth convex-concave saddle-point problems. Oper. Res. Lett. 43(2), 209–214 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  39. Luke, D.R., Shefi, R.: A globally linearly convergent method for pointwise quadratically supportable convex-concave saddle point problems. J. Math. Anal. Appl. 457(2), 1568–1590 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  40. Komodakis, N., Pesquet, J.C.: Playing with duality: An overview of recent primal-dual approaches for solving large-scale optimization problems. IEEE Signal Process. Mag. 32, 31–54 (2015)

    Article  Google Scholar 

  41. He, B.S., Yuan, X.M.: Convergence analysis of primal-dual algorithms for a saddle-point problem: from contraction perspective. SIAM J. Imaging Sci. 5(1), 119–149 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  42. Condat, L., Kitahara, D., Contreras, A., Hirabayashi, A.: Proximal splitting algorithms: a tour of recent advances, with new twists. arXiv eprint, arxiv: 1912.00137 (2019)

  43. Bauschke, H.H., Combettes, P.L.: Convex Analysis and Monotone Operator Theory in Hilbert Spaces, 2nd edn. Springer, London (2017)

    Book  MATH  Google Scholar 

  44. Combettes, P.L., Vũ, B.C.: Variable metric forward-backward splitting with applications to monotone inclusions in duality. Optim. 63(9), 1289–1318 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  45. Bot, R.I., Grad, S.M., Wanka, G.: Duality in Vector Optimization. Springer-Verlag, Berlin Heidelberg (2009)

    Book  MATH  Google Scholar 

  46. Banert, S.: A relaxed forward-backward splitting algorithm for inclusions of sums of monotone operators. Master’s thesis (2012)

  47. Wang, Y.L., Yang, J.F., Yin, W.T., Zhang, Y.: A new alternating minimization algorithm for total variation image reconstruction. SIAM J. Imaging Sci. 1(3), 248–272 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  48. Beck, A., Teboulle, M.: Fast gradient-based algorithms for constrained total variation image denoising and deblurring problems. IEEE Trans. Image Process. 18(11), 2419–2434 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  49. Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distrituted optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 3, 1–122 (2010)

    Article  MATH  Google Scholar 

  50. Marquina, A., Osher, S.J.: Image super-resolution by tv-regularization and bregman iteration. J. Sci. Comput. 37, 367–382 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  51. Goldstein, T., Osher, S.: The split bregman method for \(\ell _1\)-regularized problems. SIAM J. Imaging Sci. 2, 323–343 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  52. Zhang, X., Burger, M., Osher, S.: A unified primal-dual framework based on bregman iteration. J. Sci. Comput. 46, 20–46 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  53. Chan, R.H., Tao, M., Yuan, X.M.: Constrained total variation deblurring models and fast algorithms based on alternating direction method of multipliers. SIAM J. Imaging Sci. 6(1), 680–697 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  54. Langer, A.: Investigating the influence of box-constraints on the solution of a toal variation model via an efficient primal-dual method. J. Imaging 4, 1–34 (2018)

    Article  Google Scholar 

  55. Chen, B., Tang, Y.C.: Iteative methods for computing the resolvent of the sum of a maximal monotone operator and composite operator with applications. Math. Probl. Eng. 2019, 7376263 (2019)

    MATH  Google Scholar 

  56. Liu, X.W., Tang, Y.C., Yang, Y.X.: Primal-dual algorithm to solve the constrained second-order total generalized variational model for image denoising. J. Electron. Imaging 28(4), 043017 (2019)

    Article  Google Scholar 

  57. Shi, F., Cheng, J., Wang, L., Yap, P.-T., Shen, D.: Low-rank total variation for image super-resolution. In: Medical Image Computing and Computer-Assisted Intervention-MICCAI, pp. 155–162, New York, (2013) Springer

  58. Shi, F., Cheng, J., Wang, L., Yap, P.T., Shen, D.G.: LRTV: MR image super-resolution with low-rank and total variation regularization. IEEE Tran. Med. Imaging 34(12), 2459–2466 (2015)

  59. Ma, L.Y., Xu, L., Zeng, T.Y.: Low rank prior and total variation regularization for image deblurring. J. Sci. Comput. 70, 1336–1357 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  60. Du, B., Huang, Z.Q., Wang, N., Zhang, Y.X., Jia, X.P.: Joint weighted nuclear norm and total variation regularization for hyperspectral image denoising. Int. J. Remote Sens. 39(2), 334–355 (2018)

    Article  Google Scholar 

  61. Hu, T., Li, W., Liu, N., Tao, R., Zhang, F., Scheunders, P.: Hyperspectral image restoration using adaptive anisotropy total variation and nuclear norms. IEEE Trans. Geosci. Remote Sens. 59(2), 1516–1533 (2021)

    Article  Google Scholar 

  62. Micchelli, C.A., Shen, L., Xu, Y.: Proximity algorithms for image models: denoising. Inverse Probl. 27, 045009 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  63. Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: From error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)

    Article  Google Scholar 

  64. Chan, S.H., Wang, X.R., Elgendy, O.: Plug-and-play ADMM for image resotration: fixed point convergence and applications. IEEE Trans. Comp. Imaging 3(5), 84–98 (2017)

  65. Mäkinen, Y., Azzari, L., Foi, A.: Collaborative filtering of correlated noise: exact transform-domain variance for improved shrinkage and patch matching. IEEE Trans. Image Process. 29, 8339–8354 (2020)

    Article  MATH  Google Scholar 

Download references

Acknowledgements

We thank Professor Liyan Ma for sharing the code. We are thankful to Professor Ming Yan for the helpful discussions. We would like to thank the authors of PnP\(\_\)BM3D and BM3DDEB for making their code freely available. We also thank the editor and reviewers for their recommendations and remarks, which have improved the quality of the paper.

Funding

This work was funded by the National Natural Science Foundations of China (12061045, 12001416, and 11661056).

Author information

Authors and Affiliations

  1. Department of Mathematics, Nanchang University, Nanchang, 330031, People’s Republic of China

    Yuchao Tang

  2. School of Science, Xi’an Polytechnic University, Xi’an, 710048, People’s Republic of China

    Meng Wen

  3. Department of Mathematics, The Chinese University of Hong Kong, Shatin, Hong Kong, People’s Republic of China

    Tieyong Zeng

Authors
  1. Yuchao Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Meng Wen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tieyong Zeng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yuchao Tang.

Ethics declarations

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tang, Y., Wen, M. & Zeng, T. Preconditioned Three-Operator Splitting Algorithm with Applications to Image Restoration. J Sci Comput 92, 106 (2022). https://doi.org/10.1007/s10915-022-01958-w

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10915-022-01958-w

Keywords

Mathematics Subject Classification

Search

Navigation