Skip to main content
SpringerLink
Log in

Generalized Peaceman–Rachford splitting method with substitution for convex programming

  • Original Paper
  • Published:
Optimization Letters Aims and scope Submit manuscript

Abstract

The generalized alternating direction method of multipliers (GADMM), which expands the dual step length to (0,2), is a benchmark for solving the two-block separable convex programming. Recently, there are many ADMM-based improved algorithms with indefinite term, that is, the second subproblem is linearized by a specialized indefinite matrix. In this paper, we propose a generalized proximal Peaceman–Rachford splitting method (abbreviated as GPRSM-S) with substitution step and indefinite term. We will find out the relationship between linearized parameter, dual step length and substitution factor. The global convergence and the worst-case convergence rate in nonergodic sense are established theoretically by variational inequality. Finally, some numerical results on LASSO and total variation based denoising problems are presented to verify the feasibility of the introduced method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  1. Adona, V.A., Goncalves, M.L.N., Melo, J.G.: Iteration-complexity analysis of a generalized alternating direction method of multipliers. J. Glob. Optim. 1, 1–18 (2018)

    MATH  Google Scholar 

  2. Alvarez, F., Attouch, H.: An inertial proximal method for maximal monotone operators via discretization of a nonlinear oscillator with damping. Set-Valued Anal. 9(1–2), 3–11 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  3. Bai, J., Li, J., Xu, F., Zhang, H.: Generalized symmetric admm for separable convex optimization. Comput. Optim. Appl. 70(1), 129–170 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  4. Beck, A., Teboulle, M.: A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Imaging Sci. 2(1), 183–202 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  5. Boyd, S.P., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. Arch. 3(1), 1–122 (2011)

    MATH  Google Scholar 

  6. Chang, X., Liu, S., Zhao, P., Li, X.: Convergent prediction-correction-based admm for multi-block separable convex programming. J. Comput. Appl. Math. 335, 270–288 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  7. Chen, C.H., Chan, R.H., Ma, S.Q., Yang, J.F.: Inertial proximal admm for linearly constrained separable convex optimization. SIAM J. Imaging Sci. 8, 2239–2267 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  8. Combettes, P.L., Wajs, V.R.: Signal recovery by proximal forward-backward splitting. Multiscale Modeling Simul. 4(4), 1168–1200 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  9. Corman, E., Yuan, X.: A generalized proximal point algorithm and its convergence rate. SIAM J. Optim. 24(4), 1614–1638 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  10. Donoho, D.L., Tsaig, Y.: Fast solution of \(l_1\)-norm minimization problems when the solution may be sparse. IEEE Trans. Inf. Theory 54(11), 4789–4812 (2008)

    Article  MATH  MathSciNet  Google Scholar 

  11. Dou, M.Y., Li, H., Liu, X.W.: An inertial proximal Peaceman–Rachford splitting method. Scientia Sinica (2017)

  12. Douglas, J., Rachford, H.H.: On the numerical solution of heat conduction problems in two and three space variables. Trans. Am. Math. Soc. 82, 421–439 (1956)

    Article  MathSciNet  MATH  Google Scholar 

  13. Eckstein, J., Bertsekas, D.P.: On the Douglas–Rachford splitting method and the proximal point algorithm for maximal monotone operators. Math. Program. 55(3), 293–318 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  14. Facchinei, F., Pang, J.: Finite-Dimensional Variational Inequalities and Complementarity Problems, Vol. 2, No. 1. Springer, New York (2003)

  15. Fu, X., He, B., Wang, X., Yuan, X.: Block-wise alternating direction method of multipliers with gaussian back substitution for multiple-block convex programming (2014)

  16. Gabay, D.: Chapter ix applications of the method of multipliers to variational inequalities. Stud. Math. Its Appl. 15, 299–331 (1983)

    Article  Google Scholar 

  17. Gabay, D., Mercier, B.: A dual algorithm for the solution of nonlinear variational problems via finite element approximation. Comput. Math. Appl. 2(1), 17–40 (1976)

    Article  MATH  Google Scholar 

  18. Gao, B., Ma, F.: Symmetric alternating direction method with indefinite proximal regularization for linearly constrained convex optimization. J. Optim. Theory Appl. 176(1), 178–204 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  19. Glowinski, R., Marrocco, A.: Sur l’approximation, par elements finis d’ordre un, et la resolution, par penalisation-dualit’e, d’une classe de problems de dirichlet non lineares. Ann. Stat. 9, 41–76 (1975)

    Google Scholar 

  20. Gu, Y., Jiang, B., Han, D.: A semi-proximal-based strictly contractive Peaceman–Rachford splitting method. Mathematics (2015)

  21. He, B., Ma, F., Yuan, X.: Optimal linearized alternating direction method of multipliers for convex programming. http://www.optimization-online.org (2017)

  22. He, B., Tao, M., Yuan, X.: Alternating direction method with Gaussian back substitution for separable convex programming. SIAM J. Optim. 22(2), 313–340 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  23. He, B., Yuan, X.: On non-ergodic convergence rate of Douglas–Rachford alternating direction method of multipliers. Numer. Math. 130(3), 567–577 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  24. He, B.S., Ma, F., Yuan, X.M.: Convergence study on the symmetric version of admm with larger step sizes. SIAM J. Imaging Sci. 9(3), 1467–1501 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  25. He, Y., Li, H., Liu, X.: Relaxed inertial proximal Peaceman–Rachford splitting method for separable convex programming. Front. Math. China 13(3), 1–24 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  26. Hestenes, M.R.: Multiplier and gradient methods. J. Optim. Theory Appl. 4(5), 303–320 (1969)

    Article  MathSciNet  MATH  Google Scholar 

  27. Jiang, F., Wu, Z., Cai, X.: Generalized admm with optimal indefinite proximal term for linearly constrained convex optimization. J. Ind. Manag. Optim. 13(5), 1–22 (2017)

    MathSciNet  Google Scholar 

  28. Lions, P.L., Mercier, B.: Splitting algorithms for the sum of two nonlinear operators. SIAM J. Numer. Anal. 16(6), 964–979 (1979)

    Article  MathSciNet  MATH  Google Scholar 

  29. Monteiro, R.D.C., Svaiter, B.F.: Iteration-complexity of block-decomposition algorithms and the alternating direction method of multipliers. SIAM J. Optim. 23(1), 475–507 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  30. Peaceman, D.W., Rachford, H.H.: The numerical solution of parabolic and elliptic differential equations. J. Soc. Ind. Appl. Math. 3(1), 28–41 (1955)

    Article  MathSciNet  MATH  Google Scholar 

  31. Powell, M.J.D.: A method for nonlinear constraints in minimization problems. Optimization 5(6), 283–298 (1969)

    MathSciNet  Google Scholar 

  32. Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. In: Eleventh International Conference of the Center for Nonlinear Studies on Experimental Mathematics: Computational Issues in Nonlinear Science: Computational Issues in Nonlinear Science, pp. 259–268 (1992)

  33. Sun, H., Tian, M., Sun, M.: The symmetric admm with indefinite proximal regularization and its application. J. Inequal. Appl. 2017(1), 172 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  34. Sun, M., Liu, J.: Generalized Peaceman–Rachford splitting method for separable convex programming with applications to image processing. J. Appl. Math. Comput. 51(1–2), 605–622 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  35. Tao, M., Yuan, X.: The generalized proximal point algorithm with step size 2 is not necessarily convergent. Comput. Optim. Appl. 70(3), 1–13 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  36. Tibshirani, R.: Regression shrinkage and selection via the lasso: a retrospective. J. R. Stat. Soc. Ser. B Stat. Methodol. 73(3), 273–282 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  37. Wang, J.J., Song, W.: An algorithm twisted from generalized admm for multi-block separable convex minimization models. J. Comput. Appl. Math. 309, 342–358 (2017)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

This work was supported by National Natural Science Foundation of China (No. 61877046), Natural Science Basic Research Program of Shanxi (No. 2017JM1001), Fundamental Research Funds for the Central Universities of China (No. JB150716).

Author information

Authors and Affiliations

  1. School of Mathematics and Statistics, Xidian University, Xi’an, 710071, People’s Republic of China

    Zhao Deng & Sanyang Liu

Authors
  1. Zhao Deng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sanyang Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhao Deng.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Deng, Z., Liu, S. Generalized Peaceman–Rachford splitting method with substitution for convex programming. Optim Lett 14, 1781–1802 (2020). https://doi.org/10.1007/s11590-019-01473-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11590-019-01473-2

Keywords

Search

Navigation