Advertisement

Journal of Optimization Theory and Applications

, Volume 158, Issue 2, pp 460–479 | Cite as

A Primal–Dual Splitting Method for Convex Optimization Involving Lipschitzian, Proximable and Linear Composite Terms

  • Laurent CondatEmail author
Article

Abstract

We propose a new first-order splitting algorithm for solving jointly the primal and dual formulations of large-scale convex minimization problems involving the sum of a smooth function with Lipschitzian gradient, a nonsmooth proximable function, and linear composite functions. This is a full splitting approach, in the sense that the gradient and the linear operators involved are applied explicitly without any inversion, while the nonsmooth functions are processed individually via their proximity operators. This work brings together and notably extends several classical splitting schemes, like the forward–backward and Douglas–Rachford methods, as well as the recent primal–dual method of Chambolle and Pock designed for problems with linear composite terms.

Keywords

Convex and nonsmooth optimization Operator splitting Primal–dual algorithm Forward–backward method Douglas–Rachford method Monotone inclusion Proximal method Fenchel–Rockafellar duality 

Notes

Acknowledgements

This work has been done at the author’s previous affiliation, the GREYC research center in Caen, France. The author wants to thank Jalal Fadili and Jean-Christophe Pesquet for stimulating discussions around splitting. He also thanks Isao Yamada for pointing out his article [39], which allowed to improve Lemma 4.3 in comparison with the result derived in a previous version of this manuscript. Moreover, the author is grateful to the anonymous referees and the associate editor for their valuable comments, which have contributed to the final preparation of the paper.

References

  1. 1.
    Sidky, E.Y., Jørgensen, J.H., Pan, X.: Convex optimization problem prototyping for image reconstruction in computed tomography with the Chambolle–Pock algorithm. Phys. Med. Biol. 57(10), 3065–3091 (2012) CrossRefGoogle Scholar
  2. 2.
    Mercier, B.: Topics in Finite Element Solution of Elliptic Problems. Lectures on Mathematics, vol. 63. Tata Institute of Fundamental Research, Bombay (1979) zbMATHGoogle Scholar
  3. 3.
    Lions, P.L., Mercier, B.: Splitting algorithms for the sum of two nonlinear operators. SIAM J. Numer. Anal. 16(6), 964–979 (1979) MathSciNetzbMATHCrossRefGoogle Scholar
  4. 4.
    Passty, G.B.: Ergodic convergence to a zero of the sum of monotone operators in Hilbert space. J. Math. Anal. Appl. 72, 383–390 (1979) MathSciNetzbMATHCrossRefGoogle Scholar
  5. 5.
    Eckstein, J.: Splitting methods for monotone operators with applications to parallel optimization. Ph.D. Thesis, MIT, Cambridge, MA (1989) Google Scholar
  6. 6.
    Eckstein, J., Bertsekas, D.P.: On the Douglas–Rachford splitting method and the proximal point algorithm for maximal monotone operators. Math. Program. 55, 293–318 (1992) MathSciNetzbMATHCrossRefGoogle Scholar
  7. 7.
    Combettes, P.: Solving monotone inclusions via compositions of nonexpansive averaged operators. Optimization 53(5–6), 475–504 (2004) MathSciNetzbMATHCrossRefGoogle Scholar
  8. 8.
    Moreau, J.J.: Fonctions convexes duales et points proximaux dans un espace hilbertien. C. R. Math. Acad. Sci. 255, 2897–2899 (1962) MathSciNetzbMATHGoogle Scholar
  9. 9.
    Combettes, P.L., Pesquet, J.C.: Proximal splitting methods in signal processing. In: Bauschke, H.H., Burachik, R., Combettes, P.L., Elser, V., Luke, D.R., Wolkowicz, H. (eds.) Fixed-Point Algorithms for Inverse Problems in Science and Engineering. Springer, New York (2010) Google Scholar
  10. 10.
    Goldstein, T., Osher, S.: The split Bregman method for L1-regularized problems. SIAM J. Imaging Sci. 2(2), 323–343 (2009) MathSciNetzbMATHCrossRefGoogle Scholar
  11. 11.
    Chambolle, A., Caselles, V., Cremers, D., Novaga, M., Pock, T.: An introduction to total variation for image analysis. In: Theoretical Foundations and Numerical Methods for Sparse Recovery. Radon Series Comp. Appl. Math, vol. 9, pp. 263–340. de Gruyter, Berlin (2010) Google Scholar
  12. 12.
    Becker, S., Candès, E., Grant, M.: Templates for convex cone problems with applications to sparse signal recovery. Math. Program. Comput. 3(3), 165–218 (2011) MathSciNetzbMATHCrossRefGoogle Scholar
  13. 13.
    Monteiro, R.D.C., Svaiter, B.F.: Iteration-complexity of block-decomposition algorithms and the alternating minimization augmented Lagrangian method (2010). Preprint, submitted to Math. Program. Google Scholar
  14. 14.
    Chambolle, A., Pock, T.: A first-order primal–dual algorithm for convex problems with applications to imaging. J. Math. Imaging Vis. 40(1), 120–145 (2011) MathSciNetzbMATHCrossRefGoogle Scholar
  15. 15.
    Esser, E., Zhang, X., Chan, T.: A general framework for a class of first order primal–dual algorithms for convex optimization in imaging science. SIAM J. Imaging Sci. 3(4), 1015–1046 (2010) MathSciNetzbMATHCrossRefGoogle Scholar
  16. 16.
    Zhang, X., Burger, M., Osher, S.: A unified primal-dual algorithm framework based on Bregman iteration. J. Sci. Comput. 46(1), 20–46 (2010) MathSciNetCrossRefGoogle Scholar
  17. 17.
    He, B.S., Yuan, X.M.: Convergence analysis of primal–dual algorithms for a saddle-point problem: from contraction perspective. SIAM J. Imaging Sci. 5, 119–149 (2012) MathSciNetzbMATHCrossRefGoogle Scholar
  18. 18.
    Pesquet, J.C., Pustelnik, N.: A parallel inertial proximal optimization method. Pac. J. Optim. 8(2), 273–305 (2012) MathSciNetzbMATHGoogle Scholar
  19. 19.
    Briceño-Arias, L.M., Combettes, P.L.: A monotone + skew splitting model for composite monotone inclusions in duality. SIAM J. Optim. 21(4), 1230–1250 (2011) MathSciNetzbMATHCrossRefGoogle Scholar
  20. 20.
    Combettes, P.L., Dũng, D., Vũ, B.C.: Proximity for sums of composite functions. J. Math. Anal. Appl. 380(2), 680–688 (2011) MathSciNetzbMATHCrossRefGoogle Scholar
  21. 21.
    Combettes, P.L., Pesquet, J.C.: Primal–dual splitting algorithm for solving inclusions with mixtures of composite, Lipschitzian, and parallel-sum type monotone operators. Set-Valued Var. Anal. 20(2), 307–330 (2012) MathSciNetzbMATHCrossRefGoogle Scholar
  22. 22.
    Svaiter, B.F.: On weak convergence of the Douglas–Rachford method. SIAM J. Control Optim. 49(1), 280–287 (2011) MathSciNetzbMATHCrossRefGoogle Scholar
  23. 23.
    Raguet, H., Fadili, J., Peyré, G.: Generalized forward–backward splitting (2011). Preprint. arXiv:1108.4404
  24. 24.
    Vũ, B.C.: A splitting algorithm for dual monotone inclusions involving cocoercive operators. Adv. Comput. Math. (2011, to appear). Preprint. arXiv:1110.1697. doi: 10.1007/s10444-011-9254-8
  25. 25.
    Gabay, D.: Applications of the method of multipliers to variational inequalities. In: Fortin, M., Glowinski, R. (eds.) Augmented Lagrangian Methods: Applications to the Solution of Boundary-Value Problems, Amsterdam, Netherlands (1983) Google Scholar
  26. 26.
    Tseng, P.: Applications of splitting algorithm to decomposition in convex programming and variational inequalities. SIAM J. Control Optim. 29, 119–138 (1991) MathSciNetzbMATHCrossRefGoogle Scholar
  27. 27.
    Chen, H.G.: Forward–backward splitting techniques: theory and applications. Ph.D. Thesis, University of Washington, Seattle, WA (1994) Google Scholar
  28. 28.
    Combettes, P.L., Wajs, V.R.: Signal recovery by proximal forward–backward splitting. Multiscale Model. Simul. 4, 1168–1200 (2005) MathSciNetzbMATHCrossRefGoogle Scholar
  29. 29.
    Rockafellar, R.T.: Conjugate Duality and Optimization. SIAM, Philadelphia (1974) zbMATHCrossRefGoogle Scholar
  30. 30.
    Robinson, S.M.: Composition duality and maximal monotonicity. Math. Program. 85, 1–13 (1999) MathSciNetzbMATHCrossRefGoogle Scholar
  31. 31.
    Pennanen, T.: Dualization of generalized equations of maximal monotone type. SIAM J. Optim. 10, 809–835 (2000) MathSciNetzbMATHCrossRefGoogle Scholar
  32. 32.
    Bauschke, H.H., Combettes, P.L.: Convex Analysis and Monotone Operator Theory in Hilbert Spaces. Springer, New York (2011) zbMATHCrossRefGoogle Scholar
  33. 33.
    Rockafellar, R.T.: Minimax theorems and conjugate saddle-functions. Math. Scand. 14, 151–173 (1964) MathSciNetzbMATHGoogle Scholar
  34. 34.
    McLinden, L.: An extension of Fenchel’s duality theorem to saddle functions and dual minimax problems. Pac. J. Math. 50, 135–158 (1974) MathSciNetzbMATHCrossRefGoogle Scholar
  35. 35.
    Chaux, C., Pesquet, J.C., Pustelnik, N.: Nested iterative algorithms for convex constrained image recovery problems. SIAM J. Imaging Sci. 2(2), 730–762 (2009) MathSciNetzbMATHCrossRefGoogle Scholar
  36. 36.
    Dupé, F.X., Fadili, M.J., Starck, J.L.: A proximal iteration for deconvolving Poisson noisy images using sparse representations. IEEE Trans. Image Process. 18(2), 310–321 (2009) MathSciNetCrossRefGoogle Scholar
  37. 37.
    Fadili, J.M., Peyré, G.: Total variation projection with first order schemes. IEEE Trans. Image Process. 20(3), 657–669 (2011) MathSciNetCrossRefGoogle Scholar
  38. 38.
    Chen, G., Teboulle, M.: A proximal-based decomposition method for convex minimization problems. Math. Program. 64, 81–101 (1994) MathSciNetzbMATHCrossRefGoogle Scholar
  39. 39.
    Ogura, N., Yamada, I.: Non-strictly convex minimization over the fixed point set of an asymptotically shrinking nonexpansive mapping. Numer. Funct. Anal. Optim. 23(1–2), 113–137 (2002) MathSciNetzbMATHCrossRefGoogle Scholar
  40. 40.
    Polyak, B.T.: Introduction to Optimization. Optimization Software, Inc., Publications Division, New York, USA (1987) Google Scholar

Copyright information

© Springer Science+Business Media New York 2012

Authors and Affiliations

  1. 1.GIPSA-LabCNRS—Grenoble Institute of TechnologySt. Martin d’Hères CedexFrance

Personalised recommendations