Projective method of multipliers for linearly constrained convex minimization

  • Majela Pentón MachadoEmail author


We present a method for solving linearly constrained convex optimization problems, which is based on the application of known algorithms for finding zeros of the sum of two monotone operators (presented by Eckstein and Svaiter) to the dual problem. We establish convergence rates for the new method, and we present applications to TV denoising and compressed sensing problems.


Constrained optimization Convex programming Complexity Total variation denoising Compressed sensing 

Mathematics Subject Classification

49M29 90C25 65K05 68Q25 



This work is part of the author’s Ph.D. thesis, written under the supervision of Benar Fux Svaiter at IMPA, and supported by CAPES and FAPERJ. The author would like to thank Carlos Antonio Galeano Ríos and Mauricio Romero Sicre for the many helpful suggestions on this paper, which have improved the exposition considerably.


  1. 1.
    Brezis, H.: Functional Analysis. Sobolev Spaces and Partial Differential Equations. Universitext. Springer, New York (2011)zbMATHGoogle Scholar
  2. 2.
    Burachik, R.S., Sagastizábal, C.A., Svaiter, B.F., \(\epsilon \)-enlargements of maximal monotone operators: theory and applications. In: Reformulation: nonsmooth, piecewise smooth, semismooth and smoothing methods (Lausanne, 1997), vol. 22 of Appl. Optim, pp. 25–43. Kluwer Academic Publishers, Dordrecht (1999)Google Scholar
  3. 3.
    Candès, E.J., Romberg, J., Tao, T.: Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information. IEEE Trans. Inf. Theory 52(2), 489–509 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Donoho, D.L.: Compressed sensing. IEEE Trans. Inf. Theory 52(4), 1289–1306 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    Eckstein, J.: Parallel alternating direction multiplier decomposition of convex programs. J. Optim. Theory Appl. 80(1), 39–62 (1994)MathSciNetCrossRefzbMATHGoogle Scholar
  6. 6.
    Eckstein, J.: Augmented Lagrangian and alternating direction methods for convex optimization: a tutorial and some illustrative computational results. RUTCOR Research Reports 32 (2012)Google Scholar
  7. 7.
    Eckstein, J., Bertsekas, D.P.: On the Douglas-Rachford splitting method and the proximal point algorithm for maximal monotone operators. Math. Program. Ser. A 55(3), 293–318 (1992)MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Eckstein, J., Ferris, M.C.: Operator-splitting methods for monotone affine variational inequalities, with a parallel application to optimal control. INFORMS J. Comput. 10(2), 218–235 (1998)MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    Eckstein, J., Svaiter, B.F.: A family of projective splitting methods for the sum of two maximal monotone operators. Math. Program. Ser. B 111(1–2), 173–199 (2008)MathSciNetzbMATHGoogle Scholar
  10. 10.
    Esser, E.: Applications of lagrangian-based alternating direction methods and connections to split bregman. CAM Rep. 9, 31 (2009)Google Scholar
  11. 11.
    Gabay, D.: Chapter ix applications of the method of multipliers to variational inequalities. Studies Math. Appl. 15, 299–331 (1983)Google Scholar
  12. 12.
    Gabay, D., Mercier, B.: A dual algorithm for the solution of nonlinear variational problems via finite element approximation. Comput. Math. Appl. 2(1), 17–40 (1976)CrossRefzbMATHGoogle Scholar
  13. 13.
    Glowinski, R., Marrocco, A.: Sur l’approximation, par éléments finis d’ordre un, et la résolution, par pénalisation-dualité, d’une classe de problèmes de Dirichlet non linéaires. Rev. Française Automat. Informat. Recherche Opérationnelle Sér. Rouge Anal. Numér. 9(R–2), 41–76 (1975)MathSciNetzbMATHGoogle Scholar
  14. 14.
    Goldstein, T., Osher, S.: The split Bregman method for \(L1\)-regularized problems. SIAM J. Imaging Sci. 2(2), 323–343 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  15. 15.
    He, B., Yuan, X.: On the \(O(1/n)\) convergence rate of the Douglas-Rachford alternating direction method. SIAM J. Numer. Anal. 50(2), 700–709 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  16. 16.
    Hiriart–Urruty, J.B., Lemaréchal, C.: Convex analysis and minimization algorithms II: advanced theory and bundle methods. In: Grundlehren der Mathematischen Wissenschaften, vol. 306. Springer, Berlin (1993)Google Scholar
  17. 17.
    Lions, P.-L., Mercier, B.: Splitting algorithms for the sum of two nonlinear operators. SIAM J. Numer. Anal. 16(6), 964–979 (1979)MathSciNetCrossRefzbMATHGoogle Scholar
  18. 18.
    Minty, G.J.: Monotone (nonlinear) operators in Hilbert space. Duke Math. J. 29, 341–346 (1962)MathSciNetCrossRefzbMATHGoogle Scholar
  19. 19.
    Monteiro, R.D.C., Svaiter, B.F.: Iteration-complexity of block-decomposition algorithms and the alternating direction method of multipliers. SIAM J. Optim. 23(1), 475–507 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  20. 20.
    Moreau, J.-J.: Proximité et dualité dans un espace hilbertien. Bull. Soc. Math. France 93, 273–299 (1965)MathSciNetCrossRefzbMATHGoogle Scholar
  21. 21.
    Passty, G.B.: Ergodic convergence to a zero of the sum of monotone operators in Hilbert space. J. Math. Anal. Appl. 72(2), 383–390 (1979)MathSciNetCrossRefzbMATHGoogle Scholar
  22. 22.
    Rockafellar, R.T.: Convex Analysis. Princeton Mathematical Series, No. 28. Princeton University Press, Princeton, NJ (1970)CrossRefzbMATHGoogle Scholar
  23. 23.
    Rockafellar, R.T.: On the maximal monotonicity of subdifferential mappings. Pac. J. Math. 33, 209–216 (1970)MathSciNetCrossRefzbMATHGoogle Scholar
  24. 24.
    Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms (Experimental mathematics: computational issues in nonlinear science (Los Alamos, NM, 1991)). Phys. D 60(1–4), 259–268 (1992)MathSciNetCrossRefzbMATHGoogle Scholar
  25. 25.
    Tseng, P.: Applications of a splitting algorithm to decomposition in convex programming and variational inequalities. SIAM J. Control Optim. 29(1), 119–138 (1991)MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.UFBA, Universidade Federal da BahiaSalvadorBrazil

Personalised recommendations