Skip to main content
SpringerLink
Log in

A First-Order Primal-Dual Algorithm for Convex Problems with Applications to Imaging

  • Published:
Journal of Mathematical Imaging and Vision Aims and scope Submit manuscript

Abstract

In this paper we study a first-order primal-dual algorithm for non-smooth convex optimization problems with known saddle-point structure. We prove convergence to a saddle-point with rate O(1/N) in finite dimensions for the complete class of problems. We further show accelerations of the proposed algorithm to yield improved rates on problems with some degree of smoothness. In particular we show that we can achieve O(1/N 2) convergence on problems, where the primal or the dual objective is uniformly convex, and we can show linear convergence, i.e. O(ω N) for some ω∈(0,1), on smooth problems. The wide applicability of the proposed algorithm is demonstrated on several imaging problems such as image denoising, image deconvolution, image inpainting, motion estimation and multi-label image segmentation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Arrow, K.J., Hurwicz, L., Uzawa, H.: Studies in linear and non-linear programming. In: Chenery, H.B., Johnson, S.M., Karlin, S., Marschak, T., Solow, R.M. (eds.) Stanford Mathematical Studies in the Social Sciences, vol. II. Stanford University Press, Stanford (1958)

    Google Scholar 

  2. Beck, A., Teboulle, M.: A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Imaging Sci. 2(1), 183–202 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  3. Boyle, J.P., Dykstra, R.L.: A method for finding projections onto the intersection of convex sets in Hilbert spaces. In: Advances in Order Restricted Statistical Inference, Iowa City, Iowa, 1985. Lecture Notes in Statist., vol. 37, pp. 28–47. Springer, Berlin (1986)

    Google Scholar 

  4. Brakke, K.A.: Soap films and covering spaces. J. Geom. Anal. 5(4), 445–514 (1995)

    MathSciNet  MATH  Google Scholar 

  5. Candès, E., Demanet, L., Donoho, D., Ying, L.: Fast discrete curvelet transforms. Multiscale Model. Simul. 5(3), 861–899 (2006) (electronic)

    Article  MathSciNet  MATH  Google Scholar 

  6. Chambolle, A.: An algorithm for total variation minimization and applications. J. Math. Imaging Vis. 20(1–2), 89–97 (2004). Special issue on mathematics and image analysis

    MathSciNet  Google Scholar 

  7. Chambolle, A.: Total variation minimization and a class of binary MRF models. In: Energy Minimization Methods in Computer Vision and Pattern Recognition, pp. 136–152 (2005)

    Chapter  Google Scholar 

  8. Chambolle, A., Cremers, D., Pock, T.: A convex approach for computing minimal partitions. Technical Report 649, CMAP, Ecole Polytechnique, France (2008)

  9. Chambolle, A., Pock, T.: A first-order primal-dual algorithm for convex problems with applications to imaging. http://hal.archives-ouvertes.fr/hal-00490826 (2010)

  10. Eckstein, J., Bertsekas, D.P.: On the Douglas-Rachford splitting method and the proximal point algorithm for maximal monotone operators. Math. Program., Ser. A 55(3), 293–318 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  11. Esser, E.: Applications of Lagrangian-based alternating direction methods and connections to split Bregman. CAM Reports 09-31, UCLA, Center for Applied Math. (2009)

  12. Esser, E., Zhang, X., Chan, T.: A general framework for a class of first order primal-dual algorithms for tv minimization. CAM Reports 09-67, UCLA, Center for Applied Math. (2009)

  13. Fadili, J., Peyré, G.: Total variation projection with first order schemes. http://hal.archives-ouvertes.fr/hal-00380491/ (2009)

  14. Fadili, J., Starck, J.-L., Elad, M., Donoho, D.: Mcalab: Reproducible research in signal and image decomposition and inpainting. Comput. Sci. Eng. 12(1), 44–63 (2010)

    Article  Google Scholar 

  15. Goldstein, T., Osher, S.: The split Bregman algorithm for l1 regularized problems. CAM Reports 08-29, UCLA, Center for Applied Math. (2008)

  16. He, B., Yuan, X.: Convergence analysis of primal-dual algorithms for total variation image restoration. Technical Report 2790, Optimization Online, November 2010 (available at www.optimization-online.org)

  17. Korpelevič, G.M.: An extragradient method for finding saddle points and for other problems. Èkon. Mat. Metody 12(4), 747–756 (1976)

    MATH  Google Scholar 

  18. Lawlor, G., Morgan, F.: Paired calibrations applied to soap films, immiscible fluids, and surfaces or networks minimizing other norms. Pac. J. Math. 166(1), 55–83 (1994)

    MathSciNet  MATH  Google Scholar 

  19. Lions, P.-L., Mercier, B.: Splitting algorithms for the sum of two nonlinear operators. SIAM J. Numer. Anal. 16(6), 964–979 (1979)

    Article  MathSciNet  MATH  Google Scholar 

  20. Michelot, C.: A finite algorithm for finding the projection of a point onto the canonical simplex of R n. J. Optim. Theory Appl. 50(1), 195–200 (1986)

    Article  MathSciNet  MATH  Google Scholar 

  21. Mumford, D., Shah, J.: Optimal approximation by piecewise smooth functions and associated variational problems. Commun. Pure Appl. Math. 42, 577–685 (1989)

    Article  MathSciNet  MATH  Google Scholar 

  22. Nedić, A., Ozdaglar, A.: Subgradient methods for saddle-point problems. J. Optim. Theory Appl. 142(1), 1 (2009)

    Article  MathSciNet  Google Scholar 

  23. Nemirovski, A.: Prox-method with rate of convergence O(1/t) for variational inequalities with Lipschitz continuous monotone operators and smooth convex-concave saddle point problems. SIAM J. Optim. 15(1), 229–251 (2004) (electronic)

    Article  MathSciNet  MATH  Google Scholar 

  24. Nemirovski, A., Yudin, D.: Problem Complexity and Method Efficiency in Optimization. A Wiley-Interscience Publication. Wiley, New York (1983). Translated from the Russian and with a preface by E.R. Dawson, Wiley-Interscience Series in Discrete Mathematics

    Google Scholar 

  25. Nesterov, Yu.: A method for solving the convex programming problem with convergence rate O(1/k 2). Dokl. Akad. Nauk SSSR 269(3), 543–547 (1983)

    MathSciNet  Google Scholar 

  26. Nesterov, Yu.: Introductory Lectures on Convex Optimization. Applied Optimization, vol. 87. Kluwer Academic, Boston (2004). A basic course

    MATH  Google Scholar 

  27. Nesterov, Yu.: Smooth minimization of non-smooth functions. Math. Program., Ser. A 103(1), 127–152 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  28. Nesterov, Yu.: Gradient methods for minimizing composite objective function. Technical report, CORE DISCUSSION PAPER (2007)

  29. Pock, T., Cremers, D., Bischof, H., Chambolle, A.: An algorithm for minimizing the Mumford-Shah functional. In: ICCV Proceedings, LNCS. Springer, Berlin (2009)

    Google Scholar 

  30. Popov, L.D.: A modification of the Arrow-Hurwitz method of search for saddle points. Mat. Zametki 28(5), 777–784, 803 (1980)

    MathSciNet  MATH  Google Scholar 

  31. Rockafellar, R.T.: Monotone operators and the proximal point algorithm. SIAM J. Control Optim. 14(5), 877–898 (1976)

    Article  MathSciNet  MATH  Google Scholar 

  32. Rockafellar, R.T.: Convex Analysis, Princeton Landmarks in Mathematics. Princeton University Press, Princeton (1997). Reprint of the 1970 original, Princeton Paperbacks

    MATH  Google Scholar 

  33. Rudin, L., Osher, S.J., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Physica D 60, 259–268 (1992) [also in Experimental Mathematics: Computational Issues in Nonlinear Science (Proc. Los Alamos Conf. 1991)]

    Article  MATH  Google Scholar 

  34. Shulman, D., Hervé, J.-Y.: Regularization of discontinuous flow fields. In: Proceedings Workshop on Visual Motion, pp. 81–86 (1989)

    Chapter  Google Scholar 

  35. Zach, C., Gallup, D., Frahm, J.M., Niethammer, M.: Fast global labeling for real-time stereo using multiple plane sweeps. In: Vision, Modeling, and Visualization 2008, pp. 243–252. IOS Press, Amsterdam (2008)

    Google Scholar 

  36. Zach, C., Pock, T., Bischof, H.: A duality based approach for realtime TV-L 1 optical flow. In: 29th DAGM Symposium on Pattern Recognition, pp. 214–223. Heidelberg, Germany (2007)

    Chapter  Google Scholar 

  37. Zhu, M., Chan, T.: An efficient primal-dual hybrid gradient algorithm for total variation image restoration. CAM Reports 08-34, UCLA, Center for Applied Math. (2008)

Download references

Author information

Authors and Affiliations

  1. CMAP, Ecole Polytechnique, CNRS, 91128, Palaiseau, France

    Antonin Chambolle

  2. Institute for Computer Graphics and Vision, Graz University of Technology, 8010, Graz, Austria

    Thomas Pock

Authors
  1. Antonin Chambolle
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Thomas Pock
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Antonin Chambolle.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Chambolle, A., Pock, T. A First-Order Primal-Dual Algorithm for Convex Problems with Applications to Imaging. J Math Imaging Vis 40, 120–145 (2011). https://doi.org/10.1007/s10851-010-0251-1

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10851-010-0251-1

Keywords

Search

Navigation