Journal of Mathematical Imaging and Vision

, Volume 40, Issue 1, pp 120–145

A First-Order Primal-Dual Algorithm for Convex Problems with Applications to Imaging

Article

Abstract

In this paper we study a first-order primal-dual algorithm for non-smooth convex optimization problems with known saddle-point structure. We prove convergence to a saddle-point with rate O(1/N) in finite dimensions for the complete class of problems. We further show accelerations of the proposed algorithm to yield improved rates on problems with some degree of smoothness. In particular we show that we can achieve O(1/N2) convergence on problems, where the primal or the dual objective is uniformly convex, and we can show linear convergence, i.e. O(ωN) for some ω∈(0,1), on smooth problems. The wide applicability of the proposed algorithm is demonstrated on several imaging problems such as image denoising, image deconvolution, image inpainting, motion estimation and multi-label image segmentation.

Keywords

Convex optimization Dual approaches Total variation Inverse problems Image Reconstruction 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Arrow, K.J., Hurwicz, L., Uzawa, H.: Studies in linear and non-linear programming. In: Chenery, H.B., Johnson, S.M., Karlin, S., Marschak, T., Solow, R.M. (eds.) Stanford Mathematical Studies in the Social Sciences, vol. II. Stanford University Press, Stanford (1958) Google Scholar
  2. 2.
    Beck, A., Teboulle, M.: A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Imaging Sci. 2(1), 183–202 (2009) MathSciNetMATHCrossRefGoogle Scholar
  3. 3.
    Boyle, J.P., Dykstra, R.L.: A method for finding projections onto the intersection of convex sets in Hilbert spaces. In: Advances in Order Restricted Statistical Inference, Iowa City, Iowa, 1985. Lecture Notes in Statist., vol. 37, pp. 28–47. Springer, Berlin (1986) Google Scholar
  4. 4.
    Brakke, K.A.: Soap films and covering spaces. J. Geom. Anal. 5(4), 445–514 (1995) MathSciNetMATHGoogle Scholar
  5. 5.
    Candès, E., Demanet, L., Donoho, D., Ying, L.: Fast discrete curvelet transforms. Multiscale Model. Simul. 5(3), 861–899 (2006) (electronic) MathSciNetMATHCrossRefGoogle Scholar
  6. 6.
    Chambolle, A.: An algorithm for total variation minimization and applications. J. Math. Imaging Vis. 20(1–2), 89–97 (2004). Special issue on mathematics and image analysis MathSciNetGoogle Scholar
  7. 7.
    Chambolle, A.: Total variation minimization and a class of binary MRF models. In: Energy Minimization Methods in Computer Vision and Pattern Recognition, pp. 136–152 (2005) CrossRefGoogle Scholar
  8. 8.
    Chambolle, A., Cremers, D., Pock, T.: A convex approach for computing minimal partitions. Technical Report 649, CMAP, Ecole Polytechnique, France (2008) Google Scholar
  9. 9.
    Chambolle, A., Pock, T.: A first-order primal-dual algorithm for convex problems with applications to imaging. http://hal.archives-ouvertes.fr/hal-00490826 (2010)
  10. 10.
    Eckstein, J., Bertsekas, D.P.: On the Douglas-Rachford splitting method and the proximal point algorithm for maximal monotone operators. Math. Program., Ser. A 55(3), 293–318 (1992) MathSciNetMATHCrossRefGoogle Scholar
  11. 11.
    Esser, E.: Applications of Lagrangian-based alternating direction methods and connections to split Bregman. CAM Reports 09-31, UCLA, Center for Applied Math. (2009) Google Scholar
  12. 12.
    Esser, E., Zhang, X., Chan, T.: A general framework for a class of first order primal-dual algorithms for tv minimization. CAM Reports 09-67, UCLA, Center for Applied Math. (2009) Google Scholar
  13. 13.
    Fadili, J., Peyré, G.: Total variation projection with first order schemes. http://hal.archives-ouvertes.fr/hal-00380491/ (2009)
  14. 14.
    Fadili, J., Starck, J.-L., Elad, M., Donoho, D.: Mcalab: Reproducible research in signal and image decomposition and inpainting. Comput. Sci. Eng. 12(1), 44–63 (2010) CrossRefGoogle Scholar
  15. 15.
    Goldstein, T., Osher, S.: The split Bregman algorithm for l1 regularized problems. CAM Reports 08-29, UCLA, Center for Applied Math. (2008) Google Scholar
  16. 16.
    He, B., Yuan, X.: Convergence analysis of primal-dual algorithms for total variation image restoration. Technical Report 2790, Optimization Online, November 2010 (available at www.optimization-online.org)
  17. 17.
    Korpelevič, G.M.: An extragradient method for finding saddle points and for other problems. Èkon. Mat. Metody 12(4), 747–756 (1976) MATHGoogle Scholar
  18. 18.
    Lawlor, G., Morgan, F.: Paired calibrations applied to soap films, immiscible fluids, and surfaces or networks minimizing other norms. Pac. J. Math. 166(1), 55–83 (1994) MathSciNetMATHGoogle Scholar
  19. 19.
    Lions, P.-L., Mercier, B.: Splitting algorithms for the sum of two nonlinear operators. SIAM J. Numer. Anal. 16(6), 964–979 (1979) MathSciNetMATHCrossRefGoogle Scholar
  20. 20.
    Michelot, C.: A finite algorithm for finding the projection of a point onto the canonical simplex of R n. J. Optim. Theory Appl. 50(1), 195–200 (1986) MathSciNetMATHCrossRefGoogle Scholar
  21. 21.
    Mumford, D., Shah, J.: Optimal approximation by piecewise smooth functions and associated variational problems. Commun. Pure Appl. Math. 42, 577–685 (1989) MathSciNetMATHCrossRefGoogle Scholar
  22. 22.
    Nedić, A., Ozdaglar, A.: Subgradient methods for saddle-point problems. J. Optim. Theory Appl. 142(1), 1 (2009) MathSciNetCrossRefGoogle Scholar
  23. 23.
    Nemirovski, A.: Prox-method with rate of convergence O(1/t) for variational inequalities with Lipschitz continuous monotone operators and smooth convex-concave saddle point problems. SIAM J. Optim. 15(1), 229–251 (2004) (electronic) MathSciNetMATHCrossRefGoogle Scholar
  24. 24.
    Nemirovski, A., Yudin, D.: Problem Complexity and Method Efficiency in Optimization. A Wiley-Interscience Publication. Wiley, New York (1983). Translated from the Russian and with a preface by E.R. Dawson, Wiley-Interscience Series in Discrete Mathematics Google Scholar
  25. 25.
    Nesterov, Yu.: A method for solving the convex programming problem with convergence rate O(1/k 2). Dokl. Akad. Nauk SSSR 269(3), 543–547 (1983) MathSciNetGoogle Scholar
  26. 26.
    Nesterov, Yu.: Introductory Lectures on Convex Optimization. Applied Optimization, vol. 87. Kluwer Academic, Boston (2004). A basic course MATHGoogle Scholar
  27. 27.
    Nesterov, Yu.: Smooth minimization of non-smooth functions. Math. Program., Ser. A 103(1), 127–152 (2005) MathSciNetMATHCrossRefGoogle Scholar
  28. 28.
    Nesterov, Yu.: Gradient methods for minimizing composite objective function. Technical report, CORE DISCUSSION PAPER (2007) Google Scholar
  29. 29.
    Pock, T., Cremers, D., Bischof, H., Chambolle, A.: An algorithm for minimizing the Mumford-Shah functional. In: ICCV Proceedings, LNCS. Springer, Berlin (2009) Google Scholar
  30. 30.
    Popov, L.D.: A modification of the Arrow-Hurwitz method of search for saddle points. Mat. Zametki 28(5), 777–784, 803 (1980) MathSciNetMATHGoogle Scholar
  31. 31.
    Rockafellar, R.T.: Monotone operators and the proximal point algorithm. SIAM J. Control Optim. 14(5), 877–898 (1976) MathSciNetMATHCrossRefGoogle Scholar
  32. 32.
    Rockafellar, R.T.: Convex Analysis, Princeton Landmarks in Mathematics. Princeton University Press, Princeton (1997). Reprint of the 1970 original, Princeton Paperbacks MATHGoogle Scholar
  33. 33.
    Rudin, L., Osher, S.J., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Physica D 60, 259–268 (1992) [also in Experimental Mathematics: Computational Issues in Nonlinear Science (Proc. Los Alamos Conf. 1991)] MATHCrossRefGoogle Scholar
  34. 34.
    Shulman, D., Hervé, J.-Y.: Regularization of discontinuous flow fields. In: Proceedings Workshop on Visual Motion, pp. 81–86 (1989) CrossRefGoogle Scholar
  35. 35.
    Zach, C., Gallup, D., Frahm, J.M., Niethammer, M.: Fast global labeling for real-time stereo using multiple plane sweeps. In: Vision, Modeling, and Visualization 2008, pp. 243–252. IOS Press, Amsterdam (2008) Google Scholar
  36. 36.
    Zach, C., Pock, T., Bischof, H.: A duality based approach for realtime TV-L 1 optical flow. In: 29th DAGM Symposium on Pattern Recognition, pp. 214–223. Heidelberg, Germany (2007) CrossRefGoogle Scholar
  37. 37.
    Zhu, M., Chan, T.: An efficient primal-dual hybrid gradient algorithm for total variation image restoration. CAM Reports 08-34, UCLA, Center for Applied Math. (2008) Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2010

Authors and Affiliations

  1. 1.CMAP, Ecole PolytechniqueCNRSPalaiseauFrance
  2. 2.Institute for Computer Graphics and VisionGraz University of TechnologyGrazAustria

Personalised recommendations