Augmented Lagrangian Method, Dual Methods and Split Bregman Iteration for ROF Model

  • Xue-Cheng Tai
  • Chunlin Wu
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5567)


In the recent decades the ROF model (total variation (TV) minimization) has made great successes in image restoration due to its good edge-preserving property. However, the non-differentiability of the minimization problem brings computational difficulties. Different techniques have been proposed to overcome this difficulty. Therein methods regarded to be particularly efficient include dual methods of CGM (Chan, Golub, and Mulet) [7] Chambolle [6] and split Bregman iteration [14], as well as splitting-and-penalty based method [28] [29]. In this paper, we show that most of these methods can be classified under the same framework. The dual methods and split Bregman iteration are just different iterative procedures to solve the same system resulted from a Lagrangian and penalty approach. We only show this relationship for the ROF model. However, it provides a uniform framework to understand these methods for other models. In addition, we provide some examples to illustrate the accuracy and efficiency of the proposed algorithm.


Dual Variable Image Restoration Dual Method Augmented Lagrangian Method Total Variation Minimization 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bertsekas, D.P.: Multiplier methods: a survey. Automatica 12, 133–145 (1976)MathSciNetCrossRefzbMATHGoogle Scholar
  2. 2.
    Blomgren, P., Chan, T.F.: Color TV: total variation methods for restoration of vector-valued images. IEEE Trans. Image Process. 7, 304–309 (1998)CrossRefGoogle Scholar
  3. 3.
    Bregman, L.M.: The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming. USSR Computational Mathematics and Mathematical Physics 7, 200–217 (1967)MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Bresson, X., Chan, T.F.: Fast minimization of the vectorial total variation norm and applications to color image processing. UCLA CAM Report 07-25 (2007)Google Scholar
  5. 5.
    Carter, J.L.: Dual methods for total variation – based image restoration. Ph.D. thesis, UCLA (2001)Google Scholar
  6. 6.
    Chambolle, A.: An algorithm for total variation minimization and applications. J. Math. Imaging Vis. 20, 89–97 (2004)MathSciNetCrossRefGoogle Scholar
  7. 7.
    Chan, T.F., Golub, G.H., Mulet, P.: A nonlinear primal-dual method for total variation-based image restoration. SIAM J. Sci. Comput. 20, 1964–1977 (1999)MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Chan, T., Marquina, A., Mulet, P.: High-order total variation-based image restoration. SIAM J. Sci. Comput. 22, 503–516 (2000)MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    Chan, T.F., Osher, S., Shen, J.: The digital TV filter and nonlinear denoising. IEEE Trans. Image Process. 10, 231–241 (2001)CrossRefzbMATHGoogle Scholar
  10. 10.
    Chan, T.F., Kang, S.H., Shen, J.H.: Total variation denoising and enhancement of color images based on the CB and HSV color models. J. Visual Commun. Image Repres. 12, 422–435 (2001)CrossRefGoogle Scholar
  11. 11.
    Chan, T., Esedoglu, S., Park, F.E., Yip, A.: Recent developments in total variation image restoration. UCLA CAM Report 05-01 (2005)Google Scholar
  12. 12.
    Chan, T.F., Esedoglu, S., Park, F.E.: A fourth order dual method for staircase reduction in texture extraction and image restoration problems. UCLA CAM Report 05-28 (2005)Google Scholar
  13. 13.
    Glowinski, R., Le Tallec, P.: Augmented Lagrangians and operator-splitting methods in nonlinear mechanics. SIAM, Philadelphia (1989)CrossRefzbMATHGoogle Scholar
  14. 14.
    Goldstein, T., Osher, S.: The split Bregman method for L1 regularized problems. UCLA CAM Report 08-29 (2008)Google Scholar
  15. 15.
    Hestenes, M.R.: Multiplier and gradient methods. Journal of Optimization Theory and Applications 4, 303–320 (1969)MathSciNetCrossRefzbMATHGoogle Scholar
  16. 16.
    Hinterberger, W., Scherzer, O.: Variational methods on the space of functions of bounded Hessian for convexification and denoising. Computing 76, 109–133 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    Kimmel, R., Malladi, R., Sochen, N.: Images as embedded maps and minimal surfaces: movies, color, texture, and volumetric medical images. Int’l J. Computer Vision 39, 111–129 (2000)CrossRefzbMATHGoogle Scholar
  18. 18.
    Lysaker, M., Lundervold, A., Tai, X.-C.: Noise removal using fourth-order partial differential equation with applications to medical Magnetic Resonance Images in space and time. IEEE Trans. Image Process. 12, 1579–1590 (2003)CrossRefzbMATHGoogle Scholar
  19. 19.
    Lysaker, M., Tai, X.-C.: Iterative image restoration combining total variation minimization and a second order functional. Int’l J. Computer Vision 66, 5–18 (2006)CrossRefzbMATHGoogle Scholar
  20. 20.
    Osher, S., Burger, M., Goldfarb, D., Xu, J.J., Yin, W.T.: An iterative regularization method for total variation-based image restoration. SIAM Multiscale Model. Simul. 4, 460–489 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  21. 21.
    Powell, M.J.D.: A method for nonlinear constraints in minimization problems. Optimization. In: Fletcher, R. (ed.), pp. 283–298. Academic Press, New York (1972)Google Scholar
  22. 22.
    Rockafellar, R.T.: A dual approach to solving nonlinear programming problems by unconstrained optimization. Mathematical Programming 5, 354–373 (1973)MathSciNetCrossRefzbMATHGoogle Scholar
  23. 23.
    Rudin, L., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Physica D 60, 259–268 (1992)MathSciNetCrossRefzbMATHGoogle Scholar
  24. 24.
    Sapiro, G., Ringach, D.L.: Anisotropic diffusion of multivalued images with applications to color filtering. IEEE Trans. Image Process 5, 1582–1586 (1996)CrossRefzbMATHGoogle Scholar
  25. 25.
    Scherer, O.: Denoising with higher order derivatives of bounded variation and an application to parameter estimation. Computing 60, 1–27 (1998)MathSciNetCrossRefGoogle Scholar
  26. 26.
    Steidl, G.: A note on the dual treatment of higher-order regularization functionals. Computing 76, 135–148 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  27. 27.
    Tschumperlé, D., Deriche, R.: Vector-valued image regularization with PDEs: a common framework for different applications. IEEE Trans. Pattern Anal. Machine Intell. 27, 506–517 (2005)CrossRefGoogle Scholar
  28. 28.
    Wang, Y.L., Yin, W.T., Zhang, Y.: A fast algorithm for image deblurring with total variation regularization. UCLA CAM Report 07-22 (2007)Google Scholar
  29. 29.
    Wang, Y., Yang, J., Yin, W., Zhang, Y.: A new alternating minimization algorithm for total variation image reconstruction. SIAM Journal on Imaging Sciences (to appear)Google Scholar
  30. 30.
    Yin, W.T., Osher, S., Goldfarb, D., Darbon, J.: Bregman iterative algorithms for compressend sensing and related problems. SIAM J. Imaging Sciences 1, 143–168 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  31. 31.
    You, Y.-L., Kaveh, M.: Fourth-order partial differential equation for noise removal. IEEE Trans. Image Process. 9, 1723–1730 (2000)MathSciNetCrossRefzbMATHGoogle Scholar
  32. 32.
    Zhu, M., Wright, S.J., Chan, T.F.: Duality-based algorithms for total variation image restoration. UCLA CAM Report 08-33 (2008)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Xue-Cheng Tai
    • 1
  • Chunlin Wu
    • 2
  1. 1.Division of Mathematical Science, School of Physical and Mathematical Sciences, Nanyang Technological University, Singapore Department of Mathematics,University of BergenBergenNorway
  2. 2.Division of Mathematical Science, School of Physical and Mathematical SciencesNanyang Technological UniversitySingapore

Personalised recommendations