Journal of Scientific Computing

, Volume 54, Issue 2–3, pp 350–368 | Cite as

An Efficient Algorithm for 0 Minimization in Wavelet Frame Based Image Restoration

  • Bin DongEmail author
  • Yong Zhang


Wavelet frame based models for image restoration have been extensively studied for the past decade (Chan et al. in SIAM J. Sci. Comput. 24(4):1408–1432, 2003; Cai et al. in Multiscale Model. Simul. 8(2):337–369, 2009; Elad et al. in Appl. Comput. Harmon. Anal. 19(3):340–358, 2005; Starck et al. in IEEE Trans. Image Process. 14(10):1570–1582, 2005; Shen in Proceedings of the international congress of mathematicians, vol. 4, pp. 2834–2863, 2010; Dong and Shen in IAS lecture notes series, Summer program on “The mathematics of image processing”, Park City Mathematics Institute, 2010). The success of wavelet frames in image restoration is mainly due to their capability of sparsely approximating piecewise smooth functions like images. Most of the wavelet frame based models designed in the past are based on the penalization of the 1 norm of wavelet frame coefficients, which, under certain conditions, is the right choice, as supported by theories of compressed sensing (Candes et al. in Appl. Comput. Harmon. Anal., 2010; Candes et al. in IEEE Trans. Inf. Theory 52(2):489–509, 2006; Donoho in IEEE Trans. Inf. Theory 52:1289–1306, 2006). However, the assumptions of compressed sensing may not be satisfied in practice (e.g. for image deblurring and CT image reconstruction). Recently in Zhang et al. (UCLA CAM Report, vol. 11-32, 2011), the authors propose to penalize the 0 “norm” of the wavelet frame coefficients instead, and they have demonstrated significant improvements of their method over some commonly used 1 minimization models in terms of quality of the recovered images. In this paper, we propose a new algorithm, called the mean doubly augmented Lagrangian (MDAL) method, for 0 minimizations based on the classical doubly augmented Lagrangian (DAL) method (Rockafellar in Math. Oper. Res. 97–116, 1976). Our numerical experiments show that the proposed MDAL method is not only more efficient than the method proposed by Zhang et al. (UCLA CAM Report, vol. 11-32, 2011), but can also generate recovered images with even higher quality. This study reassures the feasibility of using the 0 “norm” for image restoration problems.


Augmented Lagrangian Image restoration 0 minimization Wavelet frames 


  1. 1.
    Chan, R., Chan, T., Shen, L., Shen, Z.: Wavelet algorithms for high-resolution image reconstruction. SIAM J. Sci. Comput. 24(4), 1408–1432 (2003) MathSciNetzbMATHCrossRefGoogle Scholar
  2. 2.
    Cai, J., Osher, S., Shen, Z.: Split Bregman methods and frame based image restoration. Multiscale Model. Simul. 8(2), 337–369 (2009) MathSciNetCrossRefGoogle Scholar
  3. 3.
    Elad, M., Starck, J., Querre, P., Donoho, D.: Simultaneous cartoon and texture image inpainting using morphological component analysis (MCA). Appl. Comput. Harmon. Anal. 19(3), 340–358 (2005) MathSciNetzbMATHCrossRefGoogle Scholar
  4. 4.
    Starck, J., Elad, M., Donoho, D.: Image decomposition via the combination of sparse representations and a variational approach. IEEE Trans. Image Process. 14(10), 1570–1582 (2005) MathSciNetCrossRefGoogle Scholar
  5. 5.
    Shen, Z.: Wavelet frames and image restorations. In: Proceedings of the International Congress of Mathematicians, vol. 4, pp. 2834–2863 (2010) Google Scholar
  6. 6.
    Dong, B., Shen, Z.: MRA-based wavelet frames and applications. IAS Lecture Notes Series, Summer Program on “The Mathematics of Image Processing”. Park City Mathematics Institute (2010) Google Scholar
  7. 7.
    Candes, E., Eldar, Y., Needell, D., Randall, P.: Compressed sensing with coherent and redundant dictionaries. Appl. Comput. Harmon. Ann. (2010) Google Scholar
  8. 8.
    Candes, E., Romberg, J., Tao, T.: Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information. IEEE Trans. Inf. Theory 52(2), 489–509 (2006) MathSciNetzbMATHCrossRefGoogle Scholar
  9. 9.
    Donoho, D.: Compressed sensing. IEEE Trans. Inf. Theory 52, 1289–1306 (2006) MathSciNetCrossRefGoogle Scholar
  10. 10.
    Zhang, Y., Dong, B., Lu, Z.: 0 minimization of wavelet frame based image restoration. UCLA CAM Report, vol. 11-32 (2011) Google Scholar
  11. 11.
    Rockafellar, R.: Augmented Lagrangians and applications of the proximal point algorithm in convex programming. Math. Oper. Res. 97–116 (1976) Google Scholar
  12. 12.
    Rudin, L., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Physica D 60, 259–268 (1992) zbMATHCrossRefGoogle Scholar
  13. 13.
    Chambolle, A., Lions, P.: Image recovery via total variation minimization and related problems. Numer. Math. 76(2), 167–188 (1997) MathSciNetzbMATHCrossRefGoogle Scholar
  14. 14.
    Meyer, Y.: Oscillating Patterns in Image Processing and Nonlinear Evolution Equations: The Fifteenth Dean Jacqueline B. Lewis Memorial Lectures. AMS, Providence (2001) zbMATHGoogle Scholar
  15. 15.
    Sapiro, G.: Geometric Partial Differential Equations and Image Analysis. Cambridge University Press, Cambridge (2001) zbMATHCrossRefGoogle Scholar
  16. 16.
    Osher, S., Fedkiw, R.: Level Set Methods and Dynamic Implicit Surfaces. Springer, Berlin (2003) zbMATHGoogle Scholar
  17. 17.
    Chan, T., Esedoglu, S., Park, F., Yip, A.: Total variation image restoration: overview and recent developments. In: Handbook of Mathematical Models in Computer Vision, pp. 17–31 (2006) CrossRefGoogle Scholar
  18. 18.
    Aubert, G., Kornprobst, P.: Mathematical Problems in Image Processing: Partial Differential Equations and the Calculus of Variations. Springer, Berlin (2006) zbMATHGoogle Scholar
  19. 19.
    Chan, R., Shen, L., Shen, Z.: A framelet-based approach for image inpainting. Res. Rep. 4, 325 (2005) Google Scholar
  20. 20.
    Cai, J., Osher, S., Shen, Z.: Linearized Bregman iterations for frame-based image deblurring. SIAM J. Imaging Sci. 2(1), 226–252 (2009) MathSciNetzbMATHCrossRefGoogle Scholar
  21. 21.
    Cai, J., Dong, B., Osher, S., Shen, Z.: Image restorations: total variation, wavelet frames and beyond. J. Am. Math. Soc. (2012, accepted) Google Scholar
  22. 22.
    Ron, A., Shen, Z.: Affine systems in L 2(ℝd): the analysis of the analysis operator. J. Funct. Anal. 148(2), 408–447 (1997) MathSciNetzbMATHCrossRefGoogle Scholar
  23. 23.
    Daubechies, I.: Ten Lectures on Wavelets. CBMS-NSF Lecture Notes, vol. 61. SIAM, Philadelphia (1992) zbMATHCrossRefGoogle Scholar
  24. 24.
    Daubechies, I., Han, B., Ron, A., Shen, Z.: Framelets: Mra-based constructions of wavelet frames. Appl. Comput. Harmon. Anal. 14, 1–46 (2003) MathSciNetzbMATHCrossRefGoogle Scholar
  25. 25.
    Daubechies, I., Teschke, G., Vese, L.: Iteratively solving linear inverse problems under general convex constraints. Inverse Probl. Theor. Imaging 1(1), 29 (2007) MathSciNetzbMATHCrossRefGoogle Scholar
  26. 26.
    Fadili, M., Starck, J.: Sparse representations and Bayesian image inpainting. In: Proc. SPARS, vol. 5 (2005) Google Scholar
  27. 27.
    Fadili, M., Starck, J., Murtagh, F.: Inpainting and zooming using sparse representations. Comput. J. 52(1), 64 (2009) CrossRefGoogle Scholar
  28. 28.
    Figueiredo, M., Nowak, R.: An EM algorithm for wavelet-based image restoration. IEEE Trans. Image Process. 12(8), 906–916 (2003) MathSciNetCrossRefGoogle Scholar
  29. 29.
    Figueiredo, M., Nowak, R.: A bound optimization approach to wavelet-based image deconvolution. In: IEEE International Conference on Image Processing. ICIP 2005, vol. 2, p. II-782. IEEE, New York (2005) Google Scholar
  30. 30.
    Cai, J., Chan, R., Shen, L., Shen, Z.: Convergence analysis of tight framelet approach for missing data recovery. Adv. Comput. Math. 1–27 (2008) Google Scholar
  31. 31.
    Cai, J., Chan, R., Shen, Z.: Simultaneous cartoon and texture inpainting. In: Inverse Problems and Imaging (IPI), vol. 4, pp. 379–395 (2010) Google Scholar
  32. 32.
    Candes, E., Tao, T.: Near-optimal signal recovery from random projections: universal encoding strategies? IEEE Trans. Inf. Theory 52(12), 5406–5425 (2006) MathSciNetCrossRefGoogle Scholar
  33. 33.
    Candes, E., Tao, T.: Decoding by linear programming. IEEE Trans. Inf. Theory 51(12), 4203–4215 (2005) MathSciNetCrossRefGoogle Scholar
  34. 34.
    Lu, Z., Zhang, Y.: Penalty decomposition methods for l 0-norm minimization. Technical report, Department of Mathematics, Simon Fraser University, Canada (2010) Google Scholar
  35. 35.
    Chartrand, R.: Exact reconstructions of sparse signals via nonconvex minimization. IEEE Signal Process. Lett. 14, 707–710 (2007) CrossRefGoogle Scholar
  36. 36.
    Chartrand, R., Yin, W.: Iteratively reweighted algorithms for compressive sensing. In: 33rd International Conference on Acoustics, Speech, and Signal Processing (ICASSP) (2008) Google Scholar
  37. 37.
    Chartrand, R.: Fast algorithms for nonconvex compressive sensing: MRI reconstruction from very few data. In: IEEE International Symposium on Biomedical Imaging (ISBI) (2009) Google Scholar
  38. 38.
    Hiriart-Urruty, J., Lemaréchal, C.: Convex Analysis and Minimization Algorithms: Fundamentals. Springer, Berlin (1993) zbMATHCrossRefGoogle Scholar
  39. 39.
    Goldstein, T., Osher, S.: The split Bregman algorithm for L1 regularized problems. SIAM J. Imaging Sci. 2(2), 323–343 (2009) MathSciNetzbMATHCrossRefGoogle Scholar
  40. 40.
    Zhang, X., Burger, M., Bresson, X., Osher, S.: Bregmanized nonlocal regularization for deconvolution and sparse reconstruction. SIAM J. Imaging Sci. 3, 253 (2010) MathSciNetzbMATHCrossRefGoogle Scholar
  41. 41.
    Gabay, D., Mercier, B.: A dual algorithm for the solution of nonlinear variational problems via finite element approximation. Comput. Math. Appl. 2(1), 17–40 (1976) zbMATHCrossRefGoogle Scholar
  42. 42.
    Bertsekas, D., Tsitsiklis, J.: Parallel and Distributed Computation: Numerical Methods. Prentice-Hall, New York (1989) zbMATHGoogle Scholar
  43. 43.
    Eckstein, J., Bertsekas, D.: On the Douglas “Rachford splitting method and the proximal point algorithm for maximal monotone operators”. Math. Program. 55(1), 293–318 (1992) MathSciNetzbMATHCrossRefGoogle Scholar
  44. 44.
    Hestenes, M.: Multiplier and gradient methods. J. Optim. Theory Appl. 4(5), 303–320 (1969) MathSciNetzbMATHCrossRefGoogle Scholar
  45. 45.
    Powell, M.: A method for non-linear constraints in minimization problems. In: Fletcher, R. (ed.) Optimization, pp. 283–298. Academic Press, New York (1969) Google Scholar
  46. 46.
    Glowinski, R., Le Tallec, P.: Augmented Lagrangian and Operator-Splitting Methods in Nonlinear Mechanics. SIAM, Philadelphia (1989) zbMATHCrossRefGoogle Scholar
  47. 47.
    Esser, E.: Applications of Lagrangian-based alternating direction methods and connections to split Bregman. CAM Rep. 9, 31 (2009) Google Scholar
  48. 48.
    Tai, X., Wu, C.: Augmented Lagrangian method, dual methods and split Bregman iteration for ROF model. In: Scale Space and Variational Methods in Computer Vision, pp. 502–513 (2009) CrossRefGoogle Scholar
  49. 49.
    Donoho, D.: De-noising by soft-thresholding. IEEE Trans. Inf. Theory 41, 613–627 (1995) MathSciNetzbMATHCrossRefGoogle Scholar
  50. 50.
    Mrázek, P., Weickert, J.: Rotationally invariant wavelet shrinkage. Pattern Recognit 156–163 (2003) Google Scholar
  51. 51.
    Iusem, A.: Augmented Lagrangian methods and proximal point methods for convex optimization. Investig. Oper. 8, 11–49 (1999) Google Scholar
  52. 52.
    Eckstein, J.: Nonlinear proximal point algorithms using Bregman functions, with applications to convex programming. Math. Oper. Res. 202–226 (1993) Google Scholar
  53. 53.
    Zhu, M., Chan, T.: An efficient primal-dual hybrid gradient algorithm for total variation image restoration. Mathematics Department, UCLA, CAM Report, pp. 08-34 (2007) Google Scholar
  54. 54.
    Ma, S., Yin, W., Zhang, Y., Chakraborty, A.: An efficient algorithm for compressed mr imaging using total variation and wavelets. In: IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2008, pp. 1–8. IEEE, New York (2008) Google Scholar
  55. 55.
    Wang, Y., Yang, J., Yin, W., Zhang, Y.: A new alternating minimization algorithm for total variation image reconstruction. SIAM J. Imaging Sci. 1(3), 248–272 (2008) MathSciNetzbMATHCrossRefGoogle Scholar
  56. 56.
    Yang, J., Zhang, Y., Yin, W.: A fast tvl 1-l 2 minimization algorithm for signal reconstruction from partial Fourier data. IEEE J. Sel. Top. Signal Process. 4, 288–297 (2009) CrossRefGoogle Scholar
  57. 57.
    Wang, Y., Yin, W.: Sparse signal reconstruction via iterative support detection. SIAM J. Imaging Sci. 3(3), 462–491 (2010) MathSciNetzbMATHCrossRefGoogle Scholar
  58. 58.
    Chen, Y., Hager, W., Huang, F., Phan, D., Ye, X., Yin, W.: Fast algorithms for image reconstruction with application to partially parallel MR imaging. SIAM J. Imaging Sci. 5(1), 90–118 (2010) MathSciNetCrossRefGoogle Scholar
  59. 59.
    Cai, X., Gu, G., He, B., Yuan, X.: A relaxed customized proximal point algorithm for separable convex programming. Preprint (2011) Google Scholar
  60. 60.
    Antoniadis, A., Fan, J.: Regularization of wavelet approximations. J. Am. Stat. Assoc. 96(455), 939–967 (2001) MathSciNetzbMATHCrossRefGoogle Scholar
  61. 61.
    Donoho, D., Johnstone, I.: Threshold selection for wavelet shrinkage of noisy data. In: Engineering in Medicine and Biology Society, 1994. Engineering Advances: New Opportunities for Biomedical Engineers. Proceedings of the 16th Annual International Conference of the IEEE, pp. A24–A25. IEEE, New York (1994) Google Scholar
  62. 62.
    Larsson, T., Patriksson, M., Stromberg, A.: Ergodic convergence in subgradient optimization. Optim. Methods Softw. 9(1), 93–120 (1998) MathSciNetzbMATHCrossRefGoogle Scholar
  63. 63.
    Larsson, T., Patriksson, M., Strömberg, A.: Ergodic, primal convergence in dual subgradient schemes for convex programming. Math. Program. 86(2), 283–312 (1999) MathSciNetzbMATHCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2012

Authors and Affiliations

  1. 1.Department of MathematicsThe University of ArizonaTucsonUSA
  2. 2.Department of MathematicsSimon Fraser UniversityBurnabyCanada

Personalised recommendations