Skip to main content
SpringerLink
Log in

An Efficient Algorithm for ℓ 0 Minimization in Wavelet Frame Based Image Restoration

  • Published:
Journal of Scientific Computing Aims and scope Submit manuscript

Abstract

Wavelet frame based models for image restoration have been extensively studied for the past decade (Chan et al. in SIAM J. Sci. Comput. 24(4):1408–1432, 2003; Cai et al. in Multiscale Model. Simul. 8(2):337–369, 2009; Elad et al. in Appl. Comput. Harmon. Anal. 19(3):340–358, 2005; Starck et al. in IEEE Trans. Image Process. 14(10):1570–1582, 2005; Shen in Proceedings of the international congress of mathematicians, vol. 4, pp. 2834–2863, 2010; Dong and Shen in IAS lecture notes series, Summer program on “The mathematics of image processing”, Park City Mathematics Institute, 2010). The success of wavelet frames in image restoration is mainly due to their capability of sparsely approximating piecewise smooth functions like images. Most of the wavelet frame based models designed in the past are based on the penalization of the ℓ 1 norm of wavelet frame coefficients, which, under certain conditions, is the right choice, as supported by theories of compressed sensing (Candes et al. in Appl. Comput. Harmon. Anal., 2010; Candes et al. in IEEE Trans. Inf. Theory 52(2):489–509, 2006; Donoho in IEEE Trans. Inf. Theory 52:1289–1306, 2006). However, the assumptions of compressed sensing may not be satisfied in practice (e.g. for image deblurring and CT image reconstruction). Recently in Zhang et al. (UCLA CAM Report, vol. 11-32, 2011), the authors propose to penalize the ℓ 0 “norm” of the wavelet frame coefficients instead, and they have demonstrated significant improvements of their method over some commonly used ℓ 1 minimization models in terms of quality of the recovered images. In this paper, we propose a new algorithm, called the mean doubly augmented Lagrangian (MDAL) method, for ℓ 0 minimizations based on the classical doubly augmented Lagrangian (DAL) method (Rockafellar in Math. Oper. Res. 97–116, 1976). Our numerical experiments show that the proposed MDAL method is not only more efficient than the method proposed by Zhang et al. (UCLA CAM Report, vol. 11-32, 2011), but can also generate recovered images with even higher quality. This study reassures the feasibility of using the ℓ 0 “norm” for image restoration problems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Algorithm 1
Fig. 1
Algorithm 2
Algorithm 3
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Chan, R., Chan, T., Shen, L., Shen, Z.: Wavelet algorithms for high-resolution image reconstruction. SIAM J. Sci. Comput. 24(4), 1408–1432 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  2. Cai, J., Osher, S., Shen, Z.: Split Bregman methods and frame based image restoration. Multiscale Model. Simul. 8(2), 337–369 (2009)

    Article  MathSciNet  Google Scholar 

  3. Elad, M., Starck, J., Querre, P., Donoho, D.: Simultaneous cartoon and texture image inpainting using morphological component analysis (MCA). Appl. Comput. Harmon. Anal. 19(3), 340–358 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  4. Starck, J., Elad, M., Donoho, D.: Image decomposition via the combination of sparse representations and a variational approach. IEEE Trans. Image Process. 14(10), 1570–1582 (2005)

    Article  MathSciNet  Google Scholar 

  5. Shen, Z.: Wavelet frames and image restorations. In: Proceedings of the International Congress of Mathematicians, vol. 4, pp. 2834–2863 (2010)

    Google Scholar 

  6. Dong, B., Shen, Z.: MRA-based wavelet frames and applications. IAS Lecture Notes Series, Summer Program on “The Mathematics of Image Processing”. Park City Mathematics Institute (2010)

  7. Candes, E., Eldar, Y., Needell, D., Randall, P.: Compressed sensing with coherent and redundant dictionaries. Appl. Comput. Harmon. Ann. (2010)

  8. Candes, E., Romberg, J., Tao, T.: Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information. IEEE Trans. Inf. Theory 52(2), 489–509 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  9. Donoho, D.: Compressed sensing. IEEE Trans. Inf. Theory 52, 1289–1306 (2006)

    Article  MathSciNet  Google Scholar 

  10. Zhang, Y., Dong, B., Lu, Z.: ℓ 0 minimization of wavelet frame based image restoration. UCLA CAM Report, vol. 11-32 (2011)

  11. Rockafellar, R.: Augmented Lagrangians and applications of the proximal point algorithm in convex programming. Math. Oper. Res. 97–116 (1976)

  12. Rudin, L., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Physica D 60, 259–268 (1992)

    Article  MATH  Google Scholar 

  13. Chambolle, A., Lions, P.: Image recovery via total variation minimization and related problems. Numer. Math. 76(2), 167–188 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  14. Meyer, Y.: Oscillating Patterns in Image Processing and Nonlinear Evolution Equations: The Fifteenth Dean Jacqueline B. Lewis Memorial Lectures. AMS, Providence (2001)

    MATH  Google Scholar 

  15. Sapiro, G.: Geometric Partial Differential Equations and Image Analysis. Cambridge University Press, Cambridge (2001)

    Book  MATH  Google Scholar 

  16. Osher, S., Fedkiw, R.: Level Set Methods and Dynamic Implicit Surfaces. Springer, Berlin (2003)

    MATH  Google Scholar 

  17. Chan, T., Esedoglu, S., Park, F., Yip, A.: Total variation image restoration: overview and recent developments. In: Handbook of Mathematical Models in Computer Vision, pp. 17–31 (2006)

    Chapter  Google Scholar 

  18. Aubert, G., Kornprobst, P.: Mathematical Problems in Image Processing: Partial Differential Equations and the Calculus of Variations. Springer, Berlin (2006)

    MATH  Google Scholar 

  19. Chan, R., Shen, L., Shen, Z.: A framelet-based approach for image inpainting. Res. Rep. 4, 325 (2005)

    Google Scholar 

  20. Cai, J., Osher, S., Shen, Z.: Linearized Bregman iterations for frame-based image deblurring. SIAM J. Imaging Sci. 2(1), 226–252 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  21. Cai, J., Dong, B., Osher, S., Shen, Z.: Image restorations: total variation, wavelet frames and beyond. J. Am. Math. Soc. (2012, accepted)

  22. Ron, A., Shen, Z.: Affine systems in L 2(ℝd): the analysis of the analysis operator. J. Funct. Anal. 148(2), 408–447 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  23. Daubechies, I.: Ten Lectures on Wavelets. CBMS-NSF Lecture Notes, vol. 61. SIAM, Philadelphia (1992)

    Book  MATH  Google Scholar 

  24. Daubechies, I., Han, B., Ron, A., Shen, Z.: Framelets: Mra-based constructions of wavelet frames. Appl. Comput. Harmon. Anal. 14, 1–46 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  25. Daubechies, I., Teschke, G., Vese, L.: Iteratively solving linear inverse problems under general convex constraints. Inverse Probl. Theor. Imaging 1(1), 29 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  26. Fadili, M., Starck, J.: Sparse representations and Bayesian image inpainting. In: Proc. SPARS, vol. 5 (2005)

    Google Scholar 

  27. Fadili, M., Starck, J., Murtagh, F.: Inpainting and zooming using sparse representations. Comput. J. 52(1), 64 (2009)

    Article  Google Scholar 

  28. Figueiredo, M., Nowak, R.: An EM algorithm for wavelet-based image restoration. IEEE Trans. Image Process. 12(8), 906–916 (2003)

    Article  MathSciNet  Google Scholar 

  29. Figueiredo, M., Nowak, R.: A bound optimization approach to wavelet-based image deconvolution. In: IEEE International Conference on Image Processing. ICIP 2005, vol. 2, p. II-782. IEEE, New York (2005)

    Google Scholar 

  30. Cai, J., Chan, R., Shen, L., Shen, Z.: Convergence analysis of tight framelet approach for missing data recovery. Adv. Comput. Math. 1–27 (2008)

  31. Cai, J., Chan, R., Shen, Z.: Simultaneous cartoon and texture inpainting. In: Inverse Problems and Imaging (IPI), vol. 4, pp. 379–395 (2010)

    Google Scholar 

  32. Candes, E., Tao, T.: Near-optimal signal recovery from random projections: universal encoding strategies? IEEE Trans. Inf. Theory 52(12), 5406–5425 (2006)

    Article  MathSciNet  Google Scholar 

  33. Candes, E., Tao, T.: Decoding by linear programming. IEEE Trans. Inf. Theory 51(12), 4203–4215 (2005)

    Article  MathSciNet  Google Scholar 

  34. Lu, Z., Zhang, Y.: Penalty decomposition methods for l 0-norm minimization. Technical report, Department of Mathematics, Simon Fraser University, Canada (2010)

  35. Chartrand, R.: Exact reconstructions of sparse signals via nonconvex minimization. IEEE Signal Process. Lett. 14, 707–710 (2007)

    Article  Google Scholar 

  36. Chartrand, R., Yin, W.: Iteratively reweighted algorithms for compressive sensing. In: 33rd International Conference on Acoustics, Speech, and Signal Processing (ICASSP) (2008)

    Google Scholar 

  37. Chartrand, R.: Fast algorithms for nonconvex compressive sensing: MRI reconstruction from very few data. In: IEEE International Symposium on Biomedical Imaging (ISBI) (2009)

    Google Scholar 

  38. Hiriart-Urruty, J., Lemaréchal, C.: Convex Analysis and Minimization Algorithms: Fundamentals. Springer, Berlin (1993)

    Book  MATH  Google Scholar 

  39. Goldstein, T., Osher, S.: The split Bregman algorithm for L1 regularized problems. SIAM J. Imaging Sci. 2(2), 323–343 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  40. Zhang, X., Burger, M., Bresson, X., Osher, S.: Bregmanized nonlocal regularization for deconvolution and sparse reconstruction. SIAM J. Imaging Sci. 3, 253 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  41. Gabay, D., Mercier, B.: A dual algorithm for the solution of nonlinear variational problems via finite element approximation. Comput. Math. Appl. 2(1), 17–40 (1976)

    Article  MATH  Google Scholar 

  42. Bertsekas, D., Tsitsiklis, J.: Parallel and Distributed Computation: Numerical Methods. Prentice-Hall, New York (1989)

    MATH  Google Scholar 

  43. Eckstein, J., Bertsekas, D.: On the Douglas “Rachford splitting method and the proximal point algorithm for maximal monotone operators”. Math. Program. 55(1), 293–318 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  44. Hestenes, M.: Multiplier and gradient methods. J. Optim. Theory Appl. 4(5), 303–320 (1969)

    Article  MathSciNet  MATH  Google Scholar 

  45. Powell, M.: A method for non-linear constraints in minimization problems. In: Fletcher, R. (ed.) Optimization, pp. 283–298. Academic Press, New York (1969)

    Google Scholar 

  46. Glowinski, R., Le Tallec, P.: Augmented Lagrangian and Operator-Splitting Methods in Nonlinear Mechanics. SIAM, Philadelphia (1989)

    Book  MATH  Google Scholar 

  47. Esser, E.: Applications of Lagrangian-based alternating direction methods and connections to split Bregman. CAM Rep. 9, 31 (2009)

    Google Scholar 

  48. Tai, X., Wu, C.: Augmented Lagrangian method, dual methods and split Bregman iteration for ROF model. In: Scale Space and Variational Methods in Computer Vision, pp. 502–513 (2009)

    Chapter  Google Scholar 

  49. Donoho, D.: De-noising by soft-thresholding. IEEE Trans. Inf. Theory 41, 613–627 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  50. Mrázek, P., Weickert, J.: Rotationally invariant wavelet shrinkage. Pattern Recognit 156–163 (2003)

  51. Iusem, A.: Augmented Lagrangian methods and proximal point methods for convex optimization. Investig. Oper. 8, 11–49 (1999)

    Google Scholar 

  52. Eckstein, J.: Nonlinear proximal point algorithms using Bregman functions, with applications to convex programming. Math. Oper. Res. 202–226 (1993)

  53. Zhu, M., Chan, T.: An efficient primal-dual hybrid gradient algorithm for total variation image restoration. Mathematics Department, UCLA, CAM Report, pp. 08-34 (2007)

  54. Ma, S., Yin, W., Zhang, Y., Chakraborty, A.: An efficient algorithm for compressed mr imaging using total variation and wavelets. In: IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2008, pp. 1–8. IEEE, New York (2008)

    Google Scholar 

  55. Wang, Y., Yang, J., Yin, W., Zhang, Y.: A new alternating minimization algorithm for total variation image reconstruction. SIAM J. Imaging Sci. 1(3), 248–272 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  56. Yang, J., Zhang, Y., Yin, W.: A fast tvl 1-l 2 minimization algorithm for signal reconstruction from partial Fourier data. IEEE J. Sel. Top. Signal Process. 4, 288–297 (2009)

    Article  Google Scholar 

  57. Wang, Y., Yin, W.: Sparse signal reconstruction via iterative support detection. SIAM J. Imaging Sci. 3(3), 462–491 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  58. Chen, Y., Hager, W., Huang, F., Phan, D., Ye, X., Yin, W.: Fast algorithms for image reconstruction with application to partially parallel MR imaging. SIAM J. Imaging Sci. 5(1), 90–118 (2010)

    Article  MathSciNet  Google Scholar 

  59. Cai, X., Gu, G., He, B., Yuan, X.: A relaxed customized proximal point algorithm for separable convex programming. Preprint (2011)

  60. Antoniadis, A., Fan, J.: Regularization of wavelet approximations. J. Am. Stat. Assoc. 96(455), 939–967 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  61. Donoho, D., Johnstone, I.: Threshold selection for wavelet shrinkage of noisy data. In: Engineering in Medicine and Biology Society, 1994. Engineering Advances: New Opportunities for Biomedical Engineers. Proceedings of the 16th Annual International Conference of the IEEE, pp. A24–A25. IEEE, New York (1994)

    Google Scholar 

  62. Larsson, T., Patriksson, M., Stromberg, A.: Ergodic convergence in subgradient optimization. Optim. Methods Softw. 9(1), 93–120 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  63. Larsson, T., Patriksson, M., Strömberg, A.: Ergodic, primal convergence in dual subgradient schemes for convex programming. Math. Program. 86(2), 283–312 (1999)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Mathematics, The University of Arizona, 617 N. Santa Rita Ave., Tucson, AZ, 85721-0089, USA

    Bin Dong

  2. Department of Mathematics, Simon Fraser University, Burnaby, BC, V5A 1S6, Canada

    Yong Zhang

Authors
  1. Bin Dong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yong Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bin Dong.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Dong, B., Zhang, Y. An Efficient Algorithm for ℓ 0 Minimization in Wavelet Frame Based Image Restoration. J Sci Comput 54, 350–368 (2013). https://doi.org/10.1007/s10915-012-9597-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10915-012-9597-4

Keywords

Search

Navigation