Skip to main content
SpringerLink
Log in

Simultaneously inpainting in image and transformed domains

  • Published:
Numerische Mathematik Aims and scope Submit manuscript

Abstract

In this paper, we focus on the restoration of images that have incomplete data in either the image domain or the transformed domain or in both. The transform used can be any orthonormal or tight frame transforms such as orthonormal wavelets, tight framelets, the discrete Fourier transform, the Gabor transform, the discrete cosine transform, and the discrete local cosine transform. We propose an iterative algorithm that can restore the incomplete data in both domains simultaneously. We prove the convergence of the algorithm and derive the optimal properties of its limit. The algorithm generalizes, unifies, and simplifies the inpainting algorithm in image domains given in Cai et al. (Appl Comput Harmon Anal 24:131–149, 2008) and the inpainting algorithms in the transformed domains given in Cai et al. (SIAM J Sci Comput 30(3):1205–1227, 2008), Chan et al. (SIAM J Sci Comput 24:1408–1432, 2003; Appl Comput Harmon Anal 17:91–115, 2004). Finally, applications of the new algorithm to super-resolution image reconstruction with different zooms are presented.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Bertalmio, M., Sapiro, G., Caselles, V., Ballester, C.: Image inpainting. In: Proceedings of SIGGRAPH, New Orleans, LA, pp. 417–424 (2000)

  2. Bertalmio M., Vese L., Sapiro G., Osher S.: Simultaneous structure and texture image inpainting. IEEE Trans. Image Proc. 12, 882–889 (2003)

    Article  Google Scholar 

  3. Bertero M., Boccacci P.: Introduction to Inverse Problems in Imaging. Institute of Physics Pub., Bristol (1998)

    Book  MATH  Google Scholar 

  4. Bertero M., Boccacci P., Benedetto F.D., Robberto M.: Restoration of chopped and nodded images in infrared astronomy. Inverse Probl. 15, 345–372 (1999)

    Article  MATH  Google Scholar 

  5. Borup L., Gribonval R., Nielsen M.: Bi-framelet systems with few vanishing moments characterize Besov spaces. Appl. Comput. Harmon. Anal. 17, 3–28 (2004)

    Article  MATH  MathSciNet  Google Scholar 

  6. Bose N., Boo K.: High-resolution image reconstruction with multisensors. Int. J. Imaging Syst. Technol. 9, 294–304 (1998)

    Article  Google Scholar 

  7. Cai J.-F., Chan R., Shen L., Shen Z.: Restoration of chopped and nodded images by framelets. SIAM J. Sci. Comput. 30(3), 1205–1227 (2008)

    Article  MathSciNet  Google Scholar 

  8. Cai J.-F., Chan R., Shen Z.: A framelet-based image inpainting algorithm. Appl. Comput. Harmon. Anal. 24, 131–149 (2008)

    Article  MATH  MathSciNet  Google Scholar 

  9. Cai, J.-F., Chan, R., Shen, Z.: Simultaneous Cartoon and Texture Inpainting (2008, preprint)

  10. Cai, J.-F., Osher, S., Shen, Z.: Linearized Bregman iterations for compressed sensing. Mathematics of Computations (to appear)

  11. Cai, J.-F., Osher, S., Shen, Z.: Convergence of the linearized Bregman iteration for ℓ1-norm minimization. Mathematics of Computations (to appear)

  12. Cai J.-F., Osher S., Shen Z.: Linearized Bregman iterations for frame-based image deblurring. SIAM J. Imaging Sci. 2(1), 226–252 (2009)

    Article  Google Scholar 

  13. Candès E.J., Donoho D.L.: New tight frames of curvelets and optimal representations of objects with piecewise C 2 singularities. Comm. Pure Appl. Math. 57, 219–266 (2004)

    Article  MATH  MathSciNet  Google Scholar 

  14. Candès E. J., Romberg J., Tao T.: Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information. IEEE Trans. Inform. Theory 52, 489–509 (2006)

    Article  MathSciNet  Google Scholar 

  15. Chai A., Shen Z.: Deconvolution: a wavelet frame approach. Numer. Math. 106, 529–587 (2007)

    Article  MATH  MathSciNet  Google Scholar 

  16. Chan R., Chan T., Shen L., Shen Z.: Wavelet algorithms for high-resolution image reconstruction. SIAM J. Sci. Comput. 24, 1408–1432 (2003)

    Article  MATH  MathSciNet  Google Scholar 

  17. Chan R., Chan T., Shen L., Shen Z.: Wavelet deblurring algorithms for spatially varying blur from high-resolution image reconstruction. Linear Algebra Appl. 366, 139–155 (2003)

    Article  MATH  MathSciNet  Google Scholar 

  18. Chan R., Riemenschneider S.D., Shen L., Shen Z.: High-resolution image reconstruction with displacement errors: a framelet approach. Int. J. Imaging Syst. Technol. 14, 91–104 (2004)

    Article  MathSciNet  Google Scholar 

  19. Chan R., Riemenschneider S.D., Shen L., Shen Z.: Tight frame: the efficient way for high-resolution image reconstruction. Appl. Comput. Harmon. Anal. 17, 91–115 (2004)

    Article  MATH  MathSciNet  Google Scholar 

  20. Chan, R., Shen, L., Shen, Z.: A framelet-based approach for image inpainting. Tech. Report 2005-4. The Chinese University of Hong Kong, Feb. (2005)

  21. Chan R., Shen Z., Xia T.: A framelet algorithm for enchancing video stills. Appl. Comput. Harmon. Anal. 23, 153–170 (2007)

    Article  MATH  MathSciNet  Google Scholar 

  22. Chan T., Kang S.-H., Shen J.: Euler’s elastica and curvature-based image inpainting. SIAM J. Appl. Math. 63, 564–592 (2002)

    MATH  MathSciNet  Google Scholar 

  23. Chan T., Shen J.: Mathematical models for local nontexture inpaintings. SIAM J. Appl. Math. 62, 1019–1043 (2001)

    MathSciNet  Google Scholar 

  24. Chan T.F., Shen J., Zhou H.-M.: Total variation wavelet inpainting. J. Math. Imaging Vision 25, 107–125 (2006)

    Article  MathSciNet  Google Scholar 

  25. Combettes P., Wajs V.: Signal recovery by proximal forward-backward splitting. Multiscale Model. Simul. A SIAM Interdiscip. J. 4, 1168–1200 (2005)

    Article  MATH  MathSciNet  Google Scholar 

  26. Daubechies I.: Ten Lectures on Wavelets. CBMS Conference Series in Applied Mathematics, vol. 61. SIAM, Philadelphia (1992)

    Google Scholar 

  27. Daubechies I., Han B., Ron A., Shen Z.: Framelets: MRA-based constructions of wavelet frames. Appl. Comput. Harmon. Anal. 14, 1–46 (2003)

    Article  MATH  MathSciNet  Google Scholar 

  28. Daubechies I., Teschke G., Vese L.: Iteratively solving linear inverse problems under general convex constraints. Inverse Probl. Imaging 1, 29–46 (2007)

    MATH  MathSciNet  Google Scholar 

  29. Delaney A.H., Bresler Y.: A fast and accurate Fourier algorithm for iterative parallel-beam tomography. IEEE Trans. Image Process. 5, 740–753 (1996)

    Article  Google Scholar 

  30. Do M.N., Vetterli M.: The contourlet transform: an efficient directional multiresolution image representation. IEEE Trans. Image Process. 14, 2091–2106 (2005)

    Article  MathSciNet  Google Scholar 

  31. Donoho D., Johnstone I.: Ideal spatial adaptation by wavelet shrinkage. Biometrika 81, 425–455 (1994)

    Article  MATH  MathSciNet  Google Scholar 

  32. Elad M., Feuer A.: Restoration of a single superresolution image from several blurred, noisy and undersampled measured images. IEEE Trans. Image Process. 6, 1646–1658 (1997)

    Article  Google Scholar 

  33. Elad M., Milanfar P., Rubinstein R.: Analysis versus synthesis in signal priors. Inverse Probl. 23, 947–968 (2007)

    Article  MATH  MathSciNet  Google Scholar 

  34. Elad M., Starck J.-L., Querre P., Donoho D.: Simultaneous cartoon and texture image inpainting using morphological component analysis (MCA). Appl. Comput. Harmon. Anal. 19, 340–358 (2005)

    Article  MATH  MathSciNet  Google Scholar 

  35. Fadili, M., Starck, J.-L.: Sparse representations and bayesian image inpainting. In: Proc. SPARS’05, vol. I. Rennes, France (2005)

  36. Guleryuz O.G.: Nonlinear approximation based image recovery using adaptive sparse reconstruction and iterated denoising: Part II adaptive algorithms. IEEE Trans. Image Process. 15(3), 555–571 (2006)

    Article  Google Scholar 

  37. Hiriart-Urruty, J.-B., Lemarechal, C.: Convex analysis and minimization algorithms. Grundlehren der Mathematischen Wissenschaften [Fundamental Principles of Mathematical Sciences], vol. 305. Springer, Berlin (1993)

  38. Joshi M.V., Chaudhuri S., Panuganti R.: Super-resolution imaging: use of zoom as a cue. Image Vis. Comput. 22, 1185–1196 (2004)

    Google Scholar 

  39. Mallat S.: A Wavelet Tour of Signal Processing. Academic Press, London (1999)

    MATH  Google Scholar 

  40. Ng M.K., Bose N.: Analysis of displacement errors in high-resolution image reconstruction with multisensors. IEEE Trans. Circuits Syst. I Fundam. Theory Appl. 49, 806–813 (2002)

    Article  Google Scholar 

  41. Ng M.K., Chan R., Tang W.: A fast algorithm for deblurring models with Neumann boundary conditions. SIAM J. Sci. Comput. 21, 851–866 (2000)

    Article  MathSciNet  Google Scholar 

  42. Ron A., Shen Z.: Affine system in \({L_2(\mathbb{R}^d)}\) : the analysis of the analysis operator. J. Func. Anal. 148, 408–447 (1997)

    Article  MATH  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Temasek Laboratories, Department of Mathematics, National University of Singapore, 2 Science Drive 2, Singapore, 117543, Singapore

    Jian-Feng Cai

  2. Department of Mathematics, The Chinese University of Hong Kong, Shatin, N.T., Hong Kong, People’s Republic of China

    Raymond H. Chan

  3. Department of Mathematics, Syracuse University, Syracuse, NY, 13244, USA

    Lixin Shen

  4. Department of Mathematics, National University of Singapore, 2 Science Drive 2, Singapore, 117543, Singapore

    Zuowei Shen

Authors
  1. Jian-Feng Cai
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Raymond H. Chan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lixin Shen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zuowei Shen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Raymond H. Chan.

Additional information

R. H. Chan’s research was supported in part by HKRGC Grant 400505 and CUHK DAG 2060257.

L. Shen’s research was supported by the US National Science Foundation under grant DMS-0712827.

Z. Shen’s research was supported in part by Grant R-146-000-060-112 at the National University of Singapore.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Cai, JF., Chan, R.H., Shen, L. et al. Simultaneously inpainting in image and transformed domains. Numer. Math. 112, 509–533 (2009). https://doi.org/10.1007/s00211-009-0222-x

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00211-009-0222-x

Mathematics Subject Classification (2000)

Search

Navigation