Numerische Mathematik

, Volume 112, Issue 4, pp 509–533 | Cite as

Simultaneously inpainting in image and transformed domains

  • Jian-Feng Cai
  • Raymond H. Chan
  • Lixin Shen
  • Zuowei Shen


In this paper, we focus on the restoration of images that have incomplete data in either the image domain or the transformed domain or in both. The transform used can be any orthonormal or tight frame transforms such as orthonormal wavelets, tight framelets, the discrete Fourier transform, the Gabor transform, the discrete cosine transform, and the discrete local cosine transform. We propose an iterative algorithm that can restore the incomplete data in both domains simultaneously. We prove the convergence of the algorithm and derive the optimal properties of its limit. The algorithm generalizes, unifies, and simplifies the inpainting algorithm in image domains given in Cai et al. (Appl Comput Harmon Anal 24:131–149, 2008) and the inpainting algorithms in the transformed domains given in Cai et al. (SIAM J Sci Comput 30(3):1205–1227, 2008), Chan et al. (SIAM J Sci Comput 24:1408–1432, 2003; Appl Comput Harmon Anal 17:91–115, 2004). Finally, applications of the new algorithm to super-resolution image reconstruction with different zooms are presented.

Mathematics Subject Classification (2000)

94A08 65T60 90C90 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bertalmio, M., Sapiro, G., Caselles, V., Ballester, C.: Image inpainting. In: Proceedings of SIGGRAPH, New Orleans, LA, pp. 417–424 (2000)Google Scholar
  2. 2.
    Bertalmio M., Vese L., Sapiro G., Osher S.: Simultaneous structure and texture image inpainting. IEEE Trans. Image Proc. 12, 882–889 (2003)CrossRefGoogle Scholar
  3. 3.
    Bertero M., Boccacci P.: Introduction to Inverse Problems in Imaging. Institute of Physics Pub., Bristol (1998)MATHCrossRefGoogle Scholar
  4. 4.
    Bertero M., Boccacci P., Benedetto F.D., Robberto M.: Restoration of chopped and nodded images in infrared astronomy. Inverse Probl. 15, 345–372 (1999)MATHCrossRefGoogle Scholar
  5. 5.
    Borup L., Gribonval R., Nielsen M.: Bi-framelet systems with few vanishing moments characterize Besov spaces. Appl. Comput. Harmon. Anal. 17, 3–28 (2004)MATHCrossRefMathSciNetGoogle Scholar
  6. 6.
    Bose N., Boo K.: High-resolution image reconstruction with multisensors. Int. J. Imaging Syst. Technol. 9, 294–304 (1998)CrossRefGoogle Scholar
  7. 7.
    Cai J.-F., Chan R., Shen L., Shen Z.: Restoration of chopped and nodded images by framelets. SIAM J. Sci. Comput. 30(3), 1205–1227 (2008)CrossRefMathSciNetGoogle Scholar
  8. 8.
    Cai J.-F., Chan R., Shen Z.: A framelet-based image inpainting algorithm. Appl. Comput. Harmon. Anal. 24, 131–149 (2008)MATHCrossRefMathSciNetGoogle Scholar
  9. 9.
    Cai, J.-F., Chan, R., Shen, Z.: Simultaneous Cartoon and Texture Inpainting (2008, preprint)Google Scholar
  10. 10.
    Cai, J.-F., Osher, S., Shen, Z.: Linearized Bregman iterations for compressed sensing. Mathematics of Computations (to appear)Google Scholar
  11. 11.
    Cai, J.-F., Osher, S., Shen, Z.: Convergence of the linearized Bregman iteration for ℓ1-norm minimization. Mathematics of Computations (to appear)Google Scholar
  12. 12.
    Cai J.-F., Osher S., Shen Z.: Linearized Bregman iterations for frame-based image deblurring. SIAM J. Imaging Sci. 2(1), 226–252 (2009)CrossRefGoogle Scholar
  13. 13.
    Candès E.J., Donoho D.L.: New tight frames of curvelets and optimal representations of objects with piecewise C 2 singularities. Comm. Pure Appl. Math. 57, 219–266 (2004)MATHCrossRefMathSciNetGoogle Scholar
  14. 14.
    Candès E. J., Romberg J., Tao T.: Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information. IEEE Trans. Inform. Theory 52, 489–509 (2006)CrossRefMathSciNetGoogle Scholar
  15. 15.
    Chai A., Shen Z.: Deconvolution: a wavelet frame approach. Numer. Math. 106, 529–587 (2007)MATHCrossRefMathSciNetGoogle Scholar
  16. 16.
    Chan R., Chan T., Shen L., Shen Z.: Wavelet algorithms for high-resolution image reconstruction. SIAM J. Sci. Comput. 24, 1408–1432 (2003)MATHCrossRefMathSciNetGoogle Scholar
  17. 17.
    Chan R., Chan T., Shen L., Shen Z.: Wavelet deblurring algorithms for spatially varying blur from high-resolution image reconstruction. Linear Algebra Appl. 366, 139–155 (2003)MATHCrossRefMathSciNetGoogle Scholar
  18. 18.
    Chan R., Riemenschneider S.D., Shen L., Shen Z.: High-resolution image reconstruction with displacement errors: a framelet approach. Int. J. Imaging Syst. Technol. 14, 91–104 (2004)CrossRefMathSciNetGoogle Scholar
  19. 19.
    Chan R., Riemenschneider S.D., Shen L., Shen Z.: Tight frame: the efficient way for high-resolution image reconstruction. Appl. Comput. Harmon. Anal. 17, 91–115 (2004)MATHCrossRefMathSciNetGoogle Scholar
  20. 20.
    Chan, R., Shen, L., Shen, Z.: A framelet-based approach for image inpainting. Tech. Report 2005-4. The Chinese University of Hong Kong, Feb. (2005)Google Scholar
  21. 21.
    Chan R., Shen Z., Xia T.: A framelet algorithm for enchancing video stills. Appl. Comput. Harmon. Anal. 23, 153–170 (2007)MATHCrossRefMathSciNetGoogle Scholar
  22. 22.
    Chan T., Kang S.-H., Shen J.: Euler’s elastica and curvature-based image inpainting. SIAM J. Appl. Math. 63, 564–592 (2002)MATHMathSciNetGoogle Scholar
  23. 23.
    Chan T., Shen J.: Mathematical models for local nontexture inpaintings. SIAM J. Appl. Math. 62, 1019–1043 (2001)MathSciNetGoogle Scholar
  24. 24.
    Chan T.F., Shen J., Zhou H.-M.: Total variation wavelet inpainting. J. Math. Imaging Vision 25, 107–125 (2006)CrossRefMathSciNetGoogle Scholar
  25. 25.
    Combettes P., Wajs V.: Signal recovery by proximal forward-backward splitting. Multiscale Model. Simul. A SIAM Interdiscip. J. 4, 1168–1200 (2005)MATHCrossRefMathSciNetGoogle Scholar
  26. 26.
    Daubechies I.: Ten Lectures on Wavelets. CBMS Conference Series in Applied Mathematics, vol. 61. SIAM, Philadelphia (1992)Google Scholar
  27. 27.
    Daubechies I., Han B., Ron A., Shen Z.: Framelets: MRA-based constructions of wavelet frames. Appl. Comput. Harmon. Anal. 14, 1–46 (2003)MATHCrossRefMathSciNetGoogle Scholar
  28. 28.
    Daubechies I., Teschke G., Vese L.: Iteratively solving linear inverse problems under general convex constraints. Inverse Probl. Imaging 1, 29–46 (2007)MATHMathSciNetGoogle Scholar
  29. 29.
    Delaney A.H., Bresler Y.: A fast and accurate Fourier algorithm for iterative parallel-beam tomography. IEEE Trans. Image Process. 5, 740–753 (1996)CrossRefGoogle Scholar
  30. 30.
    Do M.N., Vetterli M.: The contourlet transform: an efficient directional multiresolution image representation. IEEE Trans. Image Process. 14, 2091–2106 (2005)CrossRefMathSciNetGoogle Scholar
  31. 31.
    Donoho D., Johnstone I.: Ideal spatial adaptation by wavelet shrinkage. Biometrika 81, 425–455 (1994)MATHCrossRefMathSciNetGoogle Scholar
  32. 32.
    Elad M., Feuer A.: Restoration of a single superresolution image from several blurred, noisy and undersampled measured images. IEEE Trans. Image Process. 6, 1646–1658 (1997)CrossRefGoogle Scholar
  33. 33.
    Elad M., Milanfar P., Rubinstein R.: Analysis versus synthesis in signal priors. Inverse Probl. 23, 947–968 (2007)MATHCrossRefMathSciNetGoogle Scholar
  34. 34.
    Elad M., Starck J.-L., Querre P., Donoho D.: Simultaneous cartoon and texture image inpainting using morphological component analysis (MCA). Appl. Comput. Harmon. Anal. 19, 340–358 (2005)MATHCrossRefMathSciNetGoogle Scholar
  35. 35.
    Fadili, M., Starck, J.-L.: Sparse representations and bayesian image inpainting. In: Proc. SPARS’05, vol. I. Rennes, France (2005)Google Scholar
  36. 36.
    Guleryuz O.G.: Nonlinear approximation based image recovery using adaptive sparse reconstruction and iterated denoising: Part II adaptive algorithms. IEEE Trans. Image Process. 15(3), 555–571 (2006)CrossRefGoogle Scholar
  37. 37.
    Hiriart-Urruty, J.-B., Lemarechal, C.: Convex analysis and minimization algorithms. Grundlehren der Mathematischen Wissenschaften [Fundamental Principles of Mathematical Sciences], vol. 305. Springer, Berlin (1993)Google Scholar
  38. 38.
    Joshi M.V., Chaudhuri S., Panuganti R.: Super-resolution imaging: use of zoom as a cue. Image Vis. Comput. 22, 1185–1196 (2004)Google Scholar
  39. 39.
    Mallat S.: A Wavelet Tour of Signal Processing. Academic Press, London (1999)MATHGoogle Scholar
  40. 40.
    Ng M.K., Bose N.: Analysis of displacement errors in high-resolution image reconstruction with multisensors. IEEE Trans. Circuits Syst. I Fundam. Theory Appl. 49, 806–813 (2002)CrossRefGoogle Scholar
  41. 41.
    Ng M.K., Chan R., Tang W.: A fast algorithm for deblurring models with Neumann boundary conditions. SIAM J. Sci. Comput. 21, 851–866 (2000)CrossRefMathSciNetGoogle Scholar
  42. 42.
    Ron A., Shen Z.: Affine system in \({L_2(\mathbb{R}^d)}\) : the analysis of the analysis operator. J. Func. Anal. 148, 408–447 (1997)MATHCrossRefMathSciNetGoogle Scholar

Copyright information

© Springer-Verlag 2009

Authors and Affiliations

  • Jian-Feng Cai
    • 1
  • Raymond H. Chan
    • 2
  • Lixin Shen
    • 3
  • Zuowei Shen
    • 4
  1. 1.Temasek Laboratories, Department of MathematicsNational University of SingaporeSingaporeSingapore
  2. 2.Department of MathematicsThe Chinese University of Hong KongHong KongPeople’s Republic of China
  3. 3.Department of MathematicsSyracuse UniversitySyracuseUSA
  4. 4.Department of MathematicsNational University of SingaporeSingaporeSingapore

Personalised recommendations