Skip to main content
SpringerLink
Log in

A Variational Framework for Non-local Image Inpainting

  • Conference paper
Energy Minimization Methods in Computer Vision and Pattern Recognition (EMMCVPR 2009)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 5681))

Abstract

Non-local methods for image denoising and inpainting have gained considerable attention in recent years. This is in part due to their superior performance in textured images, a known weakness of purely local methods. Local methods on the other hand have demonstrated to be very appropriate for the recovering of geometric structure such as image edges. The synthesis of both types of methods is a trend in current research. Variational analysis in particular is an appropriate tool for a unified treatment of local and non-local methods. In this work we propose a general variational framework for the problem of non-local image inpainting, from which several previous inpainting schemes can be derived, in addition to leading to novel ones. We explicitly study some of these, relating them to previous work and showing results on synthetic and real images.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Aujol, J.-F., Ladjal, S., Masnou, S.: Exemplar-based inpainting from a variational point of view (submitted) (2008)

    Google Scholar 

  2. Awate, S.P., Whitaker, R.T.: Unsupervised, information-theoretic, adaptive image filtering for image restoration. IEEE Trans. on PAMI 28(3), 364–376 (2006)

    Article  Google Scholar 

  3. Ballester, C., Bertalmío, M., Caselles, V., Sapiro, G., Verdera, J.: Filling-in by joint interpolation of vector fields and gray levels. IEEE Trans. on IP 10(8), 1200–1211 (2001)

    MathSciNet  MATH  Google Scholar 

  4. Bertalmío, M., Sapiro, G., Caselles, V., Ballester, C.: Image inpainting. In: Proc. of SIGGRAPH (2000)

    Google Scholar 

  5. Bertalmío, M., Vese, L., Sapiro, G., Osher, S.: Simultaneous structure and texture inpainting. IEEE Trans. on Image Processing 12(8), 882–889 (2003)

    Article  Google Scholar 

  6. Bornard, R., Lecan, E., Laborelli, L., Chenot, J.-H.: Missing data correction in still images and image sequences. In: Proc. ACM Int. Conf. on Multimedia (2002)

    Google Scholar 

  7. Bornemann, F., März, T.: Fast image inpainting based on coherence transport. J. of Math. Imag. and Vis. 28(3), 259–278 (2007)

    Article  MathSciNet  Google Scholar 

  8. Brox, T., Kleinschmidt, O., Cremers, D.: Efficient nonlocal means for denoising of textural patterns. IEEE Trans. on IP 17(7), 1057–1092 (2008)

    MathSciNet  Google Scholar 

  9. Buades, A., Coll, B., Morel, J.M.: A non local algorithm for image denoising. In: Proc. of the IEEE Conf. on CVPR, vol. 2, pp. 60–65 (2005)

    Google Scholar 

  10. Cao, F., Gousseau, Y., Masnou, S., Pérez, P.: Geometrically-guided exemplar-based inpainting (submitted) (2008)

    Google Scholar 

  11. Chan, T., Kang, S.H., Shen, J.H.: Euler’s elastica and curvature based inpaintings. SIAM J. App. Math. 63(2), 564–592 (2002)

    MathSciNet  MATH  Google Scholar 

  12. Chan, T., Shen, J.H.: Mathematical models for local nontexture inpaintings. SIAM J. App. Math. 62(3), 1019–1043 (2001)

    MathSciNet  MATH  Google Scholar 

  13. Cheng, Y.: Mean shift, mode seeking and clustering. IEEE Trans. on PAMI 17(8), 790–799 (1995)

    Article  Google Scholar 

  14. Criminisi, A., Pérez, P., Toyama, K.: Region filling and object removal by exemplar-based inpainting. IEEE Trans. on IP 13(9), 1200–1212 (2004)

    Google Scholar 

  15. Csiszár, I.: Axiomatic characterizations of information measures. Entropy 10(3), 261–273 (2008)

    Article  MATH  Google Scholar 

  16. Demanet, L., Song, B., Chan, T.: Image inpainting by correspondence maps: a deterministic approach. Technical report, UCLA (2003)

    Google Scholar 

  17. Drori, I., Cohen-Or, D., Yeshurun, H.: Fragment-based image completion. ACM Trans. on Graphics. Special issue: Proc. of ACM SIGGRAPH 22(3), 303–312 (2003)

    Article  Google Scholar 

  18. Efros, A.A., Leung, T.K.: Texture synthesis by non-parametric sampling. In: Proc. of the IEEE Conf. on CVPR, September 1999, pp. 1033–1038 (1999)

    Google Scholar 

  19. Esedoglu, S., Shen, J.H.: Digital image inpainting by the Mumford-Shah-Euler image model. European J. App. Math. 13, 353–370 (2002)

    MATH  Google Scholar 

  20. Gilboa, G., Osher, S.J.: Nonlocal linear image regularization and supervised segmentation. SIAM Mult. Mod. and Sim. 6(2), 595–630 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  21. Igehy, H., Pereira, L.: Image replacement through texture synthesis. In: Proc. of the IEEE Conf. on CVPR (October 1997)

    Google Scholar 

  22. Jaynes, E.T.: Information theory and statistical mechanics. Physical Review 106(4), 620–630 (1957)

    Article  MathSciNet  MATH  Google Scholar 

  23. Jia, J., Tang, C.-K.: Inference of segmented color and texture description by tensor voting. IEEE Trans. on PAMI 26(6), 771–786 (2004)

    Article  Google Scholar 

  24. Komodakis, N., Tziritas, G.: Image completion using efficient belief propagation via priority scheduling and dynamic pruning. IEEE Trans. on IP 16(11), 2649–2661 (2007)

    MathSciNet  Google Scholar 

  25. Levin, A., Zomet, A., Weiss, Y.: Learning how to inpaint from global image statistics. In: Proc. of ICCV (2003)

    Google Scholar 

  26. Levina, E., Bickel, P.: Texture synthesis and non-parametric resampling of random fields. Annals of Statistics 34(4) (2006)

    Google Scholar 

  27. Mahmoudi, M., Sapiro, G.: Fast image and video denoising via nonlocal means of similar neighborhoods. Signal Processing Letters 12(12), 839–842 (2005)

    Article  Google Scholar 

  28. Masnou, S.: Disocclusion: a variational approach using level lines. IEEE Trans. on IP 11(2), 68–76 (2002)

    MathSciNet  Google Scholar 

  29. Masnou, S., Morel, J.-M.: Level lines based disocclusion. In: Proc. of IEEE ICIP (1998)

    Google Scholar 

  30. Pérez, P., Gangnet, M., Blake, A.: Poisson image editing. In: Proc. of SIGGRAPH (2003)

    Google Scholar 

  31. Pérez, P., Gangnet, M., Blake, A.: PatchWorks: Example-based region tiling for image editing. Technical report, Microsoft Research (2004)

    Google Scholar 

  32. Peyré, G.: Manifold models for signals and images. Comp. Vis. and Im. Unders. 113(2), 249–260 (2009)

    Article  Google Scholar 

  33. Peyré, G., Bougleux, S., Cohen, L.: Non-local regularization of inverse problems. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part III. LNCS, vol. 5304, pp. 57–68. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  34. Protter, M., Elad, M., Takeda, H., Milanfar, P.: Generalizing the non-local-means to super-resolution reconstruction. IEEE Trans. on IP 18(1), 36–51 (2009)

    Google Scholar 

  35. Singer, A., Shkolnisky, Y., Nadler, B.: Diffusion interpretation of non-local neighborhood filters for signal denoising. SIAM J. on Imag. Sci. 2(1), 118–139 (2009)

    Article  MATH  Google Scholar 

  36. Tschumperlé, D., Deriche, R.: Vector-valued image regularization with PDE’s: a common framework for different applications. IEEE Trans. on PAMI 27(4) (2005)

    Google Scholar 

  37. Wei, L.-Y., Levoy, M.: Fast texture synthesis using tree-structured vector quantization. In: Proc. of the SIGGRAPH (2000)

    Google Scholar 

  38. Wexler, Y., Shechtman, E., Irani, M.: Space-time completion of video. IEEE Trans. on PAMI 29(3), 463–476 (2007)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Universitat Pompeu Fabra, DTIC, 08018, Barcelona, Spain

    Pablo Arias & Vicent Caselles

  2. University of Minnesota, ECE, Minneapolis, MN 55455, USA

    Guillermo Sapiro

Authors
  1. Pablo Arias
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Vicent Caselles
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Guillermo Sapiro
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Institut für Informatik III, Rö, Universität Bonn, merstraße 164, 53117, Bonn, Germany

    Daniel Cremers

  2. Computer Science Department, University of Western Ontario, N6A 5A5, London, ON, Canada

    Yuri Boykov

  3. Microsoft Research, J.J. Thomson Avenue, CB3 OFB, Cambridge, United Kingdom

    Andrew Blake

  4. Institut für Informatik III, Universität Bonn, Römerstraße 164, 53117, Bonn, Germany

    Frank R. Schmidt

Rights and permissions

Reprints and permissions

Copyright information

© 2009 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Arias, P., Caselles, V., Sapiro, G. (2009). A Variational Framework for Non-local Image Inpainting. In: Cremers, D., Boykov, Y., Blake, A., Schmidt, F.R. (eds) Energy Minimization Methods in Computer Vision and Pattern Recognition. EMMCVPR 2009. Lecture Notes in Computer Science, vol 5681. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-03641-5_26

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-03641-5_26

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-03640-8

  • Online ISBN: 978-3-642-03641-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation