Scale Consistent Image Completion

  • Michal Holtzman-Gazit
  • Irad Yavneh
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4291)


Most patch based image completion algorithms fill in missing parts of images by copying patches from the known part of the image into the unknown part. The criterion for preferring one patch over another is the compatibility or consistency of the proposed patch with the nearby region that is known or already completed. In this paper we propose adding another dimension to this consistency criterion, namely, scale. Thus, the preferred patch is chosen by evaluating its consistency with respect to smoothed (less detailed) versions of the image, as well as its surroundings in the current version. Applied recursively, this approach results in a multi-scale framework that is shown to yield a dramatic improvement in the robustness of a good existing image completion algorithm.


Patch Size Texture Synthesis Image Inpainting Image Completion Source Patch 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Ballester, C., Bertalmio, M., Caselles, V., Sapiro, G., Verdera, J.: Texture mixing and texture movie synthesis using statistical learning. IEEE Trans. Image Processing 10, 1200–1211 (2001)MATHCrossRefMathSciNetGoogle Scholar
  2. 2.
    Bertalmio, M., Sapiro, G., Caselles, V., Ballester, C.: Image inpainting. In: SIGGRAPH 2000, Computer Graphics Proceedings, ACM SIGGRAPH, pp. 417–424 (2000)Google Scholar
  3. 3.
    Chan, T., Shen, J.: Non-texture inpaintings by curvature-driven diffusions. J. Visual Communication and Image Representation 12(4), 436–449 (2001)CrossRefGoogle Scholar
  4. 4.
    Telea, A.: An image inpainting technique based on the fast marching method. Journal of Graphics Tools: JGT 9, 23–34 (2004)Google Scholar
  5. 5.
    Oliveira, M.M., Bowen, B., McKenna, R., Chang, Y.S.: Fast digital image inpainting. In: Proceeding of International Conference on VIIP, pp. 261–266 (2001)Google Scholar
  6. 6.
    Elad, M., Starck, J.L., Querre, P., Donoho, D.: Simultaneous cartoon and texture image inpainting using morphological component analysis (MCA). Journal on Applied and Computational Harmonic Analysis 19, 340–358 (2005)MATHCrossRefMathSciNetGoogle Scholar
  7. 7.
    Rares, A., Reinders, M., Biemond, J.: Edge-based image restoration. IEEE Trans. Image Processing 14, 1454–1468 (2005)CrossRefGoogle Scholar
  8. 8.
    Shih, T., Lu, L.C., Wang, Y.H., Chang, R.C.: Multi-resolution image inpainting. In: Proceedings of the 2003 International Conference on Multimedia and Expo, vol. 1, pp. 485–8 (2003)Google Scholar
  9. 9.
    Bonet, J.S.D.: Multiresolution sampling procedure for analysis and synthesis of texture images. In: Computer Graphics Proceedings, ACM SIGGRAPH, pp. 361–368 (1997)Google Scholar
  10. 10.
    Efros, A., Leung, T.: Texture synthesis by non-parametric sampling. In: ICCV proceedings, Corfu, Greece, pp. 1033–1038 (1999)Google Scholar
  11. 11.
    Harrison, P.: A non-hierarchical procedure for re-synthesis of complex textures. In: WSCG, pp. 190–197 (2001)Google Scholar
  12. 12.
    Wei, L., Levoy, M.: Fast texture synthesis using tree-structured vector quantization. In: SIGGRAPH 2000, Computer Graphics Proceedings, ACM SIGGRAPH, pp. 479–488 (2000)Google Scholar
  13. 13.
    Hertzmann, A., Jacobs, C.E., Oliver, N., Curless, B., Salesin, D.H.: Image analogies. In: SIGGRAPH 2001, Computer Graphics Proceedings, ACM SIGGRAPH, pp. 327–340 (2001)Google Scholar
  14. 14.
    Ashikhmin, M.: Synthesizing natural textures. In: Proceedings of the 2001 symposium on Interactive 3D graphics, pp. 217–226. ACM Press, New York (2001)CrossRefGoogle Scholar
  15. 15.
    Efros, A., Freeman, W.: Image quilting for texture synthesis and transfer. In: SIGGRAPH 2001, Computer Graphics Proceedings, ACM SIGGRAPH, pp. 341–346 (2001)Google Scholar
  16. 16.
    Kwatra, V., Schödl, A., Essa, I., Turk, G., Bobick, A.: Graphcut textures: Image and video synthesis using graph cuts. ACM Trans. on Graph., SIGGRAPH 2003 22, 277–286 (2003)Google Scholar
  17. 17.
    Zhang, Y., Xiao, J., Shah, M.: Region completion in a single image. In: EUROGRAPHICS (2004)Google Scholar
  18. 18.
    Jia, J., Tang, C.: Inference of segmented color and texture description by tensor voting. IEEE Trans. Pattern Anal. Mach. Intell. 26, 771–786 (2004)CrossRefGoogle Scholar
  19. 19.
    Bertalmio, M., Vese, L., Sapiro, G., Osher, S.: Simultaneous structure and texture image inpainting. In: Proc. of Computer Vision and Pattern Recognition, vol. 2, pp. 707–712 (2003)Google Scholar
  20. 20.
    Drori, I., Cohen-Or, D., Yeshurun, H.: Fragment-based image completion. ACM Trans. Graph. 22, 303–312 (2003)CrossRefGoogle Scholar
  21. 21.
    Cant, R.J., Langensiepen, C.S.: A multiscale method for automated inpainting. In: Proceeding of ESM 2003, pp. 148–153 (2003)Google Scholar
  22. 22.
    Wexler, Y., Shechtman, E., Irani, M.: Space-time video completion. In: Proc. of Computer Vision and Pattern Recognition, vol. 1, pp. 120–127 (2004)Google Scholar
  23. 23.
    Sun, J., Yuan, L., Jia, J., Shum, H.Y.: Image completion with structure propagation. ACM Trans. Graph. 24, 861–868 (2005)CrossRefGoogle Scholar
  24. 24.
    Criminisi, A., Perez, P., Toyama, K.: Region filling and object removal by exemplar-based inpainting. IEEE Trans. on Image Processing 13, 1200–1212 (2004)CrossRefGoogle Scholar
  25. 25.
    Galun, M., Sharon, E., Basri, R., Brandt, A.: Texture segmentation by multiscale aggregation of filter responses and shape elements. In: ICCV, pp. 716–723 (2003)Google Scholar
  26. 26.
    Perona, P., Malik, J.: Scale-space and edge detection using anisotropic diffusion. IEEE Trans. Pattern Anal. Mach. Intell. 12, 629–639 (1990)CrossRefGoogle Scholar
  27. 27.
    Spira, A., Kimmel, R., Sochen, N.: Efficient beltrami flow using a short time kernel. In: Griffin, L.D., Lillholm, M. (eds.) Scale-Space 2003. LNCS, vol. 2695, pp. 511–522. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  28. 28.
    Schaefer, G., Stich, M.: UCID - an uncompressed colour image database. In: Proc. SPIE, Storage and Retrieval Methods and Applications for Multimedia 2004, pp. 472–480 (2004)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Michal Holtzman-Gazit
    • 1
  • Irad Yavneh
    • 1
  1. 1.Computer Science DepartmentTechnion–I.I.T.HaifaIsrael

Personalised recommendations