A Combined PDE and Texture Synthesis Approach to Inpainting

  • Harald Grossauer
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3022)


While there is a vast amount of literature considering PDE based inpainting and inpainting by texture synthesis, only a few publications are concerned with combination of both approaches. We present a novel algorithm which combines both approaches and treats each distinct region of the image separately. Thus we are naturally lead to include a segmentation pass as a new feature. This way the correct choice of texture samples for the texture synthesis is ensured. We propose a novel concept of “local texture synthesis” which gives satisfactory results even for large domains in a complex environment.


Landau Equation Texture Synthesis Geometry Part Image Decomposition Geometry Image 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Masnou, S.: Disocclusion: A Variational Approach Using Level Lines. IEEE Transactions on Signal Processing 11(2), 68–76 (2002)MathSciNetGoogle Scholar
  2. 2.
    Masnou, S., Morel, J.-M.: Level Lines based Disocclusion. In: Proceedings of the 1998 IEEE International Conference on Image Processing, pp. 259–263 (1998)Google Scholar
  3. 3.
    Ballester, C., Bertalmio, M., Caselles, V., Sapiro, G., Verdera, J.: Filling–In by Joint Interpolation of Vector Fields and Gray Levels. IEEE Transactions on Signal Processing 10(8), 1200–1211 (2001)zbMATHMathSciNetGoogle Scholar
  4. 4.
    Bertalmio, M., Sapiro, G., Ballester, C., Caselles, V.: Image inpainting, Computer Graphics. In: SIGGRAPH 2000 (July 2000)Google Scholar
  5. 5.
    Bertalmio, M., Bertozzi, A., Sapiro, G.: Navier–Stokes, Fluid Dynamics, and Image and Video Inpainting. In: IEEE CVPR 2001, Hawaii, USA (December 2001)Google Scholar
  6. 6.
    Chan, T., Shen, J.: Mathematical Models for Local Nontexture Inpaintings. SIAM Journal of Applied Mathematics 62(3), 1019–1043 (2002)zbMATHCrossRefMathSciNetGoogle Scholar
  7. 7.
    Chan, T., Shen, J.: Non–Texture Inpainting by Curvature–Driven Diffusions (CDD). Journal of Visual Communication and Image Representation 12(4), 436–449 (2001)CrossRefGoogle Scholar
  8. 8.
    Chan, T., Kang, S., Shen, J.: Euler’s Elastica and Curvature Based Inpainting. SIAM Journal of Applied Mathematics 63(2), 564–592 (2002)zbMATHMathSciNetGoogle Scholar
  9. 9.
    Esedoglu, S., Shen, J.: Digital Inpainting Based on the Mumford–Shah–Euler Image Model. European Journal of Applied Mathematics 13, 353–370 (2002)zbMATHCrossRefMathSciNetGoogle Scholar
  10. 10.
    Oliveira, M., Bowen, B., McKenna, R., Chang, Y.: Fast Digital Inpainting. In: Proceedings of the International Conference on Visualization, Imaging and Image Processing (VIIP 2001), Marbella, Spain, pp. 261–266 (2001)Google Scholar
  11. 11.
    Grossauer, H., Scherzer, O.: Using the Complex Ginzburg–Landau Equation for Digital Inpainting in 2D and 3D. In: Griffin, L.D., Lillholm, M. (eds.) Scale-Space 2003. LNCS, vol. 2695, Springer, Heidelberg (2003)CrossRefGoogle Scholar
  12. 12.
    Igehy, H., Pereira, L.: Image replacement through texture synthesis. In: Proceedings of the 1997 IEEE International Conf. on Image Processing (1997)Google Scholar
  13. 13.
    Wei, L.-Y., Levoy, M.: Fast Texture Synthesis using Tree-structured Vector Quantization. In: Proceedings of SIGGRAPH 2000 (2000)Google Scholar
  14. 14.
    Wei, L.-Y., Levoy, M.: Order-Independent Texture Synthesis, Technical Report TR- 2002-01, Computer Science Department, Stanford University (April 2002)Google Scholar
  15. 15.
    Harrison, P.: A non–hiearachical procedure for re–synthesis of complex textures. In: WSCG 2001, Plzen, Czech Republic, pp. 190–197. University of West Bohemia (2001), 2001Google Scholar
  16. 16.
    Ashikhmin, M.: Synthesizing Natural Textures. In: Proceedings of 2001 ACM Symposium on Interactive 3D Graphics, March 19-21, pp. 217–226. Research Triangle Park, North Carolina (2001)Google Scholar
  17. 17.
    Paget, R., Longstaff, D.: Texture Synthesis via a Nonparametric Markov Random Field. In: Proceedings of DICTA-95, Digital Image Computing: Techniques and Applications, December 6-8, vol. 1, pp. 547–552 (1995)Google Scholar
  18. 18.
    Efros, A., Leung, T.: Texture Synthesis by Non-parametric Sampling. In: IEEE International Conference on Computer Vision (ICCV 1999), Corfu, Greece (September 1999)Google Scholar
  19. 19.
    Hirani, A., Totsuka, T.: Combining frequency and spatial domain information for fast interactive image noise removal. In: ACM SIGGRAPH, pp. 269–276 (1996)Google Scholar
  20. 20.
    Rane, S., Bertalmio, M., Sapiro, G.: Structure and Texture Filling-in of missing image blocks for Wireless Transmission and Compression. IEEE Transactions on Image Processing 12(3) (March 2003)Google Scholar
  21. 21.
    Bertalmio, M., Vese, L., Sapiro, G., Osher, S.: Simultaneous texture and structure image inpainting. IEEE Transactions on Image Processing 12(8) (August 2003)Google Scholar
  22. 22.
    Yamauchi, H., Haber, J., Seidel, H.-P.: Image Restoration using Multiresolution Texture Synthesis and Image Inpainting. In: Proc. Computer Graphics International (CGI) 2003, Tokyo, Japan, July 9-11, pp. 120–125 (2003)Google Scholar
  23. 23.
    Perona, P., Malik, J.: Scale–Space and Edge Detection Using Anisotropic Diffusion. IEEE Trans. on Pattern Analysis and Machine Intelligence 12(7) (July 1990)Google Scholar
  24. 24.
    Vese, L., Osher, S.: Modeling Textures with Total Variation Minimization and Oscillating Patterns in Image Processing, UCLA CAM Report 02-19 (May 2002); to appear in Journal of Scientific ComputingGoogle Scholar
  25. 25.
    Osher, S., Sole, A., Vese, L.: Image decomposition and restoration using total variation minimization and the H− 1 norm, UCLA Computational and Applied Mathematics Reports (October 2002)Google Scholar
  26. 26.
    Aujol, J.-F., et al.: Image Decomposition Application to SAR Images. In: Griffin, L.D., Lillholm, M. (eds.) Scale-Space 2003. LNCS, vol. 2695, Springer, Heidelberg (2003)CrossRefGoogle Scholar
  27. 27.
    Meyer, Y.: Oscillating Patterns in Image Processing and Nonlinear Evolution Equations. AMS University Lecture Series, vol. 22 (2002)Google Scholar
  28. 28.
    Bethuel, F., Brezis, H., Helein, F.: Ginzburg–Landau Vortices. In: Progress in Nonlinear Partial Differential Equations, vol. 13, Birkhäuser, Basel (1994)Google Scholar
  29. 29.
    Ambrosio, L., Dancer, N.: Calculus of Variations and Partial Differential Equations. Springer, Heidelberg (2000)Google Scholar
  30. 30.
    Weickert, J.: Anisotropic Diffusion in Image Processing. B.G.Teubner, Stuttgart (1998)zbMATHGoogle Scholar
  31. 31.
    Acton, S.: Multigrid Anisotropic Diffusion. IEEE Transactions on Image Processing 7(3) (March 1998)Google Scholar
  32. 32.
    Nene, S.A., Nayar, S.K.: A Simple Algorithm for Nearest Neighbor Search in High Dimensions. IEEE Trans. Pattern Anal. Machine Intell. 19, 989–1003 (1997)CrossRefGoogle Scholar
  33. 33.

Copyright information

© Springer-Verlag Berlin Heidelberg 2004

Authors and Affiliations

  • Harald Grossauer
    • 1
  1. 1.Department of Computer ScienceUniversity of InnsbruckInnsbruckAUSTRIA

Personalised recommendations