Image Inpainting Considering Brightness Change and Spatial Locality of Textures and Its Evaluation

  • Norihiko Kawai
  • Tomokazu Sato
  • Naokazu Yokoya
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5414)


Image inpainting techniques have been widely investigated to remove undesired objects in an image. Conventionally, missing parts in an image are completed by optimizing the objective function using pattern similarity. However, unnatural textures are easily generated due to two factors: (1) available samples in the image are quite limited, and (2) pattern similarity is one of the required conditions but is not sufficient for reproducing natural textures. In this paper, we propose a new energy function based on the pattern similarity considering brightness changes of sample textures (for (1)) and introducing spatial locality as an additional constraint (for (2)). The effectiveness of the proposed method is successfully demonstrated by qualitative and quantitative evaluation. Furthermore, the evaluation methods used in much inpainting research are discussed.


Image Inpainting Energy Minimization Evaluation Method 


  1. 1.
    Levin, A., Zomet, A., Weiss, Y.: Learning How to Inpaint from Global Image Statistics. In: Proc. ICCV, vol. 1, pp. 305–312 (2003)Google Scholar
  2. 2.
    Ballester, C., Bertalmio, M., Sapiro, V., Verdera, J.: Filling-In by Joint Interpolation of Vector Fields and Gray Levels. Trans. on Image Processing 10(8), 1200–1211 (2001)MathSciNetCrossRefzbMATHGoogle Scholar
  3. 3.
    Ballester, C., Caselles, V., Verdera, J., Bertalmio, M., Sapiro, G.: A Variational Model for Filling-In Gray Level and Color Images. In: Proc. ICCV, pp. 10–16 (2001)Google Scholar
  4. 4.
    Tschumperlé, D.: Curvature-preserving regularization of multi-valued images using pDE’s. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3952, pp. 295–307. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  5. 5.
    Villéger, E., Aubert, G., Blanc-Féraud, L.: Image Disocclusion Using a Probabilistic Gradient Orientation. In: Proc. ICPR, vol. 2, pp. 52–55 (2004)Google Scholar
  6. 6.
    Bertalmio, M., Bertozzi, A.L., Sapiro, G.: Navier-Stokes, Fluid Dynamics, and Image and Video Inpainting. In: Proc. CVPR, pp. 355–362 (2001)Google Scholar
  7. 7.
    Bertalmio, M., Sapiro, G., Caselles, V., Ballester, C.: Image Inpainting. In: Proc. SIGGRAPH 2000, pp. 417–424 (2000)Google Scholar
  8. 8.
    Esedoglu, S., Shen, J.: Digital Inpainting Based on the Mumford-shah-euler Image Model. European J. of Applied Mathematics 13, 353–370 (2003)MathSciNetzbMATHGoogle Scholar
  9. 9.
    Masnou, S., Morel, J.M.: Level Lines Based Disocclusion. In: Proc. ICIP, vol.  3, pp. 259–263 (1998)Google Scholar
  10. 10.
    Chan, T., Shen, J.: Non-texture Inpainting by Curvature-Driven Diffusions (CDD). J. of Visual Communication and Image Representation 12(4), 436–449 (2001)CrossRefGoogle Scholar
  11. 11.
    Chan, T., Kang, S., Shen, J., Osher, S.: Euler’s Elastica and Curvature Based Inpaintings. SIAM J. of Applied Mathematics 63(2), 564–592 (2002)MathSciNetzbMATHGoogle Scholar
  12. 12.
    Hirani, A.N., Totsuka, T.: Combining Frequency and Spatial Domain Information for Fast Interactive Image Noise Removal. In: Proc. SIGGRAPH 1996, pp. 269–276 (1996)Google Scholar
  13. 13.
    Rane, S.D., Remus, J., Sapiro, G.: Wavelet-Domain Reconstruction of Lost Blocks in Wireless Image Transmission and Packet-Switched. In: Proc. ICIP, vol. 1, pp. 309–312 (2002)Google Scholar
  14. 14.
    Amano, T.: Image Interpolation by High Dimensional Projection Based on Subspace Method. In: Proc. ICPR, vol. 4, pp. 665–668 (2004)Google Scholar
  15. 15.
    Efros, A.A., Leung, T.K.: Texture Synthesis by Non-parametric Sampling. In: Proc. ICCV, pp. 1033–1038 (1999)Google Scholar
  16. 16.
    Bornard, R., Lecan, E., Laborelli, L., Chenot, J.: Missing Data Correction in Still Images and Image Sequences. In: Proc. ACM Int. Conf. on Multimedia, pp. 355–361 (2002)Google Scholar
  17. 17.
    Criminisi, A., Pérez, P., Toyama, K.: Region Filling and Object Removal by Exemplar-Based Image Inpainting. Trans. on Image Processing 13(9), 1200–1212 (2004)CrossRefGoogle Scholar
  18. 18.
    Li, B., Qi, Y., Shen, X.: An Image Inpainting Method. In: Proc. IEEE Int. Conf. on Computer Aided Design and Computer Graphics, pp. 531–536 (2005)Google Scholar
  19. 19.
    Allène, C., Paragios, N.: Image Renaissance Using Discrete Optimization. In: Proc. ICPR, pp. 631–634 (2006)Google Scholar
  20. 20.
    Wexler, Y., Shechtman, E., Irani, M.: Space-Time Completion of Video. Trans. on PAMI 29(3), 463–476 (2007)CrossRefGoogle Scholar
  21. 21.
    Komodakis, N., Tziritas, G.: Image Completion Using Global Optimization. In: Proc. CVPR, pp. 442–452 (2006)Google Scholar
  22. 22.
    Drori, I., Cohen-Or, D., Yeshurun, H.: Fragment-Based Image Completion. In: Proc. SIGGRAPH 2003, pp. 303–312 (2003)Google Scholar
  23. 23.
    Sun, J., Yuan, L., Jia, J., Shum, H.: Image Completion with Structure Propagation. In: Proc. SIGGRAPH 2005, pp. 861–868 (2005)Google Scholar
  24. 24.
    Jia, J., Tang, C.: Image Repairing: Robust Image Synthesis by Adaptive ND Tensor Voting. In: Proc. CVPR, pp. 643–650 (2003)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Norihiko Kawai
    • 1
  • Tomokazu Sato
    • 1
  • Naokazu Yokoya
    • 1
  1. 1.Graduate School of Information ScienceNara Institute of Science and TechnologyNaraJapan

Personalised recommendations