Evaluation of Digital Inpainting Quality in the Context of Artwork Restoration

  • Alexandra Ioana Oncu
  • Ferdinand Deger
  • Jon Yngve Hardeberg
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7583)


Improved digital image inpainting algorithms could provide substantial support for future artwork restoration. However, currently, there is an acknowledged lack of quantitative metrics for image inpainting evaluation. In this paper the performance of eight inpainting algorithms is first evaluated by means of a psychophysical experiment. The ranking of the algorithms thus obtained confirms that exemplar based methods generally outperform PDE based methods. Two novel inpainting quality metrics, proposed in this paper, eight general image quality metrics and four inpainting-specific metrics are then evaluated by validation against the perceptual data. Results show that no metric can adequately predict inpainting quality over the entire image database, and that the performance of the metrics is image-dependent.


Human Visual System Mean Opinion Score Psychophysical Experiment Image Inpainting Perceptual Data 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Bertalmio, M.: Processing of Flat and Non-Flat Image Information on Arbitrary Manifolds Using Partial Differential Equations. PhD thesis, University of Minnesota (2001)Google Scholar
  2. 2.
    Telea, A.: An image inpainting technique based on the fast marching method. J. Graphics Tools 9(1), 23–24 (2004)CrossRefGoogle Scholar
  3. 3.
    Tschumperle, D., Deriche, R.: Vector-valued image regularization with PDE’s: A common framework for different applications. IEEE Transactions on Pattern Analysis and Machine Intelligence 27(4), 506–517 (2005)CrossRefGoogle Scholar
  4. 4.
    Bornemann, F., März, T.: Fast image inpainting based on coherence transport. J. Math. Imaging Vision 28(3), 259–278 (2007)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Criminisi, A., Pérez, P., Toyama, K.: Region filling and object removal by exemplar-based image inpainting. IEEE Transactions on Image Processing 13(9), 1200–1212 (2004)CrossRefGoogle Scholar
  6. 6.
    Zhou, J., Kelly, A.R.: Image inpainting based on local optimization. In: International Conference on Pattern Recognition, ICPR, pp. 4440–4443 (2010)Google Scholar
  7. 7.
    Barnes, C., Shechtman, E., Finkelstein, A., Goldman, D.B.: Patchmatch: a randomized correspondence algorithm for structural image editing. ACM Transactions on Graphics (TOG) 28(3) (2009)Google Scholar
  8. 8.
    Oliveira, M.M., Bowen, B., Mckenna, R., Chang, Y.S.: Fast digital image inpainting. In: Proceedings of the International Conference on Visualization, Imaging and Image Processing (VIIP 2001), pp. 261–266. ACTA Press (September 2001)Google Scholar
  9. 9.
    Mahalingam, V.V.: Digital inpainting algorithms and evaluation. PhD thesis, University of Kentucky (2010)Google Scholar
  10. 10.
    Ardis, P., Singhal, A.: Visual salience metrics for image inpainting. In: Proceedings of SPIE/IS&T Electronic Imaging, San Jose, CA, USA, vol. 7257 (January 2009)Google Scholar
  11. 11.
    ITU-R: Rec. ITU-R BT.500-13. Methodology for the subjective assessment of the quality of television pictures (2012)Google Scholar
  12. 12.
    Walther, D.: Interactions of visual attention and object recognition: computational modeling, algorithms, and psychophysics. PhD thesis, California Institute of Technology, Pasadena, CA (2006)Google Scholar
  13. 13.
    Bonnier, N., Schmitt, F., Brettel, H., Berche, S.: Evaluation of spatial gamut mapping algorithms. In: Color Imaging Conference, pp. 56–61 (November 2006)Google Scholar
  14. 14.
    Zhang, X., Wandell, B.A.: A spatial extension of CIELAB for digital color image reproduction. Soc. Inform. Display 96 Digest, 731–734 (1996)Google Scholar
  15. 15.
    Chandler, D.M., Hemami, S.S.: VSNR: A wavelet-based visual signal to noise ratio for natural imges. IEEE Transactions on Image Processing 16(9), 2284–2298 (2007)MathSciNetCrossRefGoogle Scholar
  16. 16.
    Pedersen, M., Hardeberg, J.Y.: A New Spatial Hue Angle Metric for Perceptual Image Difference. In: Trémeau, A., Schettini, R., Tominaga, S. (eds.) CCIW 2009. LNCS, vol. 5646, pp. 81–90. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  17. 17.
    Hong, G., Luo, M.R.: New algorithm for calculating perceived colour difference of images. Imaging Science Journal 54(2), 86–91 (2006)CrossRefGoogle Scholar
  18. 18.
    Johnson, G.M., Fairchild, M.D.: Darwinism of color image difference models. In: Proceedings of IS& T/SID 9th Color Imaging Conference, pp. 108–112 (November 2001)Google Scholar
  19. 19.
    Wang, Z., Hardeberg, J.Y.: An adaptive bilateral filter for predicting color image difference. In: Color Imaging Conference, pp. 27–31 (November 2009)Google Scholar
  20. 20.
    Kendall, M.G., Stuart, A., Ord, J.K.: Kendall’s Advanced Theory of Statistics: Classical inference and relationship, 5th edn., vol. 2. Hodder Arnold (1991)Google Scholar
  21. 21.
    Hardeberg, J.Y., Bando, E., Pedersen, M.: Evaluating colour image difference metrics for gamut-mapped images. Coloration Technology 124(4), 243–253 (2008)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Alexandra Ioana Oncu
    • 1
  • Ferdinand Deger
    • 1
  • Jon Yngve Hardeberg
    • 1
  1. 1.The Norwegian Colour and Visual Computing LaboratoryGjøvik University CollegeGjøvikNorway

Personalised recommendations