Skip to main content
SpringerLink
Log in

Impainting by Restoring Image Gradients

  • MATHEMATICAL MODELS AND COMPUTATIONAL METHODS
  • Published:
Journal of Communications Technology and Electronics Aims and scope Submit manuscript

Abstract

Impainting is used both to improve the visual perception of an image and in classical problems of recognition and robotics to remove irrelevant information from an image. Modern impainting methods of reconstruction drawing use neural networks. However, these approaches are plagued with disadvantages that prevent these algorithms from being used in practical applications of computer vision. The authors have proposed in their recent paper an impainting method based on a non-local mean (NLM) filter that outperforms other conventional image processing methods in terms of peak signal-to-noise ration (PSNR). However, this algorithm may create artifacts at the boundaries of the rendering zones that distort visual perception. In this article, we propose to draw not the image, but the gradients of the image restored and only then restore the image itself, maintaining in this way connectivity at the border of the rendering areas. This method eliminates most of the artifacts that arise when using the previously proposed method, and also surpasses it in terms of the recovery accuracy assessed using the PSNR criterion.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.
Fig. 5.

Similar content being viewed by others

REFERENCES

  1. A. N. Hirani and T. Totsuka, “Combining Frequency and Spatial Domain Information for Fast Interactive Image Noise Removal,” in Proc. 23rd Annual Conf. Computer Graphics & Interactive Techniques, New York, USA, 1996, (ACM, 1996), pp. 269–276.

  2. A. A. Efros and T. K. Leung, “Texture Synthesis by Non-Parametric Sampling,” in Proc.6th IEEE Int. Conf. on Computer Vision (ICCV’99), Corfu, Greece, Sept. 1999 (IEEE, New York, 1999), Vol. 2, pp. 1033–1038.

  3. D. J. Heeger and J. R. Bergen, “Pyramid-Based Texture Analysis/Synthesis,” in Proc. 22nd Annual Conf. on Computer Graphics and Interactive Techniques, (SIGGRAPH’95), Sept. 1995, pp. 229–238.

  4. D. Karras and G. Mertzios, “Discretization schemes and numerical approximations of pde impainting models and a comparative evaluation on novel real world mri reconstruction applications,” in Proc. IEEE Int. Workshop on Imaging Systems and Techniques (IST) (IEEE Cat. No. 04EX896), Stresa, Italy, May 14, 2004 (IEEE, Piscataway, 2004), pp. 153–158.

  5. T. Ružić and A. Pižurica, “Context-aware patch-based image inpainting using Markov random field modeling,” IEEE Trans. Image Process. 24 (1), 444–456 (2014).

  6. U. Demir and G. Unal, “Patch-based image inpainting with generative adversarial networks,” Preprint. arXiv: 1803.07422 (2018).

  7. R. A. Yeh, C. Chen, Lim T. Yian, et al., “Semantic image inpainting with deep generative models,” in Proc. IEEE Conf. on Computer Vision & Pattern Recognition, Honolulu, Hawaii, USA 2017 (IEEE, New York, 2017), pp. 5485–5493.

  8. C. Tomasi and R. Manduchi, “Bilateral Fltering for Gray and Color Images,” in Proc. Conf. on Computer Vision & Pattern Recognition (CVPR’98), Santa Barbara, CA, USA, June 23–25, 1998 (CVPR, 1998), pp. 839– 846.

  9. M. Mozerov and J. van de Weijer, “Accurate stereo natching by two-step energy minimization,” IEEE Trans. Image Process. 24, 1153–1163 (2015).

  10. M. G. Mozerov and J. van de Weijer, “One-view occlusion detection for stereo matching with a fully connected crf model,” IEEE Trans. Image Process. 28, 2936–2947 (2019).

  11. E. Ershov, V. Karnaukhov, and M. Mozerov, “Probabilistic choice between symmetric disparities in motion stereo matching for a lateral navigation system,” Opt. Eng. 55, 023101–023101 (2016).

  12. M. Mozerov and J. van de Weijer, “Improved recursive geodesic distance computation for edge preserving flter,” IEEE Trans. Image Process. 26 (8), 3696–3706 (2017).

  13. M. Mozerov, “Constrained optical ow estimation as a matching problem,” IEEE Trans. Image Process. 22 (5), 2044–2055 (2013).

  14. R. Fattal, M. Agrawala, and S. Rusinkiewicz, “Multiscale shape and detail enhancement from multi-light image collections,” ACM Trans. on Graph. (TOG/ACM) 26, 51 (2007).

  15. Z. Farbman, R. Fattal, D. Lischinski, and R. Szeliski, “Edge-preserving decompositions for multi-scale tone and detail manipulation,” ACM Trans. on Graph. (TOG/ACM) 27, 67 (2008).

  16. V. Karnaukhov and M. Mozerov, “Restoration of multispectral images by the gradient reconstruction method and estimation of the blur parameters on the basis of the multipurpose matching model,” J. Commun. Technol. Electron. 61 (12), 1426–1431 (2016).

  17. R. Ramanath and W. E. Snyder, “Adaptive demosaicking,” J. Electron. Imag. 12, 633–642 (2003).

  18. H. Winnemöller, S. Olsen, and B. Gooch, “Real-time video abstraction,” ACM Trans. on Graphics 25 (3), 1221–1226 (2006).

    Article  Google Scholar 

  19. S. Paris and F. Durand, “A fast approximation of the bilateral flter using a signal processing approach,” in Proc. Eur. Conf. on Computer Vision (ECCV), Graz, Austria, May 7–13, 2006 (Springer, 2006), pp. 568–580.

  20. A. Adams, J. Baek, and M. Davis, “Fast high-dimensional fltering using the permutohedral lattice,” Computer Graphics Forum 29 (2), 753–762 (2010).

  21. E. Gastal and M. Oliveira, “Domain transform for edge-aware image and video processing,” ACM Trans. on Graph. 30 (4), 69 (2011).

    Article  Google Scholar 

  22. M. G. Mozerov and J. van de Weijer, “Global color sparseness and a local statistics prior for fast bilateral ltering,” IEEE Trans. Image Process. 24 (12), 5842– 5853 (2015).

  23. V. Karnaukhov, V. Kober, M. Mozerov, and L. Zimina, “Impainting with a nonlocal means flter,” J. Commun. Technol. Electron. 67, 709–714 (2022).

  24. A. Buades, B. Coll, and J.-M. Morel, “A non-local algorithm for image denoising,” in IEEE Comput. Soc. Conf. on Computer Vision and Pattern Recognition (CVPR’05), San Diego, CA, USA, 2005 (IEEE, New York, 2017), Vol. 2, pp. 60–65.

  25. K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian, “Image denoising by sparse 3-d transform-domain collaborative filtering,” IEEE Trans. Image Process. 16, 2080–2095 (2007).

  26. W. Zuo, L. Zhang, C. Song, and D. Zhang, “Texture enhanced image denoising via gradient histogram preservation,” in Proc. IEEE Conf. Comput Vision & Pattern Recogn. (CVPR), 2013 (IEEE, New York, 2013), pp. 1203–1210.

  27. G. Liu, H. Zhong, and L. Jiao, “Comparing Noisy Patches for Image Denoising: A Double Noise Similarity Model,” IEEE Transactions on Image Processing 24, 862–872 (2015).

  28. V. Karnaukhov and M. Mozerov, “Fast non-local mean filter algorithm based on recursive calculation of similarity weights,” J. Commun. Technol. Electron. 63, 1475–1477 (2018).

Download references

Author information

Authors and Affiliations

  1. Kharkevich Institute for Information Transmission Problems, Russian Academy of Sciences, 127051, Moscow, Russia

    V. N. Karnaukhov, V. I. Kober & M. G. Mozerov

  2. Moscow Polytechnic University, 107023, Moscow, Russia

    L. V. Zimina

  3. Center of Scientific Research and Higher Education, B.C 22860, Ensenada, Mexico

    V. I. Kober

Authors
  1. V. N. Karnaukhov
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. V. I. Kober
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. M. G. Mozerov
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. L. V. Zimina
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to V. N. Karnaukhov, V. I. Kober, M. G. Mozerov or L. V. Zimina.

Ethics declarations

The authors declare that they have no conflicts of interest.

Additional information

Translated by M. Shmatikov

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Karnaukhov, V.N., Kober, V.I., Mozerov, M.G. et al. Impainting by Restoring Image Gradients. J. Commun. Technol. Electron. 67, 1557–1563 (2022). https://doi.org/10.1134/S1064226922120063

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1134/S1064226922120063

Search

Navigation