Abstract
Due to various factors, some parts of images can be lost. Recovering the damaged regions of images is essential. In this paper, a single image inpainting method using Wasserstein Generative Adversarial Networks (WGAN) and self-attention is proposed. The global consistency of the inpainting region is established and the Wasserstein distance is used to measure the similarity of the two distributions. Finally, self-attention is embedded to exploit the self-similarity of local features. The experiments confirm that the proposed method can recover the global correlation of corrupted images better than similar methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Arjovsky, M., Chintala, S., Bottou, L.: Wasserstein generative adversarial networks. In: Proceedings of 2017 International Conference on Machine Learning, Sydney (2017)
Ballester, C., Bertalmio, M., et al.: Filling-in by joint interpolation of vector fields and gray levels. IEEE Trans. Image Process. 10(8), 1200–1211 (2001)
Barnes, C., Shechtman, E., et al.: PatchMatch: a randomized correspondence algorithm for structural image editing. ACM Trans. Graph. 28(3), 1–11 (2009)
Bertalmio, M., Sapiro, G., et al.: Image inpainting. In: Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques, New Orleans (2000)
Carion, N., Massa, F., Synnaeve, G., et al.: End-to-end object detection with transformers. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.M. (eds.) Computer Vision – ECCV 2020, vol. 12346, pp. 213–229. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58452-8_13
Criminisi, A., Patrick, P., Kentaro, T.: Region filling and object removal by exemplar-based image inpainting. IEEE Trans. Image Process. 13(9), 1200–1212 (2004)
Dosovitskiy, A., Beyer, L., Kolesnikov, A., et al.: An image is worth 16 × 16 words: transformers for image recognition at scale (2020). arXiv:2010.11929
Efros, A.A., Freeman, W.T.: Image quilting for texture synthesis and transfer. In: Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques, Los Angeles (2001)
Efros, A.A., Leung, T.K.: Texture synthesis by non-parametric sampling. In: Proceedings of the Seventh IEEE International Conference on Computer Vision, Corfu (1999)
Goodfellow, I., et al.: Generative adversarial nets. In: Advances in Neural Information Processing Systems, vol. 27 (2014)
Gulrajani, I., et al.: Improved training of Wasserstein GANS. In: Advances in Neural Information Processing Systems, vol. 30 (2017)
Guo, M.H., Xu, T.X., et al.: Attention mechanisms in computer vision: a survey (2021). arXiv:2111.07624
He, K., Zhang, X., Ren, S., et al.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas (2016)
Hou, Q., Zhou, D., Feng, J.: Coordinate attention for efficient mobile network design (2021). arXiv:2103.02907
Hu, J., Shen, L., Albanie, S., et al.: Squeeze-and-excitation networks (2017). arXiv:1709.01507
Iizuka, S., Simo-Serra, E., Ishikawa, H.: Globally and locally consistent image completion. ACM Trans. Graph. (ToG) 36(4), 1–14 (2017)
Jaderberg, V., Simonyan, K., Zisserman, A., et al.: Spatial transformer networks (2015). arXiv:1506.02025
Kechaou, A., Martinez, M., Haurilet, M., et al.: Detective: an attentive recurrent model for sparse object detection. In: Proceedings of 25th International Conference on Pattern Recognition (ICPR), Milan (2021)
Li, J., et al.: Recurrent feature reasoning for image inpainting. In: Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Washington (2020)
Liu, G., Reda, F.A., Shih, K.J., Wang, T.-C., Tao, A., Catanzaro, B.: Image inpainting for irregular holes using partial convolutions. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11215, pp. 89–105. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01252-6_6
Mao, J., Niu, M., Bai, H., et al.: Pyramid R-CNN: towards better performance and adaptability for 3D object detection (2021). arXiv:2109.02499
Mnih, V., Heess, N., Graves, A., et al.: Recurrent models of visual attention (2014). arXiv:1406.6247
Nazeri, K., Ng, E., Joseph, T., et al.: EdgeConnect: generative image inpainting with adversarial edge learning (2019). arXiv:1901.00212
Pathak, D., Krahenbuhl, P., et al.: Context encoders: feature learning by inpainting. In: Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas (2016)
Radford, A., Metz, L., Chintala, S.: Unsupervised representation learning with deep convolutional generative adversarial networks (2015). arXiv:1511.06434
Vaswani, A., Shazeer, N., Parmar, N., et al.: Attention is all you need (2017). arXiv:1706.03762
Wang, X., Girshick, R., Gupta, A., et al.: Non-local Neural Networks. In: Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City (2018)
Wang, Y., Tao, X., et al.: Image inpainting via generative multi-column convolutional neural networks. In: Proceedings of the 32nd International Conference on Neural Information Processing Systems, Montreal (2018)
Woo, S., Park, J., Lee, J.Y., et al.: CBAM: convolutional block attention module (2018). arXiv:1807.06521
Yang, C., et al.: High-resolution image inpainting using multi-scale neural patch synthesis. In: Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Washington (2017)
Yu, J., et al.: Free-form image inpainting with gated convolution. In: Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision, Seoul (2019)
Zhao, H., Jia, J., Koltun, V.: Exploring self-attention for image recognition. In: Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle (2020)
Zheng, C., Tat-Jen, C., Cai, J.: Pluralistic image completion. In: Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach (2019)
Thanh, D.N.H., Dvoenko, S.D.: A denoising of biomedical images. Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci. XL-5/W6, 73–78 (2015), https://doi.org/10.5194/isprsarchives-XL-5-W6-73-2015
Funding
This research was funded by University of Economics Ho Chi Minh City (UEH), Vietnam.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 Springer Nature Switzerland AG
About this paper
Cite this paper
Mao, Y., Zhang, T., Fu, B., Thanh, D.N.H. (2023). Single Image Inpainting Method Using Wasserstein Generative Adversarial Networks and Self-attention. In: Rousseau, JJ., Kapralos, B. (eds) Pattern Recognition, Computer Vision, and Image Processing. ICPR 2022 International Workshops and Challenges. ICPR 2022. Lecture Notes in Computer Science, vol 13644. Springer, Cham. https://doi.org/10.1007/978-3-031-37742-6_46
Download citation
DOI: https://doi.org/10.1007/978-3-031-37742-6_46
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-37741-9
Online ISBN: 978-3-031-37742-6
eBook Packages: Computer ScienceComputer Science (R0)
