Abstract
In some image restoration algorithms of the past, they often do not consider the continuity between pixels, and the internal features of the hole region. The mapping to the image semantically does not take into account the continuity of the feature, resulting in the color of the fault. Or the deformation of the edge contour of the image. In some of the current popular algorithms and models, we can clearly see the color faults and artificial repair traces from their repair results. These discontinuities are mainly because these methods ignore the semantic relevance and feature continuity of the hole region. Therefore, if we want to get a better image repair effect. We have to improve on semantic relevance and feature continuity. We validated the effectiveness of our proposed method in image restoration tasks on the CelebA and Places2 datasets, and our results yielded a better visual experience in some images than existing methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Liu, G., Reda, F.A., Shih, K.J., Wang, T.C., Tao, A., Catanzaro, B.: Image inpainting for irregular holes using partial convolutions. arXiv preprint arXiv:1804.07723 (2018)
Yu, J., Lin, Z., Yang, J., Shen, X., Lu, X., Huang, T.S.: Free-form image inpainting with gated convolution. arXiv preprint arXiv:1806.03589 (2018)
Liu, H., Jiang, B., Xiao, Y., Yang, C.: Coherent Semantic Attention for Image Inpainting. arXiv preprint arXiv:1905.12384 (2019)
Walker, J., Doersch, C., Gupta, A., Hebert, M.: An uncertain future: forecasting from static images using variational autoencoders. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9911, pp. 835–851. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46478-7_51
Isola, P., Zhu, J.Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: CVPR (2017)
Yeh, R.A., Chen, C., Lim, T.Y., Schwing, A.G., Hasegawa-Johnson, M., Do, M.N.: Semantic image inpainting with perceptual and contextual losses. arXiv preprint arXiv:1607.07539 (2016)
Johnson, J., Alahi, A., Fei-Fei, L.: Perceptual losses for real-time style transfer and super-resolution. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9906, pp. 694–711. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46475-6_43
Yu, J., Lin, Z., Yang, J., Shen, X., Lu, X., Huang, T.S.: Generative image inpainting with contextual attention. In: Computer Vision and Pattern Recognition (cs.CV) (2018)
Goodfellow, I., et al.: Generative adversarial nets. In Advances in neural information processing systems, pp. 2672–2680 (2014)
Pathak, D., Krahenbuhl, P., Donahue, J., Darrell, T., Efros, A.A.: Context encoders: feature learning by inpainting. In: CVPR (2016)
Iizuka, S., Simo-Serra, E., Ishikawa, H.: Globally and locally consistent image completion. ACM Trans. Graph. 36(4), 1–14 (2017)
Wang, Y., Tao, X., Qi, X., Shen, X., Jia, J.: Image inpainting via generative multi-column convolutional neural networks. In: Advances in Neural Information Processing Systems (NeurIPS), pp. 329–338 (2018)
Zhang, H., Hu, Z., Luo, C., Zuo, W., Wang, M.: Semantic image inpainting with progressive generative networks. In: ACM International Conference on Multimedia (ACMMM), pp. 1939–1947 (2018)
Nazeri, K., Ng, E., Joseph, T., Qureshi, F., Ebrahimi, M.: Generative image inpainting with adversarial edge learning. arXiv preprint arXiv:1901.00212 (2019)
Xiong, W., et al.: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2019)
Liu, G., Reda, F.A., Shih, K.J., Wang, T.C., Tao, A., Catanzaro, B.: Image inpainting for irregular holes using partial convolutions. In: The European Conference on Computer Vision (ECCV), vol. 11215, pp. 89–105 (2018)
Fang, W., Zhang, F., Sheng, J., Ding, Y.: A new sequential image prediction method based on LSTM and DCGAN. CMC Comput. Mater. Continua 64, 217–231 (2019)
Fang, W., Zhang, F., Sheng, V.S., Ding, Y.: A method for improving CNN-based image recognition using DCGAN. CMC Comput. Mater. Continua 57(1), 167–178 (2018)
Pan, L., Qin, J., Chen, H., Xiang, X., Li, C., Chen, R.: Image augmentation-based food recognition with convolutional neural net works. Comput. Mater. Continua 59(1), 297–313 (2019)
Tu, Y., Lin, Y., Wang, J., Kim, J.: Semi-supervised learning with generative adversarial networks on digital signal modulation classification. Comput. Mater. Continua 55(2), 243–254 (2018)
Li, X., Liang, Y., Zhao, M., Wang, C., Jiang, Y.: Few-shot learning with generative adversarial networks based on WOA13 data. Comput. Mater. Continua 60(3), 1073–1085 (2019)
Acknowledgements
This work was supported in part by the Open Research Project of State Key Laboratory of Novel Software Technology under Grant KFKT2018B23, the Priority Academic Program Development of Jiangsu Higher Education Institutions, and the Open Project Program of the State Key Lab of CAD&CG (Grant No. A1916), Zhejiang University.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Wang, W., Gu, E., Fang, W. (2020). An Improvement of Coherent Semantic Attention for Image Inpainting. In: Sun, X., Wang, J., Bertino, E. (eds) Artificial Intelligence and Security. ICAIS 2020. Communications in Computer and Information Science, vol 1252. Springer, Singapore. https://doi.org/10.1007/978-981-15-8083-3_24
Download citation
DOI: https://doi.org/10.1007/978-981-15-8083-3_24
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-15-8082-6
Online ISBN: 978-981-15-8083-3
eBook Packages: Computer ScienceComputer Science (R0)