Advertisement

Enhance Target Features in Real-Time Arbitrary Style Transfer

  • Yuxiong Fang
  • Hong JiangEmail author
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 1075)

Abstract

Style transfer aims to transfer arbitrary visual styles to content images, and real-time arbitrary style transfer is a hot research topic, it aims real-time and arbitrary style transfer. Two current representative real-time arbitrary style transfer methods, AdaIN method and WCT method, have their own advantages and disadvantages. In this paper, the theory of the two methods were outlined, the advantages and disadvantages of the two methods were analyzed, and the influence of WCT module on the style transfer of each-level features was analyzed. Considering the combination of the two methods to overcome their shortcomings, the WCT module is added in the AdaIN method, the result feature obtained by AdaIN module is further processed by WCT module to obtain the target feature, and the content image is preprocessed by gray processing to remove some of its style features while retain content, the experimental results show that the method is feasible and effective.

Keywords

Style transfer Neural network Computer vision 

References

  1. 1.
    Gatys, L.A., Ecker, A.S., Bethge, M.: Image style transfer using convolutional neural networks. In: CVPR (2016)Google Scholar
  2. 2.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: ICLR (2015)Google Scholar
  3. 3.
    Gatys, L.A., Ecker, A.S., Bethge, M.: Texture synthesis using convolutional neural networks. In: NIPS (2015)Google Scholar
  4. 4.
    Mahendran, A., Vedaldi, A.: Understanding deep image representations by inverting them. In: CVPR (2015)Google Scholar
  5. 5.
    Jing, Y., Yang, Y., Feng, Z., Ye, J., Yu, Y., Song, M.: Neural style transfer: a review. arXiv preprint arXiv:1705.04058 (2017)
  6. 6.
    Prisma Labs, I.: Prisma: turn memories into art using artificial intelligence (2016). http://prisma-ai.com
  7. 7.
    Ostagram (2016). http://ostagram.ru
  8. 8.
    Champandard, A.J.: Deep forger: paint photos in the style of famous artists (2015). http://deepforger.com
  9. 9.
    Johnson, J., Alahi, A., Fei-Fei, L.: Perceptual losses for real-time style transfer and super-resolution. In: European Conference on Computer Vision, pp. 694–711. Springer, Heidelberg (2016)CrossRefGoogle Scholar
  10. 10.
    Li, C., Wand, M.: Precomputed real-time texture synthesis with markovian generative adversarial networks. In: European Conference on Computer Vision, pp. 702–716. Springer, Heidelberg (2016)CrossRefGoogle Scholar
  11. 11.
    Ulyanov, D., Lebedev, V., Vedaldi, A., Lempitsky, V.S.: Texture networks: feed-forward synthesis of textures and stylized images. In: ICML, pp. 1349–1357 (2016)Google Scholar
  12. 12.
    Wang, X., Oxholm, G., Zhang, D., Wang, Y.F.: Multimodal transfer: a hierarchical deep convolutional neural network for fast artistic style transfer. In: CVPR (2017)Google Scholar
  13. 13.
    Huang, X., Belongie, S.: Arbitrary style transfer in real-time with adaptive instance normalization. In ICCV (2017)Google Scholar
  14. 14.
    Li, Y., Fang, C., Yang, J., Wang, Z., Lu, X., Yang, M.-H.: Universal style transfer via feature transforms. In NIPS (2017)Google Scholar
  15. 15.
    Sanakoyeu, A., Kotovenko, D., Lang, S., Ommer, B.: A style-aware content loss for real-time HD style transfer. In: ECCV (2018)Google Scholar
  16. 16.
    Pęśko, M., Trzciński, T.: Neural comic style transfer: case study. arXiv preprint arXiv:1809.01726 (2018)
  17. 17.
    Ioffe, S., Szegedy, C.: Batch normalization: Accelerating deep network training by reducing internal covariate shift. In: JMLR, vol. 2 (2015)Google Scholar
  18. 18.
    Ulyanov, D., Vedaldi, A., Lempitsky, V.: Improved texture networks: maximizing quality and diversity in feed-forward stylization and texture synthesis. In: CVPR (2017)Google Scholar
  19. 19.
    Dumoulin, V., Shlens, J., Kudlur, M.: A learned representation for artistic style. In: ICLR (2017)Google Scholar
  20. 20.
    Majumdar, S., Bhoi, A., Jagadeesan, G.: A comprehensive comparison between neural style transfer and universal style transfer. arXiv preprint arXiv:1806.00868 (2018)

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Computer CenterEast China Normal UniversityShanghaiChina

Personalised recommendations