Advertisement

Deep Compression on Convolutional Neural Network for Artistic Style Transfer

  • Jian Hu
  • Kun He
  • John E. Hopcroft
  • Yaren Zhang
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 768)

Abstract

Deep artistic style transfer is popular yet costly as it is computationally expensive to generate artistic images using deep neural networks. We first ignore the network and only try an optimization method to generate artistic pictures, but the variation is limited. Then we speed up the style transfer by deep compression on the CNN layers of VGG. We simply remove inner ReLU functions within each convolutional block, such that each block containing two to three convolutional operation layers with ReLU in between collapses to a fully connected layer followed by a ReLU and a pooling layer. We use activation vectors in the modified network to morph the generated image. Experiments show that using the same loss function of Gatys et al. for style transfer the compressed neural network is competitive to the original VGG but is 2 to 3 times faster. The deep compression on convolutional neural networks shows alternative ways of generating artistic pictures.

Keywords

Convolutional neural network Deep compression Artistic style Back-propagation Computer vision 

Notes

Acknowledgements

This research work was supported by National Science Foundation of China (61472147, 61772219) and Shenzhen Science and Technology Planning Project (JCYJ20170307154749425).

References

  1. 1.
    Gatys, L.A., Ecker, A.S., Bethge, M.: Image style transfer using convolutional neural networks. arXiv:1508.06576v2 [cs.CV], 2 September 2015
  2. 2.
    Gatys, L.A., Ecker, A.S., Bethge, M.: Texture synthesis and the controlled generation of natural stimuli using convolutional neural networks. arXiv:1505.07376
  3. 3.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556 [cs.CV]
  4. 4.
    Russakovsky, O., Deng, J., Hao, S., et al.: ImageNet large scale visual recognition challenge. Int. J. Comput. Vis. (IJCV) 115(3), 211–252 (2015)MathSciNetCrossRefGoogle Scholar
  5. 5.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. arXiv:1512.03385 [cs.CV]
  6. 6.
    He, K., Wang, Y., Hopcroft, J.: A powerful generative model using random weights for the deep image representation. arXiv:1606.04801 [cs.CV]
  7. 7.
    Ulyanov, D., Lebedev, V., Vedaldi, A., Lempitsky, V.S.: Texture networks: feed-forward synthesis of textures and stylized images. arXiv:1603.03417 [cs.CV]
  8. 8.
    Bengio, Y., Courville, A., Vincent, P.: Representation learning: a review and new perspectives. arXiv:1206.5538 [cs.LG]
  9. 9.
    Johnson, J., Alahi, A., Fei-Fei, L.: Perceptual losses for real-time style transfer and super-resolution. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9906, pp. 694–711. Springer, Cham (2016). doi: 10.1007/978-3-319-46475-6_43 CrossRefGoogle Scholar
  10. 10.
    Mahendran, A., Vedaldi, A.: Understanding deep image representations by inverting them. In: CVPR, pp. 5188–5196 (2015)Google Scholar
  11. 11.
    Jia, Y., Shelhamer, E., Donahue, J., et al.: Caffe: convolutional architecture for fast feature embedding. In: Proceedings of the ACM International Conference on Multimedia, pp. 675–678. ACM (2014)Google Scholar
  12. 12.
    Abadi, M., et al.: TensorFlow: large-scale machine learning on heterogeneous distributed systems (2015). http://tensorflow.org/
  13. 13.
    Nikulin, Y., Novak, R.: Exploring the neural algorithm of artistic style. arXiv preprint arXiv:1602.07188

Copyright information

© Springer Nature Singapore Pte Ltd. 2017

Authors and Affiliations

  • Jian Hu
    • 1
    • 3
  • Kun He
    • 1
    • 3
  • John E. Hopcroft
    • 2
  • Yaren Zhang
    • 1
  1. 1.School of Computer Science and TechnologyHuazhong University of Science and TechnologyWuhanChina
  2. 2.Department of Computer ScienceCornell UniversityIthacaUSA
  3. 3.Shenzhen Research Institute of Huazhong University of Science and TechnologyShenzhenChina

Personalised recommendations