Deep Compression on Convolutional Neural Network for Artistic Style Transfer
Deep artistic style transfer is popular yet costly as it is computationally expensive to generate artistic images using deep neural networks. We first ignore the network and only try an optimization method to generate artistic pictures, but the variation is limited. Then we speed up the style transfer by deep compression on the CNN layers of VGG. We simply remove inner ReLU functions within each convolutional block, such that each block containing two to three convolutional operation layers with ReLU in between collapses to a fully connected layer followed by a ReLU and a pooling layer. We use activation vectors in the modified network to morph the generated image. Experiments show that using the same loss function of Gatys et al. for style transfer the compressed neural network is competitive to the original VGG but is 2 to 3 times faster. The deep compression on convolutional neural networks shows alternative ways of generating artistic pictures.
KeywordsConvolutional neural network Deep compression Artistic style Back-propagation Computer vision
This research work was supported by National Science Foundation of China (61472147, 61772219) and Shenzhen Science and Technology Planning Project (JCYJ20170307154749425).
- 1.Gatys, L.A., Ecker, A.S., Bethge, M.: Image style transfer using convolutional neural networks. arXiv:1508.06576v2 [cs.CV], 2 September 2015
- 2.Gatys, L.A., Ecker, A.S., Bethge, M.: Texture synthesis and the controlled generation of natural stimuli using convolutional neural networks. arXiv:1505.07376
- 3.Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556 [cs.CV]
- 5.He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. arXiv:1512.03385 [cs.CV]
- 6.He, K., Wang, Y., Hopcroft, J.: A powerful generative model using random weights for the deep image representation. arXiv:1606.04801 [cs.CV]
- 7.Ulyanov, D., Lebedev, V., Vedaldi, A., Lempitsky, V.S.: Texture networks: feed-forward synthesis of textures and stylized images. arXiv:1603.03417 [cs.CV]
- 8.Bengio, Y., Courville, A., Vincent, P.: Representation learning: a review and new perspectives. arXiv:1206.5538 [cs.LG]
- 10.Mahendran, A., Vedaldi, A.: Understanding deep image representations by inverting them. In: CVPR, pp. 5188–5196 (2015)Google Scholar
- 11.Jia, Y., Shelhamer, E., Donahue, J., et al.: Caffe: convolutional architecture for fast feature embedding. In: Proceedings of the ACM International Conference on Multimedia, pp. 675–678. ACM (2014)Google Scholar
- 12.Abadi, M., et al.: TensorFlow: large-scale machine learning on heterogeneous distributed systems (2015). http://tensorflow.org/
- 13.Nikulin, Y., Novak, R.: Exploring the neural algorithm of artistic style. arXiv preprint arXiv:1602.07188