Predicting Aesthetic Radar Map Using a Hierarchical Multi-task Network

  • Xin Jin
  • Le Wu
  • Xinghui Zhou
  • Geng Zhao
  • Xiaokun Zhang
  • Xiaodong Li
  • Shiming GeEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11257)


The aesthetic quality assessment of images is a challenging work in the field of computer vision because of its complex subjective semantic information. The recent research work can utilize the deep convolutional neural network to evaluate the overall score of the image. However, the focus in the field of aesthetic is often not limited to the total score of image, and multiple attribute of the aesthetic evaluation can obtain image richer aesthetic characteristics. The multi-attribute rating called Aesthetic Radar Map. In addition, traditional deep learning methods can only be predicted by classification or simple regression, and cannot output multi-dimensional information. In this paper, we propose a hierarchical multi-task dense network to make multiple regression of the properties of images. According to the total score, the scoring performance of each attribute is enhanced, and the output effect is better by optimizing the network structure. Through this method, the more sufficient aesthetic information of the image can be obtained, which is of certain guiding significance to the comprehensive evaluation of image aesthetics.


Aesthetic evaluation Neural network Computer vision 



We thank all the reviewers and ACs. This work is partially supported by the National Natural Science Foundation of China (grant numbers 61772047, 61772513, 61402021), the open funding project of CETC Big Data Research Institute Co.,Ltd., (grant number W-2018022), the Science and Technology Project of the State Archives Administrator (grant number 2015-B-10), the Open Research Fund of Beijing Key Laboratory of Big Data Technology for Food Safety (grant number BTBD-2018KF-07), Beijing Technology and Business University, and the Fundamental Research Funds for the Central Universities (grant numbers. 328201803, 328201801).


  1. 1.
    Mai, L., Jin, H., Liu, F.: Composition-preserving deep photo aesthetics assessment. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 497–506 (2016)Google Scholar
  2. 2.
    Deng, J., Dong, W., Socher, R., et al.: ImageNet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2009, pp. 248-255. IEEE (2009)Google Scholar
  3. 3.
    Karayev, S., Trentacoste, M., Han, H., et al.: Recognizing image style. arXiv preprint arXiv:1311.3715 (2013)
  4. 4.
    Lu, X., Lin, Z., Jin, H., et al.: RAPID: rating pictorial aesthetics using deep learning. In: Proceedings of the 22nd ACM International Conference on Multimedia, pp. 457–466. ACM (2014)Google Scholar
  5. 5.
    Kao, Y., Wang, C., Huang, K.: Visual aesthetic quality assessment with a regression model. In: 2015 IEEE International Conference on Image Processing (ICIP), pp. 1583–1587. IEEE (2015)Google Scholar
  6. 6.
    Lu, X., Lin, Z., Shen, X., et al.: Deep multi-patch aggregation network for image style, aesthetics, and quality estimation. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 990–998 (2015)Google Scholar
  7. 7.
    Lu, X., Lin, Z., Jin, H.: Rating image aesthetics using deep learning. IEEE Trans. Multimed. 17(11), 2021–2034 (2015)CrossRefGoogle Scholar
  8. 8.
    Dong, Z., Tian, X.: Multi-level photo quality assessment with multi-view features. Neurocomputing 168, 308–319 (2015)CrossRefGoogle Scholar
  9. 9.
    Kao, Y., Huang, K., Maybank, S.: Hierarchical aesthetic quality assessment using deep convolutional neural networks. Sig. Process. Image Commun. 47, 500–510 (2016)CrossRefGoogle Scholar
  10. 10.
    Wang, W., Zhao, M., Wang, L.: A multi-scene deep learning model for image aesthetic evaluation. Sig. Process. Image Commun. 47, 511–518 (2016)CrossRefGoogle Scholar
  11. 11.
    Ma, S., Liu, J., Chen, C.W.: A-Lamp: adaptive layout-aware multi-patch deep convolutional neural network for photo aesthetic assessment. CoRR abs/1704.00248. URL: (2017)
  12. 12.
    Kong, S., Shen, X., Lin, Z., Mech, R., Fowlkes, C.: Photo aesthetics ranking network with attributes and content adaptation. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 662–679. Springer, Cham (2016). Scholar
  13. 13.
    Jin, X., Chi, J., Peng, S., et al.: Deep image aesthetics classification using inception modules and fine-tuning connected layer. In: 2016 8th International Conference on Wireless Communications Signal Processing (WCSP), pp. 1–6. IEEE (2016)Google Scholar
  14. 14.
    Jin, X., Wu, L., Song, C., et al.: Predicting aesthetic score distribution through cumulative jensen-shannon divergence. In: Proceedings of the 32th International Conference of the America Association for Artificial Intelligence (AAAI 2018), New Orleans, Louisiana, 2–7 February 2018 (2017)Google Scholar
  15. 15.
    Kao, Y., He, R., Huang, K.: Deep aesthetic quality assessment with semantic information. IEEE Trans. Image Process. 26(3), 1482–1495 (2017)MathSciNetCrossRefGoogle Scholar
  16. 16.
    Wang, Z., Liu, D., Chang, S., et al.: Image aesthetics assessment using Deep Chatterjee’s machine. In: 2017 International Joint Conference on Neural Networks (IJCNN), pp. 941–948. IEEE (2017)Google Scholar
  17. 17.
    Wu, O., Hu, W., Gao, J.: Learning to predict the perceived visual quality of photos. In: 2011 IEEE International Conference on Computer Vision (ICCV), pp. 225–232. IEEE (2011)Google Scholar
  18. 18.
    Ke, Y., Tang, X., Jing, F.: The design of high-level features for photo quality assessment. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 419–426. IEEE (2006)Google Scholar
  19. 19.
    Murray, N., Marchesotti, L., Perronnin, F.: AVA: a large-scale database for aesthetic visual analysis. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2408–2415. IEEE (2012)Google Scholar
  20. 20.
    Iandola, F., Moskewicz, M., Karayev, S., et al.: DenseNet: implementing efficient convnet descriptor pyramids. arXiv preprint arXiv:1404.1869 (2014)
  21. 21.
    He, K., Zhang, X., Ren, S., et al.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016)Google Scholar
  22. 22.
    Chang, K.Y., Lu, K.H., Chen, C.S.: Aesthetic critiques generation for photos. In: 2017 IEEE International Conference on Computer Vision (ICCV) pp. 3534–3543. IEEE (2017)Google Scholar
  23. 23.
    Jin, B., Segovia, M.V.O., Süsstrunk, S.: Image aesthetic predictors based on weighted CNNs. In: 2016 IEEE International Conference on Image Processing (ICIP), pp. 2291–2295. IEEE (2016)Google Scholar
  24. 24.
    Hou, L., Yu, C.P., Samaras, D.: Squared earth mover’s distance-based loss for training deep neural networks. arXiv preprint arXiv:1611.05916 (2016)

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.Department of Cyber SecurityBeijing Electronic Science and Technology InstituteBeijingChina
  2. 2.CETC Big Data Research Institute Co., Ltd.GuiyangChina
  3. 3.Institute of Information EngineeringChinese Academy of SciencesBeijingChina

Personalised recommendations