Skip to main content

Predicting Aesthetic Radar Map Using a Hierarchical Multi-task Network

Part of the Lecture Notes in Computer Science book series (LNIP,volume 11257)

Abstract

The aesthetic quality assessment of images is a challenging work in the field of computer vision because of its complex subjective semantic information. The recent research work can utilize the deep convolutional neural network to evaluate the overall score of the image. However, the focus in the field of aesthetic is often not limited to the total score of image, and multiple attribute of the aesthetic evaluation can obtain image richer aesthetic characteristics. The multi-attribute rating called Aesthetic Radar Map. In addition, traditional deep learning methods can only be predicted by classification or simple regression, and cannot output multi-dimensional information. In this paper, we propose a hierarchical multi-task dense network to make multiple regression of the properties of images. According to the total score, the scoring performance of each attribute is enhanced, and the output effect is better by optimizing the network structure. Through this method, the more sufficient aesthetic information of the image can be obtained, which is of certain guiding significance to the comprehensive evaluation of image aesthetics.

Keywords

  • Aesthetic evaluation
  • Neural network
  • Computer vision

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-03335-4_4
  • Chapter length: 10 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   79.99
Price excludes VAT (USA)
  • ISBN: 978-3-030-03335-4
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   99.99
Price excludes VAT (USA)
Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.
Fig. 5.
Fig. 6.

Notes

  1. 1.

    https://gurushots.com/.

  2. 2.

    https://github.com/keras-team/keras/.

References

  1. Mai, L., Jin, H., Liu, F.: Composition-preserving deep photo aesthetics assessment. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 497–506 (2016)

    Google Scholar 

  2. Deng, J., Dong, W., Socher, R., et al.: ImageNet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2009, pp. 248-255. IEEE (2009)

    Google Scholar 

  3. Karayev, S., Trentacoste, M., Han, H., et al.: Recognizing image style. arXiv preprint arXiv:1311.3715 (2013)

  4. Lu, X., Lin, Z., Jin, H., et al.: RAPID: rating pictorial aesthetics using deep learning. In: Proceedings of the 22nd ACM International Conference on Multimedia, pp. 457–466. ACM (2014)

    Google Scholar 

  5. Kao, Y., Wang, C., Huang, K.: Visual aesthetic quality assessment with a regression model. In: 2015 IEEE International Conference on Image Processing (ICIP), pp. 1583–1587. IEEE (2015)

    Google Scholar 

  6. Lu, X., Lin, Z., Shen, X., et al.: Deep multi-patch aggregation network for image style, aesthetics, and quality estimation. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 990–998 (2015)

    Google Scholar 

  7. Lu, X., Lin, Z., Jin, H.: Rating image aesthetics using deep learning. IEEE Trans. Multimed. 17(11), 2021–2034 (2015)

    CrossRef  Google Scholar 

  8. Dong, Z., Tian, X.: Multi-level photo quality assessment with multi-view features. Neurocomputing 168, 308–319 (2015)

    CrossRef  Google Scholar 

  9. Kao, Y., Huang, K., Maybank, S.: Hierarchical aesthetic quality assessment using deep convolutional neural networks. Sig. Process. Image Commun. 47, 500–510 (2016)

    CrossRef  Google Scholar 

  10. Wang, W., Zhao, M., Wang, L.: A multi-scene deep learning model for image aesthetic evaluation. Sig. Process. Image Commun. 47, 511–518 (2016)

    CrossRef  Google Scholar 

  11. Ma, S., Liu, J., Chen, C.W.: A-Lamp: adaptive layout-aware multi-patch deep convolutional neural network for photo aesthetic assessment. CoRR abs/1704.00248. URL: http://arxiv.org/abs/1704.00248 (2017)

  12. Kong, S., Shen, X., Lin, Z., Mech, R., Fowlkes, C.: Photo aesthetics ranking network with attributes and content adaptation. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 662–679. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46448-0_40

    CrossRef  Google Scholar 

  13. Jin, X., Chi, J., Peng, S., et al.: Deep image aesthetics classification using inception modules and fine-tuning connected layer. In: 2016 8th International Conference on Wireless Communications Signal Processing (WCSP), pp. 1–6. IEEE (2016)

    Google Scholar 

  14. Jin, X., Wu, L., Song, C., et al.: Predicting aesthetic score distribution through cumulative jensen-shannon divergence. In: Proceedings of the 32th International Conference of the America Association for Artificial Intelligence (AAAI 2018), New Orleans, Louisiana, 2–7 February 2018 (2017)

    Google Scholar 

  15. Kao, Y., He, R., Huang, K.: Deep aesthetic quality assessment with semantic information. IEEE Trans. Image Process. 26(3), 1482–1495 (2017)

    MathSciNet  CrossRef  Google Scholar 

  16. Wang, Z., Liu, D., Chang, S., et al.: Image aesthetics assessment using Deep Chatterjee’s machine. In: 2017 International Joint Conference on Neural Networks (IJCNN), pp. 941–948. IEEE (2017)

    Google Scholar 

  17. Wu, O., Hu, W., Gao, J.: Learning to predict the perceived visual quality of photos. In: 2011 IEEE International Conference on Computer Vision (ICCV), pp. 225–232. IEEE (2011)

    Google Scholar 

  18. Ke, Y., Tang, X., Jing, F.: The design of high-level features for photo quality assessment. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 419–426. IEEE (2006)

    Google Scholar 

  19. Murray, N., Marchesotti, L., Perronnin, F.: AVA: a large-scale database for aesthetic visual analysis. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2408–2415. IEEE (2012)

    Google Scholar 

  20. Iandola, F., Moskewicz, M., Karayev, S., et al.: DenseNet: implementing efficient convnet descriptor pyramids. arXiv preprint arXiv:1404.1869 (2014)

  21. He, K., Zhang, X., Ren, S., et al.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016)

    Google Scholar 

  22. Chang, K.Y., Lu, K.H., Chen, C.S.: Aesthetic critiques generation for photos. In: 2017 IEEE International Conference on Computer Vision (ICCV) pp. 3534–3543. IEEE (2017)

    Google Scholar 

  23. Jin, B., Segovia, M.V.O., Süsstrunk, S.: Image aesthetic predictors based on weighted CNNs. In: 2016 IEEE International Conference on Image Processing (ICIP), pp. 2291–2295. IEEE (2016)

    Google Scholar 

  24. Hou, L., Yu, C.P., Samaras, D.: Squared earth mover’s distance-based loss for training deep neural networks. arXiv preprint arXiv:1611.05916 (2016)

Download references

Acknowledgments

We thank all the reviewers and ACs. This work is partially supported by the National Natural Science Foundation of China (grant numbers 61772047, 61772513, 61402021), the open funding project of CETC Big Data Research Institute Co.,Ltd., (grant number W-2018022), the Science and Technology Project of the State Archives Administrator (grant number 2015-B-10), the Open Research Fund of Beijing Key Laboratory of Big Data Technology for Food Safety (grant number BTBD-2018KF-07), Beijing Technology and Business University, and the Fundamental Research Funds for the Central Universities (grant numbers. 328201803, 328201801).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Shiming Ge .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2018 Springer Nature Switzerland AG

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

Jin, X. et al. (2018). Predicting Aesthetic Radar Map Using a Hierarchical Multi-task Network. In: , et al. Pattern Recognition and Computer Vision. PRCV 2018. Lecture Notes in Computer Science(), vol 11257. Springer, Cham. https://doi.org/10.1007/978-3-030-03335-4_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-03335-4_4

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-03334-7

  • Online ISBN: 978-3-030-03335-4

  • eBook Packages: Computer ScienceComputer Science (R0)