Skip to main content
SpringerLink
Log in

Technological Development of Image Aesthetics Assessment

  • Conference paper
  • First Online:
Image and Graphics (ICIG 2021)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 12890))

Included in the following conference series:

  • 2308 Accesses

Abstract

Quantitative research on aesthetics is a classic interdisciplinary research. With the rapid development of deep learning, various approaches have been made in image aesthetics assessment (IAA). Starting from the concept of image aesthetics, this report roughly follows the chronological sequence and first introduces the manual design of image aesthetic features. We divide IAA into generic image aesthetics assessment (GIAA) and personalized image aesthetics assessment (PIAA) to introduce separately in the deep learning part. Majority of approaches are GIAA, which purpose is to simulate general aesthetics. In this section, we separately reviewed representative studies of five assessment methods (aesthetic classification, aesthetic regression, aesthetic distribution, IAA with attributes, aesthetic description). Due to the subjectivity of aesthetics, human’s aesthetics will more or less deviate from the generic value. PIAA aims to model the aesthetic preferences of specific user, and the research is of great value. We introduced this novel research in the fifth section. Finally, image aesthetic datasets of different uses are summarized. We hope this comprehensive survey can be helpful to researchers in the field of image and enhance the connection between computer and art.

This work is supported by Research on Quality Evaluation Method of UHD Video based on Hevc (GJ181901).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 99.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 129.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Anderson, P., Fernando, B., Johnson, M., Gould, S.: SPICE: semantic propositional image caption evaluation. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9909, pp. 382–398. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46454-1_24

    Chapter  Google Scholar 

  2. Aydın, T.O., Smolic, A., Gross, M.: Automated aesthetic analysis of photographic images. IEEE Trans. Vis. Comput. Graph. 21(1), 31–42 (2014)

    Article  Google Scholar 

  3. Birkhoff, G.D.: Aesthetic Measure. Harvard University Press (2013)

    Google Scholar 

  4. Chang, K.Y., Lu, K.H., Chen, C.S.: Aesthetic critiques generation for photos. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 3514–3523 (2017)

    Google Scholar 

  5. Cristani, M., Vinciarelli, A., Segalin, C., Perina, A.: Unveiling the multimedia unconscious: Implicit cognitive processes and multimedia content analysis. In: Proceedings of the 21st ACM International Conference on Multimedia, pp. 213–222 (2013)

    Google Scholar 

  6. Cui, C., Liu, H., Lian, T., Nie, L., Zhu, L., Yin, Y.: Distribution-oriented aesthetics assessment with semantic-aware hybrid network. IEEE Trans. Multimedia 21(5), 1209–1220 (2018)

    Article  Google Scholar 

  7. Datta, R., Joshi, D., Li, J., Wang, J.Z.: Studying aesthetics in photographic images using a computational approach. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3953, pp. 288–301. Springer, Heidelberg (2006). https://doi.org/10.1007/11744078_23

    Chapter  Google Scholar 

  8. Dhar, S., Ordonez, V., Berg, T.L.: High level describable attributes for predicting aesthetics and interestingness. In: CVPR 2011, pp. 1657–1664. IEEE (2011)

    Google Scholar 

  9. Dong, Z., Shen, X., Li, H., Tian, X.: Photo quality assessment with DCNN that understands image well. In: He, X., Luo, S., Tao, D., Xu, C., Yang, J., Hasan, M.A. (eds.) MMM 2015. LNCS, vol. 8936, pp. 524–535. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-14442-9_57

    Chapter  Google Scholar 

  10. Elgammal, A., Liu, B., Elhoseiny, M., Mazzone, M.: CAN: Creative adversarial networks, generating “art” by learning about styles and deviating from style norms. arXiv preprint arXiv:1706.07068 (2017)

  11. Ghadiyaram, D., Bovik, A.C.: Massive online crowdsourced study of subjective and objective picture quality. IEEE Trans. Image Process. 25(1), 372–387 (2015)

    Article  MathSciNet  Google Scholar 

  12. Hou, L., Yu, C.P., Samaras, D.: Squared earth mover’s distance-based loss for training deep neural networks. arXiv preprint arXiv:1611.05916 (2016)

  13. Jin, X., et al.: Predicting aesthetic score distribution through cumulative Jensen-Shannon divergence. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32 (2018)

    Google Scholar 

  14. Jin, X., et al.: Aesthetic attributes assessment of images. In: Proceedings of the 27th ACM International Conference on Multimedia, pp. 311–319 (2019)

    Google Scholar 

  15. Jin, X., et al.: Predicting aesthetic radar map using a hierarchical multi-task network. In: Lai, J.H., et al. (eds.) PRCV 2018. LNCS, vol. 11257, pp. 41–50. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-03335-4_4

    Chapter  Google Scholar 

  16. Jin, X., Zhao, M., Chen, X., Zhao, Q., Zhu, S.-C.: Learning artistic lighting template from portrait photographs. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6314, pp. 101–114. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-15561-1_8

    Chapter  Google Scholar 

  17. Joshi, D., et al.: Aesthetics and emotions in images. IEEE Sig. Process. Mag. 28(5), 94–115 (2011)

    Article  Google Scholar 

  18. Kao, Y., He, R., Huang, K.: Deep aesthetic quality assessment with semantic information. IEEE Trans. Image Process. 26(3), 1482–1495 (2017)

    Article  MathSciNet  Google Scholar 

  19. Ke, Y., Tang, X., Jing, F.: The design of high-level features for photo quality assessment. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2006, vol. 1, pp. 419–426. IEEE (2006)

    Google Scholar 

  20. Kong, S., Shen, X., Lin, Z., Mech, R., Fowlkes, C.: Photo aesthetics ranking network with attributes and content adaptation. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 662–679. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46448-0_40

    Chapter  Google Scholar 

  21. Lakhal, S., Darmon, A., Bouchaud, J.P., Benzaquen, M.: Beauty and structural complexity. Phys. Rev. Res. 2(2), 022058 (2020)

    Article  Google Scholar 

  22. Li, L., Zhu, H., Zhao, S., Ding, G., Lin, W.: Personality-assisted multi-task learning for generic and personalized image aesthetics assessment. IEEE Trans. Image Process. 29, 3898–3910 (2020)

    Article  Google Scholar 

  23. Lu, X., Lin, Z., Jin, H., Yang, J., Wang, J.Z.: RAPID: rating pictorial aesthetics using deep learning. In: Proceedings of the 22nd ACM International Conference on Multimedia, pp. 457–466 (2014)

    Google Scholar 

  24. Lu, X., Lin, Z., Shen, X., Mech, R., Wang, J.Z.: Deep multi-patch aggregation network for image style, aesthetics, and quality estimation. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 990–998 (2015)

    Google Scholar 

  25. Luo, W., Wang, X., Tang, X.: Content-based photo quality assessment. In: 2011 International Conference on Computer Vision, pp. 2206–2213. IEEE (2011)

    Google Scholar 

  26. Ma, S., Liu, J., Wen Chen, C.: A-Lamp: adaptive layout-aware multi-patch deep convolutional neural network for photo aesthetic assessment. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4535–4544 (2017)

    Google Scholar 

  27. Machado, P., Cardoso, A.: Computing aesthetics. In: de Oliveira, F.M. (ed.) SBIA 1998. LNCS (LNAI), vol. 1515, pp. 219–228. Springer, Heidelberg (1998). https://doi.org/10.1007/10692710_23

    Chapter  Google Scholar 

  28. Mai, L., Jin, H., Liu, F.: Composition-preserving deep photo aesthetics assessment. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 497–506 (2016)

    Google Scholar 

  29. Malu, G., Bapi, R.S., Indurkhya, B.: Learning photography aesthetics with deep CNNs. arXiv preprint arXiv:1707.03981 (2017)

  30. Marchesotti, L., Perronnin, F., Larlus, D., Csurka, G.: Assessing the aesthetic quality of photographs using generic image descriptors. In: 2011 International Conference on Computer Vision, pp. 1784–1791. IEEE (2011)

    Google Scholar 

  31. Murray, N., Marchesotti, L., Perronnin, F.: AVA: a large-scale database for aesthetic visual analysis. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition, pp. 2408–2415. IEEE (2012)

    Google Scholar 

  32. Nishiyama, M., Okabe, T., Sato, I., Sato, Y.: Aesthetic quality classification of photographs based on color harmony. In: CVPR 2011, pp. 33–40. IEEE (2011)

    Google Scholar 

  33. Perc, M.: Beauty in artistic expressions through the eyes of networks and physics. J. R. Soc. Interface 17(164), 20190686 (2020)

    Article  Google Scholar 

  34. Polat, A.T., Akay, A.: Relationships between the visual preferences of urban recreation area users and various landscape design elements. Urban Forest. Urban Greening 14(3), 573–582 (2015)

    Article  Google Scholar 

  35. Ponomarenko, N., et al.: Color image database TID2013: peculiarities and preliminary results. In: European Workshop on Visual Information Processing (EUVIP), pp. 106–111. IEEE (2013)

    Google Scholar 

  36. Ren, J., Shen, X., Lin, Z., Mech, R., Foran, D.J.: Personalized image aesthetics. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 638–647 (2017)

    Google Scholar 

  37. Sheng, K., et al.: Revisiting image aesthetic assessment via self-supervised feature learning. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 5709–5716 (2020)

    Google Scholar 

  38. Sheng, K., Dong, W., Ma, C., Mei, X., Huang, F., Hu, B.G.: Attention-based multi-patch aggregation for image aesthetic assessment. In: Proceedings of the 26th ACM International Conference on Multimedia, pp. 879–886 (2018)

    Google Scholar 

  39. Talebi, H., Milanfar, P.: NIMA: neural image assessment. IEEE Trans. Image Process. 27, 3998–4011 (2017)

    Article  MathSciNet  Google Scholar 

  40. Tong, H., Li, M., Zhang, H.-J., He, J., Zhang, C.: Classification of digital photos taken by photographers or home users. In: Aizawa, K., Nakamura, Y., Satoh, S. (eds.) PCM 2004. LNCS, vol. 3331, pp. 198–205. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-30541-5_25

    Chapter  Google Scholar 

  41. Wang, W., Su, J., Li, L., Xu, X., Luo, J.: Meta-learning perspective for personalized image aesthetics assessment. In: 2019 IEEE International Conference on Image Processing (ICIP), pp. 1875–1879. IEEE (2019)

    Google Scholar 

  42. Wang, W., Yang, S., Zhang, W., Zhang, J.: Neural aesthetic image reviewer. IET Comput. Vis. 13(8), 749–758 (2019)

    Article  Google Scholar 

  43. Wong, L.K., Low, K.L.: Saliency-enhanced image aesthetics class prediction. In: 2009 16th IEEE International Conference on Image Processing (ICIP), pp. 997–1000. IEEE (2009)

    Google Scholar 

  44. Yeh, M.C., Cheng, Y.C.: Relative features for photo quality assessment. In: 2012 19th IEEE International Conference on Image Processing, pp. 2861–2864. IEEE (2012)

    Google Scholar 

  45. Zhang, J., Sclaroff, S., Lin, Z., Shen, X., Price, B., Mech, R.: Unconstrained salient object detection via proposal subset optimization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5733–5742 (2016)

    Google Scholar 

  46. Zhu, H., Li, L., Wu, J., Zhao, S., Ding, G., Shi, G.: Personalized image aesthetics assessment via meta-learning with bilevel gradient optimization. IEEE Trans. Cybern. (2020)

    Google Scholar 

Download references

Acknowledgements

This work is supported by Training of Outstanding Talents in Beijing in 2017, Research on Quality Evaluation Method of UHD Video based on HEVC (GJ181901).

Author information

Authors and Affiliations

  1. State Key Laboratory of Media Convergence and Communication, Communication University of China, Beijing, China

    Ruoyu Zou, Jiangbo Xu & Ziyu Xue

  2. Academy of Broadcasting Science (NRTA), Beijing, China

    Ziyu Xue

Authors
  1. Ruoyu Zou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jiangbo Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ziyu Xue
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Peking University, Beijing, China

    Yuxin Peng

  2. Tsinghua University, Beijing, China

    Shi-Min Hu

  3. Tampere University, Tampere, Finland

    Moncef Gabbouj

  4. Zhejiang University, Hangzhou, China

    Kun Zhou

  5. Technion – Israel Institute of Technology, Haifa, Israel

    Michael Elad

  6. Tsinghua University, Beijing, China

    Kun Xu

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zou, R., Xu, J., Xue, Z. (2021). Technological Development of Image Aesthetics Assessment. In: Peng, Y., Hu, SM., Gabbouj, M., Zhou, K., Elad, M., Xu, K. (eds) Image and Graphics. ICIG 2021. Lecture Notes in Computer Science(), vol 12890. Springer, Cham. https://doi.org/10.1007/978-3-030-87361-5_28

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-87361-5_28

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-87360-8

  • Online ISBN: 978-3-030-87361-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation