A Small Scale Multi-Column Network for Aesthetic Classification Based on Multiple Attributes

  • Chaoqun Wan
  • Xinmei Tian
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10634)


Image aesthetic quality assessment, which devotes to distinguishing whether an image is beautiful or not, has drawn a lot of attention in recent years. Recently deep learning has shown great power in data analysis and has been widely used in this field. However, on the one hand, deep learning is an end-to-end learning method that can be easily influenced by noisy data. On the other hand, prior information concluded from the experience of human perception of aesthetics, which widely applied in traditional aesthetic assessment methods, has not been effectively utilized in deep learning based aesthetic quality assessment methods. Therefore, in this paper we embed these prior information in deep learning as guidance for aesthetic quality assessment. Firstly, we design an extremely small network with only 38 K parameters for better training. Then we propose a multi-column network architecture to embed prior information into our deep learning model. We train our proposed network on AVA dataset, which is widely used for aesthetic assessment. The experimental results show that prior information indeed guides our network to learn better.


Aesthetic quality assessment Deep learning Multi-Column Prior information 



This work is supported by the 973 project 2015CB351803, NSFC No. 61572451 and No. 61390514, Youth Innovation Promotion Association CAS CX2100060016, and Fok Ying Tung Education Foundation WF2100060004.


  1. 1.
    Bhattacharya, S., Sukthankar, R., Shah, M.: A framework for photo-quality assessment and enhancement based on visual aesthetics. In: Proceedings of the 18th ACM International Conference on Multimedia, pp. 271–280. ACM (2010)Google Scholar
  2. 2.
    Datta, R., Joshi, D., Li, J., Wang, J.Z.: Studying aesthetics in photographic images using a computational approach. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3953, pp. 288–301. Springer, Heidelberg (2006). doi: 10.1007/11744078_23 CrossRefGoogle Scholar
  3. 3.
    Dhar, S., Ordonez, V., Berg, T.L.: High level describable attributes for predicting aesthetics and interestingness. In: 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1657–1664. IEEE (2011)Google Scholar
  4. 4.
    Dong, Z., Shen, X., Li, H., Tian, X.: Photo quality assessment with DCNN that understands image well. In: He, X., Luo, S., Tao, D., Xu, C., Yang, J., Hasan, M.A. (eds.) MMM 2015. LNCS, vol. 8936, pp. 524–535. Springer, Cham (2015). doi: 10.1007/978-3-319-14442-9_57 Google Scholar
  5. 5.
    Dong, Z., Tian, X.: Effective and efficient photo quality assessment. In: 2014 IEEE International Conference on Systems, Man and Cybernetics (SMC), pp. 2859–2864. IEEE (2014)Google Scholar
  6. 6.
    He, K., Zhang, X., Ren, S., Sun, J.: Delving deep into rectifiers: surpassing human-level performance on imagenet classification. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1026–1034 (2015)Google Scholar
  7. 7.
    Ke, Y., Tang, X., Jing, F.: The design of high-level features for photo quality assessment. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 419–426. IEEE (2006)Google Scholar
  8. 8.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)Google Scholar
  9. 9.
    Lin, M., Chen, Q., Yan, S.: Network in network. arXiv preprint arXiv:1312.4400 (2013)
  10. 10.
    Lu, X., Lin, Z., Jin, H., Yang, J., Wang, J.Z.: Rapid: rating pictorial aesthetics using deep learning. In: Proceedings of the 22nd ACM International Conference on Multimedia, pp. 457–466. ACM (2014)Google Scholar
  11. 11.
    Lu, X., Lin, Z., Shen, X., Mech, R., Wang, J.Z.: Deep multi-patch aggregation network for image style, aesthetics, and quality estimation. In: Proceedings of the IEEE International Conference on Computer Vision. pp. 990–998 (2015)Google Scholar
  12. 12.
    Luo, Y., Tang, X.: Photo and video quality evaluation: focusing on the subject. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008. LNCS, vol. 5304, pp. 386–399. Springer, Heidelberg (2008). doi: 10.1007/978-3-540-88690-7_29 CrossRefGoogle Scholar
  13. 13.
    Marchesotti, L., Perronnin, F., Larlus, D., Csurka, G.: Assessing the aesthetic quality of photographs using generic image descriptors. In: 2011 IEEE International Conference on Computer Vision (ICCV), pp. 1784–1791. IEEE (2011)Google Scholar
  14. 14.
    Murray, N., Marchesotti, L., Perronnin, F.: AVA: a large-scale database for aesthetic visual analysis. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2408–2415. IEEE (2012)Google Scholar
  15. 15.
    Nishiyama, M., Okabe, T., Sato, I., Sato, Y.: Aesthetic quality classification of photographs based on color harmony. In: 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 33–40. IEEE (2011)Google Scholar
  16. 16.
    Tang, X., Luo, W., Wang, X.: Content-based photo quality assessment. IEEE Trans. Multimed. 15(8), 1930–1943 (2013)CrossRefGoogle Scholar
  17. 17.
    Tian, X., Dong, Z., Yang, K., Mei, T.: Query-dependent aesthetic model with deep learning for photo quality assessment. IEEE Trans. Multimed. 17(11), 2035–2048 (2015)CrossRefGoogle Scholar
  18. 18.
    Tokumaru, M., Muranaka, N., Imanishi, S.: Color design support system considering color harmony. In: Proceedings of the 2002 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE 2002), vol. 1, pp. 378–383. IEEE (2002)Google Scholar
  19. 19.
    Tong, H., Li, M., Zhang, H.-J., He, J., Zhang, C.: Classification of digital photos taken by photographers or home users. In: Aizawa, K., Nakamura, Y., Satoh, S. (eds.) PCM 2004. LNCS, vol. 3331, pp. 198–205. Springer, Heidelberg (2004). doi: 10.1007/978-3-540-30541-5_25 CrossRefGoogle Scholar
  20. 20.
    Wang, W., Zhao, M., Wang, L., Huang, J., Cai, C., Xu, X.: A multi-scene deep learning model for image aesthetic evaluation. Signal Process. Image Commun. 47, 511–518 (2016)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.CAS Key Laboratory of Technology in Geo-spatial Information Processing and Application SystemUniversity of Science and Technology of ChinaAnhuiChina

Personalised recommendations