Abstract
A way of embedded learning convolution neural network (ELCNN) based on the image content is proposed to evaluate the image aesthetic quality in this paper. Our approach can not only solve the problem of small-scale data but also score the image aesthetic quality. First, we chose Alexnet and VGG_S to compare for confirming which is more suitable for this image aesthetic quality evaluation task. Second, to further boost the image aesthetic quality classification performance, we employ the image content to train aesthetic quality classification models. But the training samples become smaller and only using once fine-tuning cannot make full use of the small-scale data set. Third, to solve the problem in second step, a way of using twice fine-tuning continually based on the aesthetic quality label and content label respective is proposed, the classification probability of the trained CNN models is used to evaluate the image aesthetic quality. The experiments are carried on the small-scale data set of Photo Quality. The experiment results show that the classification accuracy rates of our approach are higher than the existing image aesthetic quality evaluation approaches.
Similar content being viewed by others
References
R Datta, D Joshi and J Li, Studying Aesthetics in Photographic Images Using a Computational Approach, European Conference on Computer Vision, 288 (2006).
W Luo, X Wang and X Tang, Content-Based Photo Quality Assessment, IEEE International Conference on Computer Vision, 2206 (2011).
J Shao and Y Zhou, Journal of Computational Information Systems 9, 3209 (2013).
S Dhar, V Ordonez and L T Berg, High Level Describable Attributes for Predicting Aesthetics and Interestingness, IEEE Conference on Computer Vision and Pattern Recognition, 1657 (2011).
P Obrador, L Schmidt-Hackenberg and N Oliver, The role of Image Composition in Image Aesthetics, IEEE International Conference on Image Processing, 3185 (2010).
C Wang, Y Pu and D Xu, Evaluating aesthetics quality in scenery images, National Conference on Multimedia Technology, 141 (2015).
C Wang, Y Pu, D Xu, J Zhu and Z Tao, Journal of Software 26, 20 (2015). (in Chinses)
X Lu, Z Lin, H Jin, J Yang and J Wang, RAPID: Rating Pictorial Aesthetics using Deep Learning, 22nd ACM international conference on Multimedia, 457 (2014).
L Guo L and F Li, Image Aesthetic Evaluation Using Paralleled Deep Convolution Neural Network, ar- Xiv:1505.05225, Computer Science, 2015.
Zhou Y, Lu X, Zhang J and Wang J, Joint Image and Text Representation for Aesthetics Analysis, ACM on Multimedia Conference, 262 (2016).
Z Dong, X Shen, H Li and X Tian, Photo Quality Assessment with DCNN that Understands Image Well, International Conference on Multimedia Modeling, 524 (2015).
X Tian, Z Dong, K Yang and T Mei, IEEE Transactions on Multimedia 17, 2035 (2015).
N Murray, L Marchesotti and F Perronnin, AVA: A Large-Scale Database for Aesthetic Visual Analysis, IEEE Conference on Computer Vision and Pattern Recognition, 2408 (2012).
Hui-qiang Geng, Hua Zhang, Yan-bing Xue, Mian Zhou, Guang-ping Xu and Zan Gao, Optoelectronics Letters 13, 381 (2017).
Fang Xu and Jing-hong Liu, Optoelectronic Letters 12, 473 (2016).
A Krizhevsky, I Sutskever and G Hinton, ImageNet Classification with Deep Convolutional Neural Networks, International Conference on Neural Information Processing Systems, 1097 (2012).
K Chatfield, K Simonyan, A Vedaldi and A Zisserman, Return of the Devil in the Details: Delving Deep into Convolutional Nets, arXiv:1405.3531, Computer Science, 2014.
Y Jia, E Shelhamer, J Donahue, S Karayev, J Long, R Girshick, S Guadarrama and T Darrell, Caffe: Convolutional Architecture for Fast Feature Embedding, 22nd ACM International Conference on Multimedia, 675 (2014).
Author information
Authors and Affiliations
Corresponding author
Additional information
This work has been supported by the National Natural Science Foundation of China (Nos.61271361, 61163019, 61462093 and 61761046), the Research Foundation of Yunnan Province (Nos.2014FA021 and 2014FB113), and the Digital Media Technology Key Laboratory of Universities in Yunnan Province. This paper was presented in part at the CCF Chinese Conference on Computer Vision, Tianjin, 2017. This paper was recommended by the program committee.
Rights and permissions
About this article
Cite this article
Li, Yx., Pu, Yy., Xu, D. et al. Image aesthetic quality evaluation using convolution neural network embedded learning. Optoelectron. Lett. 13, 471–475 (2017). https://doi.org/10.1007/s11801-017-7203-6
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11801-017-7203-6