Abstract
Facial beauty prediction is a challenging problem in computer vision and multimedia fields, due to the variant pose and diverse conditions. In this paper, we introduce “soft label” for each annotated facial image, and propose a novel neural network–classification and regression network (CRNet) with different branches, to simultaneously process a classification and a regression task. Besides, weighted mean squared error (MSE) and cross entropy (CE) are used as the loss function, which is robust to outliers. CRNet achieves state-of-the-art performance on SCUT-FBP and ECCV HotOrNot dataset. Experimental results demonstrate the effectiveness of the proposed method and clarify the most important facial regions for facial beauty perception.
This work was primarily supported by Foundation Research Funds for the Central Universities (Program No. 2662017JC049) and State Scholarship Fund (NO. 261606765054).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Perrett, D.I., May, K.A., Yoshikawa, S.: Facial shape and judgements of female attractiveness. Nature 368(6468), 239–42 (1994)
Rothe, R., Timofte, R., Van Gool, L.: Some like it hot-visual guidance for preference prediction. In: Proceedings CVPR 2016, pp. 1–9 (2016)
Zhang, D., Chen, F., Xu, Y.: Computer Models for Facial Beauty Analysis. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-32598-9
Xie, D., Liang, L., Jin, L., Xu, J., Li, M.: DScut-fbp: a benchmark dataset for facial beauty perception. In: 2015 IEEE International Conference on Systems, Man, and Cybernetics (SMC), pp. 1821–1826. IEEE (2015)
Gray, D., Yu, K., Xu, W., Gong, Y.: Predicting Facial Beauty without Landmarks. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6316, pp. 434–447. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-15567-3_32
Chen, F., Xiao, X., Zhang, D.: Data-driven facial beauty analysis: prediction, retrieval and manipulation. IEEE Trans. Affect. Comput. 9(2), 205–216 (2018)
Eisenthal, Y., Dror, G., Ruppin, E.: Facial attractiveness: beauty and the machine. Neural Comput. 18(1), 119–142 (2006)
Huang, G.B., Lee, H., Learned-Miller, E.: Learning hierarchical representations for face verification with convolutional deep belief networks. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2518–2525. IEEE (2012)
Kagian, A., Dror, G., Leyvand, T., Cohen-Or, D., Ruppin, E.: A humanlike predictor of facial attractiveness. In: Advances in Neural Information Processing Systems, pp. 649–656 (2007)
Rothe, R., Timofte, R., Van Gool, L.: Deep expectation of real and apparent age from a single image without facial landmarks. Int. J. Comput. Vis. 126(2–4), 144–157 (2018)
Zhang, D., Zhao, Q., Chen, F.: Quantitative analysis of human facial beautyusing geometric features. Pattern Recognit. 44(4), 940–950 (2011)
Wang, S., Shao, M., Fu, Y.: Attractive or not?: beauty prediction with attractiveness-aware encoders and robust late fusion. In: Proceedings of the 22nd ACM International Conference on Multimedia, pp. 805–808. ACM (2014)
Xu, J., Jin, L., Liang, L., Feng, Z., Xie, D., Mao, H.: Facial attractiveness prediction using psychologically inspired convolutional neural network (PI-CNN). In: IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 1657–1661 (2017)
Ranjan, R., Patel, V.M., Chellappa, R.: Hyperface: a deep multi-task learning framework for face detection, landmark localization, pose estimation, and gender recognition. IEEE Trans. Pattern Anal. Mach. Intell. 1 (2018)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on CVPR, pp. 770–778 (2016)
Yosinski, J., Clune, J., Bengio, Y., Lipson, H.: How transferable are features in deep neural networks? In: Advances in Neural Information Processing Systems, pp. 3320–3328 (2014)
Donahue, J., et al.: Decaf: a deep convolutional activation feature for generic visual recognition. In: International Conference on Machine Learning, pp. 647–655 (2014)
Goodfellow, I., Bengio, Y., Courville, A., Bengio, Y.: Deep Learning, vol. 1. MIT Press, Cambridge (2016)
Gebru, T., Hoffman, J., Fei-Fei, L.: Fine-grained recognition in the wild: a multi-task domain adaptation approach. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 1358–1367. IEEE (2017)
King, D.E.: Dlib-ml: a machine learning toolkit. JMLR.org (2009)
Paszke, A., Gross, S., Chintala, S., Chanan, G.: Tensors and dynamic neural networks in python with strong GPU acceleration, Pytorch (2017)
Ioffe, I., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. In: International Conference on Machine Learning, pp. 448–456 (2015)
Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15(1), 1929–1958 (2014)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)
Liu, S., Fan, Y.-Y., Guo, Z., Samal, A., Ali, A.: A landmark-based data-driven approach on 2.5D facial attractiveness computation. Neurocomputing 238, 168–178 (2017)
Khan, S.H., Hayat, M., Bennamoun, M., Sohel, F.A., Togneri, R.: Costsensitive learning of deep feature representations from imbalanced data. IEEE Trans. Neural Netw. Learn. Syst. 29(8), 3573–3587 (2018)
Liu, W., Wen, Y., Yu, Z., Li, M., Raj, B., Song, L.: Sphereface: deep hypersphere embedding for face recognition. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), vol. 1, p. 1 (2017)
Yang, H., Ciftci, U., Yin, L.: Facial expression recognition by de-expression residue learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2168–2177 (2018)
Yang, T.-Y., Huang, Y.-H., Lin, Y.-Y., Hsiu, P.-C., Chuang, Y.-Y.: Ssr-net: a compact soft stagewise regression network for age estimation. In: IJCAI, pp. 1078–1084 (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Xu, L., Xiang, J., Yuan, X. (2018). CRNet: Classification and Regression Neural Network for Facial Beauty Prediction. In: Hong, R., Cheng, WH., Yamasaki, T., Wang, M., Ngo, CW. (eds) Advances in Multimedia Information Processing – PCM 2018. PCM 2018. Lecture Notes in Computer Science(), vol 11166. Springer, Cham. https://doi.org/10.1007/978-3-030-00764-5_61
Download citation
DOI: https://doi.org/10.1007/978-3-030-00764-5_61
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-00763-8
Online ISBN: 978-3-030-00764-5
eBook Packages: Computer ScienceComputer Science (R0)