Abstract
In order to achieve high recognition rate, most facial expression recognition (FER) methods generate sufficient labeled facial images based on generative adversarial networks (GAN) to train model. However, these methods do not estimate the facial pose before passing the images to the generator, which affects the quality of generated images. And mode collapse is prone to occur during the training process, leading to generate a single-style facial images. To solve these problems, a FER model is proposed based on pose conditioned dendritic convolution neural network (PCD-CNN) with pose and expression. Before passing the facial images to the generator, PCD-CNN was used to process facial images, effectively estimating the facial landmarks to detect face and disentangle the pose. In order to accelerate the training speed of the model, PCD-CNN was based on the ShuffleNet-v2 framework. Every landmark of facial image was modeled by a separate ShuffleNet-DeconvNet, maintaining better performance with fewer parameters. To solve the mode collapse during image generation, we theoretically analyzed the causes, and implemented mini-batch processing on the discriminator in the model and directly calculated the statistical characteristics of the mini-batch samples. Experiments were carried out on the Multi-PIE and BU-3DFE facial expression datasets. Compared with current advanced methods, our method achieves higher accuracy 93.08%, and the training process is more stable.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Jiao, Y., Niu, Y., Zhang, Y., Li, F., Zou, C., Shi, G.: Facial attention based convolutional neural network for 2D + 3D facial expression recognition. In: IEEE Visual Communications and Image Processing, pp. 1–4 (2019)
Sariyanidi, E., Gunes, H., Cavallaro, A.: Automatic analysis of facial affect: a survey of registration, representation, and recognition. IEEE Trans. Pattern Anal. Mach. Intell. 37(6), 1113–1133 (2015)
Zhang, F., Zhang, T., Mao, Q., Xu, C.: Joint pose and expression modeling for facial expression recognition. In: Conference on Computer Vision and Pattern Recognition, pp. 3359–3368 (2018)
Kumar, A., Chellappa, R.: Disentangling 3D pose in a dendritic CNN for unconstrained 2D face alignment. In: Conference on Computer Vision and Pattern Recognition, pp. 430–439 (2018)
Dong, J., Yuan, J., Li, L., Zhong, X., Liu, W.: An efficient semantic segmentation method using pyramid ShuffleNet V2 with vortex pooling. In: IEEE 31st International Conference on Tools with Artificial Intelligence, pp. 1214–1220 (2019)
Gross, R., Matthews, I., Cohn, J., Kanade, T., Baker, S.: Multi-PIE. In: IEEE International Conference on Automatic Face & Gesture Recognition, pp. 1–8 (2008)
Zhou, W., Zhao, C., Lu, L., Zhao, Q.: Dense correspondence of 3D facial point clouds via neural network fitting. In: IEEE International Conference on Image Processing, pp. 3731–3735 (2019)
Ekweariri, A.N., Yurtkan, K.: Facial expression recognition using enhanced local binary patterns. In: 9th International Conference on Computational Intelligence and Communication Networks (CICN), pp. 43–47 (2017)
Yang, J., Zhang, F., Chen, B., Khan, S.U.: Facial expression recognition based on facial action unit. In: Tenth International Green and Sustainable Computing Conference (IGSC), pp. 1–6 (2019)
Goodfellow, I., et al.: Generative adversarial nets. In: NIPS, pp. 2672–2680 (2014)
Yin, R.: Multi-resolution generative adversarial networks for tiny-scale pedestrian detection. In: IEEE International Conference on Image Processing (ICIP), pp. 1665–1669 (2019)
Salimans, T., Goodfellow, I., Zaremba, W., Cheung, V., Radford, A., Chen, X.: Improved techniques for training GANs (2016)
Karras, T., Aila, T., Laine, S., Lehtinen, J.: Progressive growing of GANs for improved quality, stability, and variation (2017)
Zeng, Z., Gong, Q., Zhang, J.: CNN model design of gesture recognition based on Tensorflow framework. In: IEEE 3rd Information Technology, Networking, Electronic and Automation Control Conference (ITNEC), pp. 1062–1067 (2019)
Zhang, Z.: Improved adam optimizer for deep neural networks. In: IEEE/ACM 26th International Symposium on Quality of Service, Banff, pp. 1–2 (2018)
Kim, S., Kim, H.: Deep explanation model for facial expression recognition through facial action coding unit. In: IEEE International Conference on Big Data and Smart Computing (BigComp), pp. 1–4 (2019)
Divya, M.B.S., Prajwala, N.B.: Facial expression recognition by calculating euclidian distance for eigen faces using PCA. In: International Conference on Communication and Signal Processing (ICCSP), pp. 0244–0248 (2018)
Ozdemir, M.A., Elagoz, B., Alaybeyoglu, A., Sadighzadeh, R., Akan, A.: Real Time Emotion Recognition from Facial Expressions Using CNN Architecture. In: Medical Technologies Congress (TIPTEKNO), pp. 1–4 (2019)
Gacav, C., Benligiray, B., Özkan, K., Topal, C.: Facial expression recognition with FHOG features. In: 26th Signal Processing and Communications Applications Conference (SIU), pp. 1–4 (2018)
Kim, D.H., Baddar, W.J., Jang, J., Ro, Y.M.: Multi-objective based spatio-temporal feature representation learning robust to expression intensity variations for facial expression recognition. IEEE Trans. Affect. Comput. 10(2), 223–236 (2019)
Kaleekal, T., Singh, J.: Facial Expression recognition using higher order moments on facial patches. In: 10th International Conference on Computing, Communication and Networking Technologies (ICCCNT), pp. 1–7 (2019)
Huang, R., Dong, H., Yin, G., Fu, Q.: Ensembling 3D CNN framework for video recognition. In: International Joint Conference on Neural Networks (IJCNN), pp. 1–7. Budapest, Hungary (2019)
Fu, Q., Wang, X., Dong, H., Huang, R.: Spiking neurons with differential evolution algorithm for pattern classification. In: IEEE International Conference on Systems, Man and Cybernetics (SMC), Bari, Italy, pp. 152–157 (2019)
Acknowledgment
We would like to acknowledge the support from the National Science Foundation of China (61472095).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Dong, H., Xu, J., Fu, Q. (2020). Facial Expression Recognition Based on PCD-CNN with Pose and Expression. In: Zeng, J., Jing, W., Song, X., Lu, Z. (eds) Data Science. ICPCSEE 2020. Communications in Computer and Information Science, vol 1257. Springer, Singapore. https://doi.org/10.1007/978-981-15-7981-3_38
Download citation
DOI: https://doi.org/10.1007/978-981-15-7981-3_38
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-15-7980-6
Online ISBN: 978-981-15-7981-3
eBook Packages: Computer ScienceComputer Science (R0)