Abstract
Deep learning is one of the most effective and efficient methods for facial emotion recognition, but it still encounters stability and infinite feasibility problems for faces of different races. To address this issue, we proposed a novel bottleneck feature extraction (BFE) method based on the deep neural network (DNN) model for facial emotion recognition. First, we used the Haar cascade classifier with a randomly generated mask to extract the face and remove the background from the image. Second, we removed the last output layer of the VGG16 transfer learning model, which was applied only for bottleneck feature extraction. Third, we designed a DNN model with five dense layers for feature training and used the famous Cohn-Kanade dataset for model training. Finally, we compared the proposed model with the K-nearest neighbor and logistic regression models on the same dataset. The experimental results showed that our model was more stable and could achieve a higher accuracy and F-measure, up to 98.59%, than other methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Han, H., Liu, Z., An, J.: Mining mobile intelligence for wireless systems: a deep neural network approach. IEEE Comput. Intell. Mag. 2, 24–31 (2020)
Hossain, M.S., Muhammad, G.: An emotion recognition system for mobile applications. IEEE Access 5, 2281–2287 (2017). https://doi.org/10.1109/ACCESS.2017.2672829
Eleftheriadis, S., Rudovic, O., Pantic, M.: Joint facial action unit detection and feature fusion: a multi-conditional learning approach. IEEE Trans. Image Process. 25(12), 5727–5742 (2016). https://doi.org/10.1109/TIP.2016.2615288
Happy, S.L., Routray, A.: Automatic facial expression recognition using features of salient facial patches. IEEE Trans. Affect. Comput. (2015) https://doi.org/10.1109/TAFFC.2014.2386334
Ryu, B., Rivera, A.R., Kim, J., Chae, O.: Local directional ternary pattern for facial expression recognition. IEEE Trans. Image Process. 26(12), 6006–6018 (2017). https://doi.org/10.1109/TIP.2017.2726010
Celis, D., Rao, M.: Learning facial recognition biases through VAE latent representations. In: FAT/MM 2019 - Proceedings of the 1st International Workshop on Fairness, Accountability, and Transparency in MultiMedia, Co-Located with MM 2019, pp. 26–32 (2019). https://doi.org/10.1145/3347447.3356752
Shen, X., Gu, Y.: Nonconvex sparse logistic regression with weakly convex regularization. IEEE Trans. Sig. Process. 66(12), 3199–3211 (2018). https://doi.org/10.1109/TSP.2018.2824289
Zhang, C., et al.: Multi-gram CNN-based self-attention model for relation classification. IEEE Access 7, 5343–5357 (2019). https://doi.org/10.1109/ACCESS.2018.2888508
Chen, S., Pande, A., Mohapatra, P.: Sensor-assisted facial recognition: an enhanced biometric authentication system for smartphones. In: MobiSys 2014 - Proceedings of the 12th Annual International Conference on Mobile Systems, Applications, and Services, pp. 109–122 (2014). https://doi.org/10.1145/2594368.2594373
Gu, Y., Wang, Y., Liu, T., Ji, Y., Liu, Z., et al.: EmoSense: computational intelligence driven emotion sensing via wireless channel data. IEEE Trans. Emerg. Top. Comput. Intell. (2019). https://doi.org/10.1109/TETCI.2019.2902438
Kiaee, N., Hashemizadeh, E., Zarrinpanjeh, N.: Using GLCM features in Haar wavelet transformed space for moving object classification. IET Intell. Transp. Syst. 13(7), 1148–1153 (2019). https://doi.org/10.1049/iet-its.2018.5192
Sharifara, A., Rahim, M.S.M., Navabifar, F., Ebert, D., Ghaderi, A., Papakostas, M.: Enhanced facial recognition framework based on skin tone and false alarm rejection. In: ACM International Conference on Pervasive Technologies Related to Assistive Environments, Part F1285, pp. 240–241 (2017). https://doi.org/10.1145/3056540.3064967
Viola, P., Jones, M.: Robust real-time object detection. Int. J. Comput. Vis. 57, 137–154 (2001)
Liu, F., Shen, C., Lin, G.: Deep convolutional neural fields for depth estimation from a single image. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2015). https://doi.org/10.1109/cvpr.2015.7299152
Facciolo, G., Limare, N., Meinhardt-Llopis, E.: Integral images for block matching. Image Process. Line (2014). https://doi.org/10.5201/ipol.2014.57
Karpathy, A.: CS231n convolutional neural networks for visual recognition. Stanford University (2016)
Brace, N., Kemp, R., Snelgar, R., Brace, N., Kemp, R., Snelgar, R.: Discriminant analysis and logistic regression. In: SPSS for Psychologists (2016). https://doi.org/10.1007/978-1-137-57923-2_11
Vasuki, A., Govindaraju, S.: Deep neural networks for image classification. In: Deep Learning for Image Processing Applications (2017)
Singh, V., Shokeen, V., Singh, B.: Face detection by haar cascade classifier with simple and complex backgrounds images using opencv implementation. Int. J. Adv. Technol. Eng. Sci. 1(12), 33–38 (2013)
Valstar, M.F., Pantic, M.: Combined support vector machines and hidden markov models for modeling facial action temporal dynamics. In: Lew, M., Sebe, N., Huang, T.S., Bakker, E.M. (eds.) HCI 2007. LNCS, vol. 4796, pp. 118–127. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-75773-3_13
Islam, M.F., Rahman, M.M.: Metal surface defect inspection through deep neural network. In: 2018 International Conference on Mechanical, Industrial and Energy Engineering, ICMIEE 2018, Khulna, Bangladesh, p. 258 (2018)
Ma, S., Bai, L.: A face detection algorithm based on Adaboost and new Haar-Like feature. In: 2016 7th IEEE International Conference on Software Engineering and Service Science (ICSESS). IEEE (2016)
Wu, B., et al.: Fast rotation invariant multi-view face detection based on real adaboost. In: 2004 Proceedings of the Sixth IEEE International Conference on Automatic Face and Gesture Recognition. IEEE (2004)
Wang, Y., et al.: Real time facial expression recognition with adaboost. In: 2004 Proceedings of the 17th International Conference on Pattern Recognition, ICPR 2004. IEEE (2004)
Lemaître, G., Nogueira, F., Aridas, C.K.: Imbalanced-learn: a python toolbox to tackle the curse of imbalanced datasets in machine learning. J. Mach. Learn. Res. 18(1), 559–563 (2017)
Yu, G., Zhang, X., Liu, Z., Ren, F.: BeSense: leveraging WiFi channel data and computational intelligence for behavior analysis. IEEE Comput. Intell. Mag. 14(4), 31–41 (2019). https://doi.org/10.1109/MCI.2019.2937610
Acknowledgments
This work was supported by the National Natural Science Foundation of China (61834005), the Enterprise Joint Fund Project of Shaanxi Natural Science Basic Research Plan (2019JLM-11-2), the Shaanxi Key Laboratory of network data analysis and intelligent processing, and the Japan Society for the Promotion of Science (JSPS) Grants-in-Aid for Scientific Research (KAKENHI) under Grant JP18K18044.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering
About this paper
Cite this paper
Ma, T. et al. (2020). Bottleneck Feature Extraction-Based Deep Neural Network Model for Facial Emotion Recognition. In: Loke, S.W., Liu, Z., Nguyen, K., Tang, G., Ling, Z. (eds) Mobile Networks and Management. MONAMI 2020. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 338. Springer, Cham. https://doi.org/10.1007/978-3-030-64002-6_3
Download citation
DOI: https://doi.org/10.1007/978-3-030-64002-6_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-64001-9
Online ISBN: 978-3-030-64002-6
eBook Packages: Computer ScienceComputer Science (R0)