Abstract
Due to the broad application potential and business demand, facial expression recognition is one of the prevalent topics in the deep learning domain. Since most of the application domain of facial expression is deployed on devices with low computational power, this work majorly focuses on a lightweight model. Therefore, in the proposed model, a lightweight vision transformer model, MobileVit that surpasses other light weight CNN like MobileNetV3, NasNet on the ImageNet classification dataset is chosen for the classification of facial expression. The model was initially pretrained on CelebA dataset, then was trained on RAF-DB dataset, and fine-tuning of the model was carried out. For improving the generalization capability of the model, advanced augmentation methods like cut mix with basic augmentation methods are deployed. For the visualization of the model performance, interpretation of feature maps was carried out utilizing the Score-Cam technique. Squeeze-and-excitation layer was added to improve the feature extraction performance of the model. After fine-tuning of the model, the performance of the model was evaluated on KDEF, FER-2013, and AffectNEt dataset.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Karanchery S, Palaniswamy S (2021) Emotion recognition using one-shot learning for human-computer interactions, vol 1. IEEE, pp 1–8
Palaniswamy S, Tripathi S (2018) Emotion recognition from facial expressions using images with pose, illumination and age variation for human-computer/robot interaction. J ICT Res Appl 12(1)
Menon S, Swathi J, Anit S, Nair AP, Sarath S (2019) Driver face recognition and sober drunk classification using thermal images. IEEE, pp 0400–0404
Gautam K, Thangavel SK (2021) Video analytics-based facial emotion recognition system for smart buildings. Int J Comput Appl 43(9):858–867
Archana M, Nitish C, Harikumar S (2022) Real time face detection and optimal face mapping for online classes. 2161(1):012063
Keshari T, Palaniswamy S (2019) Emotion recognition using feature-level fusion of facial expressions and body gestures. In: 2019 International conference on communication and electronics systems (ICCES). IEEE, pp 1184–1189
Dosovitskiy A, Beyer L, Kolesnikov A, Weissenborn D, Zhai X, Unterthiner T, Dehghani M, Minderer M, Heigold G, Gelly S et al (2020) An image is worth \(16\,\times \,16\) words: transformers for image recognition at scale. arXiv preprint arXiv:2010.11929
Mehta S, Rastegari M (2021) Mobilevit: light-weight, general-purpose, and mobile-friendly vision transformer. arXiv preprint arXiv:2110.02178
Li S, Deng W, Du J (2017) Reliable crowdsourcing and deep locality-preserving learning for expression recognition in the wild. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2852–2861
Mollahosseini A, Hasani B, Mahoor MH (2017) Affectnet: a database for facial expression, valence, and arousal computing in the wild. IEEE Trans Affect Comput 10(1):18–31
Lundqvist D, Flykt A, Öhman A (1998) Karolinska directed emotional faces. Cogn Emot
Goodfellow IJ, Erhan D, Carrier PL, Courville A, Mirza M, Hamner B, Cukierski W, Tang Y, Thaler D, Lee D-H et al (2013) Challenges in representation learning: a report on three machine learning contests. International conference on neural information processing. Springer, Berlin, pp 117–124
Sandler M, Howard A, Zhu M, Zhmoginov A, Chen L-C (2018) Mobilenetv2: inverted residuals and linear bottlenecks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4510–4520
Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7132–7141
Farzaneh AH, Qi X (2021) Facial expression recognition in the wild via deep attentive center loss. In: Proceedings of the IEEE/CVF winter conference on applications of computer vision, pp 2402–2411
Alshamsi H, Kepuska V, Meng H (2017) Automated facial expression recognition app development on smart phones using cloud computing. In: 2017 IEEE 8th Annual ubiquitous computing, electronics and mobile communication conference (UEMCON). IEEE, pp 577–583
Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556
Shi J, Zhu S, Liang Z (2021) Learning to amend facial expression representation via de-albino and affinity. arXiv preprint arXiv:2103.10189
Ma H, Celik T, Li H-C (2021) Lightweight attention convolutional neural network through network slimming for robust facial expression recognition. Signal Image Video Process 15(7):1507–1515
Khanzada A, Bai C, Celepcikay FT (2020) Facial expression recognition with deep learning. arXiv preprint arXiv:2004.11823
Gera D, Balasubramanian S (2021) Consensual collaborative training and knowledge distillation based facial expression recognition under noisy annotations. arXiv preprint arXiv:2107.04746
Pecoraro R, Basile V, Bono V, Gallo S (2021) Local multi-head channel self-attention for facial expression recognition. arXiv preprint arXiv:2111.07224
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Devika, R., Divya Udayan, J. (2023). Lightweight Deep Learning Facial Expression Recognition Model. In: Tuba, M., Akashe, S., Joshi, A. (eds) ICT Systems and Sustainability. Lecture Notes in Networks and Systems, vol 516. Springer, Singapore. https://doi.org/10.1007/978-981-19-5221-0_49
Download citation
DOI: https://doi.org/10.1007/978-981-19-5221-0_49
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-19-5220-3
Online ISBN: 978-981-19-5221-0
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)