Abstract
Emotion being a subjective thing, leveraging knowledge and science behind labeled data and extracting the components that constitute it. With the development of deep learning in computer vision, emotion recognition has become a widely-tackled research problem. For mobility and privacy reasons, the required image processing should be local on embedded computer platforms with performance requirements and energy constraints. For this purpose, in this work, we propose a Field Programmable Gate Array (FPGA) architecture applied for this task using independent method called convolutional neural network (CNN [1]). The design flow is evaluated by implementing the previously trained CNN to recognize facial emotions from face image implemented in python on a PC. The project explains the process of porting the CNN algorithm from python to C/C++ and then executing it on a ZYNQ FPGA board. Once we have trained a network, weights from the Tensorflow model will be convert as C-arrays. After having the weights as C arrays, they can be implemented to FPGA system. This method was trained on the posed-emotion dataset (FER2013). The results show that with more fine-tuning and depth, the CNN model can outperform the state-of-the-art methods for emotion recognition. The bottleneck of the CNN [2] is the convolutional layers and that is why different solutions for that accelerator are analyzed and the performance of each solution is tested.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Brandon Rohrer. http://brohrer.github.io/how_convolutional_neural_networks_work.html
Ververidis, D., Kotropoulos, C.: Automatic speech classification to five emotional states based on gender information. In: Proceedings of the EUSIPCO2004 Conference, Austria, pp. 341–344 (2004)
Cowie, R., et al.: Emotion recognition in human computer interaction. IEEE Signal Process. Mag. 18(1), 32–80 (2001)
LeCun, Y., Bengio, Y., Hinton, G.: Deep Learning. Nature 521 (2015). https://doi.org/10.1038/nature14539
Glorot, X., Bengio, Y.: Understanding the difficulty of training deep feedforward neural networks. In: Proceedings of the 13th International Conference on Artificial Intelligence and Statistics, Sardinia, Italy, PMLR9, pp. 249–256 (2010)
Russakovsky, O., et al.: Imagenet large scale visual recognition challenge. Int. J. Comput. Vis., 211–252 (2012)
Girshick, R.: Fast R-CNN. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2015, Boston, MA, USA, pp. 1440–1448 (2015)
Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmenta. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2015. Boston, MA, USA, pp. 3431–3440 (2015)
Zhang, Y., et al.: Towards end-to-end speech recognition with deep convolutional neural networks. In: Proceedings of Interspeech Conference, San Francisco, USA (2016)
Goodfellow, I.J., et al.: Challenges in representation learning: a report on three machine learning contests. In: Lee, M., Hirose, A., Hou, Z.-G., Kil, R.M. (eds.) ICONIP 2013. LNCS, vol. 8228, pp. 117–124. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-42051-1_16
Martinez, B., Valstar, M.F.: Advances, challenges, and opportunities in automatic facial expression recognition. In: Kawulok, M., Celebi, M.E., Smolka, B. (eds.) Advances in Face Detection and Facial Image Analysis, pp. 63–100. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-25958-1_4
Britz, D.: http://www.wildml.com/2015/11/understanding-convolutional-neural-networks-for-nlp/
Anh Tran-The. http://labs.septeni-technology.jp/technote/ml-20-convolution-neural-network-part-3/
Phan-Van, H.: Neural Network Lecture, Danang University of Science and Technology (2013)
Ma, Y., et al.: Optimizing loop operation and dataflow in FPGA acceleration of deep convolutional neural networks. Arizona University (2017)
Horseman, A.: SVM for Facial Expression Recognition. A demonstrate project using SVM (2007). https://github.com/amineHorseman/facial-expression-recognition-svm/commit/aecd525367f6d77e5d274de7c6f0166d5bfa4bb9
Shima, A., Azar, F.: Convolutional neural networks for facial expression recognition. In: Proceedings of the Stanford University report (2016). http://cs231n.stanford.edu/reports/2016/pdfs/005_Report.pdf
Jain, A.: (2016). https://www.analyticsvidhya.com/blog/2016/04/deep-learning-computer-vision-introduction-convolution-neural-networks/
Szegedy, C., et al.: Going deeper with convolutions
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Le-Tien, T., Phan-Xuan, H., Nguyen-Tan, S. (2019). Light-Weight Deep Convolutional Network-Based Approach for Recognizing Emotion on FPGA Platform. In: Dang, T., Küng, J., Takizawa, M., Bui, S. (eds) Future Data and Security Engineering. FDSE 2019. Lecture Notes in Computer Science(), vol 11814. Springer, Cham. https://doi.org/10.1007/978-3-030-35653-8_19
Download citation
DOI: https://doi.org/10.1007/978-3-030-35653-8_19
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-35652-1
Online ISBN: 978-3-030-35653-8
eBook Packages: Computer ScienceComputer Science (R0)