Abstract
Facial expression recognition is a popular research topic to a wide range of applications in human-computer interaction, social robotics, and affective computing. Various attempts have been made to improve the techniques and accuracies of FER. However, one of the main challenges still persists – how to generalise across different datasets, deal with small data and reduce classifiers bias. In this paper, we explore the application of bio-inspired Spiking Neural Networks (SNN) with unsupervised learning using spike timing dependent plasticity (STDP) for FER. We have evaluated our approach on two publicly available, third-party, facial expression datasets. The results have shown that our approach has achieved consistently high accuracies (92%) in cross-dataset evaluation and exhibited a significant improvement compared with the state-of-the-art CNN and HOG feature extraction techniques. The results suggest that our approach can learn more effective feature representations, which lead to good generalisation across subjects in different ethnic groups with different facial dimensions and characteristics.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Carcagnì, P., Coco, M.D., Leo, M., Distante, C.: Facial expression recognition and histograms of oriented gradients: a comprehensive study. SpringerPlus 4(1), 645 (2015)
Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection, vol. 5, pp. 886–893 (2005)
Diehl, P., Cook, M.: Unsupervised learning of digit recognition using spike-timing-dependent plasticity. Front. Comput. Neurosci. 9, 99 (2015)
Filip, P., Andrzej, K.: Introduction to spiking neural networks: information processing. Learn. Appl. 71, 409–433 (2011)
Gilani, S.Z., Mian, A., Shafait, F., Reid, I.: Dense 3D face correspondence. IEEE Trans. Pattern Anal. Mach. Intell. 40(7), 1584–1598 (2017)
Goodman, D., Brette, R.: Brain: a simulator for spiking neural networks in python. Front. Neuroinformatics 2, 5 (2008)
Hodgkin, A.L., Huxley, A.F.: A quantitative description of membrane current and its application to conduction and excitation in nerve. Bull. Math. Biol. 52, 25–71 (1990)
Jose, J.T., Amudha, J., Sanjay, G.: A survey on spiking neural networks in image processing, pp. 107–115 (2015)
Kheradpisheh, S., Ganjtabesh, M., Thorpe, S., Masquelier, T.: Stdp-based spiking deep convolutional neural networks for object recognition. Neural Networks pp. 56–67 (2017)
Khorrami, P., Paine, T.L., Huang, T.S.: Do deep neural networks learn facial action units when doing expression recognition? CoRR 2015 (2015)
Kim, B.K., Dong, S.Y., Roh, J., Kim, G., Lee, S.Y.: Fusing aligned and non-aligned face information for automatic affect recognition in the wild: a deep learning approach, pp. 1499–1508, June 2016
Liu, Y., et al.: Facial expression recognition with PCA and LBP features extracting from active facial patches, pp. 368–373, June 2016
Lopes, A.T., de Aguiar, E., Souza, A.D., Oliveira-Santos, T.: Facial expression recognition with convolutional neural networks: coping with few data and the training sample order. Pattern Recogn. 61, 610–628 (2017)
Lucey, P., Cohn, J.F., Kanade, T., Saragih, J., Ambadar, Z., Matthews, I.: The extended Cohn-kanade dataset (ck+): a complete dataset for action unit and emotion-specified expression, pp. 94–101, June 2010
Lyons, M.J., Budynek, J., Akamatsu, S.: Automatic classification of single facial images. IEEE Trans. Pattern Anal. Mach. Intell. 21, 162–8828 (1999)
Maass, W.: Networks of spiking neurons: the third generation of neural network models. Neural Netw. 10, 1659–1671 (1997)
Majumder, A., Behera, L., Subramanian, V.K.: Automatic facial expression recognition system using deep network-based data fusion. IEEE Trans. Cybern. 99, 1–12 (2016)
Mansouri-Benssassi, E.: A decentralised multimodal integration of social signals: a bio-inspired approach. In: Proceedings of the 19th ACM International Conference on Multimodal Interaction, ICMI, pp. 633–637 (2017)
Marr, D., Hildreth, E.: Theory of edge detection. Proc. Royal Soc. London Ser. B 23, 187–217 (1980)
Mishra, B., et al.: Facial expression recognition using feature based techniques and model based techniques: a survey, pp. 589–594 (2015)
Mollahosseini, A., Chan, D., Mahoor, M.H.: Going deeper in facial expression recognition using deep neural networks, pp. 1–10, March 2016
Tie, Y., Guan, L.: A deformable 3-D facial expression model for dynamic human emotional state recognition. IEEE Trans. Circ. Syst. Video Technol. 23, 142–157 (2013)
van der Walt, S., et al.: scikit-image: image processing in python. PeerJ 2, e453 (2014)
Acknowledgement
We gratefully acknowledge the support of NVIDIA Corporation with the donation of the Quadro M5000 GPU used for this research.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Mansouri-Benssassi, E., Ye, J. (2018). Bio-Inspired Spiking Neural Networks for Facial Expression Recognition: Generalisation Investigation. In: Fagan, D., Martín-Vide, C., O'Neill, M., Vega-Rodríguez, M.A. (eds) Theory and Practice of Natural Computing. TPNC 2018. Lecture Notes in Computer Science(), vol 11324. Springer, Cham. https://doi.org/10.1007/978-3-030-04070-3_33
Download citation
DOI: https://doi.org/10.1007/978-3-030-04070-3_33
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-04069-7
Online ISBN: 978-3-030-04070-3
eBook Packages: Computer ScienceComputer Science (R0)