Abstract
Micro-Expression Recognition has been a challenging task as transitory micro-expressions only appear in a few of facial areas. In this work, we aim to improve the recognition performance by boosting the micro-variations in the learned areas. This paper proposes an architecture, deriving facial micro-variation heat areas and then integrating in conjunction with the micro-expression recognition network, to learn the micro-expression features in an end-to-end manner. The method is constructed by the Heat Areas Estimator from Heatmap (HAEH), which is to produce micro-variation heat areas as facial geometric structure, and the Temporal Facial Micro-Variation Network (TFMVN) for learning the fusion features. The method can define and capture facial heat areas significantly contributed to the micro-expressions. Our approach activates or deactivates corresponding feature maps from the heat areas to guide feature learning. We perform experiments on CASME II dataset and SAMM dataset. The experimental results show that we achieve state-of-the-art accuracy, and the method demonstrates good generalization ability for cross-dataset. Moreover, we validate three pivotal components’ effectiveness within our architecture.
This work is supported by the National Natural Science Foundation of China (No. 61672276, No. 51975294).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
A. Bulat and G. Tzimiropoulos. Binarized convolutional landmark localizers for human pose estimation and face alignment with limited resources. In: IEEE International Conference on Computer Vision, ICCV 2017, Venice, Italy, 22–29 October 2017, pp. 3726–3734 (2017)
Bulat, A., Tzimiropoulos, G.: How far are we from solving the 2D & 3D face alignment problem? (and a dataset of 230, 000 3d facial landmarks). In: IEEE International Conference on Computer Vision, ICCV 2017, Venice, Italy, 22–29 October 2017, pp. 1021–1030 (2017)
Davison, A.K., Lansley, C., Costen, N., Tan, K., Yap, M.H.: SAMM: a spontaneous micro-facial movement dataset. IEEE Trans. Affective Comput. 9(1), 116–129 (2018)
Ghiasi, G., Fowlkes, C.C.: Occlusion coherence: localizing occluded faces with a hierarchical deformable part model. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2014, Columbus, OH, USA, 23–28 June 2014, pp. 1899–1906 (2014)
Hassani, B., Mahoor, M.H.: Spatio-temporal facial expression recognition using convolutional neural networks and conditional random fields. In: 12th IEEE International Conference on Automatic Face & Gesture Recognition, FG 2017, Washington, DC, USA, May 30 - June 3, 2017, pp. 790–795 (2017)
Huan, Z., Shang, L.: Model the dynamic evolution of facial expression from image sequences. In: Advances in Knowledge Discovery and Data Mining - 22nd Pacific-Asia Conference, PAKDD 2018, Melbourne, VIC, Australia, 3–6 June 2018, Proceedings, Part II, pp. 546–557 (2018)
Jung, H., Lee, S., Yim, J., Park, S., Kim, J.: Joint fine-tuning in deep neural networks for facial expression recognition. In: 2015 IEEE International Conference on Computer Vision, ICCV 2015, Santiago, Chile, 7–13 December 2015, pp. 2983–2991 (2015)
Khor, H., See, J., Phan, R.C., Lin, W.: Enriched long-term recurrent convolutional network for facial micro-expression recognition. In: 13th IEEE International Conference on Automatic Face & Gesture Recognition, FG 2018, Xi’an, China, 15–19 May 2018, pp. 667–674 (2018)
Kim, D.H., Baddar, W.J., Ro, Y.M.: Micro-expression recognition with expression-state constrained spatio-temporal feature representations. In: Proceedings of the 2016 ACM Conference on Multimedia Conference, MM 2016, Amsterdam, The Netherlands, 15–19 October 2016, pp. 382–386 (2016)
Li, X., et al.: Towards reading hidden emotions: a comparative study of spontaneous micro-expression spotting and recognition methods. IEEE Trans. Affective Comput. 9(4), 563–577 (2018)
Liong, S., See, J., Wong, K., Phan, R.C.: Less is more: micro-expression recognition from video using apex frame. Sig. Proc. Image Comm. 62, 82–92 (2018)
J. Lv, X. Shao, J. Xing, C. Cheng, and X. Zhou. A deep regression architecture with two-stage re-initialization for high performance facial landmark detection. In 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, 21–26 July 2017, pp. 3691–3700 (2017)
Newell, A., Yang, K., Deng, J.: Stacked hourglass networks for human pose estimation. In: Computer Vision - ECCV 2016–14th European Conference, Amsterdam, The Netherlands, October 11–14 (2016), Proceedings, Part VIII, pp. 483–499 (2016)
Ngo, A.C.L., See, J., Phan, R.C.: Sparsity in dynamics of spontaneous subtle emotions: analysis and application. IEEE Trans. Affective Comput. 8(3), 396–411 (2017)
Park, S.Y., Lee, S., Ro, Y.M.: Subtle facial expression recognition using adaptive magnification of discriminative facial motion. In: Proceedings of the 23rd Annual ACM Conference on Multimedia Conference, MM 2015, Brisbane, Australia, 26–30 October 2015, pp. 911–914 (2015)
Peng, M., Wang, C., Chen, T., Liu, G., Fu, X.: Dual temporal scale convolutional neural network for micro-expression recognition. Front. Psychol. 8, 1745 (2017)
Sagonas, C., Tzimiropoulos, G., Zafeiriou, S., Pantic, M.: 300 faces in-the-wild challenge: the first facial landmark localization challenge. In: 2013 IEEE International Conference on Computer Vision Workshops, ICCV Workshops 2013, Sydney, Australia, December 1–8, 2013, pp. 397–403 (2013)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. CoRR, abs/1409.1556 (2014)
Verma, A., Qassim, H., Feinzimer, D.: Residual squeeze CNDS deep learning CNN model for very large scale places image recognition. In: 8th IEEE Annual Ubiquitous Computing, Electronics and Mobile Communication Conference, UEMCON 2017, New York City, NY, USA, 19–21 October 2017, pp. 463–469 (2017)
Wang, Y., See, J., Phan, R.C., Oh, Y.: LBP with six intersection points: reducing redundant information in LBP-TOP for micro-expression recognition. In: Computer Vision - ACCV 2014–12th Asian Conference on Computer Vision, Singapore, Singapore, 1–5 November 2014, Revised Selected Papers, Part I, pp. 525–537 (2014)
Xu, F., Zhang, J., Wang, J.Z.: Microexpression identification and categorization using a facial dynamics map. IEEE Trans. Affective Comput. 8(2), 254–267 (2017)
Yan, W.J., et al.: Casme ii: an improved spontaneous micro-expression database and the baseline evaluation. Plos One 9(1), e86041 (2014)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Zhang, M., Huan, Z., Shang, L. (2020). Micro-Expression Recognition Using Micro-Variation Boosted Heat Areas. In: Peng, Y., et al. Pattern Recognition and Computer Vision. PRCV 2020. Lecture Notes in Computer Science(), vol 12306. Springer, Cham. https://doi.org/10.1007/978-3-030-60639-8_44
Download citation
DOI: https://doi.org/10.1007/978-3-030-60639-8_44
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-60638-1
Online ISBN: 978-3-030-60639-8
eBook Packages: Computer ScienceComputer Science (R0)