Micro-Expression Recognition Using Robust Principal Component Analysis and Local Spatiotemporal Directional Features

  • Su-Jing WangEmail author
  • Wen-Jing Yan
  • Guoying Zhao
  • Xiaolan Fu
  • Chun-Guang Zhou
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8925)


One of important cues of deception detection is micro-expression. It has three characteristics: short duration, low intensity and usually local movements. These characteristics imply that micro-expression is sparse. In this paper, we use the sparse part of Robust PCA (RPCA) to extract the subtle motion information of micro-expression. The local texture features of the information are extracted by Local Spatiotemporal Directional Features (LSTD). In order to extract more effective local features, 16 Regions of Interest (ROIs) are assigned based on the Facial Action Coding System (FACS). The experimental results on two micro-expression databases show the proposed method gain better performance. Moreover, the proposed method may further be used to extract other subtle motion information (such as lip-reading, the human pulse, and micro-gesture etc.) from video.


Micro-expression recognition Sparse representation Dynamic features Local binary pattern Subtle motion extraction 


  1. 1.
    Cootes, T.F., Taylor, C.J., Cooper, D.H., Graham, J.: Active shape models-their training and application. Computer vision and image understanding 61(1), 38–59 (1995)CrossRefGoogle Scholar
  2. 2.
    Dao, M., Suo, Y., Chin, S., Tran, T.: Video frame interpolation via weighted robust principal component analysis. In: 2013 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1404–1408. IEEE (2013)Google Scholar
  3. 3.
    Eckart, C., Young, G.: The approximation of one matrix by another of lower rank. Psychometrika 1(3), 211–218 (1936)CrossRefzbMATHGoogle Scholar
  4. 4.
    Ekman, P.: Microexpression training tool (METT). University of California, San Francisco (2002)Google Scholar
  5. 5.
    Ekman, P.: Lie catching and microexpressions. The philosophy of deception pp. 118–133 (2009)Google Scholar
  6. 6.
    Ekman, P., Friesen, W.: Nonverbal leakage and clues to deception. Tech. rep, DTIC Document (1969)Google Scholar
  7. 7.
    Ekman, P., Friesen, W.V.: Facial action coding system: A technique for the measurement of facial movement, vol. 12. Consulting Psychologists Press, CA (1978)Google Scholar
  8. 8.
    Georgieva, P., De la Torre, F.: Robust principal component analysis for brain imaging. In: Mladenov, V., Koprinkova-Hristova, P., Palm, G., Villa, A.E.P., Appollini, B., Kasabov, N. (eds.) ICANN 2013. LNCS, vol. 8131, pp. 288–295. Springer, Heidelberg (2013) CrossRefGoogle Scholar
  9. 9.
    Li, X., Pfister, T., Huang, X., Zhao, G., Pietikäinen, M.: A spontaneous micro-expression database: inducement, collection and baseline. In: IEEE International Conference on Automatic Face and Gesture Recognition and Workshops (2013)Google Scholar
  10. 10.
    Lin, Z., Liu, R., Su, Z.: Linearized alternating direction method with adaptive penalty for low-rank representation. In: Neural Information Processing Systems (NIPS) (2011)Google Scholar
  11. 11.
    Matsumoto, D., Hwang, H.: Evidence for training the ability to read microexpressions of emotion. Motivation and Emotion 35(2), 181–191 (2011)CrossRefGoogle Scholar
  12. 12.
    Michael, N., Dilsizian, M., Metaxas, D., Burgoon, J.K.: Motion profiles for deception detection using visual cues. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part VI. LNCS, vol. 6316, pp. 462–475. Springer, Heidelberg (2010) CrossRefGoogle Scholar
  13. 13.
    Ojala, T., Pietikainen, M., Maenpaa, T.: Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Transactions on Pattern Analysis and Machine Intelligence 24(7), 971–987 (2002)CrossRefzbMATHGoogle Scholar
  14. 14.
    Ojansivu, V., Heikkilä, J.: Blur Insensitive Texture Classification Using Local Phase Quantization. In: Elmoataz, A., Lezoray, O., Nouboud, F., Mammass, D. (eds.) ICISP 2008 2008. LNCS, vol. 5099, pp. 236–243. Springer, Heidelberg (2008) CrossRefGoogle Scholar
  15. 15.
    Pfister, T., Li, X., Zhao, G., Pietikainen, M.: Recognising spontaneous facial micro-expressions. In: 12th IEEE International Conference on Computer Vision. pp. 1449–1456. IEEE (2011)Google Scholar
  16. 16.
    Polikovsky, S., Kameda, Y., Ohta, Y.: Facial micro-expressions recognition using high speed camera and 3D-gradient descriptor. In: 3rd International Conference on Crime Detection and Prevention. pp. 1–6. IET (2009)Google Scholar
  17. 17.
    Shi, L.C., Duan, R.N., Lu, B.L.: A robust principal component analysis algorithm for eeg-based vigilance estimation. In: 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 6623–6626. IEEE (2013)Google Scholar
  18. 18.
    Wang, L., Cheng, H.: Robust principal component analysis for sparse face recognition. In: 2013 Fourth International Conference on Intelligent Control and Information Processing (ICICIP), pp. 171–176. IEEE (2013)Google Scholar
  19. 19.
    Wang, S.J., Chen, H.L., Yan, W.J., Chen, Y.H., Fu, X.: Face recognition and micro-expression based on discriminant tensor subspace analysis plus extreme learning machine. Neural Processing Letters 39(1), 25–43 (2014)CrossRefGoogle Scholar
  20. 20.
    Wright, J., Ganesh, A., Rao, S., Peng, Y., Ma, Y.: Robust principal component analysis: exact recovery of corrupted low-rank matrices via convex optimization. In: Advances in neural information processing systems, pp. 2080–2088 (2009)Google Scholar
  21. 21.
    Yan, W.J., Li, X., Wang, S.J., Zhao, G., Liu, Y.J., Chen, Y.H., Fu, X.: CASME II: An improved spontaneous micro-expression database and the baseline evaluation. PLoS ONE 9(1), e86041 (2014)CrossRefGoogle Scholar
  22. 22.
    Yan, W.J., Wu, Q., Liang, J., Chen, Y.H., Fu, X.: How fast are the leaked facial expressions: The duration of micro-expressions. Journal of Nonverbal Behavior, pp. 1–14 (2013)Google Scholar
  23. 23.
    Zhao, G., Pietikainen, M.: Dynamic texture recognition using local binary patterns with an application to facial expressions. IEEE Transactions on Pattern Analysis and Machine Intelligence 29(6), 915–928 (2007)CrossRefGoogle Scholar
  24. 24.
    Zhao, G., Pietikäinen, M.: Visual speaker identification with spatiotemporal directional features. In: Kamel, M., Campilho, A. (eds.) ICIAR 2013. LNCS, vol. 7950, pp. 1–10. Springer, Heidelberg (2013) CrossRefGoogle Scholar
  25. 25.
    Zhou, Z., Zhao, G., Pietikainen, M.: Towards a practical lipreading system. In: 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 137–144. IEEE (2011)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Su-Jing Wang
    • 1
    • 4
    Email author
  • Wen-Jing Yan
    • 1
    • 2
  • Guoying Zhao
    • 3
  • Xiaolan Fu
    • 1
  • Chun-Guang Zhou
    • 4
  1. 1.State Key Lab of Brain and Cognitive Science, Institute of PsychologyChinese Academy of SciencesBeijingChina
  2. 2.College of Teacher EducationWenzhou UniversityWenzhouChina
  3. 3.Center for Machine Vision ResearchUniversity of OuluOuluFinland
  4. 4.College of Computer Science and TechnologyJilin UniversityChangchunChina

Personalised recommendations