Abstract
Psychological evidence suggests that the human ability to recognize facial expression improves with the addition of temporal stimuli. While the facial action coding community has largely migrated towards temporal information, the facial expression recognition community has been slow to utilize facial dynamics. This paper contrasts the contributions of static vs. temporal features, including both dense and sparse facial tracking methodologies in combination with sparse representation classification. The temporal methods of facial feature point tracking, motion history images, free form deformation, and SIFT flow are adapted for facial expression classification. Dense optical flow for facial expression recognition is successfully utilized. We show that when used in isolation, the best temporal methods are just as good as static methods. However, when fusing temporal dynamics with static imagery significant increases in facial expression classification are achieved.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Lew, M., Bakker, E.M., Sebe, N., Huang, T.S.: Human-Computer Intelligent Interaction: A Survey. In: Lew, M., Sebe, N., Huang, T.S., Bakker, E.M. (eds.) HCI 2007. LNCS, vol. 4796, pp. 1–5. Springer, Heidelberg (2007)
Pantic, M., Pentland, A., Nijholt, A., Huang, T.S.: Human computing and machine understanding of human behavior: a survey. In: ICMI 2006 and IJCAI 2007 International Workshops, Artifical Intelligence for Human Computing, Berlin, Germany (2007)
Pantic, M., Rothkrantz, L.U.M.: Automatic analysis of facial expressions: The state of the art. IEEE Transactions on Pattern Analysis and Machine Intelligence 22, 1424–1445 (2000)
Zhihong, Z., Pantic, M., Roisman, G.I., Huang, T.S.: A survey of affect recognition methods: audio, visual, and spontaneous expressions. IEEE Transactions on Pattern Analysis and Machine Intelligence 31, 39–58 (2009)
Shuai-Shi, L., Yan-Tao, T., Dong, L.: New research advances of facial expression recognition. In: Eighth International Conference on Machine Learning and Cybernetics (2009)
Martin, C., Werner, U., Gross, H.M.: A real-time facial expression recognition system based on active appearance models using gray images and edge images. In: 8th IEEE International Conference on Automatic Face & Gesture Recognition (2008)
Buciu, I., Kotropoulos, C., Pitas, I.: ICA and Gabor representation for facial expression recognition. In: Proceedings of International Conference on Image Processing (2003)
Shan, C., Gong, S., McOwan, P.W.: Facial expression recognition based on Local Binary Patterns: A comprehensive study. Image and Vision Computing 27, 803–816 (2009)
Zafeiriou, S., Petrou, M.: Sparse representations for facial expressions recognition via l1 optimization. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops, CVPRW 2010, San Francisco, CA (2010)
Ekman, P., Friesen, W.V.: The Facial Action Coding System. Consulting Psychologists Press, Inc., San Francisco (1978)
Koelstra, S., Pantic, M., Patras, I.: A dynamic texture-based approach to recognition of facial actions and their temporal models. IEEE Transactions on Pattern Analysis and Machine Intelligence 32, 1940–1954 (2010)
Curio, C., Bulthoff, H., Giese, M.: Dynamic Faces: Insights from Experiments and Computation, 1st edn. The MIT Press (2010)
Jiang, B., Valstar, M.F., Pantic, M.: Action Unit Detection Using Sparse Appearance Descriptors in Space-Time Video Volumes. In: Face and Gesture Recognition Face and Gesture Recognition, Santa Barbara, CA (2011)
Lucey, P., Cohn, J., Matthews, I., Prkachin, K., Solomon, P.: Painful Data: The UNBC-McMaster Shoulder Pain Expression Archive Database. In: Face and Gesture Recognition, Santa Barbara, CA (2011)
Bhaskaran, N., Nwogu, I., Frank, M., Govindaraju, V.: Lie to Me: Deceit Detection via Online Behavioral Learning. In: Face and Gesture Recognition, Santa Barbara, CA (2011)
Cohn, J.F., et al.: Detecting depression from facial actions and vocal prosody. In: 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops, ACII 2009 (2009)
Saragih, J., Lucey, S., Cohn, J.: Real-time Avatar Animation from a Single Image. In: Face and Gesture Recognition, Santa Barbara, CA (2011)
Bobick, A.F., Davis, J.W.: The recognition of human movement using temporal templates. IEEE Transactions on Pattern Analysis and Machine Intelligence 23, 257–267 (2001)
Ce, L., Yuen, J., Torralba, A.: SIFT flow: dense correspondence across scenes and its applications. IEEE Transactions on Pattern Analysis and Machine Intelligence 33, 978–994 (2011)
Songfan, Y., Bhanu, B.: Facial expression recognition using emotion avatar image. In: 2011 IEEE International Conference on Automatic Face & Gesture Recognition (2011)
Ptucha, R., Tsagkatakis, G., Savakis, A.: Manifold Based Sparse Representation for Robust Expression Recognition without Neutral Subtraction. In: Presented at the BeFIT 2011 Workshop, International Conference on Computer Vision, Barcelona, Spain (2011)
Olshausen, B.A., Field, D.J.: Sparse coding with an overcomplete basis set: a strategy employed by V1? Vision Research 37, 3311–3325 (1997)
Olshausen, B.A., Field, D.J.: Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381, 607–609 (1996)
Wright, J., Yang, A.Y., Ganesh, A., Sastry, S.S., Yi, M.: Robust face recognition via sparse representation. IEEE Transactions on Pattern Analysis and Machine Intelligence 31, 210–227 (2009)
He, X., Niyogi, P.: Locality Preserving Projections. In: Advances in Neural Information Processing Systems, Vancouver, Canada, vol. 16 (2003)
Cootes, T.F., Taylor, C.J., Cooper, D.H., Graham, J.: Active shape models - their training and application. Computer Vision and Image Understanding 61, 38–59 (1995)
Kotsia, I., Pitas, I.: Facial expression recognition in image sequences using geometric deformation features and support vector machines. IEEE Transactions on Image Processing 16, 172–187 (2007)
Rueckert, D., Sonoda, L.I., Hayes, C., Hill, D.L.G., Leach, M.O., Hawkes, D.J.: Nonrigid registration using free-form deformations: Application to breast MR images. IEEE Transactions on Medical Imaging 18, 712–721 (1999)
Valstar, M., Jiang, B., Mehu, M., Pantic, M., Scherer, K.R.: The First Facial Expression Recognition and Analysis Challenge. In: Face and Gesture Recognition, Santa Barbara, CA (2011)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Ptucha, R., Savakis, A. (2012). Towards the Usage of Optical Flow Temporal Features for Facial Expression Classification. In: Bebis, G., et al. Advances in Visual Computing. ISVC 2012. Lecture Notes in Computer Science, vol 7432. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-33191-6_38
Download citation
DOI: https://doi.org/10.1007/978-3-642-33191-6_38
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-33190-9
Online ISBN: 978-3-642-33191-6
eBook Packages: Computer ScienceComputer Science (R0)