Facial Expression Recognition Based on the Two-Dimensional Structure of Affect

  • Young-suk Shin
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3138)


We present an expression recognition system based on the two-dimensional structure of affect. The system is capable of identifying the various emotions using automated feature extraction. A method for extracting information about facial expressions from images is presented in three steps. In the first step, Gabor wavelet representation is constructed to provide edge extraction of major face components using the average value of the image’s 2-D Gabor wavelet coefficient histogram. In the second step, sparse features of facial expression image are extracted using fuzzy C-means clustering(FCM) algorithm on neutral faces. In the third step, features of facial expressions are extracted using the Dynamic Linking Model(DLM) on expression images. The result of facial expression recognition is compared with dimensional values of internal states derived from semantic ratings of words related to emotion by experimental subjects. The two-dimensional structure of affect recognizes not only six facial expressions related to six basic emotions (happiness, sadness, surprise, angry, fear, disgust), but also expressions of various internal states.


Facial Expression Emotion Word Expression Recognition Basic Emotion Facial Expression Recognition 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Ekman, P.: Universal and cultural difference in facial expressions of emotions. In: Cole, J.K. (ed.) Nebraska symposium on motivation, pp. 207–283. University of Nebraska Press, Lincoln (1972)Google Scholar
  2. 2.
    Ekman, P., Friesen, W.V.: Facial action coding system. Consulting Psychologists, Palo Alto, CA (1977)Google Scholar
  3. 3.
    Mase, K.: Recognition of facial expression from optical flow. IEICE Transactions, E 74 10, 3473–3483 (1991)Google Scholar
  4. 4.
    Essa, I., Pentland, A.: Coding, analysis, interpretation, and recognition of facial expressions. IEEE Transactions on Pattern Analysis and Machine Intelligence 19, 757–763 (1997)CrossRefGoogle Scholar
  5. 5.
    Lien, J.: Automatic recognition of facial expressions using hidden Markov models and estimation of expression intensity. Ph.D. Thesis, Carnegie Mellon University (1998)Google Scholar
  6. 6.
    Oliver, N., Pentland, A., Berard, F.: LAFTER:a real-time face and lips tracker with facial expression recognition. Pattern Recognition 33, 1369–1382 (2000)CrossRefGoogle Scholar
  7. 7.
    Cohen, I., Sebe, N., Garg, A., Chen, L.S., Huang, T.S.: Facial expression recognition from video sequence:temporal and static modeling. Computer Vision and Image Understanding (2003) (in press)Google Scholar
  8. 8.
    Yacoob, Y., Davis, L.S.: Recognizing human facial expression from long image sequences using optical flow. IEEE Trans. Pattern Anal. Machine Intell. 18(6), 636–642 (1996)CrossRefGoogle Scholar
  9. 9.
    Bartlett, M., Viola, P., Sejnowski, T., Larsen, J., Hager, J., Ekman, P.: Classfying Facial Action. In: Touretzky, D., et al. (eds.) Advances in Neural Information Processing Systems 8, MIT Press, Cambridge (1996)Google Scholar
  10. 10.
    Padgett, C., Cottrell, G.: Identifying emotion in static face images. In: Proceeding of the 2nd Joint Symposium on Neural Computation, vol. 5, pp. 91–101 (1995)Google Scholar
  11. 11.
    Essa, I., Pentland, A.: Facial Expression Recognition using Visually Extracted Facial Action Parameters. In: Proceedings of the International Workshop on Automatic Face and Gesture Recognition, pp. 35–40 (1995)Google Scholar
  12. 12.
    Lyons, M., Akamatsu, S.: Coding facial expressions with Gabor wavelets. In: Proceeding of the Third International Conference on Automatic Face and Gesture Recognition, pp. 200–205 (1998)Google Scholar
  13. 13.
    von der Malsburg, C.: Nervous structure with dynamical links. Ber. Bunsenges. Phy.Chem 89, 703–710 (1985)Google Scholar
  14. 14.
    Bahn, S., Hahn, J., Chung, C.: Facial expression database for mapping facial expression onto internal state. In: 1997 Emotion Conference of Korea, pp. 215–219 (1997)Google Scholar
  15. 15.
    Kim, Y., Kim, J., O.S., O.K., Chung, C.: The study of dimension of internal states through word analysis about emotion. Korean Journal of the Science of Emotion and Sensibility 1, 145–152 (1998)Google Scholar
  16. 16.
    Daugman, J.: Uncertainty relation for resolution in space, spatial frequency, and orientation optimized by two-dimensional visual cortical filters. Journal of the Optical Society of America 2, 1160–1169 (1985)CrossRefGoogle Scholar
  17. 17.
    Bezdek, J.C.: Fuzzy mathematics in pattern classification. Ph.D. thesis, Applied Math. Center, Cornell University, Ithaca (1973)Google Scholar
  18. 18.
    Karayiannis, N.B., Pai, P.-I.: Fuzzy vector quantization algorithms and their application in image compression. IEEE Transactions on Image Processing (1995)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2004

Authors and Affiliations

  • Young-suk Shin
    • 1
  1. 1.Department of Information and telecommunication EngineeringChosun UniversityGwangjuKorea

Personalised recommendations