A Hidden Markov Model Based Approach for Facial Expression Recognition in Image Sequences
One of the important properties of hidden Markov models is the ability to model sequential dependencies. In this study the applicability of hidden Markov models for emotion recognition in image sequences is investigated, i.e. the temporal aspects of facial expressions. The underlying image sequences were taken from the Cohn-Kanade database. Three different features (principal component analysis, orientation histograms and optical flow estimation) from four facial regions of interest (face, mouth, right and left eye) were extracted. The resulting twelve paired combinations of feature and region were used to evaluate hidden Markov models. The best single model with features of principal component analysis in the region face achieved a detection rate of 76.4 %. To improve these results further, two different fusion approaches were evaluated. Thus, the best fusion detection rate in this study was 86.1 %.
Unable to display preview. Download preview PDF.
- 3.Bishop, C.M.: Pattern Recognition and Machine Learning. Springer, New York (2007)Google Scholar
- 4.Cohen, I., Garg, A., Huang, T.S.: Emotion recognition from facial expressions using multilevel HMM. In: Neural Information Processing Systems (2000)Google Scholar
- 5.Cohn, J.F., Kanade, T., Tian, Y.: Comprehensive database for facial expression analysis. In: Proceedings of the 4th IEEE International Conference on Automatic Face and Gesture Recognition, pp. 46–53 (2000)Google Scholar
- 6.Darwin, C.: The Expression of the Emotions in Man and Animals, 1st edn. Oxford University Press Inc., New York (1872)Google Scholar
- 7.Lin, D.T., Chen, J.: Facial expressions classification with hierarchical radial basis function networks. In: Proceedings of the 6th International Conference on Neural Information Processing, ICONIP, pp. 1202–1207 (1999)Google Scholar
- 9.Ekman, P., Friesen, W.V.: Facial Action Coding System: A Technique for the Measurement of Facial Movement. Consulting Psychologists Press, Palo Alto (1978)Google Scholar
- 10.Freeman, W.T., Roth, M.: Orientation histograms for hand gesture recognition. In: International Workshop on Automatic Face and Gesture Recognition, pp. 296–301 (1994)Google Scholar
- 13.Lien, J.J.J., Kanade, T., Cohn, J., Li, C.: A multi-method approach for discriminating between similar facial expressions, including expression intensity estimation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR 1998) (June 1998)Google Scholar
- 14.Lisetti, C.L., Rumelhart, D.E.: Facial expression recognition using a neural network. In: Proceedings of the Eleventh International Florida Artificial Intelligence Research Society Conference, pp. 328–332. AAAI Press, Menlo Park (1998)Google Scholar
- 15.Rabiner, L., Juang, B.H.: Fundamentals of Speech Recognition. Prentice Hall PTR, Englewood Cliffs (1993)Google Scholar
- 16.Rabiner, L.R.: A tutorial on hidden markov models and selected applications in speech recognition. In: Proceedings of the IEEE, pp. 257–286 (1989)Google Scholar
- 20.Yeasin, M., Bullot, B., Sharma, R.: From facial expression to level of interest: A spatio-temporal approach. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 922–927 (2004)Google Scholar