Classification of Facial Expressions Using K-Nearest Neighbor Classifier

  • Abu Sayeed Md. Sohail
  • Prabir Bhattacharya
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4418)


In this paper, we have presented a fully automatic technique for detection and classification of the six basic facial expressions from nearly frontal face images. Facial expressions are communicated by subtle changes in one or more discrete features such as tightening the lips, raising the eyebrows, opening and closing of eyes or certain combinations of them. These discrete features can be identified through monitoring the changes in muscles movement (Action Units) located near about the regions of mouth, eyes and eyebrows. In this work, we have used eleven feature points that represent and identify the principle muscle actions as well as provide measurements of the discrete features responsible for each of the six basic human emotions. A multi-detector approach of facial feature point localization has been utilized for identifying these points of interests from the contours of facial components such as eyes, eyebrows and mouth. Feature vector composed of eleven features is then obtained by calculating the degree of displacement of these eleven feature points from a non-changeable rigid point. Finally, the obtained feature sets are used for training a K-Nearest Neighbor Classifier so that it can classify facial expressions when given to it in the form of a feature set. The developed Automatic Facial Expression Classifier has been tested on a publicly available facial expression database and on an average 90.76% successful classification rate has been achieved.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Donato, G., et al.: Classifying Facial Actions. IEEE Trans. Pattern Analysis and Machine Intelligence 21(10), 974–989 (1999)CrossRefGoogle Scholar
  2. 2.
    Mehrabian, A.: Communication without Words. Psychology Today 2(4), 53–56 (1968)Google Scholar
  3. 3.
    Van Dam, A.: Beyond WIMP. IEEE Computer Graphics and Applications 20(1), 50–51 (2000)CrossRefGoogle Scholar
  4. 4.
    Pentland, A.: Looking at People: Sensing for Ubiquitous and Wearable Computing. IEEE Trans. Pattern Analysis and Machine Intelligence 22(1), 107–119 (2005)CrossRefGoogle Scholar
  5. 5.
    Jain, A.K., Duin, R.P.W., Mao, J.: Statistical Pattern Recognition: A Review. IEEE Trans. Pattern Analysis and Machine Intelligence 22(1), 4–37 (2000)CrossRefGoogle Scholar
  6. 6.
    Chibelushi, C.C., Bourel, F.: Facial Expression Recognition: A Brief Tutorial Overview. Available Online at:”
  7. 7.
    Ekman, P., Friesen, W.V.: Unmasking the Face. Prentice Hall, Englewood Cliffs (1975)Google Scholar
  8. 8.
    Cottrell, G.W., Metcalfe, J.: EMPATH: Face, Emotion, Gender Recognition Using Holons. In: Advances in Neural Information Processing Systems, vol. 3, pp. 564–571 (1991)Google Scholar
  9. 9.
    Rahardja, A., Sowmya, A., Wilson, W.H.: A Neural Network Approach to Component Versus Holistic Recognition of Facial Expression in Images. Intelligent Robots and Computer Vision X: Algorithms and Techniques 1607, 62–70 (1991)Google Scholar
  10. 10.
    Kobayashi, H., Hara, F.: Recognition of Mixed Facial Expressions and Their Strength by a Neural Network. In: IEEE International Conference on Acoustic, Speech and Signal Processing, pp. 1495–1498. IEEE Computer Society Press, Los Alamitos (1992)Google Scholar
  11. 11.
    Vanger, P., Honlinger, R., Haken, H.: Applications of Synergetics in Decoding Facial Expression of Emotion. In: IEEE International Conference on Automatic Face and Gesture Recognition, pp. 24–29. IEEE Computer Society Press, Los Alamitos (1995)Google Scholar
  12. 12.
    Essa, I.A., Pentland, A.P.: Coding, Analysis, Interpretation and Recognition of Facial Expressions. IEEE Trans. Pattern Analysis and Machine Intelligence 19(7), 757–763 (1997)CrossRefGoogle Scholar
  13. 13.
    Lien, J.J., et al.: Automated Facial Expression Recognition Based on FACS Action Units. In: Third IEEE International Conference on Automatic Face and Gesture Recognition, pp. 390–395. IEEE Computer Society Press, Los Alamitos (1998)CrossRefGoogle Scholar
  14. 14.
    Pantic, M., Rothkrantz, L.J.M.: An Expert System for Multiple Emotional Classification of Facial Expressions. In: 11th IEEE International Conference on Tools with Artificial Intelligence, pp. 113–120. IEEE Computer Society Press, Los Alamitos (1999)Google Scholar
  15. 15.
    Feng, X., Pietikainen, M., Hadid, A.: Facial Expression Recognition with Local Binary Patterns and Linear Programming. IEEE Trans. Pattern Recognition and Image Analysis 15(2), 546–548 (2005)Google Scholar
  16. 16.
    Sohail, A.S.M., Bhattacharya, P.: Localization of Facial Feature Regions Using Anthropometric Face Model. In: First International Conference on Multidisciplinary Information Sciences and Technologies (2006)Google Scholar
  17. 17.
    Fasel, I., Fortenberry, B., Movellan, J.R.: A Generative Framework for Real Time Object Detection and Classification. Computer Vision and Image Understanding 98, 182–210 (2005)CrossRefGoogle Scholar
  18. 18.
    Efford, N.: Digital Image Processing: A Practical Introduction Using Java. Addison-Wesley, Essex (2000)Google Scholar
  19. 19.
    Ritter, G.X., Wilson, J.N.: Handbook of Computer Vision Algorithms in Image Algebra. CRC Press, Boca Raton (1996)MATHGoogle Scholar
  20. 20.
    Otsu, N.: A Threshold Selection Method from Gray Level Histograms. IEEE Trans. Systems, Man, and Cybernetics 9(1), 62–66 (1979)MathSciNetCrossRefGoogle Scholar
  21. 21.
    Dasarathy, B.V. (ed.): Nearest Neighbor: Pattern Classification Techniques (Nn Norms: Nn Pattern Classification Techniques). IEEE Computer Society Press, Los Alamitos (1991)Google Scholar
  22. 22.
    Wettschereck, D., Aha, D., Mohri, T.: A Review and Empirical Evaluation of Feature-weighting Methods for a Class of Lazy Learning Algorithms. Artificial Intelligence Review 11, 273–314 (1997)CrossRefGoogle Scholar
  23. 23.
    Dudani, S.A.: The Distance-weighted K-Nearest-Neighbor Rule. IEEE Transaction on Systems, Man and Cybernetics 6, 325–327 (1976)Google Scholar
  24. 24.
    Lyons, J., et al.: Coding Facial Expressions with Gabor Wavelets. In: Third IEEE International Conference on Automatic Face and GestureRecognition, pp. 200–205. IEEE Computer Society Press, Los Alamitos (1998)CrossRefGoogle Scholar

Copyright information

© Springer Berlin Heidelberg 2007

Authors and Affiliations

  • Abu Sayeed Md. Sohail
    • 1
  • Prabir Bhattacharya
    • 2
  1. 1.Department of Computer Science and Software Engineering, Concordia University, 1455 de Maisonneuve Blvd. West, Montreal, Quebec H3G 1M8Canada
  2. 2.Concordia Institute for Information Systems Engineering, Concordia University, 1515 St. Catherine West, Montreal, Quebec H3G 2W1Canada

Personalised recommendations