Evaluation of Smile Detection Methods with Images in Real-World Scenarios

  • Zhoucong Cui
  • Shuo Zhang
  • Jiani Hu
  • Weihong Deng
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9010)


Discriminative methods such as SVM, have been validated extremely efficient in pattern recognition issues. We present a systematic study on smile detection with different SVM classifiers. We experimented with linear SVM classifier, RBF kernel SVM classifier and a recently-proposed local linear SVM (LL-SVM) classifier. In this paper, we focus on smile detection in face images captured in real-world scenarios, such as those in GENKI4K database. In the meantime, illumination normalization, alignment and feature representation methods are also taken into consideration. Compared with the commonly used pixel-based representation, we find that local-feature-based methods achieve not only higher detection performance but also better robustness against misalignment. Almost all the illumination normalization methods have no effect on the detection accuracy. Among all the SVM classifiers, the novel LL-SVM is verified to find a balance between accuracy and efficiency. And among all the features including pixel value intensity, Gabor, LBP and HOG features, we find that HOG features are the most appropriate features to detect smiling faces, which, combined with RBF kernel SVM, achieve an accuracy of \(93.25\,\%\) on GENKI4K database.



This work was partially sponsored by National Natural Science Foundation of China (NSFC) under Grant No. 61375031, No. 61471048, and No. 61273217. This work was also supported by the Fundamental Research Funds for the Central Universities, Beijing Higher Education Young Elite Teacher Project, and the Program for New Century Excellent Talents in University.


  1. 1.
    Pantic, M., Rothkrantz, L.J.: Facial action recognition for facial expression analysis from static face images. IEEE Trans. Syst. Man Cybern. Part B Cybern. 34, 1449–1461 (2004)CrossRefGoogle Scholar
  2. 2.
    Tian, Y.L., Kanade, T., Cohn, J.F.: Recognizing action units for facial expression analysis. IEEE Trans. Pattern Anal. Mach. Intell. 23, 97–115 (2001)CrossRefGoogle Scholar
  3. 3.
    Kapoor, A., Qi, Y., Picard, R.W.: Fully automatic upper facial action recognition. In: IEEE International Workshop on Analysis and Modeling of Faces and Gestures, 2003, AMFG 2003, pp. 195–202. IEEE (2003)Google Scholar
  4. 4.
    Wen, Z., Huang, T.S.: Capturing subtle facial motions in 3D face tracking. In: Proceedings of the Ninth IEEE International Conference on Computer Vision, 2003, pp. 1343–1350. IEEE (2003)Google Scholar
  5. 5.
    Sebe, N., Sun, Y., Bakker, E., Lew, M.S., Cohen, I., Huang, T.S.: Towards authentic emotion recognition. In: IEEE International Conference on Systems, Man and Cybernetics, 2004, vol. 1, pp. 623–628. IEEE (2004)Google Scholar
  6. 6.
    Cohn, J.F., Schmidt, K.L.: The timing of facial motion in posed and spontaneous smiles. Int. J. Wavelets Multiresolut. Inf. Process. 2, 121–132 (2004)CrossRefGoogle Scholar
  7. 7.
    Cohen, I., Sebe, N., Garg, A., Chen, L.S., Huang, T.S.: Facial expression recognition from video sequences: temporal and static modeling. Comput. Vis. Image Underst. 91, 160–187 (2003)CrossRefGoogle Scholar
  8. 8.
    Zhang, Y., Ji, Q.: Active and dynamic information fusion for facial expression understanding from image sequences. IEEE Trans. Pattern Anal. Mach. Intell. 27, 699–714 (2005)CrossRefGoogle Scholar
  9. 9.
    Shinohara, Y., Otsu, N.: Facial expression recognition using fisher weight maps. In: Proceedings of the Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004, pp. 499–504. IEEE (2004)Google Scholar
  10. 10.
    Deniz, O., Castrillon, M., Lorenzo, J., Anton, L., Bueno, G.: Smile detection for user interfaces. In: Bebis, G., et al. (eds.) ISVC 2008, Part II. LNCS, vol. 5359, pp. 602–611. Springer, Heidelberg (2008) CrossRefGoogle Scholar
  11. 11.
    Deng, W., Hu, J., Guo, J.: Extended SRC: undersampled face recognition via intraclass variant dictionary. IEEE Trans. Pattern Anal. Mach. Intell. 34, 1864–1870 (2012)CrossRefGoogle Scholar
  12. 12.
    Deng, W., Hu, J., Lu, J., Guo, J.: Transform-invariant PCA: a unified approach to fully automatic face alignment, representation, and recognition. IEEE Trans. Pattern Anal. Mach. Intell. 36, 1275–1284 (2014)CrossRefGoogle Scholar
  13. 13.
    Deng, W., Hu, J., Guo, J.: Linear ranking analysis. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE (2014)Google Scholar
  14. 14.
    Ojala, T., Pietikainen, M., Maenpaa, T.: Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans. Pattern Anal. Mach. Intell. 24, 971–987 (2002)CrossRefGoogle Scholar
  15. 15.
    Ahonen, T., Hadid, A., Pietikäinen, M.: Face recognition with local binary patterns. In: Pajdla, T., Matas, J.G. (eds.) ECCV 2004. LNCS, vol. 3021, pp. 469–481. Springer, Heidelberg (2004) CrossRefGoogle Scholar
  16. 16.
    Deng, W., Hu, J., Guo, J., Cai, W., Feng, D.: Emulating biological strategies for uncontrolled face recognition. Pattern Recogn. 43, 2210–2223 (2010)CrossRefzbMATHGoogle Scholar
  17. 17.
    Deng, W., Hu, J., Guo, J., Cai, W., Feng, D.: Robust, accurate and efficient face recognition from a single training image: a uniform pursuit approach. Pattern Recogn. 43, 1748–1762 (2010)CrossRefzbMATHGoogle Scholar
  18. 18.
    Zhang, Z., Ladicky, L., Torr, P., Saffari, A.: Learning anchor planes for classification. In: Advances in Neural Information Processing Systems, pp. 1611–1619 (2011)Google Scholar
  19. 19.
    Kanade, T., Cohn, J.F., Tian, Y.: Comprehensive database for facial expression analysis. In: Proceedings of the Fourth IEEE International Conference on Automatic Face and Gesture Recognition, 2000, pp. 46–53. IEEE (2000)Google Scholar
  20. 20.
    Sim, T., Baker, S., Bsat, M.: The CMU pose, illumination, and expression database. IEEE Trans. Pattern Anal. Mach. Intell. 25, 1615–1618 (2003)CrossRefGoogle Scholar
  21. 21.
    Wang, H., Li, S.Z., Wang, Y.: Generalized quotient image. In: Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004, CVPR 2004, vol. 2, p. II-498. IEEE (2004)Google Scholar
  22. 22.
    Chen, W., Er, M.J., Wu, S.: Illumination compensation and normalization for robust face recognition using discrete cosine transform in logarithm domain. IEEE Trans. Syst. Man Cybern. B Cybern. 36, 458–466 (2006)CrossRefGoogle Scholar
  23. 23.
    Baluja, S., Rowley, H.A.: Boosting sex identification performance. Int. J. Comput. Vision 71, 111–119 (2007)CrossRefGoogle Scholar
  24. 24.
    Shan, C.: Smile detection by boosting pixel differences. IEEE Trans. Image Process. 21, 431–436 (2012)CrossRefMathSciNetGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Zhoucong Cui
    • 1
  • Shuo Zhang
    • 1
  • Jiani Hu
    • 1
  • Weihong Deng
    • 1
  1. 1.Beijing University of Posts and TelecommunicationsBeijingChina

Personalised recommendations