Facial Action Unit Recognition Using Filtered Local Binary Pattern Features with Bootstrapped and Weighted ECOC Classifiers

Part of the Studies in Computational Intelligence book series (SCI, volume 373)


Within the context face expression classification using the facial action coding system (FACS), we address the problem of detecting facial action units (AUs). The method adopted is to train a single Error-Correcting Output Code (ECOC) multiclass classifier to estimate the probabilities that each one of several commonly occurring AU groups is present in the probe image. Platt scaling is used to calibrate the ECOC outputs to probabilities and appropriate sums of these probabilities are taken to obtain a separate probability for each AU individually. Feature extraction is performed by generating a large number of local binary pattern (LBP) features and then selecting from these using fast correlation-based filtering (FCBF). The bias and variance properties of the classifier are measured and we show that both these sources of error can be reduced by enhancing ECOC through the application of bootstrapping and class-separability weighting.


Local Binary Pattern Facial Expression Recognition Facial Action Facial Action Code System Automatic Face 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Ahonen, T., Hadid, A., Pietikainen, M.: Face description with local binary patterns: Application to face recognition. IEEE Trans. Pattern Analysis and Machine Intell. 28, 2037–2041 (2006)Google Scholar
  2. 2.
    Bartlett, M.S., Littlewort, G., Frank, M.G., Lainscsek, C., Fasel, I.R., Movellan, J.R.: Fully automatic facial action recognition in spontaneous behavior. In: Proc. 7th IEEE Int. Conf. Automatic Face and Gesture Recogn., Southampton, UK, pp. 223–230. IEEE Comp. Society, Los Alamitos (2006)Google Scholar
  3. 3.
    Das, S.: Filters, wrappers and a boosting-based hybrid for feature selection. In: Brodley, C.E., Pohoreckyj Danyluk, A. (eds.) Proc. 18th Int. Conf. Machine Learning, Williamstown, MA, pp. 74–81. Morgan Kaufmann, San Francisco (2001)Google Scholar
  4. 4.
    Dietterich, T.G., Bakiri, G.: Solving multiclass learning problems via error-correcting output codes. J. Artif. Intell. Research 2, 263–286 (1995)Google Scholar
  5. 5.
    Donato, G., Bartlett, M.S., Hager, J.C., Ekman, P., Sejnowski, T.J.: Classifying facial actions. IEEE Trans. Pattern Analysis and Machine Intell. 21, 974–989 (1999)Google Scholar
  6. 6.
    Duin, R.P.W., Juszczak, P., Paclik, P., Pekalska, E., de Ridder, D., Tax, D.M.J., Verzakov, S.: PRTools 4.1, A Matlab toolbox for pattern recognition. Delft University of Technology (2007)Google Scholar
  7. 7.
    Efron, B., Tibshirani, R.J.: An introduction to the bootstrap. Chapman & Hall/CRC Press, Boca Raton (1993)Google Scholar
  8. 8.
    Ekman, P., Friesen, W.V.: The facial action coding system: A technique for the measurement of facial movement. Consulting Psychologists Press, Palo Alto (1978)Google Scholar
  9. 9.
    Geman, S., Bienenstock, E.: Neural networks and the bias/variance dilemma. Neural Comp. 4, 1–58 (1992)Google Scholar
  10. 10.
    James, G.: Majority vote classifiers: Theory and applications. PhD Dissertation, Stanford University (1998)Google Scholar
  11. 11.
    James, G.: Variance and bias for general loss functions. Machine Learning 51, 115–135 (2003)Google Scholar
  12. 12.
    Kanade, T., Cohn, J.F., Tian, Y.: Comprehensive database for facial expression analysis. In: Proc. 4th Int. Conf. Automatic Face and Gesture Recognition, Grenoble, France, pp. 46–53. IEEE Comp. Society, Los Alamitos (2000)Google Scholar
  13. 13.
    Kohavi, R., Wolpert, D.: Bias plus variance decomposition for zero-one loss functions. In: Proc. 13th Int. Conf. on Machine Learning, Bari, Italy, pp. 275–283. Morgan Kaufmann, San Francisco (1996)Google Scholar
  14. 14.
    Ojala, T., Pietikäinen, M., Mäenpää, T.: Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans. Pattern Analysis and Machine Intell. 24, 971–987 (2002)Google Scholar
  15. 15.
    Platt, J.: Probabilistic outputs for support vector machines and comparison to regularized likelihood methods. In: Smola, A.J., Bartlett, P., Scholkopf, B., Schuurmans, D. (eds.) Advances in Large Margin Classifiers, pp. 61–74. MIT Press, Cambridge (1999)Google Scholar
  16. 16.
    Shan, C., Gong, S., McOwan, P.W.: Facial expression recognition based on local binary patterns: A comprehensive study. Image and Vision Comp. 27, 803–816 (2009)Google Scholar
  17. 17.
    Smith, R.S., Windeatt, T.: Class-separability weighting and bootstrapping in error correcting output code ensembles. In: El Gayar, N., Kittler, J., Roli, F. (eds.) MCS 2010. LNCS, vol. 5997, pp. 185–194. Springer, Heidelberg (2010)Google Scholar
  18. 18.
    Smith, R.S., Windeatt, T.: A Bias-variance analysis of bootstrapped class-separability weighting for error-correcting output code ensembles. In: Proc. 22nd IEEE Int. Conf. Pattern Recogn., Istanbul, Turkey, pp. 61–64. IEEE Comp. Society, Los Alamitos (2010)Google Scholar
  19. 19.
    Tian, Y.-I., Kanade, T., Cohn, J.F.: Recognizing action units for facial expression analysis. IEEE Trans. Pattern Analysis and Machine Intell. 23, 97–115 (2001)Google Scholar
  20. 20.
    Turk, M., Pentland, A.: Eigenfaces for recognition. J. Cognitive Neuroscience 3, 71–86 (1991)Google Scholar
  21. 21.
    Windeatt, T., Smith, R.S., Dias, K.: Weighted decoding ECOC for facial action unit classification. In: Okun, O., Valentini, G. (eds.) Proc. 2nd Supervised and Unsupervised Ensemble Methods and their Applications, Patras, Greece, pp. 26–30 (2008)Google Scholar
  22. 22.
    Witten, I.H., Frank, E.: Data mining: Practical machine learning tools and techniques. Morgan Kaufmann, San Francisco (2005)Google Scholar
  23. 23.
    Yu, L., Liu, H.: Feature selection for high-dimensional data: A fast correlation-based filter solution. In: Fawcett, T., Mishra, N. (eds.) Proc. 20th Int. Conf. Machine Learning, Washington DC, pp. 856–863. AAAI Press, Menlo Park (2003)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  1. 1.13AB05, Centre for Vision, Speech and Signal ProcessingUniversity of SurreySurreyUK
  2. 2.27AB05, Centre for Vision, Speech and Signal ProcessingUniversity of SurreySurreyUK

Personalised recommendations