Ensemble Approaches to Facial Action Unit Classification

  • Terry Windeatt
  • Kaushala Dias
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5197)


Facial action unit (au) classification is an approach to face expression recognition that decouples the recognition of expression from individual actions. In this paper, upper face aus are classified using an ensemble of MLP (Multi-layer perceptron) base classifiers with feature ranking based on PCA components. This approach is compared experimentally with other popular feature-ranking methods applied to Gabor features. Experimental results on Cohn-Kanade database demonstrate that the MLP ensemble is relatively insensitive to the feature-ranking method but optimized PCA features achieve lowest error rate. When posed as a multi-class problem using Error-Correcting-Output-Coding (ECOC), error rates are comparable to two-class problems (one-versus-rest) when the number of features and base classifier are optimized.


Ensembles ECOC FACS Feature-ranking 


  1. 1.
    Tian, Y., Kanade, T., Cohn, J.F.: Recognising action units for facial expression analysis. IEEE Trans. PAMI 23(2), 97–115 (2001)CrossRefGoogle Scholar
  2. 2.
    Donato, G., Bartlett, M.S., Hager, J.C., Ekman, P., Sejnowski, T.J.: Classifying facial actions. IEEE Trans. PAMI 21(10), 974–989 (1999)CrossRefGoogle Scholar
  3. 3.
    Windeatt, T., Dias, K.: Feature-ranking ensembles for facial action unit classification. In: IAPR Third Int. Workshop on artificial neural networks in pattern recognition, Paris (accepted, 2008)Google Scholar
  4. 4.
    Windeatt, T., Prior, M.: Stopping Criteria for Ensemble-based Feature Selection. In: Haindl, M., Kittler, J., Roli, F. (eds.) MCS 2007. LNCS, vol. 4472, pp. 271–281. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  5. 5.
    Windeatt, T.: Accuracy/Diversity and Ensemble Classifier Design. IEEE Trans. Neural Networks 17(5), 287–297 (2006)CrossRefGoogle Scholar
  6. 6.
    Dietterich, T.G., Bakiri, G.: Solving multiclass learning problems via error-correcting output codes. J. Artificial Intelligence Research 2, 263–286 (1995)zbMATHGoogle Scholar
  7. 7.
    Windeatt, T., Ghaderi, R.: Coding and Decoding Strategies for multiclass learning problems. Information Fusion 4(1), 11–21 (2003)CrossRefGoogle Scholar
  8. 8.
    Freund, Y., Schapire, R.E.: A decision-theoretic generalisation of on-line learning and an application to boosting. J. of Computer and System Science 55(1), 119–139 (1997)CrossRefzbMATHGoogle Scholar
  9. 9.
    Skuruchina, M., Duin, R.P.W.: Combining feature subsets in feature selection. In: Oza, N.C., Polikar, R., Kittler, J., Roli, F. (eds.) MCS 2005. LNCS, vol. 3541, pp. 165–175. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  10. 10.
    Oza, N., Tumer, K.: Input Decimation ensembles: decorrelation through dimensionality reduction. In: Kittler, J., Roli, F. (eds.) MCS 2001. LNCS, vol. 2096, pp. 238–247. Springer, Heidelberg (2001)CrossRefGoogle Scholar
  11. 11.
    Bryll, R., Gutierrez-Osuna, R., Quek, F.: Attribute bagging: improving accuracy of classifier ensembles by using random feature subsets. Pattern Recognition 36, 1291–1302 (2003)CrossRefzbMATHGoogle Scholar
  12. 12.
    Hsu, C., Huang, H., Schuschel, D.: The ANNIGMA-wrapper approach to fast feature selection for neural nets. IEEE Trans. System, Man and Cybernetics-Part B:Cybernetics 32(2), 207–212 (2002)CrossRefGoogle Scholar
  13. 13.
    Efron, N., Intrator, N.: The effect of noisy bootstrapping on the robustness of supervised classification of gene expression data. In: IEEE Int. Workshop on Machine Learning for Signal Processing, Brazil, pp. 411–420 (2004)Google Scholar
  14. 14.
    Windeatt, T., Prior, M., Effron, N., Intrator, N.: Ensemble-based Feature Selection Criteria. In: Perner, P. (ed.) MLDM 2007. LNCS (LNAI), vol. 4571, pp. 168–182. Springer, Heidelberg (2007)Google Scholar
  15. 15.
    Bartlett, M.S., Littlewort, G., Lainscsek, C., Fasel, I.: J. Machine learning methods for fully automatic recognition of facial expressions and facial actions. In: IEEE Conf. System, Man and Cybernetics, vol. 1, pp. 592–597 (2004)Google Scholar
  16. 16.
    Silapachote, P., Karuppiah, D.R., Hanson, A.R.: Feature Selection using Adaboost for Face Expression Recognition. In: Proc. Conf. on Visualisation, Imaging and Image Processing, Marbella, Spain, pp. 84–89 (2004)Google Scholar
  17. 17.
    Heijden, F., Duin, R.P.W., Ridder, D., Tax, D.M.J.: Classification, Parameter Estimation and State Estimation. Wiley, Chichester (2004)CrossRefzbMATHGoogle Scholar
  18. 18.
    Guyon, I., Weston, J., Barnhill, S., Vapnik, V.: Gene selection for cancer classification using support vector machines. Machine Learning 46(1-3), 389–422 (2002)CrossRefzbMATHGoogle Scholar
  19. 19.
    Yu, L., Liu, H.: Efficient feature selection via analysis of relevance and redundancy. Journal of Machine Learning Research 5, 1205–1224 (2004)MathSciNetzbMATHGoogle Scholar
  20. 20.
    Kanade, T., Cohn, J.F., Tian, Y.: Comprehensive Database for facial expression analysis. In: Proc. 4th Int. Conf. automatic face and gesture recognition, Grenoble, France, pp. 46–53 (2000)Google Scholar
  21. 21.
    Bartlett, M.S., Littlewort, G., Frank, M., Lainscsek, C., Fasel, I., Movellan, J.: Fully automatic facial action recognition in spontaneous behavior. In: Proc 7th Conf. On Automatic Face and Gesture Recognition, pp. 223–238 (2006) ISBN 0-7695-2503-2Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Terry Windeatt
    • 1
  • Kaushala Dias
    • 1
  1. 1.Centre for Vision, Speech and Signal Proc (CVSSP)University of SurreyGuildfordUnited Kingdom

Personalised recommendations