Knowledge and Information Systems

, Volume 59, Issue 1, pp 219–250 | Cite as

RTCRelief-F: an effective clustering and ordering-based ensemble pruning algorithm for facial expression recognition

  • Danyang Li
  • Guihua WenEmail author
  • Zhi Hou
  • Eryang Huan
  • Yang Hu
  • Huihui Li
Regular Paper


Ensemble pruning is effective for improving the accuracy of expression recognition. This paper proposes a novel ensemble pruning algorithm called RTCRelief-F and applies it to facial expression recognition. RTCRelief-F uses a novel classifier-representation method that accounts for the interaction among classifiers and bases the classifier selection upon not only diversity but accuracy. Additionally, for the first time, RTCRelief-F, applies the Relief-F algorithm to evaluate the classifiers’ ability and resets the fusion order. Finally, the combination of a clustering-based ensemble pruning method and the ordering-based ensemble pruning method can both alleviate the dependence of a selected subset S on the adopted clustering strategies and guarantee the diversity of the selected subset S. The experimental results show that this method outperforms or competes with the original ensemble and some major state-of-the-art results on the data sets Fer2013, JAFFE, and CK\(+\).


Facial expression recognition Ensemble pruning Convolutional neural network Relative transformation Hierarchical clustering Relief-F 



This study was supported by a China National Science Foundation under Grants (60973083, 61273363), Science and Technology Planning Projects of Guangdong Province (2014A010103009, 2015A020217002), and Guangzhou Science and Technology Planning Project (201504291154480)


  1. 1.
    Cavalcanti GDC, Oliveira LS, Moura TJM et al (2016) Combining diversity measures for ensemble pruning. Pattern Recognit Lett 74:38–45CrossRefGoogle Scholar
  2. 2.
    Cai XF, Wen GH, Wei J et al (2014) Relative manifold based semi-supervised dimensionality reduction. Front Comput Sci 8(6):923–932MathSciNetCrossRefGoogle Scholar
  3. 3.
    Coates A, Lee H, Ng AY (2011) An analysis of single-layer networks in unsupervised feature learning. In: Gordon G, Dunson D (eds) 14th International conference on artificial intelligence and statistics, AISTATS 2011. Microtome Publishing, Ft. Lauderdale, FL, pp 215–223Google Scholar
  4. 4.
    Cruz RMO, Sabourin R, Cavalcanti GDC (2014) On meta-learning for dynamic ensemble selection. In: Borga M (ed) 2014 22nd International conference on pattern recognition (ICPR). Institute of Electrical and Electronics Engineers Inc, Stockholm, pp 1230–1235Google Scholar
  5. 5.
    Dai Q (2013) A novel ensemble pruning algorithm based on randomized greedy selective strategy and ballot. Neurocomputing 122:258–265CrossRefGoogle Scholar
  6. 6.
    Dai Q, Li ML (2015) Introducing randomness into greedy ensemble pruning algorithms. Appl Intell 42(3):406–429CrossRefGoogle Scholar
  7. 7.
    Dai Q, Han XM (2016) An efficient ordering-based ensemble pruning algorithm via dynamic programming. Appl Intell 44(4):816–830MathSciNetCrossRefGoogle Scholar
  8. 8.
    Dai Q, Ye R, Liu Z (2017) Considering diversity and accuracy simultaneously for ensemble pruning. Appl Soft Comput 58:75–91CrossRefGoogle Scholar
  9. 9.
    Demsar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7:1–30MathSciNetzbMATHGoogle Scholar
  10. 10.
    Fan XJ, Tjahjadi T (2017) A dynamic framework based on local Zernike moment and motion history image for facial expression recognition. Pattern Recogn 64:399–406CrossRefGoogle Scholar
  11. 11.
    Goodfellow LJ, Erhan D, Carrier PL et al (2015) Challenges in representation learning: a report on three machine learning contests. Neural Netw 64:59–63CrossRefGoogle Scholar
  12. 12.
    Guo HP, Sun F, Cheng J et al (2016) A novel margin-based measure for directed hill climbing ensemble pruning. Math Probl Eng 2016:1–11MathSciNetzbMATHGoogle Scholar
  13. 13.
    He KM, Zhang XY, Ren SQ et al (2015) Delving deep into rectifiers: surpassing human-level performance on imagenet classification. In: Bajcsy R, Hager G, Ma Y (eds) 2015 IEEE international conference on computer vision (ICCV). Institute of Electrical and Electronics Engineers Inc, Santiago, pp 1026–1034Google Scholar
  14. 14.
    Jia XB, Zhang YH, Powers D et al (2014) Multi-classifier fusion based facial expression recognition approach. KSII Trans Internet Inf Syst 8(1):196–212CrossRefGoogle Scholar
  15. 15.
    Kim BK, Roh J, Dong SY et al (2016) Hierarchical committee of deep convolutional neural networks for robust facial expression recognition. J Multimod User Interfaces 10(2):173–189CrossRefGoogle Scholar
  16. 16.
    Kira K, Rendell LA (1992) Feature selection problem: traditional methods and a new algorithm. In: Swartout W (ed) Tenth national conference on artificial intelligence, AAAI-92. Published by American Association for Artificial Intelligence, San Jose, CA, pp 129–134Google Scholar
  17. 17.
    Kononenko L (1994) Estimating attributes: analysis and extensions of RELIEF. In: Bergadano F, De Raedt L (eds) European conference on machine learning, ECML 1994. Springer, Catania, pp 171–182Google Scholar
  18. 18.
    Krawczyk B (2015) One-class classifier ensemble pruning and weighting with firefly algorithm. Neurocomputing 150:490–500CrossRefGoogle Scholar
  19. 19.
    Krizhevsky A (2009) Learning multiple layers of features from tiny images. University of TorontoGoogle Scholar
  20. 20.
    Kuncheva LI, Whitaker CJ (2003) Measures of diversity in classifier ensembles and their relationship with the ensemble accuracy. Mach Learn 51(2):181–207CrossRefzbMATHGoogle Scholar
  21. 21.
    Kuncheva LI (2013) A bound on kappa-error diagrams for analysis of classifier ensembles. IEEE Trans Knowl Data Eng 25(3):494–501CrossRefGoogle Scholar
  22. 22.
    Lecun Y, Bottou L, Bengio Y et al (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324CrossRefGoogle Scholar
  23. 23.
    Li N, Yu Y, Zhou ZH (2012) Diversity regularized ensemble pruning. In: Flach P, De Bie T, Cristianini N (eds) 2012 European conference on machine learning and principles and practice of knowledge discovery in databases, ECML-PKDD 2012. Springer, Bristol, pp 330–345Google Scholar
  24. 24.
    Liang D, Tsai CF, Dai AJ et al (2017) A novel classifier ensemble approach for financial distress prediction. Knowl Inf Syst 54:437–462CrossRefGoogle Scholar
  25. 25.
    Lin C, Chen WQ, Qiu C et al (2014) LibD3C: ensemble classifiers with a clustering and dynamic selection strategy. Neurocomputing 123:424–435CrossRefGoogle Scholar
  26. 26.
    Liu L, Wang BS, Zhong QX et al (2015) A selective ensemble method based on K-means method. In: Heilongjiang University (eds) Proceedings of 2015 4th international conference on computer science and network technology, ICCSNT 2015. Institute of Electrical and Electronics Engineers Inc, Harbin, pp 665–668Google Scholar
  27. 27.
    Lopes AT, de Aguiar E, De Souza AF et al (2017) Facial expression recognition with convolutional neural networks: coping with few data and the training sample order. Pattern Recogn 61:610–628CrossRefGoogle Scholar
  28. 28.
    Lu ZY, Wu XD, Zhu XQ, Bongard J (2010) Ensemble Pruning via Individual Contribution Ordering, In: Rao B, Krishnapuram B eds KDD ’10 Proceedings of the 16th ACM SIGKDD international conference on knowledge discovery and data mining. Association for Computing Machinery (ACM), Washington DC, pp 871–880Google Scholar
  29. 29.
    Lucey P, Cohn JF, Kanade T et al (2010) The extended Cohn–Kanade dataset (CK\(+\)): a complete dataset for action unit and emotion-specified expression. In: IEEE (eds) 2010 IEEE computer society conference on computer vision and pattern recognition workshops (CVPR workshops). IEEE Computer Society, San Francisco, CA, pp 94–101Google Scholar
  30. 30.
    Lyons M, Akamatsu S, Kamachi M et al (1998) Coding facial expressions with gabor wavelets. In: IEEE (eds) 3rd IEEE international conference on automatic face and gesture recognition. IEEE Computer Society, Nara, pp 200–205Google Scholar
  31. 31.
    Ma G, Li XX, Luo K (2005) Application of clustering in regional economy. In: Li Q, Liang TP (ed) 7th International conference on electronic commerce, ICEC05. Association for Computing Machinery (ACM), Xi’an, pp 48–51Google Scholar
  32. 32.
    Markatopoulou F, Tsoumakas G, Vlahavas L (2015) Dynamic ensemble pruning based on multi-label classification. Neurocomputing 150:501–512CrossRefGoogle Scholar
  33. 33.
    Martinez-Munoz G, Suarez A (2004) Aggregation ordering in bagging. In: Hamza MH (ed) Proceedings of the IASTED international conference on artificial intelligence and applications. Acta Press, Innsbruck, pp 258–263Google Scholar
  34. 34.
    Martinez-Munoz G, Hernandez-Lobato D, Suarez A (2009) An analysis of ensemble pruning techniques based on ordered aggregation. IEEE Trans Pattern Anal Mach Intell 31:245–259CrossRefGoogle Scholar
  35. 35.
    Matthew DZ, Rob F (2013) Stochastic pooling for regularization of deep convolutional neural network. arXiv: 1301.3557
  36. 36.
    Oleg O, Giorgio V (eds) (2009) Applications of supervised and unsupervised ensemble methods. Springer, Berlin, pp 4–5Google Scholar
  37. 37.
    Partalas L, Tsoumakas G, Vlahavas L (2010) An ensemble uncertainty aware measure for directed hill climbing ensemble pruning. Mach Learn 81(3):257–282MathSciNetCrossRefGoogle Scholar
  38. 38.
    Sun YX, Wen GH (2017) Cognitive facial expression recognition with constrained dimensionality reduction. Neurocomputing 230:397–408CrossRefGoogle Scholar
  39. 39.
    Treichler GD (1967) Are you missing the boat in training aids. Film Audio-Vis Commun 48(1):14–16Google Scholar
  40. 40.
    Wen GH, Jiang LJ, Wen J (2009) Local relative transformation with application to isometric embedding. Pattern Recognit Lett 30(3):203–211CrossRefGoogle Scholar
  41. 41.
    Wen GH, Tuo J, Jiang LJ et al (2012) Audio feature extraction for classification using relative transformation. In: IEEE (eds) 2012 3rd IEEE/IET international conference on audio, language and image processing, ICALIP 2012. IEEE Computer Society, Shanghai, pp 260–265Google Scholar
  42. 42.
    Wen GH, Li HH, Li DY (2015) An ensemble convolutional echo state networks for facial expression recognition. In: IEEE (eds) 2015 International conference on affective computing and intelligent interaction, ACII 2015. Institute of Electrical and Electronics Engineers Inc, Xi’an, pp 873–878Google Scholar
  43. 43.
    Ykhlef H, Bouchaffra D (2017) An efficient ensemble pruning approach based on simple coalitional games. Inf Fusion 2017:28–42CrossRefGoogle Scholar
  44. 44.
    Yu ZD, Zhang C (2015) Image based static facial expression recognition with multiple deep network learning. In: Zhang ZY, Cohen P (eds) Proceedings of the 2015 ACM international conference on multimodal interaction. Association for Computing Machinery, Inc, Xi’an, pp 435–442Google Scholar
  45. 45.
    Zavaschi THH, Koerich AL, Oliveira LES (2011) Facial expression recognition using ensemble of classifiers. In: Tichavsky P, Cernocky H, Prochazka A (eds) 36th IEEE international conference on acoustics, speech, and signal processing, ICASSP 2011. Institute of Electrical and Electronics Engineers Inc, Prague, pp 1489–1492Google Scholar
  46. 46.
    Zhang HX, Cao LL (2014) A spectral clustering based ensemble pruning approach. Neurocomputing 139:289–297CrossRefGoogle Scholar

Copyright information

© Springer-Verlag London Ltd., part of Springer Nature 2018

Authors and Affiliations

  • Danyang Li
    • 1
  • Guihua Wen
    • 1
    Email author
  • Zhi Hou
    • 1
  • Eryang Huan
    • 1
  • Yang Hu
    • 1
  • Huihui Li
    • 1
  1. 1.School of Computer Science and EngineeringSouth China University of TechnologyPanyu District, Guangzhou CityChina

Personalised recommendations