Evaluating real-life performance of the state-of-the-art in facial expression recognition using a novel YouTube-based datasets

  • Muhammad Hameed Siddiqi
  • Maqbool Ali
  • Mohamed Elsayed Abdelrahman Eldib
  • Asfandyar Khan
  • Oresti Banos
  • Adil Mehmood Khan
  • Sungyoung Lee
  • Hyunseung Choo
Article

DOI: 10.1007/s11042-016-4321-2

Cite this article as:
Siddiqi, M.H., Ali, M., Abdelrahman Eldib, M.E. et al. Multimed Tools Appl (2017). doi:10.1007/s11042-016-4321-2
  • 50 Downloads

Abstract

Facial expression recognition (FER) is one of the most active areas of research in computer science, due to its importance in a large number of application domains. Over the years, a great number of FER systems have been implemented, each surpassing the other in terms of classification accuracy. However, one major weakness found in the previous studies is that they have all used standard datasets for their evaluations and comparisons. Though this serves well given the needs of a fair comparison with existing systems, it is argued that this does not go in hand with the fact that these systems are built with a hope of eventually being used in the real-world. It is because these datasets assume a predefined camera setup, consist of mostly posed expressions collected in a controlled setting, using fixed background and static ambient settings, and having low variations in the face size and camera angles, which is not the case in a dynamic real-world. The contributions of this work are two-fold: firstly, using numerous online resources and also our own setup, we have collected a rich FER dataset keeping in mind the above mentioned problems. Secondly, we have chosen eleven state-of-the-art FER systems, implemented them and performed a rigorous evaluation of these systems using our dataset. The results confirm our hypothesis that even the most accurate existing FER systems are not ready to face the challenges of a dynamic real-world. We hope that our dataset would become a benchmark to assess the real-life performance of future FER systems.

Keywords

Facial expressions Classification YouTube Real-life scenarios 

Copyright information

© Springer Science+Business Media New York 2017

Authors and Affiliations

  • Muhammad Hameed Siddiqi
    • 1
  • Maqbool Ali
    • 2
  • Mohamed Elsayed Abdelrahman Eldib
    • 3
  • Asfandyar Khan
    • 4
  • Oresti Banos
    • 5
  • Adil Mehmood Khan
    • 6
  • Sungyoung Lee
    • 2
  • Hyunseung Choo
    • 1
  1. 1.Department of Computer Science and EngineeringSungkyunkwan UniversitySuwonKorea
  2. 2.Department of Computer EngineeringKyung Hee UniversitySuwonKorea
  3. 3.Department of Biomedical EngineeringKyung Hee UniversitySuwonKorea
  4. 4.Department of Computer ScienceUniversity of Science & TechnologyBannuPakistan
  5. 5.Center for Telematics and Information TechnologyUniversity of TwenteEnschedeNetherlands
  6. 6.Department of Computer ScienceInnopolis UniversityKazanRussia