Skip to main content
SpringerLink
Log in

Reliability of Results and Fairness in the Comparison of Rates Among 3D Facial Expression Recognition Works

  • Conference paper
  • First Online:
Computer Analysis of Images and Patterns (CAIP 2019)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 11678))

Included in the following conference series:

  • 1576 Accesses

Abstract

The capability of replicating experiments and comparing results is a basic premise for scientific progress. Thus, it is imperative that the conduction of validation experiments follow transparent methodological steps and be also reported in a clear way to allow accurate replication and fair comparison between results. In 3D facial expression recognition, the presented results are estimates of performance of a classification system and, therefore, have an intrinsic degree of uncertainty. Because of that, the reliability of a measure for evaluation is directly related to the concept of stability. In this work, we examine the experimental setup reported by a set of 3D facial expression recognition studies published from 2013 to 2018. This investigation revealed that the concern with stability of mean recognition rates is present in only a small portion of studies. In addition, it demonstrates that the highest rates in this domain are also, potentially, the most unstable. Those findings lead to a reflection on the fairness of comparisons in this domain.

This study was financed in part by the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior - Brasil (CAPES) - Finance Code 001.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 64.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 84.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://www.scopus.com/search/form.uri?display=basic.

  2. 2.

    https://www.webofknowledge.com/.

  3. 3.

    For the complete list of those studies, please see the supplementary material available in https://goo.gl/JUP1cM.

References

  1. Azazi, A., Lebai Lutfi, S., Venkat, I., Fernández-Martínez, F.: Towards a robust affect recognition: automatic facial expression recognition in 3D faces. Expert Syst. Appl. 42(6), 3056–3066 (2015). https://doi.org/10.1016/j.eswa.2014.10.042

    Article  Google Scholar 

  2. Azazi, A., Lutfi, S.L., Venkat, I.: Analysis and evaluation of SURF descriptors for automatic 3D facial expression recognition using different classifiers. In: 2014 4th World Congress on Information and Communication Technologies, WICT 2014, pp. 23–28. IEEE, December 2014. https://doi.org/10.1109/WICT.2014.7077296

  3. Gong, B., Wang, Y., Liu, J., Tang, X.: Automatic facial expression recognition on a single 3D face by exploring shape deformation. In: ACM International Conference on Multimedia, pp. 569–572 (2009)

    Google Scholar 

  4. Bousquet, O., Elisseeff, A.: Stability and generalization. J. Mach. Learn. Res. 2(3), 499–526 (2002). https://doi.org/10.1162/153244302760200704

    Article  MathSciNet  MATH  Google Scholar 

  5. Duda, R.O., Hart, P.E., Stork, D.G.: Algorithm-independent machine learning. In: Pattern Classification, pp. 453–516. Wiley, New York (2012). Chap. 9

    Google Scholar 

  6. Jan, A., Meng, H.: Automatic 3D facial expression recognition using geometric and textured feature fusion. In: 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), vol. 05, pp. 1–6. IEEE, May 2015. https://doi.org/10.1109/FG.2015.7284860

  7. Lemaire, P., Ardabilian, M., Chen, L., Daoudi, M.: Fully automatic 3D facial expression recognition using differential mean curvature maps and histograms of oriented gradients. In: 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG). pp. 1–7. IEEE, April 2013. https://doi.org/10.1109/FG.2013.6553821. http://ieeexplore.ieee.org/document/6553821/

  8. Li, H., Sun, J., Xu, Z., Chen, L.: Multimodal 2D+3D facial expression recognition with deep fusion convolutional neural network. IEEE Trans. Multimed. 19(12), 2816–2831 (2017). https://doi.org/10.1109/TMM.2017.2713408

    Article  Google Scholar 

  9. Li, H., et al.: An efficient multimodal 2D + 3D feature-based approach to automatic facial expression recognition. Comput. Vis. Image Underst. 140(C), 83–92 (2015). https://doi.org/10.1016/j.cviu.2015.07.005

    Article  Google Scholar 

  10. Li, Q., An, G., Ruan, Q.: 3D Facial expression recognition using orthogonal tensor marginal fisher analysis on geometric maps. In: 2017 International Conference on Wavelet Analysis and Pattern Recognition (ICWAPR), vol. 35, pp. 65–71. IEEE, July 2017. https://doi.org/10.1109/ICWAPR.2017.8076665

  11. Li, X., Ruan, Q., An, G.: 3D facial expression recognition using delta faces. In: 5th IET International Conference on Wireless, Mobile and Multimedia Networks (ICWMMN 2013), pp. 234–239. Institution of Engineering and Technology (2013). https://doi.org/10.1049/cp.2013.2415

  12. Li, X., Ruan, Q., An, G., Jin, Y., Zhao, R.: Multiple strategies to enhance automatic 3D facial expression recognition. Neurocomputing 161(C), 89–98 (2015). https://doi.org/10.1016/j.neucom.2015.02.063

    Article  Google Scholar 

  13. Savran, A., et al.: Bosphorus database for 3D face analysis. In: Schouten, B., Juul, N.C., Drygajlo, A., Tistarelli, M. (eds.) BioID 2008. LNCS, vol. 5372, pp. 47–56. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-89991-4_6

    Chapter  Google Scholar 

  14. Yang, X., Huang, D., Wang, Y., Chen, L.: Automatic 3D facial expression recognition using geometric scattering representation. In: 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), pp. 1–6. IEEE, May 2015. https://doi.org/10.1109/FG.2015.7163090

  15. Yin, L., Wei, X., Sun, Y., Wang, J., Rosato, M.J.: A 3D facial expression database for facial behavior research. In: FGR 2006: Proceedings of the 7th International Conference on Automatic Face and Gesture Recognition 2006, pp. 211–216 (2006). https://doi.org/10.1109/FGR.2006.6

  16. Zeng, W., Li, H., Chen, L., Morvan, J.M., Gu, X.D.: An automatic 3D expression recognition framework based on sparse representation of conformal images. In: 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), pp. 1–8. IEEE, April 2013. https://doi.org/10.1109/FG.2013.6553749

  17. Zhen, Q., Huang, D., Wang, Y., Chen, L.: Muscular movement model-based automatic 3D/4D facial expression recognition. IEEE Trans. Multimed. 18(7), 1438–1450 (2016). https://doi.org/10.1109/TMM.2016.2557063

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Departamento de Engenharia de Teleinformática, Centro de Tecnologia, Campus do Pici, Universidade Federal do Ceará, Fortaleza, CE, 60455-970, Brazil

    Gilderlane Ribeiro Alexandre, George André Pereira Thé & José Marques Soares

Authors
  1. Gilderlane Ribeiro Alexandre
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. George André Pereira Thé
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. José Marques Soares
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gilderlane Ribeiro Alexandre .

Editor information

Editors and Affiliations

  1. Department of Computer and Electrical Engineering and Applied Mathematics, University of Salerno, Fisciano (SA), Italy

    Mario Vento

  2. Department of Computer and Electrical Engineering and Applied Mathematics, University of Salerno, Fisciano (SA), Italy

    Gennaro Percannella

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Alexandre, G.R., Thé, G.A.P., Soares, J.M. (2019). Reliability of Results and Fairness in the Comparison of Rates Among 3D Facial Expression Recognition Works. In: Vento, M., Percannella, G. (eds) Computer Analysis of Images and Patterns. CAIP 2019. Lecture Notes in Computer Science(), vol 11678. Springer, Cham. https://doi.org/10.1007/978-3-030-29888-3_31

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-29888-3_31

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-29887-6

  • Online ISBN: 978-3-030-29888-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation