Skip to main content
SpringerLink
Log in

Fully Automated Facial Expression Recognition Using 3D Morphable Model and Mesh-Local Binary Pattern

  • Conference paper
  • First Online:
Advanced Concepts for Intelligent Vision Systems (ACIVS 2017)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 10617))

Abstract

With recent advances in artificial intelligence and pattern recognition, automatic facial expression recognition draws a great deal of interest. In this area, most of works involved 2D imagery. However, they present some challenges related to pose, illumination variation and self-occlusion. To deal with these problems, we propose to reconstruct the face in 3D space, from only one 2D image, using the 3D Morphable Model (3DMM). Thus, thanks to its robustness against pose and illumination variations, 3DMM offers high-resolution model and fast fitting functionality. Then, given the reconstructed 3D face, we extract a set of features, which are effective to describe shape changes and expression-related facial appearance, using Mesh-Local Binary Pattern (mesh-LBP). Obtained results proved the effectiveness of combining 3DMM and mesh-LBP for automatic facial expression recognition from 2D single image. In fact, to evaluate the proposed method against state-of-the-art methods, a comparative study shows that the method outperforms existing ones.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Han, H., Otto, C., Liu, X., Jain, A.K.: Demographic estimation from face images: human vs. machine performance. IEEE Trans. Pattern Anal. Mach. Intell. 37(6), 1148–1161 (2015)

    Article  Google Scholar 

  2. Phillips, P.J., Toole, A.J.O.: Comparison of human and computer performance across face recognition experiments. Image Vis. Comput. 32, 74–85 (2014)

    Article  Google Scholar 

  3. Koelstra, S., Pantic, M., Patras, I.: A dynamic texture-based method to recognition of facial actions and their temporal models. IEEE Trans. Pattern Anal. Mach. Intell. 32(11), 1940–1954 (2010)

    Article  Google Scholar 

  4. Ekman, P.: Universals and cultural differences in facial expressions of emotion. Nebr. Symp. Motiv. 207–283 (1972)

    Google Scholar 

  5. Bull, P.: Communication Under the Microscope: The Theory and Practice of Microanalysis. Routledge, Abingdon (2002)

    Book  Google Scholar 

  6. Zhen, H.D., Wang, Y., Chen, L.: Muscular movement model-based automatic 3D/4D facial expression recognition: face segmentation, shape representation, and feature fusion. IEEE Trans. Multimedia 18(7), 1438–1450 (2016)

    Article  Google Scholar 

  7. Xue, M., Mian, A., Liu, W., Li, L.: Fully automatic 3D facial expression recognition using local depth features. In: IEEE Winter Conference Applications of Computer Vision (WACV) (2014)

    Google Scholar 

  8. Li, X.L., Ruan, Q.Q., Jin, Y., An, G., Zhao, R.: Fully automatic 3D facial expression recognition using polytypic multi-block local binary patterns. Sig. Process. 108, 297–308 (2015)

    Article  Google Scholar 

  9. Jan, A., Meng, H.: Automatic 3D facial expression recognition using geometric and textured feature fusion. In: 11th IEEE, Automatic Face and Gesture Recognition (FG) (2015)

    Google Scholar 

  10. Huber, P., Hu, G., Tena, R., Mortazavian, P., Koppen, W.P., Christmas, W., Rätsch, M., Kittler, J.: A multi resolution 3D morphable face model and fitting framework. In: International Conference on Computer Vision Theory and Applications (VISAPP) (2016)

    Google Scholar 

  11. King, D.E.: Dlib-ml: a machine learning toolkit. J. Mach. Learn. Res. 10, 1755–1758 (2009)

    Google Scholar 

  12. Kazemi, V., Sullivan, J.: One millisecond face alignment with an ensemble of regression trees. computer vision and pattern recognition (CVPR). In: IEEE Conference: pp. 1867–1874 (2014)

    Google Scholar 

  13. Viola, P., Jones, M.: Robust real-time face detection. Int. J. Comput. Vis. 57(2), 137–154 (2004)

    Article  Google Scholar 

  14. Werghi, N., Tortorici, C., Berretti, S., Del Bimbo, A.: Boosting 3D LBP-based face recognition by fusing shape and texture descriptors on the mesh. IEEE Trans. Inf. Forensics Secur. 11, 964–979 (2016)

    Article  Google Scholar 

  15. Ojala, T., Pietikäinen, M., Harwood, D.: A comparative study of texture measures with classification based on featured distributions. Pattern Recogn. 29, 51–59 (1996)

    Article  Google Scholar 

  16. Savran, A., Alyüz, N., Dibeklioğlu, H., Çeliktutan, O., Gökberk, B., Sankur, B., Akarun, L.: Bosphorus database for 3D face analysis. In: Schouten, B., Juul, N.C., Drygajlo, A., Tistarelli, M. (eds.) BioID 2008. LNCS, vol. 5372, pp. 47–56. Springer, Heidelberg (2008). doi:10.1007/978-3-540-89991-4_6

    Chapter  Google Scholar 

  17. Yin, L., Wei, X., Sun, Y., Wang, J., Rosato, M.J.: A 3D facial expression database for facial behavior research. In: Conference on Automatic Face and Gesture Recognition (FGR 2006), pp. 211–216 (2006)

    Google Scholar 

  18. Hewahi, N.M., AbdulRahman, M.B.: Emotion Recognition model based on facial expressions, ethnicity and gender using backpropagation neural network. Int. J. Technol. Diffus. 3, 33–43 (2012)

    Article  Google Scholar 

  19. Yin, L., Chen, X., Sun, Y., Worm, T., Reale, M.: A high-resolution 3D dynamic facial expression database. In: IEEE Automatic Face and Gesture Recognition (2008)

    Google Scholar 

  20. Xue, M., Mian, A., Liu, W., Li, L.: Automatic 4D facial expression recognition using DCT features. In: IEEE Winter Conference Applications of Computer Vision (WACV), pp. 199–206 (2015)

    Google Scholar 

  21. Reale, M., Zhang, X., Yin, L.: Nebula feature: a space time feature for posed and spontaneous 4D facial behavior analysis. In: 2013 10th IEEE International Conference and Workshops in Automatic Face and Gesture Recognition (FG), pp. 1–8 (2013)

    Google Scholar 

  22. Yin, L., Chen, X., Sun, Y., Worm, T., Reale, M.: A high resolution 3D dynamic facial expression database. In: 8th IEEE International Conference in Automatic Face & Gesture Recognition, pp. 1–6 (2008)

    Google Scholar 

  23. Chang, C.C., Lin, C.J.: LIBSVM: A Library for support vector machines. ACM Trans.Intell. Syst. Technol. (2011)

    Google Scholar 

  24. Shan, C., Gong, S., McOwan, P.W.: Facial expression recognition based on local binary patterns: a comprehensive study. Image Vis. Comput. 27(6), 803–816 (2009)

    Article  Google Scholar 

  25. Barr, J., Bowyer, K., Flynn, P., Biswas, S.: Face recognition from video: a review. Int. J. Pattern Recogn. Artif. Intell. 26(5), 1266002 (2012)

    Article  MathSciNet  Google Scholar 

  26. Zhang, X., Gao, Y.: Face recognition across pose: a review. Pattern Recogn. 42(11), 2876–2896 (2009)

    Article  Google Scholar 

  27. Mishra, B., Fernandes, S., Abhishek, K., Alva, A., Shetty, C., Ajila, C., Shetty, D., Rao, H., Shetty, P.: Facial expression recognition using feature based techniques and model based techniques: a survey. In: Electronics and Communication Systems (ICECS), 2nd International Conference, pp. 589–594 (2015)

    Google Scholar 

  28. Sandbach, G., Zafeiriou, S., Pantic, M., Yin, L.: Static and dynamic 3D facial expression recognition: a comprehensive survey. Image Vis. Comput. 30, 683–697 (2012)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Research Team on Intelligent Systems in Imaging and Articial Vision (SIIVA), LR16ES06 Laboratoire de Recherche en Informatique, Modélisation et Traitement de l’ Information et de la Connaissance (LIMTIC), Université de Tunis El Manar, Institut Supérieur d’ Informatique, Tunis, Tunisia

    Hela Bejaoui & Walid Barhoumi

  2. Ecole Nationale d’ Ingénieurs de Carthage, Université de Carthage, Tunis, Tunisia

    Haythem Ghazouani & Walid Barhoumi

Authors
  1. Hela Bejaoui
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Haythem Ghazouani
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Walid Barhoumi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hela Bejaoui .

Editor information

Editors and Affiliations

  1. DGA, Paris, France

    Jacques Blanc-Talon

  2. University of Antwerp, Antwerp, Belgium

    Rudi Penne

  3. Ghent University - imec, Ghent, Belgium

    Wilfried Philips

  4. CSIRO Data 61, Canberra, Aust Capital Terr, Australia

    Dan Popescu

  5. University of Antwerp, Wilrijk, Belgium

    Paul Scheunders

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Bejaoui, H., Ghazouani, H., Barhoumi, W. (2017). Fully Automated Facial Expression Recognition Using 3D Morphable Model and Mesh-Local Binary Pattern. In: Blanc-Talon, J., Penne, R., Philips, W., Popescu, D., Scheunders, P. (eds) Advanced Concepts for Intelligent Vision Systems. ACIVS 2017. Lecture Notes in Computer Science(), vol 10617. Springer, Cham. https://doi.org/10.1007/978-3-319-70353-4_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-70353-4_4

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-70352-7

  • Online ISBN: 978-3-319-70353-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation