Skip to main content
SpringerLink
Log in

IFEPE: On the Impact of Facial Expression in Head Pose Estimation

  • Conference paper
  • First Online:
Pattern Recognition. ICPR International Workshops and Challenges (ICPR 2021)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 12665))

Included in the following conference series:

Abstract

The Head Pose Estimation (HPE) is the study of the angular rotations of the head along the Pitch, Yaw, and Roll axes. Widely used in facial involving methods, as face frontalization, driver attention and best surveillance frame selection, is strongly related to facial features. In this study we examine the impact of facial expressions (FE) on HPE and, in particular, we put in relation the axis more affected by the error when a specific facial expression is observable. The HPE method chosen for this purpose is based on the Partitioned Iterated Function System (PIFS). For its construction this method is dependent on the facial appearance and self-similarity. Basing on this, and using a FER network, we observed that there is an evident relation between facial expressions and pose errors. This relation go thought the facial keypoints distances and can be discriminated along the three axes, by providing an estimate of the percentages of variation in errors related to a percentage of variation in distances.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Abate, A.F., Barra, P., Bisogni, C., Nappi, M., Ricciardi, S.: Near real-time three axis head pose estimation without training. IEEE Access 7, 64256–64265 (2019). https://doi.org/10.1109/ACCESS.2019.2917451

    Article  Google Scholar 

  2. Abdi, H.: Partial least square regression (pls regression). Encyclopedia Res. Methods Soc. Sci. 6(4), 792–795 (2003)

    Google Scholar 

  3. Ariz, M., Villanueva, A., Cabeza, R.: Robust and accurate 2d-tracking-based 3d positioning method: application to head pose estimation. Comput. Vis. Image Underst. 180, 13–22 (2019)

    Article  Google Scholar 

  4. Barnsley, M.F., et al.: The Science of Fractal Images. Springer, New York (1988). https://doi.org/10.1007/978-1-4612-3784-6

    Book  Google Scholar 

  5. Barra, P., Barra, S., Bisogni, C., De Marsico, M., Nappi, M.: Web-shaped model for head pose estimation: an approach for best exemplar selection. IEEE Trans. Image Process. 29, 5457–5468 (2020)

    Article  Google Scholar 

  6. Bartlett, M.S., Littlewort, G., Frank, M., Lainscsek, C., Fasel, I., Movellan, J.: Recognizing facial expression: machine learning and application to spontaneous behavior. In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2005), vol. 2, pp. 568–573. IEEE (2005)

    Google Scholar 

  7. Belhumeur, P.N., Jacobs, D.W., Kriegman, D.J., Kumar, N.: Localizing parts of faces using a consensus of exemplars. IEEE Trans. Pattern Anal. Mach. Intell. 35(12), 2930–2940 (2013)

    Article  Google Scholar 

  8. Bisogni, C., Nappi, M., Pero, C., Ricciardi, S.: Hp2ifs: head pose estimation exploiting partitioned iterated function systems. In: 25th International Conference on Pattern Recognition (ICPR2020) (2020). https://arxiv.org/abs/2003.11536

  9. Chen, Y., Wang, J., Chen, S., Shi, Z., Cai, J.: Facial motion prior networks for facial expression recognition. In: 2019 IEEE Visual Communications and Image Processing, VCIP 2019, Sydney, Australia, December 1–4, 2019, pp. 1–4. IEEE (2019)

    Google Scholar 

  10. Czupryński, B., Strupczewski, A.: High accuracy head pose tracking survey. In: Ślȩzak, D., Schaefer, G., Vuong, S.T., Kim, Y.-S. (eds.) AMT 2014. LNCS, vol. 8610, pp. 407–420. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-09912-5_34

    Chapter  Google Scholar 

  11. Du, S., Tao, Y., Martinez, A.M.: Compound facial expressions of emotion. Proc. Natl. Acad. Sci. 111(15), E1454–E1462 (2014). https://doi.org/10.1073/pnas.1322355111

    Article  Google Scholar 

  12. Ekman, P., Friesen, W.V.: Constants across cultures in the face and emotion. J. Pers. Soc. Psychol. 17(2), 124 (1971)

    Article  Google Scholar 

  13. Fanelli, G., Weise, T., Gall, J., Van Gool, L.: Real time head pose estimation from consumer depth cameras. In: Joint Pattern Recognition Symposium (2011)

    Google Scholar 

  14. Hamming, R.W.: Error detecting and error correcting codes. Bell Syst. Tech. J. 29(2), 147–160 (1950)

    Article  MathSciNet  Google Scholar 

  15. Hough, P.V.: Method and means for recognizing complex patterns (Dec 18 1962), uS Patent 3,069,654

    Google Scholar 

  16. Hsu, H.W., Wu, T.Y., Wan, S., Wong, W.H., Lee, C.Y.: Quatnet: quaternion-based head pose estimation with multiregression loss. IEEE Trans. Multimedia 21(4), 1035–1046 (2018)

    Article  Google Scholar 

  17. Jain, D.K., Shamsolmoali, P., Sehdev, P.: Extended deep neural network for facial emotion recognition. Pattern Recogn. Lett. 120, 69–74 (2019)

    Article  Google Scholar 

  18. Kazemi, V., Sullivan, J.: One millisecond face alignment with an ensemble of regression trees. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1867–1874 (2014)

    Google Scholar 

  19. Kim, J.H., Kim, B.G., Roy, P.P., Jeong, D.M.: Efficient facial expression recognition algorithm based on hierarchical deep neural network structure. IEEE Access 7, 41273–41285 (2019)

    Article  Google Scholar 

  20. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)

    Google Scholar 

  21. Li, S., Deng, W.: Deep facial expression recognition: a survey. IEEE Trans. Affect. Comput. (2020)

    Google Scholar 

  22. Lopes, A.T., de Aguiar, E., De Souza, A.F., Oliveira-Santos, T.: Facial expression recognition with convolutional neural networks: coping with few data and the training sample order. Pattern Recogn. 61, 610–628 (2017)

    Article  Google Scholar 

  23. Lucey, P., Cohn, J.F., Kanade, T., Saragih, J., Ambadar, Z., Matthews, I.: The extended cohn-kanade dataset (ck+): a complete dataset for action unit and emotion-specified expression. In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition-Workshops, pp. 94–101. IEEE (2010)

    Google Scholar 

  24. Meng, Z., Liu, P., Cai, J., Han, S., Tong, Y.: Identity-aware convolutional neural network for facial expression recognition. In: 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017), pp. 558–565. IEEE (2017)

    Google Scholar 

  25. Mollahosseini, A., Hasani, B., Mahoor, M.H.: AffectNet: a database for facial expression, valence, and arousal computing in the wild. IEEE Trans. Affect. Comput. 10(1), 18–31 (2019)

    Article  Google Scholar 

  26. Murphy-Chutorian, E., Trivedi, M.M.: Head pose estimation in computer vision: a survey. IEEE Trans. Pattern Anal. Mach. Intell. 31(4), 607–626 (2008)

    Article  Google Scholar 

  27. Ranjan, R., Patel, V.M., Chellappa, R.: Hyperface: a deep multi-task learning framework for face detection, landmark localization, pose estimation, and gender recognition. IEEE Trans. Pattern Anal. Mach. Intell. 41(1), 121–135 (2019). https://doi.org/10.1109/TPAMI.2017.2781233

    Article  Google Scholar 

  28. Rasmussen, C.E.: Gaussian processes in machine learning. In: Bousquet, O., von Luxburg, U., Rätsch, G. (eds.) ML -2003. LNCS (LNAI), vol. 3176, pp. 63–71. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-28650-9_4

    Chapter  Google Scholar 

  29. Sagonas, C., Antonakos, E., Tzimiropoulos, G., Zafeiriou, S., Pantic, M.: 300 faces in-the-wild challenge: database and results. Image Vis. Comput. 47, 3–18 (2016)

    Article  Google Scholar 

  30. Sagonas, C., Tzimiropoulos, G., Zafeiriou, S., Pantic, M.: 300 faces in-the-wild challenge: the first facial landmark localization challenge. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, pp. 397–403 (2013)

    Google Scholar 

  31. Shan, C., Gong, S., McOwan, P.W.: Facial expression recognition based on local binary patterns: a comprehensive study. Image Vis. Comput. 27(6), 803–816 (2009)

    Article  Google Scholar 

  32. Valstar, M.F., Pantic, M.: Induced disgust, happiness and surprise: an addition to the mmi facial expression database. In: Proceedings of International Conference Language Resources and Evaluation, Workshop on EMOTION, Malta, 2019, May 2010, pp. 65–70 (2010)

    Google Scholar 

  33. Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2001. vol. 1, pp. I-I. IEEE (2001)

    Google Scholar 

  34. Wang, Y., Liang, W., Shen, J., Jia, Y., Yu, L.F.: A deep coarse-to-fine network for head pose estimation from synthetic data. Pattern Recogn. 94, 196–206 (2019)

    Article  Google Scholar 

  35. Whitehill, J., Omlin, C.W.: HAAR features for FACS au recognition. In: 7th International Conference on Automatic Face and Gesture Recognition (FGR06), pp. 5-pp. IEEE (2006)

    Google Scholar 

  36. Yang, T.Y., Chen, Y.T., Lin, Y.Y., Chuang, Y.Y.: Fsa-net: Learning fine-grained structure aggregation for head pose estimation from a single image. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1087–1096 (2019)

    Google Scholar 

  37. Zhang, F., Zhang, T., Mao, Q., Xu, C.: Joint pose and expression modeling for facial expression recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3359–3368 (2018)

    Google Scholar 

  38. Zhou, E., Fan, H., Cao, Z., Jiang, Y., Yin, Q.: Extensive facial landmark localization with coarse-to-fine convolutional network cascade. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, pp. 386–391 (2013)

    Google Scholar 

  39. Zhu, X., Lei, Z., Shi, H., Liu, X., Li, S.Z.: Face alignment across large poses: a 3d solution. In: IEEE Conference on Computer Vision and Pattern Recognition (2016)

    Google Scholar 

  40. Zhu, X., Ramanan, D.: Face detection, pose estimation, and landmark localization in the wild. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition, pp. 2879–2886. IEEE (2012)

    Google Scholar 

  41. Zhu, X., Lei, Z., Liu, X., Shi, H., Li, S.Z.: Face alignment across large poses: a 3d solution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 146–155 (2016)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, University of Salerno, Via Giovanni Paolo II, 132, Fisciano, Salerno, Italy

    Carmen Bisogni & Chiara Pero

Authors
  1. Carmen Bisogni
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chiara Pero
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Carmen Bisogni .

Editor information

Editors and Affiliations

  1. Dipartimento di Ingegneria dell’Informazione, University of Firenze, Firenze, Italy

    Alberto Del Bimbo

  2. Dipartimento di Ingegneria “Enzo Ferrari”, Università di Modena e Reggio Emilia, Modena, Italy

    Rita Cucchiara

  3. Department of Computer Science, Boston University, Boston, MA, USA

    Stan Sclaroff

  4. Dipartimento di Matematica e Informatica, University of Catania, Catania, Italy

    Giovanni Maria Farinella

  5. Cloud & AI, JD.COM, Beijing, China

    Tao Mei

  6. Dipartimento di Ingegneria dell’Informazione, University of Firenze, Firenze, Italy

    Marco Bertini

  7. Computational Sciences Department, National Institute of Astrophysics, Optics and Electronics (INAOE), Tonantzintla, Puebla, Mexico

    Hugo Jair Escalante

  8. Dipartimento di Ingegneria “Enzo Ferrari”, Università di Modena e Reggio Emilia, Modena, Italy

    Roberto Vezzani

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Bisogni, C., Pero, C. (2021). IFEPE: On the Impact of Facial Expression in Head Pose Estimation. In: Del Bimbo, A., et al. Pattern Recognition. ICPR International Workshops and Challenges. ICPR 2021. Lecture Notes in Computer Science(), vol 12665. Springer, Cham. https://doi.org/10.1007/978-3-030-68821-9_41

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-68821-9_41

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-68820-2

  • Online ISBN: 978-3-030-68821-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships

Search

Navigation