Skip to main content

Advertisement

SpringerLink
Log in

Personalized face-pose estimation network using incrementally updated face shape parameters

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

In this paper, a deep learning method is proposed for human image processing that incorporates a mechanism to update target-specific parameters. The aim is to improve system performance in situations where the target can be continuously observed. Network-based algorithms typically rely on offline training processes that use large datasets, while trained networks typically operate in a one-shot fashion. That is, each input image is processed one by one in the static network. On the other hand, many practical applications can be expected to use continuous observation rather than observation of a single image. The proposed method employs dynamic use of multiple observations to improve system performance. In this paper, the effectiveness of the proposed method adopting an iterative update process is clarified through its implementation in the task of face-pose estimation. The method consists of two separate processes: 1) sequential estimation and updating of face-shape parameters (target-specific parameters) and 2) face-pose estimation for every single image using the updated parameters. Experimental results indicate the effectiveness of the proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Murphy-Chutorian E, Trivedi MM (2009) Head pose estimation in computer vision: a survey. IEEE Trans Pattern Anal Mach Intell 31(4):607–626. https://doi.org/10.1109/TPAMI.2008.106

    Article  Google Scholar 

  2. Julina JKJ, Sharmila TS (2017) A morphological approach to detect human in video. In: 2017 international conference on computer, communication and signal processing (ICCCSP), pp 1–5. https://doi.org/10.1109/ICCCSP.2017.7944083

  3. Tsumugiwa T, Kamiyoshi A, Yokogawa R, Shibata H (2006) Development of human motion detecting device for human-machine interface. In: 2006 IEEE international conference on robotics and biomimetics, pp 239–244. https://doi.org/10.1109/ROBIO.2006.340160

  4. Hernandez-Matamoros A, Bonarini A, Escamilla-Hernandez E, Nakano-Miyatake M, Perez-Meana H (2016) Facial expression recognition with automatic segmentation of face regions using a fuzzy based classification approach. Knowl-Based Syst 110:1

    Article  Google Scholar 

  5. Hernandez-Matamoros A, Nagai T, Attamimi M, Nakano M, Perez-Meana H (2017) Facial expression recogntion in unconstrained environment. In: New trends in intelligent software methodologies, tools and techniques. IOS Press, pp 525–538

  6. Park S, Mello SD, Molchanov P, Iqbal U, Hilliges O, Kautz J (2019) Few-shot adaptive gaze estimation. In: 2019 IEEE/CVF international conference on computer vision (ICCV), pp 9367–9376. https://doi.org/10.1109/ICCV.2019.00946

  7. Lindén E, Sjöstrand J, Proutiere A (2019) Learning to personalize in appearance-based gaze tracking. In: 2019 IEEE/CVF international conference on computer vision workshop (ICCVW), pp 1140–1148. https://doi.org/10.1109/ICCVW.2019.00145

  8. He J, Pham K, Valliappan N, Xu P, Roberts C, Lagun D, Navalpakkam V (2019) On-device few-shot personalization for real-time gaze estimation. In: 2019 IEEE/CVF international conference on computer vision workshop (ICCVW), pp 1149–1158. https://doi.org/10.1109/ICCVW.2019.00146

  9. Li S, Deng W (2020) Deep facial expression recognition: a survey. IEEE Trans Affect Comput 1–1. https://doi.org/10.1109/TAFFC.2020.2981446

  10. Noroozi F, Kaminska D, Corneanu C, Sapinski T, Escalera S, Anbarjafari G (2018) Survey on emotional body gesture recognition. IEEE Trans Affect Comput 1–1. https://doi.org/10.1109/TAFFC.2018.2874986

  11. Asadi-Aghbolaghi M, Clapés A., Bellantonio M, Escalante HJ, Ponce-López V, Baró X, Guyon I, Kasaei S, Escalera S (2017) A survey on deep learning based approaches for action and gesture recognition in image sequences. In: 2017 12th IEEE international conference on automatic face gesture recognition (FG 2017), pp 476–483. https://doi.org/10.1109/FG.2017.150

  12. Kar A, Corcoran P (2017) A review and analysis of eye-gaze estimation systems, algorithms and performance evaluation methods in consumer platforms. IEEE Access 5:16495. https://doi.org/10.1109/ACCESS.2017.2735633

    Article  Google Scholar 

  13. Yu J, Hong C, Rui Y, Tao D (2017) Multitask autoencoder model for recovering human poses. IEEE Trans Ind Electron 65(6):5060

    Article  Google Scholar 

  14. Hong C, Yu J, Zhang J, Jin X, Lee KH (2018) Multimodal face-pose estimation with multitask manifold deep learning. IEEE Trans Ind Inform 15(7):3952

    Article  Google Scholar 

  15. Xiao J, Li H, Qu G, Fujita H, Cao Y, Zhu J, Huang C (2021) Hope: heatmap and offset for pose estimation. J Ambient Intell Humanized Comput 1–13

  16. Krafka K, Khosla A, Kellnhofer P, Kannan H, Bhandarkar S, Matusik W, Torralba A (2016) Eye tracking for everyone. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2176–2184

  17. Glorot X, Bengio Y (2010) Understanding the difficulty of training deep feedforward neural networks. In: Proceedings of the thirteenth international conference on artificial intelligence and statistics (JMLR Workshop and Conference Proceedings), pp 249–256

  18. Kingma DP, Ba J (2017) Adam: A method for stochastic optimization

  19. Reddi SJ, Kale S, Kumar S (2019) On the convergence of adam and beyond

  20. Fanelli G, Dantone M, Gall J, Fossati A, Van Gool L (2013) Random forests for real time 3D face analysis. Int J Comput Vision 101(3):437

    Article  Google Scholar 

  21. Ruiz N, Chong E, Rehg JM (2018) Fine-grained head pose estimation without keypoints

  22. Yang TY, Chen YT, Lin YY, Chuang YY (2019) FSA-Net: Learning fine-grained structure aggregation for head pose estimation from a single image. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR)

  23. Kazemi V, Sullivan J (2014) One millisecond face alignment with an ensemble of regression trees. In: 2014 IEEE conference on computer vision and pattern recognition, pp 1867–1874. https://doi.org/10.1109/CVPR.2014.241

  24. Zhu X, Lei Z, Liu X, Shi H, Li SZ (2016) Face alignment across large poses: A 3D solution. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR), pp 146–155. https://doi.org/10.1109/CVPR.2016.23

  25. Bulat A, Tzimiropoulos G (2017) How far are we from solving the 2D & 3D face alignment problem? (and a Dataset of 230,000 3D Facial Landmarks). 2017 IEEE international conference on computer vision (ICCV). https://doi.org/10.1109/iccv.2017.116

  26. Bin H, Chen R, Xu W, Zhou Q (2019) Improving head pose estimation using two-stage ensembles with top-k regression. Image Vis Comput 93. https://doi.org/10.1016/j.imavis.2019.11.005

  27. Cao Z, Chu Z, Liu D, Chen Y (2020) A vector-based representation to enhance head pose estimation

  28. Albiero V, Chen X, Yin X, Pang G, Hassner T (2021) img2pose: Face alignment and detection via 6dof face pose estimation

  29. Köstinger M, Wohlhart P, Roth PM, Bischof H (2011) Annotated facial landmarks in the wild: A large-scale, real-world database for facial landmark localization. In: 2011 IEEE international conference on computer vision workshops (ICCV Workshops), pp 2144–2151. https://doi.org/10.1109/ICCVW.2011.6130513

Download references

Acknowledgements

This work was supported by JSPS KAKENHI Grant Number JP18H03269, JP18K11383, JP20K21824.

Author information

Authors and Affiliations

  1. Advanced Telecommunications Research Institute International (ATR), Seika-cho, Sorakugun, Kyoto, 619-0288, Japan

    Makoto Sei & Akira Utsumi

  2. Ritsumeikan University, Kusatsu, Shiga, 525-8577, Japan

    Makoto Sei & Joo-Ho Lee

  3. University of Hyogo, Himeji, Hyogo, 671-2280, Japan

    Hirotake Yamazoe

Authors
  1. Makoto Sei
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Akira Utsumi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hirotake Yamazoe
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Joo-Ho Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Makoto Sei.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sei, M., Utsumi, A., Yamazoe, H. et al. Personalized face-pose estimation network using incrementally updated face shape parameters. Appl Intell 52, 11506–11516 (2022). https://doi.org/10.1007/s10489-021-02888-0

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-021-02888-0

Keywords

Search

Navigation