Skip to main content

Tensor Decomposition and Non-linear Manifold Modeling for 3D Head Pose Estimation

Abstract

Head pose estimation is a challenging computer vision problem with important applications in different scenarios such as human–computer interaction or face recognition. In this paper, we present a 3D head pose estimation algorithm based on non-linear manifold learning. A key feature of the proposed approach is that it allows modeling the underlying 3D manifold that results from the combination of rotation angles. To do so, we use tensor decomposition to generate separate subspaces for each variation factor and show that each of them has a clear structure that can be modeled with cosine functions from a unique shared parameter per angle. Such representation provides a deep understanding of data behavior. We show that the proposed framework can be applied to a wide variety of input features and can be used for different purposes. Firstly, we test our system on a publicly available database, which consists of 2D images and we show that the cosine functions can be used to synthesize rotated versions from an object from which we see only a 2D image at a specific angle. Further, we perform 3D head pose estimation experiments using other two types of features: automatic landmarks and histogram-based 3D descriptors. We evaluate our approach on two publicly available databases, and demonstrate that angle estimations can be performed by optimizing the combination of these cosine functions to achieve state-of-the-art performance.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Notes

  1. 1.

    Of course this applies to rotations about a single axis; the general 3D-rotation case, depending on 3 parameters, could be anyway decomposed in the product of 3 such rotation matrices, analogous to the product of \(\mathbf {f}^{(y)}(\omega ^{(y)}) \times \mathbf {f}^{(p)}( \omega ^{(p)} ) \times \mathbf {f}^{(r)}( \omega ^{(r)} )\) in Eq. 11.

  2. 2.

    This process is coined translation in the seminal work by Tenenbaum and Freeman (2000).

  3. 3.

    These samples cover approximately a viewpoint range from \(5^{\circ }\) to \(35^{\circ }\).

  4. 4.

    Such invariance, however, is only partially achieved in 3DSC since the orientation of the surface normal still leaves one degree of freedom undefined (the sphere’s azimuth) (Sukno et al. 2013).

  5. 5.

    The results obtained from the minimization are forced to comply with the rotation manifold after each iteration using nearest-neighbour search. Results without such correction would be worse than those reported and not meaningful for comparison, since this is a widespread practice. Notice that no constraints are applied to the identity subspace in any of the experiments in this paper.

  6. 6.

    We consider that a landmark is on the surface when its distance to it is relatively small as compared to the mesh resolution.

  7. 7.

    Depending on the way in which the data is captured and the extent of the considered rotations, self-occlusions may jeopardize this strategy.

  8. 8.

    All experiments in this paper have been performed following this strategy.

References

  1. Ahn, B., Park, J., & Kweon, I. S. (2014). Real-time head orientation from a monocular camera using deep neural network. In Asian conference on computer vision (pp. 82–96). Springer.

  2. Bakry, A., & Elgammal, A. (2014). Untangling object-view manifold for multiview recognition and pose estimation. In European conference on computer vision (pp. 434–449). Springer.

  3. Balasubramanian, V. N., Ye, J., & Panchanathan, S. (2007). Biased manifold embedding: A framework for person-independent head pose estimation. In Computer vision and pattern recognition (CVPR) (pp. 1–7). IEEE.

  4. Baltrušaitis, T., Robinson, P., & Morency, L. P. (2012). 3D constrained local model for rigid and non-rigid facial tracking. In Computer vision and pattern recognition (CVPR) (pp. 2610–2617). IEEE.

  5. Barros, J. M. D., Mirbach, B., Garcia, F., Varanasi, K., & Stricker, D. (2018). Fusion of keypoint tracking and facial landmark detection for real-time head pose estimation. In Winter conference on applications of computer vision (WACV) (pp. 2028–2037). IEEE.

  6. BenAbdelkader, C. (2010). Robust head pose estimation using supervised manifold learning. In European conference on computer vision (pp. 518–531). Springer.

  7. Bergqvist, G., & Larsson, E. G. (2010). The higher-order singular value decomposition: Theory and an application [lecture notes]. IEEE Signal Processing Magazine, 27(3), 151–154.

    Article  Google Scholar 

  8. Borghi, G., Fabbri, M., Vezzani, R., Calderara, S., & Cucchiara, R. (2019). Face-from-depth for head pose estimation on depth images. IEEE Transactions on Pattern Analysis and Machine Intelligence (in press).

  9. Borghi, G., Venturelli, M., Vezzani, R., & Cucchiara, R. (2017). Poseidon: Face-from-depth for driver pose estimation. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 4661–4670).

  10. Breitenstein, M. D., Kuettel, D., Weise, T., Van Gool L, & Pfister, H. (2008). Real-time face pose estimation from single range images. In Computer vision and pattern recognition (pp. 1–8). IEEE.

  11. Byrd, R. H., Nocedal, J., & Schnabel, R. B. (1994). Representations of quasi-newton matrices and their use in limited memory methods. Mathematical Programming, 63(1–3), 129–156.

    MathSciNet  Article  MATH  Google Scholar 

  12. Chen, J., Wu, J., Richter, K., Konrad, J., & Ishwar, P. (2016). Estimating head pose orientation using extremely low resolution images. In Southwest symposium on image analysis and interpretation (SSIAI) (pp. 65–68). IEEE

  13. Comon, P. (2014). Tensors: A brief introduction. Signal Processing Magazine, 31(3), 44–53.

    Article  Google Scholar 

  14. De Lathauwer, L., De Moor, B., & Vandewalle, J. (2000). A multilinear singular value decomposition. SIAM Journal on Matrix Analysis and Applications, 21(4), 1253–1278.

    MathSciNet  Article  MATH  Google Scholar 

  15. Derkach, D., Ruiz, A., & Sukno, F. M. (2017). Head pose estimation based on 3-D facial landmarks localization and regression. In 12th IEEE international conference on automatic face and gesture recognition (FG 2017) (pp. 820–827). IEEE.

  16. Derkach, D., Ruiz, A., & Sukno, F. M. (2018). 3D head pose estimation using tensor decomposition and non-linear manifold modeling. In: International conference on 3D Vision (3DV) (pp. 505–513). IEEE.

  17. Fanelli, G., Dantone, M., Gall, J., Fossati, A., & Van Gool, L. (2013). Random forests for real time 3D face analysis. International Journal of Computer Vision, 101(3), 437–458.

    Article  Google Scholar 

  18. Fanelli, G., Weise, T., Gall, J., & Van Gool, L. (2011). Real time head pose estimation from consumer depth cameras. In Joint pattern recognition symposium (pp. 101–110). Springer.

  19. Frome, A., Huber, D., Kolluri, R., Bulow, T., & Malik, J. (2004). Recognizing objects in range data using regional point descriptors. In European conference on computer vision (pp. 224–237). Springer.

  20. Fu, Y., & Huang, T. S. (2006). Graph embedded analysis for head pose estimation. In International conference on automatic face and gesture recognition (pp. 6–8). IEEE.

  21. Ghiass, R. S., Arandjelović, O., & Laurendeau, D. (2015). Highly accurate and fully automatic head pose estimation from a low quality consumer-level rgb-d sensor. In Proceedings of the 2nd workshop on computational models of social interactions: Human–Computer–Media communication (pp. 25–34). ACM.

  22. Gu, J., Yang, X., De Mello, S., & Kautz, J. (2017). Dynamic facial analysis: From bayesian filtering to recurrent neural network. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1548–1557).

  23. Johnson, A., & Hebert, M. (1999). Using spin images for efficient object recognition in cluttered 3D scenes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 21(5), 433–449.

    Article  Google Scholar 

  24. Kolda, T. G., & Bader, B. W. (2009). Tensor decompositions and applications. SIAM Review, 51(3), 455–500.

    MathSciNet  Article  MATH  Google Scholar 

  25. Lathuiliére, S., Juge, R., Mesejo, P., Muñoz-Salinas, R., & Horaud, R. (2017). Deep mixture of linear inverse regressions applied to head-pose estimation. In Conference on computer vision and pattern recognition (vol. 3, pp. 4817–4825).

  26. Lathuiliére, S., Mesejo, P., Alameda-Pineda, X., & Horaud, R. (2019). A comprehensive analysis of deep regression. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1–1 (in press).

  27. Lee, D., Yang, M. H., & Oh, S. (2015). Fast and accurate head pose estimation via random projection forests. In International conference on computer vision (pp. 1958–1966). IEEE.

  28. Lee, D., Yang, M. H., & Oh, S. (2017). Head and body orientation estimation using convolutional random projection forests. In IEEE transactions on pattern analysis and machine intelligence (pp. 1–14)

  29. Li, D., & Pedrycz, W. (2014). A central profile-based 3D face pose estimation. Pattern Recognition, 47(2), 525–534.

    Article  Google Scholar 

  30. Li, S., Ngan, K. N., Paramesran, R., & Sheng, L. (2016). Real-time head pose tracking with online face template reconstruction. IEEE Transactions on Pattern Analysis and Machine Intelligence, 38(9), 1922–1928.

    Article  Google Scholar 

  31. Liu, X., Liang, W., Wang, Y., Li, S., & Pei, M. (2016). 3D head pose estimation with convolutional neural network trained on synthetic images. In International conference on image processing (ICIP) (pp. 1289–1293). IEEE.

  32. Liu, X., Lu, H., & Li, W. (2010). Multi-manifold modeling for head pose estimation. In International conference on image processing (ICIP) (pp. 3277–3280). IEEE.

  33. Lüsi, I., Escalera, S., & Anbarjafari, G. (2016a). Human head pose estimation on SASE database using random hough regression forests. Video Analytics (pp. 137–150). Springer: Face and Facial Expression Recognition and Audience Measurement.

  34. Lüsi, I., Escarela, S., & Anbarjafari, G. (2016b). SASE: RGB-depth database for human head pose estimation. In European conference on computer vision (pp. 325–336). Springer.

  35. Lüsi, I., Jacques Junior, J. C. S., Gorbova, J., Baró X, Escalera, S., Demirel, H., Allik, J., Ozcinar, C., & Anbarjafari, G. (2017). Joint challenge on dominant and complementary emotion recognition using micro emotion features and head-pose estimation: Databases. In International conference on automatic face and gesture recognition (pp. 809–813). IEEE.

  36. Martin, M., Van De Camp, F., & Stiefelhagen, R. (2014). Real time head model creation and head pose estimation on consumer depth cameras. In International conference on 3D vision (3DV) (vol. 1, pp. 641–648). IEEE.

  37. Meyer, G. P., Gupta, S., Frosio, I., Reddy, D., & Kautz, J. (2015). Robust model-based 3D head pose estimation. In Proceedings of the IEEE international conference on computer vision (pp. 3649–3657). IEEE.

  38. Murphy-Chutorian, E., & Trivedi, M. M. (2009). Head pose estimation in computer vision: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 31(4), 607–626.

    Article  Google Scholar 

  39. Nene, S. A., Nayar, S. K., Murase, H., et al. (1996). Columbia object image library (coil-20).

  40. Padeleris, P., Zabulis, X., & Argyros, A. A. (2012). Head pose estimation on depth data based on particle swarm optimization. In Computer society conference on computer vision and pattern recognition workshops (CVPRW) (pp. 42–49). IEEE.

  41. Papazov, C., Marks, T. K., & Jones, M. (2015). Real-time 3D head pose and facial landmark estimation from depth images using triangular surface patch features. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 4722–4730).

  42. Patacchiola, M., & Cangelosi, A. (2017). Head pose estimation in the wild using convolutional neural networks and adaptive gradient methods. Pattern Recognition, 71, 132–143.

    Article  Google Scholar 

  43. Peng, X., Huang, J., Hu, Q., Zhang, S., & Metaxas, D. N. (2014). Head pose estimation by instance parameterization. In International conference on pattern recognition (ICPR) (pp. 1800–1805). IEEE.

  44. Raytchev, B., Yoda, I., & Sakaue, K. (2004). Head pose estimation by nonlinear manifold learning. In International conference on pattern recognition (ICPR) (vol. 4, pp. 462–466). IEEE.

  45. Ruiz, N., Chong, E., & Rehg, J. M. (2018). Fine-grained head pose estimation without keypoints. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops (pp. 2074–2083).

  46. Rusu, R. B., Blodow, N., & Beetz, M. (2009). Fast point feature histograms (fpfh) for 3d registration. In International conference on robotics and automation, Citeseer (pp. 3212–3217).

  47. Schmidt, M. (2012). Minfunc: Unconstrained differentiable multivariate optimization in matlab. Software available at http://www.cs.ubc.ca/schmidtm/Software/minFunc.htm.

  48. Seemann, E., Nickel, K., & Stiefelhagen, R. (2004). Head pose estimation using stereo vision for human–robot interaction. In International conference on automatic face and gesture recognition (pp. 626–631). IEEE.

  49. Sukno, F., Waddington, J., & Whelan, P. (2012). Comparing 3D descriptors for local search of craniofacial landmarks. In International symposium on visual computing (pp. 92–103). Springer.

  50. Sukno, F., Waddington, J., & Whelan, P. (2013). Rotationally invariant 3D shape contexts using asymmetry patterns. International conference on computer graphics theory and applications (pp. 7–17).

  51. Sukno, F. M., Waddington, J. L., & Whelan, P. F. (2015). 3-D facial landmark localization with asymmetry patterns and shape regression from incomplete local features. IEEE Transactions on Cybernetics, 45(9), 1717–1730.

    Article  Google Scholar 

  52. Sun, Y., & Yin, L. (2008). Automatic pose estimation of 3D facial models. In International conference on pattern recognition (pp. 1–4.).

  53. Sundararajan, K., & Woodard, D. L. (2015). Head pose estimation in the wild using approximate view manifolds. In International conference on computer vision and pattern recognition workshops (pp. 50–58). IEEE.

  54. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015). Going deeper with convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1–9).

  55. Takallou, H. M., & Kasaei, S. (2014). Head pose estimation and face recognition using a non-linear tensor-based model. IET Computer Vision, 8(1), 54–65.

    Article  Google Scholar 

  56. Tan, D. J., Tombari, F., & Navab, N. (2018). Real-time accurate 3d head tracking and pose estimation with consumer rgb-d cameras. International Journal of Computer Vision, 126(2–4), 158–183.

    MathSciNet  Article  Google Scholar 

  57. Tenenbaum, J. B., & Freeman, W. T. (1997). Separating style and content. In Advances in neural information processing systems (pp. 662–668).

  58. Tenenbaum, J. B., & Freeman, W. T. (2000). Separating style and content with bilinear models. Neural Computation, 12(6), 1247–1283.

    Article  Google Scholar 

  59. Tombari, F., Salti, S., & Di Stefano, L. (2010). Unique signatures of histograms for local surface description. In European conference on computer vision (pp. 356–369). Springer.

  60. Tulyakov, S., Vieriu, R. L., Semeniuta, S., & Sebe, N. (2014). Robust real-time extreme head pose estimation. In International conference on pattern recognition (ICPR) (pp. 2263–2268). IEEE.

  61. Vasilescu, M. A. O., & Terzopoulos, D. (2002). Multilinear analysis of image ensembles: Tensorfaces. In European conference on computer vision (pp. 447–460). Springer.

  62. Wang, B., Liang, W., Wang, Y., & Liang, Y. (2013). Head pose estimation with combined 2D SIFT and 3D HOG features. In International conference on image and graphics (ICIG) (pp. 650–655). IEEE.

  63. Wang, C., Guo, Y., & Song, X. (2017a). Head pose estimation via manifold learning. InTech: In Manifolds-current research areas.

    Book  Google Scholar 

  64. Wang, C., & Song, X. (2014). Robust head pose estimation via supervised manifold learning. Neural Networks, 53, 15–25.

    Article  MATH  Google Scholar 

  65. Wang, K., Wu, Y., & Ji, Q. (2018). Head pose estimation on low-quality images. In International conference on automatic face and gesture recognition (FG 2018) (pp. 540–547). IEEE.

  66. Wang, M., Panagakis, Y., Snape, P., Zafeiriou, S., et al. (2017b). Learning the multilinear structure of visual data. In: Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 4592–4600).

  67. Wang, Y., Liang, W., Shen, J., Jia, Y., & Yu, L. F. (2019). A deep coarse-to-fine network for head pose estimation from synthetic data. Pattern Recognition, 94, 196–206.

    Article  Google Scholar 

  68. Xu, Y., Hao, R., Yin, W., & Su, Z. (2015). Parallel matrix factorization for low-rank tensor completion. Inverse Problems and Imaging, 9(2), 601–624.

    MathSciNet  Article  MATH  Google Scholar 

  69. Yu, Y., Mora, K. A. F., & Odobez, J. M. (2017). Robust and accurate 3D head pose estimation through 3dmm and online head model reconstruction. In International conference on automatic face and gesture recognition (FG 2017) (pp. 711–718). IEEE.

  70. Zhang, H., El-Gaaly, T., Elgammal, A., & Jiang, Z. (2015). Factorization of view-object manifolds for joint object recognition and pose estimation. Computer Vision and Image Understanding, 139, 89–103.

    Article  Google Scholar 

  71. Zhao, Q., Zhang, L., & Cichocki, A. (2015). Bayesian cp factorization of incomplete tensors with automatic rank determination. IEEE Transactions on Pattern Analysis and Machine Intelligence, 37(9), 1751–1763.

    Article  Google Scholar 

  72. Zhu, Y., Xue, Z., & Li, C. (2014). Automatic head pose estimation with synchronized sub manifold embedding and random regression forests. International Journal of Signal Processing, Image Processing and Pattern Recognition, 7(3), 123–134.

    Article  Google Scholar 

Download references

Acknowledgements

This work is partly supported by the Spanish Ministry of Economy and Competitiveness under Project Grant TIN2017-90124-P, the Ramon y Cajal programme, and the Maria de Maeztu Units of Excellence Programme (MDM-2015-0502). Adria Ruiz work is partially funded by ANR grant ANR-16-CE23-0006.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Federico M. Sukno.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Communicated by Stephen Lin.

Appendix

Appendix

For the objective function of Eq. 15, partial derivatives should be computed with respect to variables \(\alpha _j^{(*)}, \beta _j^{(*)}, \gamma _j^{(*)}, \varphi _j^{(*)}\). Let us rewrite this equation as:

$$\begin{aligned}&\mathop {\mathrm{argmin}}\limits _{\alpha _j^{(*)}, \beta _j^{(*)}, \gamma _j^{(*)}, \varphi _j^{(*)}}{ \Vert u^{(*)}_{ij} - f_j( \omega _i^{(*)} ) \Vert }\\&= \mathop {\mathrm{argmin}}\limits _{\alpha _j^{(*)}, \beta _j^{(*)}, \gamma _j^{(*)}, \varphi _j^{(*)}} {E(\alpha _j^{(*)}, \beta _j^{(*)}, \gamma _j^{(*)}, \varphi _j^{(*)})} \end{aligned}$$

Where error function E can be written in element form as:

$$\begin{aligned}&E(\alpha _j^{(*)}, \beta _j^{(*)}, \gamma _j^{(*)}, \varphi _j^{(*)})\\&= \frac{1}{2} \sum _i{\big ( u^{(*)}_{ij} - ( \alpha _j^{(*)} \, \cos {( \beta _j^{(*)} \omega _i^{(*)} + \gamma _j^{(*)})} + \varphi _j^{(*)}) \big )}^2 \end{aligned}$$

Thus, partial derivatives of the function E become:

$$\begin{aligned} \frac{\partial {E}}{\partial {\alpha _j^{(*)}}}= & {} \sum _i \big ( \cos {( \beta _j^{(*)} \omega _i^{(*)} + \gamma _j^{(*)})}\\&\cdot (\alpha _j^{(*)}\cos {( \beta _j^{(*)} \omega _i^{(*)} + \gamma _j^{(*)})} - u^{(*)}_{ij} + \varphi _j^{(*)})\big )\\ \frac{\partial {E}}{\partial {\beta _j^{(*)}}}= & {} \alpha _j^{(*)}\sum _i \big ( \omega _i^{(*)} \sin {(\beta _j^{(*)}\omega _i^{(*)}+\gamma _j^{(*)})}\\&\cdot (u^{(*)}_{ij} - \alpha _j^{(*)} \cos {(\beta _j^{(*)}\omega _i^{(*)} + \gamma _j^{(*)}) - \varphi _j^{(*)}})\big )\\ \frac{\partial {E}}{\partial {\gamma _j^{(*)}}}= & {} \alpha _j^{(*)}\sum _i \big ( \sin {(\beta _j^{(*)}\omega _i^{(*)}+\gamma _j^{(*)})}\\&\cdot (u^{(*)}_{ij} - \alpha _j^{(*)}\cos {(\beta _j^{(*)}\omega _i^{(*)}+\gamma _j^{(*)})} -\varphi _j^{(*)})\big )\\ \frac{\partial {E}}{\partial {\varphi _j^{(*)}}}= & {} \sum _i \big ( \alpha _j^{(*)}\cdot \cos {(\beta _j^{(*)}\omega _i^{(*)}+\gamma _j^{(*)})}\\&+ \varphi _j^{(*)} + u^{(*)}_{ij} \big ) \end{aligned}$$

Similarly to the previous case, the error function in Eq. 12 can be written as:

$$\begin{aligned} E(\omega ^{(y)}, \omega ^{(p)}, \omega ^{(r)}, \mathbf u ^{(id)}) = \frac{1}{2} \sum _n{\big ( \mathbf x _n - \hat{\mathbf{x }}_n \big )}^2 \end{aligned}$$

where \(\mathbf x _n\) using Eq. 14 is:

$$\begin{aligned} \mathbf x _n= & {} \sum _i \sum _j \sum _k \sum _l\big ( \mathscr {W}_{ijkln}\cdot f_i^{(y)}( \omega ^{(y)} ) \cdot f_j^{(p)}( \omega ^{(p)} )\\&\cdot f_k^{(r)}( \omega ^{(r)} ) \cdot {u}^{(id)}_l \big ) \end{aligned}$$

Now we can compute partial derivatives with respect to variables \(\omega ^{(y)}, \omega ^{(p)}, \omega ^{(r)}\) and each l-th element in vector \(\mathbf u ^{(id)}\):

$$\begin{aligned} \frac{\partial {E}}{\partial {\omega ^{(y)}}}= & {} -\sum _n \big ( (\mathbf x _n - \hat{\mathbf{x }}_n ) \cdot \frac{\partial {\hat{\mathbf{x }}_n}}{\partial {\omega ^{(y)}}} \big );\\ \frac{\partial {\hat{\mathbf{x }}_n}}{\partial {\omega ^{(y)}}}= & {} -\sum _i \sum _j \sum _k \sum _l\big ( \mathscr {W}_{ijkln}\cdot \alpha ^{(y)}_i \sin (\beta ^{(y)}_i\omega ^{(y)}+\gamma ^{(y)}_i)\beta ^{(y)}_i\\&\cdot f_j^{(p)}( \omega ^{(p)} ) \cdot f_k^{(r)}( \omega ^{(r)} )\cdot {u}^{(id)}_l \big ) ;\\ \frac{\partial {E}}{\partial {\omega ^{(p)}}}= & {} -\sum _n \big ( (\mathbf x _n - \hat{\mathbf{x }}_n ) \cdot \frac{\partial {\hat{\mathbf{x }}_n}}{\partial {\omega ^{(p)}}} \big );\\ \frac{\partial {\hat{\mathbf{x }}_n}}{\partial {\omega ^{(p)}}}= & {} -\sum _i \sum _j \sum _k \sum _l\big ( \mathscr {W}_{ijkln}\cdot f_i^{(y)}( \omega ^{(y)} )\\&\cdot (\alpha ^{(p)}_j \sin (\beta ^{(p)}_j\omega ^{(p)}+\gamma ^{(p)}_j)\beta ^{(p)}_j)\cdot f_k^{(r)}( \omega ^{(r)} )\cdot {u}^{(id)}_l \big );\\ \frac{\partial {E}}{\partial {\omega ^{(r)}}}= & {} -\sum _n \big ( (\mathbf x _n - \hat{\mathbf{x }}_n ) \cdot \frac{\partial {\hat{\mathbf{x }}_n}}{\partial {\omega ^{(r)}}} \big );\\ \frac{\partial {\hat{\mathbf{x }}_n}}{\partial {\omega ^{(r)}}}= & {} -\sum _i \sum _j \sum _k \sum _l\big ( \mathscr {W}_{ijkln}\cdot f_i^{(y)}( \omega ^{(y)} )\\&\cdot f_j^{(p)}( \omega ^{(p)} ) \cdot (\alpha ^{(r)}_k \sin (\beta ^{(r)}_k\omega ^{(r)}+\gamma ^{(r)}_k)\beta ^{(r)}_k) \cdot {u}^{(id)}_l \big );\\ \frac{\partial {E}}{\partial \mathbf{u ^{(id)}_l}}= & {} -\sum _n \big ( (\mathbf x _n - \hat{\mathbf{x }}_n ) \cdot \frac{\partial {\hat{\mathbf{x }}_n}}{\partial \mathbf{u ^{(id)}_l}} \big ); \\ \frac{\partial {\hat{\mathbf{x }}_n}}{\partial \mathbf{u ^{(id)}_l}}= & {} \sum _i \sum _j \sum _k \big ( \mathscr {W}_{ijkln}\cdot f_i^{(y)}( \omega ^{(y)} )\cdot f_j^{(p)}( \omega ^{(p)} ) \cdot f_k^{(r)}( \omega ^{(r)} )\big ) \end{aligned}$$

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Derkach, D., Ruiz, A. & Sukno, F.M. Tensor Decomposition and Non-linear Manifold Modeling for 3D Head Pose Estimation. Int J Comput Vis 127, 1565–1585 (2019). https://doi.org/10.1007/s11263-019-01208-x

Download citation

Keywords

  • 3D head pose
  • Manifold learning
  • Tensor decomposition