Non-rigid Face Modelling Using Shape Priors

  • Alessio Del Bue
  • Xavier Lladó
  • Lourdes Agapito
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3723)


Non-rigid 3D shape recovery is an inherently ambiguous problem. Given a specific rigid motion, different non-rigid shapes can be found that fit the measurements. To solve this ambiguity prior knowledge on the shape and motion should be used to constrain the solution. This paper is based on the observation that often not all the points on a moving and deforming surface such as a human face are undergoing non-rigid motion. Some of the points are frequently on rigid parts of the structure – for instance the nose – while others lie on deformable areas. First we develop a segmentation algorithm to separate rigid and non-rigid motion. Once this segmentation is available, the rigid points can be used to estimate the overall rigid motion and to constrain the underlying mean shape. We propose two reconstruction algorithms and show that improved reconstructions can be obtained when the priors on the shape are used on synthetic and real data.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Aanæs, H., Kahl, F.: Estimation of deformable structure and motion. In: Workshop on Vision and Modelling of Dynamic Scenes, ECCV 2002, Copenhagen, Denmark (2002)Google Scholar
  2. 2.
    Brand, M.: Morphable models from video. In: Proc. IEEE Conference on Computer Vision and Pattern Recognition, Kauai, Hawaii (December 2001)Google Scholar
  3. 3.
    Bregler, C., Hertzmann, A., Biermann, H.: Recovering non-rigid 3d shape from image streams. In: Proc. IEEE Conference on Computer Vision and Pattern Recognition, Hilton Head, South Carolina, pp. 690–696 (June 2000)Google Scholar
  4. 4.
    Del Bue, A., Smeraldi, F., Agapito, L.: Non-rigid structure from motion using non-parametric tracking and non-linear optimization. In: Proceedings of the 2004 Conference on Computer Vision and Pattern Recognition Workshop (CVPRW 2004), Washington, DC, USA, vol. 1 (2004)Google Scholar
  5. 5.
    Golub, G.H., Van Loan, C.F.: Matrix Computation, 2nd edn. John Hopkins University Press, Baltimore (1991)Google Scholar
  6. 6.
    Hansen, P.C.: Regularization, gsvd and truncated gsvd. BIT 29(3), 491–504 (1989)zbMATHCrossRefMathSciNetGoogle Scholar
  7. 7.
    Jain, A.K., Zongker, D.: Feature selection: Evaluation, application, and small sample performace. IEEE Transactions on Pattern Analysis and Machine Intelligence 19(2), 153–158 (1997)CrossRefGoogle Scholar
  8. 8.
    Kim, T., Hong. K.-S.: Estimating approximate shape and motion of deformable objects with a monocular view. In: Proc. Asian Conference on Computer Vision, Jeju Island, Korea (January 2004)Google Scholar
  9. 9.
    Kittler, J.: Feature selection and extraction. In: Young, T.Y., Fu, K.S. (eds.) HPRIP, Orlando, FL, pp. 59–83. Academic Press, London (1986)Google Scholar
  10. 10.
    Roy-Chowdhury, A.: A measure of deformability of shapes with applications to human motion analysis. In: IEEE Conference in Computer Vision and Pattern Recognition, vol. 1, pp. 398–404 (June 2005)Google Scholar
  11. 11.
    Sugaya, Y., Kanatani, K.: Multi-stage optimization for multi-body motion segmentation. IEICE Transactions on Information and Systems E87-D(7), 1935–1942 (2004)Google Scholar
  12. 12.
    Tomasi, C., Kanade, T.: Shape and motion from image streams under orthography: A factorization approach. International Journal in Computer Vision 9(2), 137–154 (1992)CrossRefGoogle Scholar
  13. 13.
    Torresani, L., Yang, D., Alexander, E., Bregler, C.: Tracking and modeling non-rigid objects with rank constraints. In: Proc. IEEE Conference on Computer Vision and Pattern Recognition, Kauai, Hawaii (2001)Google Scholar
  14. 14.
    Torresani, L., Hertzmann, A.: Automatic non-rigid 3d modeling from video. In: Proc. 8th European Conference on Computer Vision, Prague, Czech Republic, pp. 299–312 (May 2004)Google Scholar
  15. 15.
    Torresani, L., Hertzmann, A., Bregler, C.: Learning non-rigid 3d shape from 2d motion. In: Thrun, S., Saul, L., Schölkopf, B. (eds.) Advances in Neural Information Processing Systems, vol. 16, MIT Press, Cambridge (2004)Google Scholar
  16. 16.
    Vidal, R., Hartley, R.: Motion segmentation with missing data using powerfactorization and gpca. In: IEEE Conference on Computer Vision and Pattern Recognition, Washington D.C., vol. 2, pp. 310–316 (June 2004)Google Scholar
  17. 17.
    Xiao, J., Chai, J., Kanade, T.: A closed-form solution to non-rigid shape and motion recovery. In: Proc. 8th European Conference on Computer Vision, Prague, Czech Republic (May 2004)Google Scholar
  18. 18.
    Yezzi, A.J., Soatto, S.: Deformotion: Deforming motion, shape average and the joint registration and approximation of structures in images. International Journal of Computer Vision 53(2), 153–167 (2003)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  • Alessio Del Bue
    • 1
  • Xavier Lladó
    • 1
  • Lourdes Agapito
    • 1
  1. 1.Queen Mary University of LondonLondonUK

Personalised recommendations