Perspective Nonrigid Shape and Motion Recovery

  • Richard Hartley
  • René Vidal
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5302)


We present a closed form solution to the nonrigid shape and motion (NRSM) problem from point correspondences in multiple perspective uncalibrated views. Under the assumption that the nonrigid object deforms as a linear combination of K rigid shapes, we show that the NRSM problem can be viewed as a reconstruction problem from multiple projections from ℙ3K to ℙ2. Therefore, one can linearly solve for the projection matrices by factorizing a multifocal tensor. However, this projective reconstruction in ℙ3K does not satisfy the constraints of the NRSM problem, because it is computed only up to a projective transformation in ℙ3K . Our key contribution is to show that, by exploiting algebraic dependencies among the entries of the projection matrices, one can upgrade the projective reconstruction to determine the affine configuration of the points in ℝ3, and the motion of the camera relative to their centroid. Moreover, if K ≥ 2, then either by using calibrated cameras, or by assuming a camera with fixed internal parameters, it is possible to compute the Euclidean structure by a closed form method.


  1. 1.
    Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision, 2nd edn. Cambridge (2004)Google Scholar
  2. 2.
    Ma, Y., Soatto, S., Kosecka, J., Sastry, S.: An Invitation to 3D Vision: From Images to Geometric Models. Springer, Heidelberg (2003)zbMATHGoogle Scholar
  3. 3.
    Brand, M.: Morphable 3D models from video. In: Conference on Computer Vision and Pattern Recognition, pp. 456–463 (2001)Google Scholar
  4. 4.
    Brand, M., Bhotika, R.: Flexible flow for 3D nonrigid tracking and shape recovery. In: Conference on Computer Vision and Pattern Recognition, pp. 315–322 (2001)Google Scholar
  5. 5.
    Bregler, C., Hertzmann, A., Biermann, H.: Recovering non-rigid 3D shape from image streams. In: Conference on Computer Vision and Pattern Recognition, pp. 2690–2696 (2000)Google Scholar
  6. 6.
    Torresani, L., Bregler, C.: Space-time tracking. In: European Conference on Computer Vision, pp. 801–812 (2002)Google Scholar
  7. 7.
    Torresani, L., Yang, D., Alexander, E., Bregler, C.: Tracking and modeling non-rigid objects with rank constraints. In: Conference on Computer Vision and Pattern Recognition, pp. 493–500 (2001)Google Scholar
  8. 8.
    Xiao, J., Chai, J., Kanade, T.: A closed-form solution to non-rigid shape and motion recovery. International Journal of Computer Vision 67, 233–246 (2006)CrossRefzbMATHGoogle Scholar
  9. 9.
    Tomasi, C., Kanade, T.: Shape and motion from image streams under orthography. International Journal of Computer Vision 9, 137–154 (1992)CrossRefGoogle Scholar
  10. 10.
    Xiao, J., Kanade, T.: Non-rigid shape and motion recovery: Degenerate deformations. In: Conference on Computer Vision and Pattern Recognition, pp. 668–675 (2004)Google Scholar
  11. 11.
    Sturm, P., Triggs, B.: A factorization based algorithm for multi–image projective structure and motion. In: European Conference on Computer Vision, pp. 709–720 (1996)Google Scholar
  12. 12.
    Xiao, J., Kanade, T.: Uncalibrated perspective reconstruction of deformable structures. In: IEEE International Conference on Computer Vision, pp. 1075–1082 (2005)Google Scholar
  13. 13.
    Vidal, R., Abretske, D.: Nonrigid shape and motion from multiple perspective views. In: European Conference on Computer Vision, pp. 205–218 (2006)Google Scholar
  14. 14.
    Hartley, R., Schaffalitzky, F.: Reconstruction from projections using Grassmann tensors. In: European Conference on Computer Vision, pp. 363–375 (2004)Google Scholar
  15. 15.
    Oliensis, J., Hartley, R.: Iterative extensions of the Sturm/Triggs algorithm: convergence and nonconvergence. IEEE Transactions on Pattern Analysis and Machine Intelligence 29, 2217–2233 (2007)CrossRefGoogle Scholar
  16. 16.
    Aanaes, H., Kahl, F.: Estimation of deformable structure and motion. In: ECCV Workshop on Vision and Modelling of Dynamic Scenes (2002)Google Scholar
  17. 17.
    Mahamud, S., Hebert, M., Omori, Y., Ponce, J.: Provably-convergent iterative methods for projective structure from motion. In: Conference on Computer Vision and Pattern Recognition, vol. I, pp. 1018–1025 (2001)Google Scholar
  18. 18.
    Costeira, J., Kanade, T.: A multibody factorization method for independently moving objects. International Journal of Computer Vision 29, 159–179 (1998)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Richard Hartley
    • 1
  • René Vidal
    • 2
  1. 1.Australian National University and NICTA, Canberra, ACTAustralia
  2. 2.Center for Imaging ScienceJohns Hopkins UniversityBaltimoreUSA

Personalised recommendations