A Closed-Form Solution to Non-rigid Shape and Motion Recovery

  • Jing Xiao
  • Jin-xiang Chai
  • Takeo Kanade
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3024)

Abstract

Recovery of three dimensional (3D) shape and motion of non-static scenes from a monocular video sequence is important for applications like robot navigation and human computer interaction. If every point in the scene randomly moves, it is impossible to recover the non-rigid shapes. In practice, many non-rigid objects, e.g. the human face under various expressions, deform with certain structures. Their shapes can be regarded as a weighted combination of certain shape bases. Shape and motion recovery under such situations has attracted much interest. Previous work on this problem [6,4,13] utilized only orthonormality constraints on the camera rotations (rotation constraints). This paper proves that using only the rotation constraints results in ambiguous and invalid solutions. The ambiguity arises from the fact that the shape bases are not unique because their linear transformation is a new set of eligible bases. To eliminate the ambiguity, we propose a set of novel constraints, basis constraints, which uniquely determine the shape bases. We prove that, under the weak-perspective projection model, enforcing both the basis and the rotation constraints leads to a closed-form solution to the problem of non-rigid shape and motion recovery. The accuracy and robustness of our closed-form solution is evaluated quantitatively on synthetic data and qualitatively on real video sequences.

Keywords

Reconstruction Error Shape Base Dynamic Scene Basis Constraint Motion Recovery 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Baker, S., Matthews, I.: Equivalence and Efficiency of Image Alignment Algorithms. In: Proc. Int. Conf. Computer Vision and Pattern Recognition (2001)Google Scholar
  2. 2.
    Bascle, B., Blake, A.: Separability of Pose and Expression in Facial Tracing and Animation. In: Proc. Int. Conf. Computer Vision, pp. 323–328 (1998)Google Scholar
  3. 3.
    Blanz, V., Vetter, T.: A morphable model for the synthesis of 3D faces. In: Proc. SIGGRAPH 1999, pp. 187–194 (1999)Google Scholar
  4. 4.
    Brand, M.: Morphable 3D Models from Video. In: Proc. Int. Conf. Computer Vision and Pattern Recognition (2001)Google Scholar
  5. 5.
    Brand, M., Bhotika, R.: Flexible Flow for 3D Nonrigid Tracking and Shape Recovery. In: Proc. Int. Conf. Computer Vision and Pattern Recognition (2001)Google Scholar
  6. 6.
    Bregler, C., Hertzmann, A., Biermann, H.: Recovering Non-Rigid 3D Shape from Image Streams. In: Proc. Int. Conf. Computer Vision and Pattern Recognition (2000)Google Scholar
  7. 7.
    Chai, J., Xiao, J., Hodgins, J.: Vision-based Control of 3D Facial Animation. In: Eurographics/ACM Symposium on Computer Animation (2003)Google Scholar
  8. 8.
    Costeira, J., Kanade, T.: A multibody factorization method for independently moving-objects. Int. Journal of Computer Vision 29(3), 159–179 (1998)CrossRefGoogle Scholar
  9. 9.
    Gokturk, S.B., Bouguet, J.Y., Grzeszczuk, R.: A data driven model for monocular face tracking. In: Proc. Int. Conf. Computer Vision (2001)Google Scholar
  10. 10.
    Han, M., Kanade, T.: Reconstruction of a Scene with Multiple Linearly Moving Objects. Proc. Int. Conf. Computer Vision and Pattern Recognition (2000)Google Scholar
  11. 11.
    Poelman, C., Kanade, T.: A paraperspective factorization method for shape and motion recovery. IEEE Trans. Pattern Analysis and Machine Intelligence 19(3), 206–218 (1997)CrossRefGoogle Scholar
  12. 12.
    Tomasi, C., Kanade, T.: Shape and motion from image streams under orthography: A factorization method. Int. Journal of Computer Vision 9(2), 137–154 (1992)CrossRefGoogle Scholar
  13. 13.
    Torresani, L., Yang, D., Alexander, G., Bregler, C.: Tracking and Modeling Non- Rigid Objects with Rank Constraints. In: Proc. Int. Conf. Computer Vision and Pattern Recognition (2001)Google Scholar
  14. 14.
    Triggs, B.: Factorization Methods for Projective Structure and Motion. Proc. Int. Conf. Computer Vision and Pattern Recognition (1996)Google Scholar
  15. 15.
    Vidal, R., Soatto, S., Ma, Y., Sastry, S.: Segmentation of Dynamic Scenes from the Multibody Fundamental Matrix. In: ECCV Workshop on Vision and Modeling of Dynamic Scenes (2002)Google Scholar
  16. 16.
    Wolf, L., Shashua, A.: Two-body Segmentation from Two Perspective Views. In: Proc. Int. Conf. Computer Vision and Pattern Recognition (2001)Google Scholar
  17. 17.
    Wolf, L., Shashua, A.: On Projection Matrices Pk → P2, k = 3,., 6, and their Applications in Computer Vision. Int. Journal of Computer Vision 48(1), 53–67 (2002)MATHCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2004

Authors and Affiliations

  • Jing Xiao
    • 1
  • Jin-xiang Chai
    • 1
  • Takeo Kanade
    • 1
  1. 1.The Robotics InstituteCarnegie Mellon UniversityPittsburghUSA

Personalised recommendations