Abstract
Recovery of three dimensional (3D) shape and motion of non-static scenes from a monocular video sequence is important for applications like robot navigation and human computer interaction. If every point in the scene randomly moves, it is impossible to recover the non-rigid shapes. In practice, many non-rigid objects, e.g. the human face under various expressions, deform with certain structures. Their shapes can be regarded as a weighted combination of certain shape bases. Shape and motion recovery under such situations has attracted much interest. Previous work on this problem (Bregler, C., Hertzmann, A., and Biermann, H. 2000. In Proc. Int. Conf. Computer Vision and Pattern Recognition; Brand, M. 2001. In Proc. Int. Conf. Computer Vision and Pattern Recognition; Torresani, L., Yang, D., Alexander, G., and Bregler, C. 2001. In Proc. Int. Conf. Computer Vision and Pattern Recognition) utilized only orthonormality constraints on the camera rotations (rotation constraints). This paper proves that using only the rotation constraints results in ambiguous and invalid solutions. The ambiguity arises from the fact that the shape bases are not unique. An arbitrary linear transformation of the bases produces another set of eligible bases. To eliminate the ambiguity, we propose a set of novel constraints, basis constraints, which uniquely determine the shape bases. We prove that, under the weak-perspective projection model, enforcing both the basis and the rotation constraints leads to a closed-form solution to the problem of non-rigid shape and motion recovery. The accuracy and robustness of our closed-form solution is evaluated quantitatively on synthetic data and qualitatively on real video sequences.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Baker, S. and Matthews, I. 2001. Equivalence and efficiency of image alignment algorithms. In Proc. Int. Conf. Computer Vision and Pattern Recognition.
Bascle, B. and Blake, A. 1998. Separability of pose and expression in facial tracing and animation. In Proc. Int. Conf. Computer Vision, pp. 323–328.
Blanz, V. and Vetter, T. 1999. A morphable model for the synthesis of 3D faces. In Proc. SIGGRAPH’99, pp. 187–194.
Brand, M. 2001. Morphable 3D models from video. In Proc. Int. Conf. Computer Vision and Pattern Recognition.
Brand, M. and Bhotika, R. 2001. Flexible flow for 3D nonrigid tracking and shape recovery. In Proc. Int. Conf. Computer Vision and Pattern Recognition.
Bregler, C., Hertzmann, A. and Biermann, H. 2000. Recovering non-rigid 3D shape from image streams. In Proc. Int. Conf. Computer Vision and Pattern Recognition.
Chai, J., Xiao, J., and Hodgins, J. 2003. Vision-based control of 3D facial animation. Eurographics/ACM Symposium on Computer Animation.
Costeira, J. and Kanade, T. 1998. A multibody factorization method for independently moving-objects. Int. Journal of Computer Vision, 29(3):159–179.
Gokturk, S.B., Bouguet, J.Y., and Grzeszczuk, R. 2001. A data driven model for monocular face tracking. In Proc. Int. Conf. Computer Vision.
Han, M. and Kanade, T. 2000. Reconstruction of a scene with multiple linearly moving objects. In Proc. Int. Conf. Computer Vision and Pattern Recognition.
Hartley, R. I. and Zisserman, A. 2000. Multiple View Geometry in Computer Vision. Cambridge University Press.
Kanatani, K. 2001. Motion segmentation by subspace separation and model selection. In Proc. Int. Conf. Computer Vision.
Mahamud, S. and Hebert, M. 2000. Iterative projective reconstruction from multiple views. In Proc. Int. Conf. Computer Vision and Pattern Recognition.
Poelman, C. and Kanade, T. 1997. A paraperspective factorization method for shape and motion recovery, IEEE Trans. Pattern Analysis and Machine Intelligence, 19(3):206–218.
Schönemann, P.H. 1966. A generalized solution of the orthogonal procrustes problem. Psychometrika, 31(1):1–10.
Sugaya, Y. and Kanatani, K. 2004. Geometric structure of degeneracy for multi-body motion segmentation. ECCV Workshop on Statistical Methods in Video Processing.
Tomasi, C. and Kanade, T. 1992. Shape and motion from image streams under orthography: A factorization method. Int. Journal of Computer Vision, 9(2):137–154.
Torresani, L., Yang, D., Alexander, G., and Bregler, C. 2001. Tracking and modeling non-rigid objects with rank constraints. In Proc. Int. Conf. Computer Vision and Pattern Recognition.
Triggs, B. 1996. Factorization methods for projective structure and motion. In Proc. Int. Conf. Computer Vision and Pattern Recognition.
Vidal, R., Soatto, S., Ma, Y., and Sastry, S. 2002. Segmentation of dynamic scenes from the multibody fundamental matrix. ECCV Workshop on Vision and Modeling of Dynamic Scenes.
Vidal, R. and Hartley, R. 2004. Motion segmentation with missing data using power factorization and GPCA. In Proc. Int. Conf. Computer Vision and Pattern Recognition.
Wolf, L. and Shashua, A. 2001. Two-body segmentation from two perspective views. In Proc. Int. Conf. Computer Vision and Pattern Recognition.
Wolf, L. and Shashua, A. 2002. On projection matrices Pκ → P2,κ = 3,...,6, and their applications in computer vision. Int. J. of Computer Vision, 48(1):53–67.
Zelnik-Manor, L. and Irani, M. 2003. Degeneracies, dependencies, and their implications in multi-body and multi-sequence factorizations. In Proc. Int. Conf. Computer Vision and Pattern Recognition.
Zhang, Z., Deriche, R., Faugeras, O., and Luong, Q. 1995. A robust technique for matching two uncalibrated images through the recovery of the unknown epipolar geometry, Artificial Intelligence, 78(1/2):87–119.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Xiao, J., Chai, J. & Kanade, T. A Closed-Form Solution to Non-Rigid Shape and Motion Recovery. Int J Comput Vision 67, 233–246 (2006). https://doi.org/10.1007/s11263-005-3962-9
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11263-005-3962-9