Abstract
Multiple-view geometry and structure-from-motion are well established techniques to compute the structure of a moving rigid object. These techniques are all based on strong algebraic constraints imposed by the rigidity of the object. Unfortunately, many scenes of interest, e.g. faces or cloths, are dynamic and the rigidity constraint no longer holds. Hence, there is a need for non-rigid structure-from-motion (NRSfM) methods which can deal with dynamic scenes. A prominent framework to model deforming and moving non-rigid objects is the factorization technique where the measurements are assumed to lie in a low-dimensional subspace. Many different formulations and variations for factorization-based NRSfM have been proposed in recent years. However, due to the complex interactions between several subspaces, the distinguishing properties between two seemingly related approaches are often unclear. For example, do two approaches just vary in the optimization method used or is really a different model beneath?
In this paper, we show that these NRSfM factorization approaches are most naturally modeled with tensor algebra. This results in a clear presentation which subsumes many previous techniques. In this regard, this paper brings several strings of research together and provides a unified point of view. Moreover, the tensor formulation can be extended to the case of a camera network where multiple static affine cameras observe the same deforming and moving non-rigid object. Thanks to the insights gained through this tensor notation, a closed-form and an efficient iterative algorithm can be derived which provide a reconstruction even if there are no feature point correspondences at all between different cameras. An evaluation of the theory and algorithms on motion capture data show promising results.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Tomasi, C., Kanade, T.: Shape and motion from image streams under orthography: a factorization method. IJCV 9(2), 137–154 (1992)
Costeira, J., Kanade, T.: A multi-body factorization method for motion analysis. In: Proc. ICCV, pp. 1071–1076 (June 1995)
Tron, R., Vidal, R.: A benchmark for the comparison of 3-D motion segmentation algorithms. In: Proc. CVPR. IEEE Computer Society (2007)
Yan, J., Pollefeys, M.: A factorization-based approach for articulated nonrigid shape, motion and kinematic chain recovery from video. TPAMI 30(5), 865–877 (2008)
Tresadern, P.A., Reid, I.D.: Articulated structure from motion by factorization. In: Proc. CVPR, pp. 1110–1115. IEEE Computer Society (2005)
Bregler, C., Hertzmann, A., Biermann, H.: Recovering non-rigid 3D shape from image streams. In: Proc. CVPR, pp. 2690–2696. IEEE Computer Society (2000)
Brand, M.: Morphable 3D models from video. In: Proc. CVPR, pp. 456–463. IEEE Computer Society (2001)
Xiao, J., Chai, J.-X., Kanade, T.: A Closed-Form Solution to Non-rigid Shape and Motion Recovery. In: Pajdla, T., Matas, J. (eds.) ECCV 2004, Part IV. LNCS, vol. 3024, pp. 573–587. Springer, Heidelberg (2004)
Brand, M.: A direct method for 3D factorization of nonrigid motion observed in 2D. In: CVPR (2), pp. 122–128. IEEE Computer Society (2005)
Fayad, J., Russell, C., de Agapito, L.: Automated articulated structure and 3D shape recovery from point correspondences. In: Proc. ICCV, pp. 431–438 (2011)
Torresani, L., Hertzmann, A., Bregler, C.: Nonrigid structure-from-motion: Estimating shape and motion with hierarchical priors. TPAMI 30(5), 878–892 (2008)
Paladini, M., Bartoli, A., Agapito, L.: Sequential Non-Rigid Structure-from-Motion with the 3D-Implicit Low-Rank Shape Model. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part II. LNCS, vol. 6312, pp. 15–28. Springer, Heidelberg (2010)
Akhter, I., Sheikh, Y., Khan, S., Kanade, T.: Nonrigid structure from motion in trajectory space. In: Proc. NIPS, pp. 41–48 (2008)
Akhter, I., Sheikh, Y., Khan, S., Kanade, T.: Trajectory space: A dual representation for nonrigid structure from motion. TPAMI 33(7), 1442–1456 (2011)
Wolf, L., Zomet, A.: Correspondence-free synchronization and reconstruction in a non-rigid scene. In: Workshop on Vision and Modeling of Dynamic Scenes (2002)
Angst, R., Pollefeys, M.: Static multi-camera factorization using rigid motion. In: Proc. ICCV. IEEE Computer Society, Washington, DC (2009)
Zaheer, A., Akhter, I., Baig, M.H., Marzban, S., Khan, S.: Multiview structure from motion in trajectory space. In: Proc. ICCV, pp. 2447–2453 (2011)
Bue, A.D., de Agapito, L.: Non-rigid stereo factorization. IJCV 66(2), 193–207 (2006)
Lladó, X., Del Bue, A., Oliver, A., Salvi, J., de Agapito, L.: Reconstruction of nonrigid 3D shapes from stereo-motion. Pattern Recogn. Lett. 32, 1020–1028 (2011)
Kolda, T.G., Bader, B.W.: Tensor Decompositions and Applications. SIAM Review 51(3), 455–500 (2009)
Olsen, S.I., Bartoli, A.: Implicit non-rigid structure-from-motion with priors. J. Math. Imag. Vision 31, 233–244 (2008)
Gotardo, P.F.U., Martinez, A.M.: Non-rigid structure from motion with complementary rank-3 spaces. In: Proc. CVPR, pp. 3065–3072. IEEE (2011)
Torresani, L., Yang, D.B., Alexander, E.J., Bregler, C.: Tracking and modeling non-rigid objects with rank constraints. In: Proc. CVPR, pp. 493–500 (2001)
Irani, M.: Multi-frame optical flow estimation using subspace constraints. In: Proc. ICCV, pp. 626–633 (1999)
Hartley, R., Vidal, R.: Perspective Nonrigid Shape and Motion Recovery. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part I. LNCS, vol. 5302, pp. 276–289. Springer, Heidelberg (2008)
Angst, R., Pollefeys, M.: A unified view on deformable shape factorization: Supplemental material (2012), http://www.inf.ethz.ch/personal/rangst/publications.php
Park, H.S., Shiratori, T., Matthews, I., Sheikh, Y.: 3D Reconstruction of a Moving Point from a Series of 2D Projections. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part III. LNCS, vol. 6313, pp. 158–171. Springer, Heidelberg (2010)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Angst, R., Pollefeys, M. (2012). A Unified View on Deformable Shape Factorizations. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds) Computer Vision – ECCV 2012. ECCV 2012. Lecture Notes in Computer Science, vol 7577. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-33783-3_49
Download citation
DOI: https://doi.org/10.1007/978-3-642-33783-3_49
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-33782-6
Online ISBN: 978-3-642-33783-3
eBook Packages: Computer ScienceComputer Science (R0)