A Unified View on Deformable Shape Factorizations

  • Roland Angst
  • Marc Pollefeys
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7577)


Multiple-view geometry and structure-from-motion are well established techniques to compute the structure of a moving rigid object. These techniques are all based on strong algebraic constraints imposed by the rigidity of the object. Unfortunately, many scenes of interest, e.g. faces or cloths, are dynamic and the rigidity constraint no longer holds. Hence, there is a need for non-rigid structure-from-motion (NRSfM) methods which can deal with dynamic scenes. A prominent framework to model deforming and moving non-rigid objects is the factorization technique where the measurements are assumed to lie in a low-dimensional subspace. Many different formulations and variations for factorization-based NRSfM have been proposed in recent years. However, due to the complex interactions between several subspaces, the distinguishing properties between two seemingly related approaches are often unclear. For example, do two approaches just vary in the optimization method used or is really a different model beneath?

In this paper, we show that these NRSfM factorization approaches are most naturally modeled with tensor algebra. This results in a clear presentation which subsumes many previous techniques. In this regard, this paper brings several strings of research together and provides a unified point of view. Moreover, the tensor formulation can be extended to the case of a camera network where multiple static affine cameras observe the same deforming and moving non-rigid object. Thanks to the insights gained through this tensor notation, a closed-form and an efficient iterative algorithm can be derived which provide a reconstruction even if there are no feature point correspondences at all between different cameras. An evaluation of the theory and algorithms on motion capture data show promising results.


Discrete Cosine Transform Basis Shape Tensor Formulation Multiple Camera Alternate Little Square 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Tomasi, C., Kanade, T.: Shape and motion from image streams under orthography: a factorization method. IJCV 9(2), 137–154 (1992)CrossRefGoogle Scholar
  2. 2.
    Costeira, J., Kanade, T.: A multi-body factorization method for motion analysis. In: Proc. ICCV, pp. 1071–1076 (June 1995)Google Scholar
  3. 3.
    Tron, R., Vidal, R.: A benchmark for the comparison of 3-D motion segmentation algorithms. In: Proc. CVPR. IEEE Computer Society (2007)Google Scholar
  4. 4.
    Yan, J., Pollefeys, M.: A factorization-based approach for articulated nonrigid shape, motion and kinematic chain recovery from video. TPAMI 30(5), 865–877 (2008)CrossRefGoogle Scholar
  5. 5.
    Tresadern, P.A., Reid, I.D.: Articulated structure from motion by factorization. In: Proc. CVPR, pp. 1110–1115. IEEE Computer Society (2005)Google Scholar
  6. 6.
    Bregler, C., Hertzmann, A., Biermann, H.: Recovering non-rigid 3D shape from image streams. In: Proc. CVPR, pp. 2690–2696. IEEE Computer Society (2000)Google Scholar
  7. 7.
    Brand, M.: Morphable 3D models from video. In: Proc. CVPR, pp. 456–463. IEEE Computer Society (2001)Google Scholar
  8. 8.
    Xiao, J., Chai, J.-X., Kanade, T.: A Closed-Form Solution to Non-rigid Shape and Motion Recovery. In: Pajdla, T., Matas, J. (eds.) ECCV 2004, Part IV. LNCS, vol. 3024, pp. 573–587. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  9. 9.
    Brand, M.: A direct method for 3D factorization of nonrigid motion observed in 2D. In: CVPR (2), pp. 122–128. IEEE Computer Society (2005)Google Scholar
  10. 10.
    Fayad, J., Russell, C., de Agapito, L.: Automated articulated structure and 3D shape recovery from point correspondences. In: Proc. ICCV, pp. 431–438 (2011)Google Scholar
  11. 11.
    Torresani, L., Hertzmann, A., Bregler, C.: Nonrigid structure-from-motion: Estimating shape and motion with hierarchical priors. TPAMI 30(5), 878–892 (2008)CrossRefGoogle Scholar
  12. 12.
    Paladini, M., Bartoli, A., Agapito, L.: Sequential Non-Rigid Structure-from-Motion with the 3D-Implicit Low-Rank Shape Model. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part II. LNCS, vol. 6312, pp. 15–28. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  13. 13.
    Akhter, I., Sheikh, Y., Khan, S., Kanade, T.: Nonrigid structure from motion in trajectory space. In: Proc. NIPS, pp. 41–48 (2008)Google Scholar
  14. 14.
    Akhter, I., Sheikh, Y., Khan, S., Kanade, T.: Trajectory space: A dual representation for nonrigid structure from motion. TPAMI 33(7), 1442–1456 (2011)CrossRefGoogle Scholar
  15. 15.
    Wolf, L., Zomet, A.: Correspondence-free synchronization and reconstruction in a non-rigid scene. In: Workshop on Vision and Modeling of Dynamic Scenes (2002)Google Scholar
  16. 16.
    Angst, R., Pollefeys, M.: Static multi-camera factorization using rigid motion. In: Proc. ICCV. IEEE Computer Society, Washington, DC (2009)Google Scholar
  17. 17.
    Zaheer, A., Akhter, I., Baig, M.H., Marzban, S., Khan, S.: Multiview structure from motion in trajectory space. In: Proc. ICCV, pp. 2447–2453 (2011)Google Scholar
  18. 18.
    Bue, A.D., de Agapito, L.: Non-rigid stereo factorization. IJCV 66(2), 193–207 (2006)CrossRefGoogle Scholar
  19. 19.
    Lladó, X., Del Bue, A., Oliver, A., Salvi, J., de Agapito, L.: Reconstruction of nonrigid 3D shapes from stereo-motion. Pattern Recogn. Lett. 32, 1020–1028 (2011)CrossRefGoogle Scholar
  20. 20.
    Kolda, T.G., Bader, B.W.: Tensor Decompositions and Applications. SIAM Review 51(3), 455–500 (2009)MathSciNetzbMATHCrossRefGoogle Scholar
  21. 21.
    Olsen, S.I., Bartoli, A.: Implicit non-rigid structure-from-motion with priors. J. Math. Imag. Vision 31, 233–244 (2008)MathSciNetCrossRefGoogle Scholar
  22. 22.
    Gotardo, P.F.U., Martinez, A.M.: Non-rigid structure from motion with complementary rank-3 spaces. In: Proc. CVPR, pp. 3065–3072. IEEE (2011)Google Scholar
  23. 23.
    Torresani, L., Yang, D.B., Alexander, E.J., Bregler, C.: Tracking and modeling non-rigid objects with rank constraints. In: Proc. CVPR, pp. 493–500 (2001)Google Scholar
  24. 24.
    Irani, M.: Multi-frame optical flow estimation using subspace constraints. In: Proc. ICCV, pp. 626–633 (1999)Google Scholar
  25. 25.
    Hartley, R., Vidal, R.: Perspective Nonrigid Shape and Motion Recovery. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part I. LNCS, vol. 5302, pp. 276–289. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  26. 26.
    Angst, R., Pollefeys, M.: A unified view on deformable shape factorization: Supplemental material (2012),
  27. 27.
    Park, H.S., Shiratori, T., Matthews, I., Sheikh, Y.: 3D Reconstruction of a Moving Point from a Series of 2D Projections. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part III. LNCS, vol. 6313, pp. 158–171. Springer, Heidelberg (2010)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Roland Angst
    • 1
  • Marc Pollefeys
    • 1
  1. 1.Computer Vision and Geometry Lab, Department of Computer ScienceETH ZürichZürichSwitzerland

Personalised recommendations