3D Reconstruction of Non-Rigid Surfaces in Real-Time Using Wedge Elements

  • Antonio Agudo
  • Begoña Calvo
  • J. M. M. Montiel
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7583)


We present a new FEM (Finite Element Method) model for the 3D reconstruction of a deforming scene using as sole input a calibrated video sequence. Our approach extends the recently proposed 2D thin-plate FEM+EKF (Extended Kalman Filter) combination. Thin-plate FEM is an approximation that models a deforming 3D thin solid as a surface, and then discretizes the surface as a mesh of planar triangles. In contrast, we propose a full-fledged 3D FEM formulation where the deforming 3D solid is discretized as a mesh of 3D wedge elements. The new 3D FEM formulation provides better conditioning for the rank analysis stage necessary to remove the rigid boundary points from the formulation. We show how the proposed formulation accurately estimates deformable scenes from real imagery even for strong deformations. Crucially we also show, for the first time to the best of our knowledge, NRSfM (Non-Rigid Structure from Motion) at 30Hz real-time over real imagery. Real-time can be achieved for our 3D FEM formulation combined with an EKF resulting in accurate estimates even for small size maps.


Strong Deformation Bundle Adjustment Scene Point Visible Side Isometric Deformation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Agudo, A., Calvo, B., Montiel, J.M.M.: Finite element based sequential bayesian non-rigid structure from motion. In: CVPR (2012)Google Scholar
  2. 2.
    Zienkiewicz, O.C., Taylor, R.L.: The finite element method, vol. 1: Basic formulation and linear problems. McGraw-Hill, London (1989)Google Scholar
  3. 3.
    Bregler, C., Hertzmann, A., Biermann, H.: Recovering non-rigid 3D shape from image streams. In: CVPR (2000)Google Scholar
  4. 4.
    Paladini, M., Bartoli, A., Agapito, L.: Sequential Non-Rigid Structure-from-Motion with the 3D-Implicit Low-Rank Shape Model. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part II. LNCS, vol. 6312, pp. 15–28. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  5. 5.
    Bartoli, A., Gay-Bellile, V., Castellani, U., Peyras, J., Olsen, S., Sayd, P.: Coarse-to-fine low-rank structure-from-motion. In: CVPR (2008)Google Scholar
  6. 6.
    Del Bue, A., Llado, X., Agapito, L.: Non-rigid metric shape and motion recovery from uncalibrated images using priors. In: CVPR (2006)Google Scholar
  7. 7.
    Torresani, L., Hertzmann, A., Bregler, C.: Nonrigid structure-from motion: estimating shape and motion with hierarchical priors. PAMI, 878–892 (2008)Google Scholar
  8. 8.
    Varol, A., Salzmann, M., Tola, E., Fua, P.: Template-free monocular reconstruction of deformable surfaces. In: ICCV (2009)Google Scholar
  9. 9.
    Taylor, J., Jepson, A.D., Kutulakos, K.N.: Non-rigid structure from locally-rigid motion. In: CVPR (2010)Google Scholar
  10. 10.
    Fayad, J., Agapito, L., Del Bue, A.: Piecewise Quadratic Reconstruction of Non-Rigid Surfaces from Monocular Sequences. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part IV. LNCS, vol. 6314, pp. 297–310. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  11. 11.
    Salzmann, M., Fua, P.: Reconstructing sharply folding surfaces: A convex formulation. In: CVPR (2009)Google Scholar
  12. 12.
    Moreno-Noguer, F., Salzmann, M., Lepetit, V., Fua, R.: Capturing 3D stretchable surfaces from single images in closed from. In: CVPR (2009)Google Scholar
  13. 13.
    Ilić, S., Fua, P.: Non-linear beam model for tracking large deformation. In: ICCV (2007)Google Scholar
  14. 14.
    Greminger, M.A., Nelson, B.J.: Deformable object tracking using the boundary element method. In: CVPR (2003)Google Scholar
  15. 15.
    Agudo, A., Calvo, B., Montiel, J.M.M.: FEM models to code non-rigid EKF monocular SLAM. In: Workshop on Dynamic Shape Capture and Analysis (2011)Google Scholar
  16. 16.
    Civera, J., Davison, A.J., Montiel, J.M.M.: Inverse depth parametrization for monocular SLAM. IEEE Transactions on Robotics 24, 932–945 (2008)CrossRefGoogle Scholar
  17. 17.
    Stroud, A.H.: Approximate Calculation of Multiple Integrals. Prentice-Hall (1971)Google Scholar
  18. 18.
    Golub, G., Van Loan, C.: Matrix computations. Johns Hopkins Univ. Pr. (1996)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Antonio Agudo
    • 1
  • Begoña Calvo
    • 1
  • J. M. M. Montiel
    • 1
  1. 1.Instituto de Investigación en Ingeniería de AragónUniversidad de ZaragozaSpain

Personalised recommendations