Exploring Ambiguities for Monocular Non-rigid Shape Estimation

  • Francesc Moreno-Noguer
  • Josep M. Porta
  • Pascal Fua
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6313)


Recovering the 3D shape of deformable surfaces from single images is difficult because many different shapes have very similar projections. This is commonly addressed by restricting the set of possible shapes to linear combinations of deformation modes and by imposing additional geometric constraints. Unfortunately, because image measurements are noisy, such constraints do not always guarantee that the correct shape will be recovered. To overcome this limitation, we introduce an efficient approach to exploring the set of solutions of an objective function based on point-correspondences and to proposing a small set of candidate 3D shapes. This allows the use of additional image information to choose the best one. As a proof of concept, we use either motion or shading cues to this end and show that we can handle a complex objective function without having to solve a difficult non-linear minimization problem.


Reconstruction Error Reprojection Error Mesh Edge Wave Sequence Candidate Shape 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Perriollat, M., Hartley, R., Bartoli, A.: Monocular template-based reconstruction of inextensible surfaces. In: BMVC (2008)Google Scholar
  2. 2.
    Salzmann, M., Fua, P.: Reconstructing sharply folding surfaces: A convex formulation. In: CVPR (2009)Google Scholar
  3. 3.
    Zhu, J., Hoi, S., Xu, Z., Lyu, M.: An effective approach to 3d deformable surface tracking. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part III. LNCS, vol. 5304, pp. 766–779. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  4. 4.
    Salzmann, M., Moreno-Noguer, F., Lepetit, V., Fua, P.: Closed-form solution to non-rigid 3d surface registration. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part IV. LNCS, vol. 5305, pp. 581–594. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  5. 5.
    Moreno-Noguer, F., Salzmann, M., Lepetit, V., Fua, P.: Capturing 3d stretchable surfaces from single images in closed form. In: CVPR (2009)Google Scholar
  6. 6.
    Ecker, A., Jepson, A.D., Kutulakos, K.N.: Semidefinite programming heuristics for surface reconstruction ambiguities. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part I. LNCS, vol. 5302, pp. 127–140. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  7. 7.
    Cohen, L., Cohen, I.: Finite-element methods for active contour models and balloons for 2-d and 3-d images. PAMI 15 (1993)Google Scholar
  8. 8.
    McInerney, T., Terzopoulos, D.: A finite element model for 3d shape reconstruction and nonrigid motion tracking. In: ICCV (1993)Google Scholar
  9. 9.
    Metaxas, D., Terzopoulos, D.: Constrained deformable superquadrics and nonrigid motion tracking. PAMI 15 (1993)Google Scholar
  10. 10.
    Blanz, V., Vetter, T.: A morphable model for the synthesis of 3–d faces. In: SIGGRAPH (1999)Google Scholar
  11. 11.
    Cootes, T., Edwards, G., Taylor, C.: Active appearance models. In: Burkhardt, H., Neumann, B. (eds.) ECCV 1998. LNCS, vol. 1407, p. 484. Springer, Heidelberg (1998)CrossRefGoogle Scholar
  12. 12.
    Gumerov, N., Zandifar, A., Duraiswami, R., Davis, L.: Structure of applicable surfaces from single views. In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004. LNCS, vol. 3023, pp. 482–496. Springer, Heidelberg (2004)Google Scholar
  13. 13.
    Liang, J., DeMenthon, D., Doermann, D.: Flattening curved documents in images. In: CVPR (2005)Google Scholar
  14. 14.
    Wang, Y., Liu, Z., Hua, G., Wen, Z., Zhang, Z., Samaras, D.: Face re-lighting from a single image under harsh lighting conditions. In: CVPR (2007)Google Scholar
  15. 15.
    White, R., Forsyth, D.: Combining cues: Shape from shading and texture. In: CVPR (2006)Google Scholar
  16. 16.
    Ruanaidh, J., Fitzgerald, W.: Numerical Bayesian Methods Applied to Signal Processing. Springer, Heidelberg (1996)zbMATHGoogle Scholar
  17. 17.
    Hamerly, G., Elkan, C.: Learning the k in k-means. In: NIPS (2003)Google Scholar
  18. 18.
    Hara, K., Nishino, K., Ikeuchi, K.: Light source position and reflectance estimation from a single view without the distant illumination assumption. PAMI (2005)Google Scholar
  19. 19.
    Blake, A., Isard, M.: Active contours. Springer, Heidelberg (1998)Google Scholar
  20. 20.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. IJCV (2004)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Francesc Moreno-Noguer
    • 1
  • Josep M. Porta
    • 1
  • Pascal Fua
    • 2
  1. 1.Institut de Robòtica i Informàtica IndustrialCSIC-UPCBarcelonaSpain
  2. 2.Computer Vision LaboratoryEPFLLausanneSwitzerland

Personalised recommendations