On Mean Pose and Variability of 3D Deformable Models

  • Benjamin Allain
  • Jean-Sébastien Franco
  • Edmond Boyer
  • Tony Tung
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8690)


We present a novel methodology for the analysis of complex object shapes in motion observed by multiple video cameras. In particular, we propose to learn local surface rigidity probabilities (i.e., deformations), and to estimate a mean pose over a temporal sequence. Local deformations can be used for rigidity-based dynamic surface segmentation, while a mean pose can be used as a sequence keyframe or a cluster prototype and has therefore numerous applications, such as motion synthesis or sequential alignment for compression or morphing. We take advantage of recent advances in surface tracking techniques to formulate a generative model of 3D temporal sequences using a probabilistic framework, which conditions shape fitting over all frames to a simple set of intrinsic surface rigidity properties. Surface tracking and rigidity variable estimation can then be formulated as an Expectation-Maximization inference problem and solved by alternatively minimizing two nested fixed point iterations. We show that this framework provides a new fundamental building block for various applications of shape analysis, and achieves comparable tracking performance to state of the art surface tracking techniques on real datasets, even compared to approaches using strong kinematic priors such as rigid skeletons.


Shape dynamics Motion analysis Shape spaces 


  1. 1.
    de Aguiar, E., Stoll, C., Theobalt, C., Ahmed, N., Seidel, H.P., Thrun, S.: Performance capture from sparse multi-view video. ACM Transactions on Graphics 27(3) (2008)Google Scholar
  2. 2.
    Allard, J., Ménier, C., Raffin, B., Boyer, E., Faure, F.: Grimage: Markerless 3d interactions. SIGGRAPH - Emerging Technologies (2007)Google Scholar
  3. 3.
    Anguelov, D., Srinivasan, P., Koller, D., Thrun, S., Rodgers, J., Davis, J.: Scape: Shape completion and animation of people. ACM Transactions on Graphics 24(3) (2005)Google Scholar
  4. 4.
    Ballan, L., Cortelazzo, G.M.: Marker-less motion capture of skinned models in a four camera set-up using optical flow and silhouettes. In: 3DPVT (2008)Google Scholar
  5. 5.
    Baran, I., Popović, J.: Automatic rigging and animation of 3D characters. ACM Transactions on Graphics 26(3), 72:1–72:8 (2007)Google Scholar
  6. 6.
    Bishop, C.M.: Pattern Recognition and Machine Learning (Information Science and Statistics). Springer-Verlag New York, Inc., Secaucus (2006)Google Scholar
  7. 7.
    Botsch, M., Pauly, M., Wicke, M., Gross, M.: Adaptive space deformations based on rigid cells. Comput. Graph. Forum 26(3), 339–347 (2007)CrossRefGoogle Scholar
  8. 8.
    Bronstein, A.M., Bronstein, M.M., Kimmel, R.: Efficient computation of isometry-invariant distances between surfaces. SIAM Journal on Scientific Computing 28 (2006)Google Scholar
  9. 9.
    Cagniart, C., Boyer, E., Ilic, S.: Probabilistic deformable surface tracking from multiple videos. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part IV. LNCS, vol. 6314, pp. 326–339. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  10. 10.
    Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the em algorithm. Journal of the Royal Statistical Society, Series B (1977)Google Scholar
  11. 11.
    Franco, J., Boyer, E.: Learning temporally consistent rigidities. In: CVPR (2011)Google Scholar
  12. 12.
    Franco, J., Menier, C., Boyer, E., Raffin, B.: A distributed approach for real-time 3d modeling. In: CVPR Workshop (2004)Google Scholar
  13. 13.
    Fréchet, M.: Les éléments aléatoires de nature quelconque dans un espace distancié. Annales de l’institut Henri Poincaré 10, 215–310 (1948)Google Scholar
  14. 14.
    Granger, S., Pennec, X.: Multi-scale EM-ICP: A fast and robust approach for surface registration. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002, Part IV. LNCS, vol. 2353, pp. 418–432. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  15. 15.
    Hasler, N., Ackermann, H., Rosenhahn, B., Thormählen, T., Seidel, H.P.: Multilinear pose and body shape estimation of dressed subjects from image sets. In: CVPR (2010)Google Scholar
  16. 16.
    Hufnagel, H., Pennec, X., Ehrhardt, J., Ayache, N., Handel, H.: Generation of a Statistical Shape Model with Probabilistic Point Correspondences and EM-ICP. IJCAR 2(5) (2008)Google Scholar
  17. 17.
    Gall, J., Stoll, C., de Aguiar, E., Theobalt, C., Rosenhahn, B., Seidel, H.P.: Motion capture using joint skeleton tracking and surface estimation. In: CVPR (2009)Google Scholar
  18. 18.
    Kendall, D.: Shape manifolds, procrustean metrics, and complex projective spaces. Bulletin of the London Mathematical Society 16(2), 81–121 (1984)CrossRefzbMATHMathSciNetGoogle Scholar
  19. 19.
    Liu, Y., Gall, J., Stoll, C., Dai, Q., Seidel, H.P., Theobalt, C.: Markerless motion capture of multiple characters using multi-view image segmentation. PAMI (2013)Google Scholar
  20. 20.
    Matsuyama, T., Nobuhara, S., Takai, T., Tung, T.: 3d video and its applications. Springer (2012)Google Scholar
  21. 21.
    Mémoli, F., Sapiro, G.: Comparing Point Clouds. SGP (2004)Google Scholar
  22. 22.
    Ovsjanikov, M., Mérigot, Q., Mémoli, F., Guibas, L.J.: One point isometric matching with the heat kernel. Comput. Graph. Forum 29(5) (2010)Google Scholar
  23. 23.
    Sahilliogu, Y., Yemez, Y.: 3D Shape correspondence by isometry-driven greedy optimization. In: CVPR (2010)Google Scholar
  24. 24.
    Starck, J., Hilton, A.: Model-based multiple view reconstruction of people. In: ICCV (2003)Google Scholar
  25. 25.
    Starck, J., Hilton, A.: Spherical matching for temporal correspondence of non-rigid surfaces. In: ICCV (2005)Google Scholar
  26. 26.
    Straka, M., Hauswiesner, S., Ruether, M., Bischof, H.: Simultaneous shape and pose adaption of articulated models using linear optimization (2012)Google Scholar
  27. 27.
    Sumner, R.W., Popović, J.: Deformation Transfer for Triangle Meshes. ACM Transactions on Graphics 23(3) (2004)Google Scholar
  28. 28.
    Tung, T., Matsuyama, T.: Topology dictionary for 3d video understanding. PAMI 34(8), 1645–1647 (2012)CrossRefGoogle Scholar
  29. 29.
    Tung, T., Matsuyama, T.: Intrinsic characterization of dynamic surfaces. In: CVPR (2013)Google Scholar
  30. 30.
    Vlasic, D., Baran, I., Matusik, W., Popovic, J.: Articulated mesh animation from multi-view silhouettes. ACM Transactions on Graphics 27(3) (2008)Google Scholar
  31. 31.
    Windheuser, T., Schlickewei, U., Schmidt, F., Cremers, D.: Geometrically consistent elastic matching of 3d shapes: A linear programming solution. In: ICCV (2011)Google Scholar
  32. 32.
    Zeng, Y., Wang, C., Gu, X., Samaras, D., Paragios, N.: A Generic Deformation Model for Dense Non-Rigid Surface Registration: a Higher-Order MRF-based Approach. In: ICCV (2013)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Benjamin Allain
    • 1
  • Jean-Sébastien Franco
    • 1
  • Edmond Boyer
    • 1
  • Tony Tung
    • 2
  1. 1.LJKINRIA Grenoble Rhône-AlpesFrance
  2. 2.Graduate School of InformaticsKyoto UniversityJapan

Personalised recommendations