Temporal Surface Tracking Using Mesh Evolution

  • Kiran Varanasi
  • Andrei Zaharescu
  • Edmond Boyer
  • Radu Horaud
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5303)


In this paper, we address the problem of surface tracking in multiple camera environments and over time sequences. In order to fully track a surface undergoing significant deformations, we cast the problem as a mesh evolution over time. Such an evolution is driven by 3D displacement fields estimated between meshes recovered independently at different time frames. Geometric and photometric information is used to identify a robust set of matching vertices. This provides a sparse displacement field that is densified over the mesh by Laplacian diffusion. In contrast to existing approaches that evolve meshes, we do not assume a known model or a fixed topology. The contribution is a novel mesh evolution based framework that allows to fully track, over long sequences, an unknown surface encountering deformations, including topological changes. Results on very challenging and publicly available image based 3D mesh sequences demonstrate the ability of our framework to efficiently recover surface motions .


Topological Change Geodesic Distance Visual Hull Surface Tracking Surface Protrusion 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Supplementary material

978-3-540-88688-4_3_MOESM1_ESM.mp4 (24.1 mb)
Supplementary material (24,655 KB)


  1. 1.
    Gavrila, D., Davis, L.: 3-D model-based tracking of humans in action: a multi-view approach. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, San Francisco, USA (1996)Google Scholar
  2. 2.
    Kakadiaris, I., Metaxas, D.: Model-based estimation of 3d human motion. IEEE Transactions on PAMI 22, 1453–1459 (2000)CrossRefGoogle Scholar
  3. 3.
    Carranza, J., Theobalt, C., Magnor, M., Seidel, H.P.: Free-viewpoint video of human actors. In: Proc. ACM Siggraph 2003, San Diego, USA, pp. 569–577 (2003)Google Scholar
  4. 4.
    DeCarlo, D., Metaxas, D.: Optical flow constraints on deformable models with applications to face tracking. International Journal of Computer Vision 38(2), 99–127 (2000)CrossRefzbMATHGoogle Scholar
  5. 5.
    Salzmann, M., Pilet, J., Ilic, S., Fua, P.: Surface deformation models for non-rigid 3–d shape recovery. IEEE Transactions on PAMI 29, 1481–1487 (2007)CrossRefGoogle Scholar
  6. 6.
    Vedula, S., Rander, P., Collins, R., Kanade, T.: Three-Dimensional Scene Flow. IEEE Transactions on PAMI 27(3), 474–480 (2005)CrossRefGoogle Scholar
  7. 7.
    Neumann, J., Aloimonos, Y.: Spatio-Temporal Stereo Using Multi-Resolution Subdivision Surfaces. International Journal of Computer Vision 47, 181–193 (2002)CrossRefzbMATHGoogle Scholar
  8. 8.
    Pons, J.P., Keriven, R., Faugeras, O.: Multi-view stereo reconstruction and scene flow estimation with a global image-based matching score. International Journal of Computer Vision 72(2), 179–193 (2007)CrossRefGoogle Scholar
  9. 9.
    de Aguiar, E., Theobalt, C., Stoll, C., Seidel, H.: Marker-less Deformable Mesh Tracking for Human Shape and Motion Capture. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Minneapolis, USA (2007)Google Scholar
  10. 10.
    Anguelov, D., Srinivasan, P., Pang, H.C., Koller, D., Thrun, S., Davis, J.: The correlated correspondence algorithm for unsupervised registration of nonrigid surfaces. In: Proceedings of Conference on Neural Information Processing Systems, Cambridge, USA (2004)Google Scholar
  11. 11.
    Bronstein, A., Bronstein, M., Kimmel, R.: Calculus of non-rigid surfaces for geometry and texture manipulation. IEEE Transaction on Visualization and Computer Graphics 13(5), 902–913 (2007)CrossRefGoogle Scholar
  12. 12.
    Starck, J., Hilton, A.: Correspondence labelling for wide-time free-form surface matching. In: Proceedings of the 11th International Conference on Computer Vision, Rio de Janeiro, Brazil (2007)Google Scholar
  13. 13.
    Osher, S., Fedkiw, R.: Level Set Methods and Dynamic Implicit Surfaces. Springer, Heidelberg (2003)CrossRefzbMATHGoogle Scholar
  14. 14.
    Montagnat, J., Delingette, H., Scapel, N., Ayache, N.: Representation, shape, topology and evolution of deformable surfaces. application to 3d medical image segmentation. Technical Report 3954, INRIA (2000)Google Scholar
  15. 15.
    Bickel, B., Botsch, M., Angst, R., Matusik, W., Otaduy, M., Pfister, H., Gross, M.: Multi-scale capture of facial geometry and motion. In: ACM Computer Graphics (Proceedings SIGGRAPH) (2007)Google Scholar
  16. 16.
    Carceroni, R., Kutulakos, K.: Multi-View Scene Capture by Surfel Sampling: From Video Streams to Non-Rigid 3D Motion, Shape and Reflectance. International Journal of Computer Vision 49(2-3), 175–214 (2002)CrossRefzbMATHGoogle Scholar
  17. 17.
    Hernandez, C., Schmitt, F.: Silhouette and stereo fusion for 3D object modeling. Computer Vision and Image Understanding 96, 367–392 (2004)CrossRefGoogle Scholar
  18. 18.
    Furukawa, Y., Ponce, J.: Carved Visual Hulls for Image-Based Modeling. In: Proceedings of the 9th European Conference on Computer Vision, Graz, Austria (2006)Google Scholar
  19. 19.
    Besl, P., McKay, N.: A method for registration of 3-d shapes. IEEE Transactions on PAMI 14(2), 239–256 (1992)CrossRefGoogle Scholar
  20. 20.
    Chui, H., Rangarajan, A.: A new point matching algorithm for non-rigid registration. Computer Vision and Image Understanding 89(2-3), 114–141 (2003)CrossRefzbMATHGoogle Scholar
  21. 21.
    Zhang, D., Hebert, M.: Harmonic Maps and Their Applications in Surface Matching. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Fort Collins, USA (1999)Google Scholar
  22. 22.
    Zigelman, G., Kimmel, R., Kiryati, N.: Texture mapping using surface flattening via multidimensional scaling. IEEE Transactions on Visualization and Computer Graphics 8(2), 198–207 (2002)CrossRefGoogle Scholar
  23. 23.
    Starck, J., Hilton, A.: Spherical Matching for Temporal Correspondence of Non-Rigid Surfaces. In: Proceedings of the 10th International Conference on Computer Vision, Beijing, China (2005)Google Scholar
  24. 24.
    Sorkine, O.: Laplacian mesh processing. In: Eurographics Conference (2005)Google Scholar
  25. 25.
    Zaharescu, A., Boyer, E., Horaud, R.: Transformesh: a topology-adaptive mesh-based approach to surface evolution. In: Proceedings of the 8th Asian Conference on Computer Vision, Tokyo, Japan (2007)Google Scholar
  26. 26.
    Bay, H., Tuytelaars, T., van Gool, L.: Surf: Speeded up robust features. In: Proceedings of the 9th European Conference on Computer Vision, Graz, Austria (2006)Google Scholar
  27. 27.
    Hilaga, M., Shinagawa, Y., Kohmura, T., Kunii, T.: Topology matching for fully automatic similarity estimation of 3d shapes. In: ACM Computer Graphics (Proceedings SIGGRAPH) (2001)Google Scholar
  28. 28.
    Breen, D.E., Whitaker, R.T.: A level-set approach for the metamorphosis of solid models. IEEE Transaction on Visualization and Computer Graphics 7, 173–192 (2001)CrossRefGoogle Scholar
  29. 29.
    Starck, J., Hilton, A.: Surface capture for performance based animation. IEEE Computer Graphics and Applications 27(3), 21–31 (2007)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Kiran Varanasi
    • 1
  • Andrei Zaharescu
    • 1
  • Edmond Boyer
    • 1
  • Radu Horaud
    • 1
  1. 1.LJK - INRIA Rhône-AlpesFrance

Personalised recommendations