Abstract
In this paper, we address the problem of surface tracking in multiple camera environments and over time sequences. In order to fully track a surface undergoing significant deformations, we cast the problem as a mesh evolution over time. Such an evolution is driven by 3D displacement fields estimated between meshes recovered independently at different time frames. Geometric and photometric information is used to identify a robust set of matching vertices. This provides a sparse displacement field that is densified over the mesh by Laplacian diffusion. In contrast to existing approaches that evolve meshes, we do not assume a known model or a fixed topology. The contribution is a novel mesh evolution based framework that allows to fully track, over long sequences, an unknown surface encountering deformations, including topological changes. Results on very challenging and publicly available image based 3D mesh sequences demonstrate the ability of our framework to efficiently recover surface motions .
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Gavrila, D., Davis, L.: 3-D model-based tracking of humans in action: a multi-view approach. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, San Francisco, USA (1996)
Kakadiaris, I., Metaxas, D.: Model-based estimation of 3d human motion. IEEE Transactions on PAMI 22, 1453–1459 (2000)
Carranza, J., Theobalt, C., Magnor, M., Seidel, H.P.: Free-viewpoint video of human actors. In: Proc. ACM Siggraph 2003, San Diego, USA, pp. 569–577 (2003)
DeCarlo, D., Metaxas, D.: Optical flow constraints on deformable models with applications to face tracking. International Journal of Computer Vision 38(2), 99–127 (2000)
Salzmann, M., Pilet, J., Ilic, S., Fua, P.: Surface deformation models for non-rigid 3–d shape recovery. IEEE Transactions on PAMI 29, 1481–1487 (2007)
Vedula, S., Rander, P., Collins, R., Kanade, T.: Three-Dimensional Scene Flow. IEEE Transactions on PAMI 27(3), 474–480 (2005)
Neumann, J., Aloimonos, Y.: Spatio-Temporal Stereo Using Multi-Resolution Subdivision Surfaces. International Journal of Computer Vision 47, 181–193 (2002)
Pons, J.P., Keriven, R., Faugeras, O.: Multi-view stereo reconstruction and scene flow estimation with a global image-based matching score. International Journal of Computer Vision 72(2), 179–193 (2007)
de Aguiar, E., Theobalt, C., Stoll, C., Seidel, H.: Marker-less Deformable Mesh Tracking for Human Shape and Motion Capture. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Minneapolis, USA (2007)
Anguelov, D., Srinivasan, P., Pang, H.C., Koller, D., Thrun, S., Davis, J.: The correlated correspondence algorithm for unsupervised registration of nonrigid surfaces. In: Proceedings of Conference on Neural Information Processing Systems, Cambridge, USA (2004)
Bronstein, A., Bronstein, M., Kimmel, R.: Calculus of non-rigid surfaces for geometry and texture manipulation. IEEE Transaction on Visualization and Computer Graphics 13(5), 902–913 (2007)
Starck, J., Hilton, A.: Correspondence labelling for wide-time free-form surface matching. In: Proceedings of the 11th International Conference on Computer Vision, Rio de Janeiro, Brazil (2007)
Osher, S., Fedkiw, R.: Level Set Methods and Dynamic Implicit Surfaces. Springer, Heidelberg (2003)
Montagnat, J., Delingette, H., Scapel, N., Ayache, N.: Representation, shape, topology and evolution of deformable surfaces. application to 3d medical image segmentation. Technical Report 3954, INRIA (2000)
Bickel, B., Botsch, M., Angst, R., Matusik, W., Otaduy, M., Pfister, H., Gross, M.: Multi-scale capture of facial geometry and motion. In: ACM Computer Graphics (Proceedings SIGGRAPH) (2007)
Carceroni, R., Kutulakos, K.: Multi-View Scene Capture by Surfel Sampling: From Video Streams to Non-Rigid 3D Motion, Shape and Reflectance. International Journal of Computer Vision 49(2-3), 175–214 (2002)
Hernandez, C., Schmitt, F.: Silhouette and stereo fusion for 3D object modeling. Computer Vision and Image Understanding 96, 367–392 (2004)
Furukawa, Y., Ponce, J.: Carved Visual Hulls for Image-Based Modeling. In: Proceedings of the 9th European Conference on Computer Vision, Graz, Austria (2006)
Besl, P., McKay, N.: A method for registration of 3-d shapes. IEEE Transactions on PAMI 14(2), 239–256 (1992)
Chui, H., Rangarajan, A.: A new point matching algorithm for non-rigid registration. Computer Vision and Image Understanding 89(2-3), 114–141 (2003)
Zhang, D., Hebert, M.: Harmonic Maps and Their Applications in Surface Matching. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Fort Collins, USA (1999)
Zigelman, G., Kimmel, R., Kiryati, N.: Texture mapping using surface flattening via multidimensional scaling. IEEE Transactions on Visualization and Computer Graphics 8(2), 198–207 (2002)
Starck, J., Hilton, A.: Spherical Matching for Temporal Correspondence of Non-Rigid Surfaces. In: Proceedings of the 10th International Conference on Computer Vision, Beijing, China (2005)
Sorkine, O.: Laplacian mesh processing. In: Eurographics Conference (2005)
Zaharescu, A., Boyer, E., Horaud, R.: Transformesh: a topology-adaptive mesh-based approach to surface evolution. In: Proceedings of the 8th Asian Conference on Computer Vision, Tokyo, Japan (2007)
Bay, H., Tuytelaars, T., van Gool, L.: Surf: Speeded up robust features. In: Proceedings of the 9th European Conference on Computer Vision, Graz, Austria (2006)
Hilaga, M., Shinagawa, Y., Kohmura, T., Kunii, T.: Topology matching for fully automatic similarity estimation of 3d shapes. In: ACM Computer Graphics (Proceedings SIGGRAPH) (2001)
Breen, D.E., Whitaker, R.T.: A level-set approach for the metamorphosis of solid models. IEEE Transaction on Visualization and Computer Graphics 7, 173–192 (2001)
Starck, J., Hilton, A.: Surface capture for performance based animation. IEEE Computer Graphics and Applications 27(3), 21–31 (2007)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Electronic Supplementary Material
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Varanasi, K., Zaharescu, A., Boyer, E., Horaud, R. (2008). Temporal Surface Tracking Using Mesh Evolution. In: Forsyth, D., Torr, P., Zisserman, A. (eds) Computer Vision – ECCV 2008. ECCV 2008. Lecture Notes in Computer Science, vol 5303. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-88688-4_3
Download citation
DOI: https://doi.org/10.1007/978-3-540-88688-4_3
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-88685-3
Online ISBN: 978-3-540-88688-4
eBook Packages: Computer ScienceComputer Science (R0)