Skip to main content
SpringerLink
Log in

Invariant shape descriptor for 3D video encoding

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

This paper presents a novel approach to represent spatio-temporal visual information. We introduce a surface-based shape model whose structure is invariant to surface variations over time to describe 3D dynamic surfaces (e.g., 3D video obtained from multiview video capture). The descriptor is defined as a graph lying on object surfaces and anchored to invariant local features (e.g., surface point extrema). Geodesic consistency-based priors are used as cues within a probabilistic framework to maintain the graph invariant, even though the surfaces undergo non-rigid deformations. Our contribution brings to 3D geometric data a temporally invariant structure that relies only on intrinsic surface properties, and is independent of surface parameterization (i.e., surface mesh connectivity). The proposed descriptor can therefore be used for efficient dynamic surface encoding, through transformation into 2D (geometry) images, as its structure can provide an invariant representation for dynamic 3D mesh models. Various experiments on challenging publicly available datasets are performed to assess invariant property and performance of the descriptor.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Notes

  1. A path on a surface is a set of points linked two-by-two by a line.

References

  1. de Aguiar, E., Stoll, C., Theobalt, C., Ahmed, N., Seidel, H.P., Thrun, S.: Performance capture from sparse multi-view video. ACM Trans Graphics 27(3), 98:1–98:10 (2008)

    Google Scholar 

  2. Alexa, M., Müllen, W.: Representing animations by principal components. Computer Gr. Forum 19(3), 411–418 (2000)

    Google Scholar 

  3. Allard, J., Ménier, C., Raffin, B., Boyer, E., Faure, F.: Grimage: Markerless 3d Interactions. ACM SIGGRAPH—Emerging Technologies (2007)

  4. Alliez, P., Gotsman, C.: Recent advances in compression of 3d meshes. Adv. Multiresolution Geometric Model. Springer-Verlag (2005)

  5. Baran, I., Popovic, J.: Automatic rigging and animation of 3d characters. ACM Trans Gr. 26(3), 27 (2007)

    Article  Google Scholar 

  6. Blum, H.: A Transformation for Extracting New Descriptors of Shape. Models for the Perception of Speech and Visual Form. MIT Press, pp. 362–380 (1967)

  7. Briceno, H., Sandler, P., McMillian, L., Gortler, S., Hoppe, H.: Geometry videos: a new representation for 3d animations. In: Eurographics/SIGGRAPH Symp Computer, Animation, pp. 136–146 (2003)

  8. Bronstein, A.M., Bronstein, M.M., Kimmel, R.: Calculus of non-rigid surfaces for geometry and texture manipulation. In: IEEE Trans Vis. Computer Gr., pp. 902–913 (2007)

  9. Cagniart, C., Boyer, E., Ilic, S.: Probabilistic deformable surface tracking from multiple videos. In: Proceedings of European Conf Computer Vision (2010)

  10. Carr, N., Hoberock, J., Crane, K., Hart, J.: Rectangular multi-chart geometry images. In: Proceedings of Eurographics Symp Geometry Processing, pp. 181–190 (2006)

  11. Carranza, J., Theobalt, C., Magnor, M., Seidel, H.P.: Free-viewpoint video of human actors. ACM Trans Gr. 22(3), 569–577 (2003)

    Article  Google Scholar 

  12. Cornea, N., Silver, D., Yuan, X., Balasubramanian, R.: Computing hierarchical curve skeletons of 3d objects. Vis. Computer J. 21(11), 945–955 (2005)

    Article  Google Scholar 

  13. Edelsbrunner, H., Harer, J., Mascarenhas, A., Pascucci, V.: Time-varying reeb graphs for continuous space-time data. In: Proceedings of Symp Computational Geometry (2004)

  14. Erickson, J., Har-Peled, S.: Optimally cutting a surface into a disk. Discrete Comput. Geometry 31(1), 37–59 (2004)

    Article  MATH  MathSciNet  Google Scholar 

  15. Floater, M.: Parametrization and smooth approximation of surface triangulations. Computer Aided Geometric Design 14(3), 231–250 (1997)

    Article  MATH  MathSciNet  Google Scholar 

  16. Forsyth, D.A., Mundy, J.L., Zisserman, A., Coelho, C., Heller, A., Rothwell, C.: Invariant descriptors for 3d object recognition and pose. IEEE Trans Pattern Anal. Mach. Intell. 13(10), 971–991 (1991)

    Google Scholar 

  17. Franco, J., Menier, C., Boyer, E., Raffin, B.: A distributed approach for real-time 3d modeling. In: Proceedings of IEEE Conf Computer Vision Pattern Recognition Workshop on real-time 3D sensors and their applications (2004)

  18. Gu, X., Gortler, S., Hoppe, H.: Geometry images. ACM Trans. Gr. (SIGGRAPH) 21(3), 355–361 (2002)

  19. Guo, Y.W., Wang, J., Cui, X.F., Peng, Q.S.: A New Constrained Texture Mapping Method. Entertainment Computing. ICEC Springer, Springer LNCS, Berlin Heidelberg (2005)

    Google Scholar 

  20. Habe, H., Katsura, Y., Matsuyama, T.: Skin-off: representation and compression scheme for 3d video. In: Proceedings of picture coding symposium (2004)

  21. Hilaga, M., Shinagawa, Y., Kohmura, T., Kunii, T.L.: Topology matching for fully automatic similarity estimation of 3d shapes. ACM SIGGRAPH, pp. 203–212 (2001)

  22. Huang, P., Tung, T., Nobuhara, S., Hilton, A., Matsuyama, T.: Comparison of skeleton and non-skeleton shape descriptors for 3d video. In: Proceedings of 3DPVT (2010)

  23. Jiang, H., Liu, H., Tan, P., Zhang, G., Bao, H.: 3d reconstruction of dynamic scenes with multiple handheld cameras. In: Proceedings of European Conf computer vision (2012)

  24. Kanade, T., Yoshida, A., Oda, K., Kano, H., Tanaka, M.: A stereo machine for video-rate dense depth mapping and its new applications. In: Proceedings of IEEE Conf computer vision pattern recognition (1996)

  25. Karni, Z., Gotsman, C.: Compression of soft-body animation sequence. Computers Gr. 28, 25–34 (2004)

    Article  Google Scholar 

  26. Kavan, L., Collins, S., Žára, J., O’Sullivan, C.: Skinning with dual quaternions. In: Proceedings of Symposium on interactive 3D graphics and games, pp. 39–46 (2007)

  27. Klein, T., Ertl, T.: Scale-space tracking of critical points in 3d vector fields. In: Proceedings of topology-based methods in visualization (2005)

  28. Mamou, K., Zaharia, T., Preteux, F., Stefanoski, N., Ostermann, J.: Frame-based compression of animated meshes in mpeg-4. In: Proceedings of IEEE Int’l Conf Multimedia and Expo (2008)

  29. Matsuyama, T., Wu, X., Takai, T., Nobuhara, S.: Real-time 3d shape reconstruction, dynamic 3d mesh deformation, and high fidelity visualization for 3d video. Computer Vis. Image Understand. 96(3), 393–434 (2004)

    Article  Google Scholar 

  30. Matsuyama, T., Nobuhara, S., Takai, T., Tung, T.: 3d Video and its Applications. Springer, London (2012)

  31. Mémoli, F., Sapiro, G.: A theoretical and computational framework for isometry invariant recognition of point cloud data. Found Comput. Math. 5(3), 313–347 (2005)

    Article  MATH  MathSciNet  Google Scholar 

  32. Morse, M.: The calculus of variations in the large. American Mathematical Society, Colloquium Publication, New York, p 18 (1934)

  33. Mortara, M., Patanè, G.: Affine-invariant skeleton of 3d shapes. In: Proceedings of Shape Modeling International (2002)

  34. Mundy, J,, Zisserman, A.: Geometric invariance in computer vision. MIT Press, Cambridge (1992)

  35. CR Cignoni, P., Scopigno, R.: Metro: measuring error on simplified surfaces. Computer Gr. Forum 17(2), 167–174 (1998)

    Article  Google Scholar 

  36. Palagyi, K., Kuba, A.: A parallel 3d 12-subiteration thinning algorithm. Graph Models Image Proc 61(4), 199–221 (1999)

    Article  Google Scholar 

  37. Pascucci, V., Scorzelli, G., Bremer, P.T., Mascarenhas, A.: Robust on-line computation of reeb graphs: simplicity and speed. ACM Trans Gr. 26 (3), 58:1–58:9 (2007)

    Google Scholar 

  38. Reeb, G.: On the singular points of a completely integrable pfaff form or of a numerical function. Comptes Rendus Acad. Sci. Paris 222, 847–849 (1946)

    MATH  MathSciNet  Google Scholar 

  39. Rothganger, F., Lazebnik, S., Schmid, C., Ponce, J.: 3d object modeling and recognition using local affine-invariant image descriptors and multi-view spatial constraints. Int’l J. Computer Vis. 66(3), 231–259 (2006)

    Article  Google Scholar 

  40. Saboret, L., Alliez, P., Lévy, B.: Planar parameterization of triangulated surface meshes, 40 edn. In: CGAL reference manual CGAL Editorial Board (2012)

  41. Sander, P., Wood, Z., Gortler, S., Snyder, J., Hoppe, H.: Multi-chart geometry images. In: Proceedings of Eurographics Symp geometry processing, pp. 146–155 (2003)

  42. Seitz, S., Curless, B., Diebel, J., Scharstein, D., Szeliski, R.: A comparison and evaluation of multi-view stereo reconstruction algorithms. In: Proceedings of IEEE Conf computer vision pattern recognition (2006)

  43. Starck, J., Hilton, A.: Spherical matching for temporal correspondence of non-rigid surfaces. In: Proceedings of IEEE Int’l Conf computer vision (2005)

  44. Starck, J., Hilton, A.: Surface capture for performance-based animation. IEEE Computer Gr. Appl. 27(3), 21–31 (2007)

    Google Scholar 

  45. Sumner, R.W., Popovic, J.: Deformation transfer for triangle meshes. ACM Trans Gr. 23(3), 399–405 (2004)

    Google Scholar 

  46. Taubin, G., Rossignac, J.: Geometric compression through topological surgery. ACM Trans Gr. 17(2), 84–115 (1998)

    Article  Google Scholar 

  47. Tung, T., Matsuyama, T.: Dynamic surface matching by geodesic mapping for 3d animation transfer. In: Proceedings of IEEE Conf computer vision pattern recognition (2010)

  48. Tung, T., Matsuyama, T.: Invariant surface-based shape descriptor for dynamic surface encoding. In: Proceedings of Asian Conf computer vision (2012a)

  49. Tung, T., Matsuyama, T.: Topology dictionary for 3d video understanding. IEEE Trans Pattern Anal. Mach. Intell. 34(8), 1645–1657 (2012b)

    Article  Google Scholar 

  50. Tung, T., Schmitt, F.: The augmented multiresolution reeb graph approach for content-based retrieval of 3d shapes (code on webpage). Int’l J. Shape Model. 11(1), 91–120 (2005)

    Article  Google Scholar 

  51. Tung, T., Schmitt, F., Matsuyama, T.: Topology matching for 3d video compression. In: Proceedings of IEEE Conf computer vision pattern recognition (2007)

  52. Vlasic, D., Baran, I., Matusik, W., Popovic, J.: Articulated mesh animation from multi-view silhouettes. ACM Trans Gr. 27(3), 97:1–97:3 (2008)

    Google Scholar 

Download references

Acknowledgments

This work was supported in part by the JST-CREST project “Creation of Human-Harmonized Information Technology for Convivial Society”. The authors thank Dr. Lyndon Hill for his preliminary work on this project.

Author information

Authors and Affiliations

  1. Kyoto University, Graduate School of Informatics, Sakyo, Kyoto, Japan

    Tony Tung & Takashi Matsuyama

Authors
  1. Tony Tung
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Takashi Matsuyama
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tony Tung.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Tung, T., Matsuyama, T. Invariant shape descriptor for 3D video encoding. Vis Comput 31, 311–324 (2015). https://doi.org/10.1007/s00371-014-0925-6

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-014-0925-6

Keywords

Search

Navigation