Advertisement

Hamiltonian Dynamics for Real-World Shape Interpolation

Conference paper
  • 837 Downloads
Part of the Lecture Notes in Computer Science book series (LNCS, volume 12349)

Abstract

We revisit the classical problem of 3D shape interpolation and propose a novel, physically plausible approach based on Hamiltonian dynamics. While most prior work focuses on synthetic input shapes, our formulation is designed to be applicable to real-world scans with imperfect input correspondences and various types of noise. To that end, we use recent progress on dynamic thin shell simulation and divergence-free shape deformation and combine them to address the inverse problem of finding a plausible intermediate sequence for two input shapes. In comparison to prior work that mainly focuses on small distortion of consecutive frames, we explicitly model volume preservation and momentum conservation, as well as an anisotropic local distortion model. We argue that, in order to get a robust interpolation for imperfect inputs, we need to model the input noise explicitly which results in an alignment based formulation. Finally, we show a qualitative and quantitative improvement over prior work on a broad range of synthetic and scanned data. Besides being more robust to noisy inputs, our method yields exactly volume preserving intermediate shapes, avoids self-intersections and is scalable to high resolution scans.

Keywords

Shape interpolation Registration 3D computer vision 

Notes

Acknowledgements

We would like to thank Zorah Lähner and Aysim Toker for useful discussions. We gratefully acknowledge the support of the ERC Consolidator Grant “3D Reloaded”.

Supplementary material

504439_1_En_11_MOESM1_ESM.pdf (897 kb)
Supplementary material 1 (pdf 896 KB)

References

  1. 1.
    Alexa, M.: Differential coordinates for local mesh morphing and deformation. Vis. Comput. 19(2–3), 105–114 (2003)CrossRefGoogle Scholar
  2. 2.
    Alexa, M., Cohen-Or, D., Levin, D.: As-rigid-as-possible shape interpolation. In: Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH 2000, pp. 157–164 (2000)Google Scholar
  3. 3.
    Allen, B., Curless, B., Popović, Z., Hertzmann, A.: Learning a correlated model of identity and pose-dependent body shape variation for real-time synthesis. In: Proceedings of the 2006 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, pp. 147–156. Eurographics Association (2006)Google Scholar
  4. 4.
    Angelidis, A., Cani, M.P., Wyvill, G., King, S.: Swirling-sweepers: constant-volume modeling. Graph. Models 68(4), 324–332 (2006)CrossRefGoogle Scholar
  5. 5.
    Anguelov, D., Srinivasan, P., Koller, D., Thrun, S., Rodgers, J., Davis, J.: Scape: shape completion and animation of people. In: ACM Transactions on Graphics (TOG), vol. 24, pp. 408–416. ACM (2005)Google Scholar
  6. 6.
    Baek, S.Y., Lim, J., Lee, K.: Isometric shape interpolation. Comput. Graph. 46, 257–263 (2015)CrossRefGoogle Scholar
  7. 7.
    Bogo, F., Romero, J., Loper, M., Black, M.J.: FAUST: dataset and evaluation for 3D mesh registration. In: Proceedings IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, Piscataway, June 2014Google Scholar
  8. 8.
    Boscaini, D., Masci, J., Rodolà, E., Bronstein, M.: Learning shape correspondence with anisotropic convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 3189–3197 (2016)Google Scholar
  9. 9.
    Botsch, M., Pauly, M., Gross, M.H., Kobbelt, L.: Primo: coupled prisms for intuitive surface modeling. In: Symposium on Geometry Processing, pp. 11–20. No. CONF (2006)Google Scholar
  10. 10.
    Bouaziz, S., Martin, S., Liu, T., Kavan, L., Pauly, M.: Projective dynamics: fusing constraint projections for fast simulation. ACM Trans. Graph. (TOG) 33(4), 1–11 (2014)CrossRefGoogle Scholar
  11. 11.
    Brandt, C., Eisemann, E., Hildebrandt, K.: Hyper-reduced projective dynamics. ACM Trans. Graph. (TOG) 37(4), 1–13 (2018)CrossRefGoogle Scholar
  12. 12.
    Brandt, C., von Tycowicz, C., Hildebrandt, K.: Geometric flows of curves in shape space for processing motion of deformable objects. In: Computer Graphics Forum, vol. 35, pp. 295–305. Wiley Online Library (2016)Google Scholar
  13. 13.
    Bronstein, A.M., Bronstein, M.M., Kimmel, R.: Numerical geometry of non-rigid shapes. Springer (2008). http://tosca.cs.technion.ac.il/book/resources_data.html
  14. 14.
    Charpiat, G., Faugeras, O., Keriven, R.: Approximations of shape metrics and application to shape warping and empirical shape statistics. Found. Comput. Math. 5(1), 1–58 (2005)MathSciNetCrossRefGoogle Scholar
  15. 15.
    Charpiat, G., Maurel, P., Pons, J.P., Keriven, R., Faugeras, O.: Generalized gradients: priors on minimization flows. Int. J. Comput. Vision 73(3), 325–344 (2007)CrossRefGoogle Scholar
  16. 16.
    Chen, R., Weber, O., Keren, D., Ben-Chen, M.: Planar shape interpolation with bounded distortion. ACM Trans. Graph. (TOG) 32(4), 1–12 (2013)zbMATHGoogle Scholar
  17. 17.
    Dyke, R., Stride, C., Lai, Y., Rosin, P.: SHREC-19: shape correspondence with isometric and non-isometric deformations (2019)Google Scholar
  18. 18.
    Eckstein, I., Pons, J.P., Tong, Y., Kuo, C.C., Desbrun, M.: Generalized surface flows for mesh processing. In: Proceedings of the fifth Eurographics Symposium on Geometry Processing, pp. 183–192. Eurographics Association (2007)Google Scholar
  19. 19.
    Eisenberger, M., Lähner, Z., Cremers, D.: Divergence-free shape correspondence by deformation. In: Computer Graphics Forum, vol. 38, pp. 1–12. Wiley Online Library (2019)Google Scholar
  20. 20.
    Eisenberger, M., Lähner, Z., Cremers, D.: Smooth shells: Multi-scale shape registration with functional maps. arXiv preprint arXiv:1905.12512 (2019)
  21. 21.
    Fletcher, P.T., Lu, C., Pizer, S.M., Joshi, S.: Principal geodesic analysis for the study of nonlinear statistics of shape. IEEE Trans. Med. Imaging 23(8), 995–1005 (2004)CrossRefGoogle Scholar
  22. 22.
    von Funck, W., Theisel, H., Seidel, H.P.: Vector field based shape deformations. In: ACM Transactions on Graphics (TOG), vol. 25, pp. 1118–1125. ACM (2006)Google Scholar
  23. 23.
    Goldenthal, R., Harmon, D., Fattal, R., Bercovier, M., Grinspun, E.: Efficient simulation of inextensible cloth. In: ACM SIGGRAPH 2007 Papers, pp. 49-es (2007)Google Scholar
  24. 24.
    Grinspun, E., Hirani, A.N., Desbrun, M., Schröder, P.: Discrete shells. In: Proceedings of the 2003 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, pp. 62–67. Eurographics Association (2003)Google Scholar
  25. 25.
    Groueix, T., Fisher, M., Kim, V.G., Russell, B.C., Aubry, M.: 3D-CODED: 3D correspondences by deep deformation. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11206, pp. 235–251. Springer, Cham (2018).  https://doi.org/10.1007/978-3-030-01216-8_15CrossRefGoogle Scholar
  26. 26.
    Heeren, B., Rumpf, M., Schröder, P., Wardetzky, M., Wirth, B.: Exploring the geometry of the space of shells. In: Computer Graphics Forum, vol. 33, pp. 247–256. Wiley Online Library (2014)Google Scholar
  27. 27.
    Heeren, B., Rumpf, M., Schröder, P., Wardetzky, M., Wirth, B.: Splines in the space of shells. Comput. Graph. Forum 35(5), 111–120 (2016)CrossRefGoogle Scholar
  28. 28.
    Heeren, B., Rumpf, M., Wardetzky, M., Wirth, B.: Time-discrete geodesics in the space of shells. In: Computer Graphics Forum, vol. 31, pp. 1755–1764. Wiley Online Library (2012)Google Scholar
  29. 29.
    Heeren, B., Zhang, C., Rumpf, M., Smith, W.: Principal geodesic analysis in the space of discrete shells. In: Computer Graphics Forum, vol. 37, pp. 173–184. Wiley Online Library (2018)Google Scholar
  30. 30.
    Hormann, K., Greiner, G.: Mips: An efficient global parametrization method. Technical report, Erlangen-Nuernberg University (Germany) Computer Graphics Group (2000)Google Scholar
  31. 31.
    Huber, P., Perl, R., Rumpf, M.: Smooth interpolation of key frames in a riemannian shell space. Comput. Aided Geom. Des. 52, 313–328 (2017)MathSciNetCrossRefGoogle Scholar
  32. 32.
    Kilian, M., Mitra, N.J., Pottmann, H.: Geometric modeling in shape space. In: ACM Transactions on Graphics (TOG), vol. 26, p. 64. ACM (2007)Google Scholar
  33. 33.
    Kim, V.G., Lipman, Y., Funkhouser, T.A.: Blended intrinsic maps. Trans. Graph. (TOG) 30, 4 (2011)Google Scholar
  34. 34.
    Lipman, Y., Sorkine, O., Cohen-Or, D., Levin, D., Rossi, C., Seidel, H.P.: Differential coordinates for interactive mesh editing. In: Proceedings Shape Modeling Applications, 2004, pp. 181–190. IEEE (2004)Google Scholar
  35. 35.
    Lipman, Y., Sorkine, O., Levin, D., Cohen-Or, D.: Linear rotation-invariant coordinates for meshes. ACM Trans. Graph. (TOG) 24(3), 479–487 (2005)CrossRefGoogle Scholar
  36. 36.
    Litany, O., Remez, T., Rodolà, E., Bronstein, A., Bronstein, M.: Deep functional maps: structured prediction for dense shape correspondence. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 5659–5667 (2017)Google Scholar
  37. 37.
    Macklin, M., Müller, M.: Position based fluids. ACM Trans. Graph. (TOG) 32(4), 1–12 (2013)CrossRefGoogle Scholar
  38. 38.
    Martin, S., Thomaszewski, B., Grinspun, E., Gross, M.: Example-based elastic materials. In: ACM SIGGRAPH 2011 Papers, pp. 1–8 (2011)Google Scholar
  39. 39.
    Masci, J., Boscaini, D., Bronstein, M., Vandergheynst, P.: Geodesic convolutional neural networks on riemannian manifolds. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, pp. 37–45 (2015)Google Scholar
  40. 40.
    McLachlan, G.J.: Mahalanobis distance. Resonance 4(6), 20–26 (1999)CrossRefGoogle Scholar
  41. 41.
    Melzi, S., Ren, J., Rodola, E., Ovsjanikov, M., Wonka, P.: Zoomout: spectral upsampling for efficient shape correspondence. arXiv preprint arXiv:1904.07865 (2019)
  42. 42.
    Michor, P.W., Mumford, D.: Riemannian geometries on spaces of plane curves. arXiv preprint math/0312384 (2003)Google Scholar
  43. 43.
    Monti, F., Boscaini, D., Masci, J., Rodola, E., Svoboda, J., Bronstein, M.M.: Geometric deep learning on graphs and manifolds using mixture model CNNs. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5115–5124 (2017)Google Scholar
  44. 44.
    Müller, M., Heidelberger, B., Hennix, M., Ratcliff, J.: Position based dynamics. J. Vis. Commun. Image Represent. 18(2), 109–118 (2007)CrossRefGoogle Scholar
  45. 45.
    Newcombe, R.A., Fox, D., Seitz, S.M.: Dynamicfusion: reconstruction and tracking of non-rigid scenes in real-time. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 343–352 (2015)Google Scholar
  46. 46.
    Ovsjanikov, M., Ben-Chen, M., Solomon, J., Butscher, A., Guibas, L.: Functional maps: a flexible representation of maps between shapes. ACM Trans. Graph. (TOG) 31(4), 30 (2012)CrossRefGoogle Scholar
  47. 47.
    von Radziewsky, P., Eisemann, E., Seidel, H.P., Hildebrandt, K.: Optimized subspaces for deformation-based modeling and shape interpolation. Comput. Graph. 58, 128–138 (2016)CrossRefGoogle Scholar
  48. 48.
    Ren, J., Poulenard, A., Wonka, P., Ovsjanikov, M.: Continuous and orientation-preserving correspondences via functional maps. ACM Trans. Graph. 37(6), 248:1–248:16 (2018)Google Scholar
  49. 49.
    Rodolà, E., Cosmo, L., Bronstein, M.M., Torsello, A., Cremers, D.: Partial functional correspondence. In: Computer Graphics Forum, vol. 36, pp. 222–236. Wiley Online Library (2017)Google Scholar
  50. 50.
    Rodolà, E., Rota Bulo, S., Windheuser, T., Vestner, M., Cremers, D.: Dense non-rigid shape correspondence using random forests. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2014)Google Scholar
  51. 51.
    Sassen, J., Heeren, B., Hildebrandt, K., Rumpf, M.: Geometric optimization using nonlinear rotation-invariant coordinates. arXiv preprint arXiv:1908.11728 (2019)
  52. 52.
    Sederberg, T.W., Gao, P., Wang, G., Mu, H.: 2-D shape blending: an intrinsic solution to the vertex path problem. In: Proceedings of the 20th Annual Conference on Computer Graphics and Interactive Techniques, pp. 15–18 (1993)Google Scholar
  53. 53.
    Slavcheva, M., Baust, M., Cremers, D., Ilic, S.: Killingfusion: Non-rigid 3D reconstruction without correspondences. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1386–1395 (2017)Google Scholar
  54. 54.
    Slavcheva, M., Baust, M., Ilic, S.: Sobolevfusion: 3D reconstruction of scenes undergoing free non-rigid motion. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2646–2655 (2018)Google Scholar
  55. 55.
    Solomon, J., Ben-Chen, M., Butscher, A., Guibas, L.: As-killing-as-possible vector fields for planar deformation. In: Computer Graphics Forum, vol. 30, pp. 1543–1552. Wiley Online Library (2011)Google Scholar
  56. 56.
    Sorkine, O., Alexa, M.: As-rigid-as-possible surface modeling. In: Symposium on Geometry Processing, vol. 4, pp. 109–116 (2007)Google Scholar
  57. 57.
    Sumner, R.W., Zwicker, M., Gotsman, C., Popović, J.: Mesh-based inverse kinematics. ACM Trans. Graph. (TOG) 24(3), 488–495 (2005)CrossRefGoogle Scholar
  58. 58.
    Sundaramoorthi, G., Yezzi, A., Mennucci, A.C.: Sobolev active contours. Int. J. Comput. Vision 73(3), 345–366 (2007)CrossRefGoogle Scholar
  59. 59.
    Vestner, M., et al.: Efficient deformable shape correspondence via kernel matching. In: International Conference on 3D Vision (3DV), October 2017Google Scholar
  60. 60.
    Wirth, B., Bar, L., Rumpf, M., Sapiro, G.: A continuum mechanical approach to geodesics in shape space. Int. J. Comput. Vision 93(3), 293–318 (2011) MathSciNetCrossRefGoogle Scholar
  61. 61.
    Xu, D., Zhang, H., Wang, Q., Bao, H.: Poisson shape interpolation. In: Proceedings of the 2005 ACM Symposium on Solid and Physical Modeling, pp. 267–274 (2005)Google Scholar
  62. 62.
    Younes, Laurent: Shapes and Diffeomorphisms. AMS, vol. 171. Springer, Heidelberg (2019).  https://doi.org/10.1007/978-3-662-58496-5CrossRefzbMATHGoogle Scholar
  63. 63.
    Zhang, C., Chen, T.: Efficient feature extraction for 2D/3D objects in mesh representation. In: Proceedings 2001 International Conference on Image Processing (Cat. No. 01CH37205), vol. 3, pp. 935–938. IEEE (2001)Google Scholar
  64. 64.
    Zhang, C., Heeren, B., Rumpf, M., Smith, W.A.: Shell PCA: statistical shape modelling in shell space. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1671–1679 (2015)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Technical University of MunichGarchingGermany

Personalised recommendations