Skip to main content
SpringerLink
Log in

Intrinsic Point Cloud Interpolation via Dual Latent Space Navigation

  • Conference paper
  • First Online:
Computer Vision – ECCV 2020 (ECCV 2020)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 12347))

Included in the following conference series:

Abstract

We present a learning-based method for interpolating and manipulating 3D shapes represented as point clouds, that is explicitly designed to preserve intrinsic shape properties. Our approach is based on constructing a dual encoding space that enables shape synthesis and, at the same time, provides links to the intrinsic shape information, which is typically not available on point cloud data. Our method works in a single pass and avoids expensive optimization, employed by existing techniques. Furthermore, the strong regularization provided by our dual latent space approach also helps to improve shape recovery in challenging settings from noisy point clouds across different datasets. Extensive experiments show that our method results in more realistic and smoother interpolations compared to baselines. Both the code and our pre-trained network can be found online: https://github.com/mrakotosaon/intrinsic_interpolations.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Achlioptas, P., Diamanti, O., Mitliagkas, I., Guibas, L.: Learning representations and generative models for 3D point clouds. In: Dy, J., Krause, A. (eds.) Proceedings of the 35th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 80, pp. 40–49. Stockholmsmässan, Stockholm Sweden, 10–15 July 2018

    Google Scholar 

  2. Alexa, M., Cohen-Or, D., Levin, D.: As-rigid-as-possible shape interpolation. In: Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques, pp. 157–164. ACM Press/Addison-Wesley Publishing Co. (2000)

    Google Scholar 

  3. Anguelov, D., Srinivasan, P., Koller, D., Thrun, S., Rodgers, J., Davis, J.: Scape: shape completion and animation of people. In: ACM Transactions on Graphics (TOG), vol. 24, pp. 408–416. ACM (2005)

    Google Scholar 

  4. Arun, K.S., Huang, T.S., Blostein, S.D.: Least-squares fitting of two 3-D point sets. IEEE Trans. Pattern Anal. Mach. Intell. 1(5), 698–700 (1987)

    Article  Google Scholar 

  5. Ben-Hamu, H., Maron, H., Kezurer, I., Avineri, G., Lipman, Y.: Multi-chart generative surface modeling. In: SIGGRAPH Asia 2018 Technical Papers, p. 215. ACM (2018)

    Google Scholar 

  6. Benamou, J.D., Brenier, Y.: A computational fluid mechanics solution to the Monge-Kantorovich mass transfer problem. Numer. Math. 84(3), 375–393 (2000)

    Article  MathSciNet  Google Scholar 

  7. Bogo, F., Romero, J., Loper, M., Black, M.J.: FAUST: dataset and evaluation for 3D mesh registration. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3794–3801 (2014)

    Google Scholar 

  8. Bogo, F., Romero, J., Pons-Moll, G., Black, M.J.: Dynamic FAUST: registering human bodies in motion. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6233–6242, July 2017

    Google Scholar 

  9. Bonneel, N., Rabin, J., Peyré, G., Pfister, H.: Sliced and radon Wasserstein Barycenters of measures. J. Math. Imaging Vis. 51(1), 22–45 (2015)

    Article  MathSciNet  Google Scholar 

  10. Boscaini, D., Eynard, D., Kourounis, D., Bronstein, M.M.: Shape-from-operator: recovering shapes from intrinsic operators. In: Computer Graphics Forum, vol. 34, pp. 265–274. Wiley Online Library (2015)

    Google Scholar 

  11. Carmo, M.P.D.: Riemannian geometry. Birkhäuser (1992)

    Google Scholar 

  12. Chen, N., Klushyn, A., Kurle, R., Jiang, X., Bayer, J., van der Smagt, P.: Metrics for deep generative models. arXiv preprint arXiv:1711.01204 (2017)

  13. Chern, A., Knöppel, F., Pinkall, U., Schröder, P.: Shape from metric. ACM Trans. Graph. (TOG) 37(4), 63 (2018)

    Article  Google Scholar 

  14. Corman, E., Solomon, J., Ben-Chen, M., Guibas, L., Ovsjanikov, M.: Functional characterization of intrinsic and extrinsic geometry. ACM Trans. Graph. (TOG) 36(2), 1–17 (2017)

    Article  Google Scholar 

  15. Crane, K., Pinkall, U., Schröder, P.: Spin transformations of discrete surfaces. ACM Trans. Graph. (TOG) 30(4), 104 (2011)

    Article  Google Scholar 

  16. Freifeld, O., Black, M.J.: Lie bodies: a manifold representation of 3D human shape. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7572, pp. 1–14. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-33718-5_1

    Chapter  Google Scholar 

  17. Frenzel, M.F., Teleaga, B., Ushio, A.: Latent space cartography: generalised metric-inspired measures and measure-based transformations for generative models. arXiv preprint arXiv:1902.02113 (2019)

  18. Gao, L., Chen, S.Y., Lai, Y.K., Xia, S.: Data-driven shape interpolation and morphing editing. Comput. Graph. Forum 36(8), 19–31 (2017)

    Article  Google Scholar 

  19. Gao, L., Lai, Y.K., Huang, Q.X., Hu, S.M.: A data-driven approach to realistic shape morphing. Comput. Graph. Forum 32(2pt4), 449–457 (2013)

    Google Scholar 

  20. Gluck, H.: Almost all simply connected closed surfaces are rigid. In: Glaser, L.C., Rushing, T.B. (eds.) Geometric Topology. LNM, vol. 438, pp. 225–239. Springer, Heidelberg (1975). https://doi.org/10.1007/BFb0066118

    Chapter  Google Scholar 

  21. Groueix, T., Fisher, M., Kim, V.G., Russell, B.C., Aubry, M.: 3D-coded: 3D correspondences by deep deformation. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 230–246 (2018)

    Google Scholar 

  22. Hasler, N., Stoll, C., Sunkel, M., Rosenhahn, B., Seidel, H.P.: A statistical model of human pose and body shape. Comput. Graph. Forum 28(2), 337–346 (2009)

    Article  Google Scholar 

  23. Heeren, B., Rumpf, M., Schröder, P., Wardetzky, M., Wirth, B.: Exploring the geometry of the space of shells. In: Computer Graphics Forum, vol. 33, pp. 247–256. Wiley Online Library (2014)

    Google Scholar 

  24. Heeren, B., Rumpf, M., Schröder, P., Wardetzky, M., Wirth, B.: Splines in the space of shells. Comput. Graph. Forum 35(5), 111–120 (2016)

    Article  Google Scholar 

  25. Heeren, B., Rumpf, M., Wardetzky, M., Wirth, B.: Time-discrete geodesics in the space of shells. Comput. Graph. Forum 31(5), 1755–1764 (2012)

    Article  Google Scholar 

  26. Huang, J., et al.: Subspace gradient domain mesh deformation. ACM Trans. Graph. (TOG) 25(3), 1126–1134 (2006)

    Article  Google Scholar 

  27. Huang, R., Rakotosaona, M.J., Achlioptas, P., Guibas, L., Ovsjanikov, M.: OperatorNet: recovering 3D shapes from difference operators. arXiv preprint arXiv:1904.10754 (2019)

  28. Igarashi, T., Moscovich, T., Hughes, J.F.: As-rigid-as-possible shape manipulation. ACM Trans. Graph. (TOG) 24(3), 1134–1141 (2005)

    Article  Google Scholar 

  29. Kendall, D.G.: Shape manifolds, procrustean metrics, and complex projective spaces. Bull. Lond. Math. Soc. 16(2), 81–121 (1984)

    Article  MathSciNet  Google Scholar 

  30. Kilian, M., Mitra, N.J., Pottmann, H.: Geometric modeling in shape space. ACM Trans. Graph. (TOG) 26(3), 64 (2007)

    Article  Google Scholar 

  31. Laine, S.: Feature-based metrics for exploring the latent space of generative models (2018)

    Google Scholar 

  32. Lazarus, F., Verroust, A.: Three-dimensional metamorphosis: a survey. Vis. Comput. 14(8), 373–389 (1998)

    Article  Google Scholar 

  33. Li, C.L., Zaheer, M., Zhang, Y., Poczos, B., Salakhutdinov, R.: Point cloud GAN. arXiv preprint arXiv:1810.05795 (2018)

  34. Lipman, Y., Cohen-Or, D., Gal, R., Levin, D.: Volume and shape preservation via moving frame manipulation. ACM Trans. Graph. (TOG) 26(1), 5 (2007)

    Article  Google Scholar 

  35. Liu, X., Han, Z., Wen, X., Liu, Y.S., Zwicker, M.: L2G auto-encoder: understanding point clouds by local-to-global reconstruction with hierarchical self-attention. In: Proceedings of the 27th ACM International Conference on Multimedia, pp. 989–997. ACM (2019)

    Google Scholar 

  36. Michor, P.W., Mumford, D.B.: Riemannian geometries on spaces of plane curves. J. Eur. Math. Soc. (2006)

    Google Scholar 

  37. Qi, C.R., Su, H., Mo, K., Guibas, L.J.: PointNet: deep learning on point sets for 3D classification and segmentation. In: Proceedings of the CVPR, pp. 652–660 (2017)

    Google Scholar 

  38. Qi, C.R., Yi, L., Su, H., Guibas, L.J.: PointNet++: deep hierarchical feature learning on point sets in a metric space. In: Advances in Neural Information Processing Systems, pp. 5099–5108 (2017)

    Google Scholar 

  39. von Radziewsky, P., Eisemann, E., Seidel, H.P., Hildebrandt, K.: Optimized subspaces for deformation-based modeling and shape interpolation. Comput. Graph. 58, 128–138 (2016)

    Article  Google Scholar 

  40. Sassen, J., Heeren, B., Hildebrandt, K., Rumpf, M.: Solving variational problems using nonlinear rotation-invariant coordinates. In: Bommes, D., Huang, H. (eds.) Symposium on Geometry Processing 2019- Posters. The Eurographics Association (2019)

    Google Scholar 

  41. Shao, H., Kumar, A., Thomas Fletcher, P.: The Riemannian geometry of deep generative models. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 315–323 (2018)

    Google Scholar 

  42. Shen, Y., Feng, C., Yang, Y., Tian, D.: Mining point cloud local structures by kernel correlation and graph pooling. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4548–4557 (2018)

    Google Scholar 

  43. Solomon, J., et al.: Convolutional Wasserstein distances: efficient optimal transportation on geometric domains. ACM Trans. Graph. (TOG) 34(4), 66 (2015)

    Article  Google Scholar 

  44. Varol, G., et al.: Learning from synthetic humans. In: CVPR (2017)

    Google Scholar 

  45. Vaxman, A., Müller, C., Weber, O.: Conformal mesh deformations with Möbius transformations. ACM Trans. Graph. (TOG) 34(4), 55 (2015)

    Article  Google Scholar 

  46. Von Funck, W., Theisel, H., Seidel, H.P.: Vector field based shape deformations. ACM Trans. Graph. (TOG) 25(3), 1118–1125 (2006)

    Article  Google Scholar 

  47. Wang, Y., Liu, B., Tong, Y.: Linear surface reconstruction from discrete fundamental forms on triangle meshes. In: Computer Graphics Forum, vol. 31, pp. 2277–2287. Wiley Online Library (2012)

    Google Scholar 

  48. Wirth, B., Bar, L., Rumpf, M., Sapiro, G.: A continuum mechanical approach to geodesics in shape space. Int. J. Comput. Vis. 93(3), 293–318 (2011)

    Article  MathSciNet  Google Scholar 

  49. Wu, J., Zhang, C., Xue, T., Freeman, B., Tenenbaum, J.: Learning a probabilistic latent space of object shapes via 3D generative-adversarial modeling. In: Advances in Neural Information Processing Systems, pp. 82–90 (2016)

    Google Scholar 

  50. Xu, D., Zhang, H., Wang, Q., Bao, H.: Poisson shape interpolation. Graph. Models 68(3), 268–281 (2006)

    Article  Google Scholar 

  51. Zhang, Z., Li, G., Lu, H., Ouyang, Y., Yin, M., Xian, C.: Fast as-isometric-as-possible shape interpolation. Comput. Graph. 46, 244–256 (2015)

    Article  Google Scholar 

  52. Zuffi, S., Kanazawa, A., Jacobs, D., Black, M.J.: 3D menagerie: modeling the 3D shape and pose of animals. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017

    Google Scholar 

Download references

Acknowledgements

Parts of this work were supported by the KAUST CRG-2017-3426 Award and the ERC Starting Grant No. 758800 (EXPROTEA).

Author information

Authors and Affiliations

  1. LIX, Ecole Polytechnique, IP Paris, Palaiseau, France

    Marie-Julie Rakotosaona & Maks Ovsjanikov

Authors
  1. Marie-Julie Rakotosaona
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Maks Ovsjanikov
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Marie-Julie Rakotosaona .

Editor information

Editors and Affiliations

  1. University of Oxford, Oxford, UK

    Andrea Vedaldi

  2. Graz University of Technology, Graz, Austria

    Horst Bischof

  3. University of Freiburg, Freiburg im Breisgau, Germany

    Thomas Brox

  4. University of North Carolina at Chapel Hill, Chapel Hill, NC, USA

    Jan-Michael Frahm

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 34351 KB)

Supplementary material 2 (mkv 36865 KB)

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Rakotosaona, MJ., Ovsjanikov, M. (2020). Intrinsic Point Cloud Interpolation via Dual Latent Space Navigation. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, JM. (eds) Computer Vision – ECCV 2020. ECCV 2020. Lecture Notes in Computer Science(), vol 12347. Springer, Cham. https://doi.org/10.1007/978-3-030-58536-5_39

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-58536-5_39

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-58535-8

  • Online ISBN: 978-3-030-58536-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation