Skip to main content

The Shape Part Slot Machine: Contact-Based Reasoning for Generating 3D Shapes from Parts

  • Conference paper
  • First Online:
Computer Vision – ECCV 2022 (ECCV 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13663))

Included in the following conference series:

Abstract

We present the Shape Part Slot Machine, a new method for assembling novel 3D shapes from existing parts by performing contact-based reasoning. Our method represents each shape as a graph of “slots,” where each slot is a region of contact between two shape parts. Based on this representation, we design a graph-neural-network-based model for generating new slot graphs and retrieving compatible parts, as well as a gradient-descent-based optimization scheme for assembling the retrieved parts into a complete shape that respects the generated slot graph. This approach does not require any semantic part labels; interestingly, it also does not require complete part geometries—reasoning about the slots proves sufficient to generate novel, high-quality 3D shapes. We demonstrate that our method generates shapes that outperform existing modeling-by-assembly approaches regarding quality, diversity, and structural complexity.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Averkiou, M., Kim, V., Zheng, Y., Mitra, N.J.: ShapeSynth: parameterizing model collections for coupled shape exploration and synthesis. In: Computer Graphics Forum (Special Issue of Eurographics 2014) (2014)

    Google Scholar 

  2. Chaudhuri, S., Kalogerakis, E., Giguere, S., Funkhouser, T.: ATTRIBIT: content creation with semantic attributes. In: ACM Symposium on User Interface Software and Technology (UIST) (2013)

    Google Scholar 

  3. Chaudhuri, S., Kalogerakis, E., Guibas, L., Koltun, V.: Probabilistic reasoning for assembly-based 3D modeling. ACM Trans. Graphics 30(4), 1–10 (2011)

    Article  Google Scholar 

  4. Funkhouser, T., et al.: Modeling by example. In: ACM SIGGRAPH 2004 Papers (2004)

    Google Scholar 

  5. Gao, L., et al.: SDM-Net: deep generative network for structured deformable mesh. In: SIGGRAPH Asia (2019)

    Google Scholar 

  6. Gilmer, J., Schoenholz, S.S., Riley, P.F., Vinyals, O., Dahl, G.E.: Neural message passing for quantum chemistry. CoRR arXiv:1704.01212 (2017)

  7. Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., Hochreiter, S.: GANs trained by a two time-scale update rule converge to a local nash equilibrium. In: NeurIPS (2017)

    Google Scholar 

  8. Huang, J., et al.: Generative 3D part assembly via dynamic graph learning. In: The IEEE Conference on Neural Information Processing Systems (NeurIPS) (2020)

    Google Scholar 

  9. Jain, A., Thormählen, T., Ritschel, T., Seidel, H.P.: Exploring shape variations by 3D-model decomposition and part-based recombination. Comput. Graph. Forum 31(2pt3), 631–640 (2012). https://doi.org/10.1111/j.1467-8659.2012.03042.x

  10. Jones, R.K., et al.: ShapeAssembly: learning to generate programs for 3D shape structure synthesis. ACM Trans. Graphics (TOG), SIGGRAPH Asia 2020 39(6), Article 234 (2020)

    Google Scholar 

  11. Jones, R.K., Charatan, D., Guerrero, P., Mitra, N.J., Ritchie, D.: ShapeMOD: macro operation discovery for 3D shape programs. ACM Trans. Graphics (TOG) 40(4), 1–16 (2021)

    Article  Google Scholar 

  12. Kalogerakis, E., Chaudhuri, S., Koller, D., Koltun, V.: A probabilistic model for component-based shape synthesis. ACM Trans. Graphics 31(4), 1–11 (2012)

    Article  Google Scholar 

  13. Kreavoy, V., Julius, D., Sheffer, A.: Model composition from interchangeable components. In: Proceedings of the 15th Pacific Conference on Computer Graphics and Applications, PG 2007, pp. 129–138. IEEE Computer Society, USA (2007). https://doi.org/10.1109/PG.2007.43

  14. Li, J., Xu, K., Chaudhuri, S., Yumer, E., Zhang, H., Guibas, L.: GRASS: generative recursive autoencoders for shape structures. In: SIGGRAPH 2017 (2017)

    Google Scholar 

  15. Mo, K., et al.: StructureNet: hierarchical graph networks for 3D shape generation. In: SIGGRAPH Asia (2019)

    Google Scholar 

  16. Mo, K., et al.: PartNet: a large-scale benchmark for fine-grained and hierarchical part-level 3D object understanding. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 909–918 (2019)

    Google Scholar 

  17. Schor, N., Katzir, O., Zhang, H., Cohen-Or, D.: CompoNet: learning to generate the unseen by part synthesis and composition. In: The IEEE International Conference on Computer Vision (ICCV) (2019)

    Google Scholar 

  18. Shen, C.H., Fu, H., Chen, K., Hu, S.M.: Structure recovery by part assembly. ACM Trans. Graphics 31(6) (2012). https://doi.org/10.1145/2366145.2366199

  19. Sung, M., Su, H., Kim, V.G., Chaudhuri, S., Guibas, L.: ComplementMe: weakly-supervised component suggestions for 3D modeling. ACM Trans. Graphics (TOG) 36(6), 226 (2017)

    Article  Google Scholar 

  20. Wang, H., Schor, N., Hu, R., Huang, H., Cohen-Or, D., Huang, H.: Global-to-local generative model for 3D shapes. ACM Trans. Graphics (Proc. SIGGRAPH ASIA) 37(6), 214:1–214:10 (2018)

    Google Scholar 

  21. Wu, R., Zhuang, Y., Xu, K., Zhang, H., Chen, B.: PQ-Net: a generative part seq2seq network for 3D shapes. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2020)

    Google Scholar 

  22. Wu, Z., Wang, X., Lin, D., Lischinski, D., Cohen-Or, D., Huang, H.: SAGNet: structure-aware generative network for 3D-shape modeling. ACM Trans. Graphics (Proc. SIGGRAPH 2019) 38(4), 91:1–91:14 (2019)

    Google Scholar 

  23. Xie, X., et al.: Sketch-to-design: context-based part assembly. Comput. Graph. Forum 32(8), 233–245 (2013)

    Article  Google Scholar 

  24. Xu, K., Zhang, H., Cohen-Or, D., Chen, B.: Fit and diverse: set evolution for inspiring 3D shape galleries. ACM Trans. Graphics (Proc. of SIGGRAPH 2012) 31(4), 57:1–57:10 (2012)

    Google Scholar 

  25. Yang, J., Mo, K., Lai, Y.K., Guibas, L.J., Gao, L.: DSM-Net: disentangled structured mesh net for controllable generation of fine geometry (2020)

    Google Scholar 

  26. Yin, K., Chen, Z., Chaudhuri, S., Fisher, M., Kim, V., Zhang, H.: COALESCE: component assembly by learning to synthesize connections. In: Proceedings of 3DV (2020)

    Google Scholar 

  27. Zou, C., Yumer, E., Yang, J., Ceylan, D., Hoiem, D.: 3D-PRNN: generating shape primitives with recurrent neural networks. In: ICCV 2017 (2017)

    Google Scholar 

Download references

Acknowledgements

This work was funded in part by NSF award #1907547 and a gift fund from Adobe. Daniel Ritchie is an advisor to Geopipe and owns equity in the company. Geopipe is a start-up that is developing 3D technology to build immersive virtual copies of the real world with applications in various fields, including games and architecture. Minhyuk Sung acknowledges the support of NRF grant (2022R1F1A1068681), NST grant (CRC 21011), and IITP grant (2022-0-00594) funded by the Korea government (MSIT), and grants from Adobe, KT, and Samsung Electronics corporations. Part of this work was done when Kai Wang interned at Adobe.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Kai Wang .

Editor information

Editors and Affiliations

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 15423 KB)

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Wang, K., Guerrero, P., Kim, V.G., Chaudhuri, S., Sung, M., Ritchie, D. (2022). The Shape Part Slot Machine: Contact-Based Reasoning for Generating 3D Shapes from Parts. In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T. (eds) Computer Vision – ECCV 2022. ECCV 2022. Lecture Notes in Computer Science, vol 13663. Springer, Cham. https://doi.org/10.1007/978-3-031-20062-5_35

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-20062-5_35

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-20061-8

  • Online ISBN: 978-3-031-20062-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics