Skip to main content
SpringerLink
Log in

Three-stage generative network for single-view point cloud completion

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

3D shape completion from single-view scan is an important task for follow-up applications such as recognition and segmentation, but it is challenging due to the critical sparsity and structural incompleteness of single-view point clouds. In this paper, a three-stage generative network (TSGN) is proposed for single-view point cloud completion, which generates fine-grained dense point clouds step by step and effectively overcomes the ubiquitous problem—the imbalance between general and individual characteristics. In the first stage, an encoder–decoder network consumes a partial point cloud and generates a rough sparse point cloud inferring the complete geometric shape. Then, a bi-channel residual network is designed to refine the preliminary result with assistance of the original partial input. A local-based folding network is introduced in the last stage to extract local context information from the revised result and build a dense point cloud with finer-grained details. Experiments on ShapeNet dataset and KITTI dataset validate the effectiveness of TSGN. The results on ShapeNet demonstrate the competitive performance on both CD and EMD.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Marini, S., Patané, G., Spagnuolo, M., et al.: Spectral feature selection for shape characterization and classification. Vis Comput 27, 1005 (2011)

    Article  Google Scholar 

  2. Luciano, L., Ben Hamza, A.: Deep similarity network fusion for 3D shape classification. Vis Comput 35, 1171–1180 (2019)

    Article  Google Scholar 

  3. Sun, Y., Miao, Y., Chen, J., et al.: PGCNet: patch graph convolutional network for point cloud segmentation of indoor scenes. Vis Comput 36, 2407–2418 (2020)

    Article  Google Scholar 

  4. Fan, Y., Wang, M., Geng, N., et al.: A self-adaptive segmentation method for a point cloud. Vis Comput 34, 659–673 (2018)

    Article  Google Scholar 

  5. Dai, A., Qi, C. R., Nießner, M.: Shape completion using 3D-encoder-predictor CNNs and shape synthesis. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, pp. 6545–6554 (2017). doi: https://doi.org/10.1109/CVPR.2017.693

  6. Han, X, Li, Z., Huang, H., Kalogerakis, E., Yu, Y.: High-resolution shape completion using deep neural networks for global structure and local geometry inference. In: 2017 IEEE International Conference on Computer Vision (ICCV), Venice, 2017, pp. 85–93, doi: https://doi.org/10.1109/ICCV.2017.19.

  7. Litany, O., Bronstein, A., Bronstein, M., Makadia, A.: Deformable shape completion with graph convolutional autoencoders. In: 2018 IEEE/CVF conference on computer vision and pattern recognition, Salt Lake City, UT, pp. 1886–1895 (2018). doi: https://doi.org/10.1109/CVPR.2018.00202

  8. Pan, J., Han, X., Chen, W., Tang, J., Jia, K.: Deep mesh reconstruction from single RGB images via topology modification networks. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Korea (South), pp. 9963–9972 (2019). doi: https://doi.org/10.1109/ICCV.2019.01006

  9. Yuan, W., Khot, T., Held, D., Mertz, C., Hebert, M.: PCN: point completion network. In: 2018 International Conference on 3D Vision (3DV), Verona, pp. 728–737 (2018). doi:https://doi.org/10.1109/3DV.2018.00088

  10. Tchapmi, L. P., Kosaraju, V., Rezatofighi, H., Reid, I., Savarese, S.: TopNet: structural point cloud decoder. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, pp. 383–392 2019. doi: https://doi.org/10.1109/CVPR.2019.00047

  11. Huang, Z., Yu, Y., Xu, J., Ni, F., Le, X.: PF-net: point fractal network for 3D point cloud completion. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, pp. 7659–7667 (2020). doi: https://doi.org/10.1109/CVPR42600.2020.00768

  12. Liu, M., Sheng, L., Yang, S., Shao, J., Hu, S.: Morphing and sampling network for dense point cloud completion. arXiv e-prints (2019)

  13. Wang, X., Ang, M. H., Lee, G. H.: Cascaded refinement network for point cloud completion. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, pp. 787–796 (2020). doi: https://doi.org/10.1109/CVPR42600.2020.00087

  14. Sung, M., Kim, V., Angst, R., Guibas, L.: Data-driven structural priors for shape completion. ACM Trans. Graphics 34, 1–11 (2015). https://doi.org/10.1145/2816795.2818094

    Article  Google Scholar 

  15. Sipiran, V., Gregor, R., Schreck, T.: Approximate symmetry detection in partial 3D meshes. Comput. Graphics Forum (2014). doi: https://doi.org/10.1111/cgf.12481

  16. Thrun, S., Wegbreit, B.: Shape from symmetry. In: Tenth IEEE International Conference on Computer Vision (ICCV'05), vol. 1, Beijing, 2005, pp. 1824–1831 vol. 2. doi: https://doi.org/10.1109/ICCV.2005.221

  17. Rock, J., Gupta, T., Thorsen, J., Gwak, J., Shin, D., Hoiem, D.: Completing 3D object shape from one depth image. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, pp. 2484–2493 (2015). doi: https://doi.org/10.1109/CVPR.2015.7298863

  18. Li, D., Shao, T., Wu, H., Zhou, K.: Shape completion from a single RGBD image. IEEE Trans. Visual Comput. Graphics 23(7), 1809–1822 (2017). https://doi.org/10.1109/TVCG.2016.2553102

    Article  Google Scholar 

  19. Pauly, M., Mitra, N., Giesen, J., Gross, M., Guibas, L.: Example-based 3D scan completion. SGP.

  20. Sharma, A., Grau, O., Fritz, M.: VConv-DAE: deep volumetric shape learning without object labels. arXiv e-prints (2016)

  21. Smith, E., Meger, D.: Improved adversarial systems for 3D object generation and reconstruction, arXiv e-prints (2017)

  22. Charles, R. Q., Su, H., Kaichun, M., Guibas, L. J.: PointNet: deep learning on point sets for 3D classification and segmentation. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, 2017, pp. 77–85, doi: https://doi.org/10.1109/CVPR.2017.16

  23. Charles, R. Q., Yi, L., Su, H., Guibas, L. J.: "PointNet++: deep hierarchical feature learning on point sets in a metric space. arXiv e-prints (2017)

  24. Achlioptas, P., Diamanti, O., Mitliagkas, I., Guibas, L.: Learning representations and generative models for 3D point clouds," arXiv e-prints (2017)

  25. Fan, H., Su, H., Guibas, L.: A point set generation network for 3D object reconstruction from a single image. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, pp. 2463–2471 (2017). doi: https://doi.org/10.1109/CVPR.2017.264

  26. Yang, Y., Feng, C., Shen, Y., Tian, D.: FoldingNet: point cloud auto-encoder via deep grid deformation. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, pp. 206–215 (2018). doi: https://doi.org/10.1109/CVPR.2018.00029

  27. Groueix, T., Fisher, M., Kim, V. G., Russell, B. C., Aubry, M.: A Papier-Mache approach to learning 3D surface generation. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, pp. 216–224 (2018). doi: https://doi.org/10.1109/CVPR.2018.00030

  28. Wu, Z. et al.: 3D ShapeNets: a deep representation for volumetric shapes. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA, pp. 1912–1920 (2015). doi: https://doi.org/10.1109/CVPR.2015.7298801

  29. Geiger, A., et al.: Vision meets robotics: the KITTI dataset. Int. J. Robot. Res. 32(11), 1231–1237 (2013)

    Article  Google Scholar 

Download references

Acknowledgements

This work is supported by the Special Project on Basic Research of Frontier Leading Technology of Jiangsu Province of China (Grant No. BK20192004C) and Natural Science Foundation of Jiangsu Province of China (Grant No. BK20181269).

Author information

Authors and Affiliations

  1. School of Automation, Southeast University, Nanjing, 210096, Jiangsu, China

    Bingling Xiao & Feipeng Da

  2. Key Laboratory of Measurement and Control of Complex Systems of Engineering, Ministry of Education, Nanjing, 210096, Jiangsu, China

    Bingling Xiao & Feipeng Da

  3. Shenzhen Research Institute, Southeast University, Shenzhen, 518063, Guangdong, China

    Feipeng Da

Authors
  1. Bingling Xiao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Feipeng Da
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Feipeng Da.

Ethics declarations

Conflict of interest

No conflict of interest exists in the submission of this manuscript, and manuscript is approved by all authors for publication. I would like to declare on behalf of my co-authors that the work described was original research that has not been published previously and not under consideration for publication elsewhere, in whole or in part. All the authors listed have approved the manuscript that is enclosed.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xiao, B., Da, F. Three-stage generative network for single-view point cloud completion. Vis Comput 38, 4373–4382 (2022). https://doi.org/10.1007/s00371-021-02301-4

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-021-02301-4

Keywords

Search

Navigation