Skip to main content
SpringerLink
Log in

Spotlights: Probing Shapes from Spherical Viewpoints

  • Conference paper
  • First Online:
Computer Vision – ACCV 2022 (ACCV 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13841))

Included in the following conference series:

  • 423 Accesses

Abstract

Recent years have witnessed the surge of learned representations that directly build upon point clouds. Inspired by spherical multi-view scanners, we propose a novel sampling model called Spotlights to represent a 3D shape as a compact 1D array of depth values. It simulates the configuration of cameras evenly distributed on a sphere, where each virtual camera casts light rays from its principal point to probe for possible intersections with the object surrounded by the sphere. The structured point cloud is hence given implicitly as a function of depths. We provide a detailed geometric analysis of this new sampling scheme and prove its effectiveness in the context of the point cloud completion task. Experimental results on both synthetic and real dataset demonstrate that our method achieves competitive accuracy and consistency while at a lower computational cost. The code and dataset will be released at https://github.com/goldoak/Spotlights.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Achlioptas, P., Diamanti, O., Mitliagkas, I., Guibas, L.J.: Learning representations and generative models for 3D point clouds. In: Proceedings of the IEEE International Conference on Machine Learning (ICML), pp. 40–49 (2018)

    Google Scholar 

  2. Campos, C., Elvira, R., Rodríguez, J.J.G., Montiel, J.M., Tardós, J.D.: ORB-SLAM3: an accurate open-source library for visual, visual-inertial and multi-map slam. IEEE Trans. Rob. (T-RO) 37(6), 1874–1890 (2021)

    Google Scholar 

  3. Chang, A.X., et al.: Shapenet: an information-rich 3D model repository. arXiv preprint arXiv:1512.03012 (2015)

  4. Chen, D.Y., Tian, X.P., Shen, Y.T., Ouhyoung, M.: On visual similarity based 3d model retrieval. In: Computer Graphics Forum, vol. 22, pp. 223–232. Wiley Online Library (2003)

    Google Scholar 

  5. Dai, A., Qi, C.R., Nießner, M.: Shape completion using 3D-encoder-predictor cnns and shape synthesis. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5868–5877 (2017)

    Google Scholar 

  6. Davis, J., Marschner, S.R., Garr, M., Levoy, M.: Filling holes in complex surfaces using volumetric diffusion. In: In Proceedings of FIrst International Symposium on 3D Data Processing Visualization and Transmission (2002)

    Google Scholar 

  7. Deng, J., Shi, S., Li, P., Zhou, W., Zhang, Y., Li, H.: Voxel r-cnn: towards high performance voxel-based 3d object detection. arXiv preprint arXiv:2012.15712 1(2), 4 (2020)

  8. Dosovitskiy, A., Ros, G., Codevilla, F., Lopez, A., Koltun, V.: CARLA: an open urban driving simulator. In: Proceedings of the 1st Annual Conference on Robot Learning, pp. 1–16 (2017)

    Google Scholar 

  9. Fan, H., Su, H., Guibas, L.J.: A point set generation network for 3D object reconstruction from a single image. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 605–613 (2017)

    Google Scholar 

  10. Furukawa, Y., Ponce, J.: Accurate, dense, and robust multi-view stereopsis. IEEE Trans. Pattern Anal. Mach. Intell. (PAMI) 32, 1362–1376 (2010)

    Article  Google Scholar 

  11. Garsthagen, R.: An open source, low-cost, multi camera full-body 3D scanner. In: Proceedings of 5th International Conference on 3D Body Scanning Technologies, pp. 174–183 (2014)

    Google Scholar 

  12. Groueix, T., Fisher, M., Kim, V., Russell, B., Aubry, M.: AtlasNet: a papier-mache approach to learning 3d surface generation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2018)

    Google Scholar 

  13. Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision, 2nd edn. Cambridge University Press, Cambridge (2003)

    MATH  Google Scholar 

  14. Huang, Z., Yu, Y., Xu, J., Ni, F., Le, X.: PF-Net: point fractal network for 3D point cloud completion. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7662–7670 (2020)

    Google Scholar 

  15. Keinert, B., Innmann, M., Sänger, M., Stamminger, M.: Spherical fibonacci mapping. ACM Trans. Graph. (TOG) 34(6), 1–7 (2015)

    Article  Google Scholar 

  16. Lang, A.H., Vora, S., Caesar, H., Zhou, L., Yang, J., Beijbom, O.: Pointpillars: fast encoders for object detection from point clouds. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12697–12705 (2019)

    Google Scholar 

  17. Li, P., Qin, T., Shen, S.: Stereo vision-based semantic 3D object and ego-motion tracking for autonomous driving. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 646–661 (2018)

    Google Scholar 

  18. Li, R., Li, X., Hui, K.H., Fu, C.W.: Sp-gan: sphere-guided 3D shape generation and manipulation. ACM Trans. Graph. 40(4) (2021)

    Google Scholar 

  19. Li, Y., Dai, A., Guibas, L., Nießner, M.: Database-assisted object retrieval for real-time 3d reconstruction. Comput. Graph. Forum 34, 435–446 (2015)

    Article  Google Scholar 

  20. Liao, Y., Xie, J., Geiger, A.: KITTI-360: a novel dataset and benchmarks for urban scene understanding in 2D and 3D. arXiv preprint arXiv:2109.13410 (2021)

  21. McCormac, J., Clark, R., Bloesch, M., Davison, A., Leutenegger, S.: Fusion++: volumetric object-level slam. In: Proceedings of the International Conference on 3D Vision (3DV), pp. 32–41 (2018)

    Google Scholar 

  22. Newcombe, R., et al.: KinectFusion: real-time dense surface mapping and tracking. In: Proceedings of the International Symposium on Mixed and Augmented Reality (ISMAR) (2011)

    Google Scholar 

  23. Newcombe, R., Lovegrove, S., Davison, A.: DTAM: dense tracking and mapping in real-time. In: Proceedings of the International Conference on Computer Vision (ICCV), pp. 2320–2327 (2011)

    Google Scholar 

  24. Park, J.J., Florence, P., Straub, J., Newcombe, R., Lovegrove, S.: DeepSDF: learning continuous signed distance functions for shape representation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2019)

    Google Scholar 

  25. Pauly, M., Mitra, N.J., Giesen, J., Gross, M.H., Guibas, L.J.: Example-based 3D scan completion (EPFL-CONF-149337), pp. 23–32 (2005)

    Google Scholar 

  26. Pesce, M., Galantucci, L., Percoco, G., Lavecchia, F.: A low-cost multi camera 3D scanning system for quality measurement of non-static subjects. Procedia CIRP 28, 88–93 (2015)

    Article  Google Scholar 

  27. Qi, C.R., Chen, X., Litany, O., Guibas, L.J.: Imvotenet: boosting 3D object detection in point clouds with image votes. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4404–4413 (2020)

    Google Scholar 

  28. Qi, C.R., Litany, O., He, K., Guibas, L.J.: Deep hough voting for 3D object detection in point clouds. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 9277–9286 (2019)

    Google Scholar 

  29. Qi, C.R., Liu, W., Wu, C., Su, H., Guibas, L.J.: Frustum pointnets for 3D object detection from rgb-d data. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 918–927 (2018)

    Google Scholar 

  30. Qi, C.R., Su, H., Mo, K., Guibas, L.J.: Pointnet: deep learning on point sets for 3D classification and segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 652–660 (2017)

    Google Scholar 

  31. Qi, C.R., Yi, L., Su, H., Guibas, L.J.: Pointnet++: deep hierarchical feature learning on point sets in a metric space. Adv. Neural Inf. Process. Syst. 30, 1–10 (2017)

    Google Scholar 

  32. Seitz, S.M., Curless, B., Diebel, J., Scharstein, D., Szeliski, R.: A comparison and evaluation of multi-view stereo reconstruction algorithms. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2006), vol. 1, pp. 519–528. IEEE (2006)

    Google Scholar 

  33. Shan, T., Englot, B., Meyers, D.: LIO-SAM: tightly-coupled lidar inertial odometry via smoothing and mapping. In: Proceedings of the IEEE/RSJ Conference on Intelligent Robots and Systems (IROS) (2020)

    Google Scholar 

  34. Shin, D., Fowlkes, C.C., Hoiem, D.: Pixels, voxels, and views: a study of shape representations for single view 3D object shape prediction. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2018)

    Google Scholar 

  35. Shin, D., Fowlkes, C.C., Hoiem, D.: Pixels, voxels, and views: a study of shape representations for single view 3D object shape prediction. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3061–3069 (2018)

    Google Scholar 

  36. Sucar, E., Wada, K., Davison, A.: NodeSLAM: neural object descriptors for multi-view shape reconstruction. In: Proceedings of the International Conference on 3D Vision (3DV) (2020)

    Google Scholar 

  37. Tchapmi, L.P., Kosaraju, V., Rezatofighi, H., Reid, I., Savarese, S.: Topnet: structural point cloud decoder. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 383–392 (2019)

    Google Scholar 

  38. Thrun, S., Wegbreit, B.: Shape from symmetry. In: Proceedings of the International Conference on Computer Vision (ICCV) (2005)

    Google Scholar 

  39. Wen, X., Li, T., Han, Z., Liu, Y.S.: Point cloud completion by skip-attention network with hierarchical folding. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2020)

    Google Scholar 

  40. Wen, X., et al.: PMP-Net: point cloud completion by learning multi-step point moving paths. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2021)

    Google Scholar 

  41. Wu, Z., et al.: 3D ShapeNets: a deep representation for volumetric shapes. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2015)

    Google Scholar 

  42. Xie, H., Yao, H., Zhou, S., Mao, J., Zhang, S., Sun, W.: GRNet: gridding residual network for dense point cloud completion. In: Proceedings of the European Conference on Computer Vision (ECCV) (2020)

    Google Scholar 

  43. Yang, Y., Feng, C., Shen, Y., Tian, D.: Foldingnet: point cloud auto-encoder via deep grid deformation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 206–215 (2018)

    Google Scholar 

  44. Yuan, W., Khot, T., Held, D., Mertz, C., Hebert, M.: PCN: point completion network. In: Proceedings of the International Conference on 3D Vision (3DV), pp. 728–737. IEEE (2018)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. ShanghaiTech University, Shanghai, China

    Jiaxin Wei, Wenqing Jiang, Sören Schwertfeger & Laurent Kneip

  2. RoboZone, Midea Inc., Foshan, China

    Lige Liu, Ran Cheng, Minghao Xu, Xinyu Jiang & Tao Sun

Authors
  1. Jiaxin Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lige Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ran Cheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Wenqing Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Minghao Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Xinyu Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Tao Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Sören Schwertfeger
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Laurent Kneip
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tao Sun .

Editor information

Editors and Affiliations

  1. University of Wollongong, Wollongong, NSW, Australia

    Lei Wang

  2. University of Bonn, Bonn, Germany

    Juergen Gall

  3. University of Adelaide, Adelaide, SA, Australia

    Tat-Jun Chin

  4. National Institute of Informatics, Tokyo, Japan

    Imari Sato

  5. Johns Hopkins University, Baltimore, MD, USA

    Rama Chellappa

1 Electronic supplementary material

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Wei, J. et al. (2023). Spotlights: Probing Shapes from Spherical Viewpoints. In: Wang, L., Gall, J., Chin, TJ., Sato, I., Chellappa, R. (eds) Computer Vision – ACCV 2022. ACCV 2022. Lecture Notes in Computer Science, vol 13841. Springer, Cham. https://doi.org/10.1007/978-3-031-26319-4_28

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-26319-4_28

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-26318-7

  • Online ISBN: 978-3-031-26319-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation