Skip to main content
SpringerLink
Log in

Point-Level Label-Free Segmentation Framework for 3D Point Cloud Semantic Mining

  • Conference paper
  • First Online:
Advanced Data Mining and Applications (ADMA 2023)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 14176))

Included in the following conference series:

  • 777 Accesses

Abstract

3D point cloud data semantic mining plays a key role in 3D scene understanding. Although recent point cloud semantic mining methods have achieved great success, they require large amounts of expensive manual annotated data. More importantly, the lack of large-scale annotated datasets limits those approaches in many real-world applications, especially for point-level semantic mining tasks such as point cloud semantic segmentation. In this work, we propose a novel point cloud segmentation framework, called Point-level Label-free Segmentation framework (PLS), that does not require point-level annotations. In this framework, the point cloud semantic mining task is formulated as a clustering problem based on mutual information. Meanwhile, our method can directly predict clusters that correspond to the given semantic classes in a single feed-forward pass of a neural network. We apply the proposed PLS to the shape part segmentation task. Experiments on the benchmark ShapeNetPart dataset demonstrate that our method has the ability to discover clusters that match semantic classes, and it can produce comparable results with methods using incomplete labels on several categories.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Chen, Z., Yin, K., Fisher, M., Chaudhuri, S., Zhang, H.: Bae-net: branched autoencoder for shape co-segmentation. In: International Conference on Computer Vision, pp. 8490–8499 (2019)

    Google Scholar 

  2. Choy, C., Gwak, J., Savarese, S.: 4d spatio-temporal convnets: Minkowski convolutional neural networks. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 3075–3084 (2019)

    Google Scholar 

  3. Choy, C., Park, J., Koltun, V.: Fully convolutional geometric features. In: International Conference on Computer Vision, pp. 8958–8966 (2019)

    Google Scholar 

  4. Deng, H., Birdal, T., Ilic, S.: PPF-foldnet: unsupervised learning of rotation invariant 3d local descriptors. In: European Conference on Computer Vision, pp. 602–618 (2018)

    Google Scholar 

  5. Du, A., Pang, S., Huang, X., Zhang, J., Wu, Q.: Exploring long-short-term context for point cloud semantic segmentation. In: IEEE International Conference on Image Processing, pp. 2755–2759. IEEE (2020)

    Google Scholar 

  6. Graham, B., Engelcke, M., van der Maaten, L.: 3d semantic segmentation with submanifold sparse convolutional networks. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 9224–9232 (2018)

    Google Scholar 

  7. Hou, J., Graham, B., Nießner, M.: Exploring data-efficient 3D scene understanding with contrastive scene contexts. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 15587–15597 (2021)

    Google Scholar 

  8. Hou, Y., Zhu, X., Ma, Y., Loy, C.C., Li, Y.: Point-to-voxel knowledge distillation for lidar semantic segmentation. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 8479–8488 (2022)

    Google Scholar 

  9. Hu, Q., et al.: SQN: weakly-supervised semantic segmentation of large-scale 3D point clouds. In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T. (eds.) Computer Vision – ECCV 2022: 17th European Conference, Tel Aviv, 23–27 October 2022, Proceedings, Part XXVII, pp. 600–619. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-19812-0_35

  10. Ji, X., Henriques, J.F., Vedaldi, A.: Invariant information clustering for unsupervised image classification and segmentation. In: International Conference on Computer Vision, pp. 9865–9874 (2019)

    Google Scholar 

  11. Lai, X., et al.: Stratified transformer for 3d point cloud segmentation. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 8500–8509 (2022)

    Google Scholar 

  12. Li, J., Dai, H., Han, H., Ding, Y.: Mseg3d: multi-modal 3d semantic segmentation for autonomous driving. arXiv preprint arXiv:2303.08600 (2023)

  13. Li, J., Chen, B.M., Hee Lee, G.: So-net: self-organizing network for point cloud analysis. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 9397–9406 (2018)

    Google Scholar 

  14. Liu, F., Liu, X.: Learning implicit functions for topology-varying dense 3d shape correspondence. In: Advances Neural Information Processing Systems, vol. 33, pp. 4823–4834 (2020)

    Google Scholar 

  15. Liu, Z., Qi, X., Fu, C.W.: One thing one click: a self-training approach for weakly supervised 3d semantic segmentation. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1726–1736 (2021)

    Google Scholar 

  16. Mei, G., et al.: Data augmentation-free unsupervised learning for 3d point cloud understanding. In: Britain Machine Visual Conference (2022)

    Google Scholar 

  17. Niu, C., Li, M., Xu, K., Zhang, H.: Rim-net: recursive implicit fields for unsupervised learning of hierarchical shape structures. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11779–11788 (2022)

    Google Scholar 

  18. Pang, Y., Wang, W., Tay, F.E.H., Liu, W., Tian, Y., Yuan, L.: Masked autoencoders for point cloud self-supervised learning. In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T. (eds.) Computer Vision – ECCV 2022: 17th European Conference, Tel Aviv, 23–27 October 2022, Proceedings, Part II, pp. 604–621. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-20086-1_35

  19. Qi, C.R., Su, H., Mo, K., Guibas, L.J.: Pointnet: deep learning on point sets for 3d classification and segmentation. In: IEEE Conference on Computer Vision Pattern Recognition, pp. 652–660 (2017)

    Google Scholar 

  20. Qi, C.R., Yi, L., Su, H., Guibas, L.J.: Pointnet++: deep hierarchical feature learning on point sets in a metric space. In: Advances Neural Information Processing Systems, pp. 5099–5108 (2017)

    Google Scholar 

  21. Sauder, J., Sievers, B.: Self-supervised deep learning on point clouds by reconstructing space. In: Advances Neural Information Processing Systems, pp. 12962–12972 (2019)

    Google Scholar 

  22. Shi, J., Malik, J.: Normalized cuts and image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 22(8), 888–905 (2000)

    Article  Google Scholar 

  23. Tarvainen, A., Valpola, H.: Mean teachers are better role models: weight-averaged consistency targets improve semi-supervised deep learning results. In: Advances Neural Information Processing Systems, vol. 30, pp. 1195–1204 (2017)

    Google Scholar 

  24. Tatarchenko, M., Park, J., Koltun, V., Zhou, Q.Y.: Tangent convolutions for dense prediction in 3d. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 3887–3896 (2018). https://doi.org/10.1109/CVPR.2018.00409

  25. Thomas, H., Qi, C.R., Deschaud, J.E., Marcotegui, B., Goulette, F., Guibas, L.J.: Kpconv: flexible and deformable convolution for point clouds. In: International Conference on Computer Vision, pp. 6411–6420 (2019)

    Google Scholar 

  26. Wang, P.S., Yang, Y.Q., Zou, Q.F., Wu, Z., Liu, Y., Tong, X.: Unsupervised 3d learning for shape analysis via multiresolution instance discrimination. In: AAAI, vol. 35, pp. 2773–2781 (2021)

    Google Scholar 

  27. Wei, J., Lin, G., Yap, K.H., Hung, T.Y., Xie, L.: Multi-path region mining for weakly supervised 3d semantic segmentation on point clouds. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 4384–4393 (2020)

    Google Scholar 

  28. Wu, W., Qi, Z., Fuxin, L.: Pointconv: deep convolutional networks on 3d point clouds. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 9621–9630 (2019)

    Google Scholar 

  29. Xie, S., Gu, J., Guo, D., Qi, C.R., Guibas, L., Litany, O.: PointContrast: unsupervised pre-training for 3D point cloud understanding. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12348, pp. 574–591. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58580-8_34

  30. Xu, C., et al.: You only group once: efficient point-cloud processing with token representation and relation inference module. In: IEEE International Conference on Intelligent Robots and Systems, pp. 4589–4596 (2021). https://doi.org/10.1109/IROS51168.2021.9636858

  31. Xu, X., Lee, G.H.: Weakly supervised semantic point cloud segmentation: towards 10x fewer labels. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 13706–13715 (2020)

    Google Scholar 

  32. Yang, J., Lee, C., Ahn, P., Lee, H., Yi, E., Kim, J.: Pbp-net: point projection and back-projection network for 3d point cloud segmentation. In: IEEE International Conference on Intelligent Robots and Systems, pp. 8469–8475 (2020). https://doi.org/10.1109/IROS45743.2020.9341776

  33. Yi, L., et al.: A scalable active framework for region annotation in 3d shape collections. ACM Trans. Graph. 35(6), 1–12 (2016)

    Google Scholar 

  34. Zheng, Z., Yang, Y.: Rectifying pseudo label learning via uncertainty estimation for domain adaptive semantic segmentation. Int. J. Comput. Vis. 129(4), 1106–1120 (2021)

    Article  Google Scholar 

Download references

Acknowledgement

This work is supported in part by the National Natural Science Foundation of China under Grants No. 62206128 and in part by the National Computational Infrastructure (NCI) through Australian Government.

Author information

Authors and Affiliations

  1. University of New South Wales, Canberra, Australia

    Anan Du

  2. Nanjing University of Science and Technology, Nanjing, China

    Shuchao Pang

  3. Macquarie University, Sydney, Australia

    Shuchao Pang & Mehmet Orgun

Authors
  1. Anan Du
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shuchao Pang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mehmet Orgun
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shuchao Pang .

Editor information

Editors and Affiliations

  1. Northeastern University, Shenyang, China

    Xiaochun Yang

  2. The University of Indonesia, Depok, Indonesia

    Heru Suhartanto

  3. Beijing Institute of Technology, Beijing, China

    Guoren Wang

  4. Northeastern University, Shenyang, China

    Bin Wang

  5. University of Technology Sydney, Sydney, NSW, Australia

    Jing Jiang

  6. Agency for Science, Technology and Research (A*STAR), Singapore, Singapore

    Bing Li

  7. Sun Yat-sen University, Guangzhou, China

    Huaijie Zhu

  8. Anhui University, Hefei, China

    Ningning Cui

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Du, A., Pang, S., Orgun, M. (2023). Point-Level Label-Free Segmentation Framework for 3D Point Cloud Semantic Mining. In: Yang, X., et al. Advanced Data Mining and Applications. ADMA 2023. Lecture Notes in Computer Science(), vol 14176. Springer, Cham. https://doi.org/10.1007/978-3-031-46661-8_28

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-46661-8_28

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-46660-1

  • Online ISBN: 978-3-031-46661-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation