Skip to main content
SpringerLink
Log in

Topological and geometrical joint learning for 3D graph data

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Traditional convolutional neural networks (CNNs) are limited to be directly applied to 3D graph data due to their inherent grid structure. And most of graph-based learning methods use local-to-global hierarchical structure learning, and often ignore the global context. To overcome these issues, we propose two strategies: one is topological learning with 3D offset convolution, which provides learnable parameters in local graph construction, effectively expands the sampling space and improves the perception ability of diverse local structures. The other is geometrical learning with an adaptive spec-graph convolution network (AsGCN), which establishes a joint learning mechanism of local geometry in spatial domain and global structure in feature domain, and generates informative deep features through spectral filtering and weighting. Extensive experiments demonstrate that our deep features have strong discerning ability and robustness to non-rigid transformed graph data, incomplete mesh data, and better performance can be obtained compared to state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Data Availability

The datasets generated and/or analysed during the current study are available in the GCN repository at https://github.com/zizigbjuan/GCN

Some models, or code generated or used during the study are available from the corresponding author by request.

References

  1. Bansal M, Kumar M, Sachdeva M, Mittal A (2021) Transfer learning for image classification using VGG19: Caltech-101 image data set. J Ambient Intell Humaniz Comput:1–12

  2. Berwin A (1993) Bandwidth selection in kernel density estimation: a review. In CORE and Institut de Statistique Citeseer

  3. Bronstein A, Bronstein M, Guibas L, Ovsjanikov M (2011) Shape Google: geometric words and expressions for invariant shape retrieval. ACM Trans Graph 30(1):1–22

    Article  Google Scholar 

  4. Bruna J, Zaremba W, Szlam A, LeCun Y (2013) Spectral networks and locally connected networks on graphs. Computer Science

  5. Cangea C, Veličković P, Jovanović N, Kipf T, Liò P (2018) Towards sparse hierarchical graph classifiers. arXiv preprint arXiv:1811.01287

  6. Chen J, Ma T, Xiao C (2018) Fastgcn: fast learning with graph convolutional networks via importance sampling. In Proc. International Conference on Learning Representations (ICLR)

  7. Chhabra P, Garg NK (2020) M K umar. Content-based image retrieval system using ORB and SIFT features. Neural Comput & Applic 32(7):1–9

    Article  Google Scholar 

  8. Chiang W, Liu X, Si S, Li Y, Bengio S, Hsieh C (2019) Cluster-gcn: an efficient algorithm for training deep and large graph convolutional networks. In: Proc. of KDD. ACM

  9. Defferrard M, Bresson X, Vandergheynst P (2016) Convolutional neural networks on graphs with fast localized spectral filtering. In Advances in Neural Information Processing Systems, 3844–3852

  10. Gilmer J, Schoenholz S, Riley P, Vinyals O, Dahl G (2017) Neural message passing for quantum chemistry. In Proc. 34th international conference on machine learning, PMLR. Sydney, Australia: 1263–1272

  11. Hamilton W, Ying R, Leskovec J (2017) Inductive representation learning on large graphs. In NIPS

  12. Hammond D, Vandergheynst P, Gribonval R (2011) Wavelets on graphs via spectral graph theory. Appl Comput Harmon Anal 30(2):129–150

    Article  MathSciNet  MATH  Google Scholar 

  13. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In CVPR, USA: 770–778

  14. Hermosilla P, Ritschel T, Vazquez P, Vinacua A, Ropinski T (2018) Monte carlo convolution for learning on non-uniformly sampled point clouds. In SIGGRAPH Asia 2018 Technical Papers, page 235. ACM

  15. Kipf TN, Welling M (2017) Semi-supervised classification with graph convolutional networks. In: Proc. of International Conference on Learning Representations (ICLR)

  16. Klokov R, Lempitsky V (2017) Escape from Cells: deep Kd-Networks for the recognition of 3d point cloud models. In Proc. IEEE International Conference on Computer Vision (ICCV): 863–872

  17. Kumar M, Bansal M, Kumar M (2020) 2D object recognition techniques: state-of-the-art work. Archives of Computational Methods in Engineering 5:1–15

    MathSciNet  Google Scholar 

  18. Lee J, Rossi R, Kim S, Ahmed N, Koh E (2018) Attention models in graphs: A survey. arXiv preprint arXiv:1807.07984

  19. Liu X, Han Z, Liu Y, Zwicker M (2019) Point2Sequence: learning the shape representation of 3D point clouds with an attention-based sequence to sequence network. In proc. AAAI Conference on Artificial Intelligence, 33:8778–8785

  20. Liu JX, Ni BB, Li CY, Yang JC, Tian Q (2019) Dynamic points agglomeration for hierarchical point sets learning. In Proc. IEEE International Conference on Computer Vision (ICCV). Seoul, Korea (South): IEEE: 7546–7555

  21. Liu S, Wang S, Liu X et al (2020) Fuzzy Detection aided Real-time and Robust Visual Tracking under Complex Environments. IEEE Transactions on Fuzzy Systems 99:1

    Google Scholar 

  22. Liu S, Wang S, Liu X, Gandomi AH, Daneshmand M, Muhammad K, de Albuquerque VHC (2021) Human memory update strategy: a multi-layer template update mechanism for remote visual monitoring. IEEE Transactions on Multimedia 99:1

    Article  Google Scholar 

  23. Luciano L, Hamza A (2017) Deep learning with geodesic moments for 3D shape classification. Pattern Recog Lett 105:182–190

    Article  Google Scholar 

  24. Luciano L, Hamza A (2018) Deep learning with geodesic moments for 3D shape classification. Pattern Recog Lett 105:182–190

    Article  Google Scholar 

  25. Masoumi M, Li C, Hamza A (2016) A spectral graph wavelet approach for non rigid 3D shape retrieval. Pattern Recogn Lett 83:339–348

    Article  Google Scholar 

  26. Masoumi M, Li C, Hamza A (2016) A spectral graph wavelet approach for non rigid 3D shape retrieval. Pattern Recogn Lett 83:339–348

    Article  Google Scholar 

  27. Niepert M, Ahmed M, Kutzkov K (2016) Learning convolutional neural networks for graphs. In ICML, 2014–2023

  28. Qi C, Su H, Mo K, Guibas L (2017) PointNet: deep learning on point sets for 3D classification and segmentation. In CVPR: 77–85

  29. Qi C, Yi L, Su H, Guibas L (2017) PointNet++: deep hierarchical feature learning on point sets in a metric space. Adv Neural Inf Proces Syst 30:5099–5108

    Google Scholar 

  30. Ren S, He K, Girshick R, Sun J (2017) Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis & Machine Intelligence 39(6):1137–1149

    Article  Google Scholar 

  31. Scarselli F, Gori M, Tsoi A, Hagenbuchner M, Monfardini G (2009) The graph neural network model. IEEE Trans Neural Netw 20(1):61–80

    Article  Google Scholar 

  32. Simonovsky M (2017) Dynamic edge-conditioned filters in convolutional neural networks on graphs. In CVPR

  33. Thomas H, Qi C, Deschaud J et al. (2019) KPConv: flexible and deformable convolution for point clouds. In Proc. ICCV

  34. Velickovic P, Cucurull G, Casanova A, Romero A, Lio P, Bengio Y (2018) Graph attention networks. In Proc. 7th International Conference on Learning Representations (ICLR)

  35. Verma N, Boyer E, Verbee J (2018) FeaStNet: feature-steered graph convolutions for 3D shape analysis. In CVPR, 2598–2606

  36. Wang Y, Sun Y, Liu Z, Sarma S, Bronstein M, Solomon J (2018) Dynamic graph CNN for learning on point clouds. ACM Trans Graph 38(5):12

    Google Scholar 

  37. Wang X, Girshick R, Gupta A, He K (2018) Non-local neural networks. In CVPR, 7794–7803

  38. Wang C, Samari B, Siddiqi K (2018) Local spectral graph convolution for point set feature learning. In Proc. the European Conference on Computer Vision (ECCV): 52–66

  39. Wu W, Qi Z, Li F (2019) PointConv: deep convolutional networks on 3D point clouds. In CVPR

  40. Wu W, Zhang Y, Wang D, Lei Y (2020) SK-Net: deep learning on point cloud via end-to-end discovery of spatial keypoints. In Proc. Thirty-Fourth AAAI Conference on Artificial Intelligence, 6422–6429

  41. Xie J, Fang Y, Zhu F (2016) Deep shape: deep learned shape descriptor for 3D shape matching and retrieval. Comput Vis Pattern Recog

  42. Xu B, Shen H, Cao Q, Cen K, Cheng X (2019) Graph convolutional networks using heat kernel for semi-supervised learning. In Proc. 28th International Joint Conference on Artificial Intelligence. Macao, China:1928–1934

  43. Yan X, Zheng C, Li Z, Zhen S, Wang S, Cui S (2020) PointASNL: robust point clouds processing using nonlocal neural networks with adaptive sampling.In CVPR

  44. Yi L, Su H, Guo X, Guibas L (2017) Syncspeccnn: synchronize spectral cnn for 3d shape segmentation. In CVPR: 6584–6592

  45. Yu F, Koltun V (2016) Multi-scale context aggregation by dilated convolutions. In: Proc. International Conference on Learning Representations (ICLR)

  46. Zhang D, He F, Tu Z, Zou L, Chen Y (2020) Pointwise geometric and semantic learning network on 3D point clouds. Integr Comput -Aided Eng 27(1):57–75

Download references

Acknowledgements

We would like to thank the anonymous reviewers for their helpful comments. The research presented in this paper is supported by a grant from NSFC (61702246), grants from research projects of Liaoning province (2019lsktyb-084, 2020JH4/10100045, LJ2020015) and a fund of Dalian Science and Technology (2019J12GX038).

Author information

Authors and Affiliations

  1. School of Computer and Information Technology, Liaoning Normal University, Dalian, China

    Li Han, Pengyan Lan, Xue Shi, Xiaomin Wang, Jinhai He & Genyu Li

Authors
  1. Li Han
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Pengyan Lan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xue Shi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiaomin Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jinhai He
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Genyu Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Li Han.

Ethics declarations

Conflicts of interests/Competing interests

We declare that we have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Han, L., Lan, P., Shi, X. et al. Topological and geometrical joint learning for 3D graph data. Multimed Tools Appl 82, 15457–15474 (2023). https://doi.org/10.1007/s11042-022-13806-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-13806-y

Keywords

Search

Navigation