Skip to main content
SpringerLink
Log in

Multi-attribute smooth graph convolutional network for multispectral points classification

  • Article
  • Published:
Science China Technological Sciences Aims and scope Submit manuscript

Abstract

Multispectral points, as a new data source containing both spectrum and spatial geometry, opens the door to three-dimensional (3D) land cover classification at a finer scale. In this paper, we model the multispectral points as a graph and propose a multi-attribute smooth graph convolutional network (MaSGCN) for multispectral points classification. We construct the spatial graph, spectral graph, and geometric-spectral graph respectively to mine patterns in spectral, spatial, and geometric-spectral domains. Then, the multispectral points graph is generated by combining the spatial, spectral, and geometric-spectral graphs. Moreover, dimensionality features and spectrums are introduced to screen the appropriate connection points for constructing the spatial graph. For remote sensing scene classification tasks, it is usually desirable to make the classification map relatively smooth and avoid salt and pepper noise. A heat operator is then introduced to enhance the low-frequency filters and enforce the smoothness in the graph signal. Considering that different land covers have different scale characteristics, we use multiple scales instead of the single scale when leveraging heat operator on graph convolution. The experimental results on two real multispectral points data sets demonstrate the superiority of the proposed MaSGCN to several state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Joshi N, Baumann M, Ehammer A, et al. A review of the application of optical and radar remote sensing data fusion to land use mapping and monitoring. Remote Sens, 2016, 8: 70

    Article  Google Scholar 

  2. Chen W, Li X, He H, et al. A review of fine-scale land use and land cover classification in open-pit mining areas by remote sensing techniques. Remote Sens, 2018, 10: 15

    Article  Google Scholar 

  3. Fisher J, Acosta E A, Dennedy-Frank P J, et al. Impact of satellite imagery spatial resolution on land use classification accuracy and modeled water quality. Remote Sens Ecol Conservation, 2018, 4: 137–149

    Article  Google Scholar 

  4. Wang Q, Gu Y, Tuia D. Discriminative multiple kernel learning for hyperspectral image classification. IEEE Trans Geosci Remote Sens, 2016, 54: 3912–3927

    Article  Google Scholar 

  5. Gu Y, Wang Q, Xie B. Multiple kernel sparse representation for airborne LiDAR data classification. IEEE Trans Geosci Remote Sens, 2017, 55: 1085–1105

    Article  Google Scholar 

  6. Stal C, Briese C, De Maeyer P, et al. Classification of airborne laser scanning point clouds based on binomial logistic regression analysis. Int J Remote Sens, 2014, 35: 3219–3236

    Article  Google Scholar 

  7. Zhang Z, Zhang L, Tong X, et al. A multilevel point-cluster-based discriminative feature for ALS point cloud classification. IEEE Trans Geosci Remote Sens, 2016, 54: 3309–3321

    Article  Google Scholar 

  8. Hong D, Gao L, Yokoya N, et al. More diverse means better: Multimodal deep learning meets remote-sensing imagery classification. IEEE Trans Geosci Remote Sens, 2021, 59: 4340–4354

    Article  Google Scholar 

  9. Zhang L, Zhang L, Du B. Deep learning for remote sensing data: A technical tutorial on the state of the art. IEEE Geosci Remote Sens Mag, 2016, 4: 22–40

    Article  Google Scholar 

  10. Hong D, Yokoya N, Xia G S, et al. X-ModalNet: A semi-supervised deep cross-modal network for classification of remote sensing data. ISPRS J Photogrammetry Remote Sens, 2020, 167: 12–23

    Article  Google Scholar 

  11. Rasti B, Hong D, Hang R, et al. Feature extraction for hyperspectral imagery: The evolution from shallow to deep: Overview and toolbox. IEEE Geosci Remote Sens Mag, 2020, 8: 60–88

    Article  Google Scholar 

  12. Zhang L, Zhang L, Du B, et al. Hyperspectral image unsupervised classification by robust manifold matrix factorization. Inf Sci, 2019, 485: 154–169

    Article  MathSciNet  Google Scholar 

  13. Fernandez-Diaz J, Carter W, Glennie C, et al. Capability assessment and performance metrics for the titan multispectral mapping lidar. Remote Sens, 2016, 8: 936

    Article  Google Scholar 

  14. Bakuła K, Kupidura P, Jełowicki Ł. Testing of land cover classification from multispectral airborne laser scanning data. Int Arch Photogramm Remote Sens Spatial Inf Sci, 2016, XLI-B7: 161–169

    Article  Google Scholar 

  15. Wang C K, Tseng Y H, Chu H J. Airborne dual-wavelength LiDAR data for classifying land cover. Remote Sens, 2014, 6: 700–715

    Article  Google Scholar 

  16. Teo T A, Wu H M. Analysis of land cover classification using multi-wavelength LiDAR system. Appl Sci, 2017, 7: 663

    Article  Google Scholar 

  17. Matikainen L, Karila K, Hyyppä J, et al. Object-based analysis of multispectral airborne laser scanner data for land cover classification and map updating. ISPRS J Photogrammetry Remote Sens, 2017, 128: 298–313

    Article  Google Scholar 

  18. Leigh H W, Magruder L A. Using dual-wavelength, full-waveform airborne lidar for surface classification and vegetation characterization. J Appl Remote Sens, 2016, 10: 045001

    Article  Google Scholar 

  19. Zou X, Zhao G, Li J, et al. 3D land cover classification based on multispectral lidar point clouds. Int Arch Photogramm Remote Sens Spatial Inf Sci, 2016, XLI-B1: 741–747

    Article  Google Scholar 

  20. Yu Y, Guan H, Li D, et al. A hybrid capsule network for land cover classification using multispectral LiDAR data. IEEE Geosci Remote Sens Lett, 2020, 17: 1263–1267

    Article  Google Scholar 

  21. Wichmann V, Bremer M, Lindenberger J, et al. Evaluating the potential of multispectral airborne lidar for topographic mapping and land cover classification. ISPRS Ann Photogramm Remote Sens Spatial Inf Sci, 2015, II-3/W5: 113–119

    Article  Google Scholar 

  22. Teledyne O. Titan world’s first multispectral LiDAR. Available: http://www.teledyneoptech.com/en/products/airborne-survey/titan/

  23. Sun J, Shi S, Biwu C, et al. Combined application of 3D spectral features from multispectral LiDAR for classification. In: IEEE International Geoscience and Remote Sensing Symposium. Fort Worth, 2017. 5264–5267

  24. Ekhtari N, Glennie C, Fernandez-Diaz J C. Classification of multispectral lidar point clouds. In: IEEE International Geoscience and Remote Sensing Symposium. Fort Worth, 2017. 2756–2759

  25. Ekhtari N, Glennie C, Fernandez-Diaz J C. Classification of airborne multispectral Lidar point clouds for land cover mapping. IEEE J Sel Top Appl Earth Observations Remote Sens, 2018, 11: 2068–2078

    Article  Google Scholar 

  26. Miller C I, Thomas J J, Kim A M, et al. Application of image classification techniques to multispectral lidar point cloud data. In: Proceedings of Laser Radar Technol Appl XXI. Baltimore, 98320X. 2016

  27. Morsy S, Shaker A, El-Rabbany A. Multispectral LiDAR data for land cover classification of urban areas. Sensors, 2017, 17: 958

    Article  Google Scholar 

  28. He K, Zhang X, Ren S, et al. Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computervision and Pattern Recognition. Las Vegas, 2016. 770–778

  29. Ren S, He K, Girshick R, et al. Faster R-CNN: Towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell, 2017, 39: 1137–1149

    Article  Google Scholar 

  30. Feng Q, Zhu D, Yang J, et al. Multisource hyperspectral and LiDAR data fusion for urban land-use mapping based on a modified two-branch convolutional neural network. Int J Geo-Inf, 2019, 8: 28

    Article  Google Scholar 

  31. Micheli A. Neural network for graphs: A contextual constructive approach. IEEE Trans Neural Netw, 2009, 20: 498–511

    Article  Google Scholar 

  32. Defferrard M, Bresson X, Vandergheynst P. Convolutional neural networks on graphs with fast localized spectral filtering. In: 30th Conference on Neural Information Processing Systems. Barcelona, 2016. 3844–3852

  33. Monti F, Boscaini D, Masci J, et al. Geometric deep learning on graphs and manifolds using mixture model CNNs. In: IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, 2017. 5425–5434

  34. Yan S, Xiong Y, Lin D. Spatial temporal graph convolutional networks for skeleton-based action recognition. In: IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City, 2018. 1–28

  35. Bruna J, Zaremba W, Szlam A, et al. Spectral networks and locally connected networks on graphs. In: 2nd International Conference on Learning Representations. Banff, 2014

  36. Atwood J, Towsley D. Diffusion-convolutional neural networks. In: 30th Conference on Neural Information Processing Systems. Barcelona, 2016. 1993–2001

  37. Niepert M, Ahmed M, Kutzkov K. Learning convolutional neural networks for graphs. In: Proceedings of the 33rd International Conference on Machine Learning. New York, 2016. 2014–2023

  38. Henaff M, Bruna J, Lecun Y. Deep convolutional networks on graph-structured data. Computer Science, 2015

  39. Li Y, Zemel R, Brockschmidt M, et al. Gated graph sequence neural networks. In: 4th International Conference on Learning Representations. San Juan, 2016

  40. Kipf T, Welling M. Semi-supervised classification with graph convolutional networks. In: 5th International Conference on Learning Representations. Toulon, 2017

  41. Chung F. Spectral Graph Theory. Rhode Island: American Mathematical Society, 1997

    MATH  Google Scholar 

  42. Hammond D K, Vandergheynst P, Gribonval R. Wavelets on graphs via spectral graph theory. Appl Comput Harmonic Anal, 2011, 30: 129–150

    Article  MathSciNet  Google Scholar 

  43. Xu K, Li C, Tian Y, et al. Representation learning on graphs with jumping knowledge networks. In: ICML. Stockholm, 2018. 5449–5458

    Google Scholar 

  44. Wan S, Gong C, Zhong P, et al. Multiscale dynamic graph convolutional network for hyperspectral image classification. IEEE Trans Geosci Remote Sens, 2020, 58: 3162–3177

    Article  Google Scholar 

  45. Hong D, Gao L, Yao J, et al. Graph convolutional networks for hyperspectral image classification. IEEE Trans Geosci Remote Sens, 2021, 59: 5966–5978

    Article  Google Scholar 

  46. Mou L, Lu X, Li X, et al. Nonlocal graph convolutional networks for hyperspectral image classification. IEEE Trans Geosci Remote Sens, 2020, 58: 8246–8257

    Article  Google Scholar 

  47. Zhang Y, Rabbat M. A graph-CNN for 3D point cloud classification. In: IEEE International Conference on Acoustics, Speech and Signal Processing. Calgary, Alberta, 2018

  48. Te G, Hu W, Guo Z, et al. RGCNN: Regularized graph CNN for point cloud segmentation. arXiv preprint: 1806.02952, 2018

  49. Shuman D I, Narang S K, Frossard P, et al. The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains. IEEE Signal Process Mag, 2013, 30: 83–98

    Article  Google Scholar 

  50. Wang Q, Gu Y. A discriminative tensor representation model for feature extraction and classification of multispectral LiDAR data. IEEE Trans Geosci Remote Sens, 2020, 58: 1568–1586

    Article  MathSciNet  Google Scholar 

  51. Lin Y, Wang C, Zhai D, et al. Toward better boundary preserved supervoxel segmentation for 3D point clouds. ISPRS J Photogrammetry Remote Sens, 2018, 143: 39–47

    Article  Google Scholar 

  52. Demantke J, Mallet C, David N, et al. Dimensionality based scale selection in 3D LiDAR point clouds. Int Arch Photogramm Remote Sens Spat Inf Sci, 2011, 38: 97–102

    Google Scholar 

  53. Coifman R R, Lafon S. Diffusion maps. Appl Comput Harmonic Anal, 2006, 21: 5–30

    Article  MathSciNet  Google Scholar 

  54. Qin A, Shang Z, Tian J, et al. Spectral-spatial graph convolutional networks for semisupervised hyperspectral image classification. IEEE Geosci Remote Sens Lett, 2019, 16: 241–245

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Electronics and Information Engineering, Harbin Institute of Technology, Harbin, 150001, China

    QingWang Wang, YanFeng Gu & Chen Wang

  2. Huawei Technology Co., Ltd, Shanghai, 200120, China

    QingWang Wang

  3. North China Sea Marine Technical Support Center, Ministry of Natural Resources, Qingdao, 266000, China

    Min Yang

Authors
  1. QingWang Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. YanFeng Gu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Min Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chen Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Min Yang.

Additional information

This work was supported by the Key Research and Development Project of Ministry of Science and Technology (Grant No. 2017YFC1405100), and in part by the National Natural Science Foundation of Key International Cooperation (Grant No. 61720106002).

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, Q., Gu, Y., Yang, M. et al. Multi-attribute smooth graph convolutional network for multispectral points classification. Sci. China Technol. Sci. 64, 2509–2522 (2021). https://doi.org/10.1007/s11431-020-1871-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11431-020-1871-8

Search

Navigation