Skip to main content
SpringerLink
Log in

FFA-Net: fast feature aggregation network for 3D point cloud segmentation

  • ORIGINAL PAPER
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

In many practical applications like autonomous driving and robot navigation, the large-scale 3D point cloud segmentation method is required to be both fast and efficient. In this paper, a fast and efficient large-scale point cloud semantic segmentation network, namely FFA-Net, is proposed. We used U-Net architecture as the backbone architecture, and used the random sampling algorithm to down sample the 3D points faster. To maintain the speed advantage of random sampling and protect integral information of 3D point cloud, we designed a lightweight operator, called fast feature aggregation (FFA) operator, to learn local features of 3D point cloud efficiently, and we equipped this FFA operator in a dilated residual block to aggregate local features within a larger receptive field hierarchically. This operator contains only minimal of learnable parameters, which makes the segmentation network not only improved in segmentation performance, but also in computation speed. Extensive experimental results on three large-scale 3D point cloud benchmarks have verified the effectiveness of our method in both segmentation performances and speed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Lawin, F.J., Danelljan, M., Tosteberg, P., Bhat, G., Khan, F.S., Felsberg, M.: Deep projective 3D semantic segmentation. In: International Conference on Computer Analysis of Images and Patterns, pp. 95–107 (2017)

  2. Boulch, A., Le Saux, B., Audebert, N.: Unstructured point cloud semantic labeling using deep segmentation networks. In: Eurographics Workshop on 3D Object Retrieval, pp. 17–24 (2017)

  3. Milioto, A., Vizzo, I., Behley, J., Stachniss, C.: Rangenet++: fast and accurate lidar semantic segmentation. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 4213–4220 (2019)

  4. Huang, J., You, S.: Point cloud labeling using 3D convolutional neural network. In: International Conference on Pattern Recognition, pp. 2670–2675 (2016)

  5. Rethage, D., Wald, J., Sturm, J., Navab, N., Tombari, F.: Fully-convolutional point networks for large-scale point clouds. In: European Conference on Computer Vision, pp. 596–611 (2018)

  6. Meng, H.Y., Gao, L., Lai, Y.K., Manocha, D.: Vv-net: voxel vae net with group convolutions for point cloud segmentation. In: IEEE/CVF International Conference on Computer Vision, pp. 8500–8508 (2019)

  7. Qi, C.R., Su, H., Mo, K., Guibas, L.J.: Pointnet: deep learning on point sets for 3D classification and segmentation. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 652–660 (2017)

  8. Jiang, M., Wu, Y., Zhao, T., Zhao, Z., Lu, C.: Pointsift: a sift-like network module for 3D point cloud semantic segmentation (2018). arXiv:1807.00652

  9. Zhao, H., Jiang, L., Fu, C.W., Jia, J.: Pointweb: enhancing local neighborhood features for point cloud processing. In IEEE Conference on Computer Vision and Pattern Recognition, pp. 5565–5573 (2019)

  10. Qi, C.R., Yi, L., Su, H., Guibas, L.J.: Pointnet++: deep hierarchical feature learning on point sets in a metric space. In: Advances in Neural Information Processing Systems, p. 30 (2017)

  11. Li, Y., Bu, R., Sun, M., Wu, W., Di, X., Chen, B.: Pointcnn: convolution on x-transformed points. In: Advances in Neural Information Processing Systems, p. 31 (2018)

  12. Wu, W., Qi, Z., Fuxin, L.: Pointconv: deep convolutional networks on 3D point clouds. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9621–9630 (2019)

  13. Wang, Y., Sun, Y., Liu, Z., Sarma, S.E., Bronstein, M.M., Solomon, J.M.: Dynamic graph cnn for learning on point clouds. ACM Trans. Graph. 38(5), 1–12 (2019)

    Article  Google Scholar 

  14. Groh, F., Wieschollek, P., Lensch, H.: Flex-convolution: million-scale point-cloud learning beyond grid-worlds (2018). arXiv:1803.07289

  15. Dovrat, O., Lang, I., Avidan, S.: Learning to sample. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2755–2764 (2019)

  16. Lang, I., Manor, A., Avidan, S.: Samplenet: Differentiable point cloud sampling. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7578–7588 (2020)

  17. Yan, X., Zheng, C., Li, Z., Wang, S., Cui, S.: Pointasnl: robust point clouds processing using nonlocal neural networks with adaptive sampling. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5589–5598 (2020)

  18. Hu, Q., Yang, B., Xie, L., Rosa, S., Guo, Y., Wang, Z., Trigoni, N., Markham, A.: Randla-net: efficient semantic segmentation of large-scale point clouds. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.11108–11117 (2020)

  19. Liu, Z., Hu, H., Cao, Y., Zhang, Z., Tong, X.: A closer look at local aggregation operators in point cloud analysis. In: European Conference on Computer Vision, pp. 326–342 (2020)

  20. Liu, X., Qi, C.R., Guibas, L.J.: Flownet3d: learning scene flow in 3D point clouds. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 529–537 (2019)

  21. Nezhadarya, E., Taghavi, E., Razani, R., Liu, B., Luo, J.: Adaptive hierarchical down-sampling for point cloud classification. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12956–12964 (2020)

  22. Zhang, Z., Hua, B.S., Yeung, S.K.: Shellnet: Efficient point cloud convolutional neural networks using concentric shells statistics. In: IEEE/CVF International Conference on Computer Vision, pp. 1607–1616 (2019)

  23. Li, G., Muller, M., Thabet, A., Ghanem, B.: Deepgcns: can gcns go as deep as cnns. In: IEEE/CVF International Conference on Computer Vision, pp. 9267–9276 (2019)

  24. Hua, B.S., Tran, M.K., Yeung, S.K.: Pointwise convolutional neural networks. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 984–993 (2018)

  25. Mao, J., Wang, X., Li, H.: Interpolated convolutional networks for 3D point cloud understanding. In: IEEE/CVF International Conference on Computer Vision, pp. 1578–1587 (2019)

  26. Lan, S., Yu, R., Yu, G., Davis, L.S.: Modeling local geometric structure of 3D point clouds using geo-cnn. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 998–1008 (2019)

  27. Thomas, H., Qi, C.R., Deschaud, J.E., Marcotegui, B., Goulette, F., Guibas, L.J.: Kpconv: flexible and deformable convolution for point clouds. In: IEEE/CVF International Conference on Computer Vision, pp. 6411–6420 (2019)

  28. Liu, Y., Fan, B., Xiang, S., Pan, C.: Relation-shape convolutional neural network for point cloud analysis. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8895–8904 (2019)

  29. Behley, J., Garbade, M., Milioto, A., Quenzel, J., Behnke, S., Stachniss, C., Gall, J.: Semantickitti: a dataset for semantic scene understanding of lidar sequences. In: IEEE/CVF International Conference on Computer Vision, pp. 9297–9307 (2019)

  30. Hackel, T., Savinov, N., Ladicky, L., Wegner, J.D., Schindler, K., Pollefeys, M.: Semantic3d. net: a new large-scale point cloud classification benchmark (2017). arXiv:1704.03847

  31. Armeni, I., Sax, S., Zamir, A.R., Savarese, S.: Joint 2d-3d-semantic data for indoor scene understanding (2017). arXiv:1702.01105

  32. Landrieu, L., Simonovsky, M.: Large-scale point cloud semantic segmentation with superpoint graphs. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 4558–4567 (2018)

  33. Su, H., Jampani, V., Sun, D., Maji, S., Kalogerakis, E., Yang, M.H., Kautz, J.: Splatnet: sparse lattice networks for point cloud processing. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2530–2539 (2018)

  34. Tatarchenko, M., Park, J., Koltun, V., Zhou, Q.Y.: Tangent convolutions for dense prediction in 3D. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 3887–3896 (2018)

  35. Wu, B., Wan, A., Yue, X., Keutzer, K.: SqueezeSeg: convolutional neural nets with recurrent crf for real-time road-object segmentation from 3D lidar point cloud (2017). arXiv:1710.07368

  36. Wu, B., Zhou, X., Zhao, S., Yue, X., Keutzer, K.: Squeezesegv2: improved model structure and unsupervised domain adaptation for road-object segmentation from a lidar point cloud. In: International Conference on Robotics and Automation, pp. 4376–4382 (2019)

  37. Li, S., Chen, X., Liu, Y., Dai, D., Stachniss, C., Gall, J.: Multi-scale interaction for real-time lidar data segmentation on an embedded platform. IEEE Robot. Autom. Lett. 7(2), 738–745 (2021)

    Article  Google Scholar 

  38. Boulch, A., Saux, B.L., Audebert, N.: Unstructured point cloud semantic labeling using deep segmentation networks. In: Eurographics Workshop on 3D Object Retrieval, pp. 17–24 (2017)

  39. Tchapmi, L., Choy, C., Armeni, I., Gwak, J., Savarese, S.: Segcloud: semantic segmentation of 3D point clouds. In: International Conference on 3D Vision, pp. 537–547 (2017)

  40. Thomas, H., Goulette, F., Deschaud, J.E., Marcotegui, B., LeGall, Y.: Semantic classification of 3D point clouds with multiscale spherical neighborhoods. In: International Conference on 3D Vision, pp. 390–398 (2018)

  41. Roynard, X., Deschaud, J.E., Goulette, F.: Classification of point cloud scenes with multiscale voxel deep network (2018). arXiv:1804.03583

  42. Wang, L., Huang, Y., Hou, Y., Zhang, S., Shan, J.: Graph attention convolution for point cloud semantic segmentation. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10296–10305 (2019)

  43. Ye, X., Li, J., Huang, H., Du, L., Zhang, X.: 3D recurrent neural networks with context fusion for point cloud semantic segmentation. In: European Conference on Computer Vision, pp. 403–417 (2018)

  44. Chen, L.Z., Li, X.Y., Fan, D.P., Wang, K., Lu, S.P., Cheng, M.M.: LSANet: feature learning on point sets by local spatial aware layer (2019). arXiv:1905.05442

  45. Jiang, L., Zhao, H., Liu, S., Shen, X., Fu, C.W., Jia, J.: Hierarchical point-edge interaction network for point cloud semantic segmentation. In: International Conference on Computer Vision, pp. 10432–10440 (2019)

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China (Grant No. 61973029, 62273034, 62076026), and the Scientific and Technological Innovation Foundation of Foshan (BK21BF004).

Author information

Authors and Affiliations

  1. Beijing Engineering Research Center of Industrial Spectrum Imaging, School of Automation and Electrical Engineering, University of Science and Technology Beijing, Beijing, 100083, China

    Ruting Cheng, Hui Zeng, Xuan Wang & Tianmeng Zhao

  2. Shunde Innovation School, University of Science and Technology Beijing, Foshan, 528399, China

    Hui Zeng & Xuan Wang

  3. Beijing Institute of Electronic System Engineering, Beijing, 100854, China

    Baoqing Zhang

Authors
  1. Ruting Cheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hui Zeng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Baoqing Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xuan Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Tianmeng Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hui Zeng.

Ethics declarations

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cheng, R., Zeng, H., Zhang, B. et al. FFA-Net: fast feature aggregation network for 3D point cloud segmentation. Machine Vision and Applications 34, 80 (2023). https://doi.org/10.1007/s00138-023-01434-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00138-023-01434-x

Keywords

Search

Navigation