Skip to main content
SpringerLink
Log in

Semantic segmentation of surface from lidar point cloud

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Mapping the environment for robot navigation is an important and challenging task in SLAM (Simultaneous Localization And Mapping). Lidar sensor can produce near accurate 3D map of the environment in real time in form of point clouds. Though the point cloud data is adequate for building the map of the environment, processing millions of points in a point cloud is found to be computationally expensive. In this paper, we propose a fast algorithm that can be used to extract semantically labelled surface segments from the cloud in real time for direct navigational use or for higher level contextual scene reconstruction. First, a single scan from a spinning Lidar is used to generate a mesh of sampled cloud points. The generated mesh is further used for surface normal computation of a set of points on the basis of which surface segments are estimated. A novel descriptor is proposed to represent the surface segments. This descriptor is used to determine the surface class (semantic label) of the segments with the help of a classifier. These semantic surface segments can be further utilized for geometric reconstruction of objects in the scene or for optimized trajectory planning of a robot. The proposed method is compared with a number of point cloud segmentation methods and state of the art semantic segmentation methods to demonstrate its efficacy in terms of speed and accuracy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Altman NS (1992) An introduction to kernel and nearest-neighbor nonparametric regression. Amer Stat 46(3):175–185

    MathSciNet  Google Scholar 

  2. Bassier M, Bonduel M, Van Genechten B, Vergauwen M (2017) Segmentation of large unstructured point clouds using octree-based region growing and conditional random fields. Int Arch Photogr Remote Sens Spatial Inf Sci 42(2W8):25–30

    Article  Google Scholar 

  3. Ben-Shabat Y, Avraham T, Lindenbaum M, Fischer A (2018) Graph based over-segmentation methods for 3d point clouds. Comput Vis Image Underst 174:12–23

    Article  Google Scholar 

  4. Bhanu B, Lee S, Ho C-C, Henderson T (1986) Range data processing: Representation of surfaces by edges. In: Proceedings of the eighth international conference on pattern recognition. IEEE Computer Society Press, pp 236–238

  5. Feng C, Taguchi Y, Kamat VR (2014) Fast plane extraction in organized point clouds using agglomerative hierarchical clustering. In: 2014 IEEE International Conference on Robotics and Automation (ICRA). IEEE, pp 6218–6225

  6. Fu K, Fan D-P, Ji G-P, Zhao Q (2020) Jl-dcf: Joint learning and densely-cooperative fusion framework for rgb-d salient object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 3052–3062

  7. Geurts P, Ernst D, Wehenkel L (2006) Extremely randomized trees. Mach Learn 63(1):3–42

    Article  Google Scholar 

  8. Golovinskiy A, Funkhouser T (2009) Min-cut based segmentation of point clouds. In: 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops. IEEE, pp 39–46

  9. Gschwandtner M, Kwitt R, Uhl A, Pree W (2011) Blensor: Blender sensor simulation toolbox. In International Symposium on Visual Computing. Springer, pp 199–208

  10. Hackel T, Wegner JD, Schindler K (2016) Fast semantic segmentation of 3d point clouds with strongly varying density. ISPRS Ann Photogr Remote Sens Spatial Inf Sci 3(3):177–184

    Article  Google Scholar 

  11. Himmelsbach M, Hundelshausen FV, Wuensche H-J (2010) Fast segmentation of 3d point clouds for ground vehicles. In: 2010 IEEE Intelligent Vehicles Symposium. IEEE, pp 560–565

  12. Ho TK (1995) Random decision forests. In: Proceedings of 3rd international conference on document analysis and recognition, vol 1. IEEE, pp 278–282

  13. Hu Q, Yang B, Xie L, Rosa S, Guo Y, Wang Z, Trigoni N, Markham A (2020) Randla-net: Efficient semantic segmentation of large-scale point clouds. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 11108–11117

  14. Ioannou Y, Taati B, Harrap R (2012) M. Greenspan. Difference of normals as a multi-scale operator in unorganized point clouds. In: 2012 Second International Conference on 3D Imaging, Modeling, Processing, Visualization & Transmission. IEEE, pp 501–508

  15. Jiang XY, Meier U, Bunke H (1996) Fast range image segmentation using high-level segmentation primitives. In: Proceedings Third IEEE Workshop on Applications of Computer Vision WACV’96. IEEE, pp 83–88

  16. Jiang M, Wu Y, Zhao T, Zhao Z, Lu C (2018) Pointsift: A sift-like network module for 3d point cloud semantic segmentation. arXiv:1807.00652

  17. Landrieu L, Simonovsky M (2018) Large-scale point cloud semantic segmentation with superpoint graphs. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 4558–4567

  18. Lee Y, Lin Y, Wahba G (2004) Multicategory support vector machines: Theory and application to the classification of microarray data and satellite radiance data. J Am Stat Assoc 99(465):67–81

    Article  MathSciNet  Google Scholar 

  19. Li M, Yin D (2017) A fast segmentation method of sparse point clouds. In 2017 29th Chinese Control And Decision Conference (CCDC). IEEE, pp 3561–3565

  20. Li Y, Bu R, Sun M, Wu W, Di X, Chen B (2018) Pointcnn: Convolution on x-transformed points. In: Advances in neural information processing systems, pp 820–830

  21. Li G, Muller M, Thabet A, Ghanem B (2019) Deepgcns: Can gcns go as deep as cnns?. In: Proceedings of the IEEE International Conference on Computer Vision, pp 9267–9276

  22. Liu Z, Tang H, Lin Y, Han S (2019) Point-voxel cnn for efficient 3d deep learning. In: Advances in Neural Information Processing Systems, pp 963–973

  23. Moosmann F, Pink O, Stiller C (2009) Segmentation of 3d lidar data in non-flat urban environments using a local convexity criterion. In: 2009 IEEE Intelligent Vehicles Symposium. IEEE, pp 215–220

  24. Mukherjee A, Das SD, Ghosh J, Chowdhury AS, Saha SK (2019) Fast geometric surface based segmentation of point cloud from lidar data. In International Conference on Pattern Recognition and Machine Intelligence. Springer, pp 415–423

  25. Nguyen A, Le B (2013) 3d point cloud segmentation: A survey. In 2013 6th IEEE conference on robotics, automation and mechatronics (RAM). IEEE, pp 225–230

  26. Qi CR, Su H, Mo K, Guibas LJ (2017a) Pointnet: Deep learning on point sets for 3d classification and segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 652–660

  27. Qi CR, Yi L, Su H, Guibas LJ (2017b) Pointnet++ Deep hierarchical feature learning on point sets in a metric space. In: Advances in neural information processing systems, pp 5099–5108

  28. Quinlan JR (1986) Induction of decision trees. Mach Learn 1 (1):81–106

    Google Scholar 

  29. Rusu RB, Cousins S (2011) 3d is here: Point cloud library (pcl). In 2011 IEEE international conference on robotics and automation. IEEE, pp 1–4

  30. Rusu RB, Holzbach A, Blodow N, Beetz M (2009) Fast geometric point labeling using conditional random fields. In: 2009 IEEE/RSJ International Conference On Intelligent Robots and Systems. IEEE, pp 7–12

  31. Srivastava S, Lall B (2019) Deeppoint3d: Learning discriminative local descriptors using deep metric learning on 3d point clouds. Pattern Recogn Lett 127:27–36

    Article  Google Scholar 

  32. Tarsha-Kurdi F, Landes T, Grussenmeyer P (2007) Hough-transform and extended ransac algorithms for automatic detection of 3d building roof planes from lidar data. In: ISPRS Workshop on Laser Scanning 2007 and SilviLaser 2007, vol 36, pp 407–412

  33. Vo A-V, Truong-Hong L, Laefer DF, Bertolotto M (2015) Octree-based region growing for point cloud segmentation. ISPRS J Photogramm Remote Sens 104:88–100

    Article  Google Scholar 

  34. Wang Y, Sun Y, Liu Z, Sarma SE, Bronstein MM, Solomon JM (2019) Dynamic graph cnn for learning on point clouds. Acm Trans Graph (tog) 38(5):1–12

    Article  Google Scholar 

  35. Wicaksono SB, Wibisono A, Jatmiko W, Gamal A, Wisesa HA (2019) Semantic segmentation on lidar point cloud in urban area using deep learning. In: 2019 International Workshop on Big Data and Information Security (IWBIS). IEEE, pp 63–66

  36. Zermas D, Izzat I, Papanikolopoulos N (2017) Fast segmentation of 3d point clouds: A paradigm on lidar data for autonomous vehicle applications. In: 2017 IEEE International Conference on Robotics and Automation (ICRA). IEEE, pp 5067–5073

  37. Zhan Q, Liang Y, Xiao Y (2009) Color-based segmentation of point clouds. Laser Scann 38(3):155–161

    Google Scholar 

  38. Zhang J, Fan D-P, Dai Y, Anwar S, Saleh FS, Zhang T, Barnes N (2020a) Uc-net: uncertainty inspired rgb-d saliency detection via conditional variational autoencoders. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 8582–8591

  39. Zhang Y, Zhou Z, David P, Yue X, Xi Z, Foroosh H (2020b) Polarnet: An improved grid representation for online lidar point clouds semantic segmentation. arXiv:2003.14032

  40. Zhao N, Chua T-S, Lee GH (2020) Few-shot 3d point cloud semantic segmentation. arXiv:2006.12052

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, Engineering, Jadavpur University, Kolkata, India

    Aritra Mukherjee & Sanjoy Kumar Saha

  2. Department of Electronics, Telecommunication Engineering, Jadavpur University, Kolkata, India

    Sourya Dipta Das, Jasorsi Ghosh & Ananda S. Chowdhury

Authors
  1. Aritra Mukherjee
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sourya Dipta Das
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jasorsi Ghosh
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ananda S. Chowdhury
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Sanjoy Kumar Saha
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sanjoy Kumar Saha.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mukherjee, A., Das, S.D., Ghosh, J. et al. Semantic segmentation of surface from lidar point cloud. Multimed Tools Appl 80, 35171–35191 (2021). https://doi.org/10.1007/s11042-020-09841-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-09841-2

Keywords

Search

Navigation