Skip to main content
SpringerLink
Log in

RobNet: real-time road-object 3D point cloud segmentation based on SqueezeNet and cyclic CRF

  • Focus
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

In order to realize real-time 3D environment perception of UAVs and autopilot in low-altitude complex road scenes, a neural network model RobNet based on SqueezeNet and cyclic CRF for real-time 3D point cloud segmentation is proposed to segment the road objects in real time. Firstly, the unordered, scattered 3D point cloud data are preprocessed into a standard data format similar to an image by a spherical mapping method. Then, at the macro-level of the model design, the SqueezeNet network with the residual connection optimization is selected as the basic network of the model, and then, the conditional random field (CRF) algorithm which is processed into the cyclic network structure is used to refine the segmentation result. Finally, the construction of the basic network, the cyclic network and the network parameter settings in the model is elaborated at the micro-level. The experimental results show that the RobNet model proposed in this paper can segment the target in the road scene better. The segmentation callback rate of the three types of vehicles, pedestrians and cyclists is increased by 28, 2 and 17%, respectively, compared with the VoxelNet network. The higher callback rate is in line with the safe movement specifications for drones and autonomous driving. At the same time, the proposed model parameters are small, 98.5% smaller than the classic network AlexNet, and are easy to deploy on a platform with limited computing resources. The RobNet model in the Robot Operating System (ROS) framework engineering deployment and implementation experimental data shows that the model meets the real-time and stability requirements of the drone and automatic driving application, engineering code can run in real time at 12 Hz, the standard deviation of each frame’s running time is around 4.5 ms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

References

  • Bhushan K, Gupta BB (2017) Security challenges in cloud computing: state-of-art. Int J Big Data Intell 4(2):81–107

    Article  Google Scholar 

  • Chen LC, Papandreou G, Kokkinos I et al (2016) DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Trans Pattern Anal Mach Intell 40(4):834–848

    Article  Google Scholar 

  • Chen L, Hu X, Xu T et al (2017a) Turn signal detection during nighttime by CNN detector and perceptual hashing tracking. IEEE Trans Intell Transp Syst 18:1–12

    Article  Google Scholar 

  • Chen X, Kundu K, Zhu Y et al (2017b) 3D object proposals using stereo imagery for accurate object class detection. IEEE Trans Pattern Anal Mach Intell 40:1

    Google Scholar 

  • Engelmann F, Kontogianni T, Hermans A, et al. (2017) Exploring spatial context for 3D semantic segmentation of point clouds. In: 2017 IEEE international conference on computer vision workshop (ICCVW). IEEE

  • Hayashi A, Shirako J, Tiotto E, Ho R, Sarkar V (2019) Performance evaluation of OpenMP’s target construct on GPUs-exploring compiler optimisations. IJHPCN 13(1):54–69

    Article  Google Scholar 

  • Hussain K, Mohd Salleh MN, Leman AM (2016) Optimization of ANFIS using mine blast algorithm for predicting strength of Malaysian small medium enterprises. In: International conference on fuzzy systems and knowledge discovery. IEEE

  • Jie L, Jian C, Lei W (2018) Design of multi-mode UAV human–computer interaction system. In: 2017 IEEE international conference on unmanned systems (ICUS)

  • Li L, Zhu H, Yang G, Qian J (2014) Referenceless measure of blocking artifacts by Tchebichef kernel analysis. IEEE Signal Process Lett 21(1):122–125

    Article  Google Scholar 

  • Li L, Wu D, Wu J, Li H, Lin W, Kot AC (2016a) Image sharpness assessment by sparse representation. IEEE Trans Multimed 18(6):1085–1097

    Article  Google Scholar 

  • Li L, Lin W, Wang X, Yang G, Bahrami K, Kot AC (2016b) No-reference image blur assessment based on discrete orthogonal moments. IEEE Trans Cybern 46(1):39–50

    Article  Google Scholar 

  • Li L, Xia W, Lin W, Fang Y, Wang S (2017) No-reference and robust image sharpness evaluation based on multiscale spatial and spectral features. IEEE Trans Multimed 19(5):1030–1040

    Article  Google Scholar 

  • Liao Z, Gao L, Zhou T et al (2019) An oil painters recognition method based on cluster multiple kernel learning algorithm. IEEE Access 7:26842–26854

    Article  Google Scholar 

  • Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: CVPR

  • Marmanis D, Datcu M, Esch T et al (2016) Deep learning earth observation classification using ImageNet pretrained networks. IEEE Geosci Remote Sens Lett 13(1):105–109

    Article  Google Scholar 

  • Qi CR, Su H, Mo K, et al. (2017) PointNet: deep learning on point sets for 3D classification and segmentation. In: IEEE conference on computer vision and pattern recognition (CVPR)

  • Pan Jeng-Shyang, Kong Lingping, Sung Tien-Wen, Tsai Pei-Wei, Snasel Waclav (2018) α-fraction first strategy for hierarchical wireless sensor networks. J Internet Technol 19(6):1717–1726

    Google Scholar 

  • Pani D, Barabino G, Citi L et al (2016) Real-time neural signals decoding onto off-the-shelf DSP processors for neuroprosthetic applications. IEEE Trans Neural Syst Rehabilit Eng 24:1

    Article  Google Scholar 

  • Rubino C, Crocco M, Bue AD (2018) 3D object localisation from multi-view image detections. IEEE Trans Pattern Anal Mach Intell 40(99):1

    Google Scholar 

  • Schlosser J, Chow CK, Kira Z (2016) Fusing lidar and images for pedestrian detection using convolutional neural networks. In: 2016 IEEE international conference on robotics and automation (ICRA). IEEE, pp 2198–2205

  • Shin MO, Oh GM, Kim SW et al (2017) Real-time and accurate segmentation of 3-D point clouds based on Gaussian process regression. IEEE Trans Intell Transp Syst 18:1–15

    Article  Google Scholar 

  • Szegedy C, Liu W, Jia Y, et al. (2015) Going deeper with convolutions. In: 2015 IEEE conference on computer vision and pattern recognition (CVPR). IEEE

  • Wang DZ, Posner I, Newman P (2012) What could move? Finding cars, pedestrians and bicyclists in 3d laser data. In: Robotics and automation (ICRA)

  • Wen H, Wu J, Pan F et al (2019) Deep-learning-based physical layer authentication for industrial wireless sensor networks. Sensors 19(11):2440

    Article  Google Scholar 

  • Wolf D, Prankl J, Vincze M (2015) Fast semantic segmentation of 3D point clouds using a dense CRF with learned parameters. In: IEEE international conference on robotics and automation

  • Wu J, Guo S, Huang H et al (2018) Information and communications technologies for sustainable development goals: state-of-the-art. Needs Perspect IEEE Commun Surv Tutor 20(3):2389–2406

    Article  Google Scholar 

  • Xiong T et al (2018) Robust student’s-t mixture modelling via Markov random field and its application in image segmentation. High Perform Comput Netw 11(4):342–350

    Google Scholar 

  • Xu L, Wan P, Wang Y, Liang T (2019) A similarity algorithm based on hamming distance used to detect malicious users in cooperative spectrum sensing. Int J High Perform Comput Netw 14(1):112–119

    Article  Google Scholar 

  • Yi L, Kunya G, Zhuo D et al (2019) Design and implementation of an Openflow SDN controller in NS-3 discrete-event network simulator. J High Perform Computing and Networking 14(1):17–29

    Article  Google Scholar 

  • Yu Y, Li J, Guan H et al (2015a) Semiautomated extraction of street light poles from mobile LiDAR point-clouds. IEEE Trans Geosci Remote Sens 53(3):1374–1386

    Article  Google Scholar 

  • Yu Y, Li J, Guan H et al (2015b) Learning hierarchical features for automated extraction of road markings from 3-D mobile LiDAR point clouds. IEEE J Sel Topics Appl Earth Obs Remote Sens 8(2):709–726

    Article  Google Scholar 

  • Zhang Y, Wu J, Cai J (2016) Compact representation of high-dimensional feature vectors for large-scale image recognition and retrieval. IEEE Trans Image Process A Publ IEEE Signal Process Soc 25(5):2407–2419

    Article  MathSciNet  Google Scholar 

  • Zheng S, Jayasumana S, Romera-Paredes B, Vineet V, Su Z, Du D, Huang C, Torr PH (2015) Conditional random fields as recurrent neural networks. In: Proceedings of the IEEE international conference on computer vision, pp 1529–1537

  • Zhou Y, Tuzel O (2018) VoxelNet: end-to-end learning for point cloud based 3D object detection. In: IEEE/CVF conference on computer vision and pattern recognition

Download references

Acknowledgements

We would like to thank the anonymous reviewers and the associate editor for their valuable comments and suggestions to improve the quality of the manuscript. This work was supported by National Nature Science Foundation of China (NSFC) under Grants 61671356, 61703403, 61601352.

Author information

Authors and Affiliations

  1. School of Aerospace Science and Technology, Xidian University, Xi’an, 710118, China

    Wei Sun, Zhenhao Zhang & Jie Huang

Authors
  1. Wei Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhenhao Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jie Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wei Sun.

Ethics declarations

Conflict of interest

The authors declared that they have no conflicts of interest to this work. We declare that we do not have any commercial or associative interest that represents a conflict of interest in connection with the work submitted.

Additional information

Communicated by B. B. Gupta.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sun, W., Zhang, Z. & Huang, J. RobNet: real-time road-object 3D point cloud segmentation based on SqueezeNet and cyclic CRF. Soft Comput 24, 5805–5818 (2020). https://doi.org/10.1007/s00500-019-04355-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-019-04355-y

Keywords

Search

Navigation