Skip to main content
SpringerLink
Log in

An Efficient Deep Learning Approach Using Improved Generative Adversarial Networks for Incomplete Information Completion of Self-driving Vehicles

  • Published:
Journal of Grid Computing Aims and scope Submit manuscript

Abstract

Autonomous driving is a key technology for intelligent logistics in the Industrial Internet of Things (IIoT). In autonomous driving, the appearance of incomplete point clouds that lose geometric and semantic information is inevitable due to the limitations of occlusion, sensor resolution, and viewing angle when Light Detection And Ranging (LiDAR) is applied. The existence of incomplete point clouds, especially incomplete vehicle point clouds, would lead to a reduction in the accuracy of object detection, traffic alerts, and collision avoidance for autonomous driving vehicles. Existing point cloud completion networks, such as the Point Fractal Network (PF-Net), focus on the accuracy of point cloud completion without considering the efficiency of the inference process, which makes it difficult for them to be deployed for vehicle point cloud repair in autonomous driving. To address this problem, in this paper, we propose an efficient deep learning approach to repair incomplete vehicle point clouds in autonomous driving accurately and efficiently. In the proposed method, an efficient downsampling algorithm that combines incremental sampling and one-time sampling is presented to improve the inference speed of the PF-Net based on Generative Adversarial Network (GAN). To evaluate the performance of the proposed method, a real dataset is used, and autonomous driving scenes are created, where three incomplete vehicle point clouds with 5 different sizes are used for three autonomous driving situations. The improved PF-Net can achieve speedups of over 19x with almost the same accuracy when compared to the original PF-Net. Experimental results demonstrate that the improved PF-Net can be applied to efficiently complete vehicle point clouds in autonomous driving.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Data Availability

The datasets generated during and/or analysed during the current study are available in the figshare repository, https://doi.org/10.6084/m9.figshare.19126883.v3.

References

  1. Khalil, R.A., Saeed, N., Masood, M., Fard, Y.M., Alouini, M.-S., Al-Naffouri, T.Y.: Deep learning in the industrial internet of things: Potentials, challenges, and emerging applications. IEEE Internet of Things Journal 8(14), 11016–11040 (2021)

    Article  Google Scholar 

  2. Barbareschi, M., Casola, V., Debenedictis, A., La Montagna, E., Mazzocca, N.: On the adoption of physically unclonable functions to secure iiot devices. IEEE Transactions on Industrial Informatics (2021)

  3. Liao, D., Li, H., Sun, G., Zhang, M., Chang, V.: Location and trajectory privacy preservation in 5g-enabled vehicle social network services. J. Netw. Comput. Appl. 110, 108–118 (2018)

    Article  Google Scholar 

  4. Maciel, D.B., Neto, E.P., Costa, K.B., Lima, M.P., Lopes, V.G., Neto, A.V., Silva, F.S.D., Sampaio, S.C.: Cloud-network slicing mano towards an efficient iot-cloud continuum. J. Grid. Comput., 19(4) (2021)

  5. Naas, M.I., Lemarchand, L., Raipin, P., Boukhobza, J.: Iot data replication and consistency management in fog computing. J. Grid. Comput., 19(3) (2021)

  6. Fang, J., Zhou, D., Yan, F., Zhao, T., Zhang, F., Ma, Y., Wang, L., Yang, R.: Augmented lidar simulator for autonomous driving. IEEE Robot Autom Lett 5(2), 1931–1938 (2020)

    Article  Google Scholar 

  7. Huang, Z., Yu, Y., Xu, J., Ni, F., Le, X.: Pf-net: Point fractal network for 3d point cloud completion. 7659–7667 (2020)

  8. Chen, C., Xiang, H., Qiu, T., Wang, C., Zhou, Y., Chang, V.: A rear-end collision prediction scheme based on deep learning in the internet of vehicles. J. Parallel Distrib. Comput. 117, 192–204 (2018)

    Article  Google Scholar 

  9. Skala, V.: Rbf interpolation with csrbf of large data sets. 108, 2433–2437 (2017)

  10. Mei, G., Tian, H.: Impact of data layouts on the efficiency of gpu-accelerated idw interpolation. SpringerPlus 5(1), 1–18 (2016)

    Article  Google Scholar 

  11. Breglia, A., Capozzoli, A., Curcio, C., Liseno, A.: Nufft-based interpolation in backprojection algorithms. IEEE Geoscience and Remote Sensing Letters (2020)

  12. Qi, C.R., Su, H., Mo, K., Guibas, L.J.: Pointnet:, Deep learning on point sets for 3d classification and segmentation. volume 2017-January (2017)

  13. Han, X., Li, Z., Huang, H., Kalogerakis, E., Yu, Y.: High-resolution shape completion using deep neural networks for global structure and local geometry inference. volume 2017-October pages 85–93 (2017)

  14. Li, C.-L., Zaheer, M., Zhang, Y., Póczos, B., Salakhutdinov, R.: Point cloud gan (2019)

  15. Wu, J., Zhang, C., Xue, T., Freeman, W.T., Tenenbaum, J.B.: Learning a probabilistic latent space of object shapes via 3d generative-adversarial modeling (2016)

  16. Li, Y., Dai, A., Guibas, L., Nießner, M.: Database-assisted object retrieval for real-time 3d reconstruction. Computer Graphics Forum 34(2), 435–446 (2015)

    Article  Google Scholar 

  17. Yuan, W., Khot, T., Held, D., Mertz, C., Hebert, M.: Pcn:, Point completion network (2018)

  18. Achlioptas, P., Diamanti, O., Mitliagkas, I., Guibas, L.: Learning representations and generative models for 3d point clouds (2018)

  19. Yang, Y., Feng, C., Shen, Y., Tian, D.: Foldingnet: Point cloud auto-encoder via deep grid deformation. 206–215 (2018)

  20. Stutz, D., Geiger, A.: Learning 3d shape completion under weak supervision. Int. J. Comput. Vis. 128(5), 1162–1181 (2020)

    Article  Google Scholar 

  21. Dai, A., Qi, C. R., Nieß, ner, M.: Shape completion using 3d-encoder-predictor cnns and shape synthesis. volume 2017-January (2017)

  22. Dai, A., Ritchie, D., Bokeloh, M., Reed, S., Sturm, J., Niebner, M.: Scancomplete:, Large-scale scene completion and semantic segmentation for 3d scans (2018)

  23. Sarmad, M., Lee, H.J., Kim, Y.M.: Rl-gan-net: A reinforcement learning agent controlled gan network for real-time point cloud shape completion. volume 2019-June pages 5891–5900 (2019)

  24. Ouyang, Z., Cui, J., Dong, X., Li, Y., Niu, J.: Saccadefork: a lightweight multi-sensor fusion-based target detector. Information Fusion 77, 172–183 (2022)

    Article  Google Scholar 

  25. Zheng, Q., Sun, J.: Effective point cloud analysis using multi-scale features. Sensors, 21(16) (2021)

  26. Wu, W., Qi, Z., Fuxin, L.: Pointconv: Deep convolutional networks on 3d point clouds. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2019-June (2019)

  27. Izadi, S., Kim, D., Hilliges, O., Molyneaux, D., Newcombe, R., Kohli, P., Shotton, J., Hodges, S., Freeman, D., Davison, A., Fitzgibbon, A.: Kinectfusion:, Real-time 3d reconstruction and interaction using a moving depth camera. 559–568 (2011)

  28. Morales, A., Piella, G., Sukno, F.M.: Survey on 3d face reconstruction from uncalibrated images. Computer Science Review, 40 (2021)

  29. Shi, S., Wang, X., Li, H.: Pointrcnn: 3d object proposal generation and detection from point cloud. volume 2019-June pages 770–779 (2019)

  30. Yang, G., Huang, X., Hao, Z., Liu, M.-Y., Belongie, S., Hariharan, B.: Pointflow:, 3d point cloud generation with continuous normalizing flows. volume 2019-October (2019)

  31. Kim, K., Kim, C., Jang, C., Sunwoo, M., Jo, K.: Deep learning-based dynamic object classification using lidar point cloud augmented by layer-based accumulation for intelligent vehicles. Expert Systems with Applications, 167 (2021)

  32. Han, X.-F., Yan, X.-Y., Sun, S.-J.: Novel methods for noisy 3d point cloud based object recognition. Multimed. Tools Appl. 80(17), 26121–26143 (2021)

    Article  Google Scholar 

  33. Qi, C.R., Yi, L., Su, H., Guibas, L.J.: Pointnet++:, Deep hierarchical feature learning on point sets in a metric space. volume 2017-December, 5100–5109 (2017)

  34. Mikšová, D., Rieser, C., Filzmoser, P., Middleton, M., Sutinen, R.: Identification of mineralization in geochemistry for grid sampling using generalized additive models Mathematical Geosciences (2021)

  35. Xu, Z., Deng, D., Shimada, K.: Autonomous uav exploration of dynamic environments via incremental sampling and probabilistic roadmap. IEEE Robot. Autom. Lett. 6(2), 2729–2736 (2021)

    Article  Google Scholar 

  36. Zhang, X., Zong, L., You, Q., Yong, X.: Sampling for nyström extension-based spectral clustering: Incremental perspective and novel analysis. ACM Transactions on Knowledge Discovery from Data, 11(1) (2016)

  37. Sun, G., Yu, M., Liao, D., Chang, V.: Analytical exploration of energy savings for parked vehicles to enhance vanet connectivity. IEEE Trans. Intell. Transp. Syst. 20(5), 1749–1761 (2019)

    Article  Google Scholar 

  38. Birek, L., Grzywaczewski, A., Iqbal, R., Doctor, F., Chang, V.: A novel big data analytics and intelligent technique to predict driver’s intent. Comput. Ind. 99, 226–240 (2018)

    Article  Google Scholar 

  39. Sui, P., Yang, X.: A privacy-preserving compression storage method for large trajectory data in road network. Journal of Grid Computing 16(2), 229–245 (2018)

    Article  Google Scholar 

  40. Lin, J.-W., Arul, J.M., Kao, J.-T.: A bottom-up tree based storage approach for efficient iot data analytics in cloud systems. J. Grid Comput., 19(1) (2021)

  41. Goodfellow, I.J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial nets. 3, 2672–2680 (2014)

  42. Malyshkin, V.E.: Parallel computing technologies 2018: Automatic parallel implementation of applications. J. Supercomput 75(12), 7747–7749 (2019)

    Article  Google Scholar 

  43. Huo, Z., Mei, G., Casolla, G., Giampaolo, F.: Designing an efficient parallel spectral clustering algorithm on multi-core processors in julia. J. Parallel Distrib. Comput. 138, 211–221 (2020)

    Article  Google Scholar 

  44. Cai, Y., Cui, X., Li, G., Liu, W.: A parallel finite element procedure for contact-impact problems using edge-based smooth triangular element and gpu. Comput. Phys. Commun. 225, 47–58 (2018)

    Article  Google Scholar 

  45. Lu, W.: Improved k-means clustering algorithm for big data mining under hadoop parallel framework. J. Grid. Comput. 18(2), 239–250 (2020)

    Article  Google Scholar 

  46. Yi, L., Kim, V.G., Ceylan, D., Shen, I.-C., Yan, M., Su, H., Lu, C., Huang, Q., Sheffer, A., Guibas, L.: A scalable active framework for region annotation in 3d shape collections. ACM Transactions on Graphics, 35(6) (2016)

  47. Stutz, D., Geiger, A.: Learning 3d shape completion under weak supervision. Int. J. Comput. Vis. 128(5), 1162–1181 (2020)

    Article  Google Scholar 

  48. Geiger, A., Lenz, P., Urtasun, R.: Are we ready for autonomous driving? the kitti vision benchmark suite. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition, pp 3354–3361 (2012)

  49. Geiger, A., Lenz, P., Stiller, C., Urtasun, R.: Vision meets robotics: the kitti dataset. Int. J. Robot. Res. 32(11), 1231–1237 (2013)

    Article  Google Scholar 

  50. Klokov, R., Boyer, E., Verbeek, J.: Discrete point flow networks for efficient point cloud generation. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 12368 LNCS: 694–710 (2020)

  51. Pacal, I., Karaboga, D.: A robust real-time deep learning based automatic polyp detection system. Comput. Biol. Med., 134 (2021)

  52. Smistad, E.: Fast: A framework for high-performance medical image computing and visualization (2021)

  53. Fang, J., Liu, Q., Li, J.: A deployment scheme of yolov5 with inference optimizations based on the triton inference server 441–445 (2021)

Download references

Acknowledgements

This research was jointly supported by the National Natural Science Foundation of China (Grant No. 11602235), and the Fundamental Research Funds for China Central Universities (2652018091).

Author information

Authors and Affiliations

  1. School of Engineering and Technolgy, China University of Geosciences, Beijing, China

    Jingzhi Tu & Gang Mei

  2. Department of Mathematics and Applications “R. Caccioppoli”, University of Naples Federico II, Campania, Italy

    Francesco Piccialli

Authors
  1. Jingzhi Tu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gang Mei
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Francesco Piccialli
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gang Mei.

Ethics declarations

Conflict of Interests

The authors declare that they have no conflict of interest.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Please note that a preprint version of this paper has been posted on arXiv at: 2109.02629.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tu, J., Mei, G. & Piccialli, F. An Efficient Deep Learning Approach Using Improved Generative Adversarial Networks for Incomplete Information Completion of Self-driving Vehicles. J Grid Computing 20, 21 (2022). https://doi.org/10.1007/s10723-022-09610-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10723-022-09610-5

Keywords

Search

Navigation