Skip to main content
SpringerLink
Log in

Simple linear iterative clustering based low-cost pseudo-LiDAR for 3D object detection in autonomous driving

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

The paper presents a low-cost and LiDAR-free approach to efficiently detect 3D objects from stereo camera images, towards autonomous driving applications. It is first proposed to exploit the simple linear iterative clustering algorithm to segment stereo images into superpixel feature maps. The segmented superpixel maps are then used to estimate a depth map. By utilizing the depth map and stereo images, a 3D point cloud can be generated; and the 3D data is considered as pseudo-LiDAR representation as it is similar to measurements collected by a LiDAR sensor. The generated pseudo-LiDAR point cloud can ultimately be fed into any the state-of-the-art LiDAR based 3D object detection techniques to localize objects. By doing this, the proposed approach can effectively detect 3D objects by only employing low-cost stereo cameras, which can save tens of thousands of dollars on LiDAR costs from the existing LiDAR based methods. Effectiveness of the proposed algorithm was evaluated in the real-world KITTI dataset where the obtained results are about 1.33% better than those obtained by the benchmarking pseudo-LiDAR++ method (You et al. 2020).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data Availability

The datasets analysed during the current study are available from the following public domain resource: http://www.cvlibs.net/datasets/kitti/

References

  1. Achanta R, Shaji A, Smith K, Lucchi A, Fua P, Süsstrunk S (2012) SLIC Superpixels compared to state-of-the-art superpixel methods. IEEE Trans Pattern Anal Mach Intell 34(11):2274–2282. https://doi.org/10.1109/TPAMI.2012.120

    Article  Google Scholar 

  2. Arnold E, Al-Jarrah OY, Dianati M, Fallah S, Oxtoby D, Mouzakitis A (2019) A survey on 3d object detection methods for autonomous driving applications. IEEE Trans Intell Transp Syst 20(10):3782–3795. https://doi.org/10.1109/TITS.2019.2892405

    Article  Google Scholar 

  3. Ban Z, Liu J, Cao L (2018) Superpixel segmentation using gaussian mixture model. IEEE Trans Image Process 27(8):4105–4117. https://doi.org/10.1109/TIP.2018.2836306

    Article  MathSciNet  MATH  Google Scholar 

  4. Cai Y, Luan T, Gao H, Wang H, Chen L, Li Y, Sotelo MA, Li Z (2021) Yolov4-5d: an effective and efficient object detector for autonomous driving. IEEE Trans Instrum Meas 70:1–13. https://doi.org/10.1109/TIM.2021.3065438

    Article  Google Scholar 

  5. Cambra AB, Muñoz A., Murillo AC, Guerrero JJ, Gutierrez D (2014) Improving depth estimation using superpixels. In: Munoz A, Vazquez P-P (eds) Spanish computer graphics conference (CEIG), pp 1–9

  6. Chang J-R, Chen Y-S (2018) Pyramid stereo matching network. In: 2018 IEEE/CVF Conference on computer vision and pattern recognition, pp 5410–5418, DOI https://doi.org/10.1109/CVPR.2018.00567, (to appear in print)

  7. Chen X, Kundu K, Zhu Y, Berneshawi AG, Ma H, Fidler S, Urtasun R (2015) 3D object proposals for accurate object class detection. In: Cortes C, Lawrence N, Lee D, Sugiyama M, Garnett R (eds) Advances in neural information processing systems, pp 1–9

  8. Chen X, Kundu K, Zhu Y, Ma H, Fidler S, Urtasun R (2018) 3D object proposals using stereo imagery for accurate object class detection. IEEE Trans Pattern Anal Mach Intell 40(5):1259–1272. https://doi.org/10.1109/TPAMI.2017.2706685

    Article  Google Scholar 

  9. Chen X, Ma H, Wan J, Li B, Xia T (2017) Multi-view 3D object detection network for autonomous driving. In: 2017 IEEE Conference on computer vision and pattern recognition (CVPR), pp 6526–6534, DOI https://doi.org/10.1109/CVPR.2017.691, (to appear in print)

  10. Concha A, Civera J (2014) Using superpixels in monocular SLAM. In: 2014 IEEE international conference on robotics and automation (ICRA). IEEE, pp 365–372

  11. Connolly C, Fleiss T (1997) A study of efficiency and accuracy in the transformation from rgb to cielab color space. IEEE Trans Image Process 6(7):1046–1048. https://doi.org/10.1109/83.597279

    Article  Google Scholar 

  12. Fischler MA, Bolles RC (1981) Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun ACM 24(6):381–395. https://doi.org/10.1145/358669.358692

    Article  MathSciNet  Google Scholar 

  13. Fu H, Gong M, Wang C, Batmanghelich K, Tao D (2018) Deep ordinal regression network for monocular depth estimation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2002–2011

  14. Geiger A, Lenz P, Stiller C, Urtasun R (2013) Vision meets robotics: the KITTI dataset. Int J Robot Res 32(11):1231–1237. https://doi.org/10.1177/0278364913491297

    Article  Google Scholar 

  15. Geiger A, Lenz P, Urtasun R (2012) Are we ready for autonomous driving? the KITTI vision benchmark suite. In: 2012 IEEE Conference on computer vision and pattern recognition, pp 3354–3361, DOI https://doi.org/10.1109/CVPR.2012.6248074, (to appear in print)

  16. Hu H, Zhao T, Wang Q, Gao F, He L, Gao Z (2021) Monocular 3-d vehicle detection using a cascade network for autonomous driving. IEEE Trans Instrum Meas 70:1–13. https://doi.org/10.1109/TIM.2021.3094622

    Google Scholar 

  17. Kanungo T, Mount DM, Netanyahu NS, Piatko CD, Silverman R, Wu AY (2002) An efficient k-means clustering algorithm: analysis and implementation. IEEE Trans Pattern Anal Mach Intell 24(7):881–892. https://doi.org/10.1109/TPAMI.2002.1017616

    Article  MATH  Google Scholar 

  18. Ku J, Mozifian M, Lee J, Harakeh A, Waslander SL (2018) Joint 3D proposal generation and object detection from view aggregation. In: 2018 IEEE/RSJ International conference on intelligent robots and systems (IROS), pp 1–8, DOI https://doi.org/10.1109/IROS.2018.8594049, (to appear in print)

  19. Li X, Zhou Y, Hua B (2021) Study of a multi-beam lidar perception assessment model for real-time autonomous driving. IEEE Trans Instrum Meas 70:1–15. https://doi.org/10.1109/TIM.2021.3094230

    Article  Google Scholar 

  20. Lin C, Tian D, Duan X, Zhou J, Zhao D, Cao D (2022) 3d-dfm: anchor-free multimodal 3-d object detection with dynamic fusion module for autonomous driving. IEEE Transactions on Neural Networks and Learning Systems 1–11. https://doi.org/10.1109/TNNLS.2022.3171553

  21. Lipson A (2017) Low cost small size liDAR for automotive. Google Patents. US Patent 9,831,630

  22. Ma C, Guo Y, Lei Y, An W (2019) Binary volumetric convolutional neural networks for 3-d object recognition. IEEE Trans Instrum Meas 68 (1):38–48. https://doi.org/10.1109/TIM.2018.2840598

    Article  Google Scholar 

  23. Mayer N, Ilg E, Häusser P, Fischer P, Cremers D, Dosovitskiy A, Brox T (2016) A large dataset to train convolutional networks for disparity, optical flow, and scene flow estimation. In: 2016 IEEE Conference on computer vision and pattern recognition (CVPR), pp 4040–4048, DOI https://doi.org/10.1109/CVPR.2016.438, (to appear in print)

  24. Meyer GP, Charland J, Hegde D, Laddha A, Vallespi-Gonzalez C (2019) Sensor fusion for joint 3D object detection and semantic segmentation. In: 2019 IEEE/CVF Conference on computer vision and pattern recognition workshops (CVPRW), pp 1230–1237, DOI https://doi.org/10.1109/CVPRW.2019.00162, (to appear in print)

  25. Pang S, Morris D, Radha H (2020) Clocs: camera-lidar object candidates fusion for 3d object detection. In: 2020 IEEE/RSJ International conference on intelligent robots and systems (IROS), pp 10386–10393. https://doi.org/10.1109/IROS45743.2020.9341791

  26. Pang S, Morris D, Radha H (2022) Fast-clocs: fast camera-lidar object candidates fusion for 3d object detection. In: 2022 IEEE/CVF Winter conference on applications of computer vision (WACV), pp 3747–3756, DOI https://doi.org/10.1109/WACV51458.2022.00380, (to appear in print)

  27. Qi CR, Liu W, Wu C, Su H, Guibas LJ (2018) Frustum PointNets for 3D object detection from RGB-d data. In: 2018 IEEE/CVF Conference on computer vision and pattern recognition, pp 918–927, DOI https://doi.org/10.1109/CVPR.2018.00102, (to appear in print)

  28. Qian R, Lai X, Li X (2022) 3d object detection for autonomous driving: a survey. Pattern Recognition 108796. https://doi.org/10.1016/j.patcog.2022.108796

  29. Shi S, Wang X, Li H (2019) PointRCNN: 3D object proposal generation and detection from point cloud. In: 2019 IEEE/CVF Conference on computer vision and pattern recognition (CVPR), pp 770–779, DOI https://doi.org/10.1109/CVPR.2019.00086, (to appear in print)

  30. Stilgoe J (2020) Who’s driving innovation? new technologies and the collaborative state. Palgrave Macmillan Cham. https://doi.org/10.1007/978-3-030-32320-2

  31. Wang Y, Chao W-L, Garg D, Hariharan B, Campbell M, Weinberger KQ (2019) Pseudo-liDAR from visual depth estimation: bridging the gap in 3D object detection for autonomous driving. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 8445–8453

  32. Yan Z, Sun L, Krajník T, Ruichek Y (2020) EU Long-term dataset with multiple sensors for autonomous driving. In: 2020 IEEE/RSJ International conference on intelligent robots and systems (IROS), pp 10697–10704, DOI https://doi.org/10.1109/IROS45743.2020.9341406, (to appear in print)

  33. Yang B, Luo W, Urtasun R (2018) PIXOR: Real-Time 3D object detection from point clouds. In: 2018 IEEE/CVF Conference on computer vision and pattern recognition, pp 7652–7660, DOI https://doi.org/10.1109/CVPR.2018.00798, (to appear in print)

  34. You Y, Wang Y, Chao W-L, Garg D, Pleiss G, Hariharan B, Campbell M, Weinberger KQ (2020) Pseudo-liDAR++: accurate depth for 3D object detection in autonomous driving. In: International conference on learning representations (ICLR)

  35. Zhang Z, Hong W-C (2021) Application of variational mode decomposition and chaotic grey wolf optimizer with support vector regression for forecasting electric loads. Knowl-Based Syst 228:107297. https://doi.org/10.1016/j.knosys.2021.107297

    Article  Google Scholar 

  36. Zhou Y, Tuzel O (2018) Voxelnet: end-to-end learning for point cloud based 3D object detection. In: 2018 IEEE/CVF Conference on computer vision and pattern recognition, pp 4490–4499, DOI https://doi.org/10.1109/CVPR.2018.00472, (to appear in print)

Download references

Acknowledgements

We would like to thank the College of Engineering & Computer Science, the Australian National University, for allowing us to use the GPU Cluster. Moreover, we are grateful to Dr. Nick Barnes and the other staff at the Australian National University for their useful comments and feedback.

Author information

Authors and Affiliations

  1. College of Engineering & Computer Science, The Australian National University, Canberra, 0200, ACT, Australia

    Duy Le

  2. School of Engineering, Information Technology and Physical Sciences, Federation University Australia, Churchill, 3842, VIC, Australia

    Linh Nguyen

Authors
  1. Duy Le
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Linh Nguyen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Linh Nguyen.

Ethics declarations

Conflict of Interests

The authors have no relevant financial or non-financial interests to disclose.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Le, D., Nguyen, L. Simple linear iterative clustering based low-cost pseudo-LiDAR for 3D object detection in autonomous driving. Multimed Tools Appl 82, 25253–25269 (2023). https://doi.org/10.1007/s11042-023-14439-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-14439-5

Keywords

Search

Navigation