Abstract
Perception of the world around is key for autonomous driving applications. To allow better perception in many different scenarios vehicles can rely on camera and LiDAR sensors. Both LiDAR and camera provide different information about the world. However, they provide information about the same features. In this research two feature based fusion methods are proposed to combine camera and LiDAR information to improve what we know about the world around, and increase our confidence in what we detect. The two methods work by proposing a region of interest (ROI) and inferring the properties of the object in that ROI. The output of the system contains fused sensor data alongside extra inferred properties of the objects based on the fused sensor data.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
F1 is Fusion system one and F2 is Fusion system two.
References
White, F.E.: JDL, data fusion lexicon. Technical Panel for C3, vol. 15, no. 0704, p. 15 (1991)
Zhang, F., Clarke, D., Knoll, A.: Vehicle detection based on LiDAR and camera fusion, In: 17th International IEEE Conference on Intelligent Transportation Systems (ITSC), pp. 1620–1625. IEEE, October 2014
Liang, M., Yang, B., Wang, S., Urtasun, R.: Deep continuous fusion for multi-sensor 3D object detection. In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). LNCS, vol. 11220, pp. 663–678 (2018)
Xu, D., Anguelov, D., Jain, A.: PointFusion: deep sensor fusion for 3D bounding box estimation, Technical report (2018)
Shuqing, Z.: System and method for fusing radar/camera object data and LiDAR scan points. US Patent 9,476,983, February 2016
Redmon, J., Farhadi, A.: Yolov3: an incremental improvement. arXiv (2018)
Point cloud library find minimum oriented bounding box of point cloud (C++ and PCL). http://codextechnicanum.blogspot.com/2015/04/find-minimum-oriented-bounding-box-of.html
Point cloud library module KdTree. http://docs.pointclouds.org/trunk/group__kdtree.html
Point cloud library pcl::voxelgrid \(<\)pointt\(>\) class template reference. http://docs.pointclouds.org/1.8.1/classpcl_1_1_voxel_grid.html
Fischler, M.A., Bolles, R.C.: Random sample paradigm for model consensus: a apphcatlons to image fitting with analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981)
Geiger, A., Lenz, P., Urtasun, R.: Are we ready for autonomous driving? The KITTI vision benchmark suite. In: Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3354–3361 (2012)
Geiger, A., Lenz, P., Stiller, C., Urtasun, R.: Vision meets robotics: the KITTI dataset. Int. J. Robot. Res. (IJRR) 32(11), 1231–1237 (2013)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Balemans, D., Vanneste, S., de Hoog, J., Mercelis, S., Hellinckx, P. (2020). LiDAR and Camera Sensor Fusion for 2D and 3D Object Detection. In: Barolli, L., Hellinckx, P., Natwichai, J. (eds) Advances on P2P, Parallel, Grid, Cloud and Internet Computing. 3PGCIC 2019. Lecture Notes in Networks and Systems, vol 96. Springer, Cham. https://doi.org/10.1007/978-3-030-33509-0_75
Download citation
DOI: https://doi.org/10.1007/978-3-030-33509-0_75
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-33508-3
Online ISBN: 978-3-030-33509-0
eBook Packages: EngineeringEngineering (R0)