Abstract
In recent years, 3D object detection has become an important component in the fields of autonomous driving and mobile robots. Current vision-based methods may fail to achieving reliable positioning of a 3D object due to the sensing limitations of a camera sensor. In this paper, we consider a more practical object perception pipeline using combined information from both a monocular camera and a 4D millimeter-wave radar (MMWR). Initially, the human body is detected in the 2D image plane of camera to aid the human body segmentation in 4D radar point clouds. Then the detection results from both the camera and the MMWR, including the doppler velocity of the point cloud, are used as measurement to feed an Kalman filter estimator backend to realize the rapid localization of human targets.
H. Chai and Z. Zou—Equal contribution.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. In: NIPS 2015 (2015)
Voigtlaender, P., et al.: MOTS: Multi-object tracking and segmentation. In: CVPR 2019 (2019)
He, K., Gkioxari, G., Dollar, P., Girshick, R.: Mask R-CNN. In: ICCV 2017 (2017)
Bergmann, P., Meinhardt, T., Leal-Taixe, L.: Tracking without bellsand whistles. In: ICCV 2019 (2019)
Zhou, X., Koltun, V., Krähenbühl, P.: Tracking objects as points. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) Computer Vision – ECCV 2020. LNCS, vol. 12349, pp. 474–490. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58548-8_28
Braso, G., Leal-Taixe, L.: Learning a neural solver for multiple object tracking. In: CVPR 2020 (2020)
Teichman, A., Levinson, J., Thrun, S.: Towards 3D object recognition via classification of arbitrary object tracks. In: ICRA 2011 (2011)
Qi, C.R., Su, H., Mo, K., Guibas, L.J.: Pointnet: Deep learning on point sets for 3d classification and segmentation. In: CVPR 2017 (2017)
Weng, X., Wang, J., Held, D., Kitani, K.: 3D multi-object tracking: a baseline and new evaluation metrics. In: IROS 2020 (2020)
Yin, T., Zhou, X., Krahenbuhl, P.: Center-based 3d object detection and tracking. arXiv:2006.11275 (2020)
Osep, A., Mehner, W., Mathias, M., Leibe, B.: Combined image- and world-space tracking in traffic scenes. In: ICRA 2017 (2017)
Qi, C.R., Su, H., Nießner, M., Dai, A., Yan, M., Guibas, L.: Volumetric and multi-view CNNs for object classification on 3d data. In: Proceedings of Computer Vision and Pattern Recognition (CVPR) IEEE (2016)
Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: Unified, real-time object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 779–788 (2016)
Zheng, Z., Wang, P., Liu, W., Li, J., Ye, R., Ren, D.: Distance-IOU loss: faster and better learning for bounding box regression. Proc. AAAI Conf. Artif. Intell. 34, 12993–13000 (2020)
Li, X., et al.: Generalized focal loss: learning qualified and distributed bounding boxes for dense object detection. Adv. Neural. Inf. Process. Syst. 33, 21002–21012 (2020)
Arthur, D., Vassilvitskii, S.: K-Means++: the advantages of careful seeding. In: Proceedings of the Eighteenth Annual ACM-SIAM Symposium on Discrete Algorithms, SODA 2007, New Orleans, Louisiana, USA, 7–9 January 2007.ACM (2007)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 Beijing HIWING Scientific and Technological Information Institute
About this paper
Cite this paper
Chai, H., Zou, Z., Zhao, C., Pan, Q., Lyu, Y. (2024). 3D Human Target Tracking and Localization Based on Millimeter Wave Radar and Visual Fusion. In: Qu, Y., Gu, M., Niu, Y., Fu, W. (eds) Proceedings of 3rd 2023 International Conference on Autonomous Unmanned Systems (3rd ICAUS 2023). ICAUS 2023. Lecture Notes in Electrical Engineering, vol 1177. Springer, Singapore. https://doi.org/10.1007/978-981-97-1103-1_35
Download citation
DOI: https://doi.org/10.1007/978-981-97-1103-1_35
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-97-1102-4
Online ISBN: 978-981-97-1103-1
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)