Abstract
Most existing transformer-based Multi-object tracking (MOT) methods use Convolutional Neural Network (CNN) to extract features and then use a transformer to detect and track objects. However, feature extract networks in existing MOT methods cannot pay more attention to the salient regional features and capture their consecutive contextual information, resulting in the neglect of potential object areas during detection. And self-attention in the transformer generates extensive redundant attention areas, resulting in a weak correlation between detected and tracking objects during the tracking. In this paper, we propose a salient regional feature enhancement module (SFEM) to focus more on salient regional features and enhance the continuity of contextual features, it effectively avoids the neglect of some potential object areas due to occlusion and background interference. We further propose soft-sparse attention (SSA) in the transformer to strengthen the correlation between detected and tracking objects, it establishes an exact association between objects to reduce the object’s ID switch. Experimental results on the datasets of MOT17 and MOT20 show that our model significantly outperforms the state-of-the-art metrics of MOTA, IDF1, and IDSw.
This work was supported by the Scientific Research Project of Tianjin Educational Committee under Grant 2021KJ037.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Bergmann, P., Meinhardt, T., Leal-Taixé, L.: Tracking without bells and whistles. Int. Conf. Comput. Vis. (2019)
Zhou, X., Cui, J., Qu, M.: An improved multi-object tracking algorithm for autonomous driving based on DeepSORT. In: ICITE (2022)
Wang, X.: Intelligent multi-camera video surveillance: a review. Pattern Recognit. Lett. 34(1), 3–19 (2013)
Uchiyama, H., Marchand, E.: Object detection and pose tracking for augmented reality: recent approaches. In: Proc (2012)
Wang, Z., Zheng, L., Liu, Y., Li, Y., Wang, S.: Towards real-time multi-object tracking. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12356, pp. 107–122. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58621-8_7
Zhang, Y., Wang, C., Wang, X., Zeng, W., Liu, W.: Fairmot: on the fairness of detection and re-identification in multiple object tracking. In: IJCV, pp. 1–19 (2021)
Zeng, F., Dong, B., Zhang, Y., Wang, T., Zhang, X., Wei, Y.: MOTR: end-to-end multiple-object ttracking with transformer. In: ECCV (2022)
Sun, P., et al.: Transtrack: multiple-object tracking with transformer. arXiv preprint arXiv:2012.15460 (2020)
Meinhardt, T., Kirillov, A., Leal-Taixé, L., Feichtenhofer, C.: Trackformer: multi-object tracking with transformers. In: CVPR (2022)
Vaswani, A., et al.: Attention is all you need. In: NeurlPS (2017)
Jaderberg, M., Simonyan, K., Zisserman, A., Kavukcuoglu, K.: Spatial transformer networks. In: NIPS (2015)
Xu, Y., Ban, Y., Delorme, G., Gan, C., Rus, D., Alameda-Pineda, X.: TransCenter: transformers with dense representations for multiple-object tracking. In: IEEE Transactions on Pattern Analysis and Machine Intelligence (2023)
Fu, Z., Fu, Z., Liu, Q., Cai, W., Wang, Y.: SparseTT: visual tracking with sparse transformers. In: IJCA (2022)
Chu, P., Wang, J., You, Q., Ling, H., Liu, Z.: TransMOT: spatial-temporal graph transformer for multiple object tracking. In: WACA (2023)
Zhao, D., Zeng, Y.: Dynamic fusion of convolutional features based on spatial and temporal attention for visual tracking. In: IJCNN (2019)
Huang, D., Yang, M., Duan, J., Yu, S., Liu, Z.: Siamese network tracking based on feature enhancement. IEEE Access (2023)
Bi, F., Sun, J., Lei, M., Wang, Y., Sun X.: Remote sensing target tracking for UAV aerial videos based on multi-frequency feature enhancement. In: IGARSS (2020)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR (2016)
Zhu, X., Su, W., Lu, L., Li, B., Wang, X., Dai, J.: Deformable detr: deformable transformers for end-to-end object detection. In: ICLR (2020)
Zhao, G., Lin, J., Zhang, Z., Ren, X., Su Q., Sun X.: Explicit sparse transformer: concentrated attention through explicit selection. arXiv preprint arXiv:1912.11637 (2019)
Milan, A., Leal-Taixé, L., Reid, I., Roth, S., Schindler, K.: Mot16: a benchmark for multi-object tracking. arXiv preprint arXiv:1603.00831 (2016)
Dendorfer, P., et al.: Mot20: A benchmark for multi object tracking in crowded scenes. arXiv preprint arXiv:2003.09003 (2020)
Shao, S., et al.: Crowdhuman: a benchmark for detecting human in a crowd. arXiv:1805.00123 (2018)
Hornakova, A., Henschel, R., Rosenhahn, B., Swoboda, P.: Lifted disjoint paths with application in multiple object tracking. In: International Conference on Machine Learning (2020)
Zhou, X., Koltun, V., Krahenbuhl, P.: Tracking objects as points. In: ECCV, pp. 474–490 (2020)
Lin, T., et al.: Microsoft coco: common objects in context. In: ECCV (2014)
Keuper, M., Tang, S., Andres, B., Brox, T., Schiele, B.: Motion segmentation and multiple object tracking by correlation co-clustering. IEEE Trans. Pattern Anal. Mach. Intell. (2018)
Henschel, R., Leal-Taixé, L., Cremers, D., Rosenhahn, B.: Improvements to Frank-Wolfe optimization for multi-detector multi-object tracking. In: CVPR (2017)
Zhang, Y., et al.: Long-term tracking with deep tracklet association. IEEE Trans. Image Process. (2020)
Braso, G., Leal-Taixé, L.: Learning a neural solver for multiple object tracking. IEEE Conf. Comput. Vis. Pattern Recogn. (2020)
Chu, P., Ling, H.: Famnet: joint learning of feature, affinity and multi-dimensional assignment for online multiple object tracking. In: CVPR (2019)
Liu, Q., Chu, Q., Liu, B., Yu, N.: GSM: Graph similarity model for multi-object tracking. Int. Joint Conf. Art. Int. (2020)
Wang, Y., Weng, X., Kitani, K.: Joint detection and multi-object tracking with graph neural networks. In: ICRA (2021)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Liu, C., Qu, X., Ma, X., Li, R., Li, X., Chen, S. (2024). Salient Feature Enhanced Multi-object Tracking with Soft-Sparse Attention in Transformer. In: Liu, Q., et al. Pattern Recognition and Computer Vision. PRCV 2023. Lecture Notes in Computer Science, vol 14436. Springer, Singapore. https://doi.org/10.1007/978-981-99-8555-5_31
Download citation
DOI: https://doi.org/10.1007/978-981-99-8555-5_31
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-8554-8
Online ISBN: 978-981-99-8555-5
eBook Packages: Computer ScienceComputer Science (R0)