Abstract
The key to person re-identification (Re-ID) is how to extract a representative and robust depth feature of the person, which requires the model to pay attention to both global contour information and local detailed features. To extract more representative features, an effective method is to build a multi-branch deep model by duplicating the backbone structure. This method usually severely increases the computational cost, and continuous convolution and pooling operations cause the loss of detailed information. This paper proposes a lightweight multi-scale feature pyramid structure, which extracts features from network layers of different scales and aggregates them to supplement spatial detail information. Meanwhile, this paper adopts a pair of complementary attention modules, which pay attention to the discriminative areas of person features by focusing on channel aggregation and position perception, respectively. In addition, this paper proposes a multi-level orthogonal regularization method to further enhance the diversity of features. The experimental results show that the mAP of this method on the Market1501 dataset reaches 91.6%. The proposed method outperforms state-of-the-art methods and along with lower complexity.
Similar content being viewed by others
References
Zheng, L., Yang, Y., Hauptmann, A G.: Person re-identification: past, present and future[J]. (2016)
Ye, M., Shen, J., Lin, G., et al.: Deep learning for person re-identification: a survey and outlook. IEEE Trans. Pattern Anal. Mach. Intell. 44(6), 2872–2893 (2021)
Huang, H., Li, D., Zhang, Z., et al.: Adversarially occluded samples for person re-identification[C]. In: 2018 IEEE/CVF conference on computer vision and pattern recognition (CVPR). IEEE, (2018)
Hou, R., Ma, B., Chang, H., et al.: VRSTC: occlusion-free video person re-identification[J].In: 2019 IEEE/cvf conference on computer vision and pattern recognition (CVPR), (2019)
Zhao, H., Tian, M., Sun, S., et al.: Spindle net: person re-identification with human body region guided feature decomposition and fusion[C]. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR). IEEE, (2017)
Song, C., Yan, H., Ouyang, W., et al.: Mask-guided contrastive attention model for person re-identification[C]. In: 2018 IEEE/CVF conference on computer vision and pattern recognition (CVPR). IEEE, (2018)
Gou, M., Fei, X., Camps, O., et al.: Person re-identification using kernel-based metric learning methods[C]. In: computer vision–ECCV 2014. (Springer, Cham, 2014)
Rui, Z., Ouyang, W., Wang, X.: Person re-identification by salience matching[C]. In: proceedings of the 2013 IEEE international conference on computer vision. IEEE, (2013)
Guillaumin, M., Verbeek, J., Schmid, C.: Is that you? Metric learning approaches for face identification[C]. In: IEEE international conference on computer vision. IEEE, (2009)
Chen, J., Zhang, Z., Wang, Y.: Relevance metric learning for person re-identification by exploiting global similarities[C]. In: 2014 22nd international conference on pattern recognition. IEEE, (2014)
Krizhevsky, A., Sutskever, I., Hinton, G.: ImageNet classification with deep convolutional neural networks. Adv. Neural Inf. Processing Syst. 25, 2 (2012)
Szegedy, C., Liu, W., Jia, Y., et al.: Going deeper with convolutions[J]. In: IEEE computer society, (2014)
Sun, Y., Zheng, L., Yang, Y., et al.: Beyond part models: person retrieval with refined part pooling (and a strong convolutional baseline). Springer, Cham (2017)
Hao, L.: Bags of tricks and a strong baseline for deep person re-identification[C]. In: 2019 IEEE/CVF conference on computer vision and pattern recognition workshops (CVPRW). IEEE, 2019
Chi, S., Li, J., Zhang. S,, et al.: Pose-driven deep convolutional model for person re-identification[C]. In: 2017 IEEE international conference on computer vision (ICCV). IEEE, (2017)
Zhang, Z., Lan, C., Zeng, W., et al.: Relation-aware global attention for person re-identification[C]. In: 2020 IEEE/CVF conference on computer vision and pattern recognition (CVPR). IEEE, (2020)
Chen, T.et al.: ABD-Net: attentive but diverse person re-identification. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 8350–8360, https://doi.org/10.1109/ICCV.2019.00844 (2019)
Wang, G., Yuan., Y, Chen, X., et al.: Learning discriminative features with multiple granularities for person re-identification[C]. In: 2018 ACM multimedia conference. ACM, (2018)
Yang, W., Huang, H., Zhang, Z., et al.: Towards rich feature discovery with class activation maps augmentation for person re-identification[C]. In: IEEE conference on computer vision and pattern recognition 2019. IEEE, (2019)
Zheng, F., Deng, C., Sun, X., et al.: Pyramidal person re-identification via multi-loss dynamic training[C]. In: 2019 IEEE/CVF conference on computer vision and pattern recognition (CVPR). IEEE, (2019)
Wang, P., Zhao, Z., Fei, S., et al.: HOReID: deep high-order mapping enhances pose alignment for person re-identification. IEEE Trans. Image Processing 30, 2908–2922 (2021)
He, K., Zhang, X., Ren, S., et al.: Deep residual learning for image recognition. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR), pp. 770-778, https://doi.org/10.1109/CVPR.2016.90 (2016)
Cai, H., Wang, Z., Cheng, J.: Multi-scale body-part mask guided attention for person re-identification[C]. In: 2019 IEEE/CVF conference on computer vision and pattern recognition workshops (CVPRW). IEEE, (2020)
Cheng, W.A., Ls, B., Gw, B., et al.: Multi-scale multi-patch person re-identification with exclusivity regularized softmax. Neurocomputing 382, 64–70 (2020)
Liu, X., Tan, H., Tong, X., et al.: Feature preserving GAN and multi-scale feature enhancement for domain adaption person re-identification. Neurocomputing 364, 108–118 (2019)
Wu, D., Wang, C., Wu, Y., et al.: Attention deep model with multi-scale deep supervision for person re-identification. IEEE Trans. Emerg. Topics Comput. Intell. 5(1), 70–78 (2021)
Zhou, K., Yang, Y., Cavallaro, A., et al.: Omni-scale feature learning for person re-identification[C]. In: 2019 IEEE/CVF international conference on computer vision (ICCV). IEEE, 2020
Chen, Z., Lv, X., Sun, T., et al.: FLAG: feature learning with additional guidance for person search. Vis. Comput. 37(4), 685–693 (2021)
Zheng, M., Karanam, S., Wu, Z., et al.: Re-identification with consistent attentive siamese networks[C]. In: 2019 IEEE/CVF conference on computer vision and pattern recognition (CVPR). IEEE, (2019)
Liu, H., Feng, J., Qi, M., et al.: End-to-End comparative attention networks for person re-identification. IEEE Trans. Image Processing Publ. IEEE Signal Processing Soc. 26(99), 3492–3506 (2017)
Li, W., Zhu, X., Gong, S.: Harmonious attention network for person re-identification[C].In: 2018 IEEE/CVF conference on computer vision and pattern recognition. IEEE, (2018)
Xia., BN. Gong, Y., et al.: Second-order non-local Attention Networks for Person Re-identification[J]. In: 2019 IEEE/CVF international conference on computer vision (ICCV), (2019)
Chen, B., Deng, W., Hu, J.: Mixed high-order attention network for person re-identification[J]. In: 2019 IEEE/CVF international conference on computer vision (ICCV), (2019)
Zhang, L., Wu, X., Zhang, S., et al.: Branch-Cooperative OSNet for Person Re-Identification[J]. (2020)
Guo, J., Yuan, Y., Huang, L., et al.: Beyond human parts: dual part-aligned representations for person re-identification[J]. In: 2019 IEEE/CVF international conference on computer vision (ICCV), (2019)
Xie, J., Ge, Y., Zhang, J., et al.: Low-resolution assisted three-stream network for person re-identification. Vis. Comput. 38(7), 2515–2525 (2022)
Quan, R., Dong, X., Wu, Y., et al.: Auto-ReID: searching for a part-aware ConvNet for person re-identification[C]. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE, 2019.
Van Der Walt, S., Colbert, S.C., Varoquaux, G.: The numpy array: a structure for efficient numerical computation. Comput. Sci. Eng. 13(2), 22–30 (2011)
Ozay, M., Okatani, T.: Optimization on Submanifolds of Convolution Kernels in CNNs[J]. (2016)
Xuan Z, Hao L, Xing F, et al. AlignedReID: Surpassing Human-Level Performance in Person Re-Identification[J]. 2017.
Wen, Y., Zhang, K., Li, Z., et al.: A discriminative feature learning approach for deep face recognition[J]. (2016)
Zheng, L., Shen, L., Lu, T., et al.: Scalable person re-identification: a benchmark[C]. In: 2015 IEEE international conference on computer vision (ICCV). IEEE, (2015)
Ristani, E., Solera, F., Zou, R., et al. Performance measures and a data set for multi-target, multi-camera Tracking[J]. (Springer, Cham, 2016)
Wei, L., Rui, Z., Tong, X., et al.: DeepReID: deep filter pairing neural network for person re-identification[C]. In: computer vision & pattern recognition. IEEE, (2014)
Wei, L., Zhang, S., Wen, G., et al.: Person transfer GAN to bridge domain gap for person re-identification[J]. IEEE, (2018)
Cheng, W., Qian, Z., Chang, H., et al.: Mancs: a multi-task attentional network with curriculum sampling for person re-identification: 15th European Conference, Munich, Germany, September 8–14, 2018, proceedings, Part IV[C]. In: european conference on computer vision. (Springer, Cham, 2018)
Dai, Z., Chen, M., Gu, X., et al.: Batch dropblock network for person re-identification and beyond[C].In: 2019 IEEE/CVF international conference on computer vision (ICCV). IEEE, (2019)
Zhang, S., Zhang, L., Wang, W., et al.: AsNet: asymmetrical network for learning rich features in person re-identification. IEEE Signal Processing Lett. 27, 850–854 (2020)
Ni, X., Fang, L., Huttunen, H.: Adaptive L2 Regularization in Person Re-Identification[C]. In: 2020 25th international conference on pattern recognition (ICPR). (2021)
Li, H., Wu, G., Zheng, W S.: Combined depth space based architecture search for person re-identification[J]. (2021)
Li,Y., He, J., Zhang, T., et al. Diverse part discovery: occluded person re-identification with part-aware transformer[J]. (2021)
Wei, L., Zhang, S., Yao, H., et al.: GLAD: global-local-alignment descriptor for scalable person re-identification[J]. IEEE Trans. Multimed 21(4), 986–999 (2019)
Huang, H., Yang, W., Lin, J., et al.: Improve Person Re-Identification With Part Awareness Learning. IEEE Trans. Image Processing 29, 7468–7481 (2020)
Zheng, Z., Yang, X., Yu, Z., et al.: Joint discriminative and generative learning for person re-identification[C]. In: 2019 IEEE/CVF conference on computer vision and pattern recognition (CVPR). IEEE, (2020)
Acknowledgements
This work was supported by National key research and development plan project (2016YFB1200602-37).
Funding
National key research and development plan project, 2016YFB1200602-37, Minglian Wang.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare no competing interests.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Wang, P., Wang, M. & He, D. Multi-scale feature pyramid and multi-branch neural network for person re-identification. Vis Comput 39, 5185–5197 (2023). https://doi.org/10.1007/s00371-022-02653-5
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00371-022-02653-5