Skip to main content
SpringerLink
Log in

Deep Constraints Space of Medium Modality for RGB-Infrared Person Re-identification

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

Reducing the gap between modalities is key to RGB-Infrared cross-modality person re-identification. In this paper, we propose an architecture based on the Deep Constrains Space of Medium Modality (DCSMM) for RGB-Infrared person re-identification. Specifically, a Medium Modality Network (MMN) is proposed to extract fused features of RGB and grayscale images, and we combine the fused features with infrared features for constraint. In addition, we also propose a loss function termed Domain Alignment and ID Consistency Loss (DAIC), which constrains the differences between the medium modality and the infrared modality as well as within single-modality in terms of instance level. Finally, in the high-level semantic stage, we also propose a Spatial Barycenter Margin Loss (SBM) based on each identity barycenter to constrain the feature space with different identities. The proposed method is validated on two large-scale datasets SYSU-MM01 and RegDB for cross-modality person re-identification, the results show that it achieves superior performance compared with the state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Huang ZY, Qin WC, Luo F, Guan TH, Xie F, Han S, Sun DM (2021) Combination of validity aggregation and multi-scale feature for person re-identification. J Ambient Intell Humaniz Comput. https://doi.org/10.1007/s12652-021-03473-6

    Article  Google Scholar 

  2. Cheng D, Gong Y, Zhou S, Wang J, Zheng N (2016) Person re-identification by multi-channel parts-based CNN with improved triplet loss function. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1335–1344

  3. Yang J, Shen X, Tian X, Li H, Huang J, Hua X.-S (2018) Local convolutional neural networks for person re-identification. In: Proceedings of the 26th ACM international conference on multimedia, pp 1074–1082

  4. Guo J, Yuan Y, Huang L, Zhang C, Yao J. -G, Han K (2019) Beyond human parts: dual part-aligned representations for person re-identification. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 3642–3651

  5. Zhong Z, Zheng L, Luo Z, Li S, Yang Y (2020) Learning to Adapt Invariance in Memory for Person Re-identification. IEEE Trans Pattern Anal Mach Intell 43(8):2723–2738

    Google Scholar 

  6. Wei X, Li D, Hong X, Ke W, Gong Y (2020) Co-attentive lifting for infrared-visible person re-identification. In: Proceedings of the 28th ACM international conference on multimedia, pp 1028–1037

  7. Gao Y, Liang T, Jin Y et al (2021) MSO: multi-feature space joint optimization network for RGB-infrared person re-identification. In: Proceedings of the 29th ACM international conference on multimedia, pp 5257–5265

  8. Wu A, Zheng W. -S, Yu H. -X, Gong S, Lai J (2017) RGB-infrared cross-modality person re-identification. In: Proceedings of the IEEE international conference on computer vision, pp 5380–5389

  9. Hao Y, Wang N, Li J et al (2019) HSME: Hypersphere manifold embedding for visible thermal person re-identification. In: Proceedings of the AAAI conference on artificial intelligence, pp 8385–8392

  10. Ye M, Lan X, Li J, Yuen P (2018) Hierarchical discriminative learning for visible thermal person re-identification. In: Proceedings of the AAAI conference on artificial intelligence, pp 7501–7508

  11. Ye M, Lan X, Wang Z, Yuen PC (2020) Bi-Directional Center-Constrained Top-Ranking for Visible Thermal Person Re-Identification. IEEE Trans Inf Forensics Secur 15:407–419

    Article  Google Scholar 

  12. Wang Z, Wang Z, Zheng Y, Chuang YY, Satoh SI (2019) Learning to reduce dual-level discrepancy for infrared-visible person re-identification. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 618–626

  13. Zhu Y, Yang Z, Wang L, Zhao S, Hu X, Tao D (2020) Hetero-Center loss for cross-modality person re-identification. Neurocomputing 386:97–109

    Article  Google Scholar 

  14. Ye M, Shen J, J Crandall D, Shao L, Luo J (2020) Dynamic dual-attentive aggregation learning for visible-infrared person re-identification. In: European conference on computer vision, pp 229–247

  15. Wang GA, Zhang T, Yang Y, Cheng J, Chang J, Liang X (2020) Cross-modality paired-images generation for RGB-infrared person re-identification. In: Proceedings of the AAAI conference on artificial intelligence, pp 12144–12151

  16. Feng Z, Lai J, Xie X (2019) Learning modality-specific representations for visible-infrared person re-identification. IEEE Trans Image Process 29:579–590

    Article  MathSciNet  MATH  Google Scholar 

  17. Alemu L T, Pelillo M, Shah M (2019) Deep constrained dominant sets for person re-identification. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 9854–9863

  18. Li W, Zhao R, Xiao T, Wang X (2014) DeepReID: Deep Filter Pairing Neural Network for Person Re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 152–159

  19. Xiao T, Li H, Ouyang W, Wang X (2016) Learning deep feature representations with domain guided dropout for person re-identification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1249–1258

  20. Liu J, Ni B, Yan Y, Zhou P, Cheng S, Hu J (2018) Pose transferrable person re-identification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4099–4108

  21. Wang G, Yuan Y, Li J, Ge S, Zhou X (2020) Receptive multi-granularity representation for person re-identification. IEEE Trans Image Process 29:6096–6109

    Article  MATH  Google Scholar 

  22. Miao J, Wu Y, Liu P, Ding Y, Yang Y (2019) Pose-guided feature alignment for occluded person re-identification. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 542–551

  23. Gao S, Wang J, Lu H, Liu Z (2020) Pose-guided visible part matching for occluded person ReID. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 11741–11749

  24. Sohn K (2016) Improved deep metric learning with multi-class N-pair loss objective. In: Proceedings of the 30th international conference on neural information processing systems (NIPS'16), pp 1857–1865

  25. Varior RR, Haloi M, Wang G (2016) Gated siamese convolutional neural network architecture for human re-identification. In: European conference on computer vision, pp 791–808

  26. Hermans A, Beyer L, Leibe B (2017) In defense of the triplet loss for person re-identification. Preprint arXiv:1703.07737

  27. Chen W, Chen X, Zhang J, Huang K (2017) Beyond triplet loss: a deep quadruplet network for person re-identification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1320–1329

  28. Xiao Q, Luo H, Zhang C (2017) Margin sample mining loss: a deep learning based method for person re-identification. Preprint arXiv:1710.00478

  29. Ling Y, Zhong Z, Luo Z, Rota P, Li S, Sebe N (2020) Class-aware modality mix and center-guided metric learning for visible-thermal person re-identification. In: Proceedings of the 28th ACM international conference on multimedia, pp 889–897

  30. Li D, Wei X, Hong X, Gong Y (2020) Infrared-visible cross-modal person re-identification with an x modality. In: Proceedings of the AAAI conference on artificial intelligence, pp 4610–4617

  31. Kansal K, Subramanyam AV, Wang Z, Satoh SI (2020) SDL: Spectrum-disentangled representation learning for visible-infrared person re-identification. IEEE Trans Circuits Syst Video Technol 30(10):3422–3432

    Article  Google Scholar 

  32. Zhang Z, Jiang S, Huang C, Li Y, Da Xu RY (2020) RGB-IR cross-modality person ReID based on teacher-student GAN model. Pattern Recogn Lett 150:155–161

    Article  Google Scholar 

  33. Choi S, Lee S, Kim Y, Kim T, Kim C (2020) Hi-CMD: hierarchical cross-modality disentanglement for visible-infrared person re-identification. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 10254–10263

  34. Ye M, Lan X, Leng Q, Shen J (2020) Cross-modality person re-identification via modality aware collaborative ensemble learning. IEEE Trans Image Process 29:9387–9399

    Article  MATH  Google Scholar 

  35. Zhao YB, Lin JW, Xuan Q, Xi X (2019) HPILN: a feature learning framework for cross-modality person re-identification. IET Image Process 13(14):2897–2904

    Article  Google Scholar 

  36. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  37. Nguyen DT, Hong HG, Kim KW, Park KR (2017) Person recognition system based on a combination of body images from visible light and thermal cameras. Sensors 17(3):605

    Article  Google Scholar 

  38. Liu H, Cheng J, Wang W, Su Y, Bai H (2020) Enhancing the discriminative feature learning for visible-thermal cross-modality person re-identification. Neurocomputing 398:11–19

    Article  Google Scholar 

  39. Wang G, Zhang T, Cheng J, Liu S, Yang Y, Hou Z (2019) RGB-infrared cross-modality person re-identification via joint pixel and feature alignment. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 3622–3631

  40. Ye M, Shen J, Shao L (2020) Visible-infrared person re-identification via homogeneous augmented tri-modal learning. IEEE Trans Inf Forensics Security 16:728–739

    Article  Google Scholar 

  41. Park H, Lee S, Lee J, Ham B (2021) Learning by aligning: visible-infrared person re-identification using cross-modal correspondences. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 12046–12055.

  42. Fu C, Hu Y, Wu X, Mei T, He R (2021) CM-NAS: rethinking cross-modality neural architectures for visible-infrared person re-identification. Preprint arXiv:2101.08467

  43. Zhang H, Cisse M, Dauphin YN, Lopez-Paz D (2017) mixup: beyond empirical risk minimization. Preprint arXiv:1710.09412

  44. Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z (2016) Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2818–2826

Download references

Author information

Authors and Affiliations

  1. School of Microelectronics and Communication Engineering, Chongqing University, Chongqing, 400044, People’s Republic of China

    Baojin Huang, Hao Chen & Wencheng Qin

Authors
  1. Baojin Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hao Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wencheng Qin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wencheng Qin.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix

Appendix

Table

Table 4 Mentioned notations in this paper
Full size table

4 shows mentioned notations in this paper.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Huang, B., Chen, H. & Qin, W. Deep Constraints Space of Medium Modality for RGB-Infrared Person Re-identification. Neural Process Lett 55, 3007–3024 (2023). https://doi.org/10.1007/s11063-022-10995-3

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-022-10995-3

Keywords

Search

Navigation