Skip to main content

Advertisement

SpringerLink
Log in

A semi-automated system for person re-identification adaptation to cross-outfit and cross-posture scenarios

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Person re-identification (ReID) algorithms are often trained on multi-camera snapshots of individuals taken on the same day, wearing the same outfits. Models trained with such protocols often fail in many long-term, indoor applications where person matching must be done across days, necessitating that algorithms be able to adapt to changing clothing and body postures. This study presents a simple, yet effective, system to overcome this challenge in realistic settings. We collected a new dataset capturing the natural variations of office worker appearances across days. To teach a ReID algorithm to adapt, we designed a semi-automated identity labeling system that requires only a small set of identification inputs from human labelers. The system utilized instance segmentation algorithms to detect people and one-shot video segmentation algorithms to track individuals across frames. Identified footages are then fed into the image repository to continually fine-tune the ReID network. These experiments demonstrate the applicability of our proposed method in helping the ReID algorithm overcome the challenges of varied clothing and postures. Our system improves the performance (measured by mAP) compared to pre-trained benchmark by 2.46% for the standard ReID condition, by 18.19% for cross-outfit re-identification, by 22.94% for cross-posture re-identification, and by 19.17% for the cross-posture and cross-outfit setting. As such, we anticipate this method may be beneficial towards the multitude of applications that utilize machine vision to automatically recognize human subjects.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. A deep neural network framework for road side analysis and lane detection. Procedia Comput Sci 165:252–258 (2019). 2nd International Conference on Recent Trends in Advanced Computing ICRTAC -DISRUP - TIV INNOVATION, 2019 November 11-12, 2019 https://doi.org/10.1016/j.procs.2020.01.081

  2. Bao L, Wu B, Liu W (2018) Cnn in mrf: Video object segmentation via inference in a cnn-based higher-order spatio-temporal mrf. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5977–5986

  3. Bolya D, Zhou C, Xiao F, Lee YJ (2019) Yolact: Real-time instance segmentation. In: Proceedings of the IEEE international conference on computer vision, pp 9157–9166

  4. Bolya D, Zhou C, Xiao F, Lee YJ (2020) Yolact++: Better real-time instance segmentation. IEEE Transactions on Pattern Analysis And Machine Intelligence, vol PP

  5. Caelles S, Maninis KK, Pont-Tuset J, Leal-Taixé L, Cremers D, Van Gool L (2017) One-shot video object segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 221–230

  6. Chen H, Du Y, Yan L, Shi J, Huang W, Tang C (2018) Optimization of the compact gamma-ray source based on inverse compton scattering design. In: 2018 IEEE Advanced accelerator concepts workshop (AAC), pp 1–5. IEEE

  7. Chen H, Qi X, Yu L, Dou Q, Qin J, Heng PA (2017) Dcan: Deep contour-aware networks for object instance segmentation from histology images. Med Image Anal 36:135–146

    Article  Google Scholar 

  8. Chen LC, Papandreou G, Kokkinos I, Murphy K, Yuille AL (2017) Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE Trans Pattern Anal Mach Intell 40(4):834–848

    Article  Google Scholar 

  9. Chen W, Chen X, Zhang J, Huang K (2017) Beyond triplet loss: a deep quadruplet network for person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 403–412

  10. Cheng DS, Cristani M, Stoppa M, Bazzani L, Murino V (2011) Custom pictorial structures for re-identification. In: Bmvc, pp 6

  11. D’haese J, Ackhurst J, Wismeijer D, De Bruyn H, Tahmaseb A (2017) Current state of the art of computer-guided implant surgery. Periodontol 2000 73(1):121–133

    Article  Google Scholar 

  12. Fan X, Jiang W, Luo H, Fei M (2019) Spherereid: Deep hypersphere manifold embedding for person re-identification. J Vis Commun Image Represent 60:51–58

    Article  Google Scholar 

  13. Fang Q, Li H, Luo X, Ding L, Luo H, Rose TM, An W (2018) Detecting non-hardhat-use by a deep learning method from far-field surveillance videos. Autom Constr 85:1–9

    Article  Google Scholar 

  14. Gamage G, Sudasingha I, Perera I, Meedeniya D (2018) Reinstating dlib correlation human trackers under occlusions in human detection based tracking. In: 2018 18Th international conference on advances in ICT for emerging regions (ICTer). IEEE, pp 92–98

  15. Girshick R (2015) Fast r-cnn. In: Proceedings of the IEEE international conference on computer vision, pp 1440–1448

  16. Girshick R, Donahue J, Darrell T, Malik J (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 580–587

  17. Gray D, Brennan S, Tao H (2007) Evaluating appearance models for recognition, reacquisition, and tracking. In: Proc. IEEE international workshop on performance evaluation for tracking and surveillance (PETS), pp 1–7. Citeseer

  18. He K, Gkioxari G, Dollár P, Girshick R (2017) Mask r-cnn. In: Proceedings of the IEEE international conference on computer vision, pp 2961–2969

  19. Henderson P, Ferrari V (2016) End-to-end training of object class detectors for mean average precision. In: Asian conference on computer vision, pp 198–213. Springer

  20. Hirzer M, Beleznai C, Roth PM, Bischof H (2011) Person re-identification by descriptive and discriminative classification. In: Scandinavian conference on image analysis, pp 91–102. Springer

  21. Islam K (2020) Person search: New paradigm of person re-identification: A survey and outlook of recent works. Image Vis Comput 101:103970. https://doi.org/10.1016/j.imavis.2020.103970

    Article  Google Scholar 

  22. Jiang M, Li Z, Chen J (2019) Person re-identification using color features and cnn features. In: 2019 IEEE 4Th international conference on image, vision and computing (ICIVC), pp 460–462. https://doi.org/10.1109/ICIVC47709.2019.8980977

  23. Kalayeh MM, Basaran E, Gökmen M, Kamasak ME, Shah M (2018) Human semantic parsing for person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1062–1071

  24. Li D, Chen X, Zhang Z, Huang K (2017) Learning deep context-aware features over body and latent parts for person re-identification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 384–393

  25. Li W, Wang X (2013) Locally aligned feature transforms across views. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3594–3601

  26. Li W, Zhao R, Xiao T, Wang X (2014) Deepreid: Deep filter pairing neural network for person re-identification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 152–159

  27. Li X, Change Loy C (2018) Video object segmentation with joint re-identification and attention-aware mask propagation. In: Proceedings of the European Conference on Computer Vision (ECCV), pp 90–105

  28. Li Y, Weng X, Kitani KM (2021) Learning shape representations for person re-identification under clothing change. In: 2021 IEEE Winter conference on applications of computer vision (WACV), pp 2431–2440. https://doi.org/10.1109/WACV48630.2021.00248

  29. Liao S, Hu Y, Zhu X, Li S (2015) Person re-identification by local maximal occurrence representation and metric learning. In: 2015 IEEE Conference on computer vision and pattern recognition (CVPR), pp 2197–2206. https://doi.org/10.1109/CVPR.2015.7298832

  30. Lin TY, Goyal P, Girshick R, He K, Dollár P (2017) Focal loss for dense object detection. In: Proceedings of the IEEE international conference on computer vision, pp 2980–2988

  31. Lin TY, Maire M, Belongie S, Hays J, Perona P, Ramanan D, Dollár P, Zitnick CL (2014) Microsoft coco: Common objects in context. In: European conference on computer vision, pp 740–755. Springer

  32. Lin Y, Zheng L, Zheng Z, Wu Y, Hu Z, Yan C, Yang Y (2019) Improving person re-identification by attribute and identity learning. Pattern Recogn 95:151–161

    Article  Google Scholar 

  33. Liu H, Feng J, Qi M, Jiang J, Yan S (2017) End-to-end comparative attention networks for person re-identification. IEEE Trans Image Process 26(7):3492–3506

    Article  MathSciNet  Google Scholar 

  34. Liu X, Liu W, Mei T, Ma H (2016) A deep learning-based approach to progressive vehicle re-identification for urban surveillance. In: European conference on computer vision, pp 869–884. Springer

  35. Liu Y, Cheng MM, Hu X, Wang K, Bai X (2017) Richer convolutional features for edge detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3000–3009

  36. Liu Y, Shi P, Peng B, Yan H, Zhou Y, Han B, Zheng Y, Lin C, Jiang J, Fan Y et al (2019) iqiyi celebrity video identification challenge. In: Proceedings of the 27th ACM International Conference on Multimedia, pp 2516–2520

  37. Luiten J, Voigtlaender P, Leibe B (2018) Premvos: Proposal-generation, refinement and merging for video object segmentation. In: Asian conference on computer vision, pp 565–580. Springer

  38. Luo H, Jiang W, Zhang X, Fan X, Qian J, Zhang C (2019) Alignedreid++: Dynamically matching local information for person re-identification. Pattern Recogn 94:53–61

    Article  Google Scholar 

  39. Maninis KK, Caelles S, Chen Y, Pont-Tuset J, Leal-Taixé L, Cremers D, Van Gool L (2018) Video object segmentation without temporal information. IEEE Trans Pattern Anal Mach Intell 41 (6):1515–1530

    Article  Google Scholar 

  40. Martini M, Paolanti M, Frontoni E (2020) Open-world person re-identification with rgbd camera in top-view configuration for retail applications. IEEE Access 8:67756–67765

    Article  Google Scholar 

  41. Oh SW, Lee JY, Xu N, Kim SJ (2019) Video object segmentation using space-time memory networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp 9226–9235

  42. Padilla R, Netto SL, da Silva EA (2020) A survey on performance metrics for object-detection algorithms. In: 2020 International conference on systems, signals and image processing (IWSSIP). IEEE, pp 237–242

  43. Perazzi F, Khoreva A, Benenson R, Schiele B, Sorkine-Hornung A (2017) Learning video object segmentation from static images. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2663–2672

  44. Perazzi F, Pont-Tuset J, McWilliams B, Van Gool L, Gross M, Sorkine-Hornung A (2016) A benchmark dataset and evaluation methodology for video object segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 724–732

  45. Pont-Tuset J, Perazzi F, Caelles S, Arbelaez P, Sorkine-Hornung A, Van Gool L (2017) The 2017 davis challenge on video object segmentation

  46. Qian X, Fu Y, Jiang Y, Xiang T, Xue X (2017) Multi-scale deep learning architectures for person re-identification. 2017 IEEE International Conference on Computer Vision (ICCV), pp 5409–5418

  47. Rahimpour A, Liu L, Taalimi A, Song Y, Qi H (2017) Person re-identification using visual attention. In: 2017 IEEE International conference on image processing (ICIP), pp 4242–4246. IEEE

  48. Ramos S, Gehrig S, Pinggera P, Franke U, Rother C (2017) Detecting unexpected obstacles for self-driving cars: Fusing deep learning and geometric modeling. In: 2017 IEEE Intelligent vehicles symposium (IV). IEEE, pp 1025–1032

  49. Redmon J, Divvala S, Girshick R, Farhadi A (2016) You only look once: Unified, real-time object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 779–788

  50. Redmon J, Farhadi A (2018) Yolov3: An incremental improvement

  51. Ren S, He K, Girshick RB, Sun J (2015) Faster r-cnn: Towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell 39:1137–1149

    Article  Google Scholar 

  52. Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M et al (2015) Imagenet large scale visual recognition challenge. Int J Comput Vis 115(3):211–252

    Article  MathSciNet  Google Scholar 

  53. Satta R, Pala F, Fumera G, Roli F (2013) Real-time appearance-based person re-identification over multiple kinecttm cameras. In: VISAPP (2), pp. 407–410

  54. Schwartz WR, Davis LS (2009) Learning discriminative appearance-based models using partial least squares. In: 2009 XXII Brazilian symposium on computer graphics and image processing. IEEE, pp 322–329

  55. Shin Yoon J, Rameau F, Kim J, Lee S, Shin S, So Kweon I (2017) Pixel-level matching for video object segmentation using convolutional neural networks. In: Proceedings of the IEEE international conference on computer vision, pp 2167– 2176

  56. Su C, Li J, Zhang S, Xing J, Gao W, Tian Q (2017) Pose-driven deep convolutional model for person re-identification. In: Proceedings of the IEEE international conference on computer vision, pp 3960–3969

  57. Su C, Zhang S, Xing J, Gao W, Tian Q (2018) Multi-type attributes driven multi-camera person re-identification. Pattern Recogn 75:77–89

    Article  Google Scholar 

  58. Sumari FO, Machaca L, Huaman J, Clua E, Guérin J. (2020) Towards practical implementations of person re-identification from full video frames. Pattern Recognit Lett 138:513–519

    Article  Google Scholar 

  59. Sun Y, Zheng L, Yang Y, Tian Q, Wang S (2018) Beyond part models: Person retrieval with refined part pooling (and a strong convolutional baseline). In: Proceedings of the European Conference on Computer Vision (ECCV), pp 480– 496

  60. Varior RR, Haloi M, Wang G (2016) Gated siamese convolutional neural network architecture for human re-identification. In: European conference on computer vision, pp 791–808. Springer

  61. Varior RR, Shuai B, Lu J, Xu D, Wang G (2016) A siamese long short-term memory architecture for human re-identification. In: European conference on computer vision, pp 135–153. Springer

  62. Vinyals O, Blundell C, Lillicrap T, Wierstra D et al (2016) Matching networks for one shot learning. In: Advances in neural information processing systems, pp 3630–3638

  63. Wan F, Wu Y, Qian X, Chen Y, Fu Y (2020) When person re-identification meets changing clothes. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp 830–831

  64. Wang P, Chen P, Yuan Y, Liu D, Huang Z, Hou X, Cottrell G (2018) Understanding convolution for semantic segmentation. In: 2018 IEEE Winter conference on applications of computer vision (WACV), pp 1451–1460. IEEE

  65. Wang Q, Zhang L, Bertinetto L, Hu W, Torr PH (2019) Fast online object tracking and segmentation: a unifying approach. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1328–1338

  66. Wang X, Zhao R (2014) Person re-identification: System design and evaluation overview. In: Person re-identification. Springer, pp 351–370

  67. Wang Y, Chen Z, Wu F, Wang G (2018) Person re-identification with cascaded pairwise convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1470–1478

  68. Wei L, Liu X, Li J, Zhang S (2018) Vp-reid: Vehicle and person re-identification system. In: Proceedings of the 2018 ACM on International Conference on Multimedia Retrieval, pp 501–504

  69. Wei L, Zhang S, Gao W, Tian Q (2018) Person transfer gan to bridge domain gap for person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 79–88

  70. Wei L, Zhang S, Yao H, Gao W, Tian Q (2017) Glad: Global-local-alignment descriptor for pedestrian retrieval. In: Proceedings of the 25th ACM international conference on Multimedia, pp 420–428

  71. Xiang J, Lin R, Hou J, Huang W (2018) Person re-identification based on feature fusion and triplet loss function. 2018 24th International Conference on Pattern Recognition (ICPR) pp 3477–3482

  72. Xu J, Wang C (2017) An improved deep feature learning method for person re-identification. In: 2017 3Rd IEEE international conference on computer and communications (ICCC), pp 1637–1640. https://doi.org/10.1109/CompComm.2017.8322817

  73. Yang Q, Wu A, Zheng WS (2019) Person re-identification by contour sketch under moderate clothing change. IEEE Transactions on Pattern Analysis and Machine Intelligence

  74. Yao R, Lin G, Xia S, Zhao J, Yong Z (2020) Video object segmentation and tracking: a survey. ACM Trans Intell Syst Technol 11:1–47. https://doi.org/10.1145/3391743

    Article  Google Scholar 

  75. Yu S, Li S, Chen D, Zhao R, Yan J, Qiao Y (2020) Cocas: a large-scale clothes changing person dataset for re-identification. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 3400–3409

  76. Zeng K, Ning M, Wang Y, Guo Y (2020) Hierarchical clustering with hard-batch triplet loss for person re-identification. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

  77. Zhang X, Luo H, Fan X, Xiang W, Sun Y, Xiao Q, Jiang W, Zhang C, Sun J (2017) Alignedreid: Surpassing human-level performance in person re-identification

  78. Zhao H, Tian M, Sun S, Shao J, Yan J, Yi S, Wang X, Tang X (2017) Spindle net: Person re-identification with human body region guided feature decomposition and fusion. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1077–1085

  79. Zheng L, Bie Z, Sun Y, Wang J, Su C, Wang S, Tian Q (2016) Mars: a video benchmark for large-scale person re-identification. In: European conference on computer vision. Springer, pp 868–884

  80. Zheng L, Huang Y, Lu H, Yang Y (2019) Pose-invariant embedding for deep person re-identification. IEEE Trans Image Process 28(9):4500–4509

    Article  MathSciNet  Google Scholar 

  81. Zheng L, Shen L, Tian L, Wang S, Wang J, Tian Q (2015) Scalable person re-identification: a benchmark. In: Proceedings of the IEEE international conference on computer vision, pp 1116–1124

  82. Zheng L, Yang Y, Hauptmann A (2016) Person re-identification: Past present and future

  83. Zheng L, Zhang H, Sun S, Chandraker M, Yang Y, Tian Q (2017) Person re-identification in the wild. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1367–1376

  84. Zheng Z, Zheng L, Yang Y (2017) Unlabeled samples generated by gan improve the person re-identification baseline in vitro. In: Proceedings of the IEEE International Conference on Computer Vision, pp 3754–3762

  85. Zhou S, Wang J, Meng D, Xin X, Li Y, Gong Y, Zheng N (2018) Deep self-paced learning for person re-identification. Pattern Recogn 76:739–751

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by the Petchra Pra Jom Klao Master’s Degree Scholarship from King Mongkut’s University of Technology Thonburi (KMUTT), Thailand (grant number 40/2561).

Author information

Authors and Affiliations

  1. Institute of Field Robotics, King Mongkut’s University of Technology Thonburi, Bangkok, Thailand

    Woravee Chanlongrat, Teeravorn Apichanapong, Pathompong Sinngam & Warasinee Chaisangmongkon

Authors
  1. Woravee Chanlongrat
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Teeravorn Apichanapong
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Pathompong Sinngam
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Warasinee Chaisangmongkon
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Warasinee Chaisangmongkon.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chanlongrat, W., Apichanapong, T., Sinngam, P. et al. A semi-automated system for person re-identification adaptation to cross-outfit and cross-posture scenarios. Appl Intell 52, 9501–9520 (2022). https://doi.org/10.1007/s10489-021-02896-0

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-021-02896-0

Keywords

Search

Navigation