Advertisement

Multimedia Tools and Applications

, Volume 78, Issue 5, pp 5863–5880 | Cite as

Deep feature embedding learning for person re-identification based on lifted structured loss

  • Zhangping He
  • Cheolkon JungEmail author
  • Qingtao Fu
  • Zhendong Zhang
Article
  • 148 Downloads

Abstract

Person re-identification (re-id) aims at matching the same individual in videos captured by multiple cameras, and much progress has been made in recent years due to large scale pedestrian data sets and deep learning-based techniques. In this paper, we propose deep feature embedding learning for person re-id based on lifted structured loss. Triplet loss is commonly used in deep neural networks for person re-id. However, the triplet loss-based framework is not able to make full use of the batch information, and thus needs to choose hard negative samples manually that is time-consuming. To address this problem, we adopt lifted structured loss for deep neural networks that makes the network learn better feature embedding by minimizing intra-class variation and maximizing inter-class variation. Extensive experiments on Market-1501, CUHK03, CUHK01 and VIPeR data sets demonstrate the superior performance of the proposed method over state-of-the-arts in terms of the cumulative match curve (CMC) metric.

Keywords

Person re-identification Deep learning Triplet loss Lifted structured loss Convolutional neural networks 

Notes

Acknowledgements

The authors are grateful to Mr. Jun Sun in Xidian University for his contributions to various experiments and collected data.

References

  1. 1.
    Barbosa IG, Cristani M, Caputo B, Rognhaugena A, Theoharis T (2017) Looking beyond appearances: Synthetic training data for deep cnns in re-identification. Comput Vis Image Underst 167:50–62CrossRefGoogle Scholar
  2. 2.
    Chen S-Z, Guo C-C, Lai J-H (2016) Deep ranking for person re-identification via joint representation learning. IEEE Trans Image Process 25(5):2353–2367MathSciNetCrossRefGoogle Scholar
  3. 3.
    Chen D, Yuan Z, Chen B, Zheng N (2016) Similarity learning with spatial constraints for person re-identification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1268–1277Google Scholar
  4. 4.
    Chen W, Chen X, Zhang J, Huang K (2017) Beyond triplet loss: A deep quadruplet network for person re-identification. In: Proc. IEEE conference on computer vision and pattern recognition (CVPR)Google Scholar
  5. 5.
    Cheng D, Gong Y, Zhou S, Wang J, Zheng N (2016) Person re-identification by multi-channel parts-based cnn with improved triplet loss function. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 1335–1344Google Scholar
  6. 6.
    Ding S, Lin L, Wang G, Chao H (2015) Deep feature learning with relative distance comparison for person re-identification. Pattern Recogn 48(10):2993–3003CrossRefGoogle Scholar
  7. 7.
    Felzenszwalb PF, Girshick R, McAllester D, Ramanan D (2010) Object detection with discriminatively trained partbased models. IEEE Trans Pattern Anal Mach Intell 32(9):1627–1645CrossRefGoogle Scholar
  8. 8.
    Geng M, Wang Y, Xiang T, Tian Y (2016) Deep transfer learning for person re-identification. arXiv:1611.05244zz
  9. 9.
    Girshick R, Donahue J, Darrell T, Malik J (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 580–587Google Scholar
  10. 10.
    Gray D, Brennan S, Tao H (2007) Evaluating appearance models for recognition, reacquisition, and tracking. In: Proc. IEEE international workshop on performance evaluation for tracking and surveillance (PETS), vol 3, pp 1–7Google Scholar
  11. 11.
    Guo C-C, Chen S-Z, Lai J-H, Hu X-J, Shi S-C (2014) Multi-shot person re-identification with automatic ambiguity inference and removal. In: Proc. 22nd international conference on pattern recognition (ICPR), pp 3540–3545Google Scholar
  12. 12.
    He K, Zhang X, Ren S, Sun J (2015) Deep residual learning for image recognition. arXiv:1512.03385
  13. 13.
    He Z, Zhang Z, Jung C (2018) Deep feature embedding learning for person re-identification using lifted structured loss. In: Proceedings of IEEE conference on acoustics, speech and signal processing (ICASSP), pp 1957–1961Google Scholar
  14. 14.
    Hermans A, Beyer L, Leibe B (2017) In defense of the triplet loss for person re-identification. arXiv:1703.07737
  15. 15.
    Jin H, Wang X, Liao S, Li SZ (2017) Deep person re-identification with improved embedding. arXiv:1705.03332
  16. 16.
    Jones M, Marks T (2015) An improved deep learning architecture for person re-identification. In: Proc. IEEE conference on computer vision and pattern recognition (CVPR)Google Scholar
  17. 17.
    Karen S, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556
  18. 18.
    Khamis S, Kuo C-H, Singh VK, Shet V D, Davis LS (2014) Joint learning for attribute-consistent person re-identification. In: Proc. ECCV Workshops, vol 3, pp 134–146Google Scholar
  19. 19.
    Koestinger M, Hirzer M, Wohlhart P, Roth PM, Bischof H (2012) Large scale metric learning from equivalence constraints. In: Proc. IEEE conference on computer vision and pattern recognition (CVPR), pp 2288–2295Google Scholar
  20. 20.
    Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems (NIPS), pp 1097–1105Google Scholar
  21. 21.
    Li W, Wang X (2013) Locally aligned feature transforms across views. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 3594-3601Google Scholar
  22. 22.
    Li W, Zhao R, Xiao T, Wang X (2014) Deepreid, Deep filter pairing neural network for person re-identification. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR)Google Scholar
  23. 23.
    Liao S, Hu Y, Zhu X, Li SZ (2015) Person re-identification by local maximal occurrence representation and metric learning. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 2197–2206Google Scholar
  24. 24.
    Liu H, Feng J, Qi M, Jiang J, Yan S (2017) End-to-end comparative attention networks for person re-identification. IEEE Trans Image Process 26(7):3492–3506MathSciNetCrossRefGoogle Scholar
  25. 25.
    Liu X, Song M, Zhao Q, Tao D, Chen C, Bu J (2012) Attribute-restricted latent topic model for person reidentification. Pattern Recogn 45(12):4204–4213CrossRefGoogle Scholar
  26. 26.
    Nanda A, Chauhan DS, Sa PK, Bakshi S (2017) Illumination and scale invariant relevant visual features with hypergraph-based learning for multi-shot person re-identification. Multimedia Tools and Applications, pp 1–26Google Scholar
  27. 27.
    Nanda A, Sa PK, Choudhury SK, Bakshi S, Majhi B (2017) A Neuromorphic Person Re-Identification Framework for Video Surveillance. IEEE Access 5:6471–6482Google Scholar
  28. 28.
    Paisitkriangkrai S, Shen C, van den Hengel A (2015) Learning to rank in person re-identification with metric ensembles. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 1846–1855Google Scholar
  29. 29.
    Schroff F, Kalenichenko D, Philbin J (2015) Facenet: A unified embedding for face recognition and clustering. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 815–823Google Scholar
  30. 30.
    Shi H, Yang Y, Zhu X, Liao S, Lei Z, Zheng W, Li S Z (2016) Embedding deep metric for person re-identification: A study against large variations. In: Proc. European conference on computer vision (ECCV), pp 732–748, SpringerGoogle Scholar
  31. 31.
    Song H, Xiang Y, Jegelka S, Savarese S (2016) Deep metric learning via lifted structured feature embedding. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4004–4012Google Scholar
  32. 32.
    Subramaniam A, Chatterjee M, Mittal A (2016) Deep neural networks with inexact matching for person re-identification. In: Advances in neural information processing systems (NIPS), pp 2667–2675Google Scholar
  33. 33.
    Varior RR, Haloi M, Wang G (2016) Gated siamese convolutional neural network architecture for human re-identification. In: Proc. European conference on computer vision (ECCV), pp 791–808, SpringerGoogle Scholar
  34. 34.
    Varior RR, Shuai B, Lu J, Xu D, Wang G (2016) A siamese long short-term memory architecture for human re-identification. In: Proc. European Conference on Computer Vision, pp 135–153, SpringerGoogle Scholar
  35. 35.
    Wang F, Zuo W, Lin L, Zhang D, Zhang L (2016) Joint learning of single-image and cross-image representations for person re-identification. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 1288–1296Google Scholar
  36. 36.
    Weinberger KQ, Saul LK (2009) Distance metric learning for large margin nearest neighbor classification. J Mach Learn Res 10:207–244zbMATHGoogle Scholar
  37. 37.
    Xiao T, Li H, Ouyang W, Wang X (2016) Learning deep feature representations with domain guided dropout for person re-identification. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 1249–1258Google Scholar
  38. 38.
    Xiong F, Gou M, Camps O, Sznaier M (2014) Person re-identification using kernel-based metric learning methods. In: Proc. European conference on computer vision (ECCV), pp 1–16, SpringerGoogle Scholar
  39. 39.
    Yi D, Lei Z, Liao S, Li SZ (2014) Deep metric learning for person re-identification. In: Proc. 22nd international conference on pattern recognition (ICPR), pp 34–39Google Scholar
  40. 40.
    Zhang L, Xiang T, Gong S (2016) Learning a discriminative null space for person re-identification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1239–1248Google Scholar
  41. 41.
    Zhang Y, Li B, Lu H, Irie A, Ruan X (2016) Sample specific svm learning for person re-identification. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 1278–1287Google Scholar
  42. 42.
    Zhao H, Tian M, Sun S, Shao J, Yan J, Yi S, Wang X, Tang X (2017) Spindle net: Person re-identification with human body region guided feature decomposition and fusion. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 907–915Google Scholar
  43. 43.
    Zhao R, Ouyang W, Wang X (2013) Person re-identification by salience matching. In: Proceedings of the IEEE international conference on computer vision (ICCV), pp 2528–2535Google Scholar
  44. 44.
    Zheng L, Shen L, Tian L, Wang S, Wang J, Tian Q (2015) Scalable person re-identification: A benchmark. In: Proc. IEEE international conference on computer vision (ICCV)Google Scholar
  45. 45.
    Zheng L, Yang Y, Hauptmann AG (2016) Person reidentification: Past, present and future. arXiv:1610.02984
  46. 46.
    Zheng Z, Zheng L, Yang Y (2016) A discriminatively learned cnn embedding for person re-identification. arXiv:1611.05666
  47. 47.
    Zheng Z, Zheng L, Yang Y (2017) Unlabeled samples generated by gan improve the person re-identification baseline in vitro. In: Proceedings of the IEEE international conference on computer vision (ICCV)Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.School of Electronic EngineeringXidian UniversityXianChina

Personalised recommendations