Advertisement

Video-based person re-identification using a novel feature extraction and fusion technique

We’re sorry, something doesn't seem to be working properly.

Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.

  • 40 Accesses

Abstract

Person re-identification has received extensive attention in the academic community. In this paper, a novel multiple feature fusion network (MPFF-Net) is proposed for video-based person re-identification. The proposed network is used to obtain the robust and discriminative feature representation for describing the pedestrian in the video, which contains the hand-crafted and deep-learned parts. First, the image-level features of all consecutive frames are extracted. Then the hand-crafted branch uses these descriptors to obtain the average feature of the video and the information of frame-to-frame differences. The deep-learned branch is based on the bidirectional LSTM (BiLSTM) network. It is responsible for aggregating frame-wise representations of human regions and yielding sequence-level features. Furthermore, the problem of misalignment is taken into account in this branch. Finally, the hand-crafted and deep-learned parts are considered to be complementary, and the fusion of them can help to capture the complete information of the video. Extensive experiments are conducted on the iLIDS-VID, PRID2011 and MARS datasets. The results demonstrate that the proposed algorithm outperforms state-of-the-art video-based re-identification methods.

This is a preview of subscription content, log in to check access.

Access options

Buy single article

Instant unlimited access to the full article PDF.

US$ 39.95

Price includes VAT for USA

Subscribe to journal

Immediate online access to all issues from 2019. Subscription will auto renew annually.

US$ 199

This is the net price. Taxes to be calculated in checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4

References

  1. 1.

    Bedagkar Gala A, Shah S (2014) Editor’s choice article: a survey of approaches and trends in person re-identification. Image Vis Comput 32(4):270–286

  2. 2.

    Boulgouris N V, Hatzinakos D, Plataniotis K N (2005) Gait recognition: a challenging signal processing technology for biometric identification. Signal Process Mag IEEE 22(6):78–90

  3. 3.

    Cheng D, Gong Y, Zhou S et al (2016) Person re-identification by multi-channel parts-based cnn with improved triplet loss function, pp 1335–1344

  4. 4.

    Dai J, Zhang P, Wang D, Lu H, Wang H (2019) Video person re-identification by temporal residual learning. IEEE Trans Image Process 28(3):1366–1377

  5. 5.

    Engel C, Baumgartner P, Holzmann M et al (2010) Person re-identification by support vector ranking. Aberystwyth, UK, pp 1–11

  6. 6.

    Gao Y, Ma J, Yuille AL (2017) Semi-supervised sparse representation based classification for face recognition with insufficient labeled samples. IEEE Trans Image Process 26(5):2545–2560

  7. 7.

    Gheissari N, Sebastian TB et al (2006) Person reidentification using spatiotemporal appearance, pp 1528–1535

  8. 8.

    Gong S, Cristani M, Yan S, et al. (2014) Person Re-Identification. Springer Publishing Company, Incorporated

  9. 9.

    Gray D, Tao H (2008) Viewpoint invariant pedestrian recognition with an ensemble of localized features, pp 262–275

  10. 10.

    He Z, Jung C, Fu Q, Zhang Z (2018) Deep feature embedding learning for person re-identification based on lifted structured loss. Multimed Tools Appl 78 (5):5863–5880

  11. 11.

    Hirzer M, Beleznai C, Roth P, et al. (2011) Person re-identification by descriptive and discriminative classification. Scandinavian Conference on Image Analysis 6688:91–102

  12. 12.

    Huang W, Liang C, Yu Y, Wang Z, Ruan W, Hu R (2018) Video-based person re-identification via self paced weighting. In: Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, (AAAI-18), the 30th innovative Applications of Artificial Intelligence, New Orleans, pp 2273–2280

  13. 13.

    Kan S, Cen Y, He Z, Zhang Z, Zhang L, Wang Y (2019) Supervised deep feature embedding with hand crafted feature. IEEE Trans Image Process 28 (12):5809–5823

  14. 14.

    Klaser A, Marszalek M, Schmid C (2008) A spatio-temporal descriptor based on 3d-gradients

  15. 15.

    Ksibi S, Mejdoub M, Amar C B (2018) Deep salient-gaussian fisher vector encoding of the spatio-temporal trajectory structures for person re-identification. Multimed Tools Appl 78(2):1583–1611

  16. 16.

    Kviatkovsky I, Adam A, Rivlin E (2013) Color invariants for person reidentification. IEEE Trans Pattern Anal Mach Intell 35(7):1622–1634

  17. 17.

    Li D, Chen X, Zhang Z, Huang K (2017) Learning deep context-aware features over body and latent parts for person re-identification. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 7398–7407

  18. 18.

    Li T, Sun L, Chong H, Jian G (2018) Person re-identification using salient region matching game. Multimed Tools Appl (1C3) 77(16):21393–21415

  19. 19.

    Liao S, Hu Y, Zhu X et al (2015) Person re-identification by local maximal occurrence representation and metric learning, pp 2197–2206

  20. 20.

    Liu C, Gong S, Loy CC et al (2012) Person re-identification: What features are important? European Conference on Computer Vision (ECCV). Berlin, Heidelberg, pp 391–401

  21. 21.

    Liu K, Ma B, Zhang W et al (2015) A spatio-temporal appearance representation for viceo-based pedestrian re-identification, pp 3810–3818

  22. 22.

    Liu H, Feng J, Qi M, Jiang J, Yan S (2016) End-to-end comparative attention networks for person re-identification. IEEE Trans Image Process PP(99):1–1

  23. 23.

    Liu H, Jie Z, Jayashree K, et al. (2017) Video-based person re-identification with accumulative motion context. IEEE Trans Circ Syst Video Technol PP(99):1–1

  24. 24.

    Liu Z, Wang Y, Li A (2018) Hierarchical integration of rich features for video-based person re-identification. IEEE Trans Circ Syst Video Technol 29 (12):3646–3659

  25. 25.

    Liu Y, Song N, Han Y (2019) Multi-cue fusion: Discriminative enhancing for person re-identification. J Vis Commun Image Represent 58:46–52

  26. 26.

    Luo Y, Liu T, Tao D, Xu C (2015a) Multiview matrix completion for multilabel image classification. IEEE Trans Image Process 24(8):2355–2368

  27. 27.

    Luo Y, Tao D, Ramamohanarao K, Xu C, Wen Y (2015b) Tensor canonical correlation analysis for multi-view dimension reduction. IEEE Trans Knowl Data Eng 27(11):3111–3124

  28. 28.

    Luo Y, Wen Y, Tao D, Gui J, Xu C (2016) Large margin multi-modal multi-task feature extraction for image classification. IEEE Trans Image Process 25 (1):414–427

  29. 29.

    Ma J, Yong M, Chang L (2018) Infrared and visible image fusion methods and applications: a survey. Inf Fusion 45:153–178

  30. 30.

    Matsukawa T, Okabe T, Suzuki E et al (2016) Hierarchical gaussian descriptor for person re-identification, pp 1363–1372

  31. 31.

    McLaughlin N, Rincon J, Miller P (2016) In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 1325–1334

  32. 32.

    Mclaughlin N, Rincon J, Miller P (2016) Recurrent convolutional network for video-based person re-identification, pp 1325–1334

  33. 33.

    Mignon A, Jurie F (2012) Pcca: A new approach for distance learning from sparse pairwise constraints, pp 2666–2672

  34. 34.

    Roth PM, Wohlhart P, Hirzer M et al (2012) Large scale metric learning from equivalence constraints, pp 2288–2295

  35. 35.

    Wang T, Gong S, Zhu X, et al. (2014) Person re-identification by video ranking. European Conference on Computer Visio (ECCV) 8692:688–703

  36. 36.

    Wang F, Zhang C, Chen S, Ying G, Lv J (2018) Engineering hand-designed and deeply-learned features for person re-identification. Pattern Recognition Letters

  37. 37.

    Wu S, Chen Y, Li Xea (2016) An enhanced deep feature representation for person re-identification. In: 2016 IEEE Winter Conference on Applications of Computer Vision (WACV):1–8

  38. 38.

    Xiao T, Li H, Wea O (2016) Learning deep feature representations with domain guided dropout for person re-identification, pp 1249–1258

  39. 39.

    Xu S, Cheng Y, Gu K et al (2017) Jointly attentive spatial-temporal pooling networks for video-based person re-identification, pp 4743–4752

  40. 40.

    Yan Y, Ni B, Song Z, et al. (2016) Person re-identification via recurrent feature aggregation. European Conference on Computer Vision (ECCV) 9910:701–716

  41. 41.

    You J, Wu A, Li X et al (2016) Top-push video-based person re-identification, pp 1345–1353

  42. 42.

    Zhang W, He X, Lu W, Qiao H, Li Y (2019) Feature aggregation with reinforcement learning for video-based person re-identification. IEEE Trans Neural Netw Learn Syst 30(12):3847–3852

  43. 43.

    Zheng W, Gong S, Xiang T (2011) Person re-identification by probabilistic relative distance comparison. USA, pp 649–656

  44. 44.

    Zheng W, Gong S, Xiang T (2013) Reidentification by relative distance comparison. IEEE Trans Pattern Anal Mach Intell 35(3):653–668

  45. 45.

    Zheng L, Yang Y, Hauptmann GA (2016a) Person re-identification: Past, present and future

  46. 46.

    Zheng L, Zhi B, Sun Y et al (2016b) Mars: A video benchmark for large-scale person re-identification, vol 9910

  47. 47.

    Zheng Z, Zheng L, Yang Y (2019) Pedestrian alignment network for large-scale person re-identification. IEEE Transactions on Circuits and Systems for Video Technology 29(10):3037–3045

  48. 48.

    Zhou K, Yang Y, Cavallaro A, Xiang T (2019) Omni-scale feature learning for person re-identification. arXiv:190500953

  49. 49.

    Zhou S, Wang J, Meng D, Liang Y, Gong Y, Zheng N (2019) Discriminative feature learning with foreground attention for person re-identification. IEEE Trans Image Process 28(9):4671–4684

  50. 50.

    Zhu X, Jing X, Wu F et al (2016a) Video-based person re-identification by simultaneously learning intra-video and inter-video distance metrics, pp 3552–3558

  51. 51.

    Zhu X, Jing XY, Wu F, Feng H (2016b) Video-based person re-identification by simultaneously learning intra-video and inter-video distance metrics. IEEE Trans Image Process PP(99):1–1

Download references

Acknowledgements

This work was supported in part by the National Natural Science Foundation of China under Grant 61471201 and Grant 61501260, and in part by the Priority Academic Program Development of Jiangsu Higher Education Institutions and in part by Postgraduate Research & Practice Innovation Program of Jiangsu Province KYCX18_0890.

Author information

Correspondence to Feng Liu.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Song, W., Zheng, J., Wu, Y. et al. Video-based person re-identification using a novel feature extraction and fusion technique. Multimed Tools Appl (2020). https://doi.org/10.1007/s11042-019-08432-0

Download citation

Keywords

  • Person re-identification
  • Video
  • Feature representation
  • Hand-crafted
  • Deep-learned