Skip to main content
SpringerLink
Log in

List-wise learning-to-rank with convolutional neural networks for person re-identification

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

In this paper, we present a novel machine learning-based image ranking approach using Convolutional Neural Networks (CNN). Our proposed method relies on a similarity metric learning algorithm operating on lists of image examples and a loss function taking into account the ranking in these lists with respect to different query images. This comprises two major contributions: (1) Rank lists instead of image pairs or triplets are used for training, thus integrating more explicitly the order of similarity and relations between sets of images. (2) A weighting is introduced in the loss function based on two evaluation measures: the mean average precision and the rank 1 score. We evaluated our approach on two different computer vision applications that are commonly formulated as ranking problems: person re-identification and image retrieval with several public benchmarks and showed that our new loss function outperforms other common functions and that our method achieves state-of-the-art performance compared to existing approaches from the literature.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Beyer, L., Breuers, S., Kurin, V., Leibe, B.: Towards a principled integration of multi-camera re-identification and tracking through optimal bayes filters. In: International Conference on Computer Vision Workshops (2017)

  2. Chaaraoui, A.A., Padilla-López, J.R., Ferrández-Pastor, F.J., Nieto-Hidalgo, M., Flórez-Revuelta, F.: A vision-based system for intelligent monitoring: human behaviour analysis and privacy by context. Sensors 14(5), 8895 (2014)

    Article  Google Scholar 

  3. Vezzani, R., Baltieri, D., Cucchiara, R.: People re-identification in surveillance and forensics: a survey. ACM Comput. Surv. CSUR 46(2), 29 (2013)

    Google Scholar 

  4. Farenzena, M., Bazzani, L., Perina, A., Murino, V., Cristani, M.: Person re-identification by symmetry-driven accumulation of local features. In: Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition (IEEE), pp. 2360–2367 (2010)

  5. Ma, B., Su, Y., Jurie, F.: Local descriptors encoded by fisher vectors for person re-identification. In: Proceedings of the European Conference on Computer Vision, pp. 413–422 (2012)

  6. Mignon, A., Jurie, F.: Pcca: a new approach for distance learning from sparse pairwise constraints. In: Computer Vision and Pattern Recognition (CVPR) (IEEE) , pp. 2666–2672 (2012)

  7. Li, W., Wang, X.: Locally aligned feature transforms across views. In: Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition, pp. 3594–3601 (2013)

  8. Zhao, R., Ouyang, W., Wang, X.: Person re-identification by salience matching. In: Proceedings of the International Conference on Computer Vision, pp. 2528–2535 (2013)

  9. Koestinger, M., Hirzer, M., Wohlhart, P., Roth, P.M., Bischof, H.: Large scale metric learning from equivalence constraints. In: Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition, pp. 2288–2295 (2012)

  10. Pedagadi, S., Orwell, J., Velastin, S., Boghossian, B.: Local fisher discriminant analysis for pedestrian re-identification. In: Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition, pp. 3318–3325 (2013)

  11. Xiong, F., Gou, M., Camps, O., Sznaier, M.: Person using kernel-based metric learning methods. In: Proceedings of the European Conference on Computer Vision, pp. 1–16 (2014)

  12. Liao, S., Hu, Y., Zhu, X., Li, S.Z.: Person re-identification by local maximal occurrence representation and metric learning. In: Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition (2015)

  13. Burges, C.J., Ragno, R., Le, Q.V.: Learning to rank with nonsmooth cost functions. In: Neural Information Processing Systems, pp. 193–200 (2007)

  14. Herbrich, R.: Large margin rank boundaries for ordinal regression. Advances in large margin classifiers pp. 115–132 (2000)

  15. Prosser, B.J., Zheng, W.S., Gong, S., Xiang, T., Mary, Q.: Person re-identification by support vector ranking. In: Proceedings of the British Machine Vision Conference (BMVC) (2010)

  16. Burges, C., Shaked, T., Renshaw, E., Lazier, A., Deeds, M., Hamilton, N., Hullender, G.: Learning to rank using gradient descent. In: Proceedings of the International Conference on Machine Learning, pp. 89–96 (2005)

  17. Cao, Z., Qin, T., Liu, T.Y., Tsai, M.F., Li, H.: Learning to rank: from pairwise approach to listwise approach. In: Proceedings of the International Conference on Machine Learning, pp. 129–136 (2007)

  18. Xia, F., Liu, T.Y., Wang, J., Zhang, W., Li, H.: Listwise approach to learning to rank: theory and algorithm. In: Proceedings of the International Conference on Machine Learning, pp. 1192–1199 (2008)

  19. Wang, J., Wang, Z., Gao, C., Sang, N., Huang, R.: Deeplist: Learning deep features with adaptive listwise constraint for person re-identification. IEEE Trans. Circuits Syst. Video Technol. 27(3), 513 (2017)

    Article  Google Scholar 

  20. Kalayeh, M.M., Basaran, E., Gökmen, M., Kamasak, M.E., Shah, M.: Human semantic parsing for person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2018)

  21. Tian, M., Yi, S., Li, H., Li, S., Zhang, X., Shi, J., Yan, J., Wang, X.: Eliminating background bias for robust person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2018)

  22. Song, C., Huang, Y., Ouyang, W., Wang, L.: Mask-guided contrastive attention model for person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2018)

  23. Li, W., Zhu, X., Gong, S.: Harmonious attention network for person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2018)

  24. Xu, J., Zhao, R., Zhu, F., Wang, H., Ouyang, W.: Attention-aware compositional network for person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2119–2128 (2018)

  25. Si, J., Zhang, H., Li, C.G., Kuen, J., Kong, X., Kot, A.C., Wang, G.: Dual attention matching network for context-aware feature sequence based person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2018)

  26. Huang, Y., Xu, J., Wu, Q., Zheng, Z., Zhang, Z., Zhang, J.: Multi-pseudo regularized label for generated samples in person re-identification. IEEE Trans. Image Process. 28(3), 1391 (2019)

    Article  MathSciNet  Google Scholar 

  27. Zheng, Z., Yang, X., Yu, Z., Zheng, L., Li, Y., Kautz, J.: Joint discriminative and generative learning for person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2019)

  28. Zheng, Z., Zheng, L., Yang, Y.: A discriminatively learned CNN embedding for person re-identification. ACM Transactions on Multimedia Computing, Communications and Applications 14(1) (2017)

  29. Chang, X., Hospedales, T.M., Xiang, T.: Multi-level factorisation net for person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2018)

  30. Yi, D., Lei, Z., Liao, S., Li, S.Z.: Deep metric learning for person re-identification. In: Proceedings of the IEEE International Conference on International Conference on Pattern Recognition, pp. 34–39 (2014)

  31. Ahmed, E., Jones, M., Marks, T.K.: An improved deep learning architecture for person re-identification. In: Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition, pp. 3908–3916 (2015)

  32. Li, W., Zhao, R., Xiao, T., Wang, X.: Deepreid:deep filter pairing neural network for person re-identification. In: Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition, pp. 152–159 (2014)

  33. Ding, S., Lin, L., Wang, G., Chao, H.: Deep feature learning with relative distance comparison for person re-identification. Pattern Recognit. 48(10), 2993 (2015)

    Article  Google Scholar 

  34. Cheng, D., Gong, Y., Zhou, S., Wang, J., Zheng, N.: Person re-identification by multichannel parts-based cnn with improved triplet loss function. In: Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition, pp. 1335–1344 (2016)

  35. Chen, W., Chen, X., Zhang, J., Huang, K.: Beyond triplet loss: a deep quadruplet network for person re-identification. In: Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition, vol. 2 (2017)

  36. Shi, H., Yang, Y., Zhu, X., Liao, S., Lei, Z., Zheng, W., Li, S.Z.: Embedding deep metric for person re-identification: a study against large variations. In: European Conference on Computer Vision, pp. 732–748 (2016)

  37. Hermans, A., Beyer, L., Leibe, B.: In defense of the triplet loss for person re-identification. arXiv:1703.07737 (2017)

  38. Wang, J., Leung, T., Rosenberg, C., Wang, J., Philbin, J., Chen, B., Wu, Y. et al.: Learning fine-grained image similarity with deep ranking. In: CVPR (2014)

  39. Schroff, F., Kalenichenko, D., Philbin, J.: A unified embedding for face recognition and clustering. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 815–823 (2015)

  40. Zheng, L., Shen, L., Tian, L., Wang, S., Wang, J., Tian, Q.: Scalable person re-identification: a benchmark. In: International Conference on Computer Vision, pp. 1116–1124 (2015)

  41. Zheng, Z., Zheng, L., Yang, Y.: Unlabeled samples generated by gan improve the person re-identification baseline in vitro. In: International Conference on Computer Vision (2017)

  42. Zhong, Z., Zheng, L., Cao, D., Li, S.: Re-ranking person re-identification with kreciprocal encoding. In: Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition (2017)

  43. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)

  44. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

  45. Yu, R., Zhou, Z., Bai, S., Bai, X.: Divide and fuse: A re-ranking approach for person re-identification. In: Proceedings of the British Machine Vision Conference (BMVC) (2017)

  46. Schumann, A., Stiefelhagen, R.: Person re-identification by deep learning attribute complementary information. In: Computer Vision and Pattern Recognition Workshops (CVPRW) (IEEE), pp. 1435–1443 (2017)

  47. Sun, Y., Zheng, L., Deng, W., Wang, S.: Svdnet for pedestrian retrieval. In: International Conference on Computer Vision (2017)

  48. Li, W., Zhu, X., Gong, S.: Person re-identification by deep joint learning of multi-loss classification. In: International Joint Conference on Artificial Intelligence (2017)

  49. Chen, Y., Zhu, X., Gong, S.: Person re-identification by deep learning multi-scale representations. In: Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition, pp. 2590–2600 (2017)

  50. Jegou, H., Douze, M., Schmid, C.: Hamming embedding and weak geometric consistency for large scale image search. In: European Conference on Computer Vision, pp. 304–317 (2008)

  51. Babenko, A., Slesarev, A., Chigorin, A., Lempitsky, V.: Neural codes for image retrieval. In: European Conference on Computer Vision, pp. 584–599 (2014)

  52. Gordo, A., Almazán, J., Revaud, J., Larlus, D.: Deep image retrieval: Learning global representations for image search. In: European Conference on Computer Vision, pp. 241–257 (2016)

  53. Tolias, G., Sicre, R., Jégou, H.: Particular object retrieval with integral max-pooling of cnn activations. In: International Conference on Learning Representations (2016)

  54. Arandjelovic, R., Gronat, P., Torii, A., Pajdla, T., Sivic, J.: Netvlad: Cnn architecture for weakly supervised place recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5297–5307 (2016)

  55. Kalantidis, Y., Mellina, C., Osindero, S.: Cross-dimensional weighting for aggregated deep convolutional features. In: European Conference on Computer Vision, pp. 685–701 (2016)

  56. Radenović, F., Tolias, G., Chum, O.: Cnn image retrieval learns from bow: Unsupervised fine-tuning with hard examples. In: European Conference on Computer Vision, pp. 3–20 (2016)

Download references

Author information

Authors and Affiliations

  1. LIRIS, CNRS, INSA-Lyon, Université de Lyon, 69621, Villeurbanne, France

    Yiqiang Chen, Stefan Duffner & Atilla Baskurt

  2. Thales Services, ThereSIS, Palaiseau, France

    Andrei Stoian & Jean-Yves Dufour

Authors
  1. Yiqiang Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Stefan Duffner
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Andrei Stoian
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jean-Yves Dufour
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Atilla Baskurt
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Stefan Duffner.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, Y., Duffner, S., Stoian, A. et al. List-wise learning-to-rank with convolutional neural networks for person re-identification. Machine Vision and Applications 32, 50 (2021). https://doi.org/10.1007/s00138-021-01170-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00138-021-01170-0

Keywords

Search

Navigation