Skip to main content
SpringerLink
Log in

Semi-supervised learning for person re-identification based on style-transfer-generated data by CycleGANs

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

Person re-identification (re-ID) is an exceedingly significant branch in the field of computer vision, especially for video surveillance. It is still a challenge to obtain more labeled training data and use them reasonably for more precise matching, though the person re-ID performance has been improved significantly. In order to solve this challenge, this study proposes a semi-supervised learning algorithm for data augmentation, the style-transfer-generated data as an extra class (STGDEC), which is aided by the Cycle-Consistent Adversarial Networks (CycleGANs) in generating extra unlabeled training data. Specifically, the algorithm firstly trains the CycleGANs and Deep Convolutional Generative Adversarial Networks so as to generate large amounts of unlabeled data. Secondly, we propose an adaptive receptive field module to expand the size of receptive fields and select the appropriate receptive field features dynamically in order to learn more contextual information and discriminative feature representation and embed the module in the backbone network easily. Thirdly, we use the combination of label smoothing regularization for outliers and an extra class loss to regularize the generated data and encourage the network not to be too confident to the ground-truth. Finally, this paper proposes three training strategies for the combination of standard dataset and generated samples. Comprehensive experiments based on the STGDEC are conducted, and these results show that the proposed algorithm gains a significant improvement over the baseline, the Basel. + LSRO and state-of-the-art approaches of person re-ID in many cases.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Zheng, L., Yang, Y., Hauptmann, A.G.: Person re-identification: past, present and future. arXiv:1610.02984 (2016)

  2. Zhong, Z., Zheng, L., Zheng, Z., Li, S., Yang, Y.: Camera style adaptation for person re-identification. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5157–5166 (2018)

  3. Ahmed, E., Jones, M., Marks, T.K.: An improved deep learning architecture for person re-identification. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3908–3916 (2015)

  4. Chen, S., Guo, C., Lai, J.: Deep ranking for person re-identification via joint representation learning. IEEE Trans. Image Process. (TIP) 25(5), 2353–2367 (2016)

    Article  MathSciNet  Google Scholar 

  5. Liao, S., Hu, Y., Zhu, X., Li, S.Z.: Person re-identification by local maximal occurrence representation and metric learning. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2197–2206 (2015)

  6. Paisitkriangkrai, S., Shen, C., van den Hengel, A.: Learning to rank in person re-identification with metric ensembles. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1846–1855 (2015)

  7. Tao, D., Guo, Y., Song, M., Li, Y., Yu, Z., Tang, Y.Y.: Person re-identification by dual-regularized KISS metric learning. IEEE Trans. Image Process. (TIP) 25(6), 2726–2738 (2016)

    Article  MathSciNet  Google Scholar 

  8. Wu, L., Shen, C., van den Hengel, A.: Deep linear discriminant analysis on fisher networks: a hybrid architecture for person re-identification. Pattern Recognit. (PR) 65, 238–250 (2017)

    Article  Google Scholar 

  9. Zheng, Z., Zheng, L., Yang, Y.: Unlabeled samples generated by gan improve the person re-identification baseline in vitro. In: IEEE International Conference on Computer Vision (ICCV), pp. 3774–3782 (2017)

  10. Huang, Y., Xu, J., Wu, Q., Zheng, Z., Zhang, Z., Zhang, J.: Multi-pseudo regularized label for generated data in person re-identification. IEEE Trans. Image Process. (TIP) 28(3), 1391–1403 (2019)

    Article  MathSciNet  Google Scholar 

  11. Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial nets. In: Neural Information Processing Systems (NIPS), pp. 2672–2680 (2014)

  12. Xin, X., Wang, J., Xie, R., Zhou, S., Huang, W., Zheng, N.: Semi-supervised person re-identification using multi-view clustering. Pattern Recognit. (PR) 88, 285–297 (2019)

    Article  Google Scholar 

  13. Liang, C., Huang, B., Hu, R., Zhang, C., Jing, X., Xiao, J.: A unsupervised person re-identification method using model based representation and ranking. In: Proceedings of the 23rd ACM International Conference on Multimedia, pp. 771–774 (2015)

  14. Peng, P., Xiang, T., Wang, Y., Pontil, M., Gong, S., Huang, T., Tian, Y.: Unsupervised cross-dataset transfer learning for person re-identification. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1306–1315 (2016)

  15. Zhu. J., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: IEEE International Conference on Computer Vision (ICCV), pp. 2242–2251 (2017)

  16. Radford, A., Metz, L., Chintala, S.: Unsupervised representation learning with deep convolutional generative adversarial networks. arxiv:1511.06434 (2016)

  17. Zheng, L., Shen, L., Tian, L., Wang, S., Wang, J., Tian, Q.: Scalable person re-identification: a benchmark. In: IEEE International Conference on Computer Vision (ICCV), pp. 1116–1124 (2015)

  18. Gatys, L.A., Ecker, A.S., Bethge, M.: Image style transfer using convolutional neural networks. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2414–2423 (2016)

  19. Isola, P., Zhu, J.-Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1125–1134 (2017)

  20. Liu, M.-Y., Tuzel, O.: Coupled generative adversarial networks. In: Neural Information Processing Systems (NIPS), pp. 469–477 (2016)

  21. Taigman, Y., Polyak, A., Wolf, L.: Unsupervised cross-domain image generation. arxiv:1611.02200 (2016)

  22. Zhao, L., Li, X., Zhuang, Y., Wang, J.: Deeply-learned part-aligned representations for person re-identification. In: IEEE International Conference on Computer Vision (ICCV), pp. 3219–3228 (2017)

  23. Lin, Y., Zheng, L., Zheng, Z., Wu, Y., Hu, Z., Yan, C., Yang, Y.: Improving person re-identification by attribute and identity learning. Pattern Recognit. (PR) 95, 151–161 (2019)

    Article  Google Scholar 

  24. He, L., Liang, J., Li, H., Sun, Z.: Deep spatial feature reconstruction for partial person re-identification: alignment-free approach. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7073–7082 (2018)

  25. Zheng, L., Huang, Y., Lu, H., Yang, Y.: Pose-invariant embedding for deep person re-identification. IEEE Trans. Image Process. (TIP) 28(9), 4500–4509 (2019)

    Article  MathSciNet  Google Scholar 

  26. Yi, D., Lei, Z., Liao, S., Li, S.Z.: Deep metric learning for person re-identification. In: 2014 22nd International Conference on Pattern Recognition (ICPR), pp. 34–39 (2014)

  27. Li, W., Zhao, R., Xiao, T., Wang, X.: Deepreid: deep filter pairing neural network for person re-identification. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 152–159 (2014)

  28. Figueira, D., Bazzani, L., Minh, H.Q., Cristani, M., Bernardino, A., Murino, V.: Semi-supervised multi-feature learning for person re-identification. In: IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), pp. 111–116 (2013)

  29. Liu, X., Song, M., Tao, D., Zhou. X., Chen, C., Bu, J.: Semi-supervised coupled dictionary learning for person re-identification. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3550–3557 (2014)

  30. Yang, X., Wang, M., Hong, R., Tian, Q., Rui, Y.: Enhancing person re-identification in a self-trained subspace. ACM Trans. Multimed. Comput. Commun. Appl. (TOMCCAP) 13(3), 27:1-27:23 (2017)

    Google Scholar 

  31. Liu, Y., Song, G., Shao, J., Jin, X., Wang, X.: Transductive centroid projection for semi-supervised large-scale recognition. In: European Conference on Computer Vision (ECCV), pp. 70–86 (2018)

  32. Salimans, T., Goodfellow, I., Zaremba, W., Cheung, V., Radford, A., Chen, X.: Improved techniques for training gans. In: Neural Information Processing Systems (NIPS), pp. 2234–2242 (2016)

  33. Ding, G., Zhang, S., Khan, S., Tang, Z., Zhang, J., Porikli, F.: Feature affinity based pseudo labeling for semi-supervised person re-identification. IEEE Trans. Multimed. (TOM) 21(11), 2891–2902 (2019)

    Article  Google Scholar 

  34. Sun, Y., Zheng, L., Yang, Y., Tian, Q., Wang, S.: Beyond part models: person retrieval with refined part pooling (and a strong convolutional baseline). In: The European Conference on Computer Vision (ECCV), pp. 501–518 (2018)

  35. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.: Going deeper with convolutions. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1–9 (2015)

  36. Hu, J., Shen, L., Sun, G.: Squeeze-and-Excitation Networks. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7132–7141 (2018)

  37. Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2818–2826 (2016)

  38. Felzenszwalb, P.F., Girshick, R.B., McAllester, D., Ramanan, D.: Object detection with discriminatively trained part-based models. IEEE Trans. Pattern Anal. Mach. Intell. (TPAMI) 32(9), 1627–1645 (2010)

    Article  Google Scholar 

  39. Ristani, E., Solera, F., Zou, R., Cucchiara, R., Tomasi, C.: Performance measures and a data set for multi-target, multi-camera tracking. In: European Conference on Computer Vision (ECCV), pp. 17–35 (2016)

  40. Lee, D.-H.: Pseudo-label: the simple and efficient semi-supervised learning method for deep neural networks. In: Workshop on Challenges in Representation Learning, ICML, pp. 2 (2013)

  41. Vedaldi, A., Lenc, K.: Matconvnet: convolutional neural networks for matlab. In: Proceedings of the 23rd ACM International Conference on Multimedia, pp. 689–692 (2015)

  42. Luo, H., Gu, Y., Liao, X., Lai, S., Jiang, W.: Bags of tricks and a strong baseline for deep person re-identification. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pp. 0 (2019)

  43. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv:1412.6980 (2014)

  44. Ustinova, E., Ganin, Y., Lempitsky, V.: Multi bilinear convolutional neural networks for person re-identification. arXiv:1512.05300 (2015)

  45. Chen, D., Yuan, Z., Chen, B., Zheng, N.: Similarity learning with spatial constraints for person re-identification. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1268–1277 (2016)

  46. Zhang, L., Xiang, T., Gong, S.: Learning a discriminative null space for person re-identification. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1239–1248 (2016)

  47. Varior, R.R., Haloi, M., Wang, G.: Gated siamese convolutional neural network architecture for human re-identification. In: The European Conference on Computer Vision (ECCV), pp. 791–808 (2016)

  48. Barbosa, I.B., Cristani, M., Caputo, B., Rognhaugen, A., Theoharis, T.: Looking beyond appearances: synthetic training data for deep CNNs in re-identification. Comput. Vis. Image Underst. (CVIU) 167, 50–62 (2018)

    Article  Google Scholar 

  49. Zhao, H., Tian, M., Sun, S., Shao, J., Yan, J., Yi, S., Wang, X., Tang, X.: Spindle net: person re-identification with human body region guided feature decomposition and fusion. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 907–915 (2017)

  50. Li, W., Zhu, X., Gong, S.: Harmonious attention network for person re-identification. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2285–2294 (2018)

Download references

Acknowledgements

This work is partly supported by National Natural Science Foundation of China (No. 61973066), Distinguished Creative Talent Program of Liaoning Colleges and Universities (LR2019027), and Fundamental Research Funds for the Central Universities (N182608004, N2004022).

Author information

Authors and Affiliations

  1. Faculty of Robot Science and Engineering, Northeastern University, Shenyang, 110819, China

    Shangdong Zhu & Song Wang

  2. College of Information Science and Engineering, Northeastern University, Shenyang, 110819, China

    Yunzhou Zhang, Ruilong Li & Shuangwei Liu

  3. Intelligent Systems Research Centre, University of Ulster, Derry, BT52 1SA, UK

    Sonya Coleman

Authors
  1. Shangdong Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yunzhou Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sonya Coleman
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Song Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ruilong Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Shuangwei Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yunzhou Zhang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhu, S., Zhang, Y., Coleman, S. et al. Semi-supervised learning for person re-identification based on style-transfer-generated data by CycleGANs. Machine Vision and Applications 32, 122 (2021). https://doi.org/10.1007/s00138-021-01239-w

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00138-021-01239-w

Keywords

Search

Navigation