Model compression via pruning and knowledge distillation for person re-identification

Abstract

Person re-identification (ReID) is an important problem in intelligent monitoring. Recently, with the development of deep learning, convolutional neural networks have achieved state-of-the-art performance on person ReID problems. However, the deep neural network models used by these methods tend to have large number of parameters and high computational cost, thereby hindering their deployment on resource-constraint devices or real-time applications. In this study, we propose a method that distills the knowledge to a pruned model to reduce the parameters, which can be divided into two stages: one is to apply unstructured pruning method on over-parameterized models, whereas the other is to carry out representation and metric learning-based knowledge distillation on the model after pruning to improve performance. Finally, the proposed method can effectively reduce the total number of parameters by 8.4 with only 0.1% drop of rank-1 accuracy on the Market1501 dataset and no drop of rank-1 accuracy on the DukeMTMC-reID dataset.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

References

  1. Ashok A, Rhinehart N, Beainy F, Kitani KM (2018) N2n learning: network to network compression via policy gradient reinforcement learning. In: ICLR 2018: International conference on learning representations 2018

  2. Chen LC, Papandreou G, Kokkinos I, Murphy K, Yuille A (2018a) Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE Trans Pattern Anal Mach Intell 40(4):834–848. https://doi.org/10.1109/TPAMI.2017.2699184

    Article  Google Scholar 

  3. Chen Y, Zhang Z, Wang N (2018b) Darkrank: Accelerating deep metric learning via cross sample similarities transfer. In: AAAI-18 AAAI conference on artificial intelligence, pp 2852–2859

  4. Cheng D, Gong Y, Zhou S, Wang J, Zheng N (2016) Person re-identification by multi-channel parts-based cnn with improved triplet loss function. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR), pp 1335–1344. https://doi.org/10.1109/CVPR.2016.149

  5. Cheng J, Wang P, Li G, Hu Q, Lu H (2018) Recent advances in efficient computation of deep convolutional neural networks. J Zhejiang Univ Sci C 19(1):64–77. https://doi.org/10.1631/FITEE.1700789

    Article  Google Scholar 

  6. Chollet F (2017) Xception: Deep learning with depthwise separable convolutions. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR), pp 1800–1807. https://doi.org/10.1109/CVPR.2017.195

  7. Denil M, Shakibi B, Dinh L, Ranzato M, de Freitas N (2013) Predicting parameters in deep learning. Adv Neural Inf Process Syst 26:2148–2156

    Google Scholar 

  8. Han S, Pool J, Tran J, Dally WJ (2015) Learning both weights and connections for efficient neural networks. Adv Neural Inf Process Syst 28:1135–1143

    Google Scholar 

  9. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR), pp 770–778. https://doi.org/10.1109/CVPR.2016.90

  10. He Y, Liu P, Wang Z, Hu Z, Yang Y (2019) Filter pruning via geometric median for deep convolutional neural networks acceleration. In: 2019 IEEE conference on computer vision and pattern recognition (CVPR), pp 4340–4349. https://doi.org/10.1109/CVPR.2019.00447

  11. Hermans A, Beyer L, Leibe B (2017) In defense of the triplet loss for person re-identification. arXiv:1703.07737

  12. Hinton GE, Vinyals O, Dean J (2015) Distilling the knowledge in a neural network. arXiv:1503.02531

  13. Ioffe S, Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. In: Proceedings of the 32nd international conference on machine learning, pp 448–456

  14. Izutov E (2018) Fast and accurate person re-identification with rmnet. arXiv:1812.02465

  15. Kim YD, Park E, Yoo S, Choi T, Yang L, Shin D (2016) Compression of deep convolutional neural networks for fast and low power mobile applications. In: ICLR 2016: International conference on learning representations 2016

  16. Kullback S, Leibler RA (1951) On information and sufficiency. Ann Math Stat 22(1):79–86. https://doi.org/10.1214/aoms/1177729694

    MathSciNet  Article  MATH  Google Scholar 

  17. Lemaire C, Achkar A, Jodoin PM (2018) Structured pruning of neural networks with budget-aware regularization. arXiv:1811.09332

  18. Leng C, Dou Z, Li H, Zhu S, Jin R (2018) Extremely low bit neural network: squeeze the last bit out with admm. In: AAAI-18 AAAI conference on artificial intelligence, pp 3466–3473

  19. Li H, Kadav A, Durdanovic I, Samet H, Graf HP (2017) Pruning filters for efficient convnets. In: ICLR 2017: International conference on learning representations 2017

  20. Li M, Zuo T, Li R, White M, Zheng W (2018) Accelerating large scale knowledge distillation via dynamic importance sampling. arXiv:1812.00914

  21. Lin S, Ji R, Yan C, Zhang B, Cao L, Ye Q, Huang F, Doermann DS (2019a) Towards optimal structured cnn pruning via generative adversarial learning. In: 2019 IEEE conference on computer vision and pattern recognition (CVPR), pp 2790–2799. https://doi.org/10.1016/10.1109/CVPR.2019.00290

  22. Lin Y, Zheng L, Zheng Z, Wu Y, Yang Y (2019b) Improving person re-identification by attribute and identity learning. Pattern Recogn 95:151–161. https://doi.org/10.1016/j.patcog.2019.06.006

    Article  Google Scholar 

  23. Liu Z, Li J, Shen Z, Huang G, Yan S, Zhang C (2017) Learning efficient convolutional networks through network slimming. In: 2017 IEEE international conference on computer vision (ICCV), pp 2755–2763. https://doi.org/10.1109/ICCV.2017.298

  24. Liu X, Ounifi H, Gherbi A, Li W, Cheriet M (2019) A hybrid GPU-FPGA based design methodology for enhancing machine learning applications performance. J Ambient Intell Human Comput. https://doi.org/10.1007/s12652-019-01357-4

    Article  Google Scholar 

  25. Luo H, Gu Y, Liao X, Lai S, Jiang W (2019) Bag of tricks and a strong baseline for deep person re-identification. arXiv:1903.07071

  26. Matsukawa T, Suzuki E (2016) Person re-identification using cnn features learned from combination of attributes. In: 2016 23rd International conference on pattern recognition (ICPR), pp 2428–2433

  27. Mnih V, Heess N, Graves A, Kavukcuoglu K (2014) Recurrent models of visual attention. Adv Neural Inf Process Syst 27:2204–2212

    Google Scholar 

  28. Ristani E, Tomasi C (2018) Features for multi-target multi-camera tracking and re-identification. In: 2018 IEEE conference on computer vision and pattern recognition (CVPR), pp 6036–6046. https://doi.org/10.1109/CVPR.2018.00632

  29. Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein MS, Berg AC, Fei-Fei L (2015) Imagenet large scale visual recognition challenge. Int J Comput Vision 115(3):211–252. https://doi.org/10.1007/s11263-015-0816-y

    MathSciNet  Article  Google Scholar 

  30. Sandler MB, Howard A, Zhu M, Zhmoginov A, Chen LC (2018) Mobilenetv2: inverted residuals and linear bottlenecks. In: 2018 IEEE conference on computer vision and pattern recognition (CVPR), pp 4510–4520. https://doi.org/10.1109/CVPR.2018.00474

  31. Sun Y, Zheng L, Deng W, Wang S (2017) Svdnet for pedestrian retrieval. In: 2017 IEEE international conference on computer vision (ICCV), pp 3820–3828. https://doi.org/10.1109/ICCV.2017.410

  32. Sun Y, Zheng L, Yang Y, Tian Q, Wang S (2018) Beyond part models: person retrieval with refined part pooling (and a strong convolutional baseline). In: Proceedings of the European conference on computer vision (ECCV), pp 480–496. https://doi.org/10.1007/978-3-030-01225-0_30

  33. Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z (2016) Rethinking the inception architecture for computer vision. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR), pp 2818–2826. https://doi.org/10.1109/CVPR.2016.308

  34. Varior RR, Shuai B, Lu J, Xu D, Wang G (2016) A siamese long short-term memory architecture for human re-identification. In: Proceedings of the European conference on computer vision (ECCV), pp 135–153. https://doi.org/10.1007/978-3-319-46478-7_9

  35. Wang G, Yuan Y, Chen X, Li J, Zhou X (2018a) Learning discriminative features with multiple granularities for person re-identification. In: Proceedings of the 26th ACM international conference on multimedia, pp 274–282. https://doi.org/10.1145/3240508.3240552

  36. Wang P, Hu Q, Zhang Y, Zhang C, Liu Y, Cheng J (2018b) Two-step quantization for low-bit neural networks. In: 2018 IEEE conference on computer vision and pattern recognition (CVPR), pp 4376–4384. https://doi.org/10.1109/CVPR.2018.00460

  37. Xiong F, Xiao Y, Cao Z, Gong K, Fang Z, Zhou JT (2019) Towards good practices on building effective cnn baseline model for person re-identification. In: Tenth international conference on graphics and image processing (ICGIP 2018)

  38. Ye J, Wang L, Li G, Chen D, Zhe S, Chu X, Xu Z (2018) Learning compact recurrent neural networks with block-term tensor decomposition. In: 2018 IEEE conference on computer vision and pattern recognition (CVPR), pp 9378–9387. https://doi.org/10.1109/CVPR.2018.00977

  39. Yim J, Joo D, Bae J, Kim J (2017) A gift from knowledge distillation: fast optimization, network minimization and transfer learning. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR), pp 7130–7138. https://doi.org/10.1109/CVPR.2017.754

  40. Yu L, Yazici VO, Liu X, van de Weijer J, Cheng Y, Ramisa A (2019) Learning metrics from teachers: compact networks for image embedding. In: 2019 IEEE conference on computer vision and pattern recognition (CVPR), pp 2907–2916. https://doi.org/10.1109/CVPR.2019.00302

  41. Yuan X, Ren L, Lu J, Zhou J (2019) Enhanced bayesian compression via deep reinforcement learning. In: 2019 IEEE conference on computer vision and pattern recognition (CVPR), pp 6946–6955. https://doi.org/10.1109/CVPR.2019.00711

  42. Zagoruyko S, Komodakis N (2017) Paying more attention to attention: improving the performance of convolutional neural networks via attention transfer. In: ICLR 2017: International conference on learning representations 2017

  43. Zavalamondragon LA, Lamichhane B, Zhang L, De Haan G (2019) Cnn-skelpose: a cnn-based skeleton estimation algorithm for clinical applications. J Ambient Intell Human Comput. https://doi.org/10.1007/s12652-019-01259-5

  44. Zhang X, Zou J, He K, Sun J (2016) Accelerating very deep convolutional networks for classification and detection. IEEE Trans Pattern Anal Mach Intell 38(10):1943–1955. https://doi.org/10.1109/TPAMI.2015.2502579

    Article  Google Scholar 

  45. Zhang X, Luo H, Fan X, Xiang W, Sun Y, Xiao Q, Jiang W, Zhang C, Sun J (2017) Alignedreid: surpassing human-level performance in person re-identification. arXiv:1711.08184

  46. Zhang X, Zhou X, Lin M, Sun J (2018) Shufflenet: an extremely efficient convolutional neural network for mobile devices. In: 2018 IEEE conference on computer vision and pattern recognition (CVPR), pp 6848–6856. https://doi.org/10.1109/CVPR.2018.00716

  47. Zhao H, Tian M, Sun S, Shao J, Yan J, Yi S, Wang X, Tang X (2017) Spindle net: person re-identification with human body region guided feature decomposition and fusion. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR), pp 907–915. https://doi.org/10.1109/CVPR.2017.103

  48. Zheng L, Shen L, Tian L, Wang S, Wang J, Tian Q (2015) Scalable person re-identification: a benchmark. In: 2015 IEEE international conference on computer vision (ICCV), pp 1116–1124. https://doi.org/10.1109/ICCV.2015.133

  49. Zheng Z, Zheng L, Yang Y (2017a) A discriminatively learned cnn embedding for person reidentification. ACM Trans Multimedia Comput Commun Appl 14(1):13. https://doi.org/10.1145/3159171

    Article  Google Scholar 

  50. Zheng Z, Zheng L, Yang Y (2017b) Unlabeled samples generated by gan improve the person re-identification baseline in vitro. In: 2017 IEEE international conference on computer vision (ICCV), pp 3774–3782. https://doi.org/10.1109/ICCV.2017.405

  51. Zheng F, Deng C, Sun X, Jiang X, Guo X, Yu Z, Huang F, Ji R (2019) Pyramidal person re-identification via multi-loss dynamic training. In: 2019 IEEE conference on computer vision and pattern recognition (CVPR), pp 8514–8522. https://doi.org/10.1109/CVPR.2019.00871

  52. Zhong Z, Zheng L, Cao D, Li S (2017) Re-ranking person re-identification with k-reciprocal encoding. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR), pp 3652–3661. https://doi.org/10.1109/CVPR.2017.389

  53. Zhou B, Khosla A, Lapedriza A, Oliva A, Torralba A (2016) Learning deep features for discriminative localization. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR), pp 2921–2929. https://doi.org/10.1109/CVPR.2016.319

Download references

Acknowledgements

This study was funded by the National Natural Science Foundation of China under Grant 61633019, the Science Foundation of Chinese Aerospace Industry under Grant JCKY2018204B053 and the Autonomous Research Project of the State Key Laboratory of Industrial Control Technology, China (Grant No. ICT1917).

Author information

Affiliations

Authors

Corresponding author

Correspondence to Wei Jiang.

Ethics declarations

Conflicts of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Xie, H., Jiang, W., Luo, H. et al. Model compression via pruning and knowledge distillation for person re-identification. J Ambient Intell Human Comput 12, 2149–2161 (2021). https://doi.org/10.1007/s12652-020-02312-4

Download citation

Keywords

  • Person re-identification
  • Model compression and acceleration
  • Parameter pruning
  • Knowledge distillation