Skip to main content
SpringerLink
Log in

Deep-ReID: deep features and autoencoder assisted image patching strategy for person re-identification in smart cities surveillance

  • 1158T: Role of Computer Vision in Smart Cities: Applications and Research Challenges
  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Person Re-identification (P-ReID) task searches for true matches of a given query from a large repository of non-overlapping camera’s images/videos. In smart cities surveillance, P-ReID is challenging due to variation in human’s appearance, illumination affects, and difference in viewpoints. The mainstream approaches achieve P-ReID by implementing supervised learning strategies, requiring exhaustive manual annotation, which is probably erroneous due to human involvement. In addition, the employed methods use high-dimensional feature maps to identify a person, which is not a realistic approach in terms of storage resources and computational complexity. To tackle these issues, we incorporate learned features and deep autoencoder under the umbrella of a unified framework for P-ReID. First, we apply a unique image patching strategy by dividing the input image into two parts (upper and lower) and acquire learned features from fully connected layer of a pretrained Convolutional Neural Network (CNN) model for both patches. To achieve efficient and high performance, the proposed framework utilizes a self-tuned autoencoder to acquire low-dimensional representative features. The obtained features are matched with the patterns of database via cosine similarity measurement to re-identify a person’s appearance. The proposed framework provides a trade-off between time complexity and accuracy, where a lightweight model can be incorporated with reduced number of autoencoder layers to obtain fast and comparatively flexible results. The major novelty of the proposed framework includes implementation of a hybrid network mechanism for P-ReID, which shows convincing real-time results and best fits for smart cities surveillance. The proposed framework is tested over several P-ReID datasets to prove its influence over the existing works with reduced computational complexity.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Ahmed E, M. Jones, and T.K. Marks (2015) An improved deep learning architecture for person re-identification. in Proceedings of the IEEE conference on computer vision and pattern recognition

  2. Alghamdi A et al (2020) Detection of myocardial infarction based on novel deep transfer learning methods for urban healthcare in smart cities. Multimed Tools Appl:1–22

  3. Alghamdi AS, Polat K, Alghoson A, Alshdadi AA, Abd el-Latif AA (2020) Gaussian process regression (GPR) based non-invasive continuous blood pressure prediction method from cuff oscillometric signals. Appl Acoust 164:107256

    Article  Google Scholar 

  4. Almasawa MO, Elrefaei LA, Moria K (2019) A survey on deep Learning-based person re-identification systems. IEEE Access 7:175228–175247

    Article  Google Scholar 

  5. Barbosa IB, Cristani M, Caputo B, Rognhaugen A, Theoharis T (2018) Looking beyond appearances: synthetic training data for deep cnns in re-identification. Comput Vis Image Underst 167:50–62

    Article  Google Scholar 

  6. Chen W, et al. (2016) A multi-task deep network for person re-identification. arXiv preprint arXiv:1607.05369

  7. Chen Y-C, Zhu X, Zheng WS, Lai JH (2017) Person re-identification by camera correlation aware feature augmentation. IEEE Trans Pattern Anal Mach Intell 40(2):392–408

    Article  Google Scholar 

  8. Chen W, et al. (2017) Beyond triplet loss: a deep quadruplet network for person re-identification. in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition

  9. Cheng D, et al. (2016) Person re-identification by multi-channel parts-based cnn with improved triplet loss function. in Proceedings of the iEEE conference on computer vision and pattern recognition

  10. Chu H et al (2019) Local region partition for person re-identification. Multimed Tools Appl:1–17

  11. Da Silva, I.N., et al. (2017) Artificial neural networks. Cham: Springer International Publishing : p. 39.

  12. Dai C, Peng C, Chen M (2020) Selective transfer cycle GAN for unsupervised person re-identification. Multimed Tools Appl:1–17

  13. Ding S, Lin L, Wang G, Chao H (2015) Deep feature learning with relative distance comparison for person re-identification. Pattern Recogn 48(10):2993–3003

    Article  Google Scholar 

  14. Fan D.-P., et al. (2019) Shifting more attention to video salient object detection. in Proceedings of the IEEE conference on computer vision and pattern recognition.

  15. Fu K, Zhao Q, Yu-Hua Gu I, Yang J (2019) Deepside: A general deep framework for salient object detection. Neurocomputing 356:69–82

    Article  Google Scholar 

  16. Fu, K., et al. (2020) Jl-dcf: Joint learning and densely-cooperative fusion framework for rgb-d salient object detection. in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

  17. Gong, S., et al. (2014) The re-identification challenge, in Person re-identification. Springer. p. 1–20.

  18. Hong C, Yu J, Wan J, Tao D, Wang M (2015) Multimodal deep autoencoder for human pose recovery. IEEE Trans Image Process 24(12):5659–5670

    Article  MathSciNet  Google Scholar 

  19. Huang, Y., et al. (2020) Real-world person re-identification via degradation invariance learning. in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.

  20. Hussain, T., et al. (2019) Multi-view summarization and activity recognition meet edge computing in IoT environments. IEEE Internet of Things Journal. https://doi.org/10.1109/JIOT.2020.3027483

  21. Hussain T et al (2020) A comprehensive survey on multi-view video summarization. Pattern Recogn:107567

  22. Iandola, F.N., et al. (2016) SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and< 0.5 MB model size. arXiv preprint arXiv:1602.07360

  23. Jia J et al (2020) View-specific subspace learning and re-ranking for semi-supervised person re-identification. Pattern Recogn 108:107568

    Article  Google Scholar 

  24. Jiao S, Wang J, Hu G, Pan Z, du L, Zhang J (2019) Joint attention mechanism for person re-identification. IEEE Access 7:90497–90506

    Article  Google Scholar 

  25. Jose, C. and F. Fleuret (2016) Scalable metric learning via weighted approximate rank component analysis. In European conference on computer vision. Springer.

  26. Khan SU, Baik R (2020) MPPIF-net: identification of plasmodium falciparum parasite mitochondrial proteins using deep features with multilayer bi-directional LSTM. Processes 8(6):725

    Article  Google Scholar 

  27. Khan MJ, Rahman M (2020) Person re-identification by discriminative local features of overlapping stripes. Symmetry 12(4):647

    Article  Google Scholar 

  28. Khan SU, Haq IU, Rho S, Baik SW, Lee MY (2019) Cover the violence: a novel deep-Learning-based approach towards violence-detection in movies. Appl Sci 9(22):4963

    Article  Google Scholar 

  29. Krizhevsky, A., I. Sutskever, and G.E. Hinton (2012) Imagenet classification with deep convolutional neural networks. in Advances in neural information processing systems

  30. Li H, Kuang Z, Yu Z, Luo J (2020) Structure alignment of attributes and visual features for cross-dataset person re-identification. Pattern Recogn 106:107414

    Article  Google Scholar 

  31. Li K et al (2020) 2 object detection with convolutional neural networks. Deep Learning in Computer Vision: Principles and Applications 30(31):41

    Article  Google Scholar 

  32. Lin Y, Wu Y, Yan C, Xu M, Yang Y (2020) Unsupervised person re-identification via cross-camera similarity exploration. IEEE Trans Image Process 29:5481–5490

    Article  Google Scholar 

  33. Liu Y, Yang H, Zhao Q (2019) Hierarchical feature aggregation from body parts for misalignment robust person re-identification. Appl Sci 9(11):2255

    Article  Google Scholar 

  34. Lu, X., et al. (2019) See more, know more: Unsupervised video object segmentation with co-attention siamese networks. in Proceedings of the IEEE conference on computer vision and pattern recognition

  35. Öztürk Ş (2020) Stacked auto-encoder based tagging with deep features for content-based medical image retrieval. Expert Syst Appl 161:113693

    Article  Google Scholar 

  36. Perwaiz N, Fraz MM, Shahzad M (2018) Person re-identification using hybrid representation reinforced by metric learning. IEEE Access 6:77334–77349

    Article  Google Scholar 

  37. Sang H, Wang C, He D, Liu Q (2019) Multi-information flow CNN and attribute-aided reranking for person reidentification. Computational intelligence and neuroscience 2019:1–12

    Article  Google Scholar 

  38. Shi Y et al (2020) Learning refined attribute-aligned network with attribute selection for person re-identification. Neurocomputing:22071

  39. Tang Y, Yang X, Wang N, Song B, Gao X (2020) CGAN-TM: a novel domain-to-domain transferring method for person re-identification. IEEE Trans Image Process 29:5641–5651

    Article  MathSciNet  Google Scholar 

  40. Tao D, Jin L, Wang Y, Yuan Y, Li X (2013) Person re-identification by regularized smoothing kiss metric learning. IEEE Transactions on Circuits and Systems for Video Technology 23(10):1675–1685

    Article  Google Scholar 

  41. Ul Haq I, Ullah A, Muhammad K, Lee MY, Baik SW (2019) Personalized movie summarization using deep cnn-assisted facial expression recognition. Complexity 2019:110

  42. Ullah A, et al., Conflux LSTMs network: a novel approach for multi-view action recognition. Neurocomputing

  43. Ullah A, Muhammad K, Haq IU, Baik SW (2019) Action recognition using optimized deep autoencoder and CNN for surveillance data streams of non-stationary environments. Futur Gener Comput Syst 96:386–397

    Article  Google Scholar 

  44. Ullah, A., et al., 5 Deep LSTM-based sequence learning approaches for action and activity recognition. Deep Learning in Computer Vision: Principles and Applications, 2020: p. 127.

  45. Ustinova E, Ganin Y, Lempitsky V (2017) Multi-region bilinear convolutional neural networks for person re-identification. In 2017 14th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS). IEEE.

  46. Vadivel, A., A. Majumdar, and S. Sural. (2003) Performance comparison of distance metrics in content-based image retrieval applications. in International Conference on Information Technology (CIT), Bhubaneswar, India

  47. Valmadre J, et al. (2017) End-to-end representation learning for correlation filter based tracking. in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition

  48. Wang Z et al (2017) Person reidentification via discrepancy matrix and matrix metric. IEEE transactions on cybernetics 48(10):3006–3020

    Article  Google Scholar 

  49. Wang F, Zhang C, Chen S, Ying G, Lv J (2020) Engineering hand-designed and deeply-learned features for person re-identification. Pattern Recogn Lett 130:293–298

    Article  Google Scholar 

  50. Wang S, Xu X, Liu L, Tian J (2020) Multi-level feature fusion model-based real-time person re-identification for forensics. J Real-Time Image Proc 17(1):73–81

    Article  Google Scholar 

  51. Watson G, Bhalerao A (2018) Person reidentification using deep foreground appearance modeling. Journal of Electronic Imaging 27(5):051215

    Article  Google Scholar 

  52. Wu S et al (2016) An enhanced deep feature representation for person re-identification. In 2016 IEEE winter conference on applications of computer vision (WACV). IEEE.

  53. Wu L, Shen C, Van Den Hengel A (2017) Deep linear discriminant analysis on fisher networks: a hybrid architecture for person re-identification. Pattern Recogn 65:238–250

    Article  Google Scholar 

  54. Wu D, Zheng SJ, Yuan CA, Huang DS (2019) A deep model with combined losses for person re-identification. Cogn Syst Res 54:74–82

    Article  Google Scholar 

  55. Xiao T, et al. (2016) Learning deep feature representations with domain guided dropout for person re-identification. in Proceedings of the IEEE conference on computer vision and pattern recognition

  56. Xu M, Tang Z, Yao Y, Yao L, Liu H, Xu J (2017) Deep learning for person reidentification using support vector machines. Advances in Multimedia 2017:1–12

    Article  Google Scholar 

  57. Yang X, Chen P (2020) Person re-identification based on multi-scale convolutional network. Multimed Tools Appl 79(13):9299–9313

    Article  Google Scholar 

  58. Yu J, Tao D, Li J, Cheng J (2014) Semantic preserving distance metric learning and applications. Inf Sci 281:674–686

    Article  MathSciNet  Google Scholar 

  59. Zeng, K., et al. (2020) Hierarchical clustering with hard-batch triplet loss for person re-identification. in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

  60. Zhang, L., T. Xiang, and S. Gong (2016) Learning a discriminative null space for person re-identification. in Proceedings of the IEEE conference on computer vision and pattern recognition

  61. Zhang Q, Yang LT, Yan Z, Chen Z, Li P (2018) An efficient deep learning model to predict cloud workload for industry informatics. IEEE transactions on industrial informatics 14(7):3170–3178

    Article  Google Scholar 

  62. Zhang J, et al. (2020) UC-Net: uncertainty inspired rgb-d saliency detection via conditional variational autoencoders. in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

  63. Zhao L, (2017) et al. Deeply-learned part-aligned representations for person re-identification. in Proceedings of the IEEE international conference on computer vision

  64. Zhao H, et al. (2017) Spindle net: Person re-identification with human body region guided feature decomposition and fusion. in Proceedings of the IEEE conference on computer vision and pattern recognition

  65. Zheng L et al. (2015) Scalable person re-identification: A benchmark. in Proceedings of the IEEE international conference on computer vision

  66. Zheng Z, Zheng L, Yang Y (2017) A discriminatively learned cnn embedding for person reidentification. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM) 14(1):1–20

    Google Scholar 

  67. Zheng Z, et al. (2019) Joint discriminative and generative learning for person re-identification. in Proceedings of the IEEE conference on computer vision and pattern recognition

  68. Zhong Z et al (2020) Learning to adapt invariance in memory for person re-identification. IEEE Trans Pattern Anal Mach Intell

  69. Zhou S, et al. (2017) Point to set similarity based deep feature learning for person re-identification. in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition

  70. Zhu X, Jing XY, You X, Zhang X, Zhang T (2018) Video-based person re-identification by simultaneously learning intra-video and inter-video distance metrics. IEEE Trans Image Process 27(11):5683–5695

    Article  MathSciNet  Google Scholar 

  71. Zhu X, Jing XY, Zhang F, Zhang X, You X, Cui X (2019) Distance learning by mining hard and easy negative samples for person re-identification. Pattern Recogn 95:211–222

    Article  Google Scholar 

Download references

Acknowledgments

This work was supported by Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (2019-0-00136, Development of AI-Convergence Technologies for Smart City Industry Productivity Innovation).

Author information

Authors and Affiliations

  1. Sejong University, Seoul, 143-747, Republic of Korea

    Samee Ullah Khan, Tanveer Hussain, Amin Ullah & Sung Wook Baik

Authors
  1. Samee Ullah Khan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tanveer Hussain
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Amin Ullah
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sung Wook Baik
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sung Wook Baik.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Khan, S.U., Hussain, T., Ullah, A. et al. Deep-ReID: deep features and autoencoder assisted image patching strategy for person re-identification in smart cities surveillance. Multimed Tools Appl 83, 15079–15100 (2024). https://doi.org/10.1007/s11042-020-10145-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-10145-8

Keywords

Search

Navigation