Skip to main content
SpringerLink
Log in

Person Re-identification with Spatial Multi-granularity Feature Exploration for Social Risk Situational Assessment

  • Published:
Cognitive Computation Aims and scope Submit manuscript

Abstract

Recently, the “human-oriented” concept of security development has become a consensus among all countries. This depends mainly on intelligent surveillance systems that can support person re-identification (Re-ID) technology to empower social risk situational assessment applications. However, existing Re-ID methods mainly focus on single and fixed convolutional operations for feature extraction, ignoring the multi-dimensional spatial association of the human body, which limits the performance of Re-ID. Human cognition when identifying people does not solely rely on visual cues of the individual in sight, but also on his/her behavioral and gestural characteristics. To solve this issue and inspired by the aforementioned cognitive mechanism of the human brain, this study developed a spatial multi-granularity feature exploration (SMGFE) model for person Re-ID. The proposed SMGFE model comprises two main steps: (i) a multi-granularity feature exploration strategy and (ii) a human spatial association scheme. The former mainly includes coarse (original person images), medium (multi-regional divided person images), and fine-tuned (keypoints of the human body) level features, which form the multi-granularity feature representation. An undirected graph model was then developed to construct multi-dimensional spatial relations for each person. Finally, the unified optimization strategy was applied to train the framework to achieve promising accuracy. We evaluated the proposed algorithm on frequently used and benchmark person Re-ID datasets (Market-1501 and DukeMTMC-reID). The cumulative match curve (CMC) and mean average precision (mAP), which are the common measuring criteria for most person Re-ID methods reported to date, were used to verify the experimental results. Experiments show that our proposed algorithm achieved unrivaled performance levels. In addition, based on the spatial multi-granularity feature exploration strategy, the time efficiency of the proposed method for detecting specific instances can reach O(n), making it suitable for deployment in low-resource terminals for security risk assessment, including Android/iOS analysis servers, urban safety risk surveillance systems, and warning platforms for situational awareness.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Data Availability

All the experiments were exclusively conducted on open-source software (Pytorch). The datasets analyzed during the study are available as public resources on the official websites of the two datasets, namely Market-1501 [45] and DukeMTMC-reID [46].

Notes

  1. https://www.jimay.com/support/, accessed on August 25, 2023.

  2. https://www.jimay.com/support/, accessed on August 25, 2023.

  3. https://www.flir.cn/, accessed on August 25, 2023.

References

  1. D’Aniello G. Fuzzy logic for situation awareness: a systematic review. J Ambient Intell Humaniz Comput. 2023;14(4):4419–38.

    Article  Google Scholar 

  2. Bellman K, Landauer C, Dutt N, Esterle L, Herkersdorf A, Jantsch A, TaheriNejad N, Lewis PR, Platzner M, Tammemäe K. Self-aware cyber-physical systems. ACM Trans Cyber-Phys Syst. 2020;4(4):1–26.

    Article  Google Scholar 

  3. Gong S, Xiang T, Gong S, Xiang T. Person re-identification. Springer. 2011.

  4. Bedagkar-Gala A, Shah SK. A survey of approaches and trends in person re-identification. Image Vis Comput. 2014;32(4):270–86.

    Article  Google Scholar 

  5. Suljagic H, Bayraktar E, Celebi N. Similarity based person re-identification for multi-object tracking using deep Siamese network. Neural Comput Appl. 2022;34(20):18171–82.

    Article  Google Scholar 

  6. Owayjan M, Dergham A, Haber G, Fakih N, Hamoush A, Abdo E. Face recognition security system. In: New Trends in Networking, Computing, E-learning, Systems Sciences, and Engineering. Springer; 2015. p. 343–8.

  7. Zhou W, Lian J, Zhu S, Wu Y, Wang D-H. Vehicle re-identification by separating representative spatial features. Cognit Comput. 2023;1–16.

  8. Sun D, Huang J, Hu L, Tang J, Ding Z. Multitask multigranularity aggregation with global-guided attention for video person re-identification. IEEE Trans Circuits Syst Video Technol. 2022;32(11):7758–71.

    Article  Google Scholar 

  9. Wang G, Yuan Y, Chen X, Li J, Zhou X. Learning discriminative features with multiple granularities for person re-identification. In: International Conference on Multimedia. ACM; 2018. p. 274–82.

  10. Zou G, Fu G, Peng X, Liu Y, Gao M, Liu Z. Person re-identification based on metric learning: a survey. Multimed Tools Appl. 2021;80(17):26855–88.

    Article  Google Scholar 

  11. Liu D, Wu L, Hong R, Ge Z, Shen J, Boussaid F, Bennamoun M. Generative metric learning for adversarially robust open-world person re-identification. ACM Trans Multimed Comput Commun Appl. 2023;19(1):1–19.

    Article  Google Scholar 

  12. Medi PR, Krishna GS, Nemani P, Vollala S, Kumar S. A novel end-to-end framework for occluded pixel reconstruction with spatio-temporal features for improved person re-identification. arXiv:2304.07721 [Preprint]. 2023. Available from: http://arxiv.org/abs/2304.07721.

  13. Lu A, Zhang Z, Huang Y, Zhang Y, Li C, Tang J, Wang L. Illumination distillation framework for nighttime person re-identification and a new benchmark. IEEE Trans Multimed. 2023.

  14. Zhuang Z, Wei L, Xie L, Zhang T, Zhang H, Wu H, Ai H, Tian Q. Rethinking the distribution gap of person re-identification with camera-based batch normalization. In: Proceedings of the European Conference on Computer Vision. Springer; 2020. p. 140–57.

  15. Xiong M, Chen D, Lu X. Mobile person re-identification with a lightweight trident CNN. Sci China Inf Sci. 2020;63:1–3.

    Article  Google Scholar 

  16. D’Aniello G, Gravina R, Gaeta M, Fortino G. Situation awareness in multi-user wearable computing systems. In: International Conference on Cognitive and Computational Aspects of Situation Management. IEEE; 2022. p. 133–8.

  17. Verhulsdonck G, Weible JL, Helser S, Hajduk N. Smart cities, playable cities, and cybersecurity: a systematic review. Int J Human-Comput Interact. 2023;39(2):378–90.

    Article  Google Scholar 

  18. Savastano M, Suciu M-C, Gorelova I, Stativă G-A. How smart is mobility in smart cities? An analysis of citizens’ value perceptions through ICT applications. Cities. 2023;132:1040–71.

    Article  Google Scholar 

  19. Martín A, Fuentes-Hurtado F, Naranjo V, Camacho D. Evolving deep neural networks architectures for android malware classification. In: IEEE Congress on Evolutionary Computation. IEEE; 2017. p. 1659–66.

  20. Zhao S. Graph-based multi-granularity person. In: Proceedings of International Conference on Image, Vision and Intelligent Systems 2022 (ICIVIS 2022), vol. 1019. Springer Nature; 2023. p. 11–31.

  21. Pan H, Chen Y, He Z. Multi-granularity graph pooling for video-based person re-identification. Neural Netw. 2023;160:22–33.

    Article  Google Scholar 

  22. Gong X, Yao Z, Li X, Fan Y, Luo B, Fan J, Lao B. LAG-Net: multi-granularity network for person re-identification via local attention system. IEEE Trans Multimedia. 2021;24:217–29.

    Article  Google Scholar 

  23. Tu M, Zhu K, Guo H, Miao Q, Zhao C, Zhu G, Qiao H, Huang G, Tang M, Wang J. Multi-granularity mutual learning network for object re-identification. IEEE Trans Intell Transp Syst. 2022;23(9):15178–89.

    Article  Google Scholar 

  24. Wang Y, Zhang H, Miao D, Pedrycz W. Multi-granularity re-ranking for visible-infrared person re-identification. CAAI Trans Intell Technol. 2023.

  25. Pan H, Bai Y, He Z, Zhang C. AAGCN: adjacency-aware graph convolutional network for person re-identification. Knowl-Based Syst. 2022;236:107300.

    Article  Google Scholar 

  26. Huang M, Hou C, Yang Q, Wang Z. Reasoning and tuning: graph attention network for occluded person re-identification. IEEE Trans Image Process. 2023;32:1568–82.

    Article  Google Scholar 

  27. Xian Y, Yang J, Yu F, Zhang J, Sun X. Graph-based self-learning for robust person re-identification. In: Proceedings of the IEEE Winter Conference on Applications of Computer Vision. IEEE; 2023. p. 4789–98.

  28. Liu S, Huang S, Fu W, Lin JC-W. A descriptive human visual cognitive strategy using graph neural network for facial expression recognition. Int J Mach Learn Cybern. 2022;1–17.

  29. Alonazi M, Alshahrani HM, Kouki F, Almalki NS, Mahmud A, Majdoubi J. Deep convolutional neural network with symbiotic organism search-based human activity recognition for cognitive health assessment. Biomimetics. 2023;8(7):554.

    Article  Google Scholar 

  30. Cushen GA. A person re-identification system for mobile devices. In: 2015 11th International Conference on Signal-Image Technology & Internet-Based Systems (SITIS). IEEE; 2015. p. 125–31.

  31. Maqsood M, Yasmin S, Gillani S, Bukhari M, Rho S, Yeo S-S. An efficient deep learning-assisted person re-identification solution for intelligent video surveillance in smart cities. Front Comp Sci. 2023;17(4):174329.

    Article  Google Scholar 

  32. Dong C, Zhou J, An Q, Jiang F, Chen S, Pan L, Liu X. Optimizing performance in federated person re-identification through benchmark evaluation for blockchain-integrated smart UAV delivery systems. Drones. 2023;7(7):413.

    Article  Google Scholar 

  33. Andrade RO, Yoo SG. Cognitive security: a comprehensive study of cognitive science in cybersecurity. J Inf Secur Appl. 2019;48:1023–52.

    Google Scholar 

  34. Mostafavi A, Yuan F. Smart flood resilience: harnessing community-scale big data for predictive flood risk monitoring, rapid impact assessment, and situational awareness. In: EGU General Assembly Conference Abstracts. 2022. p. EGU22–781.

  35. Zhong X, Zhang X, Zhang P. Pipeline risk big data intelligent decision-making system based on machine learning and situation awareness. Neural Comput Appl. 2022;34(18):15221–39.

    Article  Google Scholar 

  36. Mirza IB, Georgakopoulos D, Yavari A. Cyber-physical-social awareness platform for comprehensive situation awareness. Sensors. 2023;23(2):08–22.

    Article  Google Scholar 

  37. Grajzl P, Murrell P. A macrohistory of legal evolution and coevolution: property, procedure, and contract in early-modern English caselaw. Int Rev Law Econ. 2023;73:106–13.

    Article  Google Scholar 

  38. Yang T, Zhu S, Chen C, Yan S, Zhang M, Willis A. MutualNet: adaptive ConvNet via mutual learning from network width and resolution. In: Proceedings of the European Conference on Computer Vision. Springer; 2020. p. 299–315.

  39. Li Y, Jia S, Li Q. BalanceHRNet: an effective network for bottom-up human pose estimation. Neural Netw. 2023;161:297–305.

    Article  Google Scholar 

  40. King DE. Dlib-ml: a machine learning toolkit. J Mach Learn Res. 2009;10:1755–8.

    Google Scholar 

  41. Guo Y, Huang J, Xiong M, Wang Z, Hu X, Wang J, Hijji M. Facial expressions recognition with multi-region divided attention networks for smart education cloud applications. Neurocomputing. 2022;493:119–28.

    Article  Google Scholar 

  42. Daverio P, Chaudhry HN, Margara A, Rossi M. Temporal pattern recognition in graph data structures. In: International Conference on Big Data. IEEE; 2021. p. 2753–63.

  43. Wang Z, Baladandayuthapani V, Kaseb AO, Amin HM, Hassan MM, Wang W, Morris JS. Bayesian edge regression in undirected graphical models to characterize interpatient heterogeneity in cancer. J Am Stat Assoc. 2022;117(538):533–46.

    Article  MathSciNet  Google Scholar 

  44. Peng J, Wang P, Zhou N, Zhu J. Partial correlation estimation by joint sparse regression models. J Am Stat Assoc. 2009;104(486):735–46.

    Article  MathSciNet  Google Scholar 

  45. Zheng L, Shen L, Tian L, Wang S, Wang J, Tian Q. Scalable person re-identification: a benchmark. In: Proceedings of the IEEE International Conference on Computer Vision. IEEE; 2015. p. 1116–24.

  46. Guo J, Yuan Y, Huang L, Zhang C, Yao J-G, Han K. Beyond human parts: dual part-aligned representations for person re-identification. In: Proceedings of the International Conference on Computer Vision. IEEE; 2019. p. 3642–51.

  47. Gray D, Brennan S, Tao H. Evaluating appearance models for recognition, reacquisition, and tracking. In: International Workshop on Performance Evaluation for Tracking and Surveillance, vol. 3. IEEE; 2007. p. 1–7.

  48. Recht B, Roelofs R, Schmidt L, Shankar V. Do ImageNet classifiers generalize to ImageNet? In: International Conference on Machine Learning. PMLR; 2019. p. 5389–400.

  49. Fu Y, Wei Y, Zhou Y, Shi H, Huang G, Wang X, Yao Z, Huang T. Horizontal pyramid matching for person re-identification. In: Proceedings of the AAAI conference on artificial intelligence, vol. 33. AAAI; 2019. p. 8295–302.

  50. Zeng K, Ning M, Wang Y, Guo Y. Hierarchical clustering with hard-batch triplet loss for person re-identification. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition. IEEE; 2020. p. 13657–65.

  51. Zhai Y, Lu S, Ye Q, Shan X, Chen J, Ji R, Tian Y. Ad-cluster: Augmented discriminative clustering for domain adaptive person re-identification. In: Proceedings of the Conference on Computer Vision and Pattern Recognition. IEEE. 2020; p. 9021–30.

  52. Quispe R, Pedrini H. Top-DB-Net: top dropblock for activation enhancement in person re-identification. In: Proceedings of the International Conference on Pattern Recognition. IEEE; 2021. p. 2980–7.

  53. Chen Y, Wang H, Sun X, Fan B, Tang C, Zeng H. Deep attention aware feature learning for person re-identification. Patt Recogn. 2022;126:108567.

    Article  Google Scholar 

  54. Ye M, Li H, Du B, Shen J, Shao L, Hoi SCH. Collaborative refining for person re-identification with label noise. IEEE Trans Image Process. 2022;31(2):379–91.

    Article  Google Scholar 

  55. Li J, Zhang S, Tian Q, Wang M, Gao W. Pose-guided representation learning for person re-identification. IEEE Trans Pattern Anal Mach Intell. 2022;44(2):622–35.

    Article  Google Scholar 

  56. Xiang J, Huang Z, Jiang X, Hou J. Similarity learning with deep CRF for person re-identification. Patt Recogn. 2023;135:109151.

    Article  Google Scholar 

  57. Zhou J, Roy SK, Fang P, Harandi M, Petersson L. Cross-correlated attention networks for person re-identification. Image Vis Comput. 2020;100:1031–9.

    Article  Google Scholar 

  58. Travin A, Shur M, Strelets M, Spalart P. Detached-eddy simulations past a circular cylinder. Flow Turbul Combust. 2000;63(1–4):293–313.

    Article  Google Scholar 

  59. Hernández A, Panizo Á, Camacho D. An ensemble algorithm based on deep learning for tuberculosis classification. In: International Conference on Intelligent Data Engineering and Automated Learning. Springer; 2019. p. 145–54.

  60. Horita F, Baptista J, de Albuquerque JP. Exploring the use of IOT data for heightened situational awareness in centralised monitoring control rooms. Inf Syst Front. 2023;25(1):275–90.

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported and funded by the Deanship of Scientific Research at Imam Mohammad Ibn Saud Islamic University (IMSIU) (grant number IMSIU-RP23058).

Author information

Authors and Affiliations

  1. School of Computer Science and Artificial Intelligence, Wuhan Textile University, Wuhan, 430200, China

    Mingfu Xiong

  2. Hubei Technology Exchange, Hubei Provincial Department of Science and Technology, Wuhan, 430064, China

    Hanmei Chen

  3. Department of Information and Technology, Hubei Branch of PICC Property and Casualty Company Limited, Wuhan, 430000, China

    Yi Wen

  4. Information Systems Department, College of Computer and Information Sciences, Imam Mohammad Ibn Saud Islamic University (IMSIU), Riyadh, 11432, Saudi Arabia

    Abdul Khader Jilani Saudagar

  5. TECNALIA, Basque Research & Technology Alliance (BRTA), Derio, 48160, Spain

    Javier Del Ser

  6. Department of Communications Engineering, University of the Basque Country (UPV/EHU), Bilbao, 48013, Spain

    Javier Del Ser

  7. Visual Analytics for Knowledge Laboratory (VIS2KNOW Lab), Department of Applied Artificial Intelligence, School of Convergence, College of Computing and Informatics, Sungkyunkwan University, Seoul, 03063, Republic of Korea

    Khan Muhammad

Authors
  1. Mingfu Xiong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hanmei Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yi Wen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Abdul Khader Jilani Saudagar
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Javier Del Ser
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Khan Muhammad
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Hanmei Chen or Khan Muhammad.

Ethics declarations

Ethical Approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Informed Consent

Informed consent was not required as no humans or animals were involved.

Conflict of Interest

Javier Del Ser is an editorial board member of Cognitive Computation. The authors declare that they have no other conflict of interest regarding this work.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xiong, M., Chen, H., Wen, Y. et al. Person Re-identification with Spatial Multi-granularity Feature Exploration for Social Risk Situational Assessment. Cogn Comput (2024). https://doi.org/10.1007/s12559-024-10249-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s12559-024-10249-5

Keywords

Search

Navigation