Advertisement

Short-Term Person Re-identification Using RGB, Depth and Skeleton Information of RGB-D Sensors

  • Zeynab Imani
  • Hadi SoltanizadehEmail author
  • Ali A. Orouji
Research Paper
  • 9 Downloads

Abstract

Person re-identification is to search for a correct match for a person of interest across different camera views among a large number of impostors. While the approaches based on RGB modality have been widely studied to re-identify people, other modalities could be exploited as additional information sources, like depth and skeleton modalities. In this paper, we perform multi- and single-modal person re-identification using RGB, depth and skeleton modalities obtained by RGB-D sensors. First of all, the depth and RGB images are divided into three regions of head, torso and legs. Then, each region is explained by the histograms of local vector patterns (LVP). In depth and RGB modalities, the depth values of pixels and the gray levels of pixels are used for extracting LVPs, respectively. The skeleton features are extracted by computing the various Euclidean distances for the joint points of skeleton images. Then, features extracted by different modalities are combined as double and triple combinations using score-level fusion. The experiments are evaluated on two RGBD-ID and KinectREID databases, and results illustrate the acceptable performance of the proposed method.

Keywords

Person re-identification LVP Skeleton Score-level fusion 

References

  1. Barbosa BI, Cristani M, Bue AD, Bazzani L, Murino V (2012) Re-identification with RGB-D sensors. In: 1st international workshop on re-identification, pp 433–442Google Scholar
  2. Bazzani L, Cristani M, Perina A, Murino V (2012) Multiple-shot person re-identification by chromatic and epitomic analyses. Pattern Recognit Lett 33:898–903CrossRefGoogle Scholar
  3. Bazzani L, Cristani M, Murino V (2013) Symmetry-driven accumulation of local features for human characterization and re-identification. Comput Vis Image Underst 117:130–144CrossRefGoogle Scholar
  4. Chahla C, Snoussi H, Abdallah F, Dornaika F (2017) Discriminant quaternion local binary pattern embedding for person reidentification through prototype formation and color categorization. Eng Appl Artif Intell 58:27–33CrossRefGoogle Scholar
  5. Doretto G, Sebastian Th, Tu P, Rittscher J (2011) Appearance-based person reidentification in camera networks: problem overview and current approaches. J Ambient Intell Hum Comput 2:127–151CrossRefGoogle Scholar
  6. Fan KCh, Hung TY (2014) A novel local pattern descriptor—local vector pattern in high-order derivative space for face recognition. IEEE Trans Image Process 23:2877–2891MathSciNetCrossRefzbMATHGoogle Scholar
  7. Fang W, Hu HM, Hu Z, Liao Sh, Li B (2017) Perceptual hash-based feature description for person re-identification. Neurocomput.  https://doi.org/10.1016/j.neucom.2017.07.019 Google Scholar
  8. Farenzena M, Bazzani L, Perina A, Murino V, Cristani M (2013) Symmetry-driven accumulation of local features for human characterization and re-identification. Comput Vis Image Underst 117:130–144CrossRefGoogle Scholar
  9. Fathi A, Nilchi AN (2012) Noise tolerant local binary pattern operator for efficient texture analysis. Pattern Recognit Lett 33:1093–1100CrossRefGoogle Scholar
  10. Forssen PE (2007) Maximally stable colour regions for recognition and matching. In: IEEE conference on computer vision and pattern recognition (CVPR), pp 1–8Google Scholar
  11. Gala AB, Shah ShK (2014) A survey of approaches and trends in person re-identification. Image Vis Comput 32:270–286CrossRefGoogle Scholar
  12. Gala AB, Shah ShK (2011) Multiple person re-identification using part based spatio-temporal color appearance model. In: IEEE International conference on computer vision workshops (ICCV Workshops), pp 1721–1728Google Scholar
  13. Guo Z, Zhang L, Zhang D (2010a) A completed modeling of local binary pattern operator for texture classification. IEEE Trans Image Process 19:1657–1663MathSciNetCrossRefzbMATHGoogle Scholar
  14. Guo Z, Zhang L, Zhang D (2010b) Rotation invariant texture classification using LBP variance (LBPV) with global matching. Pattern Recognit 43:706–719CrossRefzbMATHGoogle Scholar
  15. Hadid A (2008) The local binary pattern approach and its applications to face analysis. In: 1st Workshop on image processing theory, tools and applications (IPTA), pp 1–9Google Scholar
  16. Huang X, Li SZ, Wang Y (2004) Shape localization based on statistical method using extended local binary pattern. In: Proceedings of the 3rd international conference on image and graphics (ICIG), pp 184–187Google Scholar
  17. Huang D, Shan C, Ardabilian M, Wang Y, Chen L (2011) Local binary patterns and its application to facial image analysis: a survey. IEEE Trans Syst Man Cybern Part C 41:765–781CrossRefGoogle Scholar
  18. Imani Z, Soltanizadeh H (2016) Person reidentification using local pattern descriptors and anthropometric measures from videos of kinect sensor. IEEE Sensors J 16:6227–6238CrossRefGoogle Scholar
  19. Imani Z, Soltanizadeh H (2018) Histogram of the node strength and histogram of the edge weight: two new features for RGB-D person re-identification. Sci China Inf Sci 61:092108.  https://doi.org/10.1007/s11432-016-9086-8 CrossRefGoogle Scholar
  20. Imani Z, Soltanizadeh H (2019) Local binary pattern, local derivative pattern and skeleton features for RGB-D person re-identification. Natl Acad Sci Lett 42:233–238.  https://doi.org/10.1007/s40009-018-0736-9 CrossRefGoogle Scholar
  21. Jin H, Liu Q, Lu H, Tong X (2004) Face detection using improved LBP under Bayesian framework. In: Proceedings of the 3rd international conference on image and graphics, pp 306–309Google Scholar
  22. Krucki K, Asari V, Borel-Donohue C, Bunker D (2014) Human re-identification in multi-camera systems. In: IEEE applied imagery pattern recognition workshop (AIPR), pp 1–7Google Scholar
  23. Layne R, Hospedales T, Gong S (2012) Person re-identification by attributes. In: Proceedings of the British machine vision conference (BMVC)Google Scholar
  24. Liu C, Gong S, Loy CC, Lin X (2012) Person re-identification: What features are important? In: Proceedings of the 1st international workshop on re-identification, pp 391–401Google Scholar
  25. Liu X, Song M, Tao D, Zhou X, Chen C, Bu J (2014) Semi-supervised coupled dictionary learning for person re-identification. In: IEEE conference on computer vision and pattern recognition (CVPR), pp 3550–3557Google Scholar
  26. Lowe DG (2004) Distinctive image features from scale-invariant key-points. Int J Comput Vis 60:91–110CrossRefGoogle Scholar
  27. Ma B, Su Y, Jurie F (2014) Covariance descriptor based on bio-inspired features for person re-identification and face verification. Image Vis Comput 32:379–390CrossRefGoogle Scholar
  28. Martinel N, Micheloni C (2012) Re-identify people in wide area camera network. In: IEEE conference on computer vision and pattern recognition workshops (CVPRW), RI, USA, pp 31–36Google Scholar
  29. Munaro M, Fossati A, Basso A, Menegatti E, Gool LV (2014) One-shot person re-identification with a consumer depth camera. In: Advances in computer vision and pattern recognition. Springer, Berlin, pp 161–181Google Scholar
  30. Murala S, Wu QMJ (2015) Spherical symmetric 3D local ternary patterns for natural, texture and biomedical image indexing and retrieval. Neurocomput 149:1502–1514CrossRefGoogle Scholar
  31. Murala S, Maheshwari RP, Balasubramanian R (2012) Local tetra patterns: a new feature descriptor for content-based image retrieval. IEEE Trans Image Process 21:2874–2886MathSciNetCrossRefzbMATHGoogle Scholar
  32. Nanni L, Lumini A, Brahnam S (2010) Local binary patterns variants as texture descriptors for medical image analysis. Artif Intell Med 49:117–125CrossRefGoogle Scholar
  33. Nanni L, Brahnam S, Lumini A (2012) A simple method for improving local binary patterns by considering non-uniform patterns. Pattern Recognit 45:3844–3852CrossRefGoogle Scholar
  34. Ojala T, Pietikäinen M, Harwood D (1996) A comparative study of texture measures with classification based on feature distributions. Pattern Recognit 29:51–59CrossRefGoogle Scholar
  35. Oliveira IO, Pio JL (2009) People re-identification in a camera network. In: IEEE international conference on dependable, autonomic and secure computing (DASC), Chengdu, China, pp 461–466Google Scholar
  36. Pala F, Satta R, Fumera G, Roli F (2015) Multi-modal person re-identification using RGB-D cameras. IEEE Trans Circuit Syst Video Tech 26:788–799CrossRefGoogle Scholar
  37. Prosser B, Zheng WS, Gong S, Xiang T (2010) Person re-identification by support vector ranking. In: Proceedings of the British machine vision conference (BMVA), pp 21.1–21.11Google Scholar
  38. Ross A, Jain A (2003) Information fusion in biometrics. Pattern Recognit Let 24:2115–2125CrossRefGoogle Scholar
  39. Schwartz WR, Davis LS (2009) Learning discriminative appearance-based models using partial least squares. In: 22nd Brazilian symposium on computer graphics and image processing (SIBGRAPI), pp 322–329Google Scholar
  40. Sun N, Zheng W, Sun C, Zou C, Zhao L (2006) Gender classification based on boosting local binary pattern. In: Proceedings of 3rd international symposium on neural networks, pp 194–201Google Scholar
  41. Tao D, Guo Y, Song M, Li Y, Yu Zh, Tang YY (2016) Person re-identification by dual-regularized KISS metric learning. IEEE Trans Image Process 25:2726–2738MathSciNetCrossRefzbMATHGoogle Scholar
  42. Vezzani R, Baltieri D, Cucchiara R (2014) People reidentification in surveillance and forensics: a survey. ACM Comput Surveys 46:29Google Scholar
  43. Wu L, Shen C, Hengel A (2017) Deep linear discriminant analysis on Fisher networks: a hybrid architecture for person re-identification. Pattern Recognit 65:238–250CrossRefGoogle Scholar
  44. Xiang ZJ, Chen Q, Liu Y (2014) Person re-identification by fuzzy space color histogram. Multimed Tools Appl 73:91–107CrossRefGoogle Scholar
  45. Xiong F, Gou M, Camps O, Sznaier M (2014) Person re-identification using kernel-based metric learning methods. In: 13th European conference on computer vision (ECCV), pp 1–16Google Scholar
  46. Yang B, Chen S (2013) A comparative study on local binary pattern (LBP) based face recognition: LBP histogram versus LBP image. Neurocomput 120:365–379CrossRefGoogle Scholar
  47. Zhang Y, Li S (2011) Gabor-lbp based region covariance descriptor for person re-identification. In: Sixth International Conference on Image and Graphics (ICIG), pp 368–371Google Scholar
  48. Zhang B, Gao Y, Zhao S, Liu J (2010) Local derivative pattern versus local binary pattern: face recognition with high order local pattern descriptor. IEEE Trans Image Process 19:533–544MathSciNetCrossRefzbMATHGoogle Scholar
  49. Zhao R, Ouyang W, Wang X (2013) Unsupervised salience learning for person re-identification. In: IEEE conference on computer vision and pattern recognition (CVPR), pp 3586–3593Google Scholar
  50. Zhao Y, Zhao X, Luo R, Liu Y (2015) Person Re-identification by encoding free energy feature maps. Multimed Tools Appl 75:4795–4813CrossRefGoogle Scholar
  51. Zheng W, Gong S, Xiang T (2011) Person re-identification by probabilistic relative distance comparison. In: IEEE Conference on computer vision and pattern recognition (CVPR), pp 649–656Google Scholar
  52. Zheng W, Gong S, Xiang T (2013) Reidentification by relative distance comparison. IEEE Trans Pattern Anal Mach Intell 35:653–668CrossRefGoogle Scholar

Copyright information

© Shiraz University 2019

Authors and Affiliations

  • Zeynab Imani
    • 1
  • Hadi Soltanizadeh
    • 1
    Email author
  • Ali A. Orouji
    • 1
  1. 1.Electrical and Computer Engineering FacultySemnanIran

Personalised recommendations