Illumination and scale invariant relevant visual features with hypergraph-based learning for multi-shot person re-identification

  • Aparajita Nanda
  • Dushyant Singh Chauhan
  • Pankaj K. Sa
  • Sambit Bakshi
Article
  • 125 Downloads

Abstract

Person re-identification which aims at matching people across disjoint cameras has received increasing attention due to the widespread use of video surveillance applications. Existing methods concentrate either on robust feature extraction or view-invariant feature transformation. However, the extracted features suffer from various limitations such as color inconsistency and scale variations. Besides, during matching, a probe is compared against each gallery instance which represents only the pairwise relationship and ignores the high order relationship among them. To address these issues, we propose a multi-shot person re-identification framework that first performs a preprocessing task on images to address illumination variations for maintaining the color consistency. Subsequently, we formulate an approach to handle scale variations in the pedestrian appearances for keeping them with relatively a fixed scale ratio. Overlapped visual patches representing appearance cues are then extracted from the processed images. A structured multi-class feature selection approach is employed to select a set of relevant patches that simultaneously discriminates all distinct persons. These selected patches use a hypergraph to represent the visual association among a probe and gallery images. Finally, for matching, we formulate a hypergraph-based learning scheme, which considers both the pairwise and high-order association among the probe and gallery images. The hypergraph structure is then optimized to yield an improved similarity score for a probe against each gallery instance. The effectiveness of our proposed framework is validated on three public datasets and comparison with state-of-the-art methods shows the superior performance of our framework.

Keywords

Video surveillance Person re-identification Illumination variations Scale variations Multi-camera Multi-class group LASSO Hypergraph learning 

Notes

Acknowledgment

This work is supported by Grant Number SB/FTP/ETA-0059/2014 by Science and Engineering Research Board (SERB), Department of Science & Technology, Government of India.

References

  1. 1.
    An L, Chen X, Yang S (2016) Person re-identification via hypergraph-based matching. Neurocomputing 182:247–254CrossRefGoogle Scholar
  2. 2.
    Avraham T, Gurvich I, Lindenbaum M, Markovitch S (2012) Learning implicit transfer for person re-identification European Conference on Computer Vision (ECCV). Springer, Berlin, pp 381–390, DOI  10.1007/978-3-642-33863-2_38, (to appear in print)
  3. 3.
    Bak S, Corvee E, Bremond F, Thonnat M (2012) Boosted human re-identification using Riemannian manifolds. Image Vis Comput 30(6):443–452. doi: 10.1016/j.imavis.2011.08.008 CrossRefGoogle Scholar
  4. 4.
    Bazzani L, Cristani M, Perina A, Murino V (2012) Multiple-shot person re-identification by chromatic and Epitomic analyses. Pattern Recogn Lett 33(7):898–903CrossRefGoogle Scholar
  5. 5.
    Cheng DS, Cristani M, Stoppa M, Bazzani L, Murino V (2011) Custom pictorial structures for re-identification British Machine Vision Conference (BMVC), vol 1, p 6, DOI  10.5244/C25.68
  6. 6.
    Corvee E, Bremond F, Thonnat M et al (2010) Person re-identification using spatial covariance regions of human body parts Advanced Video and Signal based Surveillance (AVSS). IEEE, pp 435–440, DOI  10.1109/AVSS.2010.34, (to appear in print)
  7. 7.
    Cui J, Liu Y, Xu Y, Zhao H, Zha H (2013) Tracking generic human motion via fusion of low-and high-dimensional approaches. IEEE Trans Syst Man Cybern Syst 43(4):996–1002CrossRefGoogle Scholar
  8. 8.
    da Vinci L The Da Vinci Notebooks, pp 1–224. Profile. DOI ISBN 1-86197-987-8Google Scholar
  9. 9.
    Farenzena M, Bazzani L, Perina A, Murino V, Cristani M (2010) Person re-identification by symmetry-driven accumulation of local features Computer Vision and Pattern Recognition (CVPR). IEEE, pp 2360–2367, DOI  10.1109/CVPR.2010.5539926, (to appear in print)
  10. 10.
    Gao Y, Wang M, Tao D, Ji R, Dai Q (2012) 3-D object retrieval and recognition with hypergraph analysis. IEEE Trans Image Process 21(9):4290–4303MathSciNetCrossRefGoogle Scholar
  11. 11.
    Jobson DJ, Rahman Zu, Woodell GA (1997) A multiscale retinex for bridging the gap between color images and the human observation of scenes. IEEE Trans Image Process 6(7):965–976CrossRefGoogle Scholar
  12. 12.
    Jobson DJ, Rahman Zu, Woodell GA (1997) Properties and performance of a center/surround retinex. IEEE Trans Image Process 6(3):451–462CrossRefGoogle Scholar
  13. 13.
    Jojic N, Perina A, Cristani M, Murino V, Frey B (2009) Stel component analysis: Modeling spatial correlations in image class structure Computer Vision and Pattern Recognition (CVPR). IEEE, pp 2044–2051, DOI  10.1109/CVPRW.2009.5206581, (to appear in print)
  14. 14.
    Karanam S, Li Y, Radke RJ (2015) Person re-identification with discriminatively trained viewpoint invariant dictionaries International Conference on Computer Vision (ICCV). IEEE, pp 4516–4524, DOI  10.1109/ICCV.2015.513, (to appear in print)
  15. 15.
    Karanam S, Li Y, Radke RJ (2015) Sparse re-id: Block sparsity for person re-identification Computer Vision and Pattern Recognition Workshops. IEEE, pp 33–40, DOI doi: 10.1109/CVPRW.2015.7301392, (to appear in print)
  16. 16.
    Koestinger M, Hirzer M, Wohlhart P, Roth PM, Bischof H (2012) Large scale metric learning from equivalence constraints Computer Vision and Pattern Recognition (CVPR). IEEE, pp 2288–2295, DOI  10.1109/CVPR.2012.6247939, (to appear in print)
  17. 17.
    Kviatkovsky I, Adam A, Rivlin E (2013) Color invariants for person reidentification. Transactions on Pattern Analysis and Machine Intelligence 35 (7):1622–1634. doi: 10.1109/TPAMI.2012.246 CrossRefGoogle Scholar
  18. 18.
    Li Y, Wu Z, Radke RJ (2015) Multi-shot re-identification with random-projection-based random forests Applications of Computer Vision. IEEE, pp 373–380, DOI  10.1109/WACV.2015.56, (to appear in print)
  19. 19.
    Li Z, Chang S, Liang F, Huang T, Cao L, Smith J (2013) Learning locally-adaptive decision functions for person verification Computer Vision and Pattern Recognition (CVPR). IEEE, pp 3610–3617, DOI  10.1109/CVPR.2013.463, (to appear in print)
  20. 20.
    Liao S, Hu Y, Zhu X, Li SZ (2015) Person re-identification by local maximal occurrence representation and metric learning Computer Vision and Pattern Recognition (CVPR). IEEE, pp 2197–2206, DOI  10.1109/CVPR.2015.7298832, (to appear in print)
  21. 21.
    Liu L, Cheng L, Liu Y, Jia Y, Rosenblum DS (2016) Recognizing complex activities by a probabilistic interval-based model AAAI, vol 30, pp 1266–1272Google Scholar
  22. 22.
    Liu Y, Cui J, Zhao H, Zha H (2012) Fusion of low-and high-dimensional approaches by trackers sampling for generic human motion tracking International Conference on Pattern Recognition (ICPR). IEEE, pp 898–901Google Scholar
  23. 23.
    Liu Y, Nie L, Han L, Zhang L, Rosenblum DS (2015) Action2activity: Recognizing complex activities from sensor data Conference on Artificial IntelligenceGoogle Scholar
  24. 24.
    Liu Y, Nie L, Liu L, Rosenblum DS (2016) From action to activity: Sensor-based activity recognition. Neurocomputing 181:108–115CrossRefGoogle Scholar
  25. 25.
    Lu Y, Wei Y, Liu L, Zhong J, Sun L, Liu Y (2016) Towards unsupervised physical activity recognition using smartphone accelerometers, Multimedia Tools and Applications 1–19Google Scholar
  26. 26.
    Ma B, Su Y, Jurie F (2012) Bicov: a novel image representation for person re-identification and face verification British Machive Vision Conference (BMVC), p 11, DOI  10.5244/C.26.57, (to appear in print)
  27. 27.
    Ma B, Su Y, Jurie F (2012) Local descriptors encoded by Fisher vectors for person re-identification European Conference on Computer Vision (ECCV). Springer, Berlin, pp 413–422, DOI  10.1007/978-3-642-33863-2_41, (to appear in print)
  28. 28.
    Ma B, Su Y, Jurie F (2014) Covariance descriptor based on bio-inspired features for person re-identification and face verification. Image Vis Comput 32(6):379–390. doi: 10.1016/j.imavis.2014.04.002 CrossRefGoogle Scholar
  29. 29.
    Martinel N, Micheloni C (2015) Classification of local Eigen-dissimilarities for person re-identification. IEEE Signal Process Lett 22(4):455–459. doi: 10.1109/LSP.2014.2362573 CrossRefGoogle Scholar
  30. 30.
    Martinel N, Micheloni C, Foresti GL (2014) Saliency weighted features for person re-identification European Conference on Computer Vision (ECCV). Springer, Berlin, pp 191–208, DOI  10.1007/978-3-319-16199-0_14, (to appear in print)
  31. 31.
    Ojala T, Pietikainen M, Maenpaa T (2002) Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans Pattern Anal Mach Intell 24(7):971–987CrossRefMATHGoogle Scholar
  32. 32.
    Pedagadi S, Orwell J, Velastin S, Boghossian B (2013) Local Fisher discriminant analysis for pedestrian re-identification Computer Vision and Pattern Recognition (CVPR). IEEE, pp 3318–3325, DOI  10.1109/CVPR.2013.426, (to appear in print)
  33. 33.
    Schwartz WR, Davis LS (2009) Learning discriminative appearance-based models using partial least squares XXII Brazilian Symposium on Computer Graphics and Image Processing (SIBGRAPI). IEEE, pp 322–329, DOI  10.1109/SIBGRAPI.2009.42, (to appear in print)
  34. 34.
    Tao D, Jin L, Wang Y, Li X (2015) Person reidentification by minimum classification error-based kiss metric learning. IEEE Trans Cybern 45(2):242–252CrossRefGoogle Scholar
  35. 35.
    Van De Weijer J, Schmid C (2006) Coloring local feature extraction European Conference on Computer Vision (ECCV). Springer, Berlin, pp 334–348Google Scholar
  36. 36.
    Vezzani R, Baltieri D, Cucchiara R (2013) People reidentification in surveillance and forensics: a survey. ACM Comput Surv (CSUR) 46(2):29CrossRefGoogle Scholar
  37. 37.
    Wang T, Gong S, Zhu X, Wang S (2014) Person re-identification by video ranking European Conference on Computer Vision (ECCV). Springer, Berlin, pp 688–703, DOI  10.1007/978-3-319-10593-2_45, (to appear in print)
  38. 38.
    Wu Y, Minoh M, Mukunoki M (2013) Collaboratively regularized nearest points for set based recognition British Machine Vision Conference (BMVC), vol 2, p 5Google Scholar
  39. 39.
    Wu Y, Minoh M, Mukunoki M, Lao S (2012) Set based discriminative ranking for recognition European Conference on Computer Vision (ECCV). Springer, Berlin, pp 497–510, DOI  10.1007/978-3-642-33712-3_36, (to appear in print)
  40. 40.
    Wu Y, Mukunoki M, Minoh M (2013) Locality-constrained collaborative sparse approximation for multiple-shot person re-identification Asian Conference on Pattern Recognition. IEEE, pp 140–144Google Scholar
  41. 41.
    Wu Z, Li Y, Radke RJ (2015) Viewpoint invariant human re-identification in camera networks using pose priors and subject-discriminative features. IEEE Trans Pattern Anal Mach Intell 37(5):1095–1108. doi: 10.1109/TPAMI.2014.2360373 CrossRefGoogle Scholar
  42. 42.
    Xie Y, Yu H, Gong X, Dong Z, Gao Y (2015) Learning visual-spatial saliency for multiple-shot person re-identification. Signal Process Lett 22(11):1854–1858. doi: 10.1109/LSP.2015.2440294 CrossRefGoogle Scholar
  43. 43.
    Xiong F, Gou M, Camps O, Sznaier M (2014) Person re-identification using kernel-based metric learning methods European Conference on Computer Vision (ECCV). Springer, Berlin, pp 1–16, DOI  10.1007/978-3-319-10584-0_1, (to appear in print)
  44. 44.
    Zhang G, Wang Y, Kato J, Marutani T, Mase K (2012) Local distance comparison for multiple-shot people re-identification Asian Conference on Computer Vision (ACCV). Springer, Berlin, pp 677–690, DOI  10.1007/978-3-642-37431-9_52, (to appear in print)
  45. 45.
    Zhao R, Ouyang W, Wang X (2013) Person re-identification by salience matching International Conference on Computer Vision (ICCV). IEEE, pp 2528–2535, DOI  10.1109/ICCV.2013.314, (to appear in print)
  46. 46.
    Zhao R, Ouyang W, Wang X (2013) Unsupervised salience learning for person re-identification Computer Vision and Pattern Recognition (CVPR). IEEE, pp 3586–3593, DOI  10.1109/CVPR.2013.460, (to appear in print)
  47. 47.
    Zheng WS, Gong S, Xiang T (2011) Person re-identification by probabilistic relative distance comparison Computer Vision and Pattern Recognition (CVPR). IEEE, pp 649–656, DOI  10.1109/CVPR.2011.5995598, (to appear in print)
  48. 48.
    Zheng WS, Gong S, Xiang T (2013) Reidentification by relative distance comparison. IEEE Trans Pattern Anal Mach Intell 35(3):653–668. doi: 10.1109/TPAMI.2012.138 CrossRefGoogle Scholar
  49. 49.
    Zhou D, Huang J, Schölkopf B (2006) Learning with hypergraphs: clustering, classification, and embedding NIPS, vol 19, pp 1633–1640Google Scholar
  50. 50.
    Zini L, Noceti N, Fusco G, Odone F (2015) Structured multi-class feature selection with an application to face recognition. Pattern Recogn Lett 55:35–41CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2017

Authors and Affiliations

  • Aparajita Nanda
    • 1
  • Dushyant Singh Chauhan
    • 1
  • Pankaj K. Sa
    • 1
  • Sambit Bakshi
    • 1
  1. 1.Department of Computer Science & EngineeringNational Institute of TechnologyRourkelaIndia

Personalised recommendations