Skip to main content

Advertisement

SpringerLink
Log in

People search based on attributes description provided by an eyewitness for video surveillance applications

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

People search based on attributes description presents a paramount task for several forensics and surveillance applications. The aim is to locate a suspect or to find a missing person in public areas. However, semantic attributes provide a natural interface for this system as they present human understandable properties. These features can cover the whole body characteristics by describing the worn bags, carried objects, clothes, accessories, etc. Detecting semantic attributes under uncontrolled acquisition conditions still remains a challenging task. Most of state-of-the-art approaches assume independence among attributes where each attribute classifier is trained independently based on low-level features extracted from training samples. In this paper, we propose a novel people search system based on attributes description that relies on several components. An interactive query verification algorithm is introduced to prevent search failure. In addition, an attribute classification method that relies on two steps is introduced. We start by selecting the most relevant features in attribute adaptive way. Then, we explored the interactions among attributes to predict a semantic trait by involving the independent attribute classifier and the other correlated attribute classifiers. Several experiments were conducted to validate the effectiveness of the proposed people search system on the challenging VIPeR, CUHK, and HDA+ datasets benchmark.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19

Similar content being viewed by others

References

  1. Adjeroh D, Cao D, Piccirilli M, Ross A (2010) Predictability and correlation in human metrology. In: 2010 IEEE international workshop on information forensics and security, pp 1–6. https://doi.org/10.1109/WIFS.2010.5711470

  2. Agrawal R, Srikant R (1994) Fast algorithms for mining association rules in large databases. In: Proceedings of the 20th international conference on very large data bases, VLDB ’94. Morgan Kaufmann Publishers Inc., San Francisco, pp 487–499. http://dl.acm.org/citation.cfm?id=645920.672836

  3. Almudhahka NY, Nixon MS, Hare JS (2016) Unconstrained human identification using comparative facial soft biometrics. In: 2016 IEEE 8th international conference on biometrics theory, applications and systems (BTAS). IEEE, pp 1–6

  4. An L, Chen X, Kafai M, Yang S, Bhanu B (2013) Improving person re-identification by soft biometrics based reranking. In: 2013 seventh international conference on distributed smart cameras (ICDSC), pp 1–6. https://doi.org/10.1109/ICDSC.2013.6778216

  5. Antol S, Zitnick CL, Parikh D (2014) Zero-shot learning via visual abstraction. Springer International Publishing, Cham, pp 401–416. https://doi.org/10.1007/978-3-319-10593-2_27

    Google Scholar 

  6. Antonie ML, Zaïane OR (2004) Mining positive and negative association rules: an approach for confined rules. In: European conference on principles of data mining and knowledge discovery. Springer, pp 27–38

  7. Breiman L (2001) Random forests. Mach Learn 45 (1):5–32. https://doi.org/10.1023/A:1010933404324

    Article  MATH  Google Scholar 

  8. Caruana R, Niculescu-Mizil A (2006) An empirical comparison of supervised learning algorithms. In: Proceedings of the 23rd international conference on machine learning. ACM, pp 161–168

  9. Chen BC, Chen YY, Kuo YH, Hsu WH (2013) Scalable face image retrieval using attribute-enhanced sparse codewords. IEEE Trans Multimedia 15(5):1163–1173. https://doi.org/10.1109/TMM.2013.2242460

    Article  Google Scholar 

  10. Cybenko G (1989) Approximation by superpositions of a sigmoidal function. Math Control Signals Syst 2(4):303–314. https://doi.org/10.1007/BF02551274

    Article  MathSciNet  MATH  Google Scholar 

  11. Demšar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7:1–30

  12. Denman S, Halstead M, Fookes C, Sridharan S (2017) Locating people in surveillance video using soft biometric traits. In: Handbook of biometrics for forensic science. Springer, pp 267–288

  13. Dollar P, Wojek C, Schiele B, Perona P (2012) Pedestrian detection: an evaluation of the state of the art. IEEE Trans Pattern Anal Mach Intell 34(4):743–761

    Article  Google Scholar 

  14. Farhadi A, Endres I, Hoiem D, Forsyth D (2009) Describing objects by their attributes. In: IEEE conference on computer vision and pattern recognition, 2009. CVPR 2009, pp 1778–1785. https://doi.org/10.1109/CVPR.2009.5206772

  15. Feris R, Bobbitt R, Brown L, Pankanti S (2014) Attribute-based people search: lessons learnt from a practical surveillance system. In: Proceedings of international conference on multimedia retrieval, ICMR ’14. ACM, New York, pp 153:153–153:160. https://doi.org/10.1145/2578726.2578732

  16. Figueira D, Taiana M, Nambiar A, Nascimento J, Bernardino A (2014) The hda + data set for research on fully automated re-identification systems. In: European conference on computer vision. Springer, pp 241–255

  17. Frikha M, Fendri E, Hammami M (2014) A new appearance signature for real time person re-identification. Springer International Publishing, Cham, pp 175–182. https://doi.org/10.1007/978-3-319-10840-7_22

    Google Scholar 

  18. Gray D, Brennan S, Tao H (2007) Evaluating appearance models for recognition, reacquisition, and tracking. In: Proceedings of IEEE international workshop on performance evaluation for tracking and surveillance (PETS), vol 3. Citeseer

  19. Han J, Pauwels EJ, de Zeeuw PM, de With PH (2012) Employing a rgb-d sensor for real-time tracking of humans across multiple re-entries in a smart environment. IEEE Trans Consum Electron 58(2):255–263

    Article  Google Scholar 

  20. https://www.youtube.com/watch?v=eha_KqdSvCI. Online. Accessed 20 April 2016

  21. http://www.bbc.com/news/magazine-22191033. Online. Accessed 19 Sept 2015

  22. Jain AK, Farrokhnia F (1991) Unsupervised texture segmentation using gabor filters. Pattern Recogn 24(12):1167–1186. https://doi.org/10.1016/0031-3203(91)90143-S. http://www.sciencedirect.com/science/article/pii/003132039190143S

    Article  Google Scholar 

  23. Kalantidis Y, Kennedy L, Li LJ (2013) Getting the look: clothing recognition and segmentation for automatic product suggestions in everyday photos. In: Proceedings of the 3rd ACM conference on international conference on multimedia retrieval, ICMR ’13. ACM, New York, pp 105–112. https://doi.org/10.1145/2461466.2461485

  24. Kittler J, Hatef M, Duin RP, Matas J (1998) On combining classifiers. IEEE Trans Pattern Anal Mach Intell 20(3):226–239

    Article  Google Scholar 

  25. Kovashka A, Parikh D, Grauman K (2015) Whittlesearch: interactive image search with relative attribute feedback. Int J Comput Vis 115(2):185–210. https://doi.org/10.1007/s11263-015-0814-0

    Article  MathSciNet  Google Scholar 

  26. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp 1097–1105

  27. Kumar N, Berg A, Belhumeur PN, Nayar S (2011) Describable visual attributes for face verification and image search. IEEE Trans Pattern Anal Mach Intell 33(10):1962–1977. https://doi.org/10.1109/TPAMI.2011.48

    Article  Google Scholar 

  28. Lampert CH, Nickisch H, Harmeling S (2009) Learning to detect unseen object classes by between-class attribute transfer. In: IEEE conference on computer vision and pattern recognition, 2009. CVPR 2009, pp 951–958. https://doi.org/10.1109/CVPR.2009.5206594

  29. Lampert CH, Nickisch H, Harmeling S (2014) Attribute-based classification for zero-shot visual object categorization. IEEE Trans Pattern Anal Mach Intell 36 (3):453–465. https://doi.org/10.1109/TPAMI.2013.140

    Article  Google Scholar 

  30. Layne R, Hospedales TM, Gong S, Mary Q (2012) Person re-identification by attributes. In: BMVC, vol 2, p 8

  31. Layne R, Hospedales TM, Gong S (2014) Attributes-based re-identification. Springer London, London, pp 93–117. https://doi.org/10.1007/978-1-4471-6296-4_5

    Google Scholar 

  32. Li W, Wang X (2013) Locally aligned feature transforms across views. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3594–3601

  33. Liao S, Hu Y, Zhu X, Li SZ (2015) Person re-identification by local maximal occurrence representation and metric learning. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2197–2206

  34. Liu J, Liang C, Ye M, Wang Z, Yang Y, Han Z, Sun K (2015) Person re-identification via attribute confidence and saliency. Springer International Publishing, Cham, pp 591–600. https://doi.org/10.1007/978-3-319-24075-6_57

    Google Scholar 

  35. Lumini A, Nanni L (2017) Overview of the combination of biometric matchers. Information Fusion 33:71–85

    Article  Google Scholar 

  36. Martinel N, Micheloni C, Foresti GL (2016) A pool of multiple person re-identification experts. Pattern Recogn Lett 71:23–30

    Article  Google Scholar 

  37. Martínez AM, Kak AC (2001) Pca versus lda. IEEE Trans Pattern Anal Mach Intell 23(2):228–233

    Article  Google Scholar 

  38. Nambiar A, Bernardino A, Nascimento J (2015) Shape context for soft biometrics in person re-identification and database retrieval. Pattern Recogn Lett 68:297–305

    Article  Google Scholar 

  39. Nguyen NB, Nguyen VH, Duc TN, Le DD, Duong DA (2015) AttRel: an approach to person re-identification by exploiting attribute relationships. Springer International Publishing, Cham, pp 50–60. https://doi.org/10.1007/978-3-319-14442-9_5

    Google Scholar 

  40. Odone F, Barla A, Verri A (2005) Building kernels from binary strings for image matching. IEEE Trans Image Process 14(2):169–180

    Article  MathSciNet  Google Scholar 

  41. Provost F, Fawcett T (2001) Robust classification for imprecise environments. Mach Learn 42(3):203–231

    Article  Google Scholar 

  42. Saghafi MA (2014) Review of person re-identification techniques. IET Comput Vis 8:455–474(19). http://digital-library.theiet.org/content/journals/10.1049/iet-cvi.2013.0180

    Article  Google Scholar 

  43. Schmid C (2001) Constructing models for content-based image retrieval. In: Proceedings of the 2001 IEEE computer society conference on computer vision and pattern recognition, 2001. CVPR 2001, vol 2, pp II–39–II–45. https://doi.org/10.1109/CVPR.2001.990922

  44. Shih FY (2010) Image processing and pattern recognition: fundamentals and techniques. Wiley, New York

    Book  Google Scholar 

  45. Swain MJ, Ballard DH (1991) Color indexing. Int J Comput Vis 7(1):11–32. https://doi.org/10.1007/BF00130487

    Article  Google Scholar 

  46. Tan S, Zheng F, Liu L, Han J, Shao L (2016) Dense invariant feature based support vector ranking for cross-camera person re-identification. IEEE Trans Circuits Syst Video Technol 28:356–363

    Article  Google Scholar 

  47. Umeda T, Sun Y, Irie G, Sudo K, Kinebuchi T (2016) Attribute discovery for person re-identification. Springer International Publishing, Cham, pp 268–276. https://doi.org/10.1007/978-3-319-27674-8_24

    Google Scholar 

  48. Vaquero DA, Feris RS, Tran D, Brown L, Hampapur A, Turk M (2009) Attribute-based people search in surveillance environments. In: 2009 workshop on applications of computer vision (WACV), pp 1–8. https://doi.org/10.1109/WACV.2009.5403131

  49. Vezzani R, Baltieri D, Cucchiara R (2013) People reidentification in surveillance and forensics: a survey. ACM Comput Surv 46(2):29:1–29:37. https://doi.org/10.1145/2543581.2543596

    Article  Google Scholar 

  50. Wang X (2013) Intelligent multi-camera video surveillance: a review. Pattern Recogn Lett 34(1):3–19. https://doi.org/10.1016/j.patrec.2012.07.005. http://www.sciencedirect.com/science/article/pii/S016786551200219X. Extracting Semantics from Multi-Spectrum Video

    Article  Google Scholar 

  51. Ye M, Liang C, Wang Z, Leng Q, Chen J, Liu J (2015) Specific person retrieval via incomplete text description. In: Proceedings of the 5th ACM on international conference on multimedia retrieval, ICMR ’15. ACM, New York, pp 547–550. https://doi.org/10.1145/2671188.2749347

  52. Zhu J, Liao S, Lei Z, Li SZ (2015) Improve pedestrian attribute classification by weighted interactions from other attributes. Springer International Publishing, Cham, pp 545–557. https://doi.org/10.1007/978-3-319-16634-6_40

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. MIRACL-Faculty of Economics and Management, Sfax University, Sfax, Tunisia

    Mayssa Frikha

  2. MIRACL-Faculty of Sciences, Sfax University, Sfax, Tunisia

    Emna Fendri & Mohamed Hammami

Authors
  1. Mayssa Frikha
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Emna Fendri
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mohamed Hammami
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mayssa Frikha.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Frikha, M., Fendri, E. & Hammami, M. People search based on attributes description provided by an eyewitness for video surveillance applications. Multimed Tools Appl 78, 2045–2072 (2019). https://doi.org/10.1007/s11042-018-6245-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-018-6245-5

Keywords

Search

Navigation