Skip to main content
SpringerLink
Log in

A novel visible spectrum images-based pedestrian detection and tracking system for surveillance in non-controlled environments

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

For many vision applications, robust detection and tracking of pedestrians in image sequences are essential. In this paper, a hybrid system for pedestrian detection and tracking is presented. The proposed method is achieved in four major stages. First, the given image is segmented by exploiting the motion information to generate regions of interests where pedestrians are likely to exist. In the second stage, the ROIs are subjected to a selection process in order to keep only significant ones. Then, a hybrid feature is defined to classify the ROIs generated from the previous step. It entails combining the so-called GLBP-Color together with the histograms of oriented optical flow (HOOF). Artificial neural networks (ANN), Adaboost and support vector machine (SVM) classifiers have been tested together with the introduced hybrid feature and the first one is adopted as it outperforms the other two. The last step performs the tracking of the detected pedestrians using the so-called Color-BMA, which is an extension of the classical block matching (BMA) algorithm to the RGB color space. The proposed method has been tested in non-controlled environments with a collection of common databasets that are well known in the surveillance research community (CAVIAR, PETS 2006 and PETS 2009). The obtained results are satisfactory when compared to the recent state of the art approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

Similar content being viewed by others

References

  1. Akhloufi MA, Porcher C, Bendada A (2013) Fusion of thermal infrared and visible spectrum images for robust pedestrian tracking. In: Proceedings of SPIE, vol 9076

  2. Arora M, Kumar M, Garg NK (2018) Facial emotion recognition system based on pca and gradient features. Natl Acad Sci Lett 41(6):365–368

    Article  Google Scholar 

  3. Banerji S, Verma A, Liu C (2012) Lbp and color descriptors for image classification. In: Cross Disciplinary Biometric Systems. Springer, pp 205–225

  4. Bansal M, Kumar M, Kumar M (2020) 2d object recognition techniques: state-of-the-art work. Arch Comput Methods Eng:1–15

  5. Barnich O, Van Droogenbroeck M (2010) Vibe: A universal background subtraction algorithm for video sequences. IEEE Trans Image process 20(6):1709–1724

    Article  MathSciNet  Google Scholar 

  6. Bay H, Ess A, Tuytelaars T, Van Gool L (2008) Speeded-up robust features (surf). Comput Vis Image Understand 110(3):346–359

    Article  Google Scholar 

  7. Bertozzi M, Broggi A, Fascioli A, Tibaldi A, Chapuis R, Chausse F (2004) Pedestrian localization and tracking system with kalman filtering. In: IEEE Intelligent Vehicles Symposium, 2004. IEEE, pp 584–589

  8. Bianco S, Mazzini D, Pau DP, Schettini R (2015) Local detectors and compact descriptors for visual search: a quantitative comparison. Digital Signal Process 44:1–13

    Article  Google Scholar 

  9. Broggi A, Fascioli A, Fedriga I, Tibaldi A, Rose MD (2003) Stereo-based preprocessing for human shape localization in unstructured environments. In: Proceedings of IEEE Intelligent Vehicles Symp, pp 410–415

  10. Choudhury SK, Sa PK, Padhy RP, Sharma S, Bakshi S (2018) Improved pedestrian detection using motion segmentation and silhouette orientation. Multimed Tools Appl 77(11):13075–13114

    Article  Google Scholar 

  11. Conde C, Moctezuma D, De Diego IM, Cabello E (2013) Hogg: Gabor and hog-based human detection for surveillance in non-controlled environments. Neurocomputing 100:19–30

    Article  Google Scholar 

  12. Corvee E, Bremond F, Thonnat M, et al. (2010) Person re-identification using haar-based and dcd-based signature. In: 2010 7th IEEE International Conference on Advanced Video and Signal Based Surveillance. IEEE, pp 1–8

  13. Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: Computer Vision and Pattern Recognition, 2005. CVPR 2005. IEEE Computer Society Conference on, vol 1. IEEE, pp 886–893

  14. Dash PP, Patra D (2019) Efficient visual tracking using multi-feature regularized robust sparse coding and quantum particle filter based localization. J Ambient Intell Human Comput 10(2):449–462

    Article  Google Scholar 

  15. Daugman JG (1985) Uncertainty relation for resolution in space, spatial frequency, and orientation optimized by two-dimensional visual cortical filters. JOSA A 2(7):1160–1169

    Article  Google Scholar 

  16. El Ansari M, Lahmyed R, Trémeau A (2018) A hybrid pedestrian detection system based on visible images and lidar data.. In: VISIGRAPP (5: VISAPP), pp 325–334

  17. Ellahyani A, El Ansari M, El Jaafari I (2016) Traffic sign detection and recognition based on random forests. Appl Soft Comput 46:805–815

    Article  Google Scholar 

  18. Espinace P, Kollar T, Roy N, Soto A (2013) Indoor scene recognition by a mobile robot through adaptive object detection. Robot Auton Syst 61(9):932–947

    Article  Google Scholar 

  19. Freire-Obregón D, Castrillón-Santana M, Barra P, Bisogni C, Nappi M (2020) An attention recurrent model for human cooperation detection. Comput Vis Image Underst 197:102991

    Article  Google Scholar 

  20. Freund Y, Schapire R, Abe N (1999) A short introduction to boosting. J-Japan Soc Artif Intell 14(771-780):1612

    Google Scholar 

  21. Gavrila DM (2000) Pedestrian detection from a moving vehicle. In: European conference on computer vision. Springer, pp 37–49

  22. Gupta S, Kumar M (2020) Forensic document examination system using boosting and bagging methodologies. Soft Comput 24(7):5409–5426

    Article  Google Scholar 

  23. Jain AK, Mao J, Mohiuddin KM (1996) Artificial neural networks: A tutorial. Computer 29(3):31–44

    Article  Google Scholar 

  24. Jiang N, Xu J, Goto S (2012) Pedestrian detection using gradient local binary patterns. IEICE Trans Fundam Electron Commun Comput Sci 95(8):1280–1287

    Article  Google Scholar 

  25. Jiang N, Xu J, Yu W, Goto S (2013) Gradient local binary patterns for human detection. In: 2013 IEEE International Symposium on Circuits and Systems (ISCAS2013). IEEE, pp 978–981

  26. Jiang Z (2017) Camera network analysis for visual surveillance in industrial electronic context. Multimed Tools Appl:1–16

  27. Jungling K, Arens M (2009) Feature based person detection beyond the visible spectrum. In: 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops. IEEE, pp 30–37

  28. Kerkaou Z, El Ansari M, Masmoudi L, Lahmyed R (2021) Dense spatio-temporal stereo matching for intelligent driving systems. IET Image Process 15(3):715–723

    Article  Google Scholar 

  29. Kumar M, Jindal MK, Sharma RK (2013) Pca-based offline handwritten character recognition system. SmartCR 3(5):346–357

    Article  Google Scholar 

  30. Kumar M, Jindal MK, Sharma RK, Jindal SR (2020) Performance evaluation of classifiers for the recognition of offline handwritten gurmukhi characters and numerals: a study. Artif Intell Rev 53(3):2075–2097

    Article  Google Scholar 

  31. Kumar M, Jindal SR (2019) Fusion of rgb and hsv colour space for foggy image quality enhancement. Multimed Tools Appl 78(8):9791–9799

    Article  Google Scholar 

  32. Kumar M, Singh N, Kumar R, Goel S, Kumar K (2021) Gait recognition based on vision systems: A systematic survey. J Vis Commun Image Represent 75:103052

    Article  Google Scholar 

  33. Kwon J, Lee KM (2010) Visual tracking decomposition. In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. IEEE, pp 1269–1276

  34. Lahmyed R, El Ansari M (2016) Multisensors-based pedestrian detection system. In: 2016 IEEE/ACS 13th International Conference of Computer Systems and Applications (AICCSA). IEEE, pp 1–4

  35. Lahmyed R, El Ansari M, Ellahyani A (2019) A new thermal infrared and visible spectrum images-based pedestrian detection system. Multimed Tools Appl 78(12):15861–15885

    Article  Google Scholar 

  36. Lahmyed R, El Ansari M, Tremeau A, Kerkaou Z (2018) Camera-light detection and ranging data fusion-based system for pedestrian detection. J Electron Imaging 27(6):063011

    Google Scholar 

  37. Laptev I (2006) Improvements of object detection using boosted histograms.. In: BMVC, vol 3. Citeseer, pp 949–958

  38. Lee MW, Nevatia R (2006) Human pose tracking using multi-level structured models. In: European conference on computer vision. Springer, pp 368–381

  39. Li J, Gong W, Li W, Liu X (2010) Robust pedestrian detection in thermal infrared imagery using the wavelet transform. Infrared Phys Technol 53(4):267–273

    Article  Google Scholar 

  40. Liao S, Zhao G, Kellokumpu V, Pietikäinen M, Li SZ (2010) Modeling pixel process with scale invariant local patterns for background subtraction in complex scenes. In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. IEEE, pp 1301–1306

  41. Lim J, Kim W (2013) Detecting and tracking of multiple pedestrians using motion, color information and the adaboost algorithm. Multimed Tools Appl 65(1):161–179

    Article  Google Scholar 

  42. Liu Y, Zeng L, Huang Y (2014) An efficient hog–albp feature for pedestrian detection. SIViP 8(1):125–134

    Article  Google Scholar 

  43. Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110

    Article  Google Scholar 

  44. Lucas BD, Kanade T, et al. (1981) An iterative image registration technique with an application to stereo vision

  45. Luo Y, Yin D, Wang A, Wu W (2018) Pedestrian tracking in surveillance video based on modified cnn. Multimed Tools Appl 77(18):24041–24058

    Article  Google Scholar 

  46. Madadlou A, Emam-Djomeh Z, Mousavi ME, Ehsani M, Javanmard M, Sheehan D (2009) Response surface optimization of an artificial neural network for predicting the size of re-assembled casein micelles. Comput Electron Agric 68(2):216–221

    Article  Google Scholar 

  47. Mikolajczyk K, Schmid C, Zisserman A (2004) Human detection based on a probabilistic assembly of robust part detectors. In: European Conference on Computer Vision. Springer, pp 69–82

  48. Ojala T, Pietikäinen M, Harwood D (1996) A comparative study of texture measures with classification based on featured distributions. Pattern recognition 29(1):51–59

    Article  Google Scholar 

  49. Olmeda D, Premebida C, Nunes U, Armingol JM, de la Escalera A (2013) Pedestrian detection in far infrared images. Integr Comput-Aided Eng 20(4):347–360

    Article  Google Scholar 

  50. Papageorgiou C, Poggio T (1999) Trainable pedestrian detection. In: Image Processing, 1999. ICIP 99. Proceedings. 1999 International Conference on, vol 4. IEEE, pp 35–39

  51. Papageorgiou C, Poggio T (2000) A trainable system for object detection. Int J Comput Vis 38(1):15–33

    Article  Google Scholar 

  52. Perš J, Sulić V, Kristan M, Perše M, Polanec K, Kovačič S (2010) Histograms of optical flow for efficient representation of body motion. Pattern Recogn Lett 31(11):1369–1376

    Article  Google Scholar 

  53. Porikli F, Tuzel O, Meer P (2006) Covariance tracking using model update based on lie algebra. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06), vol 1. IEEE, pp 728–735

  54. Rittscher J, Tu PH, Krahnstoever N (2005) Simultaneous estimation of segmentation and shape. In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), vol 2. IEEE, pp 486–493

  55. Schick B, Schmidt S (2013) Evaluation of video-based driver assistance systems with sensor data fusion by using virtual test driving. In: Proceedings of the FISITA 2012 World Automotive Congress. Springer, pp 1363–1375

  56. Shahrokni A, Ellis A, Ferryman J (2009) Overall evaluation of the pets2009 results

  57. Shashua A, Gdalyahu Y, Hayun G (2004) Pedestrian detection for driving assistance systems: Single-frame classification and system level performance. In: IEEE Intelligent Vehicles Symposium, 2004. IEEE, pp 1–6

  58. Shechtman E, Irani M (2007) Matching local self-similarities across images and videos. In: Computer Vision and Pattern Recognition, 2007. CVPR’07. IEEE Conference on. IEEE, pp 1–8

  59. Smith K, Gatica-Perez D, Odobez J-M (2005) Using particles to track varying numbers of interacting people. In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), vol 1. IEEE, pp 962–969

  60. Stauffer C, Grimson WEL (1999) Adaptive background mixture models for real-time tracking. In: Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149), vol 2. IEEE, pp 246–252

  61. Thirde D, Li L, Ferryman F (2006) Overview of the pets2006 challenge. In: Proceedings of 9th IEEE International Workshop on Performance Evaluation of Tracking and Surveillance (PETS 2006), pp 47–50

  62. Trefnỳ J, Matas J (2010) Extended set of local binary patterns for rapid object detection. In: Computer vision winter workshop, pp 1–7

  63. Vapnik VN, Vapnik V (1998) Statistical learning theory, vol 1. Wiley, New York

  64. Varga D, Havasi L, Szirányi T (2015) Pedestrian detection in surveillance videos based on cs-lbp feature. In: 2015 International Conference on Models and Technologies for Intelligent Transportation Systems (MT-ITS). IEEE, pp 413–417

  65. Viola P, Jones MJ, Snow D (2005) Detecting pedestrians using patterns of motion and appearance. Int J Comput Vis 63(2):153–161

    Article  Google Scholar 

  66. Walk S, Majer N, Schindler K, Schiele B (2010) New features and insights for pedestrian detection. In: 2010 IEEE Computer society conference on computer vision and pattern recognition. IEEE, pp 1030–1037

  67. Wang D, Lu H, Yang M-H (2012) Online object tracking with sparse prototypes. IEEE Trans Image Process 22(1):314–325

    Article  MathSciNet  Google Scholar 

  68. Wu H, Liu N, Luo X, Su J, Chen L (2014) Real-time background subtraction-based video surveillance of people by integrating local texture patterns. SIViP 8(4):665–676

    Article  Google Scholar 

  69. Wu Y, Lim J, Yang M-H (2013) Online object tracking: A benchmark. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2411–2418

  70. Xiao F, Liu B, Li R (2020) Pedestrian object detection with fusion of visual attention mechanism and semantic computation. Multimed Tools Appl 79(21):14593–14607

    Article  Google Scholar 

  71. Yang J, Liang W, Jia Y (2012) Face pose estimation with combined 2d and 3d hog features. In: Pattern Recognition (ICPR), 2012 21st International Conference on. IEEE, pp 2492–2495

  72. Yang T, Fu D, Pan S (2017) Pedestrian tracking for infrared image sequence based on trajectory manifold of spatio-temporal slice. Multimed Tools Appl 76(8):11021–11035

    Article  Google Scholar 

  73. Yao S, Pan S, Wang T, Zheng C, Shen W, Chong Y (2015) A new pedestrian detection method based on combined hog and lss features. Neurocomputing 151:1006–1014

    Article  Google Scholar 

  74. Zhang S, Bauckhage C, Klein DA, Cremers AB (2013) Moving pedestrian detection based on motion segmentation. In: 2013 IEEE Workshop on Robot Vision (WORV). IEEE, pp 102–107

Download references

Author information

Authors and Affiliations

  1. LabSIV, Department of Computer Science, Faculty of Science, Ibn Zohr University, BP 8106, 80000, Agadir, Morocco

    Redouan Lahmyed, Mohamed El Ansari & Zakaria Kerkaou

  2. Informatics and Applications Laboratory, Department of Computer Science, Faculty of Science, My Ismail University, Meknès, Morocco

    Mohamed El Ansari

Authors
  1. Redouan Lahmyed
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mohamed El Ansari
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zakaria Kerkaou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Redouan Lahmyed.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This research work was supported by the National Center for Scientific and technical Research (CNRST), research grant No: 20UIZ2015.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lahmyed, R., El Ansari, M. & Kerkaou, Z. A novel visible spectrum images-based pedestrian detection and tracking system for surveillance in non-controlled environments. Multimed Tools Appl 81, 39275–39309 (2022). https://doi.org/10.1007/s11042-022-13026-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-13026-4

Keywords

Search

Navigation