Neural Computing and Applications

, Volume 28, Supplement 1, pp 127–141 | Cite as

Online adaptive multiple pedestrian tracking in monocular surveillance video

  • Zhihui Wang
  • Sook YoonEmail author
  • Dong Sun Park
Original Article


Automatic online multiple pedestrian tracking is a rather important and challenging task in the field of machine vision. A new multiple pedestrian tracking system is proposed in this paper, which combines pedestrian detection, motion prediction, target matching and adaptive location adjustment methods. The clip-split strategy was adopted for optimization of the detected pedestrian candidates, which resulted in great improvement of the tracking accuracies, especially when the marginal areas of the detected target candidates contained background scenes. For each frame, the proposed adaptive location adjustment method was used to adjust the location and scale of the targets to deal with drifting problems where necessary, especially after severe occlusions. Experimental results on three challenging real-world datasets demonstrated that the proposed tracker has excellent performance over other state-of-the-art trackers based on MOT metrics.


Visual tracking Pedestrian detection Clip-split strategy Adaptive location adjustment MOT metrics 



This research was supported by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (2013R1A1A2013778) and also supported (in part) by Research Funds of Mokpo National University in 2013.


  1. 1.
    Li X, Hu W, Shen C, Zhang Z, Dick A, Hengel AVD (2008) A survey of appearance models in visual object tracking. ACM Trans Intel Syst Technol 4(4): Article 58Google Scholar
  2. 2.
    Yilmaz A, Javed O, Shah M (2006) Object tracking: a survey, ACM Comput Surv 38(4):Article 13Google Scholar
  3. 3.
    Smeulders AWM, Chu DM, Cucchiara R, Calderara S, Dehghan A, Shah M (2014) Visual tracking: an experimental survey. IEEE Trans Pattern Anal Mach Intell 36(7):1442–1468CrossRefGoogle Scholar
  4. 4.
    Huang CM, Fu LC (2011) Multitarget visual tracking based effective surveillance with cooperation of multiple active cameras. IEEE Trans Syst Man Cybern Part B Cybern 41(1):234–247CrossRefGoogle Scholar
  5. 5.
    Renno J, Orwell J, Jones GA (2002) Learning surveillance tracking models for the self-calibrated ground plane. In: Proceedings of the 2002 British machine vision conference (BMVC 2002), pp 1–10Google Scholar
  6. 6.
    Moeslund TB, Hilton A, Krüger V (2006) A survey of advances in vision-based human motion capture and analysis. Comput Vision Image Underst 104(2):90–126CrossRefGoogle Scholar
  7. 7.
    Luck JP, Debrunner C, Hoff W, He Q, Small DE (2002) Development and analysis of a real-time human motion tracking system. In: Proceedings of the sixth IEEE workshop on applications of computer vision (WACV 2002), Orlando, pp 196–202Google Scholar
  8. 8.
    Führ G, Jung CR (2014) Combining patch matching and detection for robust pedestrian tracking in monocular calibrated cameras. Pattern Recogn Lett 39:11–20CrossRefGoogle Scholar
  9. 9.
    Possegger H, Mauthner T, Roth PM, Bischof H (2014) Occlusion geodesics for online multi-object tracking. In: Proceedings of the 2014 IEEE conference on computer vision and pattern recognition (CVPR 2014), Columbus, pp 1306–1313Google Scholar
  10. 10.
    Führ G, Jung CR (2012) Robust patch-based pedestrian tracking using monocular calibrated cameras. In: Proceedings of the 25th SIBGRAPI conference on graphics, patterns and images (SIBGRAPI 2012), Ouro Preto, pp 166–173Google Scholar
  11. 11.
    Veenman CJ, Reinders MJ, Backer E (2001) Resolving motion correspondence for densely moving points. IEEE Trans Pattern Anal Mach Intell 23(1):54–72CrossRefGoogle Scholar
  12. 12.
    Serby D, Meier EK, Gool LV (2004) Probabilistic object tracking using multiple features. In: Proceedings of the 17th international conference on pattern recognition (ICPR 2004), Cambridge, vol 2, pp 184–187Google Scholar
  13. 13.
    Comaniciu D, Ramesh V, Meer P (2003) Kernel-based object tracking. IEEE Trans Pattern Anal Mach Intell 25(5):564–577CrossRefGoogle Scholar
  14. 14.
    Yilmaz A, Li X, Shah M (2004) Contour-based object tracking with occlusion handling in video acquired using mobile cameras. IEEE Trans Pattern Anal Mach Intell 26(11):1531–1536CrossRefGoogle Scholar
  15. 15.
    Ali A, Aggarwal JK (2001) Segmentation and recognition of continuous human activity. In: Proceedings of the 2001 IEEE workshop on detection and recognition of events in video, Vancouver, pp 28–35Google Scholar
  16. 16.
    Edwards GJ, Taylor CJ, Cootes TF (1998) Interpreting face images using active appearance models. In: Proceedings of the third IEEE international conference on automatic face and gesture recognition, Nara, pp 300–305Google Scholar
  17. 17.
    Black MJ, Jepson AD (1998) Eigentracking: robust matching and tracking of articulated objects using a view-based representation. Int J Comput Vision 26(1):63–84CrossRefGoogle Scholar
  18. 18.
    Dicle C, Camps OI, Sznaier M (2013) The way they move: tracking multiple targets with similar appearance. In: Proceedings of the 2013 IEEE international conference on computer vision (ICCV 2013), Sydney, pp 2304–2311Google Scholar
  19. 19.
    Pirsiavash H, Ramanan D, Fowlkes CC (2011) Globally-optimal greedy algorithms for tracking a variable number of objects. In: Proceedings of the 2011 IEEE conference on computer vision and pattern recognition (CVPR 2011), Colorado, pp 1201–1208Google Scholar
  20. 20.
    Kuhn HW (2005) The Hungarian method for the assignment problem. Naval Res Log 52(1):7–21CrossRefGoogle Scholar
  21. 21.
    Andriyenko A, Schindler K (2010) Globally optimal multi-target tracking on a hexagonal lattice. In: Proceedings of the 11th European conference on computer vision (ECCV 2010), Heraklion, Crete, pp 466–479Google Scholar
  22. 22.
    Babenko B, Yang M-H, Belongie S (2011) Robust object tracking with online multiple instance learning. IEEE Trans Pattern Anal Mach Intell 33(8):1619–1632CrossRefGoogle Scholar
  23. 23.
    Zhang K, Song H (2013) Real-time visual tracking via online weighted multiple instance learning. Pattern Recogn 46:397–411CrossRefzbMATHGoogle Scholar
  24. 24.
    Quan W, Chen JX, Yu N (2014) Robust object tracking enhanced random ferns. Visual Comput 30(4):351–358CrossRefGoogle Scholar
  25. 25.
    Freund Y, Schapire RE (1996) Experiments with a new boosting algorithm. In: Proceedings of the 13th international conference on machine learning, Bari, pp 148–156Google Scholar
  26. 26.
    Mallapragada PK, Jin R, Jain AK, Liu Y (2009) Semiboost: boosting for semi-supervised learning. IEEE Trans Pattern Anal Mach Intell 31(11):2000–2014CrossRefGoogle Scholar
  27. 27.
    Xu X-S, Jiang Y, Xue X, Zhou Z-H (2012) Semisupervised multi-instance multi-label learning for video annotation task. In: Proceedings of the 20th ACM international conference on multimedia, Nara, pp 737–740Google Scholar
  28. 28.
    Haritaoglu I, Harwood D, Davis LS (1998) W\(^4\)S: a real-time system for detecting and tracking people in \(2\frac{1}{2}\)D. In: Proceedings of the 5th European conference on computer vision (ECCV’98), Freiburg, pp 877–892Google Scholar
  29. 29.
    Shen Y, Hu W, Liu J, Yang M, Wei B, Chou CT (2012) Efficient background subtraction for real-time tracking in embedded camera networks. In: Proceedings of the 10th ACM conference on embedded network sensor systems (SenSys ’10), Toronto, pp 295–308Google Scholar
  30. 30.
    PETS 2009 (2009) Eleventh IEEE international workshop on performance evaluation of tracking and surveillance, 2009. Available:
  31. 31.
    Benfold B, Reid I (2011) Stable multi-target tracking in real-time surveillance video. In: Proceedings of the 2011 IEEE conference on computer vision and pattern recognition (CVPR 2011), Colorado, pp 3457–3464Google Scholar
  32. 32.
    Astola J, Haavisto P, Neuvo Y (1990) Vector median filters. Proc IEEE 78(4):678–689CrossRefGoogle Scholar
  33. 33.
    Han J, Kamber M (2001) Data mining: concepts and techniques. Morgan Kaufman, BurlingtonzbMATHGoogle Scholar
  34. 34.
    Barnich O, Van Droogenbroeck M (2011) ViBe: a universal background subtraction algorithm for video sequences. IEEE Trans Image Process 20(6):1709–1724MathSciNetCrossRefzbMATHGoogle Scholar
  35. 35.
    Dollár P, Belongie S, Perona P (2010) The fastest pedestrian detector in the West. In: Proceedings of the British machine vision conference 2010 (BMVC), Aberystwyth, Wales, pp 68.1–68.11Google Scholar
  36. 36.
    Hofmann M, Tiefenbacher P, Rigoll G (2012) Background segmentation with feedback: the pixel-based adaptive segmenter. In: Proceedings of the 2012 IEEE computer society conference on computer vision and pattern recognition workshops (CVPRW 2012), Providence, Rhode Island, pp 38–43Google Scholar
  37. 37.
    Freund Y, Schapire RE (1996) Experiments with a new boosting algorithm. In: Proceedings of the 13th international conference on machine learning (ICML’ 96), Bari, vol 96, pp 148–156Google Scholar
  38. 38.
    Overett G, Petersson L, Brewer N, Andersson L, Pettersson N (2008) A new pedestrian dataset for supervised learning. In: Proceedings of the 2008 IEEE intelligent vehicles symposium, Eindhoven, pp 373–378Google Scholar
  39. 39.
    Keller C, Enzweiler M, Gavrila DM (2011) A new benchmark for stereo-based pedestrian detection. In: Proceedings of the IEEE intelligent vehicles symposium, Baden-BadenGoogle Scholar
  40. 40.
    Wang Z, Yoon S, Xie SJ, Lu Y, Park DS (2014) A high accuracy pedestrian detection system combining a cascade AdaBoost detector and random vector functional-link net. Sci World J 2014:105089. doi: 10.1155/2014/105089 Google Scholar
  41. 41.
    Ayazoglu M, Sznaier M, Camps OI (2012) Fast algorithms for structured robust principal component analysis. In: Proceedings of the 2012 IEEE conference on computer vision and pattern recognition (CVPR 2012), Providence, Rhode Island, pp 1704–1711Google Scholar
  42. 42.
    Zhang K, Zhang L, Yang M (2014) Fast compressive tracking. IEEE Trans Pattern Anal Mach Intell 36(10):2002–2015CrossRefGoogle Scholar
  43. 43.
    Kalal Z, Mikolajczyk K, Matas J (2010) Tracking-learning-detection. IEEE Trans Pattern Anal Mach Intell 34(7):1409–1422CrossRefGoogle Scholar
  44. 44.
    Felzenszwalb PF, Girshick RB, McAllester D, Ramanan D (2010) Object detection with discriminatively trained part-based models. IEEE Trans Pattern Anal Mach Intell 32(9):1627–1645CrossRefGoogle Scholar
  45. 45.
    Keni B, Rainer S (2008) Evaluating multiple object tracking performance: the CLEAR MOT metrics. EURASIP J Image Video Process 2008:246309. doi: 10.1155/2008/246309 Google Scholar

Copyright information

© The Natural Computing Applications Forum 2016

Authors and Affiliations

  1. 1.Department of Electronics EngineeringChonbuk National UniversityJeonjuSouth Korea
  2. 2.Department of Multimedia EngineeringMokpo National UniversityJeonnamSouth Korea
  3. 3.IT Convergence Research CenterChonbuk National UniversityJeonjuSouth Korea
  4. 4.Division of Electronics EngineeringChonbuk National UniversityJeonjuSouth Korea

Personalised recommendations