The paper proposes a method to detect failures in object tracking. Detection is done with the help of two types of errors, namely jump and stop errors. Jump errors occur when an abrupt change in object’s motion is observed, whereas stop errors occur when a moving object remains stationary for longer duration at any point. In our framework, moving objects are first tracked using well-known trackers and their trajectories are obtained. Discrepancies between trajectories are measured. We have shown that the proposed method can be reliable for detection of tracking failures. This can help to find error-free trajectories that are essential in various computer vision tasks. We have also shown that the tracking performance can be further improved while processing the output trajectories without much knowledge about the underlying tracking algorithms. The effect of tracking failure is investigated to identify erroneous trajectories. It has been observed that when a tracker fails, velocity profile of the moving object usually changes significantly. Based on this hypothesis, erroneous trajectories are detected and a set of error-free trajectories are marked and grouped. Two recently proposed tracking algorithms, namely real-time compressive tracker (CT) and real-time L1-tracker (L1APG), have been used to track the objects. We have tested our framework on five publicly available datasets containing more than 300 trajectories. Our experiments reveal that average classification rate of erroneous trajectories can be as high as 80.4% when objects are tracked using L1APG tracker. Accuracy can be as high as 81.2% when applied on trajectories obtained using CT tracker. Average accuracy of tracking increases significantly (19.2% with respect to L1APG tracker and 24.8% with respect to CT tracker) when the decision is taken using a fused framework.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Tax calculation will be finalised during checkout.
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
Tax calculation will be finalised during checkout.
Adam A, Rivlin E, Shimshoni I (2006) Robust fragments-based tracking using the integral histogram. In: Proceedings of the 2006 IEEE computer society conference on computer vision and pattern recognition (CVPR’06). IEEE, vol 1, pp 798–805
Ahmed A, Dogra D, Kar S, Kim B, Hill P, Bhaskar H (2016) Localization of region of interest in surveillance scene. Multimed Tools Appl. doi:10.1007/s11042-016-3762-y
Ali S, Shah M (2007) A lagrangian particle dynamics approach for crowd flow segmentation and stability analysis. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, pp 1–6
Andriyenko A, Schindler K (2011) Multi-target tracking by continuous energy minimization. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, pp 1265–1272
Babenko YMH B, Belongie S (2011) Robust object tracking with online multiple instance learning. IEEE Trans Pattern Anal Mach Intell 33(8):1619–1632
Bae SH, Yoon KJ (2014) Robust online multi-object tracking based on tracklet confidence and online discriminative appearance learning. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1218–1225
Bailer C, Pagani A, Stricker D (2014) A superior tracking approach: Building a strong tracker through fusion. In: Computer vision–ECCV 2014. Springer, pp 170–185
Bao C, Wu Y, Ling H, Ji H (2012) Real time robust l1 tracker using accelerated proximal gradient approach. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, pp 1830–1837
Benfold B, Reid I (2011) Stable multi-target tracking in real-time surveillance video. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, pp 3457–3464
Bernardin K, Stiefelhagen R (2008) Evaluating multiple object tracking performance: the clear mot metrics. J Image Video Process 2008:1
Bibby C, Reid I (2008) Robust real-time visual tracking using pixel-wise posteriors. In: Computer vision–ECCV 2008. Springer, pp 831–844
Birchfield S (2007) Klt: An implementation of the Kanade–Lucas–Tomasi feature tracker
Comaniciu D, Ramesh V, Meer P (2000) Real-time tracking of non-rigid objects using mean shift. Proc IEEE Comput Soc Conf Comput Vis Pattern Recogn 2:142–149
Comaniciu D, Ramesh V, Meer P (2003) Kernel-based object tracking. IEEE Trans Pattern Anal Mach Intell 25(5):564–577
Dehghan A, Idrees H, Zamir A, Shah M (2014) Automatic detection and tracking of pedestrians in videos with various crowd densities. In: Proceedings of the pedestrian and evacuation dynamics, pp 3–19
Doğan S, Temiz M, Külür S (2010) Real time speed estimation of moving vehicles from side view images from an uncalibrated video camera. Sensors 10(5):4805–4824
Dogra D, Ahmed A, Bhaskar H (2015a) Interest area localization using trajectory analysis in surveillance scenes. In: Proceedings of the 10th international conference on computer vision theory and applications, pp 478–485
Dogra D, Reddy R, Subramanyam K, Ahmed A, Bhaskar H (2015b) Scene representation and anomalous activity detection using weighted region association graph. In: Proceedings of the 10th international conference on computer vision theory and applications, pp 104–112
Dogra D, Ahmed A, Bhaskar H (2016) Smart video summarization using mealy machine-based trajectory modelling for surveillance applications. Multimed Tools Appl 75:6373–6401
Doyle D, Jennings A, Black J (2014) Optical flow background estimation for real-time pan/tilt camera object tracking. Measurement 48:195–207
Ekin A, Tekalp A, Mehrotra R (2003) Automatic soccer video analysis and summarization. IEEE Trans Image Process 12(7):796–807
Gall J, Yao A, Razavi N, Gool LV, Lempitsky V (2011) Hough forests for object detection, tracking, and action recognition. IEEE Trans Pattern Anal Mach Intell 33(11):2188–2202
Hua Y, Alahari K, Schmid C (2015) Online object tracking with proposal selection. In: Proceedings of the IEEE international conference on computer vision, pp 3092–3100
Jurie F, Dhome M (2001) Real time tracking of 3d objects with occultations. In: Proceedings of international conference on image processing. IEEE, vol 1, pp 413–416
Kaempchen N, Buehler M, Dietmayer K (2005) Feature-level fusion for free-form object tracking using laser scanner and video. In: Intelligent vehicles symposium, 2005. Proceedings. IEEE, pp 453–458
Khan M, Valstar M, Pridmore T (2013) A multiple motion model tracker handling occlusion and rapid motion variation. In: British machine vision workshop
Kim S, Kwak S, Feyereisl J, Han B (2012) Online multi-target tracking by large margin structured learning. In: Asian conference on computer vision. Springer, pp 98–111
Lalonde M, Beaulieu M, Gagnon L (2001) Fast and robust optic disc detection using pyramidal decomposition and hausdorff-based template matching. IEEE Trans Med Imaging 20(11):1193–1200
Lan X, Ma A, Yuen P (2014) Multi-cue visual tracking using robust feature-level fusion based on joint sparse representation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1194–1201
Leibe B, Schindler K, Gool L (2007) Coupled detection and trajectory estimation for multi-object tracking. In: IEEE 11th international conference on computer vision (ICCV), IEEE, pp 1–8
Li F, Cao T, Song Z, Wang W (2015) Object tracking based on locality-constrained linear coding joint sparse representation appearance model. In: IEEE 16th international conference on communication technology (ICCT). IEEE, pp 892–896
Li X, Hu W, Shen C, Zhang Z, Dick A, Hengel A (2013) A survey of appearance models in visual object tracking. ACM Trans Intell Syst Technol (TIST) 4(4):58
Li Y, Huang C, Nevatia R (2009) Learning to associate: hybrid boosted multi-target tracker for crowded scene. In: IEEE conference on computer vision and pattern recognition, 2009 (CVPR, 2009). IEEE, pp 2953–2960
Long C, Hua G, Kapoor A (2015) A joint gaussian process model for active visual recognition with expertise estimation in crowdsourcing. Int J Comput Vis. doi:10.1007/s11263-015-0834-9
Ma L, Lu J, Feng J, Zhou J (2015) Multiple feature fusion via weighted entropy for visual tracking. In: Proceedings of the IEEE international conference on computer vision, pp 3128–3136
Mariano V, Min J, Park J, Kasturi R, Mihalcik D, Li H, Doermann D, Drayer T (2002) Performance evaluation of object detection algorithms. In: Proceedings of 16th international conference on pattern recognition, 2002. IEEE, vol 3, pp 965–969
Martín R, Martínez J (2014) Evaluation of bounding box level fusion of single target video object trackers. In: Hybrid artificial intelligence systems. Springer, pp 200–210
MS Temiz SK, Dogan S (2012) Real time speed estimation from monocular video. In: International archives of the photogrammetry, remote sensing and spatial information sciences, pp 427–432
Nummiaro K, Koller-Meier E, Gool LV (2002) Object tracking with an adaptive color-based particle filter. In: Pattern recognition. Springer, pp 353–360
Ogiela L, Ogiela MR (2015) Comparison of cognitive information systems supporting management tasks. In: 2015 international conference on intelligent networking and collaborative systems (INCOS). IEEE, pp 49–56
Papadourakis V, Argyros A (2010) Multiple objects tracking in the presence of long-term occlusions. Comput Vis Image Underst 114(7):835–846
Pettersen S, Johansen D, Oth (2014) Soccer video and player position dataset. In: Proceedings of the 5th ACM multimedia systems conference, pp 18–23
Power D, Sharda R, Burstein F (2015) Decision support systems. Wiley Online Library, London
Ristic B, Arulampalam S, Gordon N (2004) Beyond the kalman filter. IEEE Aerosp Electron Syst Mag 19(7):37–38
Rousseeuw P (1987) Silhouettes: a graphical aid to the interpretation and validation of cluster analysis. J Comput Appl Math 20:53–65
Saini R, Ahmed A, Dogra DP, Roy PP (2017a) Classification of object trajectories represented by high-level features using unsupervised learning. In: Proceedings of international conference on computer vision and image processing. Springer, pp 273–284
Saini R, Ahmed A, Dogra DP, Roy PP (2017b) Surveillance scene segmentation based on trajectory classification using supervised learning. In: Proceedings of international conference on computer vision and image processing. Springer, pp 261–271
Schoepflin T, Dailey D (2003) Dynamic camera calibration of roadside traffic management cameras for vehicle speed estimation. IEEE Trans Intell Transp Syst 4(2):90–98
Shah M, Javed O, Shafique K (2007) Automated visual surveillance in realistic scenarios. IEEE Multimed 14(1):30–39
Shao J, Loy C, Wang X (2014) Scene-independent group profiling in crowd. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, pp 2227–2234
Shin J, Kim S, Kang S, Lee S, Paik J, Abidi B, Abidi M (2005) Optical flow-based real-time object tracking using non-prior training active feature model. Real-Time Imaging 11(3):204–218
Smeulders A, Chu D, Cucchiara R, Calderara S, Dehghan A, Shah M (2014) Visual tracking: an experimental survey. IEEE Trans Pattern Anal Mach Intell 36(7):1442–1468
Stauffer C, Grimson W (2000) Learning patterns of activity using real-time tracking. IEEE Trans Pattern Anal Mach Intell 22(8):747–757
Takala V, Pietikainen M (2007) Multi-object tracking using color, texture and motion. In: IEEE conference on computer vision and pattern recognition (CVPR). IEEE, pp 1–7
Tang F, Brennan S, Zhao Q, Tao H (2007) Co-tracking using semi-supervised support vector machines. In: IEEE 11th international conference on computer vision (ICCV). IEEE, pp 1–8
Wang X, Ma K, Ng G, Grimson W (2011) Trajectory analysis and semantic region modeling using nonparametric hierarchical bayesian models. Int J Comput Vis 95(3):287–312
Wu H, Sankaranarayanan A, Chellappa R (2010) Online empirical evaluation of tracking algorithms. IEEE Trans Pattern Anal Mach Intell 32(8):1443–1458
Wu Y, Lim J, Yang M (2015) Object tracking benchmark. IEEE Trans Pattern Anal Mach Intell 37(9):1834–1848
Xiang Y, Alahi A, Savarese S (2015) Learning to track: online multi-object tracking by decision making. In: Proc IEEE Int Conf Comput Vis, pp 4705–4713
Yilmaz A (2007) Object tracking by asymmetric kernel mean shift with automatic scale and orientation selection. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, pp 1–6
Yilmaz A, Li X, Shah M (2004) Contour-based object tracking with occlusion handling in video acquired using mobile cameras. IEEE Trans Pattern Anal Mach Intell 26(11):1531–1536
Yilmaz A, Javed O, Shah M (2006) Object tracking: a survey. ACM Comput Surv 38(4):13
Zhang K, Zhang L, Yang M (2012) Real-time compressive tracking. In: Proceedings of the European conference on computer vision, pp 864–877
Zhong Y, Jain A, Dubuisson-Jolly M (2000) Object tracking using deformable templates. IEEE Trans Pattern Anal Mach Intell 22(5):544–549
Zhou B, Wang X, Tang X (2012) Understanding collective crowd behaviors: learning a mixture model of dynamic pedestrian-agents. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, pp 2871–2878
Zhou B, Tang X, Zhang H, Wang X (2014a) Measuring crowd collectiveness. IEEE Trans Pattern Anal Mach Intell 36(8):1586–1599
Zhou B, Tang X, Wang X (2015) Learning collective crowd behaviors with dynamic pedestrian-agents. Int J Comput Vis 111(1):50–68
Zhou H, Yuan Y, Shi C (2009) Object tracking using sift features and mean shift. Comput Vis Image Underst 113(3):345–352
Zhou Y, Rao C, Lu Q, Bai X, Liu W (2011) Multiple feature fusion for object tracking. In: Intelligent science and intelligent data engineering. Springer, pp 145–152
Zhou Z, Wu D, Peng X, Zhu Z, Luo K (2014b) Object tracking based on camshift with multi-feature fusion. J Softw 9(1):147–153
Zivkovic Z, Krose B (2004) An em-like algorithm for color-histogram-based object tracking. In: Proceedings of the 2004 IEEE computer society conference on computer vision and pattern recognition (CVPR). IEEE, vol 1, pp I–798
This study has not been funded by any agency.
Conflict of interest
The authors declare that they have no conflict of interest regarding the publication of this paper.
This article does not contain any studies on human participants or animals.
Consent was obtained from all volunteers/individual participants included in the study.
Communicated by V. Loia.
About this article
Cite this article
Ahmed, S.A., Dogra, D.P., Kar, S. et al. Unsupervised classification of erroneous video object trajectories. Soft Comput 22, 4703–4721 (2018). https://doi.org/10.1007/s00500-017-2656-x
- Object tracking
- Tracker fusion
- Unsupervised tracking