Skip to main content
SpringerLink
Log in

An extended TLD tracking algorithm using co-training learning for low frame rate videos

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

The conventional tracking-learning-detection (TLD) algorithm is sensitive to illumination changes, clutter, significant changes of target shape between consecutive frames. In addition, low frame rate scenarios result in drift in object position or even missing the object. To solve these problems and enhance the tracking robustness, in this paper, TLD algorithm was extended in two folds. First, Kalman filter and Mean-shift algorithms were combined for the tracking part and second, co-training semi-supervised learning algorithm was used for the learning part of the conventional TLD structure. The Kalman filter estimates the position of the target in the next frame based on the previous positions of the target. This reduces tracking failure. On the other hands, the Mean-shift tracking algorithm is robust to rotation, partial occlusion and scale changing. In the learning part of TLD structure, two training tracking algorithms with two independent classifiers were run on the current frame simultaneously. Its structure makes data of both pools (color features and target templates) update by the results of other algorithm, in addition to the results of the corresponding algorithm in each of the tracking and detection algorithms. Therefore, classifiers can learn faster changing features of the target during the consecutive frames in online tracking process. Finally, the extended structure can solve the problem of lost object in LFR videos tracking and other similar challenges simultaneously. In terms of overlap ratio metric, comparing with conventional TLD and extended kernelized correlation filters (EKCF) algorithms, the success rate of our algorithm under various scenarios has increased by 161.03% and 255.82% respectively and under other scenarios, in terms of precision metric, it has increased by 18.46% and 1479.47%, respectively. Accordingly, comparative evaluations of the proposed method to other top state-of-the-art tracking algorithms under various scenarios present in TB-100 dataset, the superior performance of the proposed algorithm was demonstrated compared to other tracking algorithms in terms of tracking robustness, stability and real-time performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Data availability

The data that support the finding of this study are publicly available from “http://cvlab.hanyang.ac.kr/tracker_benchmark/datasets.html”.

References

  1. Ahmad S, Mehfuz S, Mebarek-Oudina F, Beg J, (2022) RSM analysis based cloud access security broker: a systematic literature review. Cluster Comput, pp 1–31

  2. Bao C, Wu Y, Ling H, Ji H (2012) Real time robust l1 tracker using accelerated proximal gradient approach. In: 2012 IEEE conference on computer vision and pattern recognition, pp 1830–1837

  3. Bhatti UA, Huang M, Wu D, Zhang Y, Mehmood A, Han H (2019) Recommendation system using feature extraction and pattern recognition in clinical care systems. Enterprise Inf Syst 13(3):329–351

    Article  Google Scholar 

  4. Bhatti UA, Yu Z, Chanussot J, Zeeshan Z, Yuan L, Luo W, Nawaz SA, Bhatti MA, Ain QU, Mehmood A (2021) Local similarity-based spatial–spectral fusion hyperspectral image classification with deep CNN and Gabor filtering. IEEE Trans Geosci Remote Sens 60:1–15

    Article  Google Scholar 

  5. Boroujeni HS, Charkari NM, Behrouzifar M, Makhsoos PT (2011) Tracking multiple variable-sizes moving objects in LFR videos using a novel genetic algorithm approach. In: Knowledge technology week, pp 143–153

  6. Carneiro G, Nascimento JC (2011) Incremental on-line semi-supervised learning for segmenting the left ventricle of the heart from ultrasound data. In: IEEE international conference on computer vision, pp 1700–1707

  7. Chowdhary CL, Patel PV, Kathrotia KJ, Attique M, Perumal K, Ijaz MF (2020) Analytical study of hybrid techniques for image encryption and decryption. Sensors 20(18):5162

    Article  Google Scholar 

  8. Chuang MC, Hwang JN, Kuo FF, Shan MK, Williams K (2014) Recognizing live fish species by hierarchical partial classification based on the exponential benefit. In: IEEE international conference on image processing (ICIP), pp 5232–5236

  9. Dai W, Chang T, Su K, Wang Q (2016) Improved TLD target algorithm based on feature fusion. In: 2nd workshop on advanced research and Technology in Industry Applications (WARTIA-16), pp 398–403

  10. Dong E, Deng M, Tong J, Jia C, Du S (2019) Moving vehicle tracking based on improved tracking–learning–detection algorithm. IET Comput Vis 13(8):730–741

    Article  Google Scholar 

  11. Duan H, Zhang Y, Sheng W, Chen X, Ren J (2020) An improved TLD target tracking algorithm combined with kernel correlation filtering. IOP Conf Ser: Earth Environ Sci 440(3):032015

  12. Everingham M, Van Gool L, Williams CK, Winn J, Zisserman A (2010) The pascal visual object classes (voc) challenge. Int J Comput Vis 88(2):303–338

    Article  Google Scholar 

  13. Fang Y, Liu J, Li J, Cheng J, Hu J, Yi D, Xiao X, Bhatti UA (2022) Robust zero-watermarking algorithm for medical images based on SIFT and Bandelet-DCT. Multimed Tools Appl 81(12):16863–16879

    Article  Google Scholar 

  14. Gaxiola LN, Diaz-Ramirez VH, Tapia JJ, García-Martínez P (2016) Target tracking with dynamically adaptive correlation. Opt Commun 365:140–149

  15. Godec M, Roth PM, Bischof H (2013) Hough-based tracking of non-rigid objects. Comput Vis Image Underst 117(10):1245–1256

    Article  Google Scholar 

  16. Henriques JF, Caseiro R, Martins P, Batista J (2014) High-speed tracking with kernelized correlation filters. IEEE Trans Pattern Anal Mach Intell 37(3):583–596

    Article  Google Scholar 

  17. Hu J, Cai M, Li J (2017) An improved TLD method based on color feature. In: 2017 29th Chinese control and decision conference (CCDC), pp 6096–6101

  18. Hussain R, Karbhari Y, Ijaz MF, Woźniak M, Singh PK, Sarkar R (2021) Revise-net: exploiting reverse attention mechanism for salient object detection. Remote Sens 13(23):4941

    Article  Google Scholar 

  19. Jalali V, Kaur D (2020) A study of classification and feature extraction techniques for brain tumor detection. Int J Multimed Inf Retriev 9:271–290

  20. Jeong J, Yoon TS, Park JB (2017) Mean shift tracker combined with online learning-based detector and Kalman filtering for real-time tracking. Expert Syst Appl 79:194–206

    Article  Google Scholar 

  21. Kalal Z, Mikolajczyk K, Matas J (2011) Tracking-learning-detection. IEEE Trans Pattern Anal Mach Intell 34(7):1409–1422

    Article  Google Scholar 

  22. Lee G, Mallipeddi R, Lee M (2017) Trajectory-based vehicle tracking at low frame rates. Expert Syst Appl 80:46–57

    Article  Google Scholar 

  23. Li Y, Ai H, Yamashita T, Lao S, Kawade M (2008) Tracking in low frame rate video: a cascade particle filter with discriminative observers of different life spans. IEEE Trans Pattern Anal Mach Intell 30(10):1728–1740

    Article  Google Scholar 

  24. Li Z, Chen J, Schraudolph NN (2008) An improved mean-shift tracker with kernel prediction and scale optimisation targeting for low-frame-rate video tracking. In: 2008 19th international conference on pattern recognition, pp 1051–1055

  25. Li T, Zhao WJ, Yang S, Li C (2016) An improved TLD object tracking algorithm. In eighth international conference on digital image processing (ICDIP 2016), pp 100330–100334

  26. Li T, Li J, Liu J, Huang M, Chen YW, Bhatti UA (2022) Robust watermarking algorithm for medical images based on log-polar transform. EURASIP J Wirel Commun Netw 24:1–11

  27. Miao F, Cang-ju X (2017) Improved TLD target tracking method based on frame difference. Electron Design Eng 7

  28. Moridvaisi H, Razzazi F, Pourmina MA, Dousti M (2020) An extended KCF tracking algorithm based on TLD structure in low frame rate videos. Multimed Tools Appl 79(29):20995–21012

    Article  Google Scholar 

  29. Nawaz SA, Li J, Bhatti UA, Bazai SU, Zafar A, Bhatti MA, Mehmood A, Ain QU, Shoukat MU (2021) A hybrid approach to forecast the COVID-19 epidemic trend. Plos One 16(10):e0256971

    Article  Google Scholar 

  30. Nummiaro K, Koller-Meier E, Van Gool L (2003) An adaptive color-based particle filter. Image Vis Comput 21(1):99–110

    Article  MATH  Google Scholar 

  31. Nyo MT, Mebarek-Oudina F, Hlaing SS, Khan NA (2022) Otsu’s thresholding technique for MRI image brain tumor segmentation. Multimed Tools Appl:1–13

  32. Oron S, Bar-Hillel A, Levi D, Avidan S (2015) Locally orderless tracking. Int J Comput Vis 111(2):213–228

    Article  MathSciNet  MATH  Google Scholar 

  33. Otoom M, Al-Louzi M (2021) Enhanced TLD-based video object-tracking implementation tested on embedded platforms. J Real-Time Image Process 18:937–952

  34. Palaniappan K, Bunyak F, Kumar P, Ersoy I, Jaeger S, Ganguli K, Haridas A, Fraser J, Rao RM, Seetharaman G (2010) Efficient feature extraction and likelihood fusion for vehicle tracking in low frame rate airborne video. In: 2010 13th international conference on information fusion, pp 1–8

  35. Pan Z, Liu S, Fu W (2017) A review of visual moving target tracking. Multimed Tools Appl 76(16):16989–17018

    Article  Google Scholar 

  36. Pang Y, Shen D, Chen G, Liang P, Pham K, Blasch E, Wang Z, Ling H (2013) Low frame rate video target localization and tracking testbed. In ground/air multisensor interoperability, integration, and networking for persistent ISR IV 8742(1):87420T

  37. Park E, Ju H, Jeong YM, Min SY (2015) Tracking-learning-detection adopted unsupervised learning algorithm. In: 2015 seventh international conference on knowledge and systems engineering (KSE), pp 234–237

  38. Piao S, Berns K (2014) Multi-object tracking based on tracking-learning-detection framework. Field Assist Robot-Adv Syst Algorithms: 74–87

  39. Porikli F, Yilmaz A (2012) Object detection and tracking. In: Video analytics for business intelligence, pp 3–41

  40. Sathish PK, Balaji S (2018) A complete person re-identification model using kernel-PCA-based Gabor-filtered hybrid descriptors. Int J Multimed Inf Retriev 7(4):221–229

    Article  Google Scholar 

  41. Siqueira DL, MacHado AMC (2016) People detection and tracking in low frame-rate dynamic scenes. IEEE Lat Am Trans 14(4):1966–1971

    Article  Google Scholar 

  42. Song Z, Cong Z, Yanan Z, Yuren D (2017) An improved TLD target tracking algorithm based on mean shift. In: 2017 13th IEEE international conference on Electronic Measurement & Instruments (ICEMI), pp 387–391

  43. Taman J, Nkapkop JDD, Ijaz MF, Prasad PK, Tsafack N, Saha A, Kengne J, Son Y (2021) Dynamical properties of ion-acoustic waves in space plasma and its application to image encryption. IEEE Access 9:18762–18782

    Article  Google Scholar 

  44. Wang F, Liu E, Yang J, Liu R (2007) Target tracking based on a hybrid tracker with two hierarchical appearance models. Measure Sci Technol 18(8):2546

    Article  Google Scholar 

  45. Wang T, Qin R, Chen Y, Snoussi H, Choi C (2019) A reinforcement learning approach for UAV target searching and tracking. Multimed Tools Appl 78(4):4347–4364

    Article  Google Scholar 

  46. Weiwei K, Daibing Z, Shulong Z, Dianle Z, Boxin Z, Zhiwei Z, Zhaowei M, Dengqing T, Jianwei Z (2015) Autonomous track and land a MAV using a modified tracking-learning-detection framework. In: 2015 34th Chinese control conference (CCC), pp 5359–5366

  47. Wu Y, Shen B, Ling H (2012) Online robust image alignment via iterative convex optimization. In: 2012 IEEE conference on computer vision and pattern recognition, pp 1808–1814

  48. Xu T, Huang C, He Q, Guan G, Zhang Y (2016) An improved TLD target tracking algorithm. In: 2016 IEEE international conference on information and automation (ICIA), pp 2051–2055

  49. Yang, X, Zhu, S, Xia, S, Zhou, D, (2019) A new TLD target tracking method based on improved correlation filter and adaptive scale. Vis Comput, pp.1-13

  50. Yu L, Zheng T, Shi Q (2016) Image tracking algorithm improvement based on TLD frame. J Sign Process Image Process Pattern Recogn 9(5):431–440

    Google Scholar 

  51. Yue C, Yu W, Hou, Z, Ma S (2020) Visual tracking algorithm based on color name histogram. IOP Conf Ser: Mater Sci Eng 790(1):012063

  52. Zanina MA, Pavlov VA, Zavjalov SV, Volvenko SV (2018) TLD object tracking algorithm improved with particle filter. In: 2018 41st international conference on telecommunications and signal processing (TSP), pp 1–4

  53. Zhang T, Fei S, Lu H, Li X (2009) Modified particle filter for object tracking in low frame rate video. In: Proceedings of the 48h IEEE conference on decision and control (CDC) held jointly with 2009 28th Chinese control conference, pp 2552–2557

  54. Zhang X, Hu W, Xie N, Bao H, Maybank S (2015) A robust tracking system for low frame rate video. Int J Comput Vis 115(3):279–304

    Article  MathSciNet  Google Scholar 

  55. Zhang J, Wang A, Wang M, Iwahori Y (2017) A novel target algorithm based on TLD combining with SLBP. Int J Performability Eng 13(4):458–468

  56. Zhang L, Hou J, Chen M, Li H (2020) The improved TLD algorithm that combines CAMShift and orientation prediction to realize face tracking in low illumination. In: 2020 Chinese automation congress (CAC), pp 3193–3200

  57. Zhang J, Lv X, Cheng C, Long Z (2021) TLD tracking algorithm based on feature fusion. In: 2021 China automation congress (CAC), pp 310–315

  58. Zhao L, Chen Y, Ye Q (2017) An improved TLD algorithm based on Kalman filter and SURF feature matching. AIP Conf Proc 1839(1):020214

  59. Zhen X, Fei S, Wang Y, Du W (2020) A visual object tracking algorithm based on improved TLD. Algorithms 13(1):15

    Article  Google Scholar 

  60. Zhong W, Lu H, Yang MH (2012) Robust object tracking via sparsity-based collaborative model. In: 2012 IEEE conference on computer vision and pattern recognition, pp 1838–1845

  61. Zhou J, Liu S, Qiu G, Zhang F, Sun J (2013) Tensor missing value recovery with tucker thresholding method. In: 2013 5th international conference on intelligent networking and collaborative systems, pp 716–720

  62. Zhu XJ (2005) Semi-supervised learning literature survey. http://digital.library.wisc.edu/1793/60444

Download references

Author information

Authors and Affiliations

  1. Department of Electrical and Computer Engineering, Science and Research Branch, Islamic Azad University, Tehran, Iran

    Hooman Moridvaisi, Farbod Razzazi, Mohammadali Pourmina & Massoud Dousti

Authors
  1. Hooman Moridvaisi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Farbod Razzazi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mohammadali Pourmina
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Massoud Dousti
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Farbod Razzazi.

Ethics declarations

Conflict of interests/competing interests

The authors certify that the work is original. There is no conflict of interest by anyone or any organization, no funding has been received for the conduct of this study and/or preparation of this manuscript.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Moridvaisi, H., Razzazi, F., Pourmina, M. et al. An extended TLD tracking algorithm using co-training learning for low frame rate videos. Multimed Tools Appl 82, 24743–24769 (2023). https://doi.org/10.1007/s11042-022-14106-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-14106-1

Keywords

Search

Navigation