Real-time object tracking based on an adaptive transition model and extended Kalman filter to handle full occlusion

  • Mohammad Zolfaghari
  • Hossein Ghanei-YakhdanEmail author
  • Mehran Yazdi
Original Article


In this paper, a tracker scheme is proposed that not only can face object tracking challenges but also can estimate object positions over occluded frames. In the proposed scheme, kernelized correlation filter (KCF) is considered as our basic tracker due to its high efficiency in the most object tracking challenges except occlusion and illumination variation. To improve the efficiency of KCF, the proposed method integrates an occlusion detection method, an adaptive model update and a prediction system into the KCF tracker. The occlusion detection method is based on the peak-to-sidelobe ratio of the confidence map to determine the type of occlusion. When an object is partially occluded, the object appearance model is adaptively updated to increase the accuracy of object tracking. When full occlusion occurs, the proposed predictor is run and exploits the available motion information before the occurrence of full occlusion to predict the location of the tracked object. The proposed predictor uses adaptive transition state equations to estimate the acceleration and velocity of the object needed in the extended Kalman filter (EKF) to predict object position. It also uses two quadratic equations to estimate the object trajectory. Finally, a method is proposed that exploits the estimated object positions by both EKF and the object trajectory to predict object positions over fully occluded frames. Experimental results on open datasets show that the proposed method achieved a better performance in comparison with several state-of-the-art trackers.


Extended Kalman filter Adaptive state transition model Full occlusion Model update Occlusion detection 


Compliance with ethical standards

Conflict of interest

All the authors declare that they have no conflict of interest.


  1. 1.
    Adhikari, G., Sahani, S.K., Chauhan, M.S., Das, B.K.: Fast real time object tracking based on normalized cross correlation and importance of thresholding segmentation. In: 2016 International Conference on Recent Trends in Information Technology (ICRTIT), pp. 1–5 (2016)Google Scholar
  2. 2.
    Yilmaz, A., Javed, O., Shah, M.: Object tracking: a survey. ACM Comput. Surv (CSUR) 38, 13 (2006)CrossRefGoogle Scholar
  3. 3.
    Liu, T., Wang, G., Yang, Q.: Real-time part-based visual tracking via adaptive correlation filters. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4902–4912 (2015)Google Scholar
  4. 4.
    Liu, Q., Zhao, X., Hou, Z.: Survey of single-target visual tracking methods based on online learning. IET Comput. Vision 8, 419–428 (2014)CrossRefGoogle Scholar
  5. 5.
    Wu, Y., Lim, J., Yang, M.-H.: Object tracking benchmark. IEEE Trans. Pattern Anal. Mach. Intell. 37, 1834–1848 (2015)CrossRefGoogle Scholar
  6. 6.
    Henriques, J.F., Caseiro, R., Martins, P., Batista, J.: High-speed tracking with kernelized correlation filters. IEEE Trans. Pattern Anal. Mach. Intell. 37, 583–596 (2015)CrossRefGoogle Scholar
  7. 7.
    Bolme, D.S., Beveridge, J.R., Draper, B.A., Lui, Y.M.: Visual object tracking using adaptive correlation filters. In: 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2544–2550 (2010)Google Scholar
  8. 8.
    Henriques, J.F., Caseiro, R, Martins, P., Batista, J.: Exploiting the circulant structure of tracking-by-detection with kernels. In: European Conference on Computer Vision, pp. 702–715 (2012)Google Scholar
  9. 9.
    Felzenszwalb, P.F., Girshick, R.B., McAllester, D., Ramanan, D.: Object detection with discriminatively trained part-based models. IEEE Trans. Pattern Anal. Mach. Intell. 32, 1627–1645 (2010)CrossRefGoogle Scholar
  10. 10.
    Van De Weijer, J., Schmid, C., Verbeek, J., Larlus, D.: Learning color names for real-world applications. IEEE Trans. Image Process. 18, 1512–1523 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Fazl-Ersi, E., Nooghabi, M.K.: Revisiting correlation-based filters for low-resolution and long-term visual tracking. Vis. Comput. 1–13 (2018).
  12. 12.
    Ma, C., Huang, J.-B., Yang, X., Yang, M.-H.: Hierarchical convolutional features for visual tracking. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 3074–3082 (2015)Google Scholar
  13. 13.
    Wu, Y., Lim, J., Yang, M.-H.: Online object tracking: a benchmark. In: 2013 IEEE Conference on Computer vision and pattern recognition (CVPR), pp. 2411–2418 (2013)Google Scholar
  14. 14.
    Li, X., Hu, W., Shen, C., Zhang, Z., Dick, A., Hengel, A.V.D.: A survey of appearance models in visual object tracking. ACM Trans. Intell. Syst. Technol. 4, 58 (2013)Google Scholar
  15. 15.
    Adam, A., Rivlin, E., Shimshoni, I.: Robust fragments-based tracking using the integral histogram. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06), pp. 798–805 (2006)Google Scholar
  16. 16.
    Iraei, I., Faez, K.: Object tracking with occlusion handling using mean shift, Kalman filter and edge histogram. In 2015 2nd International Conference on Pattern Recognition and Image Analysis, pp. 4799–8445 (2015)Google Scholar
  17. 17.
    Li, D., Xu, L., Wu, Y.: Improved CAMShift object tracking based on Epanechnikov Kernel density estimation and Kalman filter. In: Control And Decision Conference (CCDC), 2017 29th Chinese, pp. 3120–3126 (2017)Google Scholar
  18. 18.
    Jeong, J.-M., Yoon, T.-S., Park, J.-B.: Kalman filter based multiple objects detection-tracking algorithm robust to occlusion. In: 2014 Proceedings of the SICE Annual Conference (SICE), pp. 941–946 (2014)Google Scholar
  19. 19.
    Tripathi, R.P., Ghosh, S., Chandle, J.: Tracking of object using optimal adaptive Kalman filter. In: 2016 IEEE International Conference on Engineering and Technology (ICETECH), pp. 1128–1131 (2016)Google Scholar
  20. 20.
    Xiao, J., Oussalah, M.: Collaborative tracking for multiple objects in the presence of inter-occlusions. IEEE Trans. Circuits Syst. Video Technol. 26, 304–318 (2016)CrossRefGoogle Scholar
  21. 21.
    Ma, C., Yang, X., Zhang, C., Yang, M.-H.: Long-term correlation tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5388–5396 (2015)Google Scholar
  22. 22.
    Mu, X., Che, J., Hu, T., Wang, Z.: A video object tracking algorithm combined Kalman filter and adaptive least squares under occlusion. In: International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI), pp. 6–10 (2016)Google Scholar
  23. 23.
    Aishwarya, R., Maik, V., Chithravathi, B.: Robust object tracking using kernalized correlation filters (KCF) and Kalman predictive estimates. In 2017 International Conference on Recent Trends in Electronics Information and Communication Technology, pp. 587–591 (2017)Google Scholar
  24. 24.
    Iswanto, I.A., Li, B.: Visual object tracking based on mean-shift and particle-Kalman filter. Proc. Comput. Sci. 116, 587–595 (2017)CrossRefGoogle Scholar
  25. 25.
    Wang, J., He, F., Zhang, X., Gao, Y.: Tracking objects through occlusions using improved Kalman filter. In: 2010 2nd International Conference on Advanced Computer Control (ICACC), pp. 223–228 (2010)Google Scholar
  26. 26.
    Danelljan, M., Häger, G., Khan, F.S., Felsberg, M.: Discriminative scale space tracking. IEEE Trans. Pattern Anal. Mach. Intell. 39, 1561–1575 (2017)CrossRefGoogle Scholar
  27. 27.
    Hu, Q., Guo, Y., Lin, Z., An, W., Cheng, H.: Robust and real-time object tracking using scale-adaptive correlation filters. In 2016 International Conference on Digital Image Computing: Techniques and Applications (DICTA), pp. 1–8 (2016)Google Scholar
  28. 28.
    Danelljan, M., Häger, G., Khan, F., Felsberg, M.: Accurate scale estimation for robust visual tracking. In: British Machine Vision Conference, Nottingham, September 1–5, 2014, pp. 1–65 (2014)Google Scholar
  29. 29.
    Schölkopf, B., Smola, A.J.: Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. MIT Press, Cambridge (2002)Google Scholar
  30. 30.
    Simon, D.: Optimal State Estimation: Kalman, H Infinity, and Nonlinear Approaches. Wiley, New York (2006)CrossRefGoogle Scholar
  31. 31.
    Zhang, H., Liu, G.: Coupled-layer based visual tracking via adaptive kernelized correlation filters. Vis Comput 34, 41–54 (2018)CrossRefGoogle Scholar
  32. 32.
    Hadi, I., Sabah, M.: Behavior formula extraction for object trajectory using curve fitting method. Int. J. Comput. Appl. 104, 28–37 (2014)Google Scholar
  33. 33.
    Kristan, M., Matas, J., Leonardis, A., Felsberg, M., Cehovin, L., Fernandez, G., et al.: The visual object tracking vot2015 challenge results. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, pp. 1–23 (2015)Google Scholar
  34. 34.
    Ross, D.A., Lim, J., Lin, R.-S., Yang, M.-H.: Incremental learning for robust visual tracking. Int. J. Comput. Vis. 77, 125–141 (2008)CrossRefGoogle Scholar
  35. 35.
    Babenko, B., Yang, M.-H., Belongie, S.: Visual tracking with online multiple instance learning. In: IEEE Conference on Computer Vision and Pattern Recognition, 2009. CVPR 2009, pp. 983–990 (2009)Google Scholar
  36. 36.
    Hare, S., Golodetz, S., Saffari, A., Vineet, V., Cheng, M.-M., Hicks, S.L., et al.: Struck: structured output tracking with kernels. IEEE Trans. Pattern Anal. Mach. Intell. 38, 2096–2109 (2016)CrossRefGoogle Scholar
  37. 37.
    Bao, C., Wu, Y., Ling, H., Ji, H.: Real time robust l1 tracker using accelerated proximal gradient approach. In 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1830–1837 (2012)Google Scholar
  38. 38.
    Zhang, K., Liu, Q., Wu, Y., Yang, M.-H.: Robust visual tracking via convolutional networks without training. IEEE Trans. Image Process. 25, 1779–1792 (2016)MathSciNetzbMATHGoogle Scholar
  39. 39.
    Kalal, Z., Matas, J., Mikolajczyk, K.: Pn learning: Bootstrapping binary classifiers by structural constraints. In: 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 49–56 (2010)Google Scholar
  40. 40.
    Ahmed, J., Jafri, M.N., Shah, M., Akbar, M.: Real-time edge-enhanced dynamic correlation and predictive open-loop car-following control for robust tracking. Mach. Vis. Appl. 19, 1–25 (2008)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Department of Electrical EngineeringYazd UniversityYazdIran
  2. 2.Department of Electronics and Telecommunication EngineeringShiraz UniversityShirazIran

Personalised recommendations