RGB-D Tracking Based on Kernelized Correlation Filter with Deep Features

  • Shuang Gu
  • Yao Lu
  • Lin Zhang
  • Jian Zhang
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10636)


This paper proposes a new RGB-D tracker which is upon Kernelized Correlation Filter(KCF) with deep features. KCF is a high-speed target tracker. However, the HOG feature used in KCF shows some weaknesses, such as not robust to noise. Therefore, we consider using RGB-D deep features in KCF, which refer to deep features of RGB and depth images and the deep features contain abundant and discriminated information for tracking. The mixture of deep features highly improves the performance of the tracker. Besides, KCF is sensitive to scale variations while depth images benefit for handling this problem. According to the principle of similar triangle, the ratio of scale variation can be observed simply. Tested over Princeton RGB-D Tracking Benchmark, Our RGB-D tracker achieves the highest accuracy when no occlusion happens. Meanwhile, we keep the high-speed tracking even if deep features are calculated during tracking and the average speed is 10 FPS.


RGB-D KCF Deep features Scale estimation 



This work is supported by the National Natural Science Foundation of China (No. 61273273) and by Research Fund for the Doctoral Program of Higher Education of China (No. 20121101110034).


  1. 1.
    Awwad, S., Piccardi, M.: Prototype-based budget maintenance for tracking in depth videos. Multimedia Tools Appl. 1–16 (2016)Google Scholar
  2. 2.
    Bibi, A., Zhang, T., Ghanem, B.: 3D part-based sparse tracker with automatic synchronization and registration. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1439–1448 (2016)Google Scholar
  3. 3.
    Danelljan, M., Hager, G., Khan, F.S., Felsberg, M.: Discriminative scale space tracking. IEEE Trans. Pattern Anal. Mach. Intell. 39(8), 1561–1575 (2016)CrossRefGoogle Scholar
  4. 4.
    Danelljan, M., Shahbaz Khan, F., Felsberg, M., Van de Weijer, J.: Adaptive color attributes for real-time visual tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1090–1097 (2014)Google Scholar
  5. 5.
    Hannuna, S., Camplani, M., Hall, J., Mirmehdi, M., Damen, D., Burghardt, T., Paiement, A., Tao, L.: DS-KCF: a real-time tracker for RGB-D data. J. Real-Time Image Proc. 1–20 (2016)Google Scholar
  6. 6.
    Henriques, J.F., Caseiro, R., Martins, P., Batista, J.: High-speed tracking with kernelized correlation filters. IEEE Trans. Pattern Anal. Mach. Intell. 37(3), 583–596 (2015)CrossRefGoogle Scholar
  7. 7.
    Kang, K., Li, H., Yan, J., Zeng, X., Yang, B., Xiao, T., Zhang, C., Wang, Z., Wang, R., Wang, X., et al.: T-cnn: Tubelets with convolutional neural networks for object detection from videos. arXiv preprint arXiv:1604.02532 (2016)
  8. 8.
    Li, Y., Zhu, J.: A Scale Adaptive Kernel Correlation Filter Tracker with Feature Integration. In: Agapito, L., Bronstein, M.M., Rother, C. (eds.) ECCV 2014. LNCS, vol. 8926, pp. 254–265. Springer, Cham (2015). doi: 10.1007/978-3-319-16181-5_18 Google Scholar
  9. 9.
    Meshgi, K., Maeda, S.I., Oba, S., Skibbe, H., Li, Y.Z., Ishii, S.: An occlusion-aware particle filter tracker to handle complex and persistent occlusions. Comput. Vis. Image Underst. 150, 81–94 (2016)CrossRefGoogle Scholar
  10. 10.
    Nam, H., Han, B.: Learning multi-domain convolutional neural networks for visual tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4293–4302 (2016)Google Scholar
  11. 11.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
  12. 12.
    Song, S., Xiao, J.: Tracking revisited using rgbd camera: Baseline and benchmark. arXiv preprint arXiv:1212.2823 (2012)

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.Beijing Laboratory of Intelligent Information Technology, School of Computer ScienceBeijing Institute of TechnologyBeijingChina
  2. 2.Advanced Analytics InstituteUniversity of Technology SydneySydneyAustralia

Personalised recommendations