Advertisement

Real-Time ‘Actor-Critic’ Tracking

  • Boyu Chen
  • Dong WangEmail author
  • Peixia Li
  • Shuang Wang
  • Huchuan Lu
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11211)

Abstract

In this work, we propose a novel tracking algorithm with real-time performance based on the ‘Actor-Critic’ framework. This framework consists of two major components: ‘Actor’ and ‘Critic’. The ‘Actor’ model aims to infer the optimal choice in a continuous action space, which directly makes the tracker move the bounding box to the object’s location in the current frame. For offline training, the ‘Critic’ model is introduced to form a ‘Actor-Critic’ framework with reinforcement learning and outputs a Q-value to guide the learning process of both ‘Actor’ and ‘Critic’ deep networks. Then, we modify the original deep deterministic policy gradient algorithm to effectively train our ‘Actor-Critic’ model for the tracking task. For online tracking, the ‘Actor’ model provides a dynamic search strategy to locate the tracked object efficiently and the ‘Critic’ model acts as a verification module to make our tracker more robust. To the best of our knowledge, this work is the first attempt to exploit the continuous action and ‘Actor-Critic’ framework for visual tracking. Extensive experimental results on popular benchmarks demonstrate that the proposed tracker performs favorably against many state-of-the-art methods, with real-time performance.

Keywords

Visual tracking Real-time tracking Reinforcement learning 

Notes

Acknowledgment

This paper was supported in part by the Natural Science Foundation of China \(\#\)61751212, \(\#\)61502070, \(\#\)61725202, \(\#\)61771088, \(\#\)61472060, \(\#\)61632006, \(\#\)91538201, and in part by the Fundamental Research Funds for the Central Universities under Grant \(\#\)DUT18JC30. This work was also supported by Alibaba Group through Alibaba Innovative Research (AIR) program.

References

  1. 1.
    Yilmaz, A., Javed, O., Shah, M.: Object tracking: a survey. ACM Comput. Surv. 38(4), 13 (2006)CrossRefGoogle Scholar
  2. 2.
    Li, X., Hu, W., Shen, C., Zhang, Z., Dick, A.R., van den Hengel, A.: A survey of appearance models in visual object tracking. ACM Trans. Intell. Syst. Technol. 4(4), 58:1–58:48 (2013)CrossRefGoogle Scholar
  3. 3.
    Li, P., Wang, D., Wang, L., Lu, H.: Deep visual tracking: review and experimental comparison. Pattern Recognit. 76, 323–338 (2018)CrossRefGoogle Scholar
  4. 4.
    Ma, C., Huang, J.B., Yang, X., Yang, M.H.: Hierarchical convolutional features for visual tracking. In: ICCV (2015)Google Scholar
  5. 5.
    Nam, H., Han, B.: Learning multi-domain convolutional neural networks for visual tracking. In: CVPR (2016)Google Scholar
  6. 6.
    Bertinetto, L., Valmadre, J., Henriques, J.F., Vedaldi, A., Torr, P.H.S.: Fully-convolutional siamese networks for object tracking. In: Hua, G., Jégou, H. (eds.) ECCV 2016. LNCS, vol. 9914, pp. 850–865. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-48881-3_56CrossRefGoogle Scholar
  7. 7.
    Danelljan, M., Bhat, G., Khan, F.S., Felsberg, M.: ECO: efficient convolution operators for tracking. In: CVPR (2017)Google Scholar
  8. 8.
    Wu, Y., Lim, J., Yang, M.: Object tracking benchmark. IEEE Trans. Pattern Anal. Mach. Intell. 37(9), 1834–1848 (2015)CrossRefGoogle Scholar
  9. 9.
    Kristan, M., et al.: The visual object tracking VOT2015 challenge results. In: ICCV (2015)Google Scholar
  10. 10.
    Yun, S., Choi, J., Yoo, Y., Yun, K., Choi, J.Y.: Action-decision networks for visual tracking with deep reinforcement learning. In: CVPR (2017)Google Scholar
  11. 11.
    Grabner, H., Bischof, H.: On-line boosting and vision. In: CVPR (2006)Google Scholar
  12. 12.
    Ross, D.A., Lim, J., Lin, R., Yang, M.: Incremental learning for robust visual tracking. Int. J. Comput. Vis. 77(1–3), 125–141 (2008)CrossRefGoogle Scholar
  13. 13.
    Babenko, B., Yang, M., Belongie, S.J.: Robust object tracking with online multiple instance learning. IEEE Trans. Pattern Anal. Mach. Intell. 33(8), 1619–1632 (2011)CrossRefGoogle Scholar
  14. 14.
    Jia, X., Lu, H., Yang, M.: Visual tracking via adaptive structural local sparse appearance model. In: CVPR (2012)Google Scholar
  15. 15.
    Zhong, W., Lu, H., Yang, M.: Robust object tracking via sparse collaborative appearance model. IEEE Trans. Image Process. 23(5), 2356–2368 (2014)MathSciNetCrossRefGoogle Scholar
  16. 16.
    Hare, S., et al.: Struck: structured output tracking with kernels. IEEE Trans. Pattern Anal. Mach. Intell. 38(10), 2096–2109 (2016)CrossRefGoogle Scholar
  17. 17.
    Li, Z., Zhang, J., Zhang, K., Li, Z.: Visual tracking with weighted adaptive local sparse appearance model via spatio-temporal context learning. IEEE Trans. Image Process. 27(9), 4478–4489 (2018)MathSciNetCrossRefGoogle Scholar
  18. 18.
    Wang, N., Yeung, D.: Learning a deep compact image representation for visual tracking. In: NIPS (2013)Google Scholar
  19. 19.
    Li, H., Li, Y., Porikli, F.: DeepTrack: learning discriminative feature representations by convolutional neural networks for visual tracking. In: BMVC (2014)Google Scholar
  20. 20.
    Henriques, J.F., Rui, C., Martins, P., Batista, J.: High-speed tracking with kernelized correlation filters. IEEE Trans. Pattern Anal. Mach. Intell. 37(3), 583–596 (2015)CrossRefGoogle Scholar
  21. 21.
    Zhu, G., Wang, J., Wu, Y., Zhang, X., Lu, H.: MC-HOG correlation tracking with saliency proposal. In: AAAI (2016)Google Scholar
  22. 22.
    Bertinetto, L., Valmadre, J., Golodetz, S., Miksik, O., Torr, P.H.S.: Staple: complementary learners for real-time tracking. In: CVPR (2016)Google Scholar
  23. 23.
    Li, Y., Zhu, J.: A scale adaptive kernel correlation filter tracker with feature integration. In: Agapito, L., Bronstein, M.M., Rother, C. (eds.) ECCV 2014. LNCS, vol. 8926, pp. 254–265. Springer, Cham (2015).  https://doi.org/10.1007/978-3-319-16181-5_18CrossRefGoogle Scholar
  24. 24.
    Danelljan, M., Häger, G., Khan, F.S., Felsberg, M.: Discriminative scale space tracking. IEEE Trans. Pattern Anal. Mach. Intell. 39(8), 1561–1575 (2017)CrossRefGoogle Scholar
  25. 25.
    Li, Y., Zhu, J., Hoi, S.C.H.: Reliable patch trackers: robust visual tracking by exploiting reliable patches. In: CVPR (2015)Google Scholar
  26. 26.
    Liu, T., Wang, G., Yang, Q.: Real-time part-based visual tracking via adaptive correlation filters. In: CVPR (2015)Google Scholar
  27. 27.
    Tang, M., Feng, J.: Multi-kernel correlation filter for visual tracking. In: ICCV (2015)Google Scholar
  28. 28.
    Zhang, T., Xu, C., Yang, M.: Multi-task correlation particle filter for robust object tracking. In: CVPR (2017)Google Scholar
  29. 29.
    Danelljan, M., Hager, G., Khan, F.S., Felsberg, M.: Learning spatially regularized correlation filters for visual tracking. In: ICCV (2015)Google Scholar
  30. 30.
    Galoogahi, H.K., Fagg, A., Lucey, S.: Learning background-aware correlation filters for visual tracking. In: ICCV (2017)Google Scholar
  31. 31.
    Danelljan, M., Robinson, A., Shahbaz Khan, F., Felsberg, M.: Beyond correlation filters: learning continuous convolution operators for visual tracking. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9909, pp. 472–488. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-46454-1_29CrossRefGoogle Scholar
  32. 32.
    Sun, C., Wang, D., Lu, H., Yang, M.H.: Learning spatial-aware regressions for visual tracking. In: CVPR (2018)Google Scholar
  33. 33.
    Sun, C., Wang, D., Lu, H., Yang, M.H.: Correlation tracking via joint discrimination and reliability learning. In: CVPR (2018)Google Scholar
  34. 34.
    Comaniciu, D., Ramesh, V., Meer, P.: Kernel-based object tracking. IEEE Trans. Pattern Anal. Mach. Intell. 25(5), 564–575 (2003)CrossRefGoogle Scholar
  35. 35.
    Baker, S., Matthews, I.A.: Lucas-kanade 20 years on: a unifying framework. Int. J. Comput. Vis. 56(3), 221–255 (2004)CrossRefGoogle Scholar
  36. 36.
    Jiang, N., Liu, W., Wu, Y.: Learning adaptive metric for robust visual tracking. IEEE Trans. Image Process. 20(8), 2288–2300 (2011)MathSciNetCrossRefGoogle Scholar
  37. 37.
    Li, P., Wang, Q.: Robust registration-based tracking by sparse representation with model update. In: Lee, K.M., Matsushita, Y., Rehg, J.M., Hu, Z. (eds.) ACCV 2012. LNCS, vol. 7726, pp. 205–216. Springer, Heidelberg (2013).  https://doi.org/10.1007/978-3-642-37431-9_16CrossRefGoogle Scholar
  38. 38.
    Ning, J., Zhang, L., Zhang, D., Wu, C.: Robust mean-shift tracking with corrected background-weighted histogram. IET Comput. Vis. 6(1), 62–69 (2012)MathSciNetCrossRefGoogle Scholar
  39. 39.
    Oron, S., Bar-Hille, A., Avidan, S.: Extended Lucas-Kanade tracking. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8693, pp. 142–156. Springer, Cham (2014).  https://doi.org/10.1007/978-3-319-10602-1_10CrossRefGoogle Scholar
  40. 40.
    Caicedo, J.C., Lazebnik, S.: Active object localization with deep reinforcement learning. In: ICCV (2015)Google Scholar
  41. 41.
    Bellver, M., Giro-i Nieto, X., Marques, F., Torres, J.: Hierarchical object detection with deep reinforcement learning. In: NIPS (2016)Google Scholar
  42. 42.
    Jayaraman, D., Grauman, K.: Look-ahead before you leap: end-to-end active recognition by forecasting the effect of motion. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9909, pp. 489–505. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-46454-1_30CrossRefGoogle Scholar
  43. 43.
    Huang, C., Lucey, S., Ramanan, D.: Learning policies for adaptive tracking with deep feature cascades. In: ICCV (2017)Google Scholar
  44. 44.
    Supancic III, J.S., Ramanan, D.: Tracking as online decision-making: learning a policy from streaming videos with reinforcement learning. In: ICCV (2017)Google Scholar
  45. 45.
    Choi, J., Kwon, J., Lee, K.M.: Visual tracking by reinforced decision making. CoRR abs/1702.06291 (2017)Google Scholar
  46. 46.
    Lillicrap, T.P., et al.: Continuous control with deep reinforcement learning. CoRR abs/1509.02971 (2015)Google Scholar
  47. 47.
    Russakovsky, O., et al.: ImageNet large scale visual recognition challenge. Int. J. Comput. Vis. 115(3), 211–252 (2015)MathSciNetCrossRefGoogle Scholar
  48. 48.
    Wu, Y., Lim, J., Yang, M.H.: Online object tracking: a benchmark. In: CVPR (2013)Google Scholar
  49. 49.
    Kristan, M., et al.: The visual object tracking VOT2016 challenge results. In: Hua, G., Jégou, H. (eds.) ECCV 2016. LNCS, vol. 9914, pp. 777–823. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-48881-3_54CrossRefGoogle Scholar
  50. 50.
    Fan, H., Ling, H.: Parallel tracking and verifying: a framework for real-time and high accuracy visual tracking. In: ICCV (2017)Google Scholar
  51. 51.
    Valmadre, J., Bertinetto, L., Henriques, J.F., Vedaldi, A., Torr, P.H.S.: End-to-end representation learning for correlation filter based tracking. In: CVPR (2017)Google Scholar
  52. 52.
    Choi, J., Chang, H.J., Yun, S., Fischer, T., Demiris, Y., Choi, J.Y.: Attentional correlation filter network for adaptive visual tracking. In: CVPR (2017)Google Scholar
  53. 53.
    Ma, C., Yang, X., Zhang, C., Yang, M.: Long-term correlation tracking. In: CVPR (2015)Google Scholar
  54. 54.
    Wang, M., Liu, Y., Huang, Z.: Large margin object tracking with circulant feature maps. In: CVPR (2017)Google Scholar
  55. 55.
    Danelljan, M., Häger, G., Khan, F.S., Felsberg, M.: Accurate scale estimation for robust visual tracking. In: BMVC (2014)Google Scholar
  56. 56.
    Wang, N., Li, S., Gupta, A., Yeung, D.: Transferring rich feature hierarchies for robust visual tracking. CoRR abs/1501.04587 (2015)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.School of Information and Communication EngineeringDalian University of TechnologyDalianChina
  2. 2.Alibaba GroupHangzhouChina

Personalised recommendations