A Benchmark and Simulator for UAV Tracking

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9905)

Abstract

In this paper, we propose a new aerial video dataset and benchmark for low altitude UAV target tracking, as well as, a photo-realistic UAV simulator that can be coupled with tracking methods. Our benchmark provides the first evaluation of many state-of-the-art and popular trackers on 123 new and fully annotated HD video sequences captured from a low-altitude aerial perspective. Among the compared trackers, we determine which ones are the most suitable for UAV tracking both in terms of tracking accuracy and run-time. The simulator can be used to evaluate tracking algorithms in real-time scenarios before they are deployed on a UAV “in the field”, as well as, generate synthetic but photo-realistic tracking datasets with automatic ground truth annotations to easily extend existing real-world datasets. Both the benchmark and simulator are made publicly available to the vision community on our website to further research in the area of object tracking from UAVs. (https://ivul.kaust.edu.sa/Pages/pub-benchmark-simulator-uav.aspx.).

Keywords

UAV tracking UAV simulator Aerial object tracking 

Notes

Acknowledgments

Research in this paper was supported by the King Abdullah University of Science and Technology (KAUST) Office of Sponsored Research.

Supplementary material

419956_1_En_27_MOESM1_ESM.pdf (2.7 mb)
Supplementary material 1 (pdf 2738 KB)

References

  1. 1.
    Babenko, B., Yang, M.H., Belongie, S.: Visual tracking with online multiple instance learning. IEEE Trans. Pattern Anal. Mach. Intell. 33(8), 1619–1632 (2010)CrossRefGoogle Scholar
  2. 2.
    Battaglia, P.W., Hamrick, J.B., Tenenbaum, J.B.: Simulation as an engine of physical scene understanding. Proc. Natl. Acad. Sci. 110(45), 18327–18332 (2013). http://www.pnas.org/content/110/45/18327.abstract
  3. 3.
    Bibi, A., Ghanem, B.: Multi-template scale-adaptive kernelized correlation filters. In: 2015 IEEE International Conference on Computer Vision Workshop (ICCVW), pp. 613–620, December 2015Google Scholar
  4. 4.
    Bibi, A., Mueller, M., Ghanem, B.: Target response adaptation for correlation filter tracking. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9910, pp. 419–433. Springer, Switzerland (2016)Google Scholar
  5. 5.
    Bolme, D.S., Beveridge, J.R., Draper, B.A., Lui, Y.M.: Visual object tracking using adaptive correlation filters. In: 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2544–2550, June 2010Google Scholar
  6. 6.
    Collins, R., Zhou, X., Teh, S.K.: An open source tracking testbed and evaluation web site. In: IEEE International Workshop on Performance Evaluation of Tracking and Surveillance (PETS 2005), January 2005Google Scholar
  7. 7.
    Danelljan, M., Hager, G., Shahbaz Khan, F., Felsberg, M.: Learning spatially regularized correlation filters for visual tracking. In: The IEEE International Conference on Computer Vision (ICCV), December 2015Google Scholar
  8. 8.
    Danelljan, M., Hger, G., Shahbaz Khan, F., Felsberg, M.: Accurate scale estimation for robust visual tracking. In: Proceedings of the British Machine Vision Conference. BMVA Press (2014)Google Scholar
  9. 9.
    Fu, C., Carrio, A., Olivares-Mendez, M., Suarez-Fernandez, R., Campoy, P.: Robust real-time vision-based aircraft tracking from unmanned aerial vehicles. In: 2014 IEEE International Conference on Robotics and Automation (ICRA), pp. 5441–5446, May 2014Google Scholar
  10. 10.
    Gaszczak, A., Breckon, T.P., Han, J.: Real-time people and vehicle detection from UAV imagery. In: Röning, J., Casasent, D.P., Hall, E.L. (eds.) IST/SPIE Electronic Imaging, vol. 7878, pp. 78780B-1–78780B-13. International Society for Optics and Photonics, January 2011Google Scholar
  11. 11.
    Grabner, H., Grabner, M., Bischof, H.: Real-time tracking via on-line boosting. In: Proceedings of the British Machine Vision Conference, pp. 6.1–6.10. BMVA Press (2006). doi: 10.5244/C.20.6
  12. 12.
    Hamalainen, P., Eriksson, S., Tanskanen, E., Kyrki, V., Lehtinen, J.: Online motion synthesis using sequential monte carlo. ACM Trans. Graph. 33(4), 51:1–51:12. http://doi.acm.org/10.1145/2601097.2601218
  13. 13.
    Hare, S., Saffari, A., Torr, P.H.S.: Struck: structured output tracking with kernels. In: 2011 International Conference on Computer Vision, pp. 263–270. IEEE, November 2011Google Scholar
  14. 14.
    Hattori, H., Naresh Boddeti, V., Kitani, K.M., Kanade, T.: Learning scene-specific pedestrian detectors without real data. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2015Google Scholar
  15. 15.
    Hejrati, M., Ramanan, D.: Analysis by synthesis: 3D object recognition by object reconstruction. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2449–2456, June 2014Google Scholar
  16. 16.
    Henriques, J.F., Caseiro, R., Martins, P., Batista, J.: High-speed tracking with kernelized correlation filters. IEEE Trans. Pattern Anal. Mach. Intell. 37(3), 583–596 (2015)CrossRefGoogle Scholar
  17. 17.
    Henriques, J.F., Caseiro, R., Martins, P., Batista, J.: Exploiting the circulant structure of tracking-by-detection with kernels. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7575, pp. 702–715. Springer, Heidelberg (2012). doi: 10.1007/978-3-642-33765-9_50 Google Scholar
  18. 18.
    Hong, Z., Chen, Z., Wang, C., Mei, X., Prokhorov, D., Tao, D.: Multi-store tracker (MUSTer): a cognitive psychology inspired approach to object tracking. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 749–758, June 2015Google Scholar
  19. 19.
    Jia, X., Lu, H., Yang, M.H.: Visual tracking via adaptive structural local sparse appearance model. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1822–1829, June 2012Google Scholar
  20. 20.
    Ju, E., Won, J., Lee, J., Choi, B., Noh, J., Choi, M.G.: Data-driven control of flapping flight. ACM Trans. Graph. 32(5), 151:1–151:12. http://doi.acm.org/10.1145/2516971.2516976
  21. 21.
    Kalal, Z., Mikolajczyk, K., Matas, J.: Tracking-learning-detection. IEEE Trans. Pattern Anal. Mach. Intell. 34(7), 1409–1422 (2011)CrossRefGoogle Scholar
  22. 22.
    Kendall, A., Salvapantula, N., Stol, K.: On-board object tracking control of a quadcopter with monocular vision. In: 2014 International Conference on Unmanned Aircraft Systems (ICUAS), pp. 404–411, May 2014Google Scholar
  23. 23.
    Kristan, M., et al.: The Visual Object Tracking VOT2014 challenge results. In: Agapito, L., Bronstein, M.M., Rother, C. (eds.) ECCV 2014. LNCS, vol. 8926, pp. 191–217. Springer, Switzerland (2015). doi: 10.1007/978-3-319-16181-5_14 Google Scholar
  24. 24.
    Li, A., Lin, M., Wu, Y., Yang, M.H., Yan, S.: NUS-PRO: a new visual tracking challenge. IEEE Trans. Pattern Anal. Mach. Intell. 38(2), 335–349 (2016)CrossRefGoogle Scholar
  25. 25.
    Liang, P., Blasch, E., Ling, H.: Encoding color information for visual tracking: algorithms and benchmark. IEEE Trans. Image Process. 24(12), 5630–5644 (2015)MathSciNetCrossRefGoogle Scholar
  26. 26.
    Liang, P., Blasch, E., Ling, H.: Encoding color information for visual tracking: algorithms and benchmark. IEEE Image Process. 24, 5630–5644 (2015). http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=7277070
  27. 27.
    Lim, H., Sinha, S.N.: Monocular localization of a moving person onboard a quadrotor MAV. In: 2015 IEEE International Conference on Robotics and Automation (ICRA), pp. 2182–2189, May 2015Google Scholar
  28. 28.
    Mueller, M., amd Neil Smith, G.S., Ghanem, B.: Persistent aerial tracking system for UAVs. In: 2016 IEEE/RSJ International Conference Intelligent Robots and Systems (IROS), October 2016Google Scholar
  29. 29.
    Naseer, T., Sturm, J., Cremers, D.: Followme: person following and gesture recognition with a quadrocopter. In: 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 624–630, November 2013Google Scholar
  30. 30.
    Nussberger, A., Grabner, H., Van Gool, L.: Aerial object tracking from an airborne platform. In: 2014 International Conference on Unmanned Aircraft Systems (ICUAS), pp. 1284–1293, May 2014Google Scholar
  31. 31.
    Papon, J., Schoeler, M.: Semantic pose using deep networks trained on synthetic RGB-D. CoRR abs/1508.00835 (2015). http://arxiv.org/abs/1508.00835
  32. 32.
    Pepik, B., Stark, M., Gehler, P., Schiele, B.: Teaching 3D geometry to deformable part models. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3362–3369, June 2012Google Scholar
  33. 33.
    Pestana, J., Sanchez-Lopez, J., Campoy, P., Saripalli, S.: Vision based GPS-denied object tracking and following for unmanned aerial vehicles. In: 2013 IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR), pp. 1–6, October 2013Google Scholar
  34. 34.
    Pollard, T., Antone, M.: Detecting and tracking all moving objects in wide-area aerial video. In: 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 15–22, June 2012Google Scholar
  35. 35.
    Portmann, J., Lynen, S., Chli, M., Siegwart, R.: People detection and tracking from aerial thermal views. In: 2014 IEEE International Conference on Robotics and Automation (ICRA), pp. 1794–1800, May 2014Google Scholar
  36. 36.
    Prokaj, J., Medioni, G.: Persistent tracking for wide area aerial surveillance. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1186–1193, June 2014Google Scholar
  37. 37.
    Qadir, A., Neubert, J., Semke, W., Schultz, R.: On-board visual tracking with Unmanned Aircraft System (UAS). In: Infotech@Aerospace Conferences. American Institute of Aeronautics and Astronautics, March 2011Google Scholar
  38. 38.
    Ross, D., Lim, J., Lin, R.S., Yang, M.H.: Incremental learning for robust visual tracking. Int. J. Comput. Vis. 77(1–3), 125–141 (2008)CrossRefGoogle Scholar
  39. 39.
    Smeulders, A.W.M., Chu, D.M., Cucchiara, R., Calderara, S., Dehghan, A., Shah, M.: Visual tracking: an experimental survey. IEEE Trans. Pattern Anal. Mach. Intell. 36(7), 1442–1468 (2014)CrossRefGoogle Scholar
  40. 40.
    Trilaksono, B.R., Triadhitama, R., Adiprawita, W., Wibowo, A., Sreenatha, A.: Hardware in the loop simulation for visual target tracking of octorotor UAV. Aircr. Eng. Aerosp. Technol. 83(6), 407–419 (2011). http://dx.doi.org/10.1108/00022661111173289
  41. 41.
    Wu, Y., Lim, J., Yang, M.H.: Object tracking benchmark. IEEE Trans. Pattern Anal. Mach. Intell. 37(9), 1834–1848 (2015)CrossRefGoogle Scholar
  42. 42.
    Wu, Y., Lim, J., Yang, M.H.: Online object tracking: a benchmark. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition, pp. 2411–2418. IEEE, June 2013Google Scholar
  43. 43.
    Zhang, J., Ma, S., Sclaroff, S.: MEEM: robust tracking via multiple experts using entropy minimization. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8694, pp. 188–203. Springer, Switzerland (2014). doi: 10.1007/978-3-319-10599-4_13 Google Scholar
  44. 44.
    Zhang, T., Bibi, A., Ghanem, B.: In defense of sparse tracking: circulant sparse tracker. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016Google Scholar
  45. 45.
    Zhang, T., Ghanem, B., Liu, S., Xu, C., Ahuja, N.: Robust visual tracking via exclusive context modeling. IEEE Trans. Cybern. 46(1), 51–63 (2016)CrossRefGoogle Scholar
  46. 46.
    Zhang, T., Ghanem, B., Xu, C., Ahuja, N.: Object tracking by occlusion detection via structured sparse learning. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 1033–1040, June 2013Google Scholar
  47. 47.
    Zhang, T., Liu, S., Xu, C., Yan, S., Ghanem, B., Ahuja, N., Yang, M.H.: Structural sparse tracking. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 150–158, June 2015Google Scholar

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  1. 1.King Abdullah University of Science and Technology (KAUST)ThuwalSaudi Arabia

Personalised recommendations