The Multi-strand Graph for a PTZ Tracker

  • Shachaf Melman
  • Yael Moses
  • Gérard Medioni
  • Yinghao Cai
Article
  • 216 Downloads

Abstract

High-resolution images can be used to resolve matching ambiguities between trajectory fragments (tracklets), which is a key challenge in multiple-target tracking. A pan–tilt–zoom (PTZ) camera, which can pan, tilt and zoom, is a powerful and efficient tool that offers both close-up views and wide area coverage on demand. The wide area enables tracking of many targets, while the close-up view allows individuals to be identified from high-resolution images of their faces. A central component of a PTZ tracking system is a scheduling algorithm that determines which target to zoom in on, particularly when the high-resolution images are also used for tracklet matching. In this paper, we study this scheduling problem from a theoretical perspective. We propose a novel data structure, the Multi-strand Tracking Graph (MSG), which represents the set of tracklets computed by a tracker and the possible associations between them. The MSG allows efficient scheduling as well as resolving of matching ambiguities between tracklets. The main feature of the MSG is the auxiliary data saved in each vertex, which allows efficient computation while avoiding time-consuming graph traversal. Synthetic data simulations are used to evaluate our scheduling algorithm and to demonstrate its superiority over a naïve one.

Keywords

Tracking Scheduling Data structure PTZ camera 

Notes

Acknowledgements

This research was supported by the Israeli Ministry of Science, Grant No. 3-8700 and by Award No. 2011-IJ-CX-K054, awarded by the National Institute of Justice, Office of Justice Programs, U.S. Department of Justice.

Supplementary material

Supplementary material 1 (avi 49205 KB)

References

  1. 1.
    Bagdanov, A., del Bimbo, A., Pernici, F.: Acquisition of high-resolution images through on-line saccade sequence planning. In: International Workshop on Video Surveillance and Sensor Networks (VSSN) (2005)Google Scholar
  2. 2.
    Cai, Y., Medioni, G.: Persistent people tracking and face capture using a PTZ camera. Mach. Vis. Appl. 27(3), 397–413 (2016)CrossRefGoogle Scholar
  3. 3.
    Cai, Y., Medioni, G., Dinh, T.: Towards a practical PTZ face detection and tracking system. In: IEEE Winter Conference on Applications of Computer Vision (WACV) (2013)Google Scholar
  4. 4.
    Costello, C., Diehl, C., Banerjee, A., Fisher, H.: Scheduling an active camera to observe people. In: International Workshop on Video Surveillance and Sensor Networks (VSSN) (2004)Google Scholar
  5. 5.
    Costello, C., Wang, I.: Surveillance camera coordination through distributed scheduling. In: IEEE Conference on Decision and Control and European Control Conference (CDC-ECC) (2005)Google Scholar
  6. 6.
    Del Bimbo, A., Pernici, F.: Towards on-line saccade planning for high-resolution image sensing. Pattern Recognit. Lett. 27(15), 1826–1834 (2006)CrossRefGoogle Scholar
  7. 7.
    Henriques, J., Caseiro, R., Batista, J.: Globally optimal solution to multi-object tracking with merged measurements. In: International Conference on Computer Vision (ICCV) (2011)Google Scholar
  8. 8.
    Krahnstoever, N., Yu, T., Lim, S., Patwardhan, K., Tu, P.: Collaborative real-time control of active cameras in large scale surveillance systems. In: Workshop on Multi-camera and Multi-modal Sensor Fusion Algorithms and Applications (2008)Google Scholar
  9. 9.
    Lim, S., Davis, L., Mittal, A.: Constructing task visibility intervals for video surveillance. Multimed. Syst. 12(3), 211–226 (2006)CrossRefGoogle Scholar
  10. 10.
    Lim, S., Davis, L., Mittal, A.: Task scheduling in large camera networks. In: Asian Conference on Computer Vision (2007)Google Scholar
  11. 11.
    Morye, A., Ding, C., Roy-Chowdhury, A., Farrell, J.: Distributed constrained optimization for bayesian opportunistic visual sensing. IEEE Trans. Control Syst. Technol. 22(6), 2302–2318 (2014)CrossRefGoogle Scholar
  12. 12.
    Natarajan, P., Hoang, T., Low, K., Kankanhalli, M.: Decision-theoretic approach to maximizing observation of multiple targets in multi-camera surveillance. In: International Conference on Autonomous Agents and Multiagent Systems (2012)Google Scholar
  13. 13.
    Neves, J.C., Proença, H.: Dynamic camera scheduling for visual surveillance in crowded scenes using Markov random fields. In: IEEE Conference on Advanced Video and Signal Based Surveillance (AVSS) (2015)Google Scholar
  14. 14.
    Nillius, P., Sullivan, J., Carlsson, S.: Multi-target tracking-linking identities using Bayesian network inference. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2006)Google Scholar
  15. 15.
    Prokaj, J., Duchaineau, M., Medioni, G.: Inferring tracklets for multi-object tracking. In: IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) (2011)Google Scholar
  16. 16.
    Qureshi, F., Terzopoulos, D.: Surveillance camera scheduling: a virtual vision approach. Multimed. Syst. 12(3), 269–283 (2006)CrossRefGoogle Scholar
  17. 17.
    Qureshi, F., Terzopoulos, D.: Surveillance in virtual reality: system design and multi-camera control. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2007)Google Scholar
  18. 18.
    Qureshi, F., Terzopoulos, D.: Planning ahead for ptz camera assignment and handoff. In: ACM/IEEE International Conference on Distributed Smart Cameras (ICDSC) (2009)Google Scholar
  19. 19.
    Reta, C., Altamirano, L., Gonzalez, J.A., Medina-Carnicer, R.: Three hypothesis algorithm with occlusion reasoning for multiple people tracking. J. Electron. Imaging 24(1), 013015–013015 (2015)CrossRefGoogle Scholar
  20. 20.
    Salvagnini, P., Pernici, F., Cristani, M., Lisanti, G., Del Bimbo, A., Murino, V.: Non-myopic information theoretic sensor management of a single pan-tilt-zoom camera for multiple object detection and tracking. Comput. Vis. Image Underst. 134, 74–88 (2015)CrossRefGoogle Scholar
  21. 21.
    Salvagnini, P., Pernici, F., Cristani, M., Lisanti, G., Masi, I., Del Bimbo, A., Murino, V.: Information theoretic sensor management for multi-target tracking with a single pan-tilt-zoom camera. In: IEEE Winter Conference on Applications of Computer Vision (WACV) (2014)Google Scholar
  22. 22.
    Sekii, T.: Robust, real-time 3D tracking of multiple objects with similar appearances. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016)Google Scholar
  23. 23.
    Sommerlade, E., Reid, I.: Information-theoretic active scene exploration. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2008)Google Scholar
  24. 24.
    Strat, T., Arambel, P., Antone, M., Rago, C., Landan, H.: A multiple-hypothesis tracking of multiple ground targets from aerial video with dynamic sensor control. In: Signal Processing, Sensor Fusion, and Target Recognition (SPIE) (2004)Google Scholar
  25. 25.
    Sullivan, J., Carlsson, S.: Tracking and labelling of interacting multiple targets. In: European Conference on Computer Vision (ECCV) (2006)Google Scholar
  26. 26.
    Wang, X., Türetken, E., Fleuret, F., Fua, P.: Tracking interacting objects optimally using integer programming. In: European Conference on Computer Vision (ECCV) (2014)Google Scholar
  27. 27.
    Ward, C., Naish, M.: Scheduling active camera resources for multiple moving targets. In: Canadian Conference on Electrical and Computer Engineering (CCECE) (2009)Google Scholar
  28. 28.
    Wheeler, F.W., Weiss, R.L., Tu, P.H.: Face recognition at a distance system for surveillance applications. In: IEEE International Conference on Biometrics: Theory Applications and Systems (BTAS) (2010)Google Scholar
  29. 29.
    Wu, Z., Kunz, T., Betke, M.: Efficient track linking methods for track graphs using network-flow and set-cover techniques. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2011)Google Scholar
  30. 30.
    Yang, B., Nevatia, R.: An online learned CRF model for multi-target tracking. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2012)Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2017

Authors and Affiliations

  1. 1.Tel Aviv UniversityTel AvivIsrael
  2. 2.The Interdisciplinary Center HerzliyaHerzliyaIsrael
  3. 3.University of Southern CaliforniaLos AngelesUSA

Personalised recommendations