Advertisement

Machine Vision and Applications

, Volume 27, Issue 3, pp 397–413 | Cite as

Persistent people tracking and face capture using a PTZ camera

  • Yinghao Cai
  • Gérard Medioni
Original Paper

Abstract

Pan–tilt–zoom (PTZ) camera is a powerful tool in far-field scenarios. However, most of the current PTZ surveillance systems require manual intervention to move the camera to the desired position. In this paper, we address the problem of persistent people tracking and face capture in uncontrolled scenarios using a single PTZ camera, which could prove most helpful in forensic applications. The system first detects and tracks pedestrians in zoomed-out mode. Then, according to a scheduler, the system selects a person to zoom in. In the zoomed-in mode, we detect a set of face images and solve the face–face association and face–person association problems. The system then zooms back out where tracking is continued as people re-appear in the view. The person–person association module associates the people on the schedule list with the people in the current view. The detected faces are associated with the corresponding people and trajectories. Due to the dynamic nature of our problem, e.g. the field of view of the camera changes because of the pan/tilt/zoom movement of the camera, all of the processes including receiving images from the camera and processing must be done in real time. To the best of our knowledge, the proposed method is the first to address the association of face images to people and trajectories using a single PTZ camera. Extensive experiments in challenging indoor and outdoor uncontrolled conditions demonstrate the effectiveness of the proposed system.

Keywords

Pan–tilt–zoom camera Multiple target tracking Face detection Pedestrian detection Face–person association 

Notes

Acknowledgments

This research was supported by Award No. 2011-IJ-CX-K054, awarded by the National Institute of Justice, Office of Justice Programs, U.S. Department of Justice and National Natural Science Foundation of China 61503381. The opinions, findings and conclusions or recommendations expressed in this publication are those of the authors and do not necessarily reflect those of the Department of Justice.

References

  1. 1.
    Andriluka, M., Roth, S., Schiele, B.: People-tracking-by-detection and people-detection-by-tracking. In: IEEE Conference on Computer Vision and Pattern Recognition (2008)Google Scholar
  2. 2.
    Babenko, B., Yang, M.H., Belongie, S.: Robust object tracking with online multiple instance learning. IEEE Trans Pattern Anal Mach Intell 33(8), 1619–1632 (2011)CrossRefGoogle Scholar
  3. 3.
    Bagdanov, A.D., Del Bimbo, A., Nunziati, W.: Improving evidential quality of surveillance imagery through active face tracking. In: International Conference on Pattern Recognition (2006)Google Scholar
  4. 4.
    Bagdanov, A.D., del Alberto, B., Federico, P.: Acquisition of high-resolution images through on-line saccade sequence planning. In: ACM International Workshop on Video Surveillance Sensor Networks, pp. 121–130 (2005)Google Scholar
  5. 5.
    Benfold, B., Reid, I.: Stable multi-target tracking in real-time surveillance video. In: Computer Vision and Pattern Recognition, pp. 3457–3464 (2011)Google Scholar
  6. 6.
    Choi, W., Savarese, S.: Multiple target tracking in world coordinate with single, minimally calibrated camera. In: European Conference on Computer Vision, pp. 553–567 (2010)Google Scholar
  7. 7.
    Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: Computer Vision and Pattern Recognition, pp. 886–893 (2005)Google Scholar
  8. 8.
    Dinh, T., Vo, N., Medioni, G.G.: Context tracker: exploring supporters and distracters in unconstrained environments. In: Computer Vision and Pattern Recognition, pp. 1177–1184 (2011a)Google Scholar
  9. 9.
    Dinh, T.B., Vo, N., Medioni, G.G.: High resolution face sequences from a PTZ network camera. In: IEEE Conference on Automatic Face and Gesture Recognition, pp. 531–538 (2011b)Google Scholar
  10. 10.
    Dinh, T.B., Yu, Q., Medioni, G.G.: Real time tracking using an active pan-tilt-zoom network camera. In: International Conference on Intelligent Robots and Systems, pp. 3786–3793 (2009)Google Scholar
  11. 11.
    Enzweiler, M., Gavrila, D.M.: Monocular pedestrian detection: survey and experiments. IEEE Trans. Pattern Anal. Mach. Intell. 31(12), 2179–2195 (2009)CrossRefGoogle Scholar
  12. 12.
    Felzenszwalb, P.F., Girshick, R.B., McAllester, D., Ramanan, D.: Object detection with discriminatively trained part-based models. IEEE Trans. Pattern Anal. Mach. Intell. 32(9), 1627–1645 (2010)CrossRefGoogle Scholar
  13. 13.
    Froba, B., Ernst, A.: Face detection with the modified census transform. In: IEEE Conference on Automatic Face and Gesture Recognition, pp. 91–96 (2004)Google Scholar
  14. 14.
    Huang, C., Nevatia, R.: High performance object detection by collaborative learning of joint ranking of granules features. In: Computer Vision and Pattern Recognition, pp. 41–48 (2010)Google Scholar
  15. 15.
    Huang, C., Wu, B., Nevatia, R.: Robust object tracking by hierarchical association of detection responses. In: European Conference on Computer Vision, pp. 788–801 (2008)Google Scholar
  16. 16.
    Kalal, Z., Mikolajczyk, K., Matas, J.: Tracking-learning-detection. IEEE Trans. Pattern Anal. Mach. Intell. 34(7), 1409–1422 (2012)CrossRefGoogle Scholar
  17. 17.
    Lee, S.G., Batkhishig, R.: Implementation of a real-time image object tracking system for PTZ cameras. Converg. Hybrid Inf. Technol. 206, 121–128 (2011)CrossRefGoogle Scholar
  18. 18.
    Lee, S.C., Nevatia, R.: Robust camera calibration tool for video surveillance camera in urban environment. In: Computer Vision and Pattern Recognition Workshops, pp. 62–67 (2011)Google Scholar
  19. 19.
    Li, Y., Bhanu, B., Lin, W.: Auction protocol for camera active control. In: International Conference on Image Processing, pp. 4325–4328 (2010)Google Scholar
  20. 20.
    Munkres, J.: Algorithms for the assignment and transportation problems. J. Soc. Ind. Appl. Math. 5(1), 32–38 (1957)MathSciNetCrossRefzbMATHGoogle Scholar
  21. 21.
    Ojala, T., Pietikäinen, M., Mäenpää, T.: Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans. Pattern Anal. Mach. Intell. 24(7), 971–987 (2002)CrossRefzbMATHGoogle Scholar
  22. 22.
    Possegger, H., Mauthner, T., Roth, P.M., Bischof, H.: Occlusion geodesics for online multi-object tracking. In: IEEE Conference on Computer Vision and Pattern Recognition (2014)Google Scholar
  23. 23.
    Qureshi, F., Terzopoulos, D.: Surveillance camera scheduling: a virtual vision approach. Multimed. Syst. 12, 269–283 (2006)CrossRefGoogle Scholar
  24. 24.
    Qureshi, F., Terzopoulos, D.: Distributed coalition formation in visual sensor networks: a virtual vision approach. Distrib. Comput. Sens. Syst. 4549, 1–20 (2007)CrossRefGoogle Scholar
  25. 25.
    Salvagnini, P., Cristani, M., Del Bue, A., Murino, V.: An experimental framework for evaluating PTZ tracking algorithms. In: International Conference on Computer Vision Systems, pp. 81–90 (2011)Google Scholar
  26. 26.
    Salvagnini, P., Pernici, F., Cristani, M., Lisanti, G., Masi, I., del Bimbo, A., Murino, V.: Information theoretic sensor management for multi-target tracking with a single pan-tilt-zoom camera. In: IEEE Winter Applications of Computer Vision Conference (2014)Google Scholar
  27. 27.
    Schumann, A., Bauml, M., Stiefelhagen, R.: Person tracking-by-detection with efficient selection of part-detectors. In: IEEE International Conference on Advanced Video and Signal Based Surveillance (2013)Google Scholar
  28. 28.
    Silberschatz, A., Galvin P.B., Gagne, G.: Process scheduling. In: Operating System Concepts (2010)Google Scholar
  29. 29.
    Smith, K., Gatica-Perez, D., Odobez, J.-M., Ba, S.: Evaluating multi-object tracking. In: Workshop on Empirical Evaluation Methods in Computer Vision (2005)Google Scholar
  30. 30.
    Sommerlade, E., Reid, I.: Information-theoretic active scene exploration. In: Computer Vision and Pattern Recognition, pp. 1–7 (2008)Google Scholar
  31. 31.
    Stauffer, C., Grimson, W.E.L.: Adaptive background mixture models for real-time tracking. In: IEEE Workshop on Visual Surveillance, pp. 246–252 (1999)Google Scholar
  32. 32.
    Vaisenberg, R., Mehrotra, S., Ramanan, D.: Semartcam scheduler: semantics driven real-time data collection from indoor camera networks to maximize event detection. J. Real-Time Image Process. 5(4), 215–230 (2010)CrossRefGoogle Scholar
  33. 33.
    Varcheie, P.D.Z., Bilodeau, G.-A.: People tracking using a network-based PTZ camera. Mach. Vis. Appl. 22(4), 671–690 (2011)CrossRefGoogle Scholar
  34. 34.
    Verma, R.C., Schmid, C., Mikolajczyk, K.: Face detection and tracking in a video by propagating detection probabilities. IEEE Trans. Pattern Anal. Mach. Intell. 25(10), 1215–1228 (2003)CrossRefGoogle Scholar
  35. 35.
    Wu, B., Nevatia, R.: Detection of multiple, partially occluded humans in a single image by bayesian combination of edgelet part detectors. In: International Conference on Computer Vision (2005)Google Scholar
  36. 36.
    Wu, B., Nevatia, R.: Detection and tracking of multiple, partially occluded humans by bayesian combination of edgelet based part detectors. Int. J. Comput. Vis. 75(2), 247–266 (2007)CrossRefGoogle Scholar
  37. 37.
    Wu, B., Nevatia, R.: Detection and segmentation of multiple, partially occluded objects by grouping, merging, assigning part detection responses. Int. J. Comput. Vis. 82, 121–128 (2009)Google Scholar
  38. 38.
    Xu, Y., Song, D.: Systems and algorithms for autonomously simultaneous observation of multiple objects using robotic PTZ cameras assisted by a wide-angle camera. In: International Conference on Intelligent Robots and Systems, pp. 3802–3807 (2009)Google Scholar
  39. 39.
    Yang, B., Nevatia, R.: Online learned discriminative part-based appearance models for multi-human tracking. In: European Conference on Computer Vision, pp. 484–498 (2012)Google Scholar
  40. 40.
    Yang, J., Wang, Y., Sowmya, A., Li, Z.: Vehicle detection and tracking with low-angle cameras. In: IEEE International Conference on Image Processing, pp. 685–688 (2010)Google Scholar
  41. 41.
    Zhang, L., Li, Y., Nevatia, R.: Global data association for multi-object tracking using network flows. In: Computer Vision and Pattern Recognition, pp. 1–8 (2008)Google Scholar
  42. 42.
    Zhao, T., Nevatia, R.: Tracking multiple humans in crowded environment. In: IEEE Conference on Computer Vision and Pattern Recognition, vol. 2 (2004)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2016

Authors and Affiliations

  1. 1.Institute of AutomationChinese Academy of SciencesBeijingChina
  2. 2.Institute for Robotics and Intelligent SystemsUniversity of Southern CaliforniaLos AngelesUSA

Personalised recommendations