Autonomous Robots

, Volume 29, Issue 1, pp 53–66 | Cite as

Systems and algorithms for autonomous and scalable crowd surveillance using robotic PTZ cameras assisted by a wide-angle camera

  • Yiliang Xu
  • Dezhen SongEmail author


We report an autonomous surveillance system with multiple pan-tilt-zoom (PTZ) cameras assisted by a fixed wide-angle camera. The wide-angle camera provides large but low resolution coverage and detects and tracks all moving objects in the scene. Based on the output of the wide-angle camera, the system generates spatiotemporal observation requests for each moving object, which are candidates for close-up views using PTZ cameras. Due to the fact that there are usually much more objects than the number of PTZ cameras, the system first assigns a subset of the requests/objects to each PTZ camera. The PTZ cameras then select the parameter settings that best satisfy the assigned competing requests to provide high resolution views of the moving objects. We propose an approximation algorithm to solve the request assignment and the camera parameter selection problems in real time. The effectiveness of the proposed system is validated in both simulation and physical experiment. In comparison with an existing work using simulation, it shows that in heavy traffic scenarios, our algorithm increases the number of observed objects by over 210%.


Surveillance PTZ camera Frame selection 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Supplementary material

10514_2010_9188_MOESM1_ESM.wmv (1.4 mb)
(WMV 1.43 MB)


  1. Alt, H., Arkin, E. M., Brönnimann, H., Erickson, J., Fekete, S. P., Knauer, C., Lenchner, J., Mitchell, J. S. B., & Whittlesey, K. (2006). Minimum-cost coverage of point sets by disks. In Symposium on computational geometry (pp. 449–458), June 2006. Google Scholar
  2. Arkin, E. M., Barequet, G., & Mitchell, J. S. B. (2006). Algorithms for two-box covering. In Symposium on computational geometry (pp. 459–467), June 2006. Google Scholar
  3. Bimbo, A. D., & Pernici, F. (2005). Distant targets identification as an on-line dynamic vehicle routing problem using an active-zooming camera. In Visual surveillance and performance evaluation of tracking and surveillance (pp. 97–104). Google Scholar
  4. Bodor, R., Morlok, R., & Papanikolopoulos, N. (2004). Dual-camera system for multi-level activity recognition. In Proceedings IEEE/RSJ international conference on intelligent robots and systems (IROS 2004) (Vol. 1, pp. 643–648), 2 Oct. 2004. Google Scholar
  5. Costello, C. J., Diehl, C. P., Banerjee, A., & Fisher, H. (2004). Scheduling an active camera to observe people. In VSSN’04: Proceedings of the ACM 2nd international workshop on video surveillance & sensor networks (pp. 39–45). New York: ACM. CrossRefGoogle Scholar
  6. Elgammal, A., Harwood, D., & Davis, L. (2000). Non-parametric model for background subtraction. In 6th European conference on computer vision (ECCV) (Vol. 2, pp. 751–761), Dublin, Ireland, June/July 2000. Google Scholar
  7. Eppstein, D. (1997). Fast construction of planar two-centers. In Proc. 8th ACM-SIAM sympos. discrete algorithms (pp. 131–138), January 1997. Google Scholar
  8. Fiore, L., Fehr, D., Bodor, R., Drenner, A., Somasundaram, G., & Papanikolopoulos, N. (2008). Multi-camera human activity monitoring. Journal of Intelligent and Robotic Systems, 52(1), 5–43. CrossRefGoogle Scholar
  9. Hampapur, A., Pankanti, S., Senior, A., Tian, Y.-L., Brown, L., & Bolle, R. (2003). Face cataloger: multi-scale imaging for relating identity to location. In Proceedings IEEE conference on advanced video and signal based surveillance (pp. 13–20), July 2003. Google Scholar
  10. Lim, S.-N., Davis, L., & Elgammal, A. (2003). Scalable image-based multi-camera visual surveillance system. In Proceedings IEEE conference on advanced video and signal based surveillance (pp. 205–212), July 2003. Google Scholar
  11. Lim, S.-N., Davis, L. S., & Mittal, A. (2006). Constructing task visibility intervals for video surveillance. Multimedia Systems, 12(3), 211–226. CrossRefGoogle Scholar
  12. Lim, S.-N., Davis, L., & Mittal, A. (2007). Task scheduling in large camera networks. In 8th Asian conference on computer vision, Tokyo, Japan (pp. 397–407). Google Scholar
  13. Megiddo, N., & Supowit, K. (1984). On the complexity of some common geometric location problems. SIAM Journal on Computing, 13, 182–196. zbMATHCrossRefMathSciNetGoogle Scholar
  14. Qureshi, F., & Terzopoulos, D. (2008). Smart camera networks in virtual reality. Proceedings of the IEEE, 96(10), 1640–1656. CrossRefGoogle Scholar
  15. Sommerlade, E., & Reid, I. (2008). Information-theoretic active scene exploration. In IEEE conference on computer vision and pattern recognition (CVRP), Anchorage, Alaska, USA. Google Scholar
  16. Song, D. (2009). Sharing a vision: systems and algorithms for collaboratively-teleoperated robotic cameras. Berlin: Springer. Google Scholar
  17. Song, D., & Goldberg, K. (2007). Approximate algorithms for a collaboratively controlled robotic camera. IEEE Transactions on Robotics, 23(5), 1061–1070. CrossRefGoogle Scholar
  18. Song, D., van der Stappen, A. F., & Goldberg, K. (2006). Exact algorithms for single frame selection on multi-axis satellites. IEEE Transactions on Automation Science and Engineering, 3(1), 16–28. CrossRefGoogle Scholar
  19. Song, D., Qin, N., & Goldberg, K. (2008). Systems, control models, and codec for collaborative observation of remote environments with an autonomous networked robotic camera. Autonomous Robots, 24(4), 435–449. CrossRefGoogle Scholar
  20. Xu, Y., & Song, D. (2009). Systems and algorithms for autonomously simultaneous observation of multiple objects using robotic ptz cameras assisted by a wide-angle camera. In International conference on intelligent robots and systems (IROS), St. Louis, USA, October 2009. Google Scholar
  21. Xu, Y., Song, D., Yi, J., & van der Stappen, F. (2008). An approximation algorithm for the least overlapping p-frame problem with non-partial coverage for networked robotic cameras. In IEEE international conference on robotics and automation (ICRA), Pasadena, CA (pp. 1011–1016), May 2008. Google Scholar
  22. Yilmaz, A., Javed, O., & Shah, M. (2006). Object tracking: a survey. ACM Computing Surveys, 38(4), 1–45. CrossRefGoogle Scholar
  23. Zhang, C., Liu, Z., Zhang, Z., & Zhao, Q. (2008). Semantic saliency driven camera control for personal remote collaboration. In IEEE 10th workshop on multimedia signal processing (pp. 28–33), Oct. 2008. Google Scholar
  24. Zhou, X., Collins, R. T., Kanade, T., & Metes, P. (2003). A master-slave system to acquire biometric imagery of humans at distance. In IWVS’03: First ACM SIGMM international workshop on video surveillance (pp. 113–120). New York: ACM. CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2010

Authors and Affiliations

  1. 1.Computer Science and Engineering DepartmentTexas A&M UniversityCollege StationUSA

Personalised recommendations