Autonomous Robots

, Volume 24, Issue 4, pp 435–449 | Cite as

Systems, control models, and codec for collaborative observation of remote environments with an autonomous networked robotic camera

  • Dezhen Song
  • Ni Qin
  • Ken Goldberg


Networked robotic cameras are becoming popular in remote observation applications such as natural observation, surveillance, and distance learning. Equipped with a high optical zoom lens and agile pan-tilt mechanisms, a networked robotic camera can cover a large region with various resolutions. The optimal selection of camera control parameters for competing observation requests and the on-demand delivery of video content for various spatiotemporal queries are two challenges in the design of such autonomous systems. For camera control, we introduce memoryless and temporal frame selection models that effectively enable collaborative control of the camera based on the competing inputs from in-situ sensors and users. For content delivery, we design a patch-based motion panorama representation and coding/decoding algorithms (codec) to allow efficient storage and computation. We present system architecture, frame selection models, user interface, and codec algorithms. We have implemented the system and extensively tested our design in real world applications including natural observation, public surveillance, distance learning, and building construction monitoring. Experiment results show that our frame selection models are robust and effective and our on-demand content delivery codec can satisfy a variety of spatiotemporal queries efficiently in terms of computation time communications bandwidth.


Frame selection Panorama video Networked cameras Tele-robot 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Agarwala, A., Zheng, C., Pal, C., Agrawala, M., Cohen, M., Curless, B., Salesin, D., & Szeliski, R. (2005). Panoramic video textures. In ACM transactions on graphics (proceedings of SIGGRAPH 2005), Los Angeles, CA, July 2005. Google Scholar
  2. Baker, S., & Nayar, S. K. (1999). A theory of single-viewpoint catadioptric image formation. International Journal of Computer Vision, 35(2), 175–196. CrossRefGoogle Scholar
  3. Bartoli, A., Dalal, N., & Horaud, R. (2004). Motion panoramas. Computer Animation and Virtual Worlds, 15, 501–517. CrossRefGoogle Scholar
  4. Benosman, R., & Kang, S. B. (2001). Panoramic vision. New York: Springer. zbMATHGoogle Scholar
  5. Cannon, D. J. (1992). Point-and-direct telerobotics: object level strategic supervisory control in unstructured interactive human-machine system environments. PhD thesis, Stanford Mechanical Engineering, June 1992. Google Scholar
  6. Chong, N., Kotoku, T., Ohba, K., Komoriya, K., Matsuhira, N., & Tanie, K. (2000). Remote coordinated controls in multiple telerobot cooperation. In IEEE international conference on robotics and automation (Vol. 4, pp. 3138–3343), April 2000. Google Scholar
  7. Collins, R., Lipton, A., & Kanade, T. (2000). Introduction to special section on video surveillance. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(8), 745–746. CrossRefGoogle Scholar
  8. Foote, J., & Kimber, D. (2000). FlyCam: practical panoramic video and automatic camera control. In IEEE international conference on multimedia and expo, ICME 2000 (Vol. 3, pp. 1419–1422), New York, NY, July 2000. Google Scholar
  9. Foote, J., & Kimber, D. (2001). Enhancing distance learning with panoramic video. In Proceedings of the 34th Hawaii international conference on system sciences (pp. 1–7). Google Scholar
  10. Goldberg, K., & Chen, B. (2001). Collaborative control of robot motion: robustness to error. In International conference on intelligent robots and systems (IROS) (Vol. 2, pp. 655–660), October 2001. Google Scholar
  11. Goldberg, K., & Siegwart, R. (Eds.). (2002). Beyond webcams: an introduction to online robots. Cambridge: MIT Press. Google Scholar
  12. Goldberg, K., Song, D., & Levandowski, A. (2003). Collaborative teleoperation using networked spatial dynamic voting. Proceedings of the IEEE, 91(3), 430–439. CrossRefGoogle Scholar
  13. Irani, M., & Anandan, P. (1998). Video indexing based on mosaic representations. Proceedings of the IEEE, 86, 905–921. CrossRefGoogle Scholar
  14. Irani, M., Hsu, S., & Anandan, P. (1995). Video compression using mosaic representations. Signal Processing: Image Communication, 7, 529–552. CrossRefGoogle Scholar
  15. Irani, M., Anandan, P., Bergen, J., Kumar, R., & Hsu, S. (1996). Efficient representations of video sequences and their applications. Signal Processing: Image Communication, 8. Google Scholar
  16. Kansal, A., Kaiser, W., Pottie, G., Srivastava, M., & Sukhatme, G. (2006). Virtual high-resolution for sensor networks. In ACM SenSys, Boulder, CO, November 1–3, 2006. Google Scholar
  17. McDonald, M., Small, D., Graves, C., & Cannon, D. (1997). Virtual collaborative control to improve intelligent robotic system efficiency and quality. In IEEE international conference on robotics and automation (Vol. 1, pp. 418–424), April 1997. Google Scholar
  18. Nayar, S. K. (1997). Catadioptric omnidirectional camera. In Proceedings of IEEE computer society conference on computer vision and pattern recognition (pp. 482–488), San Juan, Puerto Rico. Google Scholar
  19. Ng, K. T., Chan, S. C., & Shum, H. Y. (2005). Data compression and transmission aspects of panoramic videos. IEEE Transactions on Circuits and Systems for Video Technology, 15(1), 82–95. CrossRefGoogle Scholar
  20. Qin, N., Song, D., & Goldberg, K. (2006). Aligning windows of live video from an imprecise pan-tilt-zoom robotic camera into a remote panoramic display. In IEEE international conference on robotics and automation (ICRA), Orlando, FL, May 2006. Google Scholar
  21. Rav-Acha, A., Pritch, Y., Lischinski, D., & Peleg, S. (2005). Dynamosaics: video mosaics with non-chronological time. In IEEE computer society conference on computer vision and pattern recognition. Google Scholar
  22. Song, D., & Goldberg, K. (2005). Networked robotic cameras for collaborative observation of natural environments. In The 12th international symposium of robotics research (ISRR), San Francisco, CA, October 2005. Google Scholar
  23. Song, D., & Goldberg, K. (2007). Approximate algorithms for a collaboratively controlled robotic camera. IEEE Transactions on Robotics, 23(5), 1061–1070. CrossRefGoogle Scholar
  24. Song, D., Pashkevich, A., & Goldberg, K. (2003). ShareCam part II: approximate and distributed algorithms for a collaboratively controlled robotic webcam. In IEEE/RSJ international conference on intelligent robots (IROS) (Vol. 2, pp. 1087–1093), Las Vegas, NV, October 2003. Google Scholar
  25. Song, D., van der Stappen, A. F., & Goldberg, K. (2006a). Exact algorithms for single frame selection on multi-axis satellites. IEEE Transactions on Automation Science and Engineering, 3(1), 16–28. CrossRefGoogle Scholar
  26. Song, D., Qin, N., & Goldberg, K. (2006b). A minimum variance calibration algorithm for pan-tilt robotic cameras in natural environments. In IEEE international conference on robotics and automation (ICRA), Orlando, FL, May 2006. Google Scholar
  27. Swaminathan, R., & Nayar, S. K. (2000). Nonmetric calibration of wide-angle lenses and polycameras. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(10), 1172–1178. CrossRefGoogle Scholar
  28. Trucco, E., Doull, A., Odone, F., Fusiello, A., & Lane, D. (2000). Dynamic video mosaicing and augmented reality for subsea inspection and monitoring. In Oceanology international conference, Brighton, UK, March 2000. Google Scholar
  29. Xiong, Y., & Turkowski, K. (1997). Creating image-based VR using a self-calibrating fisheye lens. In Proceedings of IEEE computer society conference on computer vision and pattern recognition (pp. 237–243), San Juan, Puerto Rico, June 1997. Google Scholar
  30. Zhu, Z., Xu, G., Riseman, E. M., & Hanson, A. R. (1999). Fast generation of dynamic and multi-resolution 360-degree panorama from video sequences. In Proceedings of the IEEE international conference on multimedia computing and systems (Vol. 1, pp. 9400–9406), Florence, Italy, June 1999. Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2008

Authors and Affiliations

  1. 1.Computer Science Dept.Texas A&M UniversityCollege StationUSA
  2. 2.Depts. of IEOR and EECSUniversity of CaliforniaBerkeleyUSA

Personalised recommendations