Development of an Intelligent Omnivision Surveillance System

  • Hannes Bistry
  • Jianwei Zhang
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 215)


This publication describes an innovative intelligent omnivision video system, that stitches the images of four cameras together, resulting in one seamless image. This way the system generates a 360° view of the scene. An additional automatically controlled pan-tilt-zoom-camera provides a high resolution view of user defined regions of interest (ROI). In addition to the fusion of multiple camera images, the system has intelligent features like object detection and region-of-interest detection. The software architecture features configurable pipelines of image processing functions. All different steps in the pipeline like decoding, feature extraction, encoding, and visualization are implemented as modules inside this pipeline. It is easily possible to rearrange the pipeline and add new functions to the overall system. The pan-tilt-zoom camera is controlled by an embedded system that has been developed for this system. GPU-accelerated processing elements allows real-time panorama stitching. We show the application of our system in the field of maritime surveillance, but the system can also be used for robots.



The authors would like to thank their student assistant Johannes Liebrecht for designing the PTZ control board during his thesis, Andreas Maeder for the calibration of the panorama stitching routine, Gang Cheng for contributing the PTU control code, and the Group Kunshan Robotechn Intelligent Technology Co., Ltd. for performing real-world testing in a maritime scenario.


  1. 1.
    Bistry H, Zhang J (2009) Task oriented control of smart camera systems in the context of mobile service robots. In: IEEE/RSJ international conference on intelligent robots and systems, IROS 2009, pp 3844–3849Google Scholar
  2. 2.
    Bistry H, Zhang J (2010) A cloud computing approach to complex robot vision tasks using smart camera systems. In: IEEE/RSJ international conference on intelligent robots and systems, IROS 2010, pp 3195–3200Google Scholar
  3. 3.
    Bistry H, Vietze F, Zhang J (2009) Towards intelligent high resolution surveillance cameras. In: Künzel M, Michel B (eds) Safety and security systems in Europe, number 10 in micromaterials and nanomaterials. Micro Materials at Fraunhofer IZM Berlin and Fraunhofer ENAS Chemnitz, pp 76–79Google Scholar
  4. 4.
    Maeder A, Bistry H, Zhang J (2008) Intelligent vision systems for robotic applications. Int J Inf Acquis 5(3):259–267CrossRefGoogle Scholar
  5. 5.
    Foote J, Kimber D (2000) Flycam: practical panoramic video and automatic camera control. In: IEEE international conference on multimedia and expo, 2000. ICME 2000, vol 3. IEEE, pp 1419–1422Google Scholar
  6. 6.
    Yamazawa K, Yagi Y, Yachida M (1993) Omnidirectional imaging with hyperboloidal projection. In: Proceedings of the 1993 IEEE/RSJ international conference on intelligent robots and systems’ 93, IROS’93, vol 2. IEEE, pp 1029–1034Google Scholar
  7. 7.
    Wu J, Yang K, Xiang Q, Zhang N (2009) Design of object surveillance system based on enhanced fish-eye lens. Chin Optics Lett 7(2):142–145CrossRefGoogle Scholar
  8. 8.
    Haritaoglu I, Harwood D, Davis LS (2000) W\(^4\): real-time surveillance of people and their activities. IEEE Trans Pattern Anal Mach Intell 22(8):809–830Google Scholar
  9. 9.
    Weser M, Westhoff D, Hüser M, Zhang J (2006) Real-time fusion of multimodal tracking data and generalization of motion patterns for trajectory prediction. In: Proceeding of the IEEE international conference on information acquisition (ICIA), Shandong, China, August 2006Google Scholar
  10. 10.
    Huang YH, Hu M, Chong HL, Jia XM, Ma JX, Liu WL (2012) A survey of robot visual tracking algorithm. Key Eng Mater 501:577–582CrossRefGoogle Scholar
  11. 11.
    Huang J, Ponce SP, Park SI, Cao Y, Quek F (2008) GPU-accelerated computation for robust motion tracking using the CUDA framework. In: Proceedings of the IET international conference on visual information engineeringGoogle Scholar
  12. 12.
    Torres F, Candelas F, Puente S, Jiménez L, Fernández C, Agulló R (1999) Simulation and scheduling of real-time computer vision algorithms. Computer vision systems, pp 98–114Google Scholar
  13. 13.
    Xu R, Jin J (2005) Scheduling latency insensitive computer vision tasks. Parallel and distributed processing and applications, pp 1089–1100Google Scholar
  14. 14.
    Taymans W, Baker S, Wingo A, Bultje R, Kost S (2008) Gstreamer application development manual (
  15. 15.
    Lowe DG (1999) Object recognition from local scale-invariant features. Int Conf Comput Vis 2:1150–1157Google Scholar
  16. 16.
    Schleicher D, Bergasa LM, Barea R, Lopez E, Ocaña M, Nuevo J (2007) Real-time wide-angle stereo visual SLAM on large environments using SIFT features correction. In: Proceedings of the IEEE/RSJ international conference on international robots and system, Oct 2007, pp 3878–3883Google Scholar
  17. 17.
    Kuehnle J, Verl A, Xue Z, Ruehl S, Zoellner JM, Dillmann R, Grundmann T, Eidenberger R, Zoellner RD (2009) 6d object localization and obstacle detection for collision-free manipulation with a mobile, service robot, pp 1–6Google Scholar
  18. 18.
    Viola P, Jones M (2001) Rapid object detection using a boosted cascade of simple features. In: IEEE Computer society conference on computer vision and pattern recognition, vol 1Google Scholar
  19. 19.
    Bradski G (2000) The openCV library. Doct Dobbs J 25(11):120–126Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  1. 1.Department of InformaticsUniversity of HamburgHamburgGermany

Personalised recommendations