Multimedia Tools and Applications

, Volume 57, Issue 2, pp 315–334 | Cite as

Intelligent video surveillance system: 3-tier context-aware surveillance system with metadata



This paper presents an intelligent video surveillance system with the metadata rule for the exchange of analyzed information. We define the metadata rule for the exchange of analyzed information between intelligent video surveillance systems that automatically analyzes video data acquired from cameras. The metadata rule is to effectively index very large video surveillance databases and to unify searches and management between distributed or heterogeneous surveillance systems more efficiently. The system consists of low-level context-aware, high-level context-aware and intelligent services to generate metadata for the surveillance systems. Various contexts are acquired from physical sensors in monitoring areas for the low-level context-aware system. The situation is recognized in the high-level context-aware system by analyzing the context data collected in the low-level system. The system provides intelligent services to track moving objects in Fields Of View (FOVs) and to recognize human activities. Furthermore, the system supports real-time moving objects tracking with Panning, Tilting and Zooming (PTZ) cameras in overlapping and non-overlapping FOVs.


Object identification Object localization Object tracking CCTV Surveillance PTZ camera Metadata 



“This research was supported by the MKE(The Ministry of Knowledge Economy), Korea, under the ITRC(Information Technology Research Center) support program supervised by the NIPA(National IT Industry Promotion Agency)” (NIPA-2010-C1090-1031-0004) and this research is also supported by the Ubiquitous Computing and Network (UCN) Project, Knowledge and Economy Frontier R&D Program of the Ministry of Knowledge Economy (MKE), the Korean government, as a result of UCN’s subproject 10C2-T3-10M.


  1. 1.
    Ali A, Aggarwal J (2001) Segmentation and recognition of continuous human activity. In: Detection and recognition of events in video, 2001. Proceedings. IEEE Workshop on, pp 28–35Google Scholar
  2. 2.
    Ayers D, Shah M (2001) Monitoring human behavior from video taken in an office environment. Image Vis Comput 19(12):833–846CrossRefGoogle Scholar
  3. 3.
    Cai Q, Aggarwal JK (1996) Tracking human motion using multiple cameras. In: ICPR ’96: Proceedings of the International Conference on Pattern Recognition (ICPR ’96) Volume III-Volume 7276. Washington, DC, USA: IEEE Computer Society, pp 68–72Google Scholar
  4. 4.
    Chae YN, Kim Y-H, Choi J, Cho K, Yang HS (2009) An adaptive sensor fusion based objects tracking and human action recognition for interactive virtual environments. In: VRCAI ’09: Proceedings of the 8th International Conference on Virtual Reality Continuum and its Applications in Industry. New York, NY, USA: ACM, pp 357–362Google Scholar
  5. 5.
    Comaniciu D, Ramesh V, Meer P (2003) Kernel-based object tracking. IEEE Trans Pattern Anal Mach Intell 25:564–575CrossRefGoogle Scholar
  6. 6.
    Danielsson P (1980) Euclidean distance mapping. Comput Graph Image Process 14(3):227–248CrossRefGoogle Scholar
  7. 7.
    Duong T, Bui H, Phung D, Venkatesh S (20–25 2005) Activity recognition and abnormality detection with the switching hidden semi-markov model. In : Computer Vision and Pattern Recognition, 2005. CVPR 2005. IEEE Computer Society Conference on, vol. 1, pp 838–845Google Scholar
  8. 8.
    Google Earth API. Online Available.
  9. 9.
    Huang C-L, Liao B-Y (2001) A robust scene-change detection method for video segmentation. IEEE Trans Circuits Syst Video Technol 11(12):1281–1288CrossRefGoogle Scholar
  10. 10.
    Huang T, Russell S (1997) Object identification in a Bayesian context. In: IJCAI’97: Proceedings of the Fifteenth international joint conference on Artifical intelligence. San Francisco, CA, USA: Morgan Kaufmann Publishers Inc., p. 1276–1282Google Scholar
  11. 11.
    Kelly PH, Katkere A, Kuramura DY, Moezzi S, Chatterjee S (1995) An architecture for multiple perspective interactive video. In: MULTI- MEDIA ’95: Proceedings of the third ACM International Conference on Multimedia. New York, NY, USA: ACM, pp 201–212Google Scholar
  12. 12.
    Kettnaker V, Zabih R (Jul 1999) Counting people from multiple cameras. In: Multimedia computing and systems, 1999. IEEE International Conference on, vol. 2, pp 267–271Google Scholar
  13. 13.
    Lv F, Kang J, Nevatia R, Cohen I, Medioni G (2004) Automatic tracking and labeling of human activities in a video sequence. In PETS04Google Scholar
  14. 14.
    Nam J, Tewfik A (2005) Detection of gradual transitions in video sequences using b-spline interpolation. IEEE Trans Multimedia 7(4):667–679CrossRefGoogle Scholar
  15. 15.
    Nam Y, Ryu J, Joo Choi Y, Duke Cho W (2007) Learning spatio-temporal topology of a multi-camera network by tracking multiple people. World Acad Sci Eng Tech 4(4):254–259Google Scholar
  16. 16.
    OpenCV, Open Computer Vision Library.
  17. 17.
    Petrushin V, Wei G, Ghani R, Gershman A (28–28 2005) Multiple sensor indoor surveillance: problems and solutions. In: Machine Learning for Signal Processing, 2005 IEEE Workshop on, pp 349–354Google Scholar
  18. 18.
    Rabiner L (1989) A tutorial on hidden markov models and selected applications in speech recognition. Proc IEEE 77(2):257–286CrossRefGoogle Scholar
  19. 19.
    Ribeiro PC, Santos-victor J (2005) Human activity recognition from video: modeling, feature selection and classification architecture. In: International Workshop on Human Activity Recognition and Modeling, pp 61–70Google Scholar
  20. 20.
    Ribnick E, Atev S, Masoud O, Papanikolopoulos N, Voyles R (Nov. 2006) Real-time detection of camera tampering. In: Video and signal based surveillance, 2006. AVSS ’06. IEEE International Conference onGoogle Scholar
  21. 21.
    Serby D, Meier E, Van Gool L (23–26 2004) Probabilistic object tracking using multiple features. In: Pattern recognition, 2004. ICPR 2004. Proceedings of the 17th International Conference on, vol. 2, pp 184–187 Vol.2Google Scholar
  22. 22.
    Veenman C, Reinders M, Backer E (2001) Resolving motion correspondence for densely moving points. IEEE Trans Pattern Anal Mach Intell 23(1):54–72CrossRefGoogle Scholar
  23. 23.
    Viola P, Jones M, Snow D (2005) Detecting pedestrians using patterns of motion and appearance. Int J Comput Vis 63:153–161CrossRefGoogle Scholar
  24. 24.
    Yilmaz A, Li X, Shah M (2004) Contour-based object tracking with occlusion handling in video acquired using mobile cameras. IEEE Trans Pattern Anal Mach Intell 26(11):1531–1536CrossRefGoogle Scholar
  25. 25.
    Zelnik-Manorand L, Irani M (2001) Event-based analysis of video. In: Computer Vision and Pattern Recognition, 2001. CVPR 2001. Proceedings of the 2001 IEEE Computer Society Conference on, vol. 2, pp 123–130Google Scholar
  26. 26.
    Zhang D, Gatica-Perez D, Bengio S, McCowan I (20–25 2005) Semi-supervised adapted HMMs for unusual event detection. In Computer Vision and Pattern Recognition, 2005. CVPR 2005. IEEE Computer Society Conference on, vol. 1, pp 611–618Google Scholar
  27. 27.
    Zhao W, Wang J, Bhat D, Sakiewicz K, Nandhakumar N, Chang W (Jul 1999) Improving color based video shot detection. In: Multimedia computing and systems, 1999. IEEE International Conference on, vol. 2, pp 752–756 vol.2Google Scholar
  28. 28.
    Zhong H, Shi J, Visontai M (2004) Detecting unusual activity in video. In: Computer vision and pattern recognition, Proceedings of the 2004 IEEE Computer Society Conference on, vol. 2, pp 819–826Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2010

Authors and Affiliations

  1. 1.Center of Excellence for Ubiquitous SystemAjou UniversitySuwonSouth Korea
  2. 2.School of Electrical EngineeringKorea UniversitySeoulSouth Korea
  3. 3.Department of Computer Science and EngineeringSeoul National University of Science and TechnologySeoulSouth Korea

Personalised recommendations