Ontology-based Shot Indexing for Video Surveillance System

  • Jeongkyu Lee
  • Munther H. Abualkibash
  • Padmini K. Ramalingam


Indexing video data is very complex due to the enormous size of video files and their semantically rich contents and unstructured format. Therefore, the first step of video indexing begins with segmenting each video into a number of basic processing units. In general, the most widely used basic unit is a shot that is defined as collections of frames recorded from a single camera operation. Basically, there are two different approaches to index video using these shots. One is using low level features such as color, motion, or object. On the other hand, high-level features such as human interpretations are utilized for the indexing. Recently, a hybrid indexing combining these two approaches has been proposed to take advantages of both of them, and shows that dealing with both features has certain advantages. However, it is necessary that further improvements are required for which features can be extracted, and how they are identified. More importantly, further investigation is needed for the seamless integration of both levels for indexing purpose. To address these issues, we introduce ontology-based indexing for video surveillance system. Generally, video surveillance has non-predefined contents unlike movie or drama, and its main purposes are to detect and record abnormal events. Since ontology defines machine-interpretable definitions of basic concept in a domain, we can prevent semantic inconsistencies from human interpretations and apply it to handle surveillance data. Then, we can classify and index video data more precisely by using well-organized ontology. To illustrate the effectiveness of ontology based video indexing, we implement a Video Ontology System (VOS) to classify and index shots in a simple domain. Our experimental results show that the proposed approach is promising.


Semantic Meaning Video Shot Shot Boundary Video Surveillance System Video Segmentation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    Chupeau, B., Forest, R.: An evaluation of the effectiveness of color attributes for video indexing. In: Proc. of SPIE conf. on Storage and Retrieval for Media Databases 2001, San Jose, CA (2001) 470–481Google Scholar
  2. [2]
    Bimbo, A.D., Pala, P., Vicario, E.: Spatial arrangement of color flows for video retrieval. In: Proc. of 2001 IEEE Int’l Conf. on Multimedia and Expo(ICME), Tokyo, Japan (2001) 413–416Google Scholar
  3. [3]
    Sanchez, J.M., Binefa, X., Vitria, J., Radeva, P.: Linking visual cues and semantic terms under specific digital video domains. Journal of Visual Languages and Computing 11 (2000)Google Scholar
  4. [4]
    Wu, Y., Zhuang, Y., Pan, Y.: Content-based video retrieval integrating human perception. In: Proc. of SPIE conf. on Storage and Retrieval for Media Databases 2001, San Jose, CA (2001) 562–570Google Scholar
  5. [5]
    Oh, J., Baskaran, N.: An efficient technique for video shot indexing using low level features. In: Proc. of WORKSHOP ON MULTIMEDIA SEMANTICS 2002 in conjunction with 29th Annual Conference on Current Trends in Theory and Practice of Informatics (SOFSEM 2002), Milovy, Czech Republic (2002)Google Scholar
  6. [6]
    Noy, N., D.L., M.: Ontology development 101: A guide to creating your first ontology. In: Proc. of 19th Int’l Conf. on Conceptual Modeling (ER2000). (2000) 383–396Google Scholar
  7. [7]
    Hyvonen, E., Styrman, A., Saarela, S.: Ontology-based image retrieval. In: Proceedings of XML Finland 2002 Conference. (2002) 15– 27Google Scholar
  8. [8]
    Simoff, S.J., Maher, M.L.: Ontology-based multimedia data mining for design information retrieval. In: Proc. of the ACSE Computing Congress, Cambridge, MA (1998) 310– 320Google Scholar
  9. [9]
    Guarino, N.: Formal ontology and information systems. In: Proc. of the 1st International Conference on Formal Ontologies in Information Systems, FOIS’98, Trento, Italy (1998) 3– 15Google Scholar
  10. [10]
    Gu, N., Wang, F., Wu, G.: Ontology-based document extraction processing. In: The 7th International Conference on Computer Supported Cooperative Work in Design. (2002) 65– 67Google Scholar
  11. [11]
    Taghva, K., Borsack, J., Coombs, J., Lumos, A.C.S., Nartker, T.: Ontology-based classification of email. In: Proceedings. ITCC 2003. International Conference on Information Technology: Computers and Communications. (2003) 194– 198Google Scholar
  12. [12]
    Chang, H.H., Ko, Y.H., Hsu, J.P.: An event-driven and ontology-based approach for the delivery and. In: Proceedings. International Symposium on Multimedia Software Engineering. (2000) 103– 109Google Scholar
  13. [13]
    Schreiber, A.T., Dubbeldam, B., Wielemaker, J., Wielinga, B.: Ontology-based photo annotation. Intelligent Systems, IEEE 16 (2001) 66–7413CrossRefGoogle Scholar
  14. [14]
    Kokar, M.M., Wang, J.: Using ontologies for recognition: an example. In: Proceedings of the Fifth International Conference on Information Fusion. (2002) 1324– 1330Google Scholar
  15. [15]
    Staab, S., Santini, S., Maedche, F.N.L.S.A.: Emergent semantics. Intelligent Systems, IEEE 17 (2002) 78–86CrossRefGoogle Scholar
  16. [16]
    Oh, J., Lee, J., Vemuri, E.: An efficient technique for segmentation of key object(s) from video shots. In: Proc of International Conference on Information Technology: Coding and Computing (ITCC 2003), Las Vegas, Nevada (2003)Google Scholar
  17. [17]
    Peim, M., Franconi, E., Paton, N.W., Goble, C.A.: Query processing with description logic ontologies over object-wrapped databases. In: Scientific and Statistical Database Management, 2002. Proceedings. 14th International Conference. (2002) 27– 36Google Scholar
  18. [18]
    Zhang, Z., Xing, C., Zhou, L., Feng, J.: Cyc: A large-scale investment in knowledge infrastructure. In: Advanced Information Networking and Applications, 2003. 17th International Conference on. (2003) 628–631Google Scholar
  19. [19]
    Zhenning, X., Weiming, Z., Kaige, H., Yong, L., You, L., Daquan, T.: Study and implementation of a semantic information query system based on ontology. In: Proceedings International Conferences on Infotech and Info-net. Volume 3., Beijing (2001) 26– 31Google Scholar
  20. [20]
    Oh, J., Thenneru, M., Jiang, N.: Hierarchical video indexing based on changes of camera. In: Proc. of The Eighteenth Annual ACM Symposium on Applied Computing (SAC2003), Melbourne, Florida, USA (2003)Google Scholar
  21. [21]
    Oh, J.: Object(s) extraction from video sequences using color quantization. In: Proc. of Second Int’l workshop on Intelligent Multimedia Computing and Networking, Durham, NC (2002)Google Scholar
  22. [22]
    Cruz, I.F., Rajendran, A.: Semantic data integration in hierarchical domains. Intelligent Systems, IEEE 18 (2003) 66–73CrossRefGoogle Scholar
  23. [23]
    Lenat, D.: Cyc: A large-scale investment in knowledge infrastructure. In: Proc. of Communications of the ACM. (1995) 33– 38Google Scholar
  24. [24]
    Miller, G.: Wordnet: A lexical database for english. In: Proc. of Communication of CACM. (1995)Google Scholar
  25. [25]
    Wang, L., Khan, L., Breen, C.: Object boundary detection for ontology-based image classification. In: Proc. of Third International Workshop on Multimedia Data Mining in Conjunction with Eighth ACM SIGKDD, Edmonton, Alberta, Canada (2002) 51–61Google Scholar
  26. [26]
    Khan, L., Luo, F.: Ontology construction for information selection. In: Proceedings. 14th IEEE International Conference on Tools with Artificial Intelligence (ICTAI 2002). (2002) 122– 127Google Scholar
  27. [27]
    Khan, L., Wang, L.: Automatic ontology derivation using clustering for image classification. In: Proc. Of Eighth International Workshop on Multimedia Information Systems, Tempe, Arizona (2002) 56– 65Google Scholar
  28. [28]
    Khan, L., McLeod, D.: Audio structuring and personalized retrieval using ontologies. In: Proc. of IEEE Advances in Digital Libraries, Library of Congress, Washington, DC (2000)Google Scholar
  29. [29]
    Hunter, J.: Adding multimedia to the semantic web - building an mpeg-7 ontology (2001)Google Scholar
  30. [30]
    Hunter, J.: Enhancing the semantic interoperability of multimedia through a core ontology. IEEE Transactions on Circuits and Systems for Video Technology 13 (2003) 49–58CrossRefMathSciNetGoogle Scholar
  31. [31]
    Khan, L., McLeod, D.: Audio structuring and personalized retrieval using ontologies. In: Proceedings. Advances in Digital Libraries. (2000) 116– 126 14Google Scholar
  32. [32]
    Mao, W.Z., Bell, D.: Integrating visual ontologies and wavelets for image content retrieval. In: Proceedings. Ninth International Workshop on Database and Expert Systems Applications. (1998) 379– 384Google Scholar
  33. [33]
    Grosso, W., Eriksson, H., Ferguson, R., Gennari, J., Tu, S., Musen, M.: Knowledge modeling at the millennium (the design and evolution of protege-2000). In: Proc. of the 12th Workshop of on Knowledge Acquisition, Modeling and Management (KAW-1999), Banff, Alberta, Canada (1999)Google Scholar
  34. [34]
    Noy, N.F., Sintek, M., Decker, S., Crubezy, M., Fergerson, R.W., Musen, M.A.: Creating semantic web contents with protege-2000. Intelligent Systems, IEEE 16 (2001) 60–71CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media B.V. 2008

Authors and Affiliations

  • Jeongkyu Lee
    • 1
  • Munther H. Abualkibash
    • 1
  • Padmini K. Ramalingam
    • 1
  1. 1.Department of Computer Science and EngineeringUniversity of BridgeportBridgeport

Personalised recommendations