Using Knowledge Representation Languages for Video Annotation and Retrieval

  • M. Bertini
  • G. D’Amico
  • A. Del Bimbo
  • C. Torniai
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4027)


Effective usage of multimedia digital libraries has to deal with the problem of building efficient content annotation and retrieval tools. In particular in video domain, different techniques for manual and automatic annotation and retrieval have been proposed. Despite the existence of well-defined and extensive standards for video content description, such as MPEG-7, these languages are not explicitly designed for automatic annotation and retrieval purpose. Usage of linguistic ontologies for video annotation and retrieval is a common practice to classify video elements by establishing relationships between video contents and linguistic terms that specify domain concepts at different abstraction levels. The main issue related to the use of description languages such as MPEG-7 or linguistic ontologies is due to the fact that linguistic terms are appropriate to distinguish event and object categories but they are inadequate when they must describe specific or complex patterns of events or video entities. In this paper we propose the usage of knowledge representation languages to define ontologies enriched with visual information that can be used effectively for video annotation and retrieval. Difference between content description languages and knowledge representation languages are shown, the advantages of using enriched ontologies both for the annotation and the retrieval process are presented in terms of enhanced user experience in browsing and querying video digital libraries.


Linguistic Term Semantic Annotation Visual Concept High Level Concept Video Annotation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Leonardi, R., Migliorati, P.: Semantic indexing of multimedia documents. IEEE Multimedia 9(2), 44–51 (2002)CrossRefGoogle Scholar
  2. 2.
    Assfalg, J., Bertini, M., Colombo, C., Del Bimbo, A., Nunziati, W.: Semantic annotation of soccer videos: Automatic highlights identification. Computer Vision and Image Understanding 92(2-3), 285–305 (2003)CrossRefGoogle Scholar
  3. 3.
    Yu, X., Xu, C., Leung, H., Tian, Q., Tang, Q., Wan, K.W.: Trajectory-based ball detection and tracking with applications to semantic analysis of broadcast soccer video. In: ACM Multimedia 2003, Berkeley, CA (USA), vol. 3, pp. 11–20 (2003)Google Scholar
  4. 4.
    Reidsma, D., Kuper, J., Declerck, T., Saggion, H., Cunningham, H.: Cross document ontology based information extraction for multimedia retrieval. In: Supplementary proc. of the ICCS 2003, Dresden (2003)Google Scholar
  5. 5.
    Mezaris, V., Kompatsiaris, I., Boulgouris, N., Strintzis, M.: Real-time compressed-domain spatiotemporal segmentation and ontologies for video indexing and retrieval. IEEE Transactions on Circuits and Systems for Video Technology 14(5), 606–621 (2004)CrossRefGoogle Scholar
  6. 6.
    Jaimes, A., Tseng, B., Smith, J.R.: Modal keywords, ontologies, and reasoning for video understanding. In: Bakker, E.M., Lew, M., Huang, T.S., Sebe, N., Zhou, X.S. (eds.) CIVR 2003. LNCS, vol. 2728. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  7. 7.
    Jaimes, A., Smith, J.: Semi-automatic, data-driven construction of multimedia ontologies. In: Proc. of IEEE Int’l. Conference on Multimedia & Expo. (2003)Google Scholar
  8. 8.
    Benitez, A., Chang, S.: Automatic multimedia knowledge discovery, summarization and evaluation. IEEE Transactions on Multimedia (submitted, 2003)Google Scholar
  9. 9.
    Strintzis, J., Bloehdorn, S., Handschuh, S., Staab, S., Simou, N., Tzouvaras, V., Petridis, K., Kompatsiaris, I., Avrithis, Y.: Knowledge representation for semantic multimedia content analysis and reasoning. In: European Workshop on the Integration of Knowledge, Semantics and Digital Media Technology (2004)Google Scholar
  10. 10.
    Dasiopoulou, S., Mezaris, V., Kompatsiaris, I., Papastathis, V.K., Strintzis, M.G.: Knowledge-assisted semantic video object detection. IEEE Transactions on Circuits and Systems for Video Technology 15(10), 1210–1224 (2005)CrossRefGoogle Scholar
  11. 11.
    Hunter, J.: Adding multimedia to the semantic web: Building an MPEG-7 ontology. In: The First Semantic Web Working Symposium, SWWS 2001. Stanford University, California, USA (2001)Google Scholar
  12. 12.
    Hunter, J.: An RDF schema/DAML+OIL representation of MPEG-7 semantics. Technical Report MPEG Document: ISO/IEC JTC1/SC29/WG11 W7807, ISO/IEC (2001)Google Scholar
  13. 13.
    Tsinaraki, C., Polydoros, P., Christodoulakis, S.: Interoperability support for ontology-based video retrieval applications. In: Enser, P.G.B., Kompatsiaris, Y., O’Connor, N.E., Smeaton, A.F., Smeulders, A.W.M. (eds.) CIVR 2004. LNCS, vol. 3115, pp. 582–591. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  14. 14.
    Tsinaraki, C., Polydoros, P., Christodoulakis, S.: Interoperability of OWL with the MPEG-7 MDS. Technical report, Technical University of Crete / Laboratory of Distributed Multimedia Information Systems and Applications (TUC/MUSIC) (2004)Google Scholar
  15. 15.
    Bertini, M., Cucchiara, R., Del Bimbo, A., Torniai, C.: Video annotation with pictorially enriched ontologies. In: Proc. of IEEE Int’l. Conference on Multimedia & Expo. (2005)Google Scholar
  16. 16.
    Bertini, M., Bimbo, A.D., Torniai, C.: Enhanced ontologies for video annotation and retrieval. In: Proceedings of ACM MIR (2005)Google Scholar
  17. 17.
    Haarslev, V., Möller, R.: Description of the racer system and its applications. In: Proceedings International Workshop on Description Logics (DL 2001), Stanford, USA, August 1-3, pp. 131–141 (2001)Google Scholar
  18. 18.
    Haarslev, V., Möller, R., Wessel, M.: Querying the semantic web with racer + nrql. In: Proceedings of the KI 2004 International Workshop on Applications of Description Logics (ADL 2004), Ulm, German, September 24 (2004)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • M. Bertini
    • 1
  • G. D’Amico
    • 1
  • A. Del Bimbo
    • 1
  • C. Torniai
    • 1
  1. 1.D.S.I.Università di FirenzeItaly

Personalised recommendations