Multimedia Tools and Applications

, Volume 46, Issue 2–3, pp 331–370 | Cite as

Enquiring MPEG-7 based multimedia ontologies

  • Stamatia DasiopoulouEmail author
  • Vassilis Tzouvaras
  • Ioannis Kompatsiaris
  • Michael G. Strintzis


Machine understandable metadata forms the main prerequisite for the intelligent services envisaged in a Web, which going beyond mere data exchange and provides for effective content access, sharing and reuse. MPEG-7, despite providing a comprehensive set of tools for the standardised description of audiovisual content, is largely compromised by the use of XML that leaves the largest part of the intended semantics implicit. Aspiring to formalise MPEG-7 descriptions and enhance multimedia metadata interoperability, a number of multimedia ontologies have been proposed. Though sharing a common vision, the developed ontologies are characterised by substantial conceptual differences, reflected both in the modelling of MPEG-7 description tools as well as in the linking with domain ontologies. Delving into the principles underlying their engineering, we present a systematic survey of the state of the art MPEG-7 based multimedia ontologies, and highlight issues that hinder interoperability as well as possible directions towards their harmonisation.


Multimedia ontologies MPEG-7 Semantic Web Multimedia metadata Interoperability 



This work was partially supported by the European Commission under contracts FP6-001765 aceMedia, FP6-507482 KnowledgeWeb, and FP6-027538 BOEMIE.


  1. 1.
    Arndt R, Troncy R, Staab S, Hardman L, Vacura M (2007) COMM: designing a well-founded multimedia ontology for the web. In: Proc international Semantic Web conference, Busan, Korea, 11–15 November 2007Google Scholar
  2. 2.
    Bechhofer S, van Harmelen F, Hendler J, Horrocks I, McGuinness D, Patel-Schneider P, Stein L (2004) OWL Web ontology language reference. W3C recommendation, 10 February 2004.
  3. 3.
    Berners-Lee T, Hendler J, Lassila O (2003) The Semantic Web. Sci Am 284:34–43CrossRefGoogle Scholar
  4. 4.
    Bloehdorn S, Simou N, Tzouvaras V, Petridis K, Handschuh S, Avrithis Y, Kompatsiaris I, Staab S, Strintzis M (2004) Knowledge representation for Semantic multimedia content analysis and reasoning. In: Proc of European workshop on the integration of knowledge, semantics and digital media technology (EWIMT), London, UK, 25–26 November 2007Google Scholar
  5. 5.
    Boll S, Burger T, Celma O, Halaschek-Wiener C, Mannens E, Troncy R (2007) Multimedia vocabularies on the Semantic Web. In: W3C incubator group report, 24 July 2007Google Scholar
  6. 6.
    Brickley D, Guha R (2004) RDF vocabulary description language 1.0 - RDF schema. W3C recommendation, 10 February 2004.
  7. 7.
    Celma O, Dasiopoulou S, Hausenblas M, Little S, Tsinaraki C, Troncy R (2007) Mpeg-7 and the Semantic Web. In: W3C incubator group editor’s draft, 14 August 2007Google Scholar
  8. 8.
    Dasiopoulou S, Tzouvaras V, Kompatsiaris I, Strintzis M (2007) Capturing MPEG-7 semantics. In: Proc 2nd international conference on metadata and semantics (MTSR), Corfu, GreeceGoogle Scholar
  9. 9.
    Dasiopoulou S, Dalakleidi K, Tzouvaras V, Kompatsiaris I (2007) D3.4 - mulitmedia content and descriptor ontologies—version 2. Boemie Technical ReportGoogle Scholar
  10. 10.
    Dasiopoulou S, Saathoff C, Mylonas P, Avrithis Y, Kompatsiaris Y, Staab S, Strintzis M (2008) Introducing context and reasoning in visual content analysis: an ontology-based framework. In: Hobson PM, Kompatsiaris Y (eds) Semantic multimedia and ontologies: theory and applications. Springer, New YorkGoogle Scholar
  11. 11.
    Dasiopoulou S, Dalakleidi K, Eleftherohorinou H, Mailis T, Tzouvaras V, Papastathis V, Kompatsiaris I, Avrithis Y (2009) D3.1 - mulitmedia content and descriptor ontologies—version 1. BOEMIE Technical ReportGoogle Scholar
  12. 12.
    Dublin Core Metadata Initiative (2008) Dublin Core Metadata Element Set, Version 1.1. DCMI recommendation.
  13. 13.
    Gangemi A (2005) Ontology design patterns for Semantic Web content. In: 4th international Semantic Web conference (ISWC), Galway, Ireland, 6–10 November 2005, pp 262–276Google Scholar
  14. 14.
    Gangemi A, Guarino N, Masolo C, Oltramari A, Schneider L (2002) Sweetening ontologies with DOLCE. In: 13th international conference on knowledge engineering and knowledge management (EKAW), Siguenza, Spain, 1–4 October, pp 166–181Google Scholar
  15. 15.
    García R, Celma O (2005) Semantic integration and retrieval of multimedia metadata. In: Proc international Semantic Web conference (ISWC), Galway, IrelandGoogle Scholar
  16. 16.
    García R, Gil R (2007) Facilitating business interoperability from the Semantic Web. In: Proc 10th international conference on business information systems (BIS), Poznan, Poland, pp 220–232Google Scholar
  17. 17.
    García R, Gil R, Delgado J (2007) A web ontologies framework for digital rights management. Artif Intell Law 15(2):137–154CrossRefGoogle Scholar
  18. 18.
    Hollink L, Little S, Hunter J (2005) Evaluating the application of semantic inferencing rules to image annotation. In: 3rd international conference on knowledge capture (K-CAP), Banff, Alberta, Canada, pp 91–98Google Scholar
  19. 19.
    Hunter J (2001) Adding multimedia to the Semantic Web: building an MPEG-7 ontology. In: Proc the first Semantic Web working symposium, SWWS’01, Stanford University, California, USAGoogle Scholar
  20. 20.
    Hunter J, Drennan J, Little S (2004) Realizing the hydrogen economy through Semantic Web technologies. IEEE Intell Syst J—Special issue on eScience 19:40–47CrossRefGoogle Scholar
  21. 21.
    Karkaletsis V, Paliouras G, Spyropoulos C (2005) A bootstrapping approach to knowledge acquisition from multimedia content with ontology evolution. In: Proc international conference on adaptive knowledge representation and reasoning (AKRR), Helsinki, Finland, pp 98–105Google Scholar
  22. 22.
    Lagoze C, Hunter J (2001) The ABC ontology and model. J Digit Inf 2(2)Google Scholar
  23. 23.
    Little S, Hunter J (2004) Rules-by-example—a novel approach to semantic indexing and querying of images. In: International Semantic Web conference, pp 534–548Google Scholar
  24. 24.
    Martínez J (2002) MPEG-7: overview of MPEG-7 description tools, part 2. IEEE Multimed 9(3):83–93CrossRefGoogle Scholar
  25. 25.
    MPEG-7 (2001) Multimedia content descritpion interface. Standard no. iso/iec 15938Google Scholar
  26. 26.
    MPEG-7 Visual (2001) ISO/IEC 15938-3/FDIS information technology—multimedia content description interface - part 3 visual. ISO/IEC JTC 1/SC 29/WG 11/N4358, SidneyGoogle Scholar
  27. 27.
    MPEG-7 Audio (2001) ISO/IEC 15938-4:2001(E)/FDIS information technology—multimedia content description interface—part 4 audio. ISO/IEC JTC 1/SC 29/WG 11/N4224, SidneyGoogle Scholar
  28. 28.
    MPEG-7 MDS (2001) ISO/IEC 15938-5/FDIS information technology—multimedia content description interface—part 5 multimedia description schemes. ISO/IEC JTC 1/SC 29/WG 11/N4242, SingaporeGoogle Scholar
  29. 29.
    Nack F, van Ossenbruggen J, Hardman L (2005) That obscure object of desire: multimedia metadata on the Web, part 2. IEEE MultiMed 12(1):54–63CrossRefGoogle Scholar
  30. 30.
    Niles I, Pease A (2001) Towards a standard upper ontology. In: 2nd international conference on formal ontology in information systems (FOIS), Ogunquit, Maine, USA, 17–19 October, pp 2–9Google Scholar
  31. 31.
    Oberle D, Ankolekar A, Hitzler P, Cimiano P, Sintek M, Kiesel M, Mougouie B, Baumann S, Vembu S, Romanelli M (2007) DOLCE ergo SUMO: on foundational and domain models in the SmartWeb integrated ontology (SWIntO). J Web Sem 5(3):156–174Google Scholar
  32. 32.
    Petridis K, Bloehdorn S, Saathoff C, Simou N, Dasiopoulou S, Tzouvaras V, Handschuh S, Avrithis Y, Kompatsiaris I, Staab S (2006) Knowledge representation and semantic annotation of multimedia content. IEE Proc Vis Image Signal Process 153:255–262CrossRefGoogle Scholar
  33. 33.
    Romanelli M, Buitelaar P, Sintek M (2007) Modeling linguistic facets of multimedia content for semantic annotation. In: 2nd international conference on semantic and digital media technologies (SAMT), Genova, Italy, pp 240–251Google Scholar
  34. 34.
    Simou N, Saathoff C, Dasiopoulou S, Spyrou E, Voisine N, Tzouvaras V, Kompatsiaris I, Avrithis Y, Staab S (2005) An ontology infrastructure for multimedia reasoning. In: Proc international workshop on very low bitrate video coding (VLBV 2005), Sardinia, ItalyGoogle Scholar
  35. 35.
    SMIL 1.0 (1998) Synchronized multimedia integration language specification. W3C recommendation, 15 June.
  36. 36.
    Troncy R, Celma O, Little S, GarciaGarcía R, Tsinaraki C (2007) Mpeg-7 based multimedia ontologies: interoperability support or interoperability issue? In: 1st workshop on multimedia annotation and retrieval enabled by shared ontologies (MARESO), Genova, Italy, pp 2–16Google Scholar
  37. 37.
    Tsinaraki C, Christodoulakis S (2007) Interoperability of XML schema applications with OWL domain knowledge and Semantic Web tools. In: On the move to meaningful internet systems (OTM), confederated international conferences, Vilamoura, Portugal, pp 850–869Google Scholar
  38. 38.
    Tsinaraki C, Polydoros P, Christodoulakis S (2004) Integration of OWL ontologies in MPEG-7 and TV-anytime compliant semantic indexing. In: 16th international conference on advanced information systems engineering (CAiSE), Riga, Latvia, 7–11 June 2004, pp 398–413Google Scholar
  39. 39.
    Tsinaraki C, Polydoros P, Christodoulakis S (2007) Interoperability support between MPEG-7/21 and OWL in DS-MIRF. IEEE Trans Knowl Data Eng 19(2):219–232CrossRefGoogle Scholar
  40. 40.
    van Ossenbruggen J, Nack F, Hardman L (2004) That obscure object of desire: multimedia metadata on the Web, part 1. IEEE MultiMed 11(4):38–48CrossRefGoogle Scholar
  41. 41.
    Vembue S, Kiesel M, Sintek M, Bauman S (2006) Towards bridging the semantic gap in multimedia annotation and retrieval. In: Proc workshop on Semantic Web annotations for multimedia (SWAMM), Edinburgh, ScotlandGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2009

Authors and Affiliations

  • Stamatia Dasiopoulou
    • 1
    Email author
  • Vassilis Tzouvaras
    • 2
  • Ioannis Kompatsiaris
    • 1
  • Michael G. Strintzis
    • 1
  1. 1.Informatics and Telematics Institute, Centre for Research and Technology HellasThessalonikiGreece
  2. 2.Department of Electrical and Computer EngineeringNational Technical University of AthensZographouGreece

Personalised recommendations