A Novel Approach to Multimedia Ontology Engineering for Automated Reasoning over Audiovisual LOD Datasets

  • Leslie F. Sikos
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9621)


Multimedia reasoning, which is suitable for, among others, multimedia content analysis and high-level video scene interpretation, relies on the formal and comprehensive conceptualization of the represented knowledge domain. However, most multimedia ontologies are not exhaustive in terms of role definitions, and do not incorporate complex role inclusions and role interdependencies. In fact, most multimedia ontologies do not have a role box at all, and implement only a basic subset of the available logical constructors. Consequently, their application in multimedia reasoning is limited. To address the above issues, VidOnt, the very first multimedia ontology with \( {{\mathcal{S}\mathcal{R}\mathcal{O}\mathcal{I}\mathcal{Q}}^{{({\mathcal{D}})}}} \) expressivity and a DL-safe ruleset has been introduced for next-generation multimedia reasoning. In contrast to the common practice, the formal grounding has been set in one of the most expressive description logics, and the ontology validated with industry-leading reasoners, namely HermiT and FaCT++. This paper also presents best practices for developing multimedia ontologies, based on my ontology engineering approach.


Ontology OWL MPEG-7 Video metadata Video retrieval Linked Open Data Knowledge representation 


  1. 1.
    Meghini, C., Sebastiani, F., Straccia, U.: Reasoning about the Form and Content of Multimedia Objects. In: AAAI 1997 Spring Symposium on Intelligent Integration and Use of Text, Image, Video and Audio, pp. 89–94. AAAI Press, Menlo Park (1997)Google Scholar
  2. 2.
    Simou, N., Athanasiadis, T., Tzouvaras, V., Kollias, S.: Multimedia reasoning with f–SHIN. In: Second International Workshop on Semantic Media Adaptation and Personalization, IEEE (2007). doi: 10.1109/SMAP.2007.40
  3. 3.
    Simou, N., Saathoff, C., Dasiopoulou, S., Spyrou, E., Voisine, N., Tzouvaras, V., Kompatsiaris, I., Avrithis, Y., Staab, S.: An ontology infrastructure for multimedia reasoning. In: Atzori, L., Giusto, D.D., Leonardi, R., Pereira, F. (eds.) VLBV 2005. LNCS, vol. 3893, pp. 51–60. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  4. 4.
    Town, C.: Ontological inference for image and video analysis. Mach. Vis. Appl. 17(2), 94–115 (2006). doi: 10.1007/s00138-006-0017-3 CrossRefGoogle Scholar
  5. 5.
    Gómez-Romero, J., Patricio, M.A., García, J., Molina, J.M.: Ontology-based context representation and reasoning for object tracking and scene interpretation in video. Expert Syst. Appl. 38, 7494–7510 (2011). doi: 10.1016/j.eswa.2010.12.118 CrossRefGoogle Scholar
  6. 6.
    Möller, R., Neumann, B.: Ontology-based reasoning techniques for multimedia interpretation and retrieval. In: Semantic Multimedia and Ontologies. Springer, London (2008). doi: 10.1007/978-1-84800-076-6_3
  7. 7.
    Elleuch, N., Zarka, M., Ammar, A.B., Alimi, A.M.: A fuzzy ontology-based framework for reasoning in visual video content analysis and indexing. In: 11th International Workshop on Multimedia Data Mining (MDMKDD 2011), San Diego (2011). doi: 10.1145/2237827.2237828
  8. 8.
    Jaimes, A., Tseng, B.L., Smith, J.R.: Modal keywords, ontologies, and reasoning for video understanding. In: Bakker, E.M., Lew, M.S., Huang, T.S., Sebe, N., Zhou, X.S. (eds.) Image and Video Retrieval. LNCS, vol. 2728, pp. 248–259. Springer, Heidelberg (2003). doi: 10.1007/3-540-45113-7_25 CrossRefGoogle Scholar
  9. 9.
    Dasiopoulou, S., Heinecke, J., Saathoff, C., Strintzis, M.G.: Multimedia reasoning with natural language support. In: IEEE Sixth International Conference on Semantic Computing, pp. 413–420. IEEE (2007). doi: 10.1109/ICSC.2007.28
  10. 10.
    D’Odorico, T., Bennett, B.: Automated reasoning on vague concepts using formal ontologies, with an application to event detection on video data. In: 11th International Symposium on Logical Formalizations of Commonsense Reasoning (COMMONSENSE 2013), Ayia Napa (2013)Google Scholar
  11. 11.
    Ballan, L., Bertini, M., Del Bimbo, A., Serra, G.: Semantic annotation of soccer videos by visual instance clustering and spatial/temporal reasoning in ontologies. Multimedia Tools Appl. 48, 313–337 (2010). doi: 10.1007/s11042-009-0342-4 CrossRefGoogle Scholar
  12. 12.
    Sikos, L.F., Powers, D.M.W.: Knowledge-driven video information retrieval with LOD: from semi-structured to structured video metadata. In: Eighth Workshop on Exploiting Semantic Annotations in Information Retrieval (ESAIR 2015), Melbourne (2015). doi: 10.1145/2810133.2810141
  13. 13.
    VidOnt: The Video Ontology.
  14. 14.
    Sikos, L.F.: Mastering Structured Data on the Semantic Web: From HTML5 Microdata to Linked Open Data. Apress Media, New York (2015). doi: 10.1007/978-1-4842-1049-9 CrossRefGoogle Scholar
  15. 15.
    Motik, B., Sattler, U., Studer, R.: Query answering for OWL-DL with rules. J. Web Semant. 3(1), 41–60 (2005). doi: 10.1016/j.websem.2005.05.001 CrossRefGoogle Scholar
  16. 16.
    Hitzler, P., Krötzsch, M., Rudolph, S.: Foundations of Semantic Web Technologies. CRC Press, Boca Raton (2009)Google Scholar
  17. 17.
    Simou, N., Tzouvaras, V., Avrithis, Y., Stamou, G., Kollias, S.: A visual descriptor ontology for multimedia reasoning. In: 6th International Workshop on Image Analysis for Multimedia Interactive Services, Montreux (2005)Google Scholar
  18. 18.
    Glimm, B., Horrocks, I., Motik, B., Stoilos, G., Wang, Z.: HermiT: An OWL 2 Reasoner. J. Autom. Reasoning 53(3), 245–269 (2014). doi: 10.1007/s10817-014-9305-1 CrossRefMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2016

Authors and Affiliations

  1. 1.Flinders UniversityAdelaideAustralia

Personalised recommendations