Multimedia Tools and Applications

, Volume 49, Issue 1, pp 167–194 | Cite as

Investigating fuzzy DLs-based reasoning in semantic image analysis

  • Stamatia DasiopoulouEmail author
  • Ioannis Kompatsiaris
  • Michael G. Strintzis


Recent advances in semantic image analysis have brought forth generic methodologies to support concept learning at large scale. The attained performance however is highly variable, reflecting effects related to similarities and variations in the visual manifestations of semantically distinct concepts, much as to the limitations issuing from considering semantics solely in the form of perceptual representations. Aiming to enhance performance and improve robustness, we investigate a fuzzy DLs-based reasoning framework, which enables the integration of scene and object classifications into a semantically consistent interpretation by capturing and utilising the underlying semantic associations. Evaluation with two sets of input classifiers, configured so as to vary with respect to the wealth of concepts’ interrelations, outlines the potential of the proposed approach in the presence of semantically rich associations, while delineating the issues and challenges involved.


Fuzzy reasoning Semantic image analysis Semantic integration Fuzzy DLs Inconsistency handling 



This work was partially supported by the European Commission under contracts FP6-001765 aceMedia, FP6-507482 KnowledgeWeb and FP7-215453 WeKnowIt.


  1. 1.
    Assfalg J, Bertini M, Colombo C, Bimbo AD (2002) Semantic annotation of sports videos. IEEE Multimed 9(2):52–60CrossRefGoogle Scholar
  2. 2.
    Baader F, Calvanese D, McGuinness DL, Nardi D, Patel-Schneider PF (2003) e.: the description logic handbook: theory, implementation, and applications. In: Description logic handbook. Cambridge University Press, CambridgeGoogle Scholar
  3. 3.
    Bagdanov A, Bertini M, DelBimbo A, Serra G, Torniai C (2007) Semantic annotation and retrieval of video events using multimedia ontologies. In: Proc. IEEE international conference on semantic computing (ICSC), Irvine, CA, USAGoogle Scholar
  4. 4.
    Bechhofer S, van Harmelen F, Hendler J, Horrocks I, McGuinness D, Patel-Schneider P, Stein L (2004) OWL web ontology language reference, W3C Recommendation 10 February.
  5. 5.
    Bell D, Qi G, Liu W (2007) Approaches to inconsistency handling in description-logic based ontologies. In: Proc. OTM workshops, Vilamoura, Portugal, pp 1303–1311Google Scholar
  6. 6.
    Bobillo F, Straccia U (2008) Fuzzydl: an expressive fuzzy description logic reasoner. In: Proc. international conference on fuzzy systems (FUZZ). IEEE Computer Society, Hong Kong, pp 923–930Google Scholar
  7. 7.
    Brickley D, Guha RV (2004) RDF Vocabulary description language 1.0: RDF schema, W3C Recommendation 10 February.
  8. 8.
    Crevier D, Lepage R (1997) Knowledge-based image understanding systems: a survey. Comput Vis Image Underst 67:161–185CrossRefGoogle Scholar
  9. 9.
    Dubois D, Prade H (2001) Possibility theory, probability theory and multiple-valued logics: a clarification. Ann Math Artif Intell 32(1-4):35–66CrossRefMathSciNetGoogle Scholar
  10. 10.
    Elfers C, Herzog O, Miene A, Wagner T (2008) Qualitative abstraction and inherent uncertainty in scene recognition. In: Cohn AG, Hogg DC, Möller R, Neumann B (eds) Logic and B probabilty for scene interpretation, Dagstuhl Seminar Proceedings, WadernGoogle Scholar
  11. 11.
    Espinosa S, Kaya A, Melzer S, Möller R, Wessel M (2007) Multimedia interpretation as abduction. In: Proc. international workshop on description logics (DL), Brixen-Bressanone, ItalyGoogle Scholar
  12. 12.
    Haase P, Qi G (2007) An analysis of approaches to resolving inconsistencies in dl-based ontologies. In: Proc. international workshop on ontology dynamics (IWOD), Innsbruck, Austria, pp 97–109Google Scholar
  13. 13.
    Haase P, van Harmelen F, Huang Z, Stuckenschmidt H, Sure Y (2005) A framework for handling inconsistency in changing ontologies. In: Proc. of international semantic web conference (ISWC), Galway, Ireland, pp 353–367Google Scholar
  14. 14.
    Hanjalic A, Lienhart R, Ma W, Smith J (2008) The holy grail of multimedia information retrieval: so close or yet so far away. IEEE Proceedings (Special Issue on Multimedia Information Retrieval) 96(4):541–547Google Scholar
  15. 15.
    Hauptmann A, Yan R, Lin W (2007) How many high-level concepts will fill the semantic gap in news video retrieval? In: Proc. 6th ACM international conference on image and video retrieval (CIVR), Amsterdam, The Netherlands, pp 627–634Google Scholar
  16. 16.
    Hunter J, Drennan SL (2004) Realizing the hydrogen economy through semantic web technologies. IEEE Intelligent Systems Journal—Special Issue on eScience 19:40–47CrossRefGoogle Scholar
  17. 17.
    Kalyanpur A, Parsia B, Sirin E, Grau BC (2006) Repairing unsatisfiable concepts in owl ontologies. In: Proc. of European semantic web conference (ESWC), Budva, Montenegro, pp 170–184Google Scholar
  18. 18.
    Klir G, Yuan B (1995) Fuzzy sets and fuzzy logic: theory and applications. Prentice-Hall, Englewood CliffszbMATHGoogle Scholar
  19. 19.
    Lam J, Sleeman D, Pan J, Vasconcelos W (2008) A fine-grained approach to resolving unsatisfiable ontologies. J Data Semantics 10:62–95CrossRefGoogle Scholar
  20. 20.
    LeBorgne H, Guérin-Dugué A, O’Connor N (2007) Learning midlevel image features for natural scene and texture classification. IEEE Trans Circuits Syst Video Technol 17(3):286–297CrossRefGoogle Scholar
  21. 21.
    Little S, Hunter J (2004) Rules-by-example—a novel approach to semantic indexing and querying of images. In: International semantic web conference (ISWC), Hiroshima, Japan, pp 534–548Google Scholar
  22. 22.
    Moller R, Neumann B, Wessel M (1999) Towards computer vision with description logics: some recent progress. In: Proceedings integration of speech and image understanding, Corfu, Greece, pp 101–115Google Scholar
  23. 23.
    Moosmann F, Triggs B, Jurie F (2006) Randomized clustering forests for building fast and discriminative visual vocabularies. In: Neural information processing systems (NIPS)Google Scholar
  24. 24.
    Mylonas P, Simou N, Tzouvaras V, Avrithis Y (2007) Towards semantic multimedia indexing by classification and reasoning on textual metadata. Knowledge acquisition from multimedia content workshop, Genova, ItalyGoogle Scholar
  25. 25.
    Naphade M, Huang T (2001) A probabilistic framework for semantic video indexing, filtering, and retrieval. IEEE Trans Multimedia 3(1):141–151CrossRefGoogle Scholar
  26. 26.
    Naphade M, Huang T (2002) Extracting semantics from audio-visual content: the final frontier in multimedia retrieval. IEEE Trans Neural Netw 13(4):793–810CrossRefGoogle Scholar
  27. 27.
    Naphade M, Kennedy L, Kender J, Chang SF, Smith J, Over P, Hauptmann A (2005) A light scale concept ontology for multimedia understanding for trecvid 2005. In: RC23612 W0505-104, computer science, IBM Research ReportGoogle Scholar
  28. 28.
    Natsev A, Jiang W, Merler M, Smith J, Tesic J, Xie L Yan R (2008) IBM Research TRECVID-2008 video retrieval system. In: Proc. TREC Video Retrieval Workshop, GaithersburgGoogle Scholar
  29. 29.
    Neumann B (2008) Bayesian compositional hierarchies—a probabilistic structure for scene interpretation. In: Dagstuhl seminar proceedingsGoogle Scholar
  30. 30.
    Neumann B, Moller R (2004) On scene interpretation with description logics (FBI-B-257/04)Google Scholar
  31. 31.
    Neumann B, Möller R (2007) On scene interpretation with description logics. Image Vis Comput (Special Issue on Cognitive Vision) 26:82–101Google Scholar
  32. 32.
    Niemann H, Sagerer G, Schröder S, Kummert F (1990) Ernest: a semantic network system for pattern understanding. IEEE Trans Pattern Anal Mach Intell 12(9):883–905CrossRefGoogle Scholar
  33. 33.
    Papadopoulos G, Mylonas P, Mezaris V, Avrithis Y, Kompatsiaris I (2006) Knowledge-assisted image analysis based on context and spatial optimization. Int J Semantic Web Inf Syst 2(3):17–36Google Scholar
  34. 34.
    Petridis K, Bloehdorn S, Saathoff C, Simou N, Dasiopoulou S, Tzouvaras V, Handschuh S, Avrithis Y, Kompatsiaris I, Staab S (2006) Knowledge representation and semantic annotation of multimedia content. IEE Proc Vis Image Signal Process (Special issue on Knowledge-Based Digital Media Processing) 153:255–262Google Scholar
  35. 35.
    Rao A, Jain R (1988) Knowledge representation and control in computer vision systems. IEEE Expert 3:64–79CrossRefGoogle Scholar
  36. 36.
    Reiter R, Mackworth A (1990) A logical framework for depiction and image interpretation. Artif Intell 41:125–155CrossRefMathSciNetGoogle Scholar
  37. 37.
    Richardson M, Domingos P (2006) Markov logic networks. Mach Learn 62(1–2):107–136CrossRefGoogle Scholar
  38. 38.
    Schober JP, Hermes T, Herzog O (2004) Content-based image retrieval by ontology-based object recognition. In: Proc. KI-2004 workshop on applications of description logics (ADL), Ulm, GermanyGoogle Scholar
  39. 39.
    Simou N, Athanasiadis T, Tzouvaras V, Kollias S (2007) Multimedia reasoning with f-shin. In: 2nd international workshop on semantic media adaptation and personalization, London, UKGoogle Scholar
  40. 40.
    Smeaton A, Over P, Kraaij W (2006) Evaluation campaigns and trecvid. In: MIR ’06: proceedings of the 8th ACM international workshop on multimedia information retrieval. ACM, New York, pp 321–330CrossRefGoogle Scholar
  41. 41.
    Smeulders AWM, Worring M, Santini S, Gupta A, Jain R (2000) Content-based image retrieval at the end of the early years. IEEE Trans Pattern Anal Mach Intell 22(12):1349–1380CrossRefGoogle Scholar
  42. 42.
    Snoek C, Worring M, van Gemert J, Geusebroek J, Smeulders A (2006) The challenge problem for automated detection of 101 semantic concepts in multimedia. In: Proc. 14th ACM international conference on multimedia, Santa Barbara, CA, USA, pp 421–430Google Scholar
  43. 43.
    Snoek C, Huurnink B, Hollink L, de Rijke M, Schreiber G, Worring M (2007) Adding semantics to detectors for video retrieval. IEEE Trans Multimedia 9(5):975–986CrossRefGoogle Scholar
  44. 44.
    Snoek C, van de Sande K, deRooij O, Huurnink B, van Gemert J, Uijlings J, He J, Li X, Everts I, Nedovic V, van Liempt M, van Balen R, Yan F, Tahir M, Mikolajczyk K, Kittler J, de Rijke M, Geusebroek J, Gevers T, Worring M, Smeulders A, Koelma D (2008) The MediaMill TRECVID 2008 semantic video search engine. University of Amsterdam, AmsterdamGoogle Scholar
  45. 45.
    Stoilos G, Stamou G, Pan J (2006) Handling imprecise knowledge with fuzzy description logic. In: Proc. international workshop on description logics (DL), Lake District, UKGoogle Scholar
  46. 46.
    Stoilos G, Stamou G, Pan J, Tzouvaras V, Horrocks I (2007) Reasoning with very expressive fuzzy description logics. J Artif Intell Res (JAIR) 30:273–320zbMATHMathSciNetGoogle Scholar
  47. 47.
    Stoilos G, Stamou G, Tzouvaras V, Pan J, Horrocks I (2005) The fuzzy description logic f-SHIN. In: International workshop on uncertainty reasoning for the semantic web (URSW), Galway, IrelandGoogle Scholar
  48. 48.
    Straccia U (2001) Reasoning within fuzzy description logics. J Artif Intell Res (JAIR) 14:137–166zbMATHMathSciNetGoogle Scholar
  49. 49.
    Straccia U (2004) Transforming fuzzy description logics into classical description logics. In: Proc. European conference on logics in artificial intelligence (JELIA), Lisbon, Portugal, pp 385–399Google Scholar
  50. 50.
    Straccia U (2006) A fuzzy description logic for the semantic web. In: Sanchez E (ed) Fuzzy logic and the semantic web, capturing intelligence. Elsevier, Amsterdam, pp 73–90Google Scholar
  51. 51.
    Town C, Sinclair D (2003) A self-referential perceptual inference framework for video interpretation. In: International confernce on computer vision systems (ICVS), Graz, Austria, pp 54–67Google Scholar
  52. 52.
    Umberto S, Giulio V (2007) Dlmedia: an ontology mediated multimedia information retrieval system. In: Proc. international workshop on description logics (DL), Brixen-Bressanone, ItalyGoogle Scholar
  53. 53.
    Zadeh L (1965) Fuzzy sets. Inf Control 8(32):338–353zbMATHCrossRefMathSciNetGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2009

Authors and Affiliations

  • Stamatia Dasiopoulou
    • 1
    Email author
  • Ioannis Kompatsiaris
    • 1
  • Michael G. Strintzis
    • 1
  1. 1.Informatics and Telematics InstituteCentre for Research and Technology HellasThessalonikiGreece

Personalised recommendations