Advertisement

Mixing Low-Level and Semantic Features for Image Interpretation

A Framework and a Simple Case Study
  • Ivan DonadelloEmail author
  • Luciano Serafini
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8926)

Abstract

Semantic Content-Based Image Retrieval (SCBIR) allows users to retrieve images via complex expressions of some ontological language describing a domain of interest. SCBIR adds some flexibility to the state-of-the-art methods for image retrieval, which support query either by keywords or by image examples. The price for this additional flexibility is the generation of a semantically rich description of the image content reflecting the ontology constraints. Generating these semantic interpretations is an open research problem. This paper contributes to this research line by proposing an approach for SCBIR based on the somehow natural idea that the interpretation of a picture is an (onto) logical model of an ontology that describes the domain of the picture. We implement this idea in an unsupervised method that jointly exploits the ontological constraints and the low-level features of the image. The preliminary evaluation, presented in the paper, shows promising results.

Keywords

Computer vision Ontologies Semantic image interpretation 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Abeel, T., Van de Peer, Y., Saeys, Y.: Javaml: A machine learning library. J. Mach. Learn. Res. 10, 931–934 (2009). http://dl.acm.org/citation.cfm?id=1577069.1577103
  2. 2.
    Antanas, L., Frasconi, P., Costa, F., Tuytelaars, T., Raedt, L.D.: A relational kernel-based framework for hierarchical image understanding. In: Gimel’farb, G.L., Hancock, E.R., Imiya, A., Kuijper, A., Kudo, M., Omachi, S., Windeatt, T., Yamada, K. (eds.) SSPR/SPR. LNCS, vol. 7626, pp. 171–180. Springer, Heidelberg (2012)Google Scholar
  3. 3.
    Antanas, L., van Otterlo, M., Mogrovejo, J.O., Tuytelaars, T., Raedt, L.D.: A relational distance-based framework for hierarchical image understanding. In: Carmona, P.L., Sánchez, J.S., Fred, A.L.N. (eds.) ICPRAM (2), pp. 206–218. SciTePress (2012)Google Scholar
  4. 4.
    Atif, J., Hudelot, C., Bloch, I.: Explanatory reasoning for image understanding using formal concept analysis and description logics. IEEE Transactions on Systems, Man, and Cybernetics: Systems 44(5), 552–570 (2014)CrossRefGoogle Scholar
  5. 5.
    Baader, F., Calvanese, D., McGuinness, D.L., Nardi, D., Patel-Schneider, P.F. (eds.): The Description Logic Handbook: Theory, Implementation, and Applications. Cambridge University Press, New York (2003)Google Scholar
  6. 6.
    Bannour, H., Hudelot, C.: Towards ontologies for image interpretation and annotation. In: Martinez, J.M. (ed.) 9th International Workshop on Content-Based Multimedia Indexing, CBMI 2011, June 13–15, Madrid, Spain, pp. 211–216. IEEE (2011)Google Scholar
  7. 7.
    Dasiopoulou, S., Kompatsiaris, I., Strintzis, M.G.: Applying fuzzy dls in the extraction of image semantics. J. Data Semantics 14, 105–132 (2009)CrossRefGoogle Scholar
  8. 8.
    Diligenti, M., Gori, M., Maggini, M., Rigutini, L.: Bridging logic and kernel machines. Machine Learning 86(1), 57–88 (2012)CrossRefzbMATHMathSciNetGoogle Scholar
  9. 9.
    Espinosa, S., Kaya, A., Möller, R.: Logical formalization of multimedia interpretation. In: Paliouras, G., Spyropoulos, C.D., Tsatsaronis, G. (eds.) Multimedia Information Extraction. LNCS, vol. 6050, pp. 110–133. Springer, Heidelberg (2011). http://dx.doi.org/10.1007/978-3-642-20795-2_5
  10. 10.
    Fellbaum, C. (ed.): WordNet: an electronic lexical database. MIT Press (1998)Google Scholar
  11. 11.
    Haarslev, V., Hidde, K., Möller, R., Wessel, M.: The racerpro knowledge representation and reasoning system. Semantic Web Journal 3(3), 267–277 (2012)Google Scholar
  12. 12.
    Han, F., Zhu, S.C.: Bottom-up/top-down image parsing by attribute graph grammar. In: Tenth IEEE International Conference on Computer Vision, ICCV 2005, vol. 2, pp. 1778–1785 (October 2005)Google Scholar
  13. 13.
    Hobbs, J.R., Stickel, M.E., Appelt, D.E., Martin, P.: Interpretation as abduction. Artif. Intell. 63(1–2), 69–142 (1993). http://dx.doi.org/10.1016/0004-3702(93)90015-4
  14. 14.
    Hudelot, C., Atif, J., Bloch, I.: Fuzzy spatial relation ontology for image interpretation. Fuzzy Sets and Systems 159(15), 1929–1951 (2008); from Knowledge Representation to Information Processing and Management Selected papers from the French Fuzzy Days (LFA 2006). http://www.sciencedirect.com/science/article/pii/S0165011408001012
  15. 15.
    Kohonen, T.: The self-organizing map. Proceedings of the IEEE 78(9), 1464–1480 (1990)CrossRefGoogle Scholar
  16. 16.
    Liu, H., Bao, H., Xu, D.: Concept vector for semantic similarity and relatedness based on wordnet structure. J. Syst. Softw. 85(2), 370–381 (2012). http://dx.doi.org/10.1016/j.jss.2011.08.029
  17. 17.
    Liu, Y., Zhang, D., Lu, G., Ma, W.Y.: A survey of content-based image retrieval with high-level semantics. Pattern Recognition 40(1), 262–282 (2007). http://dx.doi.org/10.1016/j.patcog.2006.04.045
  18. 18.
    Moller, R., Neumann, B., Wessel, M.: Towards computer vision with description logics: some recent progress. In: Proceedings of the Integration of Speech and Image Understanding, pp. 101–115 (1999)Google Scholar
  19. 19.
    Neumann, B., Mller, R.: On scene interpretation with description logics. Image and Vision Computing 26(1), 82–101 (2008) cognitive Vision-Special Issue. http://www.sciencedirect.com/science/article/pii/S0262885607001394
  20. 20.
    Neumann, B., Weiss, T.: Navigating through logic-based scene models for high-level scene interpretations. In: Crowley, J.L., Piater, J.H., Vincze, M., Paletta, L. (eds.) ICVS 2003. LNCS, vol. 2626, pp. 212–222. Springer, Heidelberg (2003). http://dl.acm.org/citation.cfm?id=1765473.1765497
  21. 21.
    Oliva, A., Torralba, A.: The role of context in object recognition. Trends in Cognitive Sciences 11(12), 520–527 (2007)CrossRefGoogle Scholar
  22. 22.
    Peraldi, I.S.E., Kaya, A., Möller, R.: Formalizing multimedia interpretation based on abduction over description logic aboxes. In: Grau, B.C., Horrocks, I., Motik, B., Sattler, U. (eds.) Description Logics. CEUR Workshop Proceedings, vol. 477. CEUR-WS.org. (2009)Google Scholar
  23. 23.
    Reiter, R., Mackworth, A.K.: A logical framework for depiction and image interpretation. Artificial Intelligence 41(2), 125–155 (1989)CrossRefzbMATHMathSciNetGoogle Scholar
  24. 24.
    Richardson, M., Domingos, P.: Markov logic networks. Machine Learning 62(1–2), 107–136 (2006)CrossRefGoogle Scholar
  25. 25.
    Russell, B.C., Torralba, A., Murphy, K.P., Freeman, W.T.: Labelme: A database and web-based tool for image annotation. Int. J. Comput. Vision 77(1–3), 157–173 (2008). http://dx.doi.org/10.1007/s11263-007-0090-8
  26. 26.
    Schroder, C., Neumann, B.: On the logics of image interpretation: model-construction in a formal knowledge-representation framework. In: Proceedings. of the Int. Conf. on Image Processing, vol. 1, pp. 785–788 (September 1996)Google Scholar
  27. 27.
    Sirin, E., Parsia, B., Grau, B.C., Kalyanpur, A., Katz, Y.: Pellet: A practical owl-dl reasoner. Web Semant. 5(2), 51–53 (2007). http://dx.doi.org/10.1016/j.websem.2007.03.004
  28. 28.
    Smith, B., von Ehrenfels, C., Verlag, P.: Foundations of Gestalt theory. Philosophia Verlag Munich, Germany (1988)Google Scholar
  29. 29.
    Socher, R., Lin, C.C., Manning, C., Ng, A.Y.: Parsing natural scenes and natural language with recursive neural networks. In: Proceedings of the 28th International Conference on Machine Learning (ICML 2011), pp. 129–136 (2011)Google Scholar
  30. 30.
    Staruch, B., Staruch, B.: First order theories for partial models. Studia Logica 80(1), 105–120 (2005)CrossRefzbMATHMathSciNetGoogle Scholar
  31. 31.
    Straccia, U.: Reasoning within fuzzy description logics. J. Artif. Intell. Res. (JAIR) 14, 137–166 (2001)zbMATHMathSciNetGoogle Scholar
  32. 32.
    Zlatoff, N., Tellez, B., Baskurt, A.: Image understanding and scene models: a generic framework integrating domain knowledge and gestalt theory. In: International Conference on Image Processing, ICIP 2004, vol. 4, pp. 2355–2358 (October 2004)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  1. 1.Fondazione Bruno KesslerTrentoItaly
  2. 2.Department of Information and Communication TechnologyUniversity of TrentoTrentoItaly

Personalised recommendations