SAFIRE: Towards Standardized Semantic Rich Image Annotation

  • Christian Hentschel
  • Andreas Nürnberger
  • Ingo Schmitt
  • Sebastian Stober
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4398)


Most of the currently existing image retrieval systems make use of either low-level features or semantic (textual) annotations. A combined usage during annotation and retrieval is rarely attempted. In this paper, we propose a standardized annotation framework that integrates semantic and feature based information about the content of images. The presented approach is based on the MPEG-7 standard with some minor extensions. The proposed annotation system SAFIRE (Semantic Annotation Framework for Image REtrieval) enables the combined use of low-level features and annotations that can be assigned to arbitrary hierarchically organized image segments. Besides the framework itself, we discuss query formalisms required for this unified retrieval approach.


Fuzzy Logic Image Retrieval Query Image Image Annotation Image Collection 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bellman, R., Giertz, M.: On the Analytic Formalism of the Theory of Fuzzy Sets. Information Science 5, 149–156 (1973)CrossRefMathSciNetGoogle Scholar
  2. 2.
    Bimbo, A.D.: Visual Information Retrieval. Morgan Kaufmann, San Francisco (1999)Google Scholar
  3. 3.
    Bloehdorn, S., et al.: Semantic annotation of images and videos for multimedia analysis. In: Gómez-Pérez, A., Euzenat, J. (eds.) ESWC 2005. LNCS, vol. 3532, Springer, Heidelberg (2005)Google Scholar
  4. 4.
    Boughanem, M., Loiseau, Y., Prade, H.: Rank-ordering documents according to their relevance in information retrieval using refinements of ordered-weighted aggregations. In: Adaptive Multimedia Retrieval: User, Context, and Feedback, Postproc. of 3rd Int. Workshop, pp. 44–54. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  5. 5.
    Boulgouris, N.V., et al.: Segmentation and content-based watermarking for color image and image region indexing and retrieval. EURASIP Journal on Applied Signal Processing, 418–431 (2002)Google Scholar
  6. 6.
    Carson, C., et al.: Blobworld: Image segmentation using expectation-maximization and its application to image querying. IEEE Trans. on Pattern Analysis and Machine Intelligence 24(8), 1026–1038 (2002)CrossRefGoogle Scholar
  7. 7.
    Carson, C., et al.: Blobworld: A system for region-based image indexing and retrieval. In: Huijsmans, D.P., Smeulders, A.W.M. (eds.) VISUAL 1999. LNCS, vol. 1614, Springer, Heidelberg (1999)Google Scholar
  8. 8.
    Ciaccia, P., et al.: Imprecision and user preferences in multimedia queries: A generic algebraic approach. In: Schewe, K.-D., Thalheim, B. (eds.) FoIKS 2000. LNCS, vol. 1762, pp. 50–71. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  9. 9.
    Fagin, R.: Fuzzy Queries in Multimedia Database Systems. In: Proc. of the Seventeenth ACM SIGACT-SIGMOD-SIGART Symposium on Principles of Database Systems, Seattle, Washington, June 1-3, 1998, pp. 1–10. ACM Press, New York (1998)CrossRefGoogle Scholar
  10. 10.
    Feng, H., Chua, T.-S.: A bootstrapping approach to annotating large image collection. In: MIR ’03: Proc. of the 5th ACM SIGMM Int. Workshop on Multimedia Information Retrieval, Berkeley, California, pp. 55–62. ACM Press, New York (2003), doi:10.1145/973264.973274CrossRefGoogle Scholar
  11. 11.
    Galindo, J., Urrutia, A., Piattini, M.: Fuzzy Databases: Modeling, Design and Implementation. Idea Group Publishing, Hershey (2005)Google Scholar
  12. 12.
    Hasida, K.: The linguistic DS: Linguisitic description in MPEG-7. The Computing Research Repository (CoRR), cs.CL/0307044 (2003)Google Scholar
  13. 13.
    Hollink, L., et al.: Semantic annotation of image collections. In: Proc. of Workshop on Knowledge Markup and Semantic Annotation (KCAP’03) (2003)Google Scholar
  14. 14.
    Jain, A.K., Dubes, R.C.: Algorithms for Clustering Data. Prentice Hall, New Jersey (1988)zbMATHGoogle Scholar
  15. 15.
    Konishi, S., Yuille, A.L.: Statistical cues for domain specific image segmentation withperformance analysis. In: IEEE Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 125–132. IEEE Computer Society Press, Los Alamitos (2000)Google Scholar
  16. 16.
    Kosinov, S., Marchand-Maillet, S.: Overview of approaches to semantic augmentation of multimedia databases for efficient access and content retrieval. In: Adaptive Multimedia Retrieval, Postproc. of 1st Int. Workshop, pp. 19–35 (2004)Google Scholar
  17. 17.
    Lu, J., Ma, S.-p., Zhang, M.: Automatic image annotation based-on model space. In: Proc. of IEEE Int. Conf. on Natural Language Processing and Knowledge Engineering, pp. 455–460. IEEE Computer Society Press, Los Alamitos (2005)Google Scholar
  18. 18.
    Lux, M., Becker, J., Krottmaier, H.: Caliph & Emir: Semantic annotation and retrieval in personal digital photo libraries. In: Eder, J., Missikoff, M. (eds.) CAiSE 2003. LNCS, vol. 2681, pp. 85–89. Springer, Heidelberg (2003)Google Scholar
  19. 19.
    Martínez, J.M.: MPEG-7: Overview of MPEG-7 description tools, part 2. IEEE MultiMedia 9(3), 83–93 (2002)CrossRefGoogle Scholar
  20. 20.
    Miller, G., et al.: Five papers on WordNet. Int. Journal of Lexicography 3(4) (1990)Google Scholar
  21. 21.
    Natsev, A.P., Naphade, M.R., Tesic, J.: Learning the Semantics of Multimedia Queries and Concepts from a Small Number of Examples. In: ACM Press (ed.) Proc. of the 13th ACM Int. Conf. on Multimedia, pp. 598–607. ACM Press, New York (2005)CrossRefGoogle Scholar
  22. 22.
    Nürnberger, A., Detyniecki, M.: Adaptive multimedia retrieval: From data to user interaction. In: Do smart adaptive systems exist? - Best practice for selection and combination of intelligent methods, Springer, Heidelberg (2005)Google Scholar
  23. 23.
    Omhover, J.-F., Detyniecki, M.: Strict: An image retrieval platform for queries based on regional content. In: Enser, P.G.B., et al. (eds.) CIVR 2004. LNCS, vol. 3115, Springer, Heidelberg (2004)Google Scholar
  24. 24.
    Omhover, J.-F., Rifqi, M., Detyniecki, M.: Ranking invariance based on similarity measures in document retrieval. In: Adaptive Multimedia Retrieval: User, Context, and Feedback, Postproc. of 3rd Int. Workshop, pp. 55–64. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  25. 25.
    Rüger, S.: Putting the user in the loop: Visual resource discovery. In: Adaptive Multimedia Retrieval: User, Context, and Feedback, Postproc. of 3rd Int. Workshop, pp. 1–18. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  26. 26.
    Schmitt, I.: Basic Concepts for Unifying Queries of Database and Retrieval Systems. Technical Report 7, Fakultät für Informatik, Univ. Magdeburg (2005)Google Scholar
  27. 27.
    Schmitt, I., Schulz, N.: Similarity Relational Calculus and its Reduction to a Similarity Algebra. In: Seipel, D., Turull-Torres, J.M. (eds.) FoIKS 2004. LNCS, vol. 2942, pp. 252–272. Springer, Heidelberg (2004)Google Scholar
  28. 28.
    Schmitt, I., Schulz, N., Herstel, T.: WS-QBE: A QBE-like Query Language for Complex Multimedia Queries. In: Proc. of the 11th Int. Multimedia Modelling Conf (MMM’05), pp. 222–229. IEEE Computer Society Press, Los Alamitos (2005)CrossRefGoogle Scholar
  29. 29.
    Schulz, N., Schmitt, I.: A Survey of Weighted Scoring Rules in Multimedia Database Systems. Preprint 7, Fakultät für Informatik, Univ. Magdeburg (2002)Google Scholar
  30. 30.
    Stauffer, C., Grimson, W.E.L.: Adaptive background mixture models for real-time tracking. In: Proc. IEEE Conf. on Computer Vision and Pattern Recognition, pp. 246–252. IEEE Computer Society Press, Los Alamitos (1999)Google Scholar
  31. 31.
    Veltkamp, R.C., Tanase, M.: Content-based image retrieval systems: A survey. Technical Report UU-CS-2000-34, CS Dept., Utrecht University (2000)Google Scholar
  32. 32.
    Voisine, N., et al.: A genetic algorithm-based approach to knowledge-assisted video analysis. In: IEEE International Conference on Image Processing, IEEE Computer Society Press, Los Alamitos (2005)Google Scholar
  33. 33.
    Vossen, P.: EuroWordNet general document version 3, final, July 19 (1999)Google Scholar
  34. 34.
    Zadeh, L.A.: Fuzzy Logic. IEEE Computer 21(4), 83–93 (1988)Google Scholar

Copyright information

© Springer Berlin Heidelberg 2007

Authors and Affiliations

  • Christian Hentschel
    • 1
  • Andreas Nürnberger
    • 1
  • Ingo Schmitt
    • 1
  • Sebastian Stober
    • 1
  1. 1.Faculty of Computer Science, Otto-von-Guericke-University Magdeburg, D-39106 MagdeburgGermany

Personalised recommendations