Semantic Image Analysis Using a Learning Approach and Spatial Context

  • G. Th. Papadopoulos
  • V. Mezaris
  • S. Dasiopoulou
  • I. Kompatsiaris
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4306)


In this paper, a learning approach coupling Support Vector Machines (SVMs) and a Genetic Algorithm (GA) is presented for knowledge-assisted semantic image analysis in specific domains. Explicitly defined domain knowledge under the proposed approach includes objects of the domain of interest and their spatial relations. SVMs are employed using low-level features to extract implicit information for each object of interest via training in order to provide an initial annotation of the image regions based solely on visual features. To account for the inherent visual information ambiguity spatial context is subsequently exploited. Specifically, fuzzy spatial relations along with the previously computed initial annotations are supplied to a genetic algorithm, which uses them to decide on the globally most plausible annotation. In this work, two different fitness functions for the GA are tested and evaluated. Experiments with outdoor photographs demonstrate the performance of the proposed approaches.


Genetic Algorithm Support Vector Machine Spatial Relation Domain Ontology Spatial Context 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Smeulders, A.W.M., Worring, M., Santini, S., Gupta, A., Jain, R.: Content-based image retrieval at the end of the early years. IEEE Transactions on Pattern Analysis and Machine Intelligence 22(12), 1349–1380 (2000)CrossRefGoogle Scholar
  2. 2.
    Al-Khatib, W., Day, Y.F., Ghafoor, A., Berra, P.B.: Semantic Annotation of Images and Videos for Multimedia Analysis. In: Gómez-Pérez, A., Euzenat, J. (eds.) ESWC 2005. LNCS, vol. 3532, pp. 592–607. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  3. 3.
    Assfalg, J., Berlini, M., Del Bimbo, A., Nunziat, W., Pala, P.: Soccer Highlights Detection and Recognition using HMMs. In: IEEE International Conference on Multimedia & Expo (ICME), pp. 825–828 (2005)Google Scholar
  4. 4.
    Zhang, L., Lin, F.Z., Zhang, B.: Support Vector Machine Learning for Image Retrieval. In: International Conference on Image Processing (2001)Google Scholar
  5. 5.
    Dasiopoulou, S., Mezaris, V., Papastathis, V.K., Kompatsiaris, I., Strintzis, M.G.: Knowledge-Assisted Semantic Video Object Detection. IEEE Transactions on Circuits and Systems for Video Technology, Special Issue on Analysis and Understanding for Video Adaptation 15(10), 1210–1224 (2005)Google Scholar
  6. 6.
    Hollink, L., Little, S., Hunter, J.: Evaluating the Application of Semantic Inferencing Rules to Image Annotation. In: 3rd International Conference on Knowledge Capture (K-CAP 2005), Canada (2005)Google Scholar
  7. 7.
    Kim, K.I., Jung, K., Park, S.H., Kim, H.J.: Support vector machines for texture classification. IEEE Trans. on Pattern Analysis and Machine Intelligence 24(11), 1542–1550 (2002)CrossRefGoogle Scholar
  8. 8.
    Chapelle, O., Haffner, P., Vapnik, V.: Support vector machines for histogram-based image classification. IEEE Transactions on Neural Networks 10(5), 1055–1064 (1999)CrossRefGoogle Scholar
  9. 9.
    Mitchell, M.: An introduction to genetic algorithms. MIT Press, Cambridge (1995)Google Scholar
  10. 10.
    Voisine, N., Dasiopoulou, S., Precioso, F., Mezaris, V., Kompatsiaris, I., Strintzis, M.G.: A Genetic Algorithm-based Approach to Knowledge-assisted Video Analysis. In: Proc. IEEE International Conference on Image Processing (ICIP), Genova (2005)Google Scholar
  11. 11.
    Adamek, T., O’Connor, N., Murphy, N.: Region-based Segmentation of Images Using Syntactic Visual Features Workshop on Image Analysis for Multimedia Interactive Services (WIAMIS), Montreux Switzerland (2005)Google Scholar
  12. 12.
    MPEG-7 Visual Experimentation Model (XM). Version 10.0 ISO/IEC/JTC1/ SC29/WG11 Doc. N4062 (2001)Google Scholar
  13. 13.
    Skiadopoulos, S., Giannoukos, C., Sarkas, N., Vassiliadis, P., Sellis, T., Koubarakis, M.: 2D topological and direction relations in the world of minimum bounding circles. IEEE Transactions on Knowledge and Data Engineering 17(12), 1610–1623 (2005)CrossRefGoogle Scholar
  14. 14.
    Wang, Y., Makedon, F., Ford, J., Shen, L., Golding, D.: Generating Fuzzy Semantic Metadata Describing Spatial Relations from Images using the R-Histogram. In: JCDL 2004, Arizona USA, June 7-11 (2004)Google Scholar
  15. 15.
    Staab, S., Studer, R.: Handbook on ontologies. In: Int. Handbooks on Information Systems, Berlin, Germany. Springer, Heidelberg (2004)Google Scholar
  16. 16.
    Tax, D., Duin, R.: Using two-class classifiers for multi-class classification. In: Proc. Int. Conf. Pattern Recognition, Quebec City Canada, vol. 2, pp. 124–127 (2002)Google Scholar
  17. 17.
    Chang, C.-C., Lin, C.-J.: LIBSVM: A library for support vector machines (2001),
  18. 18.
    Goldberg, D., Deb, K.: A comparative analysis of selection schemes used in genetic algorithms. In: Foundations of Genetic Algorithms G. Rawlins, pp. 69–93 (1991)Google Scholar
  19. 19.
    Millet, C., Bloch, I., Hede, P., Moellic, P.-A.: Using relative spatial relationships to improve individual region recognition. EWIMT, London (2005)Google Scholar
  20. 20.
    Hollink, L., Nguyen, G., Schreiber, G., Wielemaker, J., Wielinga, B., Worring, M.: Adding Spatial Semantics to Image Annotations. In: Proceedings of International Workshop on Knowledge Markup and Semantic Annotation (2004)Google Scholar
  21. 21.
    Cox, E.: Fuzzy fundamentals. Spectrum IEEE 29(10), 58–61 (1992)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • G. Th. Papadopoulos
    • 1
    • 2
  • V. Mezaris
    • 2
  • S. Dasiopoulou
    • 1
    • 2
  • I. Kompatsiaris
    • 2
  1. 1.Information Processing Laboratory, Electrical and Computer Engineering DepartmentAristotle University of ThessalonikiGreece
  2. 2.Informatics and Telematics InstituteThessalonikiGreece

Personalised recommendations