A Context-Based Region Labeling Approach for Semantic Image Segmentation

  • Thanos Athanasiadis
  • Phivos Mylonas
  • Yannis Avrithis
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4306)


In this paper we present a framework for simultaneous image segmentation and region labeling leading to automatic image annotation. The proposed framework operates at semantic level using possible semantic labels to make decisions on handling image regions instead of visual features used traditionally. In order to stress its independence of a specific image segmentation approach we applied our idea on two region growing algorithms, i.e. watershed and recursive shortest spanning tree. Additionally we exploit the notion of visual context by employing fuzzy algebra and ontological taxonomic knowledge representation, incorporating in this way global information and improving region interpretation. In this process, semantic region growing labeling results are being re-adjusted appropriately, utilizing contextual knowledge in the form of domain-specific semantic concepts and relations. The performance of the overall methodology is demonstrated on a real-life still image dataset from the popular domains of beach holidays and motorsports.


Membership Degree Contextual Knowledge Relation Graph Visual Context Merge Region 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Akrivas, G., Stamou, G., Kollias, S.: Semantic Association of Multimedia Document Descriptions through Fuzzy Relational Algebra and Fuzzy Reasoning. IEEE Trans. on Systems, Man, and Cybernetics, part A 34(2) (March 2004)Google Scholar
  2. 2.
    Akrivas, G., Wallace, M., Andreou, G., Stamou, G., Kollias, S.: Context – Sensitive Semantic Query Expansion. In: Proc. of the IEEE International Conference on Artificial Intelligence Systems (ICAIS), Divnomorskoe, Russia (September 2002)Google Scholar
  3. 3.
    Andrade Neto, E.L., Woods, J.C., Khan, E., Ghanbari, M.: Region Based Analysis and Retrieval for Tracking of Semantic Objects and Provision of Augmented Information in Interactive Sport Scenes. IEEE Trans. on Multimedia 7(6), 1084–1096 (2005)CrossRefGoogle Scholar
  4. 4.
    Athanasiadis, T., Tzouvaras, V., Petridis, K., Precioso, F., Avrithis, Y., Kompatsiaris, Y.: Using a Multimedia Ontology Infrastructure for Semantic Annotation of Multimedia Content. In: Proc. of 5th International Workshop on Knowledge Markup and Semantic Annotation (SemAnnot 2005), Galway, Ireland (November 2005)Google Scholar
  5. 5.
    Athanasiadis, T., Avrithis, Y., Kollias, S.: A Semantic Region Growing Approach in Image Segmentation and Annotation. In: 1st International Workshop on Semantic Web Annotations for Multimedia (SWAMM), Edinburgh, Scotland (November 2006)Google Scholar
  6. 6.
    Benitez, A.B., Rising, H., Jrgensen, C., Leonardi, R., Bugatti, A., Hasida, K., Mehrotra, R., Tekalp, M., Ekin, A., Walker, T.: Semantics of Multimedia in MPEG-7. In: IEEE International Conference on Image Processing, vol. 1, pp. 137–140 (2002)Google Scholar
  7. 7.
    Beucher, S., Meyer, F.: The Morphological Approach to Segmentation: The Watershed Transformation. In: Doughertty, E.R. (ed.) Mathematical Morphology in Image Processing. Marcel Dekker, New York (1993)Google Scholar
  8. 8.
    Borenstein, E., Sharon, E., Ullman, S.: Combining Top-Down and Bottom-Up Segmentation. In: Computer Vision and Pattern Recognition Workshop, Washington DC, USA (June 2004)Google Scholar
  9. 9.
    Boutell, M., Luo, J., Shena, X., Brown, C.: Learning multi-label scene classification. Pattern Recognition 37(9), 1757–1771 (2004)CrossRefGoogle Scholar
  10. 10.
    Gruber, T.R.: A Translation Approach to Portable Ontology Specification. Knowledge Acquisition 5, 199–220 (1993)CrossRefGoogle Scholar
  11. 11.
    Jianping, F., Yau, D.K.Y., Elmagarmid, A.K., Aref, W.G.: Automatic image segmentation by integrating color-edge extraction and seeded region growing. IEEE Trans. on Image Processing 10(10), 1454–1466 (2001)zbMATHCrossRefGoogle Scholar
  12. 12.
    Klir, G., Yuan, B.: Fuzzy Sets and Fuzzy Logic, Theory and Applications. Prentice Hall, New Jersey (1995)zbMATHGoogle Scholar
  13. 13.
    Lee, S., Crawford, M.M.: Unsupervised classification using spatial region growing segmentation and fuzzy training. In: Proc. of the IEEE International Conference on Image Processing, Thessaloniki, Greece (2001)Google Scholar
  14. 14.
    Luo, J., Savakis, A.: Indoor vs outdoor classification of consumer photographs using low-level and semantic features. In: Proc. IEEE Int. Conf. on Image Processing (ICIP 2001) (2001)Google Scholar
  15. 15.
    Manjunath, B.S., Ohm, J.R., Vasudevan, V.V., Yamada, A.: Color and texture descriptors, Special Issue on MPEG-7. IEEE Trans. on Circuits and Systems for Video Technology 11(6), 703–715 (2001)CrossRefGoogle Scholar
  16. 16.
    Morris, O.J., Lee, M.J., Constantinides, A.G.: Graph theory for image analysis: An approach based on the shortest spanning tree. Proc. Inst. Elect. Eng. 133, 146–152 (1986)Google Scholar
  17. 17.
    Mylonas, P., Avrithis, Y.: Context modeling for multimedia analysis and use. In: Proc. of 5th Inter-national and Interdisciplinary Conference on Modeling and Using Context (CONTEXT 2005), Paris, France (July 2005)Google Scholar
  18. 18.
    Salembier, P., Marques, F.: Region-Based Representations of Image and Video - Segmentation Tools for Multimedia Services. IEEE Trans. on Circuits and Systems for Video Technology 9(8) (1999)Google Scholar
  19. 19.
    Sikora, T.: The MPEG-7 Visual standard for content description - an overview, Special Issue on MPEG-7. IEEE Trans. on Circuits and Systems for Video Technology 11(6), 696–702 (2001)CrossRefMathSciNetGoogle Scholar
  20. 20.
    ISO/IEC JTC 1/SC 29/WG 11/N3966, Text of 15938-5 FCD Information Technology – Multimedia Content Description Interface – Part 5 Multimedia Description Schemes, Singapore (2001)Google Scholar
  21. 21.

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Thanos Athanasiadis
    • 1
  • Phivos Mylonas
    • 1
  • Yannis Avrithis
    • 1
  1. 1.School of Electrical and Computer EngineeringNational Technical University of AthensZographou, AthensGreece

Personalised recommendations