Labelling Image Regions Using Wavelet Features and Spatial Prototypes

  • Carsten Saathoff
  • Marcin Grzegorzek
  • Steffen Staab
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5392)


In this paper we present an approach for image region classification that combines low-level processing with high-level scene understanding. For the low-level training, predefined image concepts are statistically modelled using wavelet features extracted directly from image pixels. For classification, features of a given test region compared with these statistical models provide probabilistic evaluations for all possible image concepts. Maximising these values themselves already leads to a classification result (label). However, in our paper they are used as an input for the high-level approach exploiting explicitly represented spatial arrangements of labels, so called spatial prototypes. We formalise the problem using Fuzzy Constraint Satisfaction Problems and Linear Programming. They provide a model with explicit knowledge that is suitable to aid the task of region labelling. Experiments performed for nearly 6000 test image regions show that combining low-level and high-level image analysis increases the labelling accuracy significantly.


  1. 1.
    Hollink, L., Schreiber, T.A., Wielinga, B.J., Worring, M.: Classification of user image descriptions. International Journal of Human-Computer Studies 61(5) (2004)Google Scholar
  2. 2.
    Barnard, K., Fan, Q., Swaminathan, R., Hoogs, A., Collins, R., Rondot, P., Kaufhold, J.: Evaluation of localized semantics: data, methodology, and experiments. International Journal of Computer Vision 77, 127–199 (2008)CrossRefGoogle Scholar
  3. 3.
    Fan, J., Gao, Y., Luo, H.: Multi-level annotation of natural scenes using dominant image components and semantic concepts. In: Proc. of ACM Multimedia 2004, pp. 540–547. ACM, New York (2004)Google Scholar
  4. 4.
    Torralba, A.: Contextual priming for object detection. Int. J. Comput. Vision 53(2), 169–191 (2003)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Grzegorzek, M., Izquierdo, E.: Statistical 3d object classification and localization with context modeling. In: Domanski, M., Stasinski, R., Bartkowiak, M. (eds.) 15th European Signal Processing Conference, Poznan, Poland, PTETiS, Poznan, pp. 1585–1589 (2007)Google Scholar
  6. 6.
    Yuan, J., Li, J., Zhang, B.: Exploiting spatial context constraints for automatic image region annotation. In: Proc. of ACM Multimedia 2007, pp. 595–604. ACM, New York (2007)Google Scholar
  7. 7.
    Panagi, P., Dasiopoulou, S., Papadopoulos, T.G., Kompatsiaris, Strintzis, M.G.: A genetic algorithm approach to ontology-driven semantic image analysis. In: Proc. of VIE 2006, pp. 132–137 (2006)Google Scholar
  8. 8.
    Saathoff, C., Staab, S.: Exploiting spatial context in image region labelling using fuzzy constraint reasoning. In: WIAMIS: Ninth International Workshop on Image Analysis for Multimedia Interactive Services (2008)Google Scholar
  9. 9.
    Manjunath, B.S., Salembier, P., Sikora, T.: Introduction to MPEG-7 - Multimedia Content Description Interface. John Willey & Sons Ltd., Chichester (2002)Google Scholar
  10. 10.
    Mallat, S.: A theory for multiresolution signal decomposition: The wavelet representation. IEEE Transactions on Pattern Analysis and Machine Intelligence 11(7), 674–693 (1989)CrossRefMATHGoogle Scholar
  11. 11.
    Webb, A.R.: Statistical Pattern Recognition. John Wiley & Sons Ltd., Chichester (2002)CrossRefMATHGoogle Scholar
  12. 12.
    Grzegorzek, M., Reinhold, M., Niemann, H.: Feature extraction with wavelet transformation for statistical object recognition. In: Kurzynski, M., Puchala, E., Wozniak, M., Zolnierek, A. (eds.) 4th International Conference on Computer Recognition Systems, Rydzyna, Poland, pp. 161–168. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  13. 13.
    Grzegorzek, M.: Appearance-Based Statistical Object Recognition Including Color and Context Modeling. Logos Verlag, Berlin (2007)Google Scholar
  14. 14.
    Ruttkay, Z.: Fuzzy constraint satisfaction. In: Proc. of Fuzzy Systems 1994, vol. 2, pp. 1263–1268 (1994)Google Scholar
  15. 15.
    Dasiopoulou, S., Heinecke, J., Saathoff, C., Strintzis, M.G.: Multimedia reasoning with natural language support. In: Proc. of ICSC 2007, pp. 413–420 (2007)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Carsten Saathoff
    • 1
  • Marcin Grzegorzek
    • 1
  • Steffen Staab
    • 1
  1. 1.ISWeb – Information Systems and Semantic Web Research Group Institute for Computer ScienceUniversity of Koblenz – LandauGermany

Personalised recommendations