Semantic Multimedia Analysis based on Region Types and Visual Context

  • Evaggelos Spyrou
  • Phivos Mylonas
  • Yannis Avrithis
Conference paper
Part of the IFIP The International Federation for Information Processing book series (IFIPAICT, volume 247)


In this paper previous work on the detection of high-level concepts within multimedia documents is extended by introducing a mid-level ontology as a means of exploiting the visual context of images in terms of the regions they consist of. More specifically, we construct a mid-level ontology, define its relations and integrate it in our knowledge modelling approach. In the past we have developed algorithms to address computationally efficient handling of visual context and extraction of mid-level characteristics and now we explain how these diverse algorithms and methodologies can be combined in order to approach a greater goal, that of semantic multimedia analysis. Early experimental results are presented using data derived from the beach domain.


Multimedia Content Region Type Model Vector Multimedia Document Visual Context 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Avrithis, Y., Doulamis, A., Doulamis, N., Kollias, S.: A stochastic framework for optimal key frame extraction from mpeg video databases. (1999)Google Scholar
  2. 2.
    Benitez, A. B., Zhong, D., Chang, S.-F., Smith, J. R., MPEG-7 MDS Content Description Tools and Applications, Lecture Notes in Computer Science, 2001.Google Scholar
  3. 3.
    Benitez, A. B., and Chang, S.-F., Image Classification Using Multimedia Knowledge Networks, Proceedings of the IEEE Int. Conf. on Image Processing (ICIP’03), Barcelona, Spain, 2003.Google Scholar
  4. 4.
    Cees, D. C. K., Snoek, G.M., Worring, M., and Smeulders, A. W., Learned lexicon-driven interactive video retrieval, 2006.Google Scholar
  5. 5.
    Chang, S.F., Sikora, T., Puri, A.: Overview of the mpeg-7 standard. IEEE trans, on Circuits and Systems for Video Technology 11(6) (2001) 688–695CrossRefGoogle Scholar
  6. 6.
    Duda, R.O., Hart, P.E., Stork, D.G.: Pattern Classification. 2 edn. Wiley Inter-science (2000)Google Scholar
  7. 7.
    Klir, G., and Yuan, B., Fuzzy Sets and Fuzzy Logic, Theory and Applications, New Jersey, Prentice Hall, 1995.zbMATHGoogle Scholar
  8. 8.
    Lewis, D., Index, Context, and Content, in Kanger, S. and Ohman, S. (Eds.), Philosophy and Grammar, Reidel Publishing, 1980.Google Scholar
  9. 9.
    McCarthy, J., Notes on Formalizing Context, in Proc. of the 13th International Joint Conference on Artificial Intelligence (IJCAI 1993), Chambéry, France, August–September 1993, pp. 81–98.Google Scholar
  10. 10.
    Miyamoto, S., Fuzzy Sets in Information Retrieval and Cluster Analysis, Kluwer Academic Publishers, Dordrecht / Boston / London, 1990.zbMATHGoogle Scholar
  11. 11.
    MPEG-7: Visual experimentation model (xm) version 10.0. ISO/IEC/ JTC1/SC29/WG11, Doc. N4062 (2001)Google Scholar
  12. 12.
    Mylonas, P., Athanasiadis, T., & Avrithis, Y. Improving image analysis using a contextual approach, In Proc. of 7th International Workshop on Image Analysis for Multimedia Interactive Services (WIAMIS), Seoul, Korea.Google Scholar
  13. 13.
    Rapantzikos, K., Avrithis, Y., Kollias, S., On the use of spatiotemporal visual attention for video classification, Proceedings of International Workshop on Very Low Bitrate Video Coding (VLBV’ 05), Sardinia, Italy, September 2005.Google Scholar
  14. 14.
    Saux, B., and Amato, G., Image classifiers for scene analysis, In Proc. of International Conference on Computer Vision and Graphics, 2004.Google Scholar
  15. 15.
    Skiadopoulos, S., Giannoukos, C, Sarkas, N., Vassiliadis, P., Sellis, T., Koubarakis, M., 2D topological and direction relations in the world of minimum bounding circles, IEEE trans, on Knowledge and Data Engineering, Vol. 17(12), pp. 16101623, 2005.CrossRefGoogle Scholar
  16. 16.
    Smeulders, A. W. M., Worring, M., Santini, S., Gupta, A. and Jain, R., Content-Based Image Retrieval at the End of the Early Years, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 22, pp. 1349–1380, 2000.CrossRefGoogle Scholar
  17. 17.
    Souvannavong, F., Merialdo, B., and Huet, B., Region-based video content indexing and retrieval, In Proc. of 4th International Workshop on Content-Based Multimedia Indexing, Riga, Latvia, 2005.Google Scholar
  18. 18.
    Spyrou, E., LeBorgne, H., Mailis, T., Cooke, E., Avrithis, Y., O’Connor, N., Fusing mpeg-7 visual descriptors for image classification, In: International Conference on Artificial Neural Networks (ICANN), 2005.Google Scholar
  19. 19.
    Staab, S., and Studer, R., Handbook on Ontologies, International Handbooks on Information Systems, Springer-Verlag, Heidelberg, 2004.Google Scholar
  20. 20.
    Tsechpenakis, G., Akrivas, G., Andreou, G., Staraou, G., and Kollias, S., Knowledge-Assisted Video Analysis and Object Detection, Proceedings of European Symposium on Intelligent Technologies, Hybrid Systems and their Implementation on Smart Adaptive Systems (Eunite02), Albufeira, Portugal, September 2002.Google Scholar
  21. 21.
    Voisine, N., Dasiopoulou, S., Mezaris, V., Spyrou, E., Athanasiadis, T., Kompat-siaris, I., Avrithis, Y., and Strintzis, M. G., Knowledge-assisted video analysis using a genetic algorithm, In Proc. of 6th International Workshop on Image Analysis for Multimedia Interactive Services (WIAMIS 2005), April 13–15, 2005.Google Scholar
  22. 22.
    W3C, RDF, Scholar
  23. 23.
    W3C, RDF Reification, Scholar

Copyright information

© International Federation for Information Processing 2007

Authors and Affiliations

  • Evaggelos Spyrou
    • 1
  • Phivos Mylonas
    • 1
  • Yannis Avrithis
    • 1
  1. 1.Image, Video and Multimedia LaboratoryNational Technical University of AthensAthensGreece

Personalised recommendations