Domain Knowledge Extension with Pictorially Enriched Ontologies

  • M. Bertini
  • R. Cucchiara
  • A. Del Bimbo
  • C. Torniai
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3691)


Classifying video elements according to some pre-defined ontology of the video content is the typical way to perform video annotation. Ontologies are built by defining relationship between linguistic terms that describe domain concepts at different abstraction levels. Linguistic terms are appropriate to distinguish specific events and object categories but they are inadequate when they must describe video entities or specific patterns of events. In these cases visual prototypes can better express pattern specifications and the diversity of visual events. To support video annotation up to the level of pattern specification enriched ontologies, that include visual concepts together with linguistic keywords, are needed. This paper presents Pictorially Enriched ontologies and provides a solution for their implementation in the soccer video domain. The pictorially enriched ontology created is used both to directly assign multimedia objects to concepts, providing a more meaningful definition than the linguistics terms, and to extend the initial knowledge of the domain, adding subclasses of highlights or new highlight classes that were not defined in the linguistic ontology. Automatic annotation of soccer clips up to the pattern specification level using a pictorially enriched ontology is discussed.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    World Wide Web Consortium, “Resource description framework (rdf),” Tech. Rep., W3C (Febraury 2004),
  2. 2.
    Leonardi, R., Migliorati, P.: Semantic indexing of multimedia documents. IEEE Multimedia 9(2), 44–51 (2002)CrossRefGoogle Scholar
  3. 3.
    Ekin, A., Murat Tekalp, A., Mehrotra, R.: Automatic soccer video analysis and summarization. IEEE Transactions on Image Processing 12(7), 796–807 (2003)CrossRefGoogle Scholar
  4. 4.
    Assfalg, J., Bertini, M., Colombo, C., Del Bimbo, A., Nunziati, W.: Semantic annotation of soccer videos: automatic highlights identification. Computer Vision and Image Understanding 92(2-3), 285–305 (2003)CrossRefGoogle Scholar
  5. 5.
    Yu, X., Xu, C., Leung, H.W., Tian, Q., Tang, Q., Wan, K.W.: Trajectory-based ball detection and tracking with applications to semantic analysis of broadcast soccer video. In: ACM Multimedia 2003, vol. 3, pp. 11–20 (2003)Google Scholar
  6. 6.
    Reidsma, D., Kuper, J., Declerck, T., Saggion, H., Cunningham, H.: Cross document ontology based information extraction for multimedia retrieval. In: Supplementary proceedings of the ICCS 2003, Dresden (July 2003)Google Scholar
  7. 7.
    Mezaris, V., Kompatsiaris, I., Boulgouris, N.V., Strintzis, M.G.: Real-time compressed-domain spatiotemporal segmentation and ontologies for video indexing and retrieval. IEEE Transactions on Circuits and Systems for Video Technology 14(5), 606–621 (2004)CrossRefGoogle Scholar
  8. 8.
    Jaimes, A., Tseng, B., Smith, J.R.: Modal keywords, ontologies, and reasoning for video understanding. In: Bakker, E.M., Lew, M., Huang, T.S., Sebe, N., Zhou, X.S. (eds.) CIVR 2003. LNCS, vol. 2728, pp. 145–150. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  9. 9.
    Jaimes, A., Smith, J.R.: Semi-automatic, data-driven construction of multimedia ontologies. In: Proc. of IEEE Int’l Conference on Multimedia & Expo. (2003)Google Scholar
  10. 10.
    Benitez, A.B., Chang, S.-F.: Automatic multimedia knowledge discovery, summarization and evaluation. IEEE Transactions on Multimedia (2003) (Submitted)Google Scholar
  11. 11.
    Strintzis, M.G., Bloehdorn, S., Handschuh, S., Staab, S., Simou, N., Tzouvaras, V., Petridis, K., Kompatsiaris, I., Avrithis, Y.: Knowledge representation for semantic multimedia content analysis and reasoning. In: European Workshop on the Integration of Knowledge, Semantics and Digital Media Technology (November 2004)Google Scholar
  12. 12.
    Bezdek, J.C.: Pattern Recognition with Fuzzy Objective Function Algorithms. Plenum Press, New York (1981)MATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  • M. Bertini
    • 1
  • R. Cucchiara
    • 2
  • A. Del Bimbo
    • 1
  • C. Torniai
    • 1
  1. 1.D.S.I. – Università di FirenzeItaly
  2. 2.D.I.I. – Università di Modena e Reggio EmiliaItaly

Personalised recommendations