Skip to main content
SpringerLink
Log in

A Collaborative Video Annotation System Based on Semantic Web Technologies

  • Published:
Cognitive Computation Aims and scope Submit manuscript

Abstract

In recent years, videos have become more and more a familiar multimedia format for common users. In particular, the advent of Web 2.0 and the spreading of video-sharing services over the Web have led to an explosion of online video content. The capability to provide broader support in accessing and exploring video content, and in general other kind of multimedia formats as images and documents, is becoming more and more important. In this context, the value of semantically structured data and metadata is recognized as a key factor both to improve search efficiency and to guarantee data interoperability. This latter aspect is critical to connect different, heterogeneous content coming from a variety of data sources. On the other hand, the annotation of video resources has been increasingly understood as a medium factor to enable deep analysis of contents and collaborative study of online digital objects. However, as existing annotation tools provide poor support for semantically structured content or in some cases express the semantics in proprietary and non-interoperable formats, such knowledge that users build by carefully annotating contents hardly crosses the boundaries of a single system and often cannot be reused by different communities (e.g., to classify content or to discover new relations among resources). In this paper, a novel Semantic Web-based annotation system is presented that enables user annotations to form semantically structured knowledge at different levels of granularity and complexity. Annotation can be reused by external applications and mixed with Web of Data sources to enable “serendipity,” the reuse of data produced for a specific task (annotation) by different people and in different contexts from the one data originated from. The main ideas behind the approach are to build on ontologies and support linking, at data level, to precise thesauri and vocabularies, as well as to the Linked Open Data cloud. By describing the software model, developed in the context of SemLib EU project, and by providing an implementation of an online video annotation tool, the main aim of this paper is to demonstrate how such technologies can enable a scenario where users annotations are created while browsing the Web, naturally shared among users, stored in machine readable format and then possibly recombined with external data and ontologies to enhance end-user experience.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. http://linkeddata.org/.

  2. http://www.europeanaconnect.eu/.

  3. http://www.swickynotes.org.

  4. http://www.im3i.eu.

  5. COST 2102 Cross-Modal Analysis of Verbal and Non-Verbal Communication http://cost2102.cs.stir.ac.uk/.

  6. Multimodal Video Annotation Survey http://www.semedia.dii.univpm.it/survey/videoannotation.

  7. Video Exhibit http://videoemotion.altervista.org/videoemotion.html.

  8. Simile Exhibit: http://www.simile-widgets.org/exhibit/.

  9. SemLib project: Semantic Web Tools for Digital Libraries.

  10. ECP 2005 CULT 038206 project, EC eContentplus programme.

  11. http://www.w3.org/2001/sw/sweo/public/UseCases/BBC/.

  12. http://dojotoolkit.org/.

  13. http://www.youtube.com/t/press_statistics.

  14. http://code.google.com/apis/youtube/getting_started.html#player_apis.

  15. http://data.labs.freebase.com/recon/query.

  16. Talis Platform http://www.talis.com/platform/.

  17. http://www.simile-widgets.org/exhibit/.

  18. openRDF.org http://www.openrdf.org/.

  19. CORS Filter: http://software.dzhuvinov.com/cors-filter.html.

  20. SemTube: Semantic YouTube Video Annotation: http://semedia.dii.univpm.it/semtube/.

  21. DBPedia spotlight, http://spotlight.dbpedia.org/demo/index.html.

  22. DBPedia lookup, http://dbpedia.org/lookup.

  23. http://metasound.dii.univpm.it/asclient/semtube/examples/files/exhibit/videoexhibit.html.

  24. http://lucene.apache.org/solr/.

  25. http://siren.sindice.com/.

  26. http://www.3medialabs.org/survey/index.php?sid=73483&lang=en.

  27. http://www.youtube.com/watch?v=e1-DV1A%5Ftlw.

References

  1. Hu W, Xie N, Li L, Zeng X, Maybank S. A survey on visual content-based video indexing and retrieval. IEEE Trans Syst Man Cybern Part C Appl Rev. 2011;41(6):797–819.

    Article  Google Scholar 

  2. Andrews P, Zaihrayeu I, Pane J. A classification of semantic annotation systems. Technical Report DISI-10-064, Ingegneria e Scienza dell’Informazione. University of Trento. 2010.

  3. Grassi M, Morbidoni C, Nucci M. Semantic web techniques application for video fragment annotation and management. In: Proceedings of the SSPnet-COST 2102 PINK international conference on analysis of verbal and nonverbal communication and enactment: the processing issues. 2011. p. 95–103.

  4. Dasiopoulou S, Giannakidou E, Litos G, Malasioti P, Kompatsiaris Y. A survey of semantic image and video annotation tools. In: Paliouras G, Spyropoulos CD, Tsatsaronis G, editors. Knowledge-driven multimedia information extraction and ontology evolution, 6050/BOEMIE EU project. Berlin: Springer; 2011.

    Google Scholar 

  5. Media Fragments URI 1.0. W3C Working Draft 24 June 2010. [Online]. Available: http://www.w3.org/TR/media-frags/.

  6. Morbidoni C, Grassi M, Nucci M. Introducing SemLib project: semantic web tools for digital libraries. To Appear in: International workshop on semantic digital archives—sustainable long-term curation perspectives of cultural heritage held as part of the 15th international conference on theory and practice of digital libraries (TPDL). 29.09.2011 in Berlin.

  7. Manola F, Miller E (eds). RDF primer W3C recommendation 10 February 2004. Available: http://www.w3.org/TR/REC-rdf-syntax.

  8. Grassi M, Morbidoni C, Piazza F. Towards semantic multimodal video annotation. Lect Notes Comput Sci. 2011;6456/2011:305–16.

    Google Scholar 

  9. Rohlfing K et al. Comparison of multimodal annotation tools—workshop report. Gesprachsforschung-OnlineZeitschrift zur verbalen Interaktion 7. 2006;7(7):99–123.

  10. Kipp M. Anvil—a generic annotation tool for multimodal dialogue. In: Proceedings of the 7th European conference on speech communication and technology (Eurospeech). Aalborg; 2001. p. 1367–70.

  11. Wittenburg P et al. ELAN: a professional framework for multimodality research. In: Proceedings of the 5th international conference on language resources and evaluation (LREC 2006). p. 1556–9.

  12. Schmidt T. Transcribing and annotating spoken language with EXMARaLDA. In: Proceedings of the LREC-workshop on XML based richly annotated corpora. Lisbon 2004. Paris: ELRA; 2004.

  13. Chebotko A et al. OntoELAN: an ontology-based linguistic multimedia annotator multimedia software engineering. In: Proceedings of the IEEE sixth international symposium on 13–15 Dec. 2004. p. 329–36.

  14. Petridis K et al. MOntoMat-Annotizer: image annotation. Linking ontologies and multimedia low-level features in KES 2006— 10th international conference on knowledge based, intelligent information and engineering systems.

  15. Video Image Annotiation (VIA) Informatics and Telematics Insitute (CERTH-ITI). Available: http://mklab.iti.gr/via/.

  16. SVAT Semantic Video Annotation Tool by JOANNEUM RESEARCH. Available: http://www.joanneum.at/en/fb2.html.

  17. YouTube Video Annotations. Available: http://www.youtube.com/it/annotations_about.

  18. VideoANT. Available: http://ant.umn.edu/.

  19. Project Pad. Available: http://dewey.at.northwestern.edu/ppad2/.

  20. Kaltura Video Editing and Annotation. Available: http://corp.kaltura.com/video_platform/video_editing.

  21. Haslhofer B, Momeni E, Gay M, Simon R. Augmenting Europeana content with linked data resources. In 6th international conference on semantic systems (I-Semantics), September 2010.

  22. Bizer C, Lehmann J, Kobilarov G, Auer S, Becker C, Cyganiak R, Hellmann S. DBpedia a crystallization point for the web of data. J Web Semant Sci Serv Agents World Wide Web. 2009;7:154–165.

    Google Scholar 

  23. Bertini M, D’Amico G, Ferracani A, Meoni M, Serra G. Sirio, orione and pan: an integrated web system for ontology-based video search and annotation. In: Proceedings of the international conference on multimedia (MM ’10). New York: ACM. p. 1625–8. doi:10.1145/1873951.1874305. http://doi.acm.org/10.1145/1873951.1874305.

  24. Holley Rose Crowdsourcing: How and Why Should Libraries Do It?. D-Lib Magazine, The Magazine of Digital Library Research. March/April, 2010.

  25. Sparql query language for rdf, W3C Recommendation, January 2008. [Online]. Available: http://www.w3.org/TR/rdf-sparql-query/.

  26. Sanderson R, Van de Sompel H (eds). Open annotation: Alpha3 data model guide 15 October 2010. Available: http://www.openannotation.org/spec/alpha3/.

  27. Kahan J, Koivunen MR. Annotea: an open RDF infrastructure for shared web annotations. In: Proceedings of the 10th international conference on World Wide Web. 2001. p. 623–32.

  28. Annotation Ontology http://code.google.com/p/annotation-ontology/.

  29. Cross-Origin Resource Sharing. W3C Working Draft 27 July 2010. Available: http://www.w3.org/TR/cors/.

  30. Grassi M. Developing HEO human emotion ontology. LNCS, vol 5707. Berlin: Springer; 2009. p. 244–51.

    Google Scholar 

  31. Dublin Core http://purl.org/dc/elements/1.1/.

  32. FOAF Vocabulary Specification 0.98. Namespace Document 9 August 2010. Available: http://xmlns.com/foaf/spec/.

  33. Grassi M, Cambria E, Hussain A, Piazza F. Sentic web: a new paradigm for managing social media affective information. Cognitive computation. Berlin: Springer. In press. doi:10.1007/s12559-011-9101-8.

  34. Cambria E. In: Roelandse M, editor. Sentic computing: techniques, tools and applications. Berlin: Springer. In press.

Download references

Acknowledgments

The research leading to these results has received funding from the European Union’s Seventh Framework Programme managed by REA-Research Executive Agency (http://ec.europa.eu/research/rea) ([FP7/2007-2013][FP7/2007-2011]) under grant agreement no. 262301. This work is also supported by COST 2102, EUCogII and SSPNET.

Author information

Authors and Affiliations

  1. Semedia (Semantic Web and Multimedia), Polytechnic University of Marche, Via Brecce Bianche I, 60131, Ancona, Italy

    Marco Grassi, Christian Morbidoni & Michele Nucci

Authors
  1. Marco Grassi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Christian Morbidoni
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Michele Nucci
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Marco Grassi.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Grassi, M., Morbidoni, C. & Nucci, M. A Collaborative Video Annotation System Based on Semantic Web Technologies. Cogn Comput 4, 497–514 (2012). https://doi.org/10.1007/s12559-012-9172-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12559-012-9172-1

Keywords

Search

Navigation