Multimedia Tools and Applications

, Volume 70, Issue 2, pp 847–867 | Cite as

Open annotations on multimedia Web resources

  • Bernhard HaslhoferEmail author
  • Robert Sanderson
  • Rainer Simon
  • Herbert van de Sompel


Many Web portals allow users to associate additional information with existing multimedia resources such as images, audio, and video. However, these portals are usually closed systems and user-generated annotations are almost always kept locked up and remain inaccessible to the Web of Data. We believe that an important step to take is the integration of multimedia annotations and the Linked Data principles. We present the current state of the Open Annotation Model, explain our design rationale, and describe how the model can represent user annotations on multimedia Web resources. Applying this model in Web portals and devices, which support user annotations, should allow clients to easily publish and consume, thus exchange annotations on multimedia Web resources via common Web standards.


Annotations Web Linked data 



The work has partly been supported by the European Commission as part of the eContentplus program (EuropeanaConnect) and by a Marie Curie International Outgoing Fellowship within the 7th Europeana Community Framework Program. The development of OAC is funded by the Andrew W. Mellon foundation.


  1. 1.
    Agosti M, Bonfiglio-Dosio G, Ferro N (2007) A historical and contemporary study on annotations to derive key features for systems design. Int J Digit Libr 8(1):1–19Google Scholar
  2. 2.
    Arndt R, Troncy R, Staab S, Hardman L, Vacura M (2007) COMM: designing a well-founded multimedia ontology for the web. In: The semantic Web, 6th International Semantic Web Conference, 2nd asian semantic Web conference, ISWC 2007 + ASWC 2007, Busan, Korea, pp 30–43, NovemberGoogle Scholar
  3. 3.
    Berners-Lee T, Fielding R, Masinter L (2005) Uniform resource identifier (URI): generic syntax. IETF, Available at Accessed 28 Feb 2012
  4. 4.
    Blustein J, Rowe D, Graff A-B (2011) Making sense in the margins: a field study of annotation. In: Proceedings of the 15th international conference on theory and practice of digital libraries: research and advanced technology for digital libraries, TPDL’11. Springer-Verlag, Berlin, Heidelberg, pp 252–259Google Scholar
  5. 5.
    Bradley J (2008) Pliny: a model for digital support of scholarship. J Digit Inf 9(1).
  6. 6.
    Ciccarese P, Ocana M, Garcia Castro LJ, Das S, Clark T (2011) An open annotation ontology for science on web 3.0. J Biomed Semantics 2(2). doi: 10.1186/2041-1480-2-S2-S4
  7. 7.
    Dodds L, Davis I (2012) Linked data patterns. Available at Accessed 28 Feb 2012
  8. 8.
    Hardman L, Bulterman DCA, van Rossum G (1994) The amsterdam hypermedia model: adding time and context to the dexter model. Commun ACM 37(2):50–62CrossRefGoogle Scholar
  9. 9.
    Haslhofer B, Jochum W, King R, Sadilek C, Schellner K (2009) The LEMO annotation framework: weaving multimedia annotations with the Web. Int J Digit Libr 10(1):15–32CrossRefGoogle Scholar
  10. 10.
    Haslhofer B, Simon R, Sanderson R, van de Sompel H (2011) The Open Annotation Collaboration (OAC) model. In: Workshop on multimedia on the Web 2011, IEEE, Graz, Austria, SeptemberGoogle Scholar
  11. 11.
    Hausenblas M, Troncy R, Raimond Y, Bürger T (2009) Interlinking multimedia: how to apply linked data principles to multimedia fragments. In: WWW 2009 workshop: Linked Data on the Web (LDOW2009), Madrid, SpainGoogle Scholar
  12. 12.
    Heath T, Bizer C Linked data: evolving the web into a global data space. In: Synthesis lectures on the semantic web: theory and technology, 1st edn 2011, vol 1:1. Morgan & Claypool, pp 1–136. Available at
  13. 13.
    Hunter J (2009) Collaborative semantic tagging and annotation systems. Ann Rev Inf Sci Technol 43(1):1–84CrossRefGoogle Scholar
  14. 14.
    Hunter J, Schroeter R (2008) Co-Annotea: a system for tagging relationships between multiple mixed-media objects. IEEE Multimed 15(3):42–53CrossRefGoogle Scholar
  15. 15.
    ISO/IEC (2006) Multimedia framework (MPEG-21) – part 17: fragment identification of MPEG resources. International Organization for Standardization, Geneva, Switzerland, DecemberGoogle Scholar
  16. 16.
    Jacobs I, Walsh N (2004) Architecture of the world wide web, 1. Position Statement W3C Video Workshop 12/13th.
  17. 17.
    Kahan J, Koivunen M-R (2001) Annotea: an open RDF infrastructure for shared web annotations. In: Proceedings of the 10th international conference on World Wide Web, WWW ’01. ACM, New York, NY, USA, pp 623–632Google Scholar
  18. 18.
    Kim HL, Scerri S, Breslin JG, Decker S, Kim HG (2008) The state of the art in tag ontologies: a semantic model for tagging and folksonomies. In: DCMI ’08: proceedings of the 2008 international conference on dublin core and metadata applications, Dublin core metadata initiative, pp 128–137Google Scholar
  19. 19.
    Klein M, Ware J, Nelson ML (2011) Rediscovering missing web pages using link neighborhood lexical signatures. In: Proceeding of the 11th annual international ACM/IEEE Joint Conference on Digital Libraries, JCDL ’11. ACM, New York, NY, USA, pp 137–140CrossRefGoogle Scholar
  20. 20.
    Koch J, Velasco CA (2009) Representing Content in RDF 1.0. W3C, Available at Accessed 28 Feb 2012
  21. 21.
    Koivunen M-R (2006) Semantic authoring by tagging with Annotea social bookmarks and topics. In: The 5th international semantic web conference (ISWC2006) - 1st Semantic Authoring and Annotation Workshop (SAAW2006)Google Scholar
  22. 22.
    Marshall CC (2000) The future of annotation in a digital (paper) world. In: Harum S, Twidale M (eds) Successes and failures of digital libraries. Urbana-Champaign: University of Illinois, pp 97–117. Available at Accessed 28 Feb 2012
  23. 23.
    Marshall CC, Bernheim Brush AJ (2004) Exploring the relationship between personal and public annotations. In: Proceedings of the 2004 Joint ACM/IEEE conference on digital libraries, pp 349–357Google Scholar
  24. 24.
    Morishima A, Nakamizo A, Iida T, Sugimoto S, Kitagawa H (2009) Bringing your dead links back to life: a comprehensive approach and lessons learned. In: Proceedings of the 20th ACM conference on hypertext and hypermedia, HT ’09. ACM, New York, NY, USA, pp 15–24CrossRefGoogle Scholar
  25. 25.
    Nack F, Lindsay AT (1999) Everything you wanted to know about MPEG-7: part 2. IEEE Multimed 6(4):64–73CrossRefGoogle Scholar
  26. 26.
    Pfeiffer S (2007) Hyperlinking to time offsets: the temporal uri specification, 12. Position Statement W3C Video Workshop 12/13th.
  27. 27.
    Phelps TA, Wilensky R (2000) Robust hyperlinks cost just five words each. Technical Report UCB/CSD-00-1091, EECS Department, University of California, BerkeleyGoogle Scholar
  28. 28.
    Popitsch N, Haslhofer B (2011) Dsnotify - a solution for event detection and link maintenance in dynamic datasets. Web Semant 9(3):266–283CrossRefGoogle Scholar
  29. 29.
    Qayyum A (2008) Analysing markings made on e-documents. Can J Inform Lib Sci 32(1/2):35–53Google Scholar
  30. 30.
    Saathoff C, Scherp A (2010) Unlocking the semantics of multimedia presentations in the web with the multimedia metadata ontology. In: Proceedings of the 19th international conference on World Wide Web, WWW ’10. ACM, New York, NY, USA, pp 831–840CrossRefGoogle Scholar
  31. 31.
    Sanderson R, Albritton B, Schwemmer R, Van de Sompel H (2011) Sharedcanvas: a collaborative model for medieval manuscript layout dissemination. In: Proceeding of the 11th annual international ACM/IEEE Joint Conference on Digital Libraries, JCDL ’11. ACM, New York, NY, USA, pp 175–184CrossRefGoogle Scholar
  32. 32.
    Sanderson R, Van de Sompel H (2010) Making web annotations persistent over time. In: Proceedings of the 10th annual Joint Conference on Digital Libraries, JCDL ’10. ACM, New York, NY, USA, pp 1–10CrossRefGoogle Scholar
  33. 33.
    Schroeter R, Hunter J, Newman A (2007) Annotating relationships between multiple mixed-media digital objects by extending annotea. In: Franconi E, Kifer M, May W (eds) ESWC, vol 4519. SpringerGoogle Scholar
  34. 34.
    Simon R, Haslhofer B, Robitza W, Momeni E (2011) Semantically augmented annotations in digitized map collections. In: Proceeding of the 11th annual international ACM/IEEE Joint Conference on Digital Libraries, JCDL ’11. ACM, New York, NY, USA, pp 199–202CrossRefGoogle Scholar
  35. 35.
    Van de Sompel H, Nelson ML, Sanderson R, Balakireva LL, Ainsworth S, Shankar H (2009) Memento: time travel for the web. Arxiv preprint. arxiv:0911.1112, November
  36. 36.
    Van de Sompel H, Sanderson R, Nelson ML, Balakireva L, Shankar H, Ainsworth S (2010) An http-based versioning mechanism for linked data. CoRR abs/1003.3661
  37. 37.
    Van Deursen D, Troncy R, Mannens E, Pfeiffer S, Lafon Y, Van de Walle R (2010) Implementing the media fragments uri specification. In: Proceedings of the 19th international conference on World Wide Web, WWW ’10. ACM, New York, NY, USA, pp 1361–1364CrossRefGoogle Scholar
  38. 38.
    Verspoor K, Cohn J, Joslyn C, Mniszewski S, Rechtsteiner A, Rocha L, Simas T (2005) Protein annotation as term categorization in the gene ontology using word proximity networks. BMC Bioinformatics 6(1). doi: 10.1186/1471-2105-6-S1-S20
  39. 39.
    W3C Media Fragments Working Group (2011) Media Fragments URI 1.0. W3C, Available at Accessed 28 Feb 2012
  40. 40.
    W3C SVG Working Group (2003) Scalable Vector Graphics (SVG) — XML Graphics for the Web. W3C Available at Accessed 28 Feb 2012

Copyright information

© Springer Science+Business Media, LLC 2012

Authors and Affiliations

  • Bernhard Haslhofer
    • 1
    Email author
  • Robert Sanderson
    • 2
  • Rainer Simon
    • 3
  • Herbert van de Sompel
    • 2
  1. 1.Department of Information ScienceCornell UniversityIthacaUSA
  2. 2.Los Alamos National LaboratoryLos AlamosUSA
  3. 3.Austrian Institute of TechnologyViennaAustria

Personalised recommendations