Automated Evaluation of Annotators for Museum Collections Using Subjective Logic

  • Davide Ceolin
  • Archana Nottamkandath
  • Wan Fokkink
Part of the IFIP Advances in Information and Communication Technology book series (IFIPAICT, volume 374)


Museums are rapidly digitizing their collections, and face a huge challenge to annotate every digitized artifact in store. Therefore they are opening up their archives for receiving annotations from experts world-wide. This paper presents an architecture for choosing the most eligible set of annotators for a given artifact, based on semantic relatedness measures between the subject matter of the artifact and topics of expertise of the annotators. We also employ mechanisms for evaluating the quality of provided annotations, and constantly manage and update the trust, reputation and expertise information of registered annotators.


  1. 1.
    BBC. Uncovering Nation’s art collection (January 2012)
  2. 2.
    Breslin, J.G., Bojars, U., Aleman-meza, B., Boley, H., Nixon, L.J., Polleres, A., Zhdanova, A.V.: Finding experts using internet-based discussions in online communities and associated social networks. In: FEWS 2007. (2007)Google Scholar
  3. 3.
    Brickley, D., Miller, L.: Foaf vocabulary specification 0.98 (January 2012),
  4. 4.
    Ceolin, D., Van Hage, W., Fokkink, W.: A trust model to estimate the quality of annotations using the Web. In: WebSci 2010 (2010)Google Scholar
  5. 5.
    Cosley, D., Frankowski, D., Terveen, L., Riedl, J.: SuggestBot: using intelligent task routing to help people find work in wikipedia. In: IUI 2007, pp. 32–41. ACM (2007)Google Scholar
  6. 6.
    Cyganiak, R., Reynolds, D., Tennison, J.: The RDF Data Cube vocabulary (February 2012),
  7. 7.
    Demartini, G.: Finding experts using wikipedia. In: FEWS 2007, pp. 33–41. (2007)Google Scholar
  8. 8.
    Emigh, W., Herring, S.C.: Collaborative authoring on the web: A genre analysis of online encyclopedias. In: HICSS 2005, p. 99a. IEEE (2005)Google Scholar
  9. 9.
    Heath, T., Motta, E.: The hoonoh ontology for describing trust relationships in information seeking. In: PICKME 2008. (2008)Google Scholar
  10. 10.
    Hollink, L., Schreiber, G., Wielinga, B.: Patterns of semantic relations to improve image content search. Journal Web Semantics 5(3), 195–203 (2007)CrossRefGoogle Scholar
  11. 11.
    IconClass. A multilingual classification system for cultural content (January 2012),
  12. 12.
    Dublin Core Metadata Initiative. Dcmi metadata terms (February 2012),
  13. 13.
    U.S institute of Museum and Library Services. Steve Social Tagging Project (January 2012)Google Scholar
  14. 14.
    Jøsang, A.: A logic for uncertain probabilities. Intl. Journal of Uncertainty, Fuzziness and Knowledge-Based Systems 9(3), 279–212 (2001)Google Scholar
  15. 15.
    Princeton University. A lexical database for English (January 2012),
  16. 16.
    Princeton University. Wordnet::Similarity (February 2012),
  17. 17.
    W3C. SKOS Simple Knowledge Organization System Primer (January 2012),
  18. 18.
    W3C. The PROV Ontology: Model and Formal Semantics (February 2012),
  19. 19.
    Wang, Y., Stash, N., Aroyo, L., Gorgels, P., Rutledge, L., Schreiber, G.: Recommendations based on semantically-enriched museum collections. Journal Web Semantics 6(4), 283–290 (2008)CrossRefGoogle Scholar
  20. 20.
    Wang, Y., Stash, N., Aroyo, L., Hollink, L.: Using semantic relations for content-based recommender systems in cultural heritage. In: WOP 2009, pp. 16–28. (2009)Google Scholar
  21. 21.
    Zhibiao, W., Palmer, M.: Verbs semantics and lexical selection. In: ACL 1994, pp. 133–138. ACL (1994)Google Scholar
  22. 22.
    Zhou, Y., Cong, G., Cui, B., Jensen, C.S., Yao, J.: Routing Questions to the Right Users in Online Communities. In: ICDE 2009. IEEE (2009)Google Scholar

Copyright information

© IFIP International Federation for Information Processing 2012

Authors and Affiliations

  • Davide Ceolin
    • 1
  • Archana Nottamkandath
    • 1
  • Wan Fokkink
    • 1
  1. 1.VU University AmsterdamAmsterdamThe Netherlands

Personalised recommendations