Skip to main content
Book cover

Research Conference on Metadata and Semantics Research

MTSR 2014: Metadata and Semantics Research pp 238–249Cite as

A Framework for the Evaluation of Automatic Metadata Enrichments

  • Conference paper

Part of the Communications in Computer and Information Science book series (CCIS,volume 478)

Abstract

Automatic enrichment of collections connects data to vocabularies, which supports the contextualization of content and adds searchable text to metadata. The paper introduces a framework of four dimensions (frequency, coverage, relevance and error rate) that measure both the suitability of the enrichment for the object and the enrichments’ contribution to search success. To verify the framework, it is applied to the evaluation of automatic enrichments in the digital library Europeana. The analysis of 100 result sets and their corresponding queries (1,121 documents total) shows the framework is a valuable tool for guiding enrichments and determining the value of enrichment efforts.

Keywords

  • Evaluation
  • Europeana
  • Digital Libraries
  • Cultural Collections
  • Framework
  • Semantic Enrichment

This is a preview of subscription content, access via your institution.

Chapter
USD   29.95
Price excludes VAT (Canada)
  • DOI: 10.1007/978-3-319-13674-5_23
  • Chapter length: 12 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   59.99
Price excludes VAT (Canada)
  • ISBN: 978-3-319-13674-5
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   79.99
Price excludes VAT (Canada)
Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Newman, D., Hagedorn, K., Chemudugunta, C., Smyth, P.: Subject Metadata Enrichment Using Statistical Topic Models. In: Proceedings of the 7th ACM/IEEE-CS Joint Conference on Digital Libraries, pp. 366–375. ACM, New York (2007)

    Google Scholar 

  2. Agirre, E., Barrena, A., Fernandez, K., Miranda, E., Otegi, A., Soroa, A.: PATHSenrich: A Web Service Prototype for Automatic Cultural Heritage Item Enrichment. In: Aalberg, T., Papatheodorou, C., Dobreva, M., Tsakonas, G., Farrugia, C.J. (eds.) TPDL 2013. LNCS, vol. 8092, pp. 462–465. Springer, Heidelberg (2013)

    CrossRef  Google Scholar 

  3. Olensky, M., Stiller, J., Dröge, E.: Poisonous India or the Importance of a Semantic and Multilingual Enrichment Strategy. In: Dodero, J.M., Palomo-Duarte, M., Karampiperis, P. (eds.) MTSR 2012. CCIS, vol. 343, pp. 252–263. Springer, Heidelberg (2012)

    CrossRef  Google Scholar 

  4. Schreiber, G., Amin, A., Aroyo, L., van Assem, M., de Boer, V., Hardman, L., et al.: Semantic annotation and search of cultural-heritage collections: The MultimediaN E-Culture demonstrator. Web Semantics: Science, Services and Agents on the World Wide Web 6(4), 243–249 (2008)

    CrossRef  Google Scholar 

  5. Jones, C.B., Purves, R., Ruas, A., Sanderson, M., Sester, M., van Kreveld, M., Weibel, R.: Spatial Information Retrieval and Geographical Ontologies an Overview of the SPIRIT Project. In: SIGIR 2002, Proceedings of the 25th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 387–388. ACM, New York (2002)

    Google Scholar 

  6. Fernie, K., Griffiths, J., Davies, R.: D 8.1 Final Report. PATHS - Personalised Access to Cultural Heritage Spaces (2014), http://www.paths-pro-ject.eu/eng/content/download/5274/40160/version/1/file/D+8+1+Final+Report_final.pdf

  7. Agirre, E., Barrena, A., Lopez de Lacalla, O., Soroa, A., Stevenson, M., Fernando, S.: Matching Cultural Heritage items to Wikipedia. In: Proceedings of the 8th International Conference on Language Resources and Evaluation, Istanbul, Turkey, pp. 1729–1735 (2012)

    Google Scholar 

  8. Shreeves, S.L., Knutson, E.M., Stvilia, B., Palmer, C.L., Twidale, M.B., Cole, T.W.: Is ‘Quality’ Metadata ‘Shareable’ Metadata? The Implications of Local Metadata Practices for Federated Collections. In: Thompson, H.A. (ed.) Proceedings of the Twelfth National Conference of the Association of College and Research Libraries, April 7-10, pp. 223–237. Association of College and Research Libraries, Minneapolis (2005)

    Google Scholar 

  9. Van Hage, W.R., Isaac, A., Aleksovski, Z.: Sample Evaluation of Ontology-Matching Systems. In: EON, vol. 2007, pp. 41–50 (November 2007)

    Google Scholar 

  10. Stiller, J., Petras, V., Gäde, M., Isaac, A.: Automatic Enrichments with Controlled Vocabularies in Europeana: Challenges and Consequences (to be published)

    Google Scholar 

  11. Stiller, J., Olensky, M., Isaac, A., Petras, V.: D7.8. Final Report on Innovative Multilingual Information Access. Europeana Version 2 (2014), http://pro.europeana.eu:9580/documents/866067/983534/D7.8%2BFinal%2BReport%2BInnovative%2BMultilingual%2BAccess

Download references

Author information

Authors and Affiliations

  1. Berlin School of Library and Information Science, Humboldt-Universität zu Berlin, Berlin, Germany

    Juliane Stiller, Marlies Olensky & Vivien Petras

Authors
  1. Juliane Stiller
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Marlies Olensky
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Vivien Petras
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Technik und Wirtschaft Fakulät, Hochschule Karlsruhe, Amalienstr. 81-87, 76133, Karlsruhe, Germany

    Sissi Closs

  2. Karlsruhe Institute of Technology, Germany

    Rudi Studer

  3. Department of Library Science & Information Systems, Alexander Technological Educational Institute of Thessaloniki, Sindos,Thessaloniki, Greece

    Emmanouel Garoufallou

  4. Information Engineering Research Unit, Computer Science Department, University of Alcalá, Ctra. Barcelona km. 33.6, 28871, Alcalá de Henares, Madrid, Spain

    Miguel-Angel Sicilia

Rights and permissions

Reprints and Permissions

Copyright information

© 2014 Springer International Publishing Switzerland

About this paper

Cite this paper

Stiller, J., Olensky, M., Petras, V. (2014). A Framework for the Evaluation of Automatic Metadata Enrichments. In: Closs, S., Studer, R., Garoufallou, E., Sicilia, MA. (eds) Metadata and Semantics Research. MTSR 2014. Communications in Computer and Information Science, vol 478. Springer, Cham. https://doi.org/10.1007/978-3-319-13674-5_23

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-13674-5_23

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-13673-8

  • Online ISBN: 978-3-319-13674-5

  • eBook Packages: Computer ScienceComputer Science (R0)