ACRyLIQ: Leveraging DBpedia for Adaptive Crowdsourcing in Linked Data Quality Assessment

  • Umair ul Hassan
  • Amrapali Zaveri
  • Edgard Marx
  • Edward Curry
  • Jens Lehmann
Conference paper

DOI: 10.1007/978-3-319-49004-5_44

Part of the Lecture Notes in Computer Science book series (LNCS, volume 10024)
Cite this paper as:
ul Hassan U., Zaveri A., Marx E., Curry E., Lehmann J. (2016) ACRyLIQ: Leveraging DBpedia for Adaptive Crowdsourcing in Linked Data Quality Assessment. In: Blomqvist E., Ciancarini P., Poggi F., Vitali F. (eds) Knowledge Engineering and Knowledge Management. EKAW 2016. Lecture Notes in Computer Science, vol 10024. Springer, Cham

Abstract

Crowdsourcing has emerged as a powerful paradigm for quality assessment and improvement of Linked Data. A major challenge of employing crowdsourcing, for quality assessment in Linked Data, is the cold-start problem: how to estimate the reliability of crowd workers and assign the most reliable workers to tasks? We address this challenge by proposing a novel approach for generating test questions from DBpedia based on the topics associated with quality assessment tasks. These test questions are used to estimate the reliability of the new workers. Subsequently, the tasks are dynamically assigned to reliable workers to help improve the accuracy of collected responses. Our proposed approach, ACRyLIQ, is evaluated using workers hired from Amazon Mechanical Turk, on two real-world Linked Data datasets. We validate the proposed approach in terms of accuracy and compare it against the baseline approach of reliability estimate using gold-standard task. The results demonstrate that our proposed approach achieves high accuracy without using gold-standard task.

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  • Umair ul Hassan
    • 1
  • Amrapali Zaveri
    • 2
  • Edgard Marx
    • 3
  • Edward Curry
    • 1
  • Jens Lehmann
    • 4
    • 5
  1. 1.Insight Centre for Data AnalyticsNational University of IrelandGalwayIreland
  2. 2.Stanford Center for Biomedical Informatics ResearchStanford UniversityStanfordUSA
  3. 3.AKSW GroupUniversity of LeipzigLeipzigGermany
  4. 4.Computer Science InstituteUniversity of BonnBonnGermany
  5. 5.Knowledge Discovery DepartmentFraunhofer IAISSankt AugustinGermany

Personalised recommendations