Crowdordering

  • Toshiko Matsui
  • Yukino Baba
  • Toshihiro Kamishima
  • Hisashi Kashima
Conference paper

DOI: 10.1007/978-3-319-06605-9_28

Part of the Lecture Notes in Computer Science book series (LNCS, volume 8444)
Cite this paper as:
Matsui T., Baba Y., Kamishima T., Kashima H. (2014) Crowdordering. In: Tseng V.S., Ho T.B., Zhou ZH., Chen A.L.P., Kao HY. (eds) Advances in Knowledge Discovery and Data Mining. PAKDD 2014. Lecture Notes in Computer Science, vol 8444. Springer, Cham

Abstract

Crowdsourcing is a promising solution to problems that are difficult for computers, but relatively easy for humans. One of the biggest challenges in crowdsourcing is quality control, since high quality results cannot be expected from crowdworkers who are not necessarily very capable or motivated. Several statistical crowdsourcing quality control methods for binary and multinomial questions have been proposed. In this paper, we consider tasks where crowdworkers are asked to arrange multiple items in the correct order. We propose a probabilistic generative model of crowd answers by extending a distance-based order model to incorporate worker ability, and propose an efficient estimation algorithm. Experiments using real crowdsourced datasets show the advantage of the proposed method over a baseline method.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Toshiko Matsui
    • 1
  • Yukino Baba
    • 1
  • Toshihiro Kamishima
    • 2
  • Hisashi Kashima
    • 1
    • 3
  1. 1.The University of TokyoJapan
  2. 2.National Institute of Advanced Industrial Science and Technology (AIST)Japan
  3. 3.JST PRESTOJapan

Personalised recommendations