Quality Control for Crowdsourcing with Spatial and Temporal Distribution

  • Gang Zhang
  • Haopeng Chen
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8223)


In the past decade, crowdsourcing has become a prospective paradigm for commercial purposes, for it brings a lot of benefits such as low cost and high immediacy, particularly in location-based services (LBS). On the other side, there also exist many problems need to be solved in crowdsourcing. For example, the quality control for crowdsourcing systems has been identified as a significant challenge, which includes how to handle massive data more efficiently, how to discriminate poor quality content in workers’ submissions and so on. In this paper, we put forward an approach to control the crowdsourcing quality from spatial and temporal distribution. Our experiments have demonstrated the effectiveness and efficiency of the approach.


crowdsourcing location-based service (LBS) quality control spatial and temporal distribution 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Howe, J.: The Rise of Crowdsourcing, Wired (June 2006), http://www.wired.com/wired/archive/14.06/crowds.html
  2. 2.
  3. 3.
    Greengard, S.: Following the crowd. Communications of the ACM 54(2), 20–22 (2011)CrossRefGoogle Scholar
  4. 4.
    Amazon Mechanical Turk, http://www.mturk.com
  5. 5.
    Alt, F., Sahami, A., Schmidt, S.A., Kramer, U., Nawaz, Z.: Location-based crowdsourcing: extending crowdsourcing to the real world. In: 6th Nordic Conference on Human-Computer Interaction: Extending Boundaries, pp. 13–22Google Scholar
  6. 6.
    Shah, S., Bao, F., Lu, C.-T., Chen, I.-R.: CROWDSAFE: crowdsourcing of crime incidents and safe routing on mobile devices. In: 19th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems, pp. 521–524Google Scholar
  7. 7.
    Hirth, M., Hoβfeld, T., Tran-Gia, P.: Cost-Optimal Validation Mechanisms and Cheat-Detection for Crowdsourcing Platforms. In: 5th International Conference on Innovative Mobile and Internet Services in Ubiquitous Computing, pp. 316–321Google Scholar
  8. 8.
    Lease, M., Yilmaz, E.: Crowdsourcing for information retrieval. Newsletter ACM SIGIR Forum Archive 45(2), 66–75 (2011)CrossRefGoogle Scholar
  9. 9.
    Venetic, P., Garcia-Molina, H.: Quality control for comparison microtasks. In: The 1st International Workshop on Crowdsourcing and Data Mining, pp. 15–21Google Scholar
  10. 10.
    Kazai, G., Kamps, J., Milic-Frayling, N.: The face of quality in crowdsourcing relevance labels: demographics, personality and labeling accuracy. In: 21st ACM International Conference on Information and Knowledge Management, pp. 2583–2586Google Scholar
  11. 11.
    Zhu, S., Kane, S., Feng, J., Sears, A.: A Crowdsourcing Quality Control Model for Tasks Distributed in Parallel. In: CHI 2012 Extended Abstracts on Human Factors in Computing Systems, pp. 2501–2506 (2012)Google Scholar
  12. 12.
    Andrew, J., Flanagin, M.J.: Metzger. The credibility of volunteered geographic information. Geo Journal, An International Journal on Geography, Published Online (July 24, 2008)Google Scholar
  13. 13.
    Mashhadi, A.J., Capra, L.: Quality control for real-time ubiquitous crowdsourcing. In: 2nd International Workshop on Ubiquitous Crowd Souring, pp. 5–8Google Scholar
  14. 14.
    Kamar, E., Horvitz, E.: Incentives for truthful reporting in crowdsourcing. In: 11th International Conference on Autonomous Agents and Multiagent Systems, vol. 3, pp. 1329–1330Google Scholar
  15. 15.
    Mason, W., Watts, D.J.: Financial incentives and the "performance of crowds". ACM SIGKDD Explorations Newsletter 11(2), 100–108 (2009)CrossRefGoogle Scholar
  16. 16.
    Dean, J., Ghemawat, S.: MapReduce: simplified data processing on large clusters. Communications of the ACM - 50th Anniversary Issue: 1958 - 2008 51(1), 107–113 (2008)CrossRefGoogle Scholar
  17. 17.
    Chen, Z., Ma, J., Cui, C., Rui, H., Huang, S.: Web page publication time detection and its application for page rank. In: 33rd International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 859–860Google Scholar
  18. 18.
    Cheng, R., Chen, J., Xie, X.: Cleaning uncertain data with quality guarantees. Journal VLDB Endowment 1(1), 722–735 (2008)Google Scholar
  19. 19.

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Gang Zhang
    • 1
  • Haopeng Chen
    • 1
  1. 1.REINS Group, School of SoftwareShanghai Jiao Tong UniversityShanghaiP.R. China

Personalised recommendations