Skip to main content
SpringerLink
Log in

Factors That Influence the Quality of Crowdsourcing

  • Conference paper
New Trends in Database and Information Systems II

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 312))

Abstract

Crowdsourcing is a technique that aims to obtain data, ideas, and funds, conduct tasks, or even solve problems with the aid of a group of people. It’s a useful technique to save money and time. The quality of data is an issue that confronts crowdsourcing websites; as the data is obtained from the crowd, and how they control the quality of data. In some of the crowdsourcing websites they have implemented mechanisms in order to manage the data quality; such as, rating, reporting, or using specific tools. In this paper, five crowdsourcing websites: Wikipedia, Amazon Mechanical Turk, YouTube, Rally Fighter, and Kickstarter are studied as cases in order to identify the possible quality assurance methods or techniques that are useful to represent crowdsourcing data.

A survey is conducted to gather general opinion about the range of reliability of crowdsourcing sites, their passion and contribution to improve the contents of these sites. Combining those to the available knowledge in the crowdsourcing research, the paper highlights the factors that influence the data quality in crowdsourcing.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Howe, J.: The Rise of Crowdsourcing. Wired magazine 14 (6), 1-4 (2006)

    Google Scholar 

  2. Webster, M.: http://www.merriam-webster.com/dictionary/crowdsourcing (accessed April 19, 2014)

  3. Webster, M.: http://www.merriam-webster.com/dictionary/quality?show=0&t=1398180177 (accessed April 19, 2014)

  4. Roebuck, K.: Data Quality: High-impact Strategies - What You Need to Know: Definitions Adoptions, Impact, Benefits, Maturity, Vendors. Emereo Publishing (2001)

    Google Scholar 

  5. Brabham, D.C.: Crowdsourcing: A Model for Leveraging Online Communities. In: Delwiche, A., Henderson, J. (eds.) The Routledge handbook of participatory cultures. Routledge, New York (2001)

    Google Scholar 

  6. Aitamurto, T., Leiponen, A., Tee, R.: The Promise of Idea Crowdsourcing – Benefits, Contexts, Limitations. white paper online available (2011)

    Google Scholar 

  7. Soresina, C.: SkipsoLabs, http://www.skipsolabs.com/en/blog/crowdsourcing/types-ofcrowdsourcing (accessed March 6, 2014)

  8. Erickson, T.: Geocentric Crowdsourcing and Smarter Cities: Enabling Urban Intelligence in Cities and Regions. Founded in the industry website, Crowdsourcing.org (September 2010)

    Google Scholar 

  9. Mashhadi, A.J., Capra, L.: Quality Control for Real-time Ubiquitous Crowdsourcing. In: Proceedings of the 2nd international workshop on Ubiquitous Crowdsouring, pp. 5–8 (2011)

    Google Scholar 

  10. Hoßfeld, T., Seufert, M., Hirth, M., Zinner, T., Tran-Gia, P., Schatz, R.: Quantification of YouTube QoE via Crowdsourcing. In: IEEE International Symposium on Multimedia, pp. 494–499 (2011)

    Google Scholar 

  11. Aker, A., El-Haj, M., Albakour, M.-D., Kruschwitz, U.: Assessing Crowdsourcing Quality through Objective Tasks. In: Proceedings of the 8th International Conference on Language Resources and Evaluation (LREC 2012), Istanbul (2012)

    Google Scholar 

  12. Oleson, D., Sorokin, A., Laughlin, G., Hester, V., Le, J., Biewald, L.: Programmatic Gold: Targeted and Scalable Quality Assurance in Crowdsourcing, Human Computation. Papers from the 2011 AAAI Workshop (WS-11-11), pp. 43–48 (2011)

    Google Scholar 

  13. Li, K., Xiao, J., Wang, Y., Wang, Q.: Analysis of the Key Factors for Software Quality in Crowdsourcing Development: An Empirical Study on TopCoder.com. In: IEEE 37th Annual Computer Software and Applications Conference, pp. 812–817 (2013)

    Google Scholar 

  14. Zaidan, O.F., Callison-Burch, C.: Crowdsourcing Translation: Professional Quality from Non-Professionals. In: Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics (ACL 2011), pp. 1220–1229 (2011)

    Google Scholar 

  15. Papineni, K., Poukos, S., Ward, T., Zhu, W.-J.: BLEU: a method for automatic evaluation of machine translation. In: Proceedings of ACL, pp. 311–318 (2002)

    Google Scholar 

  16. Ipeirotis, P.G., Provost, F., Wang, J.: Quality management on Amazon Mechanical Turk. In: Proceedings of the Second Human Computation Workshop (KDD-HCOMP 2010), Washington DC, USA (2010)

    Google Scholar 

  17. Dawid, A.P., Skene, A.M.: Maximum likelihood estimation of observer error-rates using the EM algorithm. J. Roy. Statist. Soc. C (Applied Statistics) 28(1), 20–28 (1979)

    Google Scholar 

  18. QualificationRequirement - Amazon Mechanical Turk, http://docs.aws.amazon.com/AWSMechTurk/latest/AWSMturkAPI/ApiReference_QualificationRequirementDataStructureArticle.html (accessed March 27, 2014)

  19. SpeechInk, Getting Qualified To Do Jobs (June 26, 2010), http://www.youtube.com/watch?v=yMXlCaH7VcQ (accessed March 27, 2014)

  20. Wikipedia, http://en.wikipedia.org/wiki/Wikipedia:Five_pillars (accessed April 22, 2014)

  21. Wikipedia, http://en.wikipedia.org/wiki/Wikipedia:About (accessed April 22 2014)

  22. Tools - YouTube, YouTube, https://www.youtube.com/yt/creators/tools.html (accessed April 25, 2014)

  23. YouTube Community Guidelines, YouTube, https://www.youtube.com/t/community_guidelines (accessed April 25, 2014)

  24. Other reporting options,YouTube, https://support.google.com/youtube/answer/2802057?hl=en&ref_topic=2803138 (Accessed April 25, 2014)

  25. What does the, Mark as Spam, feature do?, YouTube, https://support.google.com/youtube/answer/128036?hl=en (accessed 25 April 2014)

  26. Submit a copyright infringement notification, YouTube, https://support.google.com/youtube/answer/128036?hl=en (accessed April 25, 2014)

  27. How Content ID works, YouTube, https://support.google.com/youtube/answer/2797370?p=cid_what_is&rd=1 (accessed April 25, 2014)

  28. How to use the YouTube Content Verification Program, YouTube, https://support.google.com/youtube/answer/3010500?hl=en (accessed April 25, 2014)

  29. Rally Fighter, http://localmotors.com/rallyfighter/ (accessed April 17, 2014)

  30. Munoz, J.A.: CNN, How the Internet built a $100,000 race car (March 13, 2013), http://www.cnn.com/2013/03/12/tech/web/crowdsourced-carsxsw/ (accessed April 17, 2014)

  31. Kickstarter, https://www.kickstarter.com/ (accessed April 21, 2014)

  32. kickstarter, https://www.kickstarter.com/help/guidelines (accessed April 21, 2014)

  33. kickstarter, https://www.kickstarter.com/privacy?ref=footer (Accessed April 21, 2014)

  34. Jane, W., Watson, R.T.: Analyzing the past to present, MIS quarterly (2001), https://www.kickstarter.com/privacy?ref=footer

  35. Saunders, M., Lewis, P., Thornhill, A.: Research Methods for Business Students, 6th edn. Prentice Hall/ Pearson Education (2009)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Dept. of Software Engineering, College of Computer & Information Sciences, Prince Sultan University, Riyadh, Saudi Arabia

    May Al Sohibani, Najd Al Osaimi, Reem Al Ehaidib, Sarah Al Muhanna & Ajantha Dahanayake

Authors
  1. May Al Sohibani
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Najd Al Osaimi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Reem Al Ehaidib
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sarah Al Muhanna
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ajantha Dahanayake
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Aristotle University of Thessaloniki, Thessaloniki, Greece

    Nick Bassiliades

  2. Department of Mathematics and Informatics, Faculty of Sciences, University of Novi Sad, Novi Sad, Serbia

    Mirjana Ivanovic

  3. Faculty of Natural Science and Mathematics, Ss Cyril and Methodious University Institute for Informatics, Skopje, Macedonia

    Margita Kon-Popovska

  4. Department of Informatics, Aristotle University of Thessaloniki, University Campus, Thessaloniki, Greece

    Yannis Manolopoulos

  5. Paris Descartes University, Paris, France

    Themis Palpanas

  6. EECS Department, Northwestern University, Evanston, Illinois, USA

    Goce Trajcevski

  7. Department of Informatics, Aristotle University of Thessalonik, Thessaloniki, Greece

    Athena Vakali

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this paper

Cite this paper

Al Sohibani, M., Al Osaimi, N., Al Ehaidib, R., Al Muhanna, S., Dahanayake, A. (2015). Factors That Influence the Quality of Crowdsourcing. In: Bassiliades, N., et al. New Trends in Database and Information Systems II. Advances in Intelligent Systems and Computing, vol 312. Springer, Cham. https://doi.org/10.1007/978-3-319-10518-5_22

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-10518-5_22

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-10517-8

  • Online ISBN: 978-3-319-10518-5

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation