Distributed and Parallel Databases

, Volume 33, Issue 1, pp 123–141 | Cite as

Mobile crowdsourcing: four experiments on platforms and tasks

  • Vincenzo Della MeaEmail author
  • Eddy Maddalena
  • Stefano Mizzaro


We study whether the tasks currently proposed on crowdsourcing platforms are adequate to mobile devices. We aim at understanding both (i) which crowdsourcing platforms, among the existing ones, are more adequate to mobile devices, and (ii) which kinds of tasks are more adequate to mobile devices. Results of four diversified experiments (three user studies and one heuristic evaluation) hint that: some crowdsourcing platforms seem more adequate to mobile devices than others; some inadequacy issues seem rather superficial and can be resolved by a better task design; some kinds of tasks are more adequate than others; there might be some unexpected opportunities with mobile devices; and spam on the requester side should be taken into account.


Crowdsourcing platforms Mobile devices 



We thank the referees (especially one of them) that provided useful remarks to improve the paper, and Giorgio Brajnik and Luca Di Gaspero for their suggestions on the statistical analysis.


  1. 1.
    Alt, F., Shirazi, A.S., Schmidt, A., Kramer, U., Nawaz, Z.: Location-based crowdsourcing: extending crowdsourcing to the real world. In: Proceedings of the 6th Nordic Conference on Human-Computer Interaction: Extending Boundaries, NordiCHI ’10, pp. 13–22. ACM, New York (2010). doi: 10.1145/1868914.1868921
  2. 2.
    Amini, S., Li, Y.: CrowdLearner: Rapidly creating mobile recognizers using crowdsourcing. In: Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology, UIST ’13, pp. 163–172. ACM, New York (2013). doi: 10.1145/2501988.2502029.
  3. 3.
    Chatzimilioudis, G., Konstantinidis, A., Laoudias, C., Zeinalipour-Yazti, D.: Crowdsourcing with smartphones. Internet Comput. IEEE 16(5), 36–44 (2012)CrossRefGoogle Scholar
  4. 4.
    Della Mea, V., Maddalena, E., Mizzaro, S.: Crowdsourcing to mobile users: A study of the role of platforms and tasks. In: R. Cheng, A. Das Sarma, S. Maniu, P. Senellart (eds.) DBCrowd 2013: First VLDB Workshop on Databases and Crowdsourcing, pp. 14–25 (2013). Accessed 1 Aug 2014
  5. 5.
    Demirbas, M., Bayir, M.A., Akcora, C.G., Yilmaz, Y.S., Ferhatosmanoglu, H.: Crowd-sourced sensing and collaboration using twitter. In: Proceedings of the 2010 IEEE International Symposium on A World of Wireless, Mobile and Multimedia Networks (WoWMoM), WOWMOM ’10, pp. 1–9. IEEE Computer Society, Washington (2010). doi: 10.1109/WOWMOM.2010.5534910.
  6. 6.
    Eagle, N.: txteagle: mobile crowdsourcing. In: Proceedings of the 3rd International Conference on Internationalization, Design and Global Development: Held as Part of HCI International 2009, IDGD ’09, pp. 447–456. Springer, Berlin (2009). doi:  10.1007/978-3-642-02767-3_50.
  7. 7.
    Gupta, A., Thies, W., Cutrell, E., Balakrishnan, R.: mClerk: enabling mobile crowdsourcing in developing regions. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’12, pp. 1843–1852. ACM, New York (2012). doi:  10.1145/2207676.2208320
  8. 8.
    Howe, J.: Crowdsourcing: Why the Power of the Crowd Is Driving the Future of Business. Random House Inc, New York (2008)Google Scholar
  9. 9.
    Ipeirotis, P.G.: Analyzing the amazon mechanical turk marketplace. XRDS 17(2), 16–21 (2010). doi: 10.1145/1869086.1869094 CrossRefGoogle Scholar
  10. 10.
    Ipeirotis, P.G.: Mechanical turk: Now with 40.92 % spam. Blog post (2010). Accessed February 2014
  11. 11.
    Luon, Y., Aperjis, C., Huberman, B.: Rankr: a mobile system for crowdsourcing opinions. In: J. Zhang, J. Wilkiewicz, A. Nahapetian (eds.) Mobile Computing, Applications, and Services, Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol. 95, pp. 20–31. Springer, Berlin (2012). doi: 10.1007/978-3-642-323201_2
  12. 12.
    Meeker, M., Wu, L.: Internet Trends D11 Conference—The annual Internet Trends Report (2013). Accessed 1 Aug 2014
  13. 13.
    Musthag, M., Ganesan, D.: Labor dynamics in a mobile micro-task market. In: W.E. Mackay, S.A. Brewster, S. Bødker (eds.) CHI, pp. 641–650. ACM, New York (2013).Google Scholar
  14. 14.
    Narula, P., Gutheim, P., Rolnitzky, D., Kulkarni, A., Hartmann, B.: MobileWorks: a mobile crowdsourcing platform for workers at the bottom of the pyramid. In: Proceedings of the HCOMP’11 (2011)Google Scholar
  15. 15.
    Nielsen, J., Molich, R.: Heuristic evaluation of user interfaces. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’90, pp. 249–256. ACM, New York (1990). doi: 10.1145/97243.97281.
  16. 16.
    Phuttharak, J., Loke, S.W.: LogicCrowd: A declarative programming platform for mobile crowdsourcing. In: Proceedings of the 2013 12th IEEE International Conference on Trust, Security and Privacy in Computing and Communications, TRUSTCOM ’13, pp. 1323–1330. IEEE Computer Society, Washington (2013). doi: 10.1109/TrustCom.2013.158.
  17. 17.
    Proaps, A.B., Landers, R.N., Reddock, C.M., Cavanaugh, K.J., Kantrowitz, T.M.: Mobile and computer-based talent assessments: Implications of workload and usability. In: CHI ’14 Extended Abstracts on Human Factors in Computing Systems, CHI EA ’14, pp. 2299–2304. ACM, New York (2014). doi: 10.1145/2559206.2581136
  18. 18.
    Stevens, M., D’Hondt, E.: Crowdsourcing of pollution data using smartphones. In: Workshop on Ubiquitous Crowdsourcing (2010)Google Scholar
  19. 19.
    Väätäjä, H., Egglestone, P.: Briefing news reporting with mobile assignments: perceptions, needs and challenges. In: Proceedings of the ACM 2012 conference on Computer Supported Cooperative Work, pp. 485–494. ACM, New York (2012)Google Scholar
  20. 20.
    Väätäjä, H., Sirkkunen, E., Ahvenainen, M.: A field trial on mobile crowdsourcing of news content – factors influencing participation. In: Human-Computer Interaction-INTERACT 2013, pp. 54–73. Springer, Berlin (2013)Google Scholar
  21. 21.
    Väätäjä, H., Vainio, T., Sirkkunen, E.: Location-based crowdsourcing of hyperlocal news: Dimensions of participation preferences. In: Proceedings of the 17th ACM International Conference on Supporting Group Work, GROUP ’12, pp. 85–94. ACM, New York (2012). doi: 10.1145/2389176.2389189.
  22. 22.
    Väätäjä, H., Vainio, T., Sirkkunen, E., Salo, K.: Crowdsourced news reporting: supporting news content creation with mobile phones. In: Proceedings of the 13th International Conference on Human Computer Interaction with Mobile Devices and Services, MobileHCI ’11, pp. 435–444. ACM, New York (2011). doi: 10.1145/2037373.2037438.
  23. 23.
    Yan, T., Kumar, V., Ganesan, D.: Crowdsearch: exploiting crowds for accurate real-time image search on mobile phones. In: MobiSys ’10: Proceedings of the 8th International Conference on Mobile Systems, Applications and Services, pp. 77–90. ACM Press, New York (2010). Accessed 1 Aug 2014
  24. 24.
    Yan, T., Marzilli, M., Holmes, R., Ganesan, D., Corner, M.: mCrowd: a platform for mobile crowdsourcing. In: Proceedings of the 7th ACM Conference on Embedded Networked Sensor Systems, SenSys ’09, pp. 347–348. ACM, New York (2009). doi: 10.1145/1644038.1644094.

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  • Vincenzo Della Mea
    • 1
    Email author
  • Eddy Maddalena
    • 1
  • Stefano Mizzaro
    • 1
  1. 1.Department of Mathematics and Computer ScienceUniversity of UdineUdineItaly

Personalised recommendations