An investigation of factors affecting the visits of online crowdsourcing and labor platforms

  • Evangelos MourelatosEmail author
  • Manolis Tzagarakis


Nowadays, the economic activities have become increasingly digital since hundreds of millions of Internet users are using crowdsourcing platforms either to work at an online job as workers, or as a model of problem-solving and production as requesters. This growing workforce makes it necessary from the perspective of the online platforms, to fully understand the business dimensions of this emerging and innovative “online labor” phenomenon, which can rapidly change the future of work and work organization in the online world. This paper aims to investigate and analyze the visits of online labor platforms that offer crowdsourcing and crowdfunding services. Using websites’ metrics data drawn from Alexa for the time period 2012-2016 the paper uses Ordinary Least Squares (OLS) and Fixed Effects (FE) regression analysis to examine correlations between visits and website characteristics. The research shows that the sessions of an online labor marketplace website from mobile devices have an increasing trend to be positively correlated to the quality mechanisms a website deploys as well as on location-dependent factors. The results are expected to provide insights on how the online labor website characteristics affect their traffic and thus inform about their evolution and improvement.


Crowdsourcing Human computation Online labor Online labor markets Websites review Performance Panel regression models 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.



  1. 1.
    Abraham, K., Haltiwanger, J., Sandusky, K., Spletzer, J. (2017). Measuring the gig economy: current knowledge and open issues. In Measuring and accounting for innovation in the 21st century. University of Chicago Press.Google Scholar
  2. 2.
    Agrawal, A.K., Catalini, C., Goldfarb, A. (2011). The geography of crowdfunding (No. w16820). Cambridge: National Bureau of Economic Research.CrossRefGoogle Scholar
  3. 3.
    Allahbakhsh, M., Benatallah, B., Ignjatovic, A., Motahari-Nezhad, H.R., Bertino, E., Dustdar, S. (2013). Quality control in crowdsourcing systems. IEEE Internet Computing, 17(2), 76–81.CrossRefGoogle Scholar
  4. 4.
    Alsyouf, I. (2007). The role of maintenance in improving companies’ productivity and profitability. International Journal of Production Economics, 105(1), 70–78.CrossRefGoogle Scholar
  5. 5.
    Autor, D. (2004). Labor market intermediation: what it is, why it is growing, and where it is going. NBER Reporter, National Bureau of Economic Research.Google Scholar
  6. 6.
    Bassett, Jr., Gilbert, W., Tam, M., Knight, K. (2002). Quantile models and estimators for data analysis. Metrika, 55(1), 17–26.Google Scholar
  7. 7.
    Beck, H. (1999). Jobs on the wire: in search of the perfect labor market. Netnomics, 1(1), 71–88.CrossRefGoogle Scholar
  8. 8.
    Benwell, G.L., Deans, K.R., Ghandour, A. (2010). The relationship between website metrics and the financial performance of online businesses, ICIS Proceedings (International Conference on Information Systems).Google Scholar
  9. 9.
    Blohm, I., Zogaj, S., Bretschneider, U. (2017). How to manage crowdsourcing platforms effectively? Journal of Information Science, 60(2), 122–169.Google Scholar
  10. 10.
    Brabham, D.C. (2008). Crowdsourcing as a model for problem solving: an introduction and cases. Convergence: The International Journal of Research into New Media Technologies, 14(1), 75–90.CrossRefGoogle Scholar
  11. 11.
    Brabham, D.C. (2010). Moving the crowd at threadless: motivations for participation in a crowdsourcing application. Information, Communication & Society, 13(8), 1122–1145.CrossRefGoogle Scholar
  12. 12.
    Buchinsky, M. (1998). Recent advances in quantile regression models: a practical guideline for empirical research. Journal of Human Resources, 33(1), 88–126.CrossRefGoogle Scholar
  13. 13.
    Burtch, G., Ghose, A., Wattal, S. (2013). An empirical examination of the antecedents and consequences of contribution patterns in crowd-funded markets. Information Systems Research, 24(3), 499–519.CrossRefGoogle Scholar
  14. 14.
    Cetintas, S., Chen, D., Si, L., Shen, B., Datbayev, Z. (2011). Forecasting counts of user visits for online display advertising with probabilistic latent class models. In Proceedings of the 34th international ACM SIGIR conference on Research and development in Information Retrieval.Google Scholar
  15. 15.
    Chatzimilioudis, G., Konstantinidis, A., Laoudias, C., Zeinalipour-Yazti, D. (2012). Crowdsourcing with smartphones. IEEE Internet Computing, 16(5), 36–44.CrossRefGoogle Scholar
  16. 16.
    Cheung, C.M.K., & Lee, M.K.O. (2010). A theoretical model of intentional social action in online social networks. Decision Support Systems, 49(1), 24–30.CrossRefGoogle Scholar
  17. 17.
    Corrado, C.A., & Hulten, C.R. (2015). How do you measure a “technological revolution”? American Economic Review, 100(2), 99–104.CrossRefGoogle Scholar
  18. 18.
    Cronin, M.J. (1997). Global advantage on the internet: from corporate connectivity to international competitiveness. New York: Wiley.Google Scholar
  19. 19.
    Dalton, D.R., Todor, W.D., Spendolini, M.J., Fielding, G.J., Porter, L.W. (1980). Organization structure and performance: a critical review. Academy of Management Review, 5(1), 49–64.CrossRefGoogle Scholar
  20. 20.
    Doan, A., Ramakrishnan, R., Halevy, A. (2011). Crowdsourcing systems on the world wide web. Communications of the ACM, 54(4), 86–96.CrossRefGoogle Scholar
  21. 21.
    Donmez, P., Carbonell, J.G., Schneider, J. (2009). Efficiently learning the accuracy of labeling sources for selective sampling. In Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM.Google Scholar
  22. 22.
    Eagle, N. (2009). txteagle: mobile crowdsourcing. In International conference on internationalization, design and global development (pp. 447–456). Berlin and Heidelberg: Springer.Google Scholar
  23. 23.
    Estellés-Arolas, E., & González-Ladrón-De-Guevara, F. (2012). Towards an integrated crowdsourcing definition. Journal of Information Science, 38(2), 189–200.CrossRefGoogle Scholar
  24. 24.
    Farrell, D., & Greig, F. (2016). Paychecks, paydays, and the online platform economy: big data on income volatility, JP Morgan Chase Institute.Google Scholar
  25. 25.
    Felstiner, A. (2011). Working the crowd: employment and labor law in the crowdsourcing industry. Berkeley Journal of Employment and Labor Law, 32(1), 143–203.Google Scholar
  26. 26.
    Folta, T.B., Cooper, A.C., Baik, Y.S. (2006). Geographic cluster size and firm performance. Journal of Business Venturing, 21(2), 217–242.CrossRefGoogle Scholar
  27. 27.
    Geerts, S. (2009). Discovering crowdsourcing: theory, classification and directions for use, Unpublished Master of Science in Innovation Management thesis, Eindhoven University of Technology. at, 202009.
  28. 28.
    Goncalves, J., Hosio, S., Vukovic, M., Konomi, S.I. (2017). Mobile and situated crowdsourcing. International Journal of Human-Computer Studies, 102, 1–3.CrossRefGoogle Scholar
  29. 29.
    Graham, M., Hjorth, I., Lehdonvirta, V. (2017). Digital labour and development: impacts of global digital labour platforms and the gig economy on worker livelihoods. Transfer: European Review of Labour and Research, 23(2), 135–162.CrossRefGoogle Scholar
  30. 30.
    Gupta, A., Thies, W., Cutrell, E., Balakrishnan, R. (2012). mClerk: enabling mobile crowdsourcing in developing regions. In Proceedings of the SIGCHI conference on human factors in computing systems (pp. 1843–1852). ACM.Google Scholar
  31. 31.
    Hirth, M., Hoßfeld, T., Tran-Gia, P. (2011). Anatomy of a crowdsourcing platform-using the example of microworkers. com. In IEEE 5th international conference on innovative mobile and internet services in ubiquitous computing (IMIS).Google Scholar
  32. 32.
    Hoch, I. (1962). Estimation of production function parameters combining time series and cross-section data. Econometrica, 30(1), 34–53.CrossRefGoogle Scholar
  33. 33.
    Hoegg, R., Martignoni, R., Meckel, M., Stanoevska-Slabeva, K. (2006). Overview of business models for Web 2.0 communities. Workshop GeNeMe Dresden.Google Scholar
  34. 34.
    Horton, J.J. (2010). Online labor markets. In International workshop on internet and network economics (pp. 515–522). Berlin: Springer.Google Scholar
  35. 35.
    Horton, J.J., & Chilton, L.B. (2010). The labor economics of paid crowdsourcing. In 11th ACM Conference on Electronic Commerce, San Jose, USA (pp 209-218).Google Scholar
  36. 36.
    Horton, J., Kerr, W.R., Stanton, C. (2017). Digital labor markets and global talent flows (No. w23398). National Bureau of Economic Research.Google Scholar
  37. 37.
    Howe, J. (2006). The rise of crowdsourcing. Wired Magazine, 14(6), 1–4.Google Scholar
  38. 38.
    Huberman, B.A., Romero, D.M., Wu, F. (2009). Crowdsourcing, attention and productivity. Journal of Information Science, 35(6), 758–765.CrossRefGoogle Scholar
  39. 39.
    Huws, U., & Joyce, S. (2016). Crowd working survey: size of the UK’s ‘Gig Economy’ revealed for the first time. University of Hertfordshire.Google Scholar
  40. 40.
    Ipeirotis, P.G. (2010). Demographics of mechanical turk, NYU Working Paper No. ;CEDER-10-01. Available at SSRN:
  41. 41.
    Kässi, O., & Lehdonvirta, V. (2018). Online labour index: measuring the online gig economy for policy and research. Technological forecasting and social change.Google Scholar
  42. 42.
    Kim, S., & Lee, Y. (2006). Global online marketplace: a cross-cultural comparison of website quality. International Journal of Consumer Studies, 30(6), 533–543.CrossRefGoogle Scholar
  43. 43.
    Kim, W., Jeong, O.R., Lee, S.W. (2010). On social web sites. Information Systems, 35(2), 215–236.CrossRefGoogle Scholar
  44. 44.
    King, J. (2014). E-books for leisure and learning-the brisbane boys’ college experience. Access (Online), 28(3), 42.Google Scholar
  45. 45.
    Krippendorff, K. (2004). Content analysis: an introduction to its methodology. Sage.Google Scholar
  46. 46.
    Kuek, S.C., Paradi-Guilford, C., Fayomi, T., Imaizumi, S., Ipeirotis, P., Pina, P., Singh, M. (2015). The global opportunity in online outsourcing (No. 22284). The World Bank.Google Scholar
  47. 47.
    Lease, M. (2011). On quality control and machine learning in crowdsourcing. Human Computation, 11(11), Papers from the 2011 AAAI Workshop, San Francisco, California, USA.Google Scholar
  48. 48.
    Lehdonvirta, V., & Ernkvist, M. (2011). Knowledge map of the virtual economy: converting the virtual economy into development potential. World Bank.Google Scholar
  49. 49.
    Lo, B.W., & Sedhain, R.S. (2006). How reliable are website rankings? Implications for e-business advertising and internet search. Issues in Information Systems, 7(2), 233–238.Google Scholar
  50. 50.
    Malone, T.W., & Laubacher, R.J. (1999). The dawn of the e-lance economy. In Electronic business engineering (pp. 13–24). Heidelberg: Physica.Google Scholar
  51. 51.
    Marsden, P. (2009). Crowdsourcing. Conragious Magazine, lssue, pp. 24–8.Google Scholar
  52. 52.
    Martin, D., Carpendale, S., Gupta, N., Hoßfeld, T., Naderi, B., Redi, J., ..., Wechsung, I. (2017). Understanding the crowd: ethical and practical matters in the academic use of crowdsourcing. In Evaluation in the crowd. crowdsourcing and human-centered experiments (pp. 27–69). Cham: Springer.Google Scholar
  53. 53.
    Martin, N, Lessmann, S., Voß, S. (2008). Crowdsourcing: Systematisierung praktischer Ausprägungen und verwandter Konzepte. In Bichler, M., Hess, T., Krcmar, H., Lechner, U., Matthes, F., Picot, A., Speitkamp, B., Wolf, P. (Eds.) Multikonferenz Wirtschaftsinformatik 2008 (pp. 1251–1263). Berlin: Gito.Google Scholar
  54. 54.
    Miao, C., Yu, H., Shen, Z., Leung, C. (2016). Balancing quality and budget considerations in mobile crowdsourcing. Decision Support Systems, 90, 56–64.CrossRefGoogle Scholar
  55. 55.
    Mollick, E. (2014). The dynamics of crowdfunding: an exploratory study. Journal of Business Venturing, 29(1), 1–16.CrossRefGoogle Scholar
  56. 56.
    Mourelatos, E., Tzagarakis, M., Dimara, E. (2016). A review of online crowdsourcing platforms. South-Eastern Europe Journal of Economics, 14(1), 59–73.Google Scholar
  57. 57.
    Mukherjee, S., Sujithan, R., Subasic, P. (2014). Detecting trending topics using page visitation statistics. In Proceedings of the 23rd International Conference on World Wide Web, Seoul, Korea (pp. 347–348).Google Scholar
  58. 58.
    Mundlak, Y. (1961). Empirical production function free of management bias. Journal of Farm Economics, 43(1), 44–56.CrossRefGoogle Scholar
  59. 59.
    Nair, A. (2006). Meta-analysis of the relationship between quality management practices and firm performance—implications for quality management theory development. Journal of Operations Management, 24(6), 948–975.CrossRefGoogle Scholar
  60. 60.
    Narula, P., Gutheim, P., Rolnitzky, D., Kulkarni, A., Hartmann, B. (2011). MobileWorks: a mobile crowdsourcing platform for workers at the bottom of the pyramid. Human Computation, 11, 11. Papers from the 2011 AAAI Workshop, San Francisco, California, USA.Google Scholar
  61. 61.
    Ortega, J. L., & Aguillo, I. (2010). Differences between web sessions according to the origin of their visits. Journal of Informetrics, 4(3), 331–337.CrossRefGoogle Scholar
  62. 62.
    Peng, X., Babar, M.A., Ebert, C. (2014). Collaborative software development platforms for crowdsourcing. IEEE Software, 31(2), 30–36.CrossRefGoogle Scholar
  63. 63.
    Plaza, B. (2011). Google analytics for measuring website performance. Tourism Management, 32(3), 477–481.CrossRefGoogle Scholar
  64. 64.
    Poetz, M.K., & Schreier, M. (2012). The value of crowdsourcing: can users really compete with professionals in generating new product ideas? Journal of Product Innovation Management, 29(2), 245–256.CrossRefGoogle Scholar
  65. 65.
    Porter, M.E. (1979). The structure within industries and companies’ performance. The Review of Economics and Statistics, 61(2), 214–227.CrossRefGoogle Scholar
  66. 66.
    Porter, M.E. (1986). Changing patterns of international competition. California Management Review, 28(2), 9–40.CrossRefGoogle Scholar
  67. 67.
    Rappa, M. (2000). Business models on the web. North Carolina State University (ecommerce. ncsu. edu), 13.Google Scholar
  68. 68.
    Richard, P.J., Devinney, T.M., Yip, G.S., Johnson, G. (2009). Measuring organizational performance: towards methodological best practice. Journal of Management, 35(3), 718–804.CrossRefGoogle Scholar
  69. 69.
    Rogstadius, J., Kostakos, V., Kittur, A., Smus, B., Laredo, J., Vukovic, M. (2011). An assessment of intrinsic and extrinsic motivation on task performance in crowdsourcing markets. ICWSM, 11, 17–21.Google Scholar
  70. 70.
    Ross, J., Irani, L., Silberman, M.S., Zaldivar, A., Tomlinson, B. (2010). Who are the crowdworkers? Shifting demographics in mechanical turk. CHI’10 extended abstracts on human factors in computing systems. ACM.Google Scholar
  71. 71.
    Saxton, G.D., Oh, O., Kishore, R. (2013). Rules of crowdsourcing: models, issues, and systems of control. Information Systems Management, 30(1), 2–20.CrossRefGoogle Scholar
  72. 72.
    Schenk, E., & Guittard, C. (2011). Towards a characterization of crowdsourcing practices. Journal of Innovation Economics & Management, 1, 93–107.CrossRefGoogle Scholar
  73. 73.
    Shuen, A. (2008). Web 2.0: a strategy guide: business thinking and strategies behind successful Web 2.0 implementations. O’Reilly Media, Inc.Google Scholar
  74. 74.
    Stevanovic, D., Vlajic, N., An, A. (2011). Unsupervised clustering of web sessions to detect malicious and non-malicious website users. Procedia Computer Science, 5, 123–131.CrossRefGoogle Scholar
  75. 75.
    Straub, D., Rai, A., Klein, R. (2004). Measuring firm performance at the network level: a nomology of the business impact of digital supply networks. Journal of Management Information Systems, 21(1), 83–114.CrossRefGoogle Scholar
  76. 76.
    Surya, A., & Sharma, D. (2013). A comparative analysis of clickstream as web page importance metric. In Proceedings of 2013 IEEE Conference on Information and Communication Technologies (ICT 2013), India.Google Scholar
  77. 77.
    Thackeray, R., Neiger, B.L., Hanson, C.L., McKenzie, J.F. (2008). Enhancing promotional strategies within social marketing programs: use of Web 2.0 social media. Health Promotion Practice, 9.4, 338–343.CrossRefGoogle Scholar
  78. 78.
    Thuan, N.H., Antunes, P., Johnstone, D. (2016). Factors influencing the decision to crowdsource: a systematic literature review. Information Systems Frontiers, 18(10), 47–68.CrossRefGoogle Scholar
  79. 79.
    Tierney, H.L., & Pan, B. (2012). A poisson regression examination of the relationship between website traffic and search engine queries. NETNOMICS: Economic Research and Electronic Networking, 13(3), 155–189.CrossRefGoogle Scholar
  80. 80.
    Turel, O., & Serenko, A. (2006). Satisfaction with mobile services in Canada: an empirical investigation. Telecommunications Policy, 30(5), 314–331.CrossRefGoogle Scholar
  81. 81.
    Turel, O., & Zhang, Y. (2011). Should I e-collaborate with this group? A multilevel model of usage intentions. Information & Management, 48, 62–68.CrossRefGoogle Scholar
  82. 82.
    Vashistha, A., Vaish, R., Cutrell, E., Thies, W. (2015). The whodunit challenge: mobilizing the crowd in India. In Human-Computer Interaction (pp. 505–521). Springer International Publishing.Google Scholar
  83. 83.
    Vaughan, L, & Yang, R. (2013). Web traffic and organization performance measures: relationships and data sources examined. Journal of Informetrics, 7(3), 699–711.CrossRefGoogle Scholar
  84. 84.
    Wais, P., Lingamneni, S., Cook, D., Fennell, J., Goldenberg, B., Lubarov, D., Marin, D., Simons, H. (2010). Towards building a High-Quality workforce with mechanical turk. In Proceedings of NIPS Workshop on Computational Social Science and the Wisdom of Crowds, (NIPS) (pp. 1–5).Google Scholar
  85. 85.
    Wang, J., Ipeirotis, P, Provost, F. (2011). Managing crowdsourcing workers. The 2011 winter conference on business intelligence.Google Scholar
  86. 86.
    Wang, J., Ipeirotis, P., Provost, F. (2013). A framework for quality assurance in crowdsourcing. NYU Working Paper No. 2451/31833.Google Scholar
  87. 87.
    Weinstein, R.S. (2013). Crowdfunding in the US and abroad: What to expect when you’re expecting. Cornell Int’l LJ, 46, 427.Google Scholar
  88. 88.
    Zahran, D.I., Al-Nuaim, H.A., Rutter, M.J., Benyon, D. (2014). A comparative approach to web evaluation and website evaluation methods. International Journal of Public Information Systems, 10(1), 20–39.Google Scholar
  89. 89.
    Zhao, Y., & Zhu, Q. (2014). Evaluation on crowdsourcing research: current status and future direction. Information Systems Frontiers, 16(3), 417–434.CrossRefGoogle Scholar
  90. 90.
    Zhu, Y., Zhang, Q., Zhu, H., Yu, J., Cao, J., Ni, L.M. (2014). Towards truthful mechanisms for mobile crowdsourcing with dynamic smartphones. In IEEE 34th International Conference on Distributed Computing Systems (ICDCS) (pp. 11–20). IEEE.Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Department of EconomicsUniversity of PatrasPatrasGreece

Personalised recommendations