Abstract
This chapter evaluates the potential of the developed prototype to realize and study personalized task recommendation on existing crowdsourcing platforms and thereby improve the match between contributors and available tasks. To this intent, the following sections present a sequence of three studies that were conducted along the process of field testing and deploying the metacrowd service on the Mechanical Turk platform. Section 5.1 first describes a pilot study that gathered initial feedback from a small group of experienced contributors. Section 5.2 then lays out the details of a large-scale survey on general task search behavior and on the perceived utility of the metacrowd service. Section 5.3, finally, analyzes a dataset that was gathered during an extended period of productive use and discusses the challenges identified during the evaluation.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
References
Chilton, L. B., Horton, J. J., Miller, R. C., & Azenkot, S. (2010). Task search in a human computation market. In Proceedings of the ACM SIGKDD Workshop on Human Computation (HCOMP) (pp. 1–9). New York, NY: ACM. doi:10.1145/1837885.1837889
Cohen, J. (1988). Statistical power analysis for the behavioral sciences. Hillsdale, NJ: Lawrence Erlbaum Associates.
Desrosiers, C., & Karypis, G. (2011). A comprehensive survey of neighborhood-based recommendation methods. In F. Ricci, L. Rokach, B. Shapira, & P. B. Kantor (Eds.), Recommender systems handbook (pp. 107–144). Springer: Boston, MA. doi:10.1007/978-0-387-85820-3_4.
Herlocker, J., Konstan, J., & Riedl, J. (2000). Explaining collaborative filtering recommendations. In Proceedings of the 2000 ACM Conference on Computer Supported Cooperative Work (CSCW) (pp. 241–250). New York, NY: ACM. doi:10.1145/358916.358995
Kacmarcik, G., Leithead, T., Rossi, J., Schepers, D., Höhrmann, B., Le Hégaret, P., et al. (2014). Document object model (DOM) level 3 events specification. W3C Working Draft. Retrieved December 1, 2014, from http://www.w3.org/TR/DOM-Level-3-Events/
Krosnick, J. A., Lavrakas, P. J., & Kim, N. (2014). Survey research. In H. T. Reis & C. M. Judd (Eds.), Handbook of research methods in social and personality psychology (2nd ed., pp. 404–442). Cambridge, UK: Cambridge University Press.
Krosnick, J. A., & Presser, S. (2010). Question and questionnaire design. In P. V. Marsden & J. D. Wright (Eds.), Handbook of survey research (2nd ed., pp. 263–314). Bingley, UK: Emerald Group Publishing.
Schulze, T., Seedorf, S., Geiger, D., Kaufmann, N., & Schader, M. (2011). Exploring task properties in crowdsourcing—An empirical study on Mechanical Turk. In 19th European Conference on Information Systems, Helsinki.
Shani, G., & Gunawardana, A. (2011). Evaluating recommendation systems. In F. Ricci, L. Rokach, B. Shapira, & P. B. Kantor (Eds.), Recommender systems handbook (pp. 257–297). Boston, MA: Springer. doi:10.1007/978-0-387-85820-3_8.
Spearman, C. (1904). The proof and measurement of association between two things. The American Journal of Psychology, 15(1), 72–101. doi:10.2307/1412159.
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Geiger, D. (2016). Personalized Task Recommendation in the Field. In: Personalized Task Recommendation in Crowdsourcing Systems. Progress in IS. Springer, Cham. https://doi.org/10.1007/978-3-319-22291-2_5
Download citation
DOI: https://doi.org/10.1007/978-3-319-22291-2_5
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-22290-5
Online ISBN: 978-3-319-22291-2
eBook Packages: Business and ManagementBusiness and Management (R0)