Abstract
In crowdsourcing systems, task recommendation can help workers to find their right tasks faster as well as help requesters to receive good quality output quicker. A number of previous works adopted active learning for task recommendation in crowdsourcing systems to achieve certain accuracy with a very low cost. However, the model updating methods in previous works are not suitable for real-world applications. In our paper, we propose a generic online-updating method for learning a factor analysis model, ActivePMF on TaskRec (Probabilistic Matrix Factorization with Active Learning on Task Recommendation Framework), for crowdsourcing systems. The larger the profile of a worker (or task) is, the less important is retraining its profile on each new work done. In case of the worker (or task) having large profile, our algorithm only retrains the whole feature vector of the worker (or task) and keeps all other entries in the matrix fixed. Besides, our algorithm runs batch update to further improve the performance. Experiment results show that our online-updating approach is accurate in approximating to a full retrain while the average runtime of model update for each work done is reduced by more than 90 % (from a few minutes to several seconds).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
NAACL 2010 workshop: http://sites.google.com/site/amtworkshop2010/data-1.
References
Howe, J.: The rise of crowdsourcing. Wired 14(6), 1–4 (2006)
Jung, H.J.: Quality assurance in crowdsourcing via matrix factorization based task routing. In: International Conference on World Wide Web (2014)
Jung, H.J., Lease, M.: Improving quality of crowdsourced labels via probabilistic matrix factorization. In: Human Computation Workshop at the 26th AAAI (2012)
Karimi, R., Freudenthaler, C., Nanopoulos, A., Schmidt-Thieme, L.: Active learning for aspect model in recommender systems. In: Proceedings of IEEE CIDM 2011, pp. 162–167 (2011)
Lewis, D.D., Gale, W.A.: A sequential algorithm for training text classifiers. In: Croft, B.W., van Rijsbergen, C.J. (eds.) SIGIR 1994, pp. 3–12. Springer, London (1994)
Rendle, S., Schmidt-Thieme, L.: Online-updating regularized kernel matrix factorization models for large-scale recommender systems. In: Proceedings of RecSys 2008, pp. 251–258. ACM, New York (2008)
Salakhutdinov, R., Mnih, A.: Probabilistic matrix factorization. In: Proceedings of the Twenty-First Annual Conference on Neural Information Processing Systems, NIPS 2007, Curran Associates Inc. (2007)
Yuen, M.-C., King, I., Leung, K.-S.: A survey of crowdsourcing systems. In: SocialCom 2011, pp. 766–773. IEEE Computer Society (2011)
Yuen, M.-C., King, I., Leung, K.-S.: TaskRec: probabilistic matrix factorization in task recommendation in crowdsourcing systems. In: Huang, T., Zeng, Z., Li, C., Leung, C.S. (eds.) ICONIP 2012. LNCS, vol. 7664, pp. 516–525. Springer, Heidelberg (2012). doi:10.1007/978-3-642-34481-7_63
Yuen, M.-C., King, I., Leung, K.-S.: Probabilistic matrix factorization with active learning for quality assurance in crowdsourcing systems. In: Proceedings of the IADIS International Conference WWW/Internet 2015, Ireland (2015)
Yuen, M.-C., King, I., Leung, K.-S.: TaskRec: a task recommendation framework in crowdsourcing systems. Neural Process. Lett. 41(2), 223–238 (2015)
Acknowledgment
This research was in part supported by grants from the National Grand Fundamental Research 973 Program of China (No. 2014CB340405), the Research Grants Council of the Hong Kong Special Administrative Region, China (Project No. CUHK 14203314), and Microsoft Research Asia Regional Seed Fund in Big Data Research (Grant No. FY13-RES-SPONSOR-036).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing AG
About this paper
Cite this paper
Yuen, MC., King, I., Leung, KS. (2016). An Online-Updating Approach on Task Recommendation in Crowdsourcing Systems. In: Hirose, A., Ozawa, S., Doya, K., Ikeda, K., Lee, M., Liu, D. (eds) Neural Information Processing. ICONIP 2016. Lecture Notes in Computer Science(), vol 9947. Springer, Cham. https://doi.org/10.1007/978-3-319-46687-3_10
Download citation
DOI: https://doi.org/10.1007/978-3-319-46687-3_10
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-46686-6
Online ISBN: 978-3-319-46687-3
eBook Packages: Computer ScienceComputer Science (R0)