Learning Complex Concepts Using Crowdsourcing: A Bayesian Approach
We develop a Bayesian approach to concept learning for crowdsourcing applications. A probabilistic belief over possible concept definitions is maintained and updated according to (noisy) observations from experts, whose behaviors are modeled using discrete types. We propose recommendation techniques, inference methods, and query selection strategies to assist a user charged with choosing a configuration that satisfies some (partially known) concept. Our model is able to simultaneously learn the concept definition and the types of the experts. We evaluate our model with simulations, showing that our Bayesian strategies are effective even in large concept spaces with many uninformative experts.
KeywordsBayesian Approach Recommender System Latent Concept Current Belief True Concept
Unable to display preview. Download preview PDF.
- 2.Boutilier, C., Regan, K., Viappiani, P.: Online feature elicitation in interactive optimization. In: Proceedings of the Twenty-sixth International Conference on Machine Learning (ICML 2009), Montreal, pp. 73–80 (2009)Google Scholar
- 3.Boutilier, C., Regan, K., Viappiani, P.: Simultaneous elicitation of preference features and utility. In: Proceedings of the Twenty-fourth AAAI Conference on Artificial Intelligence (AAAI 2010), Atlanta, pp. 1160–1167 (2010)Google Scholar
- 4.Chen, S., Zhang, J., Chen, G., Zhang, C.: What if the irresponsible teachers are dominating? In: Proceedings of the Twenty-Fourth AAAI Conference on Artificial Intelligence (AAAI 2010), Atlanta, pp. 419–424 (2010)Google Scholar
- 5.Dai, P., Mausam, Weld, D.S.: Decision-theoretic control of crowd-sourced workflows. In: Proceedings of the Twenty-Fourth AAAI Conference on Artificial Intelligence (AAAI 2010), Atlanta, pp. 1168–1174 (2010)Google Scholar
- 6.Haussler, D.: Learning conjunctive concepts in structural domains. Machine Learning 4, 7–40 (1989)Google Scholar
- 10.Mitchell, T.M.: Version spaces: A candidate elimination approach to rule learning. In: Proceedings of the Fifth International Joint Conference on Artificial Intelligence (IJCAI 1977), Cambridge, pp. 305–310 (1977)Google Scholar
- 11.Shahaf, D., Horvitz, E.: Generalized task markets for human and machine computation. In: Proceedings of the Twenty-Fourth AAAI Conference on Artificial Intelligence (AAAI 2010), Atlanta, pp. 986–993 (2010)Google Scholar
- 12.Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)Google Scholar
- 13.Viappiani, P., Boutilier, C.: Optimal Bayesian recommendation sets and myopically optimal choice query sets. In: Advances in Neural Information Processing Systems (NIPS), Vancouver, vol. 23, pp. 2352–2360 (2010)Google Scholar