Abstract
Learning how to refer effectively in an expert-referral network is an emerging challenge at the intersection of Active Learning and Multi-Agent Reinforcement Learning. Distributed interval estimation learning (DIEL) was previously found to be promising for learning appropriate referral choices, compared to greedy and Q-learning methods. This paper extends these results in several directions: First, learning methods with several multi-armed bandit (MAB) algorithms are compared along with greedy variants, each optimized individually. Second, DIEL’s rapid performance gain in the early phase of learning proved equally convincing in the case of multi-hop referral, a condition not heretofore explored. Third, a robustness analysis across the learning algorithms, with an emphasis on capacity constraints and evolving networks (experts dropping out and new experts of unknown performance entering) shows rapid recovery. Fourth, the referral paradigm is successfully extended to teams of Stochastic Local Search (SLS) SAT solvers with different capabilities.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
KhudaBukhsh, A.R., Jansen, P.J., Carbonell, J.G.: Distributed learning in expert referral networks. In: European Conference on Artificial Intelligence (ECAI), vol. 2016, pp. 1620–1621 (2016)
KhudaBukhsh, A.R., Carbonell, J.G., Jansen, P.J.: Proactive skill posting in referral networks. In: Kang, B.H., Bai, Q. (eds.) AI 2016. LNCS, vol. 9992, pp. 585–596. Springer, Cham (2016). doi:10.1007/978-3-319-50127-7_52
Agrawal, R.: Sample mean based index policies with o (log n) regret for the multi-armed bandit problem. Adv. Appl. Probab. 27(4), 1054–1078 (1995)
Lai, T.L., Robbins, H.: Asymptotically efficient adaptive allocation rules. Adv. Appl. Math. 6(1), 4–22 (1985)
Audibert, J.-Y., Munos, R., Szepesvári, C.: Tuning bandit algorithms in stochastic environments. In: Hutter, M., Servedio, R.A., Takimoto, E. (eds.) ALT 2007. LNCS, vol. 4754, pp. 150–165. Springer, Heidelberg (2007). doi:10.1007/978-3-540-75225-7_15
Abdallah, S., Lesser, V.: Learning the task allocation game. In: Proceedings of AAMAS 2006, pp. 850–857. ACM (2006)
Zhang, C., Lesser, V., Shenoy, P.: A multi-agent learning approach to online distributed resource allocation. In: Proceedings of IJCAI 2009, Pasadena, CA, vol. 1, pp. 361–366 (2009)
KhudaBukhsh, A.R., Xu, L., Hoos, H.H., Leyton-Brown, K.: SATenstein: automatically building local search SAT solvers from components. Artif. Intell. 232, 20–42 (2016)
KhudaBukhsh, A.R., Carbonell, J.G., Jansen, P.J.: Proactive-DIEL in evolving referral networks. In: European Conference on Multi-Agent Systems, Springer, Heidelberg (2016)
Donmez, P., Carbonell, J.G., Bennett, P.N.: Dual strategy active learning. In: Kok, J.N., Koronacki, J., Mantaras, R.L., Matwin, S., Mladenič, D., Skowron, A. (eds.) ECML 2007. LNCS (LNAI), vol. 4701, pp. 116–127. Springer, Heidelberg (2007). doi:10.1007/978-3-540-74958-5_14
Watkins, C.J., Dayan, P.: Q-learning. Mach. Learn. 8(3–4), 279–292 (1992)
Hasselt, H.V.: Double Q-learning. In: Advances in Neural Information Processing Systems, pp. 2613–2621 (2010)
Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Mach. Learn. 47(2–3), 235–256 (2002)
Donmez, P., Carbonell, J.G.: Proactive learning: cost-sensitive active learning with multiple imperfect oracles. In: Proceedings of CIKM 2008, vol. 08, pp. 619–628 (2008)
Acknowledgements
This research is partially funded by the National Science Foundation grant EAGER-1649225.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
KhudaBukhsh, A.R., Carbonell, J.G., Jansen, P.J. (2017). Robust Learning in Expert Networks: A Comparative Analysis. In: Kryszkiewicz, M., Appice, A., Ślęzak, D., Rybinski, H., Skowron, A., Raś, Z. (eds) Foundations of Intelligent Systems. ISMIS 2017. Lecture Notes in Computer Science(), vol 10352. Springer, Cham. https://doi.org/10.1007/978-3-319-60438-1_29
Download citation
DOI: https://doi.org/10.1007/978-3-319-60438-1_29
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-60437-4
Online ISBN: 978-3-319-60438-1
eBook Packages: Computer ScienceComputer Science (R0)