Abstract
We present the Bayesian Online Prediction for Ad hoc teamwork (BOPA), a novel algorithm for ad hoc teamwork which enables a robot to collaborate, on the fly, with human teammates without any pre-coordination protocol. Unlike previous works, BOPA relies only on state observations/transitions of the environment in order to identify the task being performed by a given teammate (without observing the teammate’s actions and environment’s reward signals). We evaluate BOPA in two distinct settings, namely (i) an empirical evaluation in a simulated environment with three different types of teammates, and (ii) an experimental evaluation in a real-world environment, deploying BOPA into an ad hoc robot with the goal of assisting a human teammate in completing a given task. Our results show that BOPA is effective at correctly identifying the target task, efficient at solving the correct task in optimal and near-optimal times, scalable by adapting to different problem sizes, and robust to non-optimal teammates, such as humans.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Hoeffding’s lemma states that, given a real-valued random variable X such that \(a\le X\le b\) almost surely and any
,$$\begin{aligned} \mathbb {E}_{}\left[ e^{\lambda X}\right] \le \exp \left( \lambda \mathbb {E}_{}\left[ X\right] +\frac{\lambda ^2(b-a)^2}{8}\right) . \end{aligned}$$.
- 2.
In the PB environment, different configurations correspond to different positions for the buttons; in the ER environment, different configurations correspond to different uncharted locations in the map.
,References
Banerjee, A.: On Bayesian bounds. In: Proceedings of 23rd International Conference Machine Learning, pp. 81–88 (2006)
Barret, S., Stone, P.: An analysis framework for ad hoc teamwork tasks. In: Proceedings of 11th International Conference Autonomous Agents and Multiagent Systems, pp. 357–364 (2012)
Barrett, S.: Making Friends on the Fly: Advances in Ad Hoc Teamwork. Ph.D. thesis, The University of Texas at Austin (2014)
Barrett, S., Stone, P.: Cooperating with unknown teammates in complex domains: a robot soccer case study of ad hoc teamwork. In: Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence, pp. 2010–2016, January 2015
Brockman, G., et al.: Openai gym. arXiv preprint arXiv:1606.01540 (2016)
Christiano, P.F., Leike, J., Brown, T.B., Martic, M., Legg, S., Amodei, D.: Deep reinforcement learning from human preferences. In: Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS 2017, pp. 4302–4310. Curran Associates Inc., Red Hook (2017)
Genter, K., Laue, T., Stone, P.: Three years of the RoboCup standard platform league drop-in player competition. Auton. Agent. Multi-Agent Syst. 31(4), 790–820 (2016). https://doi.org/10.1007/s10458-016-9353-5
Melo, F.S., Sardinha, A.: Ad hoc teamwork by learning teammates’ task. Auton. Agent. Multi-Agent Syst. 30(2), 175–219 (2016)
Pepito, J.A., Locsin, R.: Can nurses remain relevant in a technologically advanced future? Int. J. Nursing Sci. 6(1), 106–110 (2019). https://doi.org/10.1016/j.ijnss.2018.09.013. http://www.sciencedirect.com/science/article/pii/S2352013218301765
Stone, P., Kaminka, G.A., Kraus, S., Rosenschein, J.S.: Ad hoc autonomous agent teams: collaboration without pre-coordination. In: Twenty-Fourth AAAI Conference on Artificial Intelligence (2010)
Tasaki, R., Kitazaki, M., Miura, J., Terashima, K.: Prototype design of medical round supporting robot “terapio”. In: 2015 IEEE International Conference on Robotics and Automation (ICRA), pp. 829–834, May 2015. https://doi.org/10.1109/ICRA.2015.7139274
Acknowledgements
This work was partially supported by national funds through FCT, Fundação para a Ciência e a Tecnologia, under project UIDB/50021/2020 (INESC-ID multi-annual funding) and the HOTSPOT project, with reference PTDC/CCI-COM/7203/2020. In addition, this material is based upon work supported by the Air Force Office of Scientific Research under award number FA9550-19-1-0020, and by TAILOR, a project funded by EU Horizon 2020 research and innovation programme under GA No 952215. The first author acknowledges the PhD grant 2020.05151.BD from FCT.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Ribeiro, J.G., Faria, M., Sardinha, A., Melo, F.S. (2021). Helping People on the Fly: Ad Hoc Teamwork for Human-Robot Teams. In: Marreiros, G., Melo, F.S., Lau, N., Lopes Cardoso, H., Reis, L.P. (eds) Progress in Artificial Intelligence. EPIA 2021. Lecture Notes in Computer Science(), vol 12981. Springer, Cham. https://doi.org/10.1007/978-3-030-86230-5_50
Download citation
DOI: https://doi.org/10.1007/978-3-030-86230-5_50
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-86229-9
Online ISBN: 978-3-030-86230-5
eBook Packages: Computer ScienceComputer Science (R0)