Abstract
The multi-robot adaptive sampling problem aims at finding trajectories for a team of robots to efficiently sample the phenomenon of interest within a given endurance budget of the robots. In this paper, we propose a robust and scalable approach using Multi-Agent Reinforcement Learning for cooperated Adaptive Sampling (MARLAS) of quasi-static environmental processes. Given a prior on the field being sampled, the proposed method learns decentralized policies for a team of robots to sample high-utility regions within a fixed budget. The multi-robot adaptive sampling problem requires the robots to coordinate with each other to avoid overlapping sampling trajectories. Therefore, we encode the estimates of neighbor positions and intermittent communication between robots into the learning process. We evaluated MARLAS over multiple performance metrics and found it to outperform other baseline multi-robot sampling techniques. Additionally, we demonstrate scalability with both the size of the robot team and the size of the region being sampled. We further demonstrate robustness to communication failures and robot failures. The experimental evaluations are conducted both in simulations on real data and in real robot experiments on demo environmental setup (The demo video can be accessed at: https://youtu.be/qRRpNC60KL4).
L. Pan and S. Manjanna—Equal contribution.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Low, K.H., Dolan, J.M., Khosla, P.: Adaptive multi-robot wide-area exploration and mapping. In: Proceedings of the 7th international joint conference on Autonomous agents and Multiagent Systems Volume 1. International Foundation for Autonomous Agents and Multiagent Systems, pp. 23–30 (2008)
Sadat, S.A., Wawerla, J., Vaughan, R.: Fractal trajectories for online non-uniform aerial coverage. In: 2015 IEEE International Conference on Robotics and Automation (ICRA), pp. 2971–2976. IEEE (2015)
Manjanna, S., Van Hoof, H., Dudek, G.: Policy search on aggregated state space for active sampling. In: Xiao, J., Kröger, T., Khatib, O. (eds.) ISER 2018. SPAR, vol. 11, pp. 211–221. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-33950-0_19
Almadhoun, R., Taha, T., Seneviratne, L., Zweiri, Y.: A survey on multi-robot coverage path planning for model reconstruction and mapping. SN Appl. Sci. 1(8), 1–24 (2019)
Salam, T., Hsieh, M.A.: Adaptive sampling and reduced-order modeling of dynamic processes by robot teams. IEEE Robot. Autom. Lett. 4(2), 477–484 (2019)
Venkataramani, R., Bresler, Y.: Perfect reconstruction formulas and bounds on aliasing error in sub-nyquist nonuniform sampling of multiband signals. IEEE Trans. Inf. Theory 46(6), 2173–2183 (2000)
Rahimi, M., Hansen, M., Kaiser, W.J., Sukhatme, G.S., Estrin, D.: Adaptive sampling for environmental field estimation using robotic sensors. In: 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 3692–3698. IEEE (2005)
Cortes, J., Martinez, S., Karatas, T., Bullo, F.: Coverage control for mobile sensing networks. IEEE Trans. Robot. Autom. 20(2), 243–255 (2004)
Durham, J.W., Carli, R., Frasca, P., Bullo, F.: Discrete partitioning and coverage control for gossiping robots. IEEE Trans. Rob. 28(2), 364–378 (2011)
Aurenhammer, F.: Voronoi diagrams-a survey of a fundamental geometric data structure. ACM Comput. Surv. (CSUR) 23(3), 345–405 (1991)
Breitenmoser, A., Schwager, M., Metzger, J.-C., Siegwart, R., Rus, D.: Voronoi coverage of non-convex environments with a group of networked robots. In: 2010 IEEE International Conference on Robotics and Automation, pp. 4982–4989. IEEE (2010)
Mishra, M., Poddar, P., Chen, J., Tokekar, P., Sujit, P.: Galopp: multi-agent deep reinforcement learning for persistent monitoring with localization constraints, arXiv preprint arXiv:2109.06831 (2021)
Chen, J., Baskaran, A., Zhang, Z., Tokekar, P.: Multi-agent reinforcement learning for visibility-based persistent monitoring. In: 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 2563–2570. IEEE (2021)
Kantaros, Y., Schlotfeldt, B., Atanasov, N., Pappas, G.J.: Sampling-based planning for non-myopic multi-robot information gathering. Auton. Robot. 45(7), 1029–1046 (2021)
Kapoutsis, A.C., Chatzichristofis, S.A., Kosmatopoulos, E.B.: DARP: divide areas algorithm for optimal multi-robot coverage path planning. J. Intell. Robot. Syst. 86(3), 663–680 (2017)
Hsu, C.D., Jeong, H., Pappas, G.J., Chaudhari, P.: Scalable reinforcement learning policies for multi-agent control. In: 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 4785–4791. IEEE (2021)
Dibangoye, J.S., Amato, C., Buffet, O., Charpillet, F.: Optimally solving Dec-POMDPs as continuous-state MDPs. J. Artif. Intell. Rese. 55, 443–497 (2016)
Khatib, O.: Real-time obstacle avoidance for manipulators and mobile robots. In: Cox, I.J., Wilfong, G.T. (eds.) Autonomous Robot Vehicles. Springer, New York, NY (1986). https://doi.org/10.1007/978-1-4613-8997-2_29
Manjanna, S., Hsieh, M.A., Dudek, G.: Scalable multi-robot system for non-myopic spatial sampling, arXiv preprint arXiv:2105.10018 (2021)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization, arXiv preprint arXiv:1412.6980 (2014)
Dai, C., Zhao, M.: Nonlinear analysis in a nutrient-algae-zooplankton system with sinking of algae. In: Abstract and Applied Analysis, vol. 2014. Hindawi (2014)
Meghjani, M., Manjanna, S., Dudek, G.: Multi-target rendezvous search. In: 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 2596–2603. IEEE (2016)
Acknowledgements
We gratefully acknowledge the support of NSF IIS 1812319 and ARL DCIST CRA W911NF-17-2-0181.
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Pan, L., Manjanna, S., Hsieh, M.A. (2024). MARLAS: Multi Agent Reinforcement Learning for Cooperated Adaptive Sampling. In: Bourgeois, J., et al. Distributed Autonomous Robotic Systems. DARS 2022. Springer Proceedings in Advanced Robotics, vol 28. Springer, Cham. https://doi.org/10.1007/978-3-031-51497-5_25
Download citation
DOI: https://doi.org/10.1007/978-3-031-51497-5_25
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-51496-8
Online ISBN: 978-3-031-51497-5
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)