Abstract
Surveillance is an interesting application for Unmanned Aerial Vehicles (UAVs). If a team of UAVs is considered, the objective is usually to act cooperatively to gather as much information as possible from a set of moving targets in the surveillance area. This is a decision-making problem with severe uncertainties involved: relying on imperfect sensors and models, UAVs need to select targets to monitor and determine the best actions to track them. Partially Observable Markov Decision Processes (POMDPs) are quite adequate for optimal decision-making under uncertainties, but they lack scalability in multi-UAV scenarios, becoming tractable only for toy problems. In this paper, we take a step forward to apply POMDP methods in real situations, where the team needs to adapt to the circumstances during the mission and foster cooperation among the team-members. We propose to split the original problem into simpler behaviors that can be modeled by scalable POMDPs. Then, those behaviors are auctioned during the mission among the UAVs, which follow different policies depending on the behavior assigned. We evaluate the performance of our approach with extensive simulations and propose an implementation with real quadcopters in a testbed scenario.
Similar content being viewed by others
References
Anderson, R., Milutinovic, D.: Anticipating stochastic observation loss during optimal target tracking by a small aerial vehicle. In: 2013 International Conference on Unmanned Aircraft Systems (ICUAS), pp 278–287. IEEE (2013)
Bar-Shalom, Y., Li, X., Kirubarajan, T.: Estimation with Applications to Tracking and Navigation. Wiley (2001)
Beard, R., McLain, T., Nelson, D., Kingston, D., Johanson, D.: Decentralized cooperative aerial surveillance using fixed-wing miniature uavs. Proc. IEEE 94(7), 1306—1324 (2006)
Bernstein, D.S., Givan, R., Immerman, N., Zilberstein, S.: The complexity of decentralized control of Markov decision processes. Math. Oper. Res. 27(4), 819—840 (2002)
Bourgault, F., Furukawa, T., Durrant-Whyte, H.: Decentralized bayesian negotiation for cooperative search. In: Proceedings of International Conference on Intelligent Robots and Systems, pp 2681–2686 (2004)
Burdakov, O., Doherty, P., Holmberg, K., Kvarnstrom, J., Olson, P.M.: Relay positioning for unmanned aerial vehicle surveillance. Int. J. Robot. Res. 29(8), 1069—1087 (2010)
Capitan, J., Merino, L., Caballero, F., Ollero, A.: Decentralized delayed-state information filter (DDSIF): A new approach for cooperative decentralized tracking. Robot. Auton. Syst. 59, 376–388 (2011). doi:10.1016/j.robot.2011.02.001 http://www.sciencedirect.com/science/ article/B6V16-526DWNX-1/2/dd823dbe262732b7d71fecdeb8cd1354
Capitan, J., Spaan, M., Merino, L., Ollero, A.: Decentralized multi-robot cooperation with auctioned POMDPs. Int. J. Robot. Res. 32(6), 650–671 (2013). doi:10.1177/0278364913483345
Cook, K., Bryan, E., Yu, H., Bai, H., Seppi, K., Beard, R.: Intelligent cooperative control for urban tracking with unmanned air vehicles. In: 2013 International Conference on Unmanned Aircraft Systems (ICUAS), pp 1–7. IEEE (2013)
Grocholsky, B., Makarenko, A., Kaupp, T., Durrant-Whyte, H. In: F. Zhao, L. Guibas (eds.) : Scalable control of decentralised sensor platforms, vol. 2634, pp 551–551. Springer, Heidelberg (2003)
He, R., Bachrach, A., Roy, N.: Efficient Planning under Uncertainty for a Target-Tracking Micro-Aerial Vehicle. In: IEEE International Conference on Robotics and Automation (2010)
Hsieh, M.A., Cowley, A., Keller, J.F., Chaimowicz, L., andCamillo, J., Taylor, B.G.V.K., End, Y., Arkin, R.C., Jung, B., Wolf, D.F., Sukhatme, G.S., MacKenzie, D.C.: Adaptive teams of autonomous aerial and ground robots for situational awareness. J. Field Rob. 24, 991—1014 (2007)
Hsu, D., Lee, W., Rong, N.: A point-based POMDP planner for target tracking. In: Proceedings of the IEEE International Conference on Robotics and Automation, pp 2644–2650 (2008)
Kaelbling, L.P., Littman, M.L., Cassandra, A.R.: Planning and acting in partially observable stochastic domains. Artif. Intell. 101, 99—134 (1998)
Matignon, L., Jeanpierre, L., Mouaddib, A.: Distributed Value Functions for Multi-Robot Exploration. In: Proceedings of the IEEE International Conference on Robotics and Automation, pp 1544–1550. St. Paul, USA (2012)
Morbidi, F., Mariottini, G.L.: On active target tracking and cooperative localization for multiple aerial vehicles. In: 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp 2229–2234. IEEE (2011)
Nair, R., Varakantham, P., Tambe, M., Yokoo, M.: Networked distributed POMDPs: A synthesis of distributed constraint optimization and POMDPs. In: Proceedings of the National Conference on Artificial Intelligence, pp 133–139 (2005)
Ong, L., Upcroft, B., Bailey, T., Ridley, M., Sukkarieh, S., Durrant-Whyte, H.: A decentralised particle filtering algorithm for multi-target tracking across multiple flight vehicles. In: IROS, pp 4539–4544 (2006)
Ong, S.C., Png, S.W., Hsu, D., Lee, W.S.: POMDPs for Robotic Tasks with Mixed Observability. In: Proceedings of the Robotics: Science and Systems Conference. Seattle, USA (2009)
Papadimitriou, C.H., Tsitsiklis, J.N.: The complexity of markov decision processes. Math. Oper. Res. 12(3), 441—450 (1987)
Poupart, P.: Exploiting Structure to Efficiently Solve Large Scale Partially Observable Markov Decision Processes. Ph.D. thesis. University of Toronto (2005)
Pynadath, D.V., Tambe, M.: The communicative multiagent team decision problem: Analyzing teamwork theories and models. J. Artif. Intell. Res. 16, 389—423 (2002)
Skoglar, P., Orguner, U., Gustafsson, F.: On information measures based on particle mixture for optimal bearings-only tracking. In: 2009 IEEE Aerospace conference, pp 1–14. IEEE (2009)
Wong, E.M., Bourgault, F., Furukawa, T.: Multi-vehicle Bayesian search for multiple lost targets. In: Proceedings of the IEEE International Conference on Robotics and Automation, pp 3169–3174 (2005)
Author information
Authors and Affiliations
Corresponding author
Additional information
This work has been supported by the FP7 EC-SAFEMOBIL Project (grant number 288082) and the PAIS-MultiRobot (TIC-7390) Regional Project.
Rights and permissions
About this article
Cite this article
Capitan, J., Merino, L. & Ollero, A. Cooperative Decision-Making Under Uncertainties for Multi-Target Surveillance with Multiples UAVs. J Intell Robot Syst 84, 371–386 (2016). https://doi.org/10.1007/s10846-015-0269-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10846-015-0269-0