Journal of Intelligent & Robotic Systems

, Volume 84, Issue 1–4, pp 371–386 | Cite as

Cooperative Decision-Making Under Uncertainties for Multi-Target Surveillance with Multiples UAVs



Surveillance is an interesting application for Unmanned Aerial Vehicles (UAVs). If a team of UAVs is considered, the objective is usually to act cooperatively to gather as much information as possible from a set of moving targets in the surveillance area. This is a decision-making problem with severe uncertainties involved: relying on imperfect sensors and models, UAVs need to select targets to monitor and determine the best actions to track them. Partially Observable Markov Decision Processes (POMDPs) are quite adequate for optimal decision-making under uncertainties, but they lack scalability in multi-UAV scenarios, becoming tractable only for toy problems. In this paper, we take a step forward to apply POMDP methods in real situations, where the team needs to adapt to the circumstances during the mission and foster cooperation among the team-members. We propose to split the original problem into simpler behaviors that can be modeled by scalable POMDPs. Then, those behaviors are auctioned during the mission among the UAVs, which follow different policies depending on the behavior assigned. We evaluate the performance of our approach with extensive simulations and propose an implementation with real quadcopters in a testbed scenario.


Planning under uncertainty Multi-target surveillance Multi-robot cooperation 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Anderson, R., Milutinovic, D.: Anticipating stochastic observation loss during optimal target tracking by a small aerial vehicle. In: 2013 International Conference on Unmanned Aircraft Systems (ICUAS), pp 278–287. IEEE (2013)Google Scholar
  2. 2.
    Bar-Shalom, Y., Li, X., Kirubarajan, T.: Estimation with Applications to Tracking and Navigation. Wiley (2001)Google Scholar
  3. 3.
    Beard, R., McLain, T., Nelson, D., Kingston, D., Johanson, D.: Decentralized cooperative aerial surveillance using fixed-wing miniature uavs. Proc. IEEE 94(7), 1306—1324 (2006)CrossRefGoogle Scholar
  4. 4.
    Bernstein, D.S., Givan, R., Immerman, N., Zilberstein, S.: The complexity of decentralized control of Markov decision processes. Math. Oper. Res. 27(4), 819—840 (2002)MathSciNetCrossRefMATHGoogle Scholar
  5. 5.
    Bourgault, F., Furukawa, T., Durrant-Whyte, H.: Decentralized bayesian negotiation for cooperative search. In: Proceedings of International Conference on Intelligent Robots and Systems, pp 2681–2686 (2004)Google Scholar
  6. 6.
    Burdakov, O., Doherty, P., Holmberg, K., Kvarnstrom, J., Olson, P.M.: Relay positioning for unmanned aerial vehicle surveillance. Int. J. Robot. Res. 29(8), 1069—1087 (2010)CrossRefGoogle Scholar
  7. 7.
    Capitan, J., Merino, L., Caballero, F., Ollero, A.: Decentralized delayed-state information filter (DDSIF): A new approach for cooperative decentralized tracking. Robot. Auton. Syst. 59, 376–388 (2011). doi: 10.1016/j.robot.2011.02.001 article/B6V16-526DWNX-1/2/dd823dbe262732b7d71fecdeb8cd1354 CrossRefGoogle Scholar
  8. 8.
    Capitan, J., Spaan, M., Merino, L., Ollero, A.: Decentralized multi-robot cooperation with auctioned POMDPs. Int. J. Robot. Res. 32(6), 650–671 (2013). doi: 10.1177/0278364913483345 CrossRefGoogle Scholar
  9. 9.
    Cook, K., Bryan, E., Yu, H., Bai, H., Seppi, K., Beard, R.: Intelligent cooperative control for urban tracking with unmanned air vehicles. In: 2013 International Conference on Unmanned Aircraft Systems (ICUAS), pp 1–7. IEEE (2013)Google Scholar
  10. 10.
    Grocholsky, B., Makarenko, A., Kaupp, T., Durrant-Whyte, H. In: F. Zhao, L. Guibas (eds.) : Scalable control of decentralised sensor platforms, vol. 2634, pp 551–551. Springer, Heidelberg (2003)Google Scholar
  11. 11.
    He, R., Bachrach, A., Roy, N.: Efficient Planning under Uncertainty for a Target-Tracking Micro-Aerial Vehicle. In: IEEE International Conference on Robotics and Automation (2010)Google Scholar
  12. 12.
    Hsieh, M.A., Cowley, A., Keller, J.F., Chaimowicz, L., andCamillo, J., Taylor, B.G.V.K., End, Y., Arkin, R.C., Jung, B., Wolf, D.F., Sukhatme, G.S., MacKenzie, D.C.: Adaptive teams of autonomous aerial and ground robots for situational awareness. J. Field Rob. 24, 991—1014 (2007)CrossRefGoogle Scholar
  13. 13.
    Hsu, D., Lee, W., Rong, N.: A point-based POMDP planner for target tracking. In: Proceedings of the IEEE International Conference on Robotics and Automation, pp 2644–2650 (2008)Google Scholar
  14. 14.
    Kaelbling, L.P., Littman, M.L., Cassandra, A.R.: Planning and acting in partially observable stochastic domains. Artif. Intell. 101, 99—134 (1998)MathSciNetCrossRefMATHGoogle Scholar
  15. 15.
    Matignon, L., Jeanpierre, L., Mouaddib, A.: Distributed Value Functions for Multi-Robot Exploration. In: Proceedings of the IEEE International Conference on Robotics and Automation, pp 1544–1550. St. Paul, USA (2012)Google Scholar
  16. 16.
    Morbidi, F., Mariottini, G.L.: On active target tracking and cooperative localization for multiple aerial vehicles. In: 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp 2229–2234. IEEE (2011)Google Scholar
  17. 17.
    Nair, R., Varakantham, P., Tambe, M., Yokoo, M.: Networked distributed POMDPs: A synthesis of distributed constraint optimization and POMDPs. In: Proceedings of the National Conference on Artificial Intelligence, pp 133–139 (2005)Google Scholar
  18. 18.
    Ong, L., Upcroft, B., Bailey, T., Ridley, M., Sukkarieh, S., Durrant-Whyte, H.: A decentralised particle filtering algorithm for multi-target tracking across multiple flight vehicles. In: IROS, pp 4539–4544 (2006)Google Scholar
  19. 19.
    Ong, S.C., Png, S.W., Hsu, D., Lee, W.S.: POMDPs for Robotic Tasks with Mixed Observability. In: Proceedings of the Robotics: Science and Systems Conference. Seattle, USA (2009)Google Scholar
  20. 20.
    Papadimitriou, C.H., Tsitsiklis, J.N.: The complexity of markov decision processes. Math. Oper. Res. 12(3), 441—450 (1987)MathSciNetCrossRefMATHGoogle Scholar
  21. 21.
    Poupart, P.: Exploiting Structure to Efficiently Solve Large Scale Partially Observable Markov Decision Processes. Ph.D. thesis. University of Toronto (2005)Google Scholar
  22. 22.
    Pynadath, D.V., Tambe, M.: The communicative multiagent team decision problem: Analyzing teamwork theories and models. J. Artif. Intell. Res. 16, 389—423 (2002)MathSciNetMATHGoogle Scholar
  23. 23.
    Skoglar, P., Orguner, U., Gustafsson, F.: On information measures based on particle mixture for optimal bearings-only tracking. In: 2009 IEEE Aerospace conference, pp 1–14. IEEE (2009)Google Scholar
  24. 24.
    Wong, E.M., Bourgault, F., Furukawa, T.: Multi-vehicle Bayesian search for multiple lost targets. In: Proceedings of the IEEE International Conference on Robotics and Automation, pp 3169–3174 (2005)Google Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2015

Authors and Affiliations

  1. 1.University of SevilleSevilleSpain
  2. 2.Pablo de Olavide UniversitySevilleSpain

Personalised recommendations