Skip to main content
Log in

Cooperative Decision-Making Under Uncertainties for Multi-Target Surveillance with Multiples UAVs

  • Published:
Journal of Intelligent & Robotic Systems Aims and scope Submit manuscript

Abstract

Surveillance is an interesting application for Unmanned Aerial Vehicles (UAVs). If a team of UAVs is considered, the objective is usually to act cooperatively to gather as much information as possible from a set of moving targets in the surveillance area. This is a decision-making problem with severe uncertainties involved: relying on imperfect sensors and models, UAVs need to select targets to monitor and determine the best actions to track them. Partially Observable Markov Decision Processes (POMDPs) are quite adequate for optimal decision-making under uncertainties, but they lack scalability in multi-UAV scenarios, becoming tractable only for toy problems. In this paper, we take a step forward to apply POMDP methods in real situations, where the team needs to adapt to the circumstances during the mission and foster cooperation among the team-members. We propose to split the original problem into simpler behaviors that can be modeled by scalable POMDPs. Then, those behaviors are auctioned during the mission among the UAVs, which follow different policies depending on the behavior assigned. We evaluate the performance of our approach with extensive simulations and propose an implementation with real quadcopters in a testbed scenario.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Anderson, R., Milutinovic, D.: Anticipating stochastic observation loss during optimal target tracking by a small aerial vehicle. In: 2013 International Conference on Unmanned Aircraft Systems (ICUAS), pp 278–287. IEEE (2013)

  2. Bar-Shalom, Y., Li, X., Kirubarajan, T.: Estimation with Applications to Tracking and Navigation. Wiley (2001)

  3. Beard, R., McLain, T., Nelson, D., Kingston, D., Johanson, D.: Decentralized cooperative aerial surveillance using fixed-wing miniature uavs. Proc. IEEE 94(7), 1306—1324 (2006)

    Article  Google Scholar 

  4. Bernstein, D.S., Givan, R., Immerman, N., Zilberstein, S.: The complexity of decentralized control of Markov decision processes. Math. Oper. Res. 27(4), 819—840 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  5. Bourgault, F., Furukawa, T., Durrant-Whyte, H.: Decentralized bayesian negotiation for cooperative search. In: Proceedings of International Conference on Intelligent Robots and Systems, pp 2681–2686 (2004)

  6. Burdakov, O., Doherty, P., Holmberg, K., Kvarnstrom, J., Olson, P.M.: Relay positioning for unmanned aerial vehicle surveillance. Int. J. Robot. Res. 29(8), 1069—1087 (2010)

    Article  Google Scholar 

  7. Capitan, J., Merino, L., Caballero, F., Ollero, A.: Decentralized delayed-state information filter (DDSIF): A new approach for cooperative decentralized tracking. Robot. Auton. Syst. 59, 376–388 (2011). doi:10.1016/j.robot.2011.02.001 http://www.sciencedirect.com/science/ article/B6V16-526DWNX-1/2/dd823dbe262732b7d71fecdeb8cd1354

    Article  Google Scholar 

  8. Capitan, J., Spaan, M., Merino, L., Ollero, A.: Decentralized multi-robot cooperation with auctioned POMDPs. Int. J. Robot. Res. 32(6), 650–671 (2013). doi:10.1177/0278364913483345

    Article  Google Scholar 

  9. Cook, K., Bryan, E., Yu, H., Bai, H., Seppi, K., Beard, R.: Intelligent cooperative control for urban tracking with unmanned air vehicles. In: 2013 International Conference on Unmanned Aircraft Systems (ICUAS), pp 1–7. IEEE (2013)

  10. Grocholsky, B., Makarenko, A., Kaupp, T., Durrant-Whyte, H. In: F. Zhao, L. Guibas (eds.) : Scalable control of decentralised sensor platforms, vol. 2634, pp 551–551. Springer, Heidelberg (2003)

  11. He, R., Bachrach, A., Roy, N.: Efficient Planning under Uncertainty for a Target-Tracking Micro-Aerial Vehicle. In: IEEE International Conference on Robotics and Automation (2010)

  12. Hsieh, M.A., Cowley, A., Keller, J.F., Chaimowicz, L., andCamillo, J., Taylor, B.G.V.K., End, Y., Arkin, R.C., Jung, B., Wolf, D.F., Sukhatme, G.S., MacKenzie, D.C.: Adaptive teams of autonomous aerial and ground robots for situational awareness. J. Field Rob. 24, 991—1014 (2007)

    Article  Google Scholar 

  13. Hsu, D., Lee, W., Rong, N.: A point-based POMDP planner for target tracking. In: Proceedings of the IEEE International Conference on Robotics and Automation, pp 2644–2650 (2008)

  14. Kaelbling, L.P., Littman, M.L., Cassandra, A.R.: Planning and acting in partially observable stochastic domains. Artif. Intell. 101, 99—134 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  15. Matignon, L., Jeanpierre, L., Mouaddib, A.: Distributed Value Functions for Multi-Robot Exploration. In: Proceedings of the IEEE International Conference on Robotics and Automation, pp 1544–1550. St. Paul, USA (2012)

  16. Morbidi, F., Mariottini, G.L.: On active target tracking and cooperative localization for multiple aerial vehicles. In: 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp 2229–2234. IEEE (2011)

  17. Nair, R., Varakantham, P., Tambe, M., Yokoo, M.: Networked distributed POMDPs: A synthesis of distributed constraint optimization and POMDPs. In: Proceedings of the National Conference on Artificial Intelligence, pp 133–139 (2005)

  18. Ong, L., Upcroft, B., Bailey, T., Ridley, M., Sukkarieh, S., Durrant-Whyte, H.: A decentralised particle filtering algorithm for multi-target tracking across multiple flight vehicles. In: IROS, pp 4539–4544 (2006)

  19. Ong, S.C., Png, S.W., Hsu, D., Lee, W.S.: POMDPs for Robotic Tasks with Mixed Observability. In: Proceedings of the Robotics: Science and Systems Conference. Seattle, USA (2009)

  20. Papadimitriou, C.H., Tsitsiklis, J.N.: The complexity of markov decision processes. Math. Oper. Res. 12(3), 441—450 (1987)

    Article  MathSciNet  MATH  Google Scholar 

  21. Poupart, P.: Exploiting Structure to Efficiently Solve Large Scale Partially Observable Markov Decision Processes. Ph.D. thesis. University of Toronto (2005)

  22. Pynadath, D.V., Tambe, M.: The communicative multiagent team decision problem: Analyzing teamwork theories and models. J. Artif. Intell. Res. 16, 389—423 (2002)

    MathSciNet  MATH  Google Scholar 

  23. Skoglar, P., Orguner, U., Gustafsson, F.: On information measures based on particle mixture for optimal bearings-only tracking. In: 2009 IEEE Aerospace conference, pp 1–14. IEEE (2009)

  24. Wong, E.M., Bourgault, F., Furukawa, T.: Multi-vehicle Bayesian search for multiple lost targets. In: Proceedings of the IEEE International Conference on Robotics and Automation, pp 3169–3174 (2005)

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to J. Capitan.

Additional information

This work has been supported by the FP7 EC-SAFEMOBIL Project (grant number 288082) and the PAIS-MultiRobot (TIC-7390) Regional Project.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Capitan, J., Merino, L. & Ollero, A. Cooperative Decision-Making Under Uncertainties for Multi-Target Surveillance with Multiples UAVs. J Intell Robot Syst 84, 371–386 (2016). https://doi.org/10.1007/s10846-015-0269-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10846-015-0269-0

Keywords

Navigation