Abstract
We study the problem of optimally coordinating multiple fixed-wing UAVs to perform vision-based target tracking, which entails that the UAVs are tasked with gathering the best joint vision-based measurements of an unpredictable ground target. We utilize an analytic expression for the error covariance associated with the fused measurements of the target’s position, and we employ stochastic fourth-order models for all vehicles, thereby incorporating a high degree of realism into the problem formulation. While dynamic programming can generate an optimal control policy that minimizes the expected value of the fused geolocation error covariance over time, it is accompanied by significant computational challenges due to the curse of dimensionality. In order to circumvent this challenge, we present a novel policy generation technique that combines simulation-based policy iteration with a robust regression scheme. The resulting control policy offers a significant advantage over alternative approaches and shows that the optimal control strategy involves coordinating the UAVs’ distances to the target rather than their viewing angles, which had been a common practice in target tracking.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Mallick, M.: Geolocation using video sensor measurements. In: IEEE Int. Conf. Information Fusion. Quebec, Canada (2007)
Collins, G.E., Stankevitz, C.R., Liese, J.: Implementation of a sensor guided flight algorithm for target tracking by small UAS. In: Ground/Air Multi-Sens. Interoperability, Integration, Netw. Persistent ISR II, vol. 8047, SPIE (2011)
Kingston, D.B.: Decentralized Control of Multiple UAVs for perimeter and target surveillance. PhD thesis, Brigham Young University (2007)
Gu, G., Chandler, P.R., Schumacher, C.J., Sparks, A., Pachter, M.: Optimum cooperative UAV sensing using a team of UAVs. IEEE Trans. Aerosp. Electron. Syst. 42, 1446–1458 (2006)
Rysdyk, R.: UAV path following for constant line-of-sight. In: Proc. 2nd AIAA Unmanned Unltd. Syst. Technol. Operations Aerosp, Land Sea Conf (2003)
Frew, E.W.: Lyapunov guidance vector fields for unmanned aircraft applications. In: Am. Control Conf. (2007)
Kim, S., Oh, H., Tsourdos, A.: Nonlinear model predictive coordinated standoff tracking of a moving ground vehicle. J. Guid. Control Dyn. 36(2), 557–566 (2013)
Ma, L., Hovakimyan, N.: Cooperative target tracking in balanced circular formation: Multiple UAVs tracking a ground vehicle. In: Am. Control Conf., pp 5386–5391. IEEE (2013)
Summers, T.H.: Cooperative Shape and Orientation Control of Autonomous Vehicle Formations. PhD thesis, University of Texas at Austin (2010)
Oh, H., Kim, S., Tsourdos, A., White, B.A.: Decentralised standoff tracking of moving targets using adaptive sliding mode control for UAVs. J. Intell. Robot. Syst., 1– 15 (2013)
Peterson, C., Paley, D.A.: Multivehicle coordination in an estimated time-varying flowfield. J. Guid. Control Dyn. 34(1), 177–191 (2011)
Anderson, R., Milutinović, D.: A stochastic approach to Dubins feedback control for target tracking. In: IEEE / RSJ Conf. Intell. Robots Syst., pp 3917–3922 (2011)
Quintero, S.A.P., Hespanha, J.P.: Vision-based target tracking with a small UAV: Optimization-based control strategies. Control Eng. Pract. 32, 28–42 (2014)
Miller, S.A., Harris, Z.A., Chong, E.K.P.: A POMDP framework for coordinated guidance of autonomous UAVs for multitarget tracking, EURASIP. J. Adv. Signal Process., 1–17 (2009)
Thrun, S., Burgard, W., Fox, D.: Probabilistic Robotics. The MIT Press (2005)
Stachura, M., Carfang, A., Frew, E.W.. In: Workshop Robotic Wirel. Sens. Netw., (Marina Del Ray, CA) Cooperative target tracking with a communication limited active sensor network (2009)
Ding, C., Morye, A.A., Farrell, J.A., Roy-Chowdhury, A. K.: Coordinated sensing and tracking for mobile camera platforms. In: Am. Control Conf., pp. 5114–5119, IEEE (2012)
Quintero, S.A.P., Papi, F., Klein, D.J., Chisci, L., Hespanha, J.P.: Optimal UAV coordination for target tracking using dynamic programming. In: IEEE Conf. Decis. Control, (Atlanta, GA) (2010)
Lalish, E., Morgansen, K., Tsukamaki, T.: Oscillatory control for constant-speed unicycle-type vehicles. In: IEEE Conf. Decis. Control (2007)
Regina, N., Zanzi, M.: UAV guidance law for ground-based target trajectory tracking and loitering. In: Aerosp. Conf., IEEE (2011)
Quintero, S.A.P., Collins, G.E., Hespanha, J.P.: Flocking with fixed-wing UAVs for distributed sensing: A stochastic optimal control approach. In: Am. Control Conf., (Washington, D.C.) (2013)
Bouchard, B., Warin, X.: Monte-carlo valorisation of American options: facts and new algorithms to improve existing methods. In: R. Carmona, P. Del Moral, P. Hu, N. Oudjane (eds.) Numer. Methods Finance,Springer Proc. Math. (2011)
Lewis, F. L., Vrabie, D., Syrmos, V.L.: 3rd. John Wiley, Hoboken, New Jersey (2012)
Guestrin, C., Hauskrecht, M., Kveton, B.: Solving factored mdps with continuous and discrete variables. In: Proc. 20th Conf. on Uncertain. in Artif. Intell., 235–242, AUAI Press (2004)
Wiegerinck, W., Broek, B.v.d., Kappen, H.: Stochastic optimal control in continuous space-time multi-agent systems. In: Proc. 22nd Conf. on Uncertain. in Artif. Intell. (2006)
Longstaff, F.A., Schwartz, E.S.: Valuing American options by simulation: A simple least-squares approach. Rev. Financial Stud. 14(1), 113–147 (2001)
Egloff, D.: Monte carlo algorithms for optimal stopping and statistical learning. Ann. Appl. Probab. 15(2), 1396–1432 (2005)
Ludkovski, M., Niemi, J.: Optimal dynamic policies for influenza management, Statistical Commun. Infect. Dis. (2010)
Bertsekas, D.P., Dynamic Programming and Optimal Control, vol. 2. Belmont, MA: Athena Scientific (2012)
Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning, 2nd edn. Springer (2009)
Belomestny, D., Kolodko, A., Schoenmakers, J.: Regression methods for stochastic control problems and their convergence analysis. SIAM J. on Control and Optim. 48(5), 3562–3588 (2010)
Vrabie, D., Vamvoudakis, K.G., Lewis, F.L.: Optimal adaptive control and differential games by reinforcement learning principles, vol. 81. IET (2013)
Oh, H., Kim, S., Shin, H.-S., Tsourdos, A., White, B.: Coordinated standoff tracking of groups of moving targets using multiple UAVs. In: Control & Automation (MED), 2013 21st Mediterranean Conf. on, 969–977, IEEE (2013)
Gramacy, R.B., Ludkovski, M.: Sequential design for optimal stopping problems,” SIAM J. on Financial Math. Note Accepted subject to minor revision (2015)
Carrillo, L., Russell, W., Hespanha, J., Collins, G.: State estimation of multiagent systems under impulsive noise and disturbances. IEEE Trans. Control Syst. Technol. 23, 13–26 (2015)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Quintero, S.A.P., Ludkovski, M. & Hespanha, J.P. Stochastic Optimal Coordination of Small UAVs for Target Tracking using Regression-based Dynamic Programming. J Intell Robot Syst 82, 135–162 (2016). https://doi.org/10.1007/s10846-015-0270-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10846-015-0270-7