Journal of Intelligent & Robotic Systems

, Volume 64, Issue 2, pp 277–298 | Cite as

Aircraft Collision Avoidance Using Monte Carlo Real-Time Belief Space Search

  • Travis B. Wolf
  • Mykel J. Kochenderfer


The aircraft collision avoidance problem can be formulated using a decision-theoretic planning framework where the optimal behavior requires balancing the competing objectives of avoiding collision and adhering to a flight plan. Due to noise in the sensor measurements and the stochasticity of intruder state trajectories, a natural representation of the problem is as a partially-observable Markov decision process (POMDP), where the underlying state of the system is Markovian and the observations depend probabilistically on the state. Many algorithms for finding approximate solutions to POMDPs exist in the literature, but they typically require discretization of the state and observation spaces. This paper investigates the introduction of a sample-based representation of state uncertainty to an existing algorithm called Real-Time Belief Space Search (RTBSS), which leverages branch-and-bound pruning to make searching the belief space for the optimal action more efficient. The resulting algorithm, called Monte Carlo Real-Time Belief Space Search (MC-RTBSS), is demonstrated on encounter scenarios in simulation using a beacon-based surveillance system and a probabilistic intruder model derived from recorded radar data.


POMDP algorithms Aircraft collision avoidance 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Espindle, L.P., Griffith, J.D., Kuchar, J.K.: Safety analysis of upgrading to TCAS version 7.1 using the 2008 U.S. correlated encounter model. Project Report ATC-349, Lincoln Laboratory, Lexington, Mass. (2009)Google Scholar
  2. 2.
    Frazzoli, E., Dahleh, M.A., Feron, E.: Real-time motion planning for agile autonomous vehicles. AIAA Journal of Aerospace Computing, Information, and Communication 25, 116–129 (2004)Google Scholar
  3. 3.
    Kaelbling, L.P., Littman, M.L., Cassandra, A.R.: Planning and acting in partially observable stochastic domains. Artif. Intell. 101, 99–134 (1998)MathSciNetzbMATHCrossRefGoogle Scholar
  4. 4.
    Kochenderfer, M., Kuchar, J., Espindle, L., Griffith, J.: Uncorrelated encounter model of the national airspace system version 1.0. Project Report ATC-345, Lincoln Laboratory, Lexington, Mass. (2008)Google Scholar
  5. 5.
    Kuchar, J.K., Drumm, A.C.: The traffic alert and collision avoidance system. Linc. Lab. J. 16(2), 277–296 (2007)Google Scholar
  6. 6.
    Kurniawati, H., Hsu, D., Lee, W.S.: SARSOP: efficient point-based POMDP planning by approximating optimally reachable belief spaces. In: Proceedings in Robotics: Science and Systems (2008)Google Scholar
  7. 7.
    Lawrence, D., Pisano, W.: Lyapunov vector fields for autonomous unmanned aircraft flight control. J. Guid. Control Dyn. 31(5), 1220–1229 (2008)CrossRefGoogle Scholar
  8. 8.
    Neapolitan, R.: Learning Bayesian Networks. Pearson Prentice Hall, Upper Saddle River, NJ (2004)Google Scholar
  9. 9.
    Paquet, S., Tobin, L., Chaib-draa, B.: Real-time decision making for large POMDPs. In: Advances in Artificial Intelligence (LNAI 3501), pp. 450–455 (2005)Google Scholar
  10. 10.
    Pineau, J., Gordon, G., Thrun, S.: Point-based value iteration: An anytime algorithm for POMDPs. In: International Joint Conference on Artificial Intelligence (IJCAI), pp. 1025–1032. Acapulco, Mexico (2003)Google Scholar
  11. 11.
    Ross, S., Pineau, J., Paquet, S., Chaib-draa, B.: Online planning algorithms for POMDPs. J. Artif. Intell. Res. 32, 663–704 (2008)MathSciNetzbMATHGoogle Scholar
  12. 12.
    RTCA: Minimum operational performance standards for traffic alert and collision avoidance system II (TCAS II) airborne equipment. Tech. rep., RTCA/DO-185A, Washington, D.C. (1997)Google Scholar
  13. 13.
    Schouwenaars, T., Mettler, B., Feron, E., How, J.: Hybrid model for trajectory planning of agile autonomous aerial vehicles. Journal of Aerospace Computing, Information, and Communication, Special Issue on Intelligent Systems 1, 629–651 (2004)Google Scholar
  14. 14.
    Smith, T., Simmons, R.G.: Heuristic search value iteration for POMDPs. In: Proc. Int. Conf. on Uncertainty in Artificial Intelligence (UAI) (2004)Google Scholar
  15. 15.
    Srinivasan, R.: Importance Sampling: Applications in Communications and Detection. Springer-Verlag, Berlin, Germany (2002)zbMATHGoogle Scholar
  16. 16.
    Sundqvist, B.G.: Auto-ACAS—robust nuisance-free collision avoidance. In: Proceedings of the 44th IEEE Conference on Decision and Control and European Control Conference, pp. 3961–3963. IEEE (2005)Google Scholar
  17. 17.
    Thrun, S.: Monte Carlo POMDPs. In: Solla, S., Leen, T., Müller, K.R. (eds.) Advances in Neural Information Processing Systems 12, pp. 1064–1070. MIT Press (2000)Google Scholar
  18. 18.
    Winder, L.F.: Hazard avoidance alerting with Markov decision processes. Ph.D. thesis, MIT, Cambridge, Mass. (2004)Google Scholar

Copyright information

© Springer Science+Business Media B.V. 2011

Authors and Affiliations

  1. 1.Lincoln LaboratoryMassachusetts Institute of TechnologyLexingtonUSA

Personalised recommendations