Online Learning Methods for Border Patrol Resource Allocation

  • Richard Klíma
  • Christopher Kiekintveld
  • Viliam Lisý
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8840)


We introduce a model for border security resource allocation with repeated interactions between attackers and defenders. The defender must learn the optimal resource allocation strategy based on historical apprehension data, balancing exploration and exploitation in the policy. We experiment with several solution methods for this online learning problem including UCB, sliding-window UCB, and EXP3. We test the learning methods against several different classes of attackers including attacker with randomly varying strategies and attackers who react adversarially to the defender’s strategy. We present experimental data to identify the optimal parameter settings for these algorithms and compare the algorithms against the different types of attackers.


security online learning multi-armed bandit problem border patrol resource allocation UCB EXP3 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    2012–2016 border patrol strategic plan. U.S. Customs and Border Protection (2012)Google Scholar
  2. 2.
    Auer, P.: Using confidence bounds for exploitation-exploration trade-offs. The Journal of Machine Learning Research 3, 397–422 (2003)MathSciNetzbMATHGoogle Scholar
  3. 3.
    Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The non-stochastic multi-armed bandit problem. SIAM Journal on Computing 32(1) (2001)Google Scholar
  4. 4.
    Fudenberg, D., Levine, D.K.: The Theory of Learning in Games. The MIT Press (1998)Google Scholar
  5. 5.
    Garivier, A., Moulines, E.: On upper-confidence bound policies for non-stationary bandit problems. Technical report (2008)Google Scholar
  6. 6.
    Kiekintveld, C., Jain, M., Tsai, J., Pita, J., Ordonez, F., Tambe, M.: Computing optimal randomized resource allocations for massive security games. In: AAMAS 2009 (2009)Google Scholar
  7. 7.
    Pita, J., Jain, M., Western, C., Portway, C., Tambe, M., Ordonez, F., Kraus, S., Parachuri, P.: Depoloyed ARMOR protection: The application of a game-theoretic model for security at the Los Angeles International Airport. In: AAMAS 2008 (Industry Track) (2008)Google Scholar
  8. 8.
    Pita, J., Tambe, M., Kiekintveld, C., Cullen, S., Steigerwald, E.: GUARDS - game theoretic security allocation on a national scale. In: AAMAS 2011 (Industry Track) (2011)Google Scholar
  9. 9.
    Predd, J., Willis, H., Setodji, C., Stelzner, C.: Using pattern analysis and systematic randomness to allocate U.S. border security resources (2012)Google Scholar
  10. 10.
    Shieh, E., An, B., Yang, R., Tambe, M., Baldwin, C., Direnzo, J., Meyer, G., Baldwin, C.W., Maule, B.J., Meyer, G.R.: PROTECT: A Deployed Game Theoretic System to Protect the Ports of the United States. In: AAMAS (2012)Google Scholar
  11. 11.
    Tambe, M.: Security and Game Theory: Algorithms, Deployed Systems, Lessons Learned. Cambridge University Press (2011)Google Scholar
  12. 12.
    Tsai, J., Rathi, S., Kiekintveld, C., Ordóñez, F., Tambe, M.: IRIS - A tools for strategic security allocation in transportation networks. In: AAMAS 2009 (Industry Track) (2009)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Richard Klíma
    • 1
    • 2
  • Christopher Kiekintveld
    • 2
  • Viliam Lisý
    • 1
  1. 1.Department of Computer Science, FEECzech Technical University in PraguePragueCzeck Republic
  2. 2.Computer Science DepartmentUniversity of Texas at El PasoEI PasoUSA

Personalised recommendations