Cluster Computing

, Volume 21, Issue 1, pp 655–666 | Cite as

Zombies Arena: fusion of reinforcement learning with augmented reality on NPC

  • Saad Razzaq
  • Fahad MaqboolEmail author
  • Maham Khalid
  • Iram Tariq
  • Aqsa Zahoor
  • Muhammad Ilyas


Augmented reality (AR) is a discipline having less cognizance but it is the door to new advance technologies. Accustomed games doesn’t facilitate user to physically interact with the surroundings which resulted into reduced learning capabilities. Our objective is to develop AR based first person shooter game empowering reinforcement learning. This act as a building block to capacitate users to interact with the physical environment. Non-player characters will be able to learn and adopt strategy more wisely after each move to capacitate players. Game is played by hundred users at different stages. Reported results are summarized in experiment section.


Reinforcement learning Non-player character Human controlled character Augmented reality First person shooter Periodic cluster-weighted rewarding 


  1. 1.
    Hou, Y., Feng, L., Ong, Y.S.: Creating human-like non-player game characters using a memetic multi-agent system. In: 2016 International Joint Conference on Neural Networks (IJCNN), pp. 177–184. IEEE (2016)Google Scholar
  2. 2.
    Glavin, Frank G., Madden, Michael G.: Adaptive shooting for bots in first person shooter games using reinforcement learning. IEEE Trans. Comput. Intell. AI Games 7(2), 180–192 (2015)CrossRefGoogle Scholar
  3. 3.
    McPartland, Michelle, Gallagher, Marcus: Reinforcement learning in first person shooter games. IEEE Trans. Comput. Intell. AI Games 3(1), 43–56 (2011)CrossRefGoogle Scholar
  4. 4.
    Glavin, Frank, G., Madden, M.G.: Learning to shoot in first person shooter games by stabilizing actions and clustering rewards for reinforcement learning. In: 2015 IEEE Conference on Computational Intelligence and Games (CIG), pp. 344–351. IEEE (2015)Google Scholar
  5. 5.
    Makarov, I., et al.: Modelling human-like behavior through reward-based approach in a first-person shooter game. In: Proceedings of the Third Workshop on Experimental Economics and Machine Learning (EEML 2016), pp. 24–33. Moscow, 18 July 2016Google Scholar
  6. 6.
    Overholtzer, C.A., Simon D.L.: Evolving AI opponents in a first-person-shooter video game. In: Proceedings of the National Conference on Artificial Intelligence, Vol. 20. No. 4, Menlo Park, CA; Cambridge, MA; London; AAAI Press; MIT Press 1999, 2005: 1620Google Scholar
  7. 7.
    Jones, Jason: Benefits of Genetic Algorithms in Simulations for Game Designers. School of Informatics, University of Buffalo, Buffalo (2003)Google Scholar
  8. 8.
    Laird, John, VanLent, Michael: Human-level AI’s killer application: interactive computer games. AI Mag. 22(2), 15 (2001)Google Scholar
  9. 9.
    Kaelbling, L.P., Littman, M.L., Moore, A.W.: Reinforcement learning: a survey. J. Artif. Intell. Res. 4, 237–285 (1996)Google Scholar
  10. 10.
    Li, J., Fan, X.: Outdoor augmented reality tracking using 3d city models and game engine. In: 7th International Congress on Image and Signal Processing (CISP), vol. 2014, pp. 104–108. IEEE (2014)Google Scholar
  11. 11.
    Kim, S.L., et al.: Using Unity 3D to facilitate mobile augmented reality game development. In: IEEE World Forum on Internet of Things (WF-IoT), vol. 2014, pp. 21–26. IEEE (2014)Google Scholar
  12. 12.
    Koutromanos, G., Styliaras, G: The buildings speak about our city: a location based augmented reality game. In: 2015 6th International Conference on Information, Intelligence, Systems and Applications (IISA), pp. 1–6. IEEE (2015)Google Scholar
  13. 13.
    Maskott, G.K., Maskott, M.B, Vrysis, L.: Serious+: a technology assisted learning space based on gaming. In: 2015 International Conference on Interactive Mobile Communication Technologies and Learning (IMCL), pp. 430–432. IEEE, (2015)Google Scholar
  14. 14.
    Emigh, M.S., et al.: Reinforcement learning in video games using nearest neighbor interpolation and metric learning. IEEE Trans. Comput. Intell. AI Games 8(1), 56–66 (2016)CrossRefGoogle Scholar
  15. 15.
    Wang, Di, Tan, Ah-Hwee: Creating autonomous adaptive agents in a real-time first-person shooter computer game. IEEE Trans. Comput. Intell. AI Games 7(2), 123–138 (2015)CrossRefGoogle Scholar
  16. 16.
    Tastan, B., Chang, Y., Sukthankar, G.: Learning to intercept opponents in first person shooter games. In: 2012 IEEE Conference on Computational Intelligence and Games (CIG), pp. 100–107. IEEE (2012)Google Scholar
  17. 17.
    Hladky, S., Bulitko, V.: An evaluation of models for predicting opponent positions in first-person shooter video games. In: CIG’08. IEEE Symposium on Computational Intelligence and Games, vol. 2008, pp. 39–46. IEEE (2008)Google Scholar
  18. 18.
    Nilsen, Trond, Linton, Steven, Looser, Julian: Motivations for augmented reality gaming. Proc. FUSE 4, 86–93 (2004)Google Scholar
  19. 19.
    Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction, vol. 1, 1st edn. MIT press, Cambridge (1998)Google Scholar
  20. 20.
    Manslow, J.: Using reinforcement learning to solve AI control problems. AI Game Program Wisdom 2, 591–601 (2004)Google Scholar
  21. 21.
    Merrick, K., Maher, M.L.: Motivated reinforcement learning for non-player characters in persistent computer game worlds. In: Proceedings of the 2006 ACM SIGCHI International Conference on Advances in Computer Entertainment Technology, p. 3. ACM (2006)Google Scholar

Copyright information

© Springer Science+Business Media New York 2017

Authors and Affiliations

  • Saad Razzaq
    • 1
  • Fahad Maqbool
    • 1
    Email author
  • Maham Khalid
    • 1
  • Iram Tariq
    • 1
  • Aqsa Zahoor
    • 1
  • Muhammad Ilyas
    • 1
  1. 1.Department of Computer Science & Information TechnologyUniversity of SargodhaSargodhaPakistan

Personalised recommendations