Skip to main content
SpringerLink
Log in

Believable Bot Navigation via Playback of Human Traces

  • Chapter
  • First Online:
Believable Bots

Abstract

Imitation is a powerful and pervasive primitive underlying  examples of intelligent behaviour in nature. Can we use it as a tool to help build artificial agents that behave like humans do? This question is studied in the context of the BotPrize competition, a Turing-like test where computer game bots compete by attempting to fool human judges into thinking they are just another human player. One problem faced by such bots is that of human-like navigation within the virtual world. This chapter describes the Human Trace Controller, a component of the \({UT{\char 94}2}\) bot which took second place in the BotPrize 2010 competition. The controller uses a database of recorded human games in order to quickly retrieve and play back relevant segments of human navigation behaviour. Empirical evidence suggests that the method of direct imitation allows the bot to effectively solve several navigation problems while moving in a human-like fashion.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
eBook
USD 16.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    http://www.botprize.org/2010.html

References

  1. Rizzolatti, G., Fogassi, L., Gallese, V.: Neurophysiological mechanisms underlying the understanding and imitation of action. Nat. Rev. Neurosci. 2(9), 661–670 (2001)

    Article  Google Scholar 

  2. Gergely, G., Bekkering, H., Király, I.: Developmental psychology: rational imitation in preverbal infants. Nature 415(6873), 755 (2002)

    Article  Google Scholar 

  3. Nicolescu, M., Matarić, M.J.: Task learning through imitation and human-robot interaction. In: Dautenhahn, K., Nehaniv, C. (eds.) Models and Mechanisms of Imitation and Social Learning in Robots, Humans and Animals: Behavioural, Social and Communicative Dimensions, Cambridge University Press, Cambridge (2005)

    Google Scholar 

  4. Hingston, P.: A Turing Test for computer game bots. IEEE Trans. Comput. Intell. AI Games 1(3), 169–186 (2009)

    Article  Google Scholar 

  5. Laird, J.E., van Lent, M.: Interactive computer games: Human-Level AI’s killer application. AI Mag. 22(2), 15–25 (2001)

    Google Scholar 

  6. Aha, D.W., Molineaux, M.: Integrating learning in interactive gaming simulators. In: Proceedings of the AAAI’04 Workshop on Challenges of Game AI, AAAI Press (2004)

    Google Scholar 

  7. Bowling, M., Fürnkranz, J., Graepel, T., Musick, R.: Machine learning and games. Mach. Learn. 63, 211–215 (2006)

    Article  Google Scholar 

  8. Molineaux, M., Aha, D.W.: TIELT: a testbed for gaming environments. In: Proceedings of the Twentieth National Conference on Artificial Intelligence (Intelligent Systems Demonstrations), AAAI Press (2005)

    Google Scholar 

  9. Schaal, S.: Learning from demonstration. In Advances in Neural Information and Processing Systems, pp. 1040–1046. Citeseer (1997)

    Google Scholar 

  10. Atkeson, C., Schaal, S.: Robot Learning from Demonstration. In: Proceedings of the Fourteenth International Conference on Machine Learning, ICML’97, pp. 12–20. Citeseer (1997)

    Google Scholar 

  11. Argall, B.D., Chernova, S., Veloso, M., Browning, B.: A survey of robot learning from demonstration. Robot. Auton. Syst. 57(5), 469–483 (2009)

    Article  Google Scholar 

  12. Thurau, C., Bauckhage, C., Sagerer, G.: Synthesizing movements for computer game characters. Lect. Notes Comput. Sci. 3175, 179–186 (2004)

    Article  Google Scholar 

  13. Stolle, M., Atkeson, C.G.: Policies Based on Trajectory Libraries. In: Proceedings of the International Conference on Robotics and Automation (ICRA 2006), (2006)

    Google Scholar 

  14. Stolle, M., Tappeiner, H., Chestnutt, J., Atkeson, C.G.: Transfer of policies based on trajectory libraries. In: Proceedings of the International Conference on Intelligent Robots and Systems, (2007)

    Google Scholar 

  15. Hladky, S., Bulitko, V.: An evaluation of models for predicting opponent positions in first-person shooter video games. In: Proceedings of the IEEE 2008 Symposium on Computational Intelligence and Games (CIG’08). Perth, Australia (2008)

    Google Scholar 

  16. Orkin, J.D., Roy, D.: Automatic learning and generation of social behavior from collective human gameplay. In: Proceedings of The 8th International Conference on Autonomous Agents and Multiagent Systems (AAMAS’09), vol. 1, pp. 385–392. (2009)

    Google Scholar 

  17. Cardamone, L., Loiacono, D., Lanzi, P.L.: Learning drivers for TORCS through imitation using supervised methods. In: Proceedings of the IEEE 2009 Symposium on Computational Intelligence and Games (CIG’09), (2009)

    Google Scholar 

  18. Aler, R., Valls, J.M., Camacho, D., Lopez, A.: Programming robosoccer agents by modeling human behavior. Expert Syst. Appl. 36(2), Part 1, 1850–1859 (2009)

    Google Scholar 

  19. Pao, H.-K., Chen, K.-T., Chang, H.-C.: Game bot detection via avatar trajectory analysis. IEEE Trans. Comput. Intell. AI Games 2(3), 162–175 (2010)

    Article  Google Scholar 

  20. Sukthankar, G., Sycara, K.: Simultaneous team assignment and behavior recognition from spatio-temporal agent traces. In: Proceedings of Twenty-First National Conference on Artificial Intelligence (AAAI-06), (2006)

    Google Scholar 

  21. Murakami, S., Sato, T., Kojima, A., Hirono, D., Kusumoto, N., Thawonmas, R.: Outline of ICE-CEC 2011 and its mechanism for learning FPS tactics. In: Extended Abstract for the Human-like Bot Workshop at the IEEE Congress on Evolutionary Computation (CEC 2011), (2011)

    Google Scholar 

  22. Epic Games, Inc. and Digital Extremes, Inc.: Unreal Tournament 2004. Atari, Inc., March 2004

    Google Scholar 

  23. Kaminka, G.A., Veloso, M.M., Schaffer, S., Sollitto, C., Adobbati, R., Marshall, A.N., Scholer, A., Tejada, S.: GameBots: a flexible test bed for multiagent team research. Commun. ACM 45(1), 43–45 (2002)

    Article  Google Scholar 

  24. Gemrot, J., Kadlec, R., Bída, M., Burkert, O., Píbil, R., Havlíček, J., Zemčák, L., Šimlovič, J., Vansa, R., Štolba, M. et al.: Pogamut 3 can assist developers in building AI (Not Only) for their videogame agents. In: Agents for Games and Simulations, pp. 1–15. (2009)

    Google Scholar 

  25. Hingston, P.: A new design for a Turing Test for bots. In: IEEE Transactions on Computational Intelligence and AI in Games, (2010)

    Google Scholar 

  26. Floyd, R.: Algorithm 97: shortest path. Commun. ACM 5(6), 345 (1962)

    Article  Google Scholar 

  27. “SQLite”. http://www.sqlite.org/

  28. Jackins, C., Tanimoto, S.: Oct-trees and their use in representing three-dimensional objects. Comput. Graph. Image Process. 14(3), 249–270 (1980)

    Article  Google Scholar 

  29. Bentley, J.L.: Multidimensional binary search trees used for associative searching. Commun. ACM 18(9), 509–517 (1975)

    Article  MathSciNet  MATH  Google Scholar 

  30. Dirichlet, G.L.: Über die Reduktion der positiven quadratischen Formen mit drei unbestimmten ganzen Zahlen. J. für die Reine und Angewandte Mathematik 40, 209–227 (1850)

    Article  MATH  Google Scholar 

  31. Voronoi, G.: Nouvelles applications des paramètres continus à la théorie des formes quadratiques. J. für die Reine und Angewandte Mathematik 133, 97–178 (1907)

    Google Scholar 

Download references

Acknowledgments

The authors would like to thank Philip Hingston for organizing the BotPrize competitions and 2K Australia for sponsoring it. The authors would also like to thank students in the Freshman Research Initiative’s Computational Intelligence in Game Design stream and members of the Neural Networks Research Group at the University of Texas and to Christopher Tanguay and Peter Djeu for participating in recordings of human game traces and for volunteering to critique and evaluate versions of \({UT{\char 94}2}\) . This research was supported in part by the NSF under grants DBI-0939454 and IIS-0915038 and by the Texas Higher Education Coordinating Board grant 003658-0036-2007.

Author information

Authors and Affiliations

  1. The University of Texas at Austin, Austin, TX 78712, USA

    Igor V. Karpov, Jacob Schrum & Risto Miikkulainen

Authors
  1. Igor V. Karpov
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jacob Schrum
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Risto Miikkulainen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Igor V. Karpov .

Editor information

Editors and Affiliations

  1. , School of Computer and Security Science, Edith Cowan University, Building 13.148, 2 Bradford Street, Mount Lawley, 6050, West Australia, Australia

    Philip Hingston

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Karpov, I.V., Schrum, J., Miikkulainen, R. (2013). Believable Bot Navigation via Playback of Human Traces. In: Hingston, P. (eds) Believable Bots. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-32323-2_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-32323-2_6

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-32322-5

  • Online ISBN: 978-3-642-32323-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation