Skip to main content

A Wise Up Visual Robot Driven by a Self-taught Neural Agent

  • 693 Accesses

Part of the Advances in Intelligent Systems and Computing book series (AISC,volume 1288)

Abstract

This paper presents a biological inspired robot capable of learning by itself high level Tic-Tac-Toe playing policies and then use this knowledge to advantageously compete with humans. The robot comprises a robotic arm, an artificial vision system and a self-motivated neural agent which has the capability to explore in a simulated ambient, new forms of game episodes that conduce toward bigger rewards. During the training phase a three terms reinforcement learning scheme is proposed, where the agent memory resources are sustained by adviser neural sub-networks, noise-balanced trained as to satisfy the look for future conditions in the control optimization predicted by the Bellman equation. In the operating phase the components merge into a wised up robot, with look ahead capacities, that mimic the abilities of ingenious human players. The achieved look ahead robotic intelligence could be useful in other complex robotic mechanisms.

Keywords

  • Clever robots
  • Self-taught agents
  • Neural networks

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-63128-4_47
  • Chapter length: 12 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   269.00
Price excludes VAT (USA)
  • ISBN: 978-3-030-63128-4
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   349.99
Price excludes VAT (USA)
Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.
Fig. 5.
Fig. 6.
Fig. 7.
Fig. 8.

References

  1. Anjomshoae, S., Najjar, A., Calvaresi, D., Främling, K.: Explainable agents and robots: results from a systematic literature review, June 2019

    Google Scholar 

  2. Brembs, B.: Genetic analysis of behavior in drosophila. In: The Oxford Handbook of Invertebrate Neurobiology, p. 71. Oxford University Press (2019)

    Google Scholar 

  3. Bösser, T.: Autonomous agents. In: Wright, J.D. (ed.) International Encyclopedia of the Social & Behavioral Sciences, 2nd edn, pp. 309–313. Elsevier, Oxford (2015)

    CrossRef  Google Scholar 

  4. Canaan, R., Salge, C., Togelius, J., Nealen, A.: Leveling the playing field - fairness in AI versus human game benchmarks, March 2019

    Google Scholar 

  5. Chang, O.: Autonomous robots and behavior initiators. In: Anbarjafari, G., Escalera, S. (eds.) Human-Robot Interaction, chapter 7. IntechOpen, Rijeka (2018)

    Google Scholar 

  6. Chang, O.: Self-programming robots boosted by neural agents. In: Wang, S., Yamamoto, V., Su, J., Yang, Y., Jones, E., Iasemidis, L., Mitchell, T. (eds.) Brain Informatics, pp. 448–457. Springer International Publishing, Cham (2018)

    CrossRef  Google Scholar 

  7. Crowley, K., Siegler, R.S.: Flexible strategy use in young children’s tic-tac-toe. Cogn. Sci. 17, 531–561 (1993)

    CrossRef  Google Scholar 

  8. Datta, S., Barua, R., Das, J.: Application of artificial intelligence in modern healthcare system. In: Pereira, L. (ed.) Alginates, chapter 8. IntechOpen, Rijeka (2020)

    Google Scholar 

  9. Do, N.: Norman do how to win at tic-tac-toe (2005)

    Google Scholar 

  10. Heining, M.: Dynamical learning: Case study on tic-tac-toe. Master’s thesis, Technische Universitat Munchen. Department of Mathematics, April 2017

    Google Scholar 

  11. Khandelwal, P., Zhang, S., Sinapov, J., Leonetti, M., Thomason, J., Yang, F., Gori, I., Svetlik, M., Khante, P., Lifschitz, V., Aggarwal, J., Mooney, R., Stone, P.: Bwibots: a platform for bridging the gap between AI and human-robot interaction research. Int. J. Robot. Res. 36, 027836491668894 (2017)

    CrossRef  Google Scholar 

  12. Li, C., Imeokparia, E., Ketzner, M., Tsahai, T.: Teaching the nao robot to play a human-robot interactive game. In: 2019 International Conference on Computational Science and Computational Intelligence (CSCI), pp. 712–715 (2019)

    Google Scholar 

  13. Ling, S.H., Lam, H.K.: Playing tic-tac-toe using genetic neural network with double transfer functions. JILSA 3, 37–44 (2011)

    CrossRef  Google Scholar 

  14. Javier, F., Martínez, P.: Research on reinforcement learning methods: a practical study, July 2017

    Google Scholar 

  15. Maye, A., Hsieh, C.H., Sugihara, G., Brembs, B.: Order in spontaneous behavior. PLoS ONE 2, e443 (2007)

    CrossRef  Google Scholar 

  16. Mela, H., Barznji, H.M.K.: Artificial intelligence and game development, January 2019

    Google Scholar 

  17. Millington, I., Funge, J.: Artificial Intelligence for Games, January 2009

    Google Scholar 

  18. Andrychowicz, O.M., Baker, B., Chociej, M., Józefowicz, R., McGrew, B., Pachocki, J.W., Petron, A., Plappert, M., Powell, G., Ray, A., Schneider, J., Sidor, S., Tobin, J., Welinder, P., Weng, L., Zaremba, W.: Learning dexterous in-hand manipulation. CoRR, abs/1808.00177 (2018)

    Google Scholar 

  19. Pérez, J.A., Deligianni, F., Ravì, D., Yang, G.Z.: Artificial intelligence and robotics. CoRR, abs/1803.10813 (2018)

    Google Scholar 

  20. Poddighe, R.: Playing tic-tac-toe with the NAO humanoid robot (2014)

    Google Scholar 

  21. Such, J., Criado, N., Vercouter, L., Rehak, M.: Intelligent cybersecurity agents. IEEE Intell. Syst. 31, 3–7 (2016)

    CrossRef  Google Scholar 

  22. Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction, Adaptive Computation and Machine Learning series. MIT Press (2018)

    Google Scholar 

  23. Wiese, E., Metta, G., Wykowska, A.: Robots as intentional agents: using neuroscientific methods to make robots appear more social. Front. Psychol. 8, 1663 (2017)

    CrossRef  Google Scholar 

  24. Zaslavsky, C., Kramer, A., (ill.).: Tic Tac Toe : And Other Three-in-A Row Games From Ancient Egypt to The Modern Computer, 1st edn. Crowell, New York (1982)

    Google Scholar 

  25. Zhinin-Vera, L., Chang, O., Valencia, R., Velastegui, R., Pilliza, G., Socasi, F.: Q-credit card fraud detector for imbalanced classification using reinforcement learning. pp. 279–286, February 2020

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Oscar Chang .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

Chang, O., Zhinin-Vera, L. (2021). A Wise Up Visual Robot Driven by a Self-taught Neural Agent. In: Arai, K., Kapoor, S., Bhatia, R. (eds) Proceedings of the Future Technologies Conference (FTC) 2020, Volume 1. FTC 2020. Advances in Intelligent Systems and Computing, vol 1288. Springer, Cham. https://doi.org/10.1007/978-3-030-63128-4_47

Download citation