Deep Reinforcement Learning Methods for Navigational Aids

  • Bijan Fakhri
  • Aaron Keech
  • Joel Schlosser
  • Ethan Brooks
  • Hemanth VenkateswaraEmail author
  • Sethuraman Panchanathan
  • Zsolt Kira
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11010)


Navigation is one of the most complex daily activities we engage in. Partly due to its complexity, navigational abilities are vulnerable to many conditions including Topographical Agnosia, Alzheimer’s Disease, and vision impairments. While navigation using solely vision remains a difficult problem in the field of assistive technology, emerging methods in Deep Reinforcement Learning and Computer Vision show promise in producing vision-based navigational aids for those with navigation impairments. To this effect, we introduce GraphMem, a Neural Computing approach to navigation tasks and compare it to several state of the art Neural Computing methods in a one-shot, 3D, first-person maze solving task. Comparing GraphMem to current methods in navigation tasks unveils insights into navigation and represents a first step towards employing these emerging techniques in navigational assistive technology.


Navigation Assistive technology Reinforcement learning Topographical agnosia 


  1. 1.
    Abadi, et al.: TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems (2016)Google Scholar
  2. 2.
    Allamanis, M., Brockschmidt, M., Khademi, M.: Learning to Represent Programs with Graphs. In: ICLR (2018)Google Scholar
  3. 3.
    Bengio, Y.: Continuous control with deep reinforcement learning. Foundations and Trends\(\textregistered \). Mach. Learn. 2(1), 1–127 (2009)Google Scholar
  4. 4.
    Bourne, R.R.A., et al.: Magnitude, temporal trends, and projections of the global prevalence of blindness and distance and near vision impairment: a systematic review and meta-analysis. Lancet Global Health 5(9), e888–e897 (2017)CrossRefGoogle Scholar
  5. 5.
    Brockman, G., et al.: OpenAI Gym (2016)Google Scholar
  6. 6.
    Duan, Y., Schulman, J., Chen, X., Bartlett, P., Sutskever, I., Abbeel, P.: RL\(\hat{}2\): fast reinforcement learning via slow reinforcement learning. arXiv, pp. 1–14 (2016)Google Scholar
  7. 7.
    Duthey, B.: Background Paper 6.11 Alzheimer Disease and other Dementias, Update on 2004. World Health Organization, pp. 1–77, February 2013Google Scholar
  8. 8.
    Graves, A., Wayne, G., Danihelka, I.: Neural Turing Machines. arXiv, pp. 1–26, October 2014Google Scholar
  9. 9.
    Graves, A., et al.: Hybrid computing using a neural network with dynamic external memory. Nat. Res. (2016)Google Scholar
  10. 10.
    Gulcehre, C., Chandar, S., Bengio, Y.: Memory Augmented Neural Networks with Wormhole Connections. arXiv, pp. 1–27 (2017)Google Scholar
  11. 11.
    Gupta, S., Davidson, J., Levine, S., Sukthankar, R., Malik, J.: Cognitive mapping and planning for visual navigation. In: CVPR (2017)Google Scholar
  12. 12.
    Heess, N., Hunt, J.J., Lillicrap, T.P., Silver, D.: Memory-based control with recurrent neural networks (2015)Google Scholar
  13. 13.
    Hochreiter, S., Urgen Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)CrossRefGoogle Scholar
  14. 14.
    Jang, E., Gu, S., Poole, B.: Categorical Reparameterization with Gumbel-Softmax. In: International Conference on Learning Representations, pp. 1–13 (2017)Google Scholar
  15. 15.
    Johnson, M., Hofmann, K., Hutton, T., Bignell, D.: The Malmo platform for artificial intelligence experimentation. In: IJCAI International Joint Conference on Artificial Intelligence, January 2016, pp. 4246–4247 (2016)Google Scholar
  16. 16.
    Karpathy, A., Li, F.F.: Deep visual-semantic alignments for generating image descriptions. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 39, no. 4, pp. 3128–3137 (2015)Google Scholar
  17. 17.
    Kempka, M., Wydmuch, M., Runc, G., Toczek, J., Jaskowski, W.: ViZDoom: a doom-based AI research platform for visual reinforcement learning. In: IEEE Conference on Computational Intelligence and Games, CIG (2017)Google Scholar
  18. 18.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Advances In Neural Information Processing Systems, pp. 1–9 (2012)Google Scholar
  19. 19.
    Mirowski, et al.: Learning to Navigate in Cities Without a Map (2018)Google Scholar
  20. 20.
    Mirowski, P., et al.: Learning to navigate in complex environments. ICLR (2017)Google Scholar
  21. 21.
    Mnih, V., et al.: Human-level control through deep reinforcement learning. Nature 518(7540), 529–533 (2015)CrossRefGoogle Scholar
  22. 22.
    Mnih, et al.: Asynchronous methods for deep reinforcement learning. arXiv preprint, vol. 48, pp. 1–28. arXiv:1602.01783v1 [cs.LG] (2016)
  23. 23.
    Mnih, V., et al.: Playing Atari with deep reinforcement learning. arXiv (2013).
  24. 24.
    Monacelli, A.M., Cushman, L.A., Kavcic, V., Duffy, C.J.: Spatial disorientation in Alzheimer’s disease: the remembrance of things passed. Neurology 61(11), 1491–1497 (2003)CrossRefGoogle Scholar
  25. 25.
    Moser, M.B., Rowland, D.C., Moser, E.I.: Place cells, grid cells, and memory. Cold Spring Harb. Perspect. Biol. 7(2), a021808 (2015)CrossRefGoogle Scholar
  26. 26.
    Oh, J., Chockalingam, V., Singh, S., Lee, H.: Control of Memory, Active Perception, and Action in Minecraft. arXiv:1605.09128 [cs] (2016)
  27. 27.
    Sanchez-Gonzalez, A., et al.: Graph networks as learnable physics engines for inference and control (2018)Google Scholar
  28. 28.
    Santoro, A., Bartunov, S., Botvinick, M., Wierstra, D., Lillicrap, T.: One-shot Learning with Memory-Augmented Neural Networks. arXiv:1605.06065 [cs], May 2016
  29. 29.
    Schulman, J., Levine, S., Jordan, M., Abbeel, P.: Trust region policy optimization. In: ICML 2015, p. 16, February 2015Google Scholar
  30. 30.
    Wolbers, T., Hegarty, M.: What determines our navigational abilities? Trends Cogn. Sci. 14(3), 138–146 (2010)CrossRefGoogle Scholar
  31. 31.
    Xie, L., Wang, S., Markham, A., Trigoni, N.: Towards Monocular Vision based Obstacle Avoidance Through Deep Reinforcement Learning. Robotics: Science and Systems Workshop 2017: New Frontiers for Deep Learning in Robotics (2017)Google Scholar
  32. 32.
    Zhang, J., Springenberg, J.T., Boedecker, J., Burgard, W.: Deep reinforcement learning with successor features for navigation across similar environments. In: IEEE International Conference on Intelligent Robots and Systems, September 2017, pp. 2371–2378 (2017)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Bijan Fakhri
    • 1
  • Aaron Keech
    • 3
  • Joel Schlosser
    • 3
  • Ethan Brooks
    • 3
  • Hemanth Venkateswara
    • 1
    Email author
  • Sethuraman Panchanathan
    • 1
  • Zsolt Kira
    • 2
    • 3
  1. 1.School of Computing, Informatics, and Decision Systems EngineeringArizona State UniversityTempeUSA
  2. 2.School of Interactive ComputingGeorgia TechAtlantaUSA
  3. 3.Georgia Tech Research InstituteAtlantaUSA

Personalised recommendations