Adaptive Landmark-Based Navigation System Using Learning Techniques

  • Bassel Zeidan
  • Sakyasingha Dasgupta
  • Florentin Wörgötter
  • Poramate Manoonpong
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8575)


The goal-directed navigational ability of animals is an essential prerequisite for them to survive. They can learn to navigate to a distal goal in a complex environment. During this long-distance navigation, they exploit environmental features, like landmarks, to guide them towards their goal. Inspired by this, we develop an adaptive landmark-based navigation system based on sequential reinforcement learning. In addition, correlation-based learning is also integrated into the system to improve learning performance. The proposed system has been applied to simulated simple wheeled and more complex hexapod robots. As a result, it allows the robots to successfully learn to navigate to distal goals in complex environments.


Goal-directed behavior Sequential reinforcement learning Correlation based learning Neural networks Walking robots 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Doya, K.: Reinforcement Learning in Continuous Time and Space. Neural Comput. 12(1), 219–245 (2000)CrossRefGoogle Scholar
  2. 2.
    Manoonpong, P., Kolodziejski, C., Woergoetter, F., Morimoto, J.: Combining Correlation-based and Reward-based Learning in Neural Control for Policy Improvement. Advances in Complex Systems 16(02-03) (2013), doi:10.1142/S021952591350015XGoogle Scholar
  3. 3.
    Hasselt, H., Wiering, M.: Reinforcement Learning in Continuous Action Spaces. In: Proceedings of the 2007 IEEE Symposium on Approximate Dynamic Programming and Reinforcement Learning, ADPRL (2007)Google Scholar
  4. 4.
    Porr, B., Woergoetter, F.: Strongly Improved Stability and Faster Convergence of Temporal Sequence Learning by Utilising Input Correlations Only. Neural Comput. 18, 1380–1412 (2006)CrossRefzbMATHGoogle Scholar
  5. 5.
    Manoonpong, P., Pasemann, F., Woergoetter, F.: Sensor-driven Neural Control for Omnidirectional Locomotion and Versatile Reactive Behaviors of Walking Machines. Robotics and Autonomous Systems 56(3), 265–288 (2008)CrossRefGoogle Scholar
  6. 6.
    Woergoetter, F., Porr, B.: Temporal Sequence Learning, Prediction, and Control - A Review of Different Models and their Relation to Biological Mechanisms. Neural Comp. 17, 245–319 (2005)CrossRefGoogle Scholar
  7. 7.
    Bakker, B., Schmidhuber, J.: Hierarchical Reinforcement Learning with Subpolicies Specializing for Learned Subgoals. In: Proceedings of the 2nd IASTED International Conference on Neural Networks and Computational Intelligence, pp. 125–130 (2004)Google Scholar
  8. 8.
    Botvinick, M.M., Niv, Y., Barto, A.C.: Hierarchically Organized Behavior and its Neural Foundations: A Reinforcement Learning Perspective. Cognition 113(3), 262–280 (2009), doi:10.1016/j.cognition.2008.08.011CrossRefGoogle Scholar
  9. 9.
    Masehian, E., Naseri, A.: Mobile Robot Online Motion Planning Using Generalized Voronoi Graphs. Journal of Industrial Engineering 5, 1–15 (2010)Google Scholar
  10. 10.
    Sheynikhovich, D., Chavarriaga, R., Strösslin, T., Gerstner, W.: Spatial Representation and Navigation in Bio-inspired Robot. In: Wermter, S., Palm, G., Elshaw, M. (eds.) Biomimetic Neural Learning. LNCS (LNAI), vol. 3575, pp. 245–264. Springer, Heidelberg (2005)Google Scholar
  11. 11.
    Ge, S.S., Cui, Y.J.: Dynamic Motion Planning for Mobile Robots Using Potential Field Method. Autonomous Robots 13(3), 207–222 (2002)CrossRefzbMATHGoogle Scholar
  12. 12.
    Arkin, R.C.: Behavior-based Robotics. MIT Press, Cambridge (1998)Google Scholar
  13. 13.
    Collett, T.S.: The Use of Visual Landmarks by Gerbils: Reaching a Goal When Landmarks are Displaced. Journal of Comparative Physiology A 160(1), 109–113 (1987)CrossRefGoogle Scholar
  14. 14.
    Dasgupta, S., Woergoetter, F., Morimoto, J., Manoonpong, P.: Neural Combinatorial Learning of Goal-directed Behavior with Reservoir Critic and Reward Modulated Hebbian Plasticity. In: 2013 IEEE International Conference on Systems, Man, and Cybernetics (SMC), pp. 993–1000 (2013)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Bassel Zeidan
    • 1
  • Sakyasingha Dasgupta
    • 2
  • Florentin Wörgötter
    • 2
  • Poramate Manoonpong
    • 2
    • 3
  1. 1.Faculty of Mathematics and Computer Science, Institute of Computer ScienceUniversity of GöttingenGöttingenGermany
  2. 2.Bernstein Center for Computational Neuroscience (BCCN)University of GöttingenGöttingenGermany
  3. 3.The Mærsk Mc-Kinney Møller InstituteUniversity of Southern DenmarkOdense MDenmark

Personalised recommendations