Autonomous Robots

, Volume 25, Issue 1–2, pp 147–169 | Cite as

Biologically-inspired robot spatial cognition based on rat neurophysiological studies

Article

Abstract

This paper presents a robot architecture with spatial cognition and navigation capabilities that captures some properties of the rat brain structures involved in learning and memory. This architecture relies on the integration of kinesthetic and visual information derived from artificial landmarks, as well as on Hebbian learning, to build a holistic topological-metric spatial representation during exploration, and employs reinforcement learning by means of an Actor-Critic architecture to enable learning and unlearning of goal locations. From a robotics perspective, this work can be placed in the gap between mapping and map exploitation currently existent in the SLAM literature. The exploitation of the cognitive map allows the robot to recognize places already visited and to find a target from any given departure location, thus enabling goal-directed navigation. From a biological perspective, this study aims at initiating a contribution to experimental neuroscience by providing the system as a tool to test with robots hypotheses concerned with the underlying mechanisms of rats’ spatial cognition. Results from different experiments with a mobile AIBO robot inspired on classical spatial tasks with rats are described, and a comparative analysis is provided in reference to the reversal task devised by O’Keefe in 1983.

Keywords

Goal-oriented navigation Hebbian learning Place recognition Rat’s hippocampus Reinforcement learning Spatial cognition Target learning Target unlearning Topological-metric mapping 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Arbib, M. A., & Lieblich, I. (1977). Motivational learning of spatial behavior. In J. Metzler (Ed.), Systems neuroscience (pp. 221–239). New York: Academic Press. Google Scholar
  2. Arleo, A., Smeraldi, F., & Gerstner, W. (2004). Cognitive navigation based on nonuniform Gabor space sampling, unsupervised growing networks, and reinforcement learning. IEEE Transactions on Neural Networks, 15(3), 639–652. CrossRefGoogle Scholar
  3. Barrera, A., & Weitzenfeld, A. (2006). Return of the rat: Biologically-inspired robotic exploration and navigation. In Proceedings of the 1st IEEE / RAS-EMBS international conference on biomedical robotics and biomechatronics (BioRob), Pisa, Italy. Google Scholar
  4. Barrera, A., & Weitzenfeld, A. (2007). Rat-inspired model of robot target learning and place recognition. In Proceedings of the 15th Mediterranean conference on control and automation (MED), Athens, Greece. Google Scholar
  5. Barto, A. G. (1995). Adaptive critics and the basal ganglia. In J. C. Houk, J. L. Davis, & D. Beiser (Eds.), Models of information processing in the basal ganglia (pp. 215–232). Cambridge: MIT Press. Google Scholar
  6. Bosse, M., Newman, P., Leonard, J., & Teller, S. (2004). SLAM in large-scale cyclic environments using the Atlas Framework. International Journal on Robotics Research, 23(12), 1113–1139. CrossRefGoogle Scholar
  7. Burgess, N., Recce, M., & O’Keefe, J. (1994). A model of hippocampal function. Neural Networks, 7(6/7), 1065–1081. CrossRefMATHGoogle Scholar
  8. Cho, J., & Sharp, P. (2001). Head direction, place, and movement correlates for cells in the rat retrosplenial cortex. Behavioral Neuroscience, 115(1), 3–25. CrossRefGoogle Scholar
  9. Collett, T. S., Cartwright, B. A., & Smith, B. A. (1986). Landmark learning and visuo-spatial memories in gerbils. Journal of Comparative Physiology A, 158, 835–851. CrossRefGoogle Scholar
  10. Cooper, B., & Mizumori, S. (1999). Retrosplenial cortex inactivation selectively impairs navigation in darkness. Neuroreport, 10(3), 625–630. CrossRefGoogle Scholar
  11. Dominey, P. F., & Arbib, M. A. (1992). A cortico-subcortical model for generation of spatially accurate sequential saccades. Cerebral Cortex, 2, 135–175. CrossRefGoogle Scholar
  12. Etienne, A., & Jeffery, K. (2004). Path integration in mammals. Hippocampus, 14(2), 180–192. CrossRefGoogle Scholar
  13. Filliat, D., & Meyer, J.-A. (2002). Global localization and topological map learning for robot navigation. In J. Hallam et al. (Eds.), From animals to animats 7. Proceedings of the seventh international conference on simulation of adaptive behavior (pp. 131–140). Cambridge: MIT Press. Google Scholar
  14. Folkesson, J., & Christensen, H. (2004). Graphical SLAM—a self-correcting map. In Proceedings of IEEE international conference on robotics and automation (ICRA), New Orleans, USA. Google Scholar
  15. Franz, M. O., Schölkopf, B., Mallot, H. A., & Bülthoff, H. (1998). Learning view graphs for robot navigation. Autonomous Robots, 5, 111–125. CrossRefGoogle Scholar
  16. Frese, U. (2006). A discussion of simultaneous localization and mapping. Autonomous Robots, 20, 25–42. CrossRefGoogle Scholar
  17. Gaussier, P., Revel, A., Banquet, J. P., & Babeau, V. (2002). From view cells and place cells to cognitive map learning: Processing stages of the hippocampal system. Biological Cybernetics, 86, 15–28. CrossRefMATHGoogle Scholar
  18. Gibson, J. J. (1966). The senses considered as perceptual systems. Boston: Houghton Mifflin. Google Scholar
  19. Gothard, K. M., Skaggs, W. E., & McNaughton, B. L. (1996). Dynamics of mismatch correction in the hippocampal ensemble code for space: Interaction between path integration and environmental cues. Journal of Neuroscience, 16(24), 8027–8040. Google Scholar
  20. Grace, A., Floresco, S., Goto, Y., & Lodge, D. (2007). Regulation of firing dopaminergic neurons and control of goal-directed behaviors. Trends in Neurosciences, 30(5), 220–227. CrossRefGoogle Scholar
  21. Granon, S., & Poucet, B. (2000). Involvement of the rat prefrontal cortex in cognitive functions: A central role for the prelimbic area. Psychobiology, 28(2), 229–237. Google Scholar
  22. Guazzelli, A., Corbacho, F. J., Bota, M., & Arbib, M. A. (1998). Affordances, motivation, and the world graph theory. Adaptive Behavior, 6(3/4), 435–471. CrossRefGoogle Scholar
  23. Guivant, J., Nebot, E., Nieto, J., & Masson, F. (2004). Navigation and mapping in large unstructured environments. The International Journal of Robotics Research, 23(4), 449–472. CrossRefGoogle Scholar
  24. Guzowski, J., & Knierim, J., Moser, E. (2004). Ensemble dynamics of hippocampal regions CA3 and CA1. Neuron, 44, 581–584. CrossRefGoogle Scholar
  25. Hähnel, D., Burgard, W., Wegbreit, B., & Thrun, S. (2003). Towards lazy data association in SLAM. In Proceedings of the 11th international symposium of robotics research (ISRR), Sienna, Italy. Google Scholar
  26. Hebb, D. O. (1949). The organization of behavior: A neuropsychological theory. New York: Wiley. Google Scholar
  27. Hernández-Rabaza, V., Barcia, J., Llorens-Martín, M., Trejo, J., & Canales, J. (2007). Spared place and object-place learning but limited spatial working memory capacity in rats with selective lesions of the dentate gyrus. Brain Research Bulletin, 72(4–6), 315–323. CrossRefGoogle Scholar
  28. Hollup, S. A., Kjelstrup, K. G., Hoff, J., Moser, M., & Moser, E. I. (2001a). Impaired recognition of the goal location during spatial navigation in rats with hippocampal lesions. The Journal of Neuroscience, 21(12), 4505–4513. Google Scholar
  29. Hollup, S. A., Molden, S., Donnett, J. G., Moser, M., & Moser, E. I. (2001b). Place fields of rat hippocampal pyramidal cells and spatial learning in the watermaze. European Journal of Neuroscience, 13, 1197–1208. CrossRefGoogle Scholar
  30. Houk, J. C., Adams, J. L., & Barto, A. G. (1995). A model of how the basal ganglia generate and use neural signals that predict reinforcement. In J. C. Houk, J. L. Davis, & D. G. Beiser (Eds.), Models of information processing in the basal ganglia (pp. 249–270). Cambridge: MIT Press. Google Scholar
  31. Jeffery, K. J., & O’Keefe, J. M. (1999). Learned interaction of visual and idiothetic cues in the control of place field orientation. Experimental Brain Research, 127, 151–161. CrossRefGoogle Scholar
  32. Kelley, A. (2004). Ventral striatal control of appetitive motivation: role in ingestive behavior and reward-related learning. Neuroscience and Biobehavioral Reviews, 27(8), 765–776. CrossRefGoogle Scholar
  33. Kuipers, B., Modayil, J., Beeson, P., MacMahon, M., & Savelli, F. (2004). Local metrical and global topological maps in the Hybrid Spatial Semantic Hierarchy. In Proceedings of IEEE international conference on robotics and automation (ICRA), New Orleans, USA. Google Scholar
  34. McNaughton, B., Mizumori, S., Barnes, C., Leonard, B., Marquis, M., & Green, E. (1994). Cortical representation of motion during unrestrained spatial navigation in the rat. Cerebral Cortex, 4, 27–39. CrossRefGoogle Scholar
  35. Milford, M., Wyeth, G., & Prasser, D. (2006). RatSLAM on the edge: Revealing a coherent representation from an overloaded rat brain. In Proceedings of IEEE/RSJ international conference on intelligent robots and systems (IROS), Beijing, China (pp. 4060–4065). Google Scholar
  36. Mittelstaedt, M., & Mittelstaedt, H. (1982). Homing by path integration in a mammal. In F. Papi & H. G. Wallraff (Eds.), Avian navigation (pp. 290–297). Berlin: Springer. Google Scholar
  37. Morris, R. G. M. (1981). Spatial localization does not require the presence of local cues. Learning and Motivation, 12, 239–260. CrossRefGoogle Scholar
  38. Movarec, H. P., & Elfes, A. (1985). High resolution maps from wide angle sonar. In Proceedings of IEEE international conference on robotics and automation (ICRA) (pp. 116–121). Google Scholar
  39. O’Keefe, J. (1983). Spatial memory within and without the hippocampal system. In W. Seifert (Ed.), Neurobiology of the hippocampus (pp. 375–403). New York: Academic Press. Google Scholar
  40. O’Keefe, J., & Nadel, L. (1978). The hippocampus as a cognitive map. London: Oxford University Press. Google Scholar
  41. Parron, C., & Save, E. (2004). Evidence for entorhinal and parietal cortices involvement in path integration in the rat. Experimental Brain Research, 159(3), 349–359. CrossRefGoogle Scholar
  42. Poucet, B. (1993). Spatial cognitive maps in animals: New hypotheses on their structure and neural mechanisms. Psychological Review, 100(2), 163–182. CrossRefGoogle Scholar
  43. Redish, A. (1997). Beyond the cognitive map. Ph.D. thesis, School of Computer Science, Carnegie Mellon University, Pittsburgh, PA. Google Scholar
  44. Redish, A., & Touretzky, D. (1997). Cognitive maps beyond the hippocampus. Hippocampus, 7(1), 15–35. CrossRefGoogle Scholar
  45. Risold, P., Thompson, R., & Swanson, L. (1997). The structural organization of connections between hypothalamus and cerebral cortex. Brain Research Reviews, 24(2–3), 197–254. CrossRefGoogle Scholar
  46. Schultz, W., Tremblay, L., & Hollerman, J. (1998). Reward prediction in primate basal ganglia and frontal cortex. Neuropharmacology, 37(4–5), 421–429. CrossRefGoogle Scholar
  47. Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning: An introduction. Cambridge: MIT Press. Google Scholar
  48. Tolman, E. (1948). Cognitive maps in rats and men. Psychological Review, 55, 189–208. CrossRefGoogle Scholar
  49. Weitzenfeld, A., Arbib, M., & Alexander, A. (2002). The neural simulation language. Cambridge: MIT Press. Google Scholar
  50. Zivkovic, Z., Bakker, B., & Kröse, B. (2005). Hierarchical map building using visual landmarks and geometric constraints. In Proceedings of IEEE/RSJ international conference on intelligent robots and systems (IROS), Edmonton, Canada (pp. 7–12). Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2007

Authors and Affiliations

  1. 1.Computer Engineering DepartmentRobotics and CANNES Laboratories at the Instituto Tecnológico Autónomo de MéxicoMéxico DFMexico

Personalised recommendations