Skip to main content
SpringerLink
Log in

Comparative Experimental Studies on Spatial Memory and Learning in Rats and Robots

  • Published:
Journal of Intelligent & Robotic Systems Aims and scope Submit manuscript

Abstract

The study of behavioral and neurophysiological mechanisms involved in rat spatial cognition provides a basis for the development of computational models and robotic experimentation of goal-oriented learning tasks. These models and robotics architectures offer neurobiologists and neuroethologists alternative platforms to study, analyze and predict spatial cognition based behaviors. In this paper we present a comparative analysis of spatial cognition in rats and robots by contrasting similar goal-oriented tasks in a cyclical maze, where studies in rat spatial cognition are used to develop computational system-level models of hippocampus and striatum integrating kinesthetic and visual information to produce a cognitive map of the environment and drive robot experimentation. During training, Hebbian learning and reinforcement learning, in the form of Actor-Critic architecture, enable robots to learn the optimal route leading to a goal from a designated fixed location in the maze. During testing, robots exploit maximum expectations of reward stored within the previously acquired cognitive map to reach the goal from different starting positions. A detailed discussion of comparative experiments in rats and robots is presented contrasting learning latency while characterizing behavioral procedures during navigation such as errors associated with the selection of a non-optimal route, body rotations, normalized length of the traveled path, and hesitations. Additionally, we present results from evaluating neural activity in rats through detection of the immediate early gene Arc to verify the engagement of hippocampus and striatum in information processing while solving the cyclical maze task, such as robots use our corresponding models of those neural structures.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Roberts, W.A.: Principles of Animal Cognition, pp. 201–230. McGraw Hill, USA (1998)

    Google Scholar 

  2. Hebb, D.O.: The Organization of Behavior: A Neuropsychological Theory. Wiley-Interscience, New York (1949)

    Google Scholar 

  3. Barto, A.G.: Adaptive critics and the basal ganglia. In: Houk, J.C., Davis, J.L., Beiser, D. (eds.) Models of Information Processing in the Basal Ganglia, pp. 215–232. MIT Press, Cambridge (1995)

    Google Scholar 

  4. O’Keefe, J.: Spatial memory within and without the hippocampal system. In: Seifert, W. (ed.) Neurobiology of the Hippocampus, pp. 375–403. Academic Press, New York (1983)

    Google Scholar 

  5. Barrera, A., Weitzenfeld, A.: Biologically-inspired robot spatial cognition based on rat neurophysiological studies. Auton. Robots, Springer 25(1–2), 147–169 (2008)

    Article  Google Scholar 

  6. Morris, R.G.M.: Spatial localization does not require the presence of local cues. Learn. Motiv. 12, 239–260 (1981)

    Article  Google Scholar 

  7. Barrera, A., Weitzenfeld, A.: Rat-inspired model of robot target learning and place recognition. In: Proceedings of the 15th Mediterranean Conference on Control and Automation—MED. Athens, Greece (2007)

  8. Arkin, R.C.: Behavioral Based Robotics. MIT Press (1998)

  9. Webb, B.: What does robotics offer animal behaviour? Anim. Behav. 60, 545–558 (2000)

    Article  Google Scholar 

  10. Weitzenfeld, A.: A prey catching and predator avoidance neural-schema architecture for single and multiple robots. J. Intell. Robot. Syst., Springer 51(2), 203–233 (2008)

    Article  Google Scholar 

  11. Weitzenfeld, A.: From schemas to neural networks: a multi-level modeling approach to biologically-inspired autonomous robotic systems. J. Robot. Auton. Syst. 56(2), 177–197 (2008)

    Article  Google Scholar 

  12. O’Keefe, J., Nadel, L.: The Hippocampus as a Cognitive Map. Oxford University Press (1978)

  13. O’Keefe, J., Dostrovsky, J.: The hippocampus as a spatial map: preliminary evidence from unit activity in the freely moving rat. Brain Res. 34(1), 171–175 (1971)

    Article  Google Scholar 

  14. Tolman, E.: Cognitive maps in rats and men. Psychol. Rev. 55, 189–208 (1948)

    Article  Google Scholar 

  15. McNaughton, B.L., Knierim, J.J., Wilson, M.A.: Vector encoding and the vestibular foundations of spatial cognition. In: Gazzaniga, M. (ed.) The Cognitive Neurosciences, pp. 585–595. MIT Press, Boston (1994)

    Google Scholar 

  16. O’Keefe, J., Conway, D.H.: Hippocampal place units in the freely moving rat: why they fire where they fire. Exp. Brain Res. 31, 573–590 (1978)

    Google Scholar 

  17. Quirk, G.J., Muller, R.U., Kubie, J.L.: The firing of hippocampal place cells in the dark depends on the rat’s recent experience. J. Neurosci. 10(6), 2008–2017 (1990)

    Google Scholar 

  18. Jeffery, K.J., O’Keefe, J.M.: Learned interaction of visual and idiothetic cues in the control of place field orientation. Exp. Brain Res. 127, 151–161 (1999)

    Article  Google Scholar 

  19. Poucet, B.: Spatial cognitive maps in animals: new hypotheses on their structure and neural mechanisms. Psychol. Rev. 100(2), 163–182 (1993)

    Article  Google Scholar 

  20. Moser, E.I., Kropff, E., Moser, M.-B.: Place cells, grid cells, and the brain’s spatial representation system. Annu. Rev. Neurosci. 31, 69–89 (2008)

    Article  Google Scholar 

  21. Risold, P., Thompson, R., Swanson, L.: The structural organization of connections between hypothalamus and cerebral cortex. Brains Res. Rev. 24(2–3), 197–254 (1997)

    Article  Google Scholar 

  22. Kelley, A.: Ventral striatal control of appetitive motivation: role in ingestive behavior and reward-related learning. Neurosci. Biobehav. Rev. 27(8), 765–776 (2004)

    Article  Google Scholar 

  23. Schultz, W., Tremblay, L., Hollerman, J.: Reward prediction in primate basal ganglia and frontal cortex. Neuropharmacology 37(4–5), 421–429 (1998)

    Article  Google Scholar 

  24. Schultz, W., Tremblay, L., Hollerman, J.: Reward processing in primate orbitofrontal cortex and basal ganglia. Cereb. Cortex 10(3), 272–283 (2000)

    Article  Google Scholar 

  25. Houk, J.C., Adams, J.L., Barto, A.G.: A model of how the basal ganglia generate and use neural signals that predict reinforcement. In: Houk, J.C., Davis, J.L., Beiser, D.G. (eds.) Models of Information Processing in the Basal Ganglia, pp. 249–270. MIT Press, Cambridge (1995)

    Google Scholar 

  26. Collett, T.S., Graham, P.: Animal navigation: path integration, visual landmarks and cognitive maps. Curr. Biol. 14(12), R475–7 (2004)

    Article  Google Scholar 

  27. Hartley, T., Burgess, N.: Complementary memory systems: competition, cooperation and compensation. Trends Neurosci. 28(4), 169–170 (2005)

    Article  Google Scholar 

  28. Lanahan, A., Worley, P.: Immediate-early genes and synaptic function. Neurobiol. Learn. Mem. 70(1–2), 37–43 (1998)

    Article  Google Scholar 

  29. Guzowski, J.F., McNaughton, B.L., Barnes, C.A., Worley, P.F.: Environment-specific expression of the immediate-early gene Arc in hippocampal neuronal ensembles. Nat. Neurosci. 2(12), 1120–1124 (1999)

    Article  Google Scholar 

  30. Ramirez-Amaya, V., Vazdarjanova, A., Mikhael, D., Rosi, S., Worley, P.F., Barnes, C.A.: Spatial exploration-induced arc mRNA and protein expression: evidence for selective, network-specific reactivation. J. Neurosci. 25, 1761–1768 (2005)

    Article  Google Scholar 

  31. Vazdarjanova, A., Ramirez-Amaya, V., Insel, N., Plummer, T.K., Rosi, S., Chowdhury, S., Mikhael, D., Worley, P.F., Guzowski, J.F., Barnes, C.A.: Spatial exploration induces ARC, a plasticity-related immediate-early gene, only in calcium/calmodulin-dependent protein kinase II-positive principal excitatory and inhibitory neurons of the rat forebrain. J. Comp. Neurol. 498(3), 317–329 (2006)

    Article  Google Scholar 

  32. Burgess, N., Recce, M., O’Keefe, J.: A model of hippocampal function. Neural Netw. 7(6–7), 1065–1081 (1994)

    Article  MATH  Google Scholar 

  33. Brown, M.A., Sharp, P.E.: Simulation of spatial learning in the Morris Water Maze by a neural network model of the hippocampal formation and nucleus accumbens. Hippocampus 5, 171–188 (1995)

    Article  Google Scholar 

  34. Redish, A., Touretzky, D.: Cognitive maps beyond the hippocampus. Hippocampus 7(1), 15–35 (1997)

    Article  Google Scholar 

  35. Guazzelli, A., Corbacho, F.J., Bota, M., Arbib, M.A.: Affordances, motivation, and the world graph theory. Adapt. Behav. 6(3–4), 435–471 (1998)

    Article  Google Scholar 

  36. Arleo, A., Gerstner, W.: Spatial cognition and neuro-mimetic navigation: a model of hippocampal place cell activity. Biol. Cybern. 83, 287–299 (2000)

    Article  Google Scholar 

  37. Gaussier, P., Revel, A., Banquet, J.P., Babeau, V.: From view cells and place cells to cognitive map learning: processing stages of the hippocampal system. Biol. Cybern. 86, 15–28 (2002)

    Article  MATH  Google Scholar 

  38. Filliat, D., Meyer, J.-A.: Global localization and topological map learning for robot navigation. In: Hallam et al. (eds.) From Animals to Animats 7 Proceedings of the Seventh International Conference on Simulation of Adaptive Behavior, pp 131–140. The MIT Press (2002)

  39. Arleo, A., Smeraldi, F., Gerstner, W.: Cognitive navigation based on nonuniform Gabor space sampling, unsupervised growing networks, and reinforcement learning. IEEE Trans. Neural Netw. 15(3), 639–652 (2004)

    Article  Google Scholar 

  40. Milford, M., Wyeth, G.: Spatial mapping and map exploitation: a bio-inspired engineering perspective. In: Winter, S., Duckham, M., Kulik, L., Kuipers, B. (eds.) Spatial Information Theory, pp. 203–221. Springer-Verlag, Heidelberg (2007)

    Chapter  Google Scholar 

  41. Parron, C., Save, E.: Evidence for entorhinal and parietal cortices involvement in path integration in the rat. Exp. Brain Res. 159(3), 349–359 (2004)

    Article  Google Scholar 

  42. Cho, J., Sharp, P.: Head direction, place, and movement correlates for cells in the rat retrosplenial cortex. Behav. Neurosci. 115(1), 3–25 (2001)

    Article  Google Scholar 

  43. McNaughton, B., Mizumori, S., Barnes, C., Leonard, B., Marquis, M., Green, E.: Cortical representation of motion during unrestrained spatial navigation in the rat. Cereb. Cortex 4, 27–39 (1994)

    Article  Google Scholar 

  44. Granon, S., Poucet, B.: Involvement of the rat prefrontal cortex in cognitive functions: a central role for the prelimbic area. Psychobiology 28(2), 229–237 (2000)

    Google Scholar 

  45. Foster, D.J., Morris, R.G.M., Dayan, P.: A model of hippocampally dependent navigation, using the temporal difference learning rule. Hippocampus 10, 1–16 (2000)

    Article  Google Scholar 

  46. Hull, C.L.: The goal gradient hypothesis and maze learning. Psychol. Rev. 39, 25–43 (1932)

    Article  Google Scholar 

  47. Weitzenfeld, A., Arbib, M., Alexander, A.: The neural simulation language. MIT Press, Cambridge (2002)

    Google Scholar 

  48. Webb, B.: Can robots make good models of biological behaviour? Behav. Brain Sci. 24, 1033–1050 (2001)

    Google Scholar 

  49. Ranck, J.B., Jr.: Head-direction cells in the deep layers of dorsal presubiculum in freely moving rats. Soc. Neurosci. Abstr. 10, 599 (1984)

    Google Scholar 

  50. Leutgeb, S., Leutgeb, J.K.: Pattern separation, pattern completion, and new neuronal codes within a continuous CA3 map. Learn. Mem. 14(11), 745–757 (2007)

    Article  Google Scholar 

  51. Ramirez-Amaya, V., Marrone, D.F., Gage, F.H., Worley, P.F., Barnes, C.A.: Integration of new neurons into functional neural networks. J. Neurosci. 26(47), 12237–12241 (2006)

    Article  Google Scholar 

  52. Guzowski, J.F., Knierim, J.J., Moser, E.I.: Ensemble dynamics of hippocampal regions CA3 and CA1. Neuron 44(4), 581–584 (2004)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computer Engineering Department–Robotics and Biorobotics Laboratories, Instituto Tecnológico Autónomo de México (ITAM), Río Hondo #1, Progreso Tizapán, CP 01080, México, DF, México

    Alejandra Barrera & Alfredo Weitzenfeld

  2. Neurobiology Institute, Plastic Neural Networks Laboratory, Universidad Nacional Autónoma de México (UNAM), Boulevard Juriquilla #3001, CP 76230, Querétaro, México

    Alejandra Cáceres & Victor Ramirez-Amaya

Authors
  1. Alejandra Barrera
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alejandra Cáceres
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alfredo Weitzenfeld
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Victor Ramirez-Amaya
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alejandra Barrera.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Barrera, A., Cáceres, A., Weitzenfeld, A. et al. Comparative Experimental Studies on Spatial Memory and Learning in Rats and Robots. J Intell Robot Syst 63, 361–397 (2011). https://doi.org/10.1007/s10846-010-9467-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10846-010-9467-y

Keywords

Search

Navigation