A spiking neural network model of spatial and visual mental imagery

Abstract

Mental imagery has long been of interest to the cognitive and neurosciences, but how it manifests itself in the mind and brain still remains unresolved. In pursuit of this, we built a spiking neural model that can perform mental rotation and mental map scanning using strategies informed by the psychology and neuroscience literature. Results: When performing mental map scanning, reaction times (RTs) for our model closely match behavioural studies (approx. 50 ms/cm), and replicate the cognitive penetrability of the task. When performing mental rotation, our model’s RTs once again closely match behavioural studies (model: 55–65°/s; studies: 60°/s), and performed the task using the same task strategy (whole unit rotation of simple and familiar objects through intermediary points). Overall, our model suggests: (1) vector-based approaches to neuro-cognitive modelling are well equipped to re-produce behavioural findings, and (2) the cognitive (in)penetrability of imagery tasks may depend on whether or not the task makes use of (non)symbolic processing.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

References

  1. Anderson JR (1996) Act: a simple theory of complex cognition. Am Psychol 51(4):355

    Article  Google Scholar 

  2. Bekolay T, Bergstra J, Hunsberger E, DeWolf T, Stewart TC, Rasmussen D, Choo X, Voelker A, Eliasmith C (2014) Nengo: a python tool for building large-scale functional brain models. Front Neuroinform 7:48

    Article  PubMed  PubMed Central  Google Scholar 

  3. Bethell-Fox CE, Shepard RN (1988) Mental rotation: effects of stimulus complexity and familiarity. J Exp Psychol Hum Percept Perform 14(1):12

    Article  Google Scholar 

  4. Borst G, Kievit RA, Thompson WL, Kosslyn SM (2011) Mental rotation is not easily cognitively penetrable. J Cognit Psychol 23(1):60–75

    Article  Google Scholar 

  5. Borst JP, Nijboer M, Taatgen N, Anderson JR (2013) A data-driven mapping of five act-r modules on the brain. In: Proceedings of the international conference on cognitive modeling (ICCM), vol 5, p 10

  6. Burgess N (2002) The hippocampus, space, and viewpoints in episodic memory. Q J Exp Psychol Sect A 55(4):1057–1080

    Article  Google Scholar 

  7. Cohen MS, Kosslyn SM, Breiter HC, DiGirolamo GJ, Thompson WL, Anderson A, Bookheimer S, Rosen BR, Belliveau J (1996) Changes in cortical activity during mental rotation a mapping study using functional mri. Brain 119(1):89–100

    Article  PubMed  Google Scholar 

  8. Cooper LA (1976) Demonstration of a mental analog of an external rotation. Percept Psychophys 19(4):296–302

    Article  Google Scholar 

  9. Denis M, Cocude M (1997) On the metric properties of visual images generated from verbal descriptions: evidence for the robustness of the mental scanning effect. Eur J Cognit Psychol 9(4):353–380

    Article  Google Scholar 

  10. Denis M, Gonc MR, Memmi D et al (1995) Mental scanning of visual images generated from verbal descriptions: towards a model of image accuracy. Neuropsychologia 33(11):1511–1530

    Article  CAS  PubMed  Google Scholar 

  11. Eliasmith C (2013) How to build a brain: a neural architecture for biological cognition. Oxford University Press, Oxford

    Google Scholar 

  12. Eliasmith C, Trujillo O (2014) The use and abuse of large-scale brain models. Curr Opin Neurobiol 25:1–6

    Article  CAS  PubMed  Google Scholar 

  13. Eliasmith C, Stewart TC, Choo X, Bekolay T, DeWolf T, Tang Y, Rasmussen D (2012) A large-scale model of the functioning brain. Science 338(6111):1202–1205

    Article  CAS  PubMed  Google Scholar 

  14. Eliasmith C, Gosmann J, Choo X (2016) Biospaun: a large-scale behaving brain model with complex neurons. arXiv preprint arXiv:160205220

  15. Farah MJ, Hammond KM, Levine DN, Calvanio R (1988) Visual and spatial mental imagery: Dissociable systems of representation. Cogn Psychol 20(4):439–462

    Article  CAS  PubMed  Google Scholar 

  16. Hegarty M, Waller D (2004) A dissociation between mental rotation and perspective-taking spatial abilities. Intelligence 32(2):175–191

    Article  Google Scholar 

  17. Herweg NA, Kahana MJ (2018) Spatial representations in the human brain. Front Hum Neurosci 12:297

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  18. Hölscher C, Tenbrink T, Wiener JM (2011) Would you follow your own route description? Cognitive strategies in urban route planning. Cognition 121(2):228–247

    Article  PubMed  Google Scholar 

  19. Hummel JE, Holyoak KJ (2003) Relational reasoning in a neurally-plausible cognitive architecture: an overview of the lisa project. Cognit Stud 10(1):58–75

    Google Scholar 

  20. Hummel JE, Holyoak KJ (2005) Relational reasoning in a neurally plausible cognitive architecture: an overview of the lisa project. Curr Dir Psychol Sci 14(3):153–157

    Article  Google Scholar 

  21. Isomura Y, Harukuni R, Takekawa T, Aizawa H, Fukai T (2009) Microcircuitry coordination of cortical motor information in self-initiation of voluntary movements. Nat Neurosci 12(12):1586

    Article  CAS  PubMed  Google Scholar 

  22. Jolicoeur P (1990) Identification of disoriented objects: a dual-systems theory. Mind Lang 5(4):387–410

    Article  Google Scholar 

  23. Just MA, Carpenter PA (1985) Cognitive coordinate systems: accounts of mental rotation and individual differences in spatial ability. Psychol Rev 92(2):137

    Article  CAS  PubMed  Google Scholar 

  24. Kajić I, Gosmann J, Stewart TC, Wennekers T, Eliasmith C (2017) A spiking neuron model of word associations for the remote associates test. Front psychol 8:99

    Article  PubMed  PubMed Central  Google Scholar 

  25. Kemps E (1999) Effects of complexity on visuo-spatial working memory. Eur J Cognit Psychol 11(3):335–356

    Article  Google Scholar 

  26. Kemps E (2001) Complexity effects in visuo-spatial working memory: implications for the role of long-term memory. Memory 9(1):13–27

    Article  CAS  PubMed  Google Scholar 

  27. Komer B, Stewart TC, Voelker AR, Eliasmith C (2019) A neural representation of continuous space using fractional binding. In: 41st annual meeting of the Cognitive Science Society, Cognitive Science Society, Montreal, QC, pp 2038–2044

  28. Kosslyn SM (1987) Seeing and imagining in the cerebral hemispheres: a computational approach. Psychol Rev 94(2):148

    Article  CAS  PubMed  Google Scholar 

  29. Kosslyn SM, Ball TM, Reiser BJ (1978) Visual images preserve metric spatial information: evidence from studies of image scanning. J Exp Psychol Hum Percept Perform 4(1):47

    Article  CAS  PubMed  Google Scholar 

  30. Kosslyn SM, Koenig O, Barrett A, Cave CB, Tang J, Gabrieli JD (1989) Evidence for two types of spatial representations: hemispheric specialization for categorical and coordinate relations. J Exp Psychol Hum Percept Perform 15(4):723

    Article  CAS  PubMed  Google Scholar 

  31. Kosslyn SM, Thompson WL, Ganis G (2006) The case for mental imagery. Oxford University Press, Oxford

    Google Scholar 

  32. Lambrey S, Amorim MA, Samson S, Noulhiane M, Hasboun D, Dupont S, Baulac M, Berthoz A (2008) Distinct visual perspective-taking strategies involve the left and right medial temporal lobe structures differently. Brain 131(2):523–534

    Article  CAS  PubMed  Google Scholar 

  33. Lund JS, Yoshioka T, Levitt JB (1993) Comparison of intrinsic connectivity in different areas of macaque monkey cerebral cortex. Cereb Cortex 3(2):148–162

    Article  CAS  PubMed  Google Scholar 

  34. Marmor GS, Zaback LA (1976) Mental rotation by the blind: does mental rotation depend on visual imagery? J Exp Psychol Hum Percept Perform 2(4):515

    Article  CAS  PubMed  Google Scholar 

  35. Marr D, Hildreth E (1980) Theory of edge detection. Proc R Soc Lond B 207(1167):187–217

    Article  CAS  PubMed  Google Scholar 

  36. McKinstry JL, Fleischer JG, Chen Y, Gall WE, Edelman GM (2016) Imagery may arise from associations formed through sensory experience: a network of spiking neurons controlling a robot learns visual sequences in order to perform a mental rotation task. PLoS ONE 11(9):e0162155

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  37. Mellet E, Bricogne S, Tzourio-Mazoyer N, Ghaem O, Petit L, Zago L, Etard O, Berthoz A, Mazoyer B, Denis M (2000) Neural correlates of topographic mental exploration: the impact of route versus survey perspective learning. Neuroimage 12(5):588–600

    Article  CAS  PubMed  Google Scholar 

  38. Mora-Sánchez A, Dreyfus G, Vialatte FB (2019) Scale-free behaviour and metastable brain-state switching driven by human cognition, an empirical approach. In: Cognitive neurodynamics, 1–16

  39. Morrison RG, Doumas LA, Richland LE (2011) A computational account of children’s analogical reasoning: balancing inhibitory control in working memory and relational representation. Dev Sci 14(3):516–529

    Article  PubMed  Google Scholar 

  40. Murray JE (1997) Flipping and spinning: spatial transformation procedures in the identification of rotated natural objects. Memory Cognit 25(1):96–105

    Article  CAS  Google Scholar 

  41. Putrino D, Brown EN, Mastaglia FL, Ghosh S (2010) Differential involvement of excitatory and inhibitory neurons of cat motor cortex in coincident spike activity related to behavioral context. J Neurosci 30(23):8048–8056

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  42. Pylyshyn ZW (1981) The imagery debate: analogue media versus tacit knowledge. Psychol Rev 88(1):16

    Article  Google Scholar 

  43. Pylyshyn ZW (2003) Seeing and visualizing: It’s not what you think. MIT press, Boston

    Google Scholar 

  44. Rao AR (2018) An oscillatory neural network model that demonstrates the benefits of multisensory learning. Cogn Neurodyn 12(5):481–499

    Article  PubMed  PubMed Central  Google Scholar 

  45. Richter W, Somorjai R, Summers R, Jarmasz M, Menon RS, Gati JS, Georgopoulos AP, Tegeler C, Ugurbil K, Kim SG (2000) Motor area activity during mental rotation studied by time-resolved single-trial fmri. J Cogn Neurosci 12(2):310–320

    Article  CAS  PubMed  Google Scholar 

  46. Rosenbloom PS (2011a) Mental imagery in a graphical cognitive architecture. In: BICA, pp 314–323

  47. Rosenbloom PS (2011b) Rethinking cognitive architecture via graphical models. Cogn Syst Res 12(2):198–209

    Article  Google Scholar 

  48. Shepard RN, Metzler J (1971) Mental rotation of three-dimensional objects. Science 171(3972):701–703

    Article  CAS  PubMed  Google Scholar 

  49. Stewart TC, Choo X, Eliasmith C, et al. (2010) Dynamic behaviour of a spiking model of action selection in the basal ganglia. In: Proceedings of the 10th international conference on cognitive modeling, Citeseer, pp 235–40

  50. Stewart TC, Tang Y, Eliasmith C (2011) A biologically realistic cleanup memory: autoassociation in spiking neurons. Cogn Syst Res 12(2):84–92

    Article  Google Scholar 

  51. Tarr MJ (1995) Rotating objects to recognize them: a case study on the role of viewpoint dependency in the recognition of three-dimensional objects. Psychon Bull Rev 2(1):55–82

    Article  CAS  PubMed  Google Scholar 

  52. Tarr MJ, Pinker S (1989) Mental rotation and orientation-dependence in shape recognition. Cogn Psychol 21(2):233–282

    Article  CAS  PubMed  Google Scholar 

  53. Thompson WL, Slotnick SD, Burrage MS, Kosslyn SM (2009) Two forms of spatial imagery: neuroimaging evidence. Psychol Sci 20(10):1245–1253

    Article  PubMed  Google Scholar 

  54. Tozzi A, Peters JF (2017) From abstract topology to real thermodynamic brain activity. Cogn Neurodyn 11(3):283–292

    Article  PubMed  PubMed Central  Google Scholar 

  55. Xie S, Tu Z (2015) Holistically-nested edge detection. In: Proceedings of the IEEE international conference on computer vision, pp 1395–1403

  56. Zacks JM (2008) Neuroimaging studies of mental rotation: a meta-analysis and review. J Cogn Neurosci 20(1):1–19

    Article  PubMed  Google Scholar 

  57. Zhang T, Pan X, Xu X, Wang R (2019) A cortical model with multi-layers to study visual attentional modulation of neurons at the synaptic level. In: Cognitive neurodynamics, pp 1–21

Download references

Acknowledgements

The authors would like to thank the three anonymous reviewers and the journal editors for their thoughtful and detailed comments, as well as everyone at the Centre for Theoretical Neuroscience at the University of Waterloo for providing a wealth of learning materials.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Sean N. Riley.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Riley, S.N., Davies, J. A spiking neural network model of spatial and visual mental imagery. Cogn Neurodyn 14, 239–251 (2020). https://doi.org/10.1007/s11571-019-09566-5

Download citation

Keywords

  • Mental imagery
  • Map scanning
  • Mental rotation
  • Visual imagery
  • Spatial imagery
  • Visuospatial imagery