Predictions and Incongruency in Object Recognition: A Cognitive Neuroscience Perspective

  • Helena Yardley
  • Leonid Perlovsky
  • Moshe Bar
Part of the Studies in Computational Intelligence book series (SCI, volume 384)


The view presented here is that visual object recognition is the product of interaction between perception and cognition, a process budding from memories of past events. Our brain is proactive, in that it is continuously aiming to predict how the future will unfold, and this inherent ability allows us to function optimally in our environment. Our memory serves as a database from which we can form analogies from our past experiences, and apply them to the current moment. We are able to recognize an object through top-down and bottom-up processing pathways, which integrate to facilitate successful and timely object recognition. Specifically, it is argued that when we encounter an object our brain asks “what is this like” and therefore draws on years of experience, stored in memory, to generate predictions that directly facilitate perception. These feed-forward and feed-back systems tune our perceptive and cognitive faculties based on a number of factors: predictive astuteness, context, personal relevance of the given event, and the degree to which its potential rarity differs from our original expectations. We discuss both computational and theoretical models of object recognition, and review evidence to support the theory that we do not merely process incoming information serially, and that during our attempts to interpret the world around us, perception relies on existing knowledge as much as it does on incoming information.


Visual Cortex Object Recognition Repetition Priming Personal Relevance Incoming Information 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bar, M.: A cortical mechanism for triggering top-down facilitation in visual object recognition. Journal of Cognitive Neuroscience 15, 600–609 (2003)CrossRefGoogle Scholar
  2. 2.
    Bar, M.: Visual objects in context. Nature Reviews Neuroscience 5(8), 617–629 (2004)MathSciNetCrossRefGoogle Scholar
  3. 3.
    Bar, M.: The proactive brain: Using analogies and associations to generate predictions. Trends in Cognitive Sciences 11(7), 280–289 (2007)CrossRefGoogle Scholar
  4. 4.
    Bar, M. (ed.): Predictions: A universal principle in the operation of the human brain (Introduction). Theme issue: Predictions in the brain: using our past to generate a future. Philosophical Transactions of the Royal Society B: Biological Sciences (2009)Google Scholar
  5. 5.
    Bar, M., Aminoff, E.: Cortical analysis of visual context. Neuron 38(2), 347–358 (2003)CrossRefGoogle Scholar
  6. 6.
    Bar, M., Kassam, K.S., et al.: Top-down facilitation of visual recognition. Proc. Natl. Acad. Sci. USA 103(2), 449–454 (2006)CrossRefGoogle Scholar
  7. 7.
    Barbas, H.: Organization of cortical afferent input to orbitofrontal areas in the rhesus monkey. Neuroscience 56(4), 841–864 (1993)CrossRefGoogle Scholar
  8. 8.
    Barbas, H.: Anatomic basis of cognitive-emotional interactions in the primate prefrontal cortex. Neuroscience and Biobehavioral Reviews 19(3), 499–510 (1995)CrossRefGoogle Scholar
  9. 9.
    Barbas, H.: Connections underlying the synthesis of cognition, memory, and emotion in primate prefrontal cortices. Brain Research Bulletin 52(5), 319–330 (2000)CrossRefGoogle Scholar
  10. 10.
    Barbey, A.K., Krueger, F., et al.: Structured event complexes in the medial prefrontal cortex support counterfactual representations for future planning. Philos. Trans. R Soc. Lond. B Biol. Sci. 364(1521), 1291–1300 (2009)CrossRefGoogle Scholar
  11. 11.
    Barrett, L.F., Bar, M.: See it with feeling: affective predictions during object perception. Philos. Trans. R Soc. Lond. B Biol. Sci. 364(1521), 1325–1334 (2009)CrossRefGoogle Scholar
  12. 12.
    Barsalou, L.W.: Simulation, situated conceptualization, and prediction. Philosophical Transactions of the Royal Society B: Biological Sciences Special Theme Issue (2009)Google Scholar
  13. 13.
    Biederman, I., Mezzanotte, R.J., et al.: Scene perception: detecting and judging objects undergoing relational violations. Cognit. Psychol. 14(2), 143–177 (1982)CrossRefGoogle Scholar
  14. 14.
    Buckner, R., Goodman, J., et al.: Functional-anatomic correlates of object priming in humans revealed by rapid presentation event-related fMRI. Neuron 20, 285–296 (1998)CrossRefGoogle Scholar
  15. 15.
    Bullier, J.: Integrated model of visual processing. Brain Research Reviews 36, 96–107 (2001)CrossRefGoogle Scholar
  16. 16.
    Bullier, J., Nowak, L.G.: Parallel versus serial processing: New vistas on the distributed organization of the visual system. Current Opinion in Neurobiology 5, 497–503 (1995)CrossRefGoogle Scholar
  17. 17.
    Cardin, V., Friston, K.J., et al.: Top-down modulations in the visual form pathway revealed with dynamic causal modeling. Cereb. Cortex 21(3), 550–562 (2011)CrossRefGoogle Scholar
  18. 18.
    Carmichael, S.T., Price, J.L.: Limbic connections of the orbital and medial prefrontal cortex in macaque monkeys. Journal of Comparative Neurology 363(4), 615–641 (1995)CrossRefGoogle Scholar
  19. 19.
    Carpenter, G.A., Grossberg, S.: Neural dynamics of category learning and recognition: Attention, memory consolidation, and amnesia. In: Davis, J., Newburgh, R., Wegman, E. (eds.) Brain Structure, Learning, and Memory, pp. 233–290. Erlbaum, Hillsdale (1987)Google Scholar
  20. 20.
    Cavada, C., Company, T., et al.: The anatomical connections of the macaque monkey orbitofrontal cortex. A review. Cerebral Cortex 10(3), 220–242 (2000)CrossRefGoogle Scholar
  21. 21.
    Chun, M.M.: Contextual cueing of visual attention. Trends Cogn. Sci. 4(5), 170–178 (2000)CrossRefGoogle Scholar
  22. 22.
    Cutler, B.L., Penrod, S.D.: Context reinstatement and eyewitness identification. In: Davies, G.M., Thomson, D.M. (eds.) Memory in context: Context in memory. John Wiley & Sons Ltd, Chichester (1988)Google Scholar
  23. 23.
    Dale, A., Liu, A.K., et al.: Dynamic statistical parametric mapping: Combining fMRI and MEG for high-resolution imaging of cortical activity. Neuron 26(1), 55–67 (2000)CrossRefGoogle Scholar
  24. 24.
    den Ouden, H.E., Friston, K.J., et al.: A dual role for prediction error in associative learning. Cereb. Cortex 19(5), 1175–1185 (2009)CrossRefGoogle Scholar
  25. 25.
    DeValois, R.L., DeValois, K.K.: Spatial vision. Oxford Science Publications, New York (1988)Google Scholar
  26. 26.
    Dhond, R.P., Buckner, R.L., Dale, A.M., Marinkovic, K., Halgren, E.: Spatiotemporal maps of activity underlying word generation and their modification during repetition priming. J. Neurosci. 21, 3564–3571 (2001)Google Scholar
  27. 27.
    Dobbins, I.G., Schnyer, D.M., et al.: Cortical activiy reductions during repetition priming can result from rapid response learning. Nature 428(6980), 316–319 (2004)CrossRefGoogle Scholar
  28. 28.
    Downing, P.E., Jiang, Y., et al.: A cortical area selective for visual processing of the human body. Science 293(5539), 2470–2473 (2001)CrossRefGoogle Scholar
  29. 29.
    Dudai, Y.: The neurobiology of memory. Oxford University Press, Oxford (1989)Google Scholar
  30. 30.
    Egner, T., Monti, J.M., et al.: Expectation and surprise determine neural population responses in the ventral visual stream. J. Neurosci. 30(49), 16601–16608 (2010)CrossRefGoogle Scholar
  31. 31.
    Engel, A.K., Fries, P., et al.: Dynamic predictions: oscillations and synchrony in top-down processing. Nature Reviews Neuroscience 2(10), 704–716 (2001)CrossRefGoogle Scholar
  32. 32.
    Esterman, M., Yantis, S.: Perceptual expectation evokes category-selective cortical activity. Cereb. Cortex 20(5), 1245–1253 (2010)CrossRefGoogle Scholar
  33. 33.
    Felleman, D.J., Van Essen, V.C.: Distributed hierarchical processing in primate visual cortex. Cerebral Cortex 1, 1–47 (1991)CrossRefGoogle Scholar
  34. 34.
    Fenske, M.J., Aminoff, E., et al.: Top-down facilitation of visual object recognition: Object-based and context-based contributions. Progress in Brain Research 155, 3–21 (2006)CrossRefGoogle Scholar
  35. 35.
    Friston, K.: A theory of cortical responses. Philos. Trans. R. Soc. Lond. B. Biol. Sci. 360(1456), 815–836 (2005)CrossRefGoogle Scholar
  36. 36.
    Gabrieli, J.D., Poldrack, R.A., et al.: The role of left prefrontal cortex in language and memory. Proceedings of the National Academy of Sciences, USA 95(3), 906–913 (1998)CrossRefGoogle Scholar
  37. 37.
    Horton, J.C., Sincich, L.C.: A new foundation for the visual cortical hierarchy. In: Gazzaniga, M. (ed.) The Cognitive Neurosciences, 3rd edn., ch. 17, pp. 233–243 (2004)Google Scholar
  38. 38.
    Ghashghaei, H., Barbas, H.: Pathways for emotion: interactions of prefrontal and anterior temporal pathways in the amygdala of the rhesus monkey. Neuroscience 115, 1261–1279 (2002)CrossRefGoogle Scholar
  39. 39.
    Ghuman, A., Bar, M., Dobbins, I.G., Schnyer, D.M.: The effects of priming on frontal-temporal communication. Proceedings of the National Academy of Science 105(24), 8405–8409 (2008)CrossRefGoogle Scholar
  40. 40.
    Goldman-Rakic, P.S., Porrino, L.J.: The primate mediodorsal (MD) nucleus and its projection to the frontal lobe. Journal of Comparative Neurology 242(4), 535–560 (1985)CrossRefGoogle Scholar
  41. 41.
    Grill-Spector, K., Kourtzi, Z., et al.: The lateral occipital complex and its role in object recognition. Vision Research 41(10-11), 1409–1422 (2001)CrossRefGoogle Scholar
  42. 42.
    Grossberg, S.: How does a brain build a cognitive code? Psychological Review 87(1), 1–51 (1980)MathSciNetCrossRefGoogle Scholar
  43. 43.
    den Ouden, H.E.M., Friston, K.J., Daw, N.D., McIntosh, A.R., Stephan, K.E.: A Dual Role for Prediction Error in Associative Learning. Cereb. Cortex 19(5), 1175–1185 (2009); first published online (September 26, 2008), doi:10.1093/cercor/bhn161CrossRefGoogle Scholar
  44. 44.
    Hawkins, J., George, D., et al.: Sequence memory for prediction, inference and behaviour. Philos. Trans. R Soc. Lond. B Biol. Sci. 364(1521), 1203–1209 (2009)CrossRefGoogle Scholar
  45. 45.
    Haxby, J.V., Gobbini, M.I., et al.: Distributed and overlapping representations of faces and objects in ventral temporal cortex. Science 293(5539), 2425–2430 (2001)CrossRefGoogle Scholar
  46. 46.
    Hebb, D.O.: The organization of behavior. Wiley, New York (1949)Google Scholar
  47. 47.
    Henson, R.N.: Neuroimaging studies of priming. Progress in Neurobiology 70(1), 53–81 (2003)MathSciNetCrossRefGoogle Scholar
  48. 48.
    Hirschfeld, G., Zwitserlood, P.: How vision is shaped by language comprehension–top-down feedback based on low-spatial frequencies. Brain Res. 1377, 78–83 (2011)CrossRefGoogle Scholar
  49. 49.
    Hubel, D.H., Wiesel, T.N.: Ferrier lecture. Functional architecture of macaque monkey visual cortex. Proc. R Soc. Lond. B Biol. Sci. 198(1130), 1–59 (1977)CrossRefGoogle Scholar
  50. 50.
    Hughes, H.C., Nozawa, G., et al.: Global precedence, spatial frequency channels, and the statistics of natural images. Journal of Cognitive Neuroscience 8(3), 197–230 (1996)CrossRefGoogle Scholar
  51. 51.
    Intraub, H.: The representation of visual scenes. Trends Cogn. Sci. 1(6), 217–222 (1997)CrossRefGoogle Scholar
  52. 52.
    Kandel, E.R., Schwartz, J.H., et al.: Principles of neural science. Elsevier, New York (1991)Google Scholar
  53. 53.
    Kersten, D., Mamassian, P., et al.: Object perception as Bayesian inference. Annu. Rev. Psychol. 55, 271–304 (2004)CrossRefGoogle Scholar
  54. 54.
    Kosslyn, S.M.: Image and Brain. MIT Press, Cambridge (1994)Google Scholar
  55. 55.
    Kveraga, K., Ghuman, A.S., et al.: Top-down predictions in the cognitive brain. Brain and Cognition 65, 145–168 (2007)CrossRefGoogle Scholar
  56. 56.
    Lamme, V.A.F., Roelfsema, P.R.: The distinct modes of vision offered by feedforward and recurrent processing. Trends in Neuroscience 23, 571–579 (2000)CrossRefGoogle Scholar
  57. 57.
    LeDoux, J.E.: The emotional brain. Simon & Schuster, New York (1996)Google Scholar
  58. 58.
    Logothetis, N.K., Sheinberg, D.L.: Visual object recognition. Annu. Rev. Neurosci. 19, 577–621 (1996)CrossRefGoogle Scholar
  59. 59.
    Malach, R., Levy, I., et al.: The topography of high-order human object areas. Trends Cogn. Sci. 6(4), 176–184 (2002)CrossRefGoogle Scholar
  60. 60.
    Malcolm, G.L., Henderson, J.M.: Combining top-down processes to guide eye movements during real-world scene search. J. Vis. 10(2), 4, 1–11 (2010)CrossRefGoogle Scholar
  61. 61.
    Marinkovic, K., Dhond, R.P., et al.: Spatiotemporal dynamics of modality-specific and supramodal word processing. Neuron 38(3), 487–497 (2003)CrossRefGoogle Scholar
  62. 62.
    Mesulam, M.: Behavioral neuroanatomy: large-scale networks, association cortex, frontal syndromes, the limbic system, and hemispheric specializations. In: Mesulam, M. (ed.) Principles of Behavioral and Cognitive Neurology, 2nd edn., pp. 1–120. Oxford University Press, New York (2000)Google Scholar
  63. 63.
    Moulton, S.T., Kosslyn, S.M.: Imagining predictions: Mental imagery as mental simulation. Philosophical Transactions of the Royal Society B: Biological Sciences Special Theme Issue (2009)Google Scholar
  64. 64.
    Mumford, D.: On the computational architecture of the neocortex. II. The role of cortico-cortical loops. Biol. Cybern. 66(3), 241–251 (1992)CrossRefGoogle Scholar
  65. 65.
    Nakamura, H., Gattass, R., et al.: The modular organization of projections for areas V1 and V2 to areas V4 and TEO in macaques. Journal of Neuroscience 13(9), 3681–3691 (1993)Google Scholar
  66. 66.
    Navon, D.: Forest before trees: The precedence of global features in visual perception. Cognitive Psychology 9, 1–32 (1977)CrossRefGoogle Scholar
  67. 67.
    Neisser, U.: Cognitive psychology. Appleton-Century –Crofts, New York (1967)Google Scholar
  68. 68.
    Oliva, A., Torralba, A.: Modeling the shape of the scene: A holistic representation of the spatial envelope. International Journal of Computer Vision 42(3), 145–175 (2001)zbMATHCrossRefGoogle Scholar
  69. 69.
    Palmer, S.E.: The effects of contextual scenes on the identification of objects. Memory and Cognition 3, 519–526 (1975)CrossRefGoogle Scholar
  70. 70.
    Perlovsky, L.I., Ilin, R.: Grounded Symbol. In: The Brain, Computational Foundations For Perceptual Symbol System. Webmed Central PSYCHOLOGY 2010, vol. 1(12), pp. WMC001357 (2010)Google Scholar
  71. 71.
    Perlovsky, L.I.: Toward Physics of the Mind: Concepts, Emotions, Consciousness, and Symbols. Phys. Life Rev. 3(1), 22–55 (2006)MathSciNetCrossRefGoogle Scholar
  72. 72.
    Perrett, D.I., Hietanen, J.K., et al.: Organization and functions of cells responsive to faces in the temporal cortex. Philosophical Transactions of the Royal Society of London, B 335, 23–30 (1992)CrossRefGoogle Scholar
  73. 73.
    Peyrin, C., Michel, C.M., et al.: The neural substrates and timing of top-down processes during coarse-to-fine categorization of visual scenes: a combined fMRI and ERP study. J. Cogn. Neurosci. 22(12), 2768–2780 (2010)CrossRefGoogle Scholar
  74. 74.
    Porrino, L.J., Crane, A.M., et al.: Direct and indirect pathways from the amygdala to the frontal lobe in rhesus monkeys. Journal of Comparative Neurology 198(1), 121–136 (1981)CrossRefGoogle Scholar
  75. 75.
    Posner, M.I., Snyder, C.R., et al.: Attention and the detection of signals. J. Exp. Psychol. 109(2), 160–174 (1980)Google Scholar
  76. 76.
    Puce, A., Allison, T., Asgari, M., Gore, J.C., McCarthy, G.: Differential sensitivity of human visual cortex to faces, letterstrings, and textures: a functional magnetic resonance imaging study. Journal of Neurosciences 16(16), 5205–5215 (1996)Google Scholar
  77. 77.
    Rao, R.P., Ballard, D.H.: Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects (see comments). Nature Neuroscience 2(1), 79–87 (1999)CrossRefGoogle Scholar
  78. 78.
    Rempel-Clower, N.L., Barbas, H.: The laminar pattern of connections between prefrontal and anterior temporal cortices in the rhesus monkey is related to cortical structure and function. Cerebral Cortex 10(9), 851–865 (2000)CrossRefGoogle Scholar
  79. 79.
    Russchen, F.T., Amaral, D.G., et al.: The afferent input to the magnocellular division of the mediodorsal thalamic nucleus in the monkey, Macaca fascicularis. Journal of Comparative Neurology 256(2), 175–210 (1987)CrossRefGoogle Scholar
  80. 80.
    Schnyer, D.M., Dobbins, I.G., Nicholls, L., Schacter, D., Verfaellie, M.: Rapid Decision Learning Alters the Repetition N400 Components in Left Frontal and Temporal Regions: Evidence from MEG Recordings During Repetition Priming. Society for Neuroscience, Washington, D.C. (2005)Google Scholar
  81. 81.
    Schyns, P.G., Oliva, A.: From blobs to boundary edges: Evidence for time- and spatial-dependent scene recognition. Psychological Science 5(4), 195–200 (1994)CrossRefGoogle Scholar
  82. 82.
    Spratling, M.W.: Predictive coding as a model of biased competition in visual attention. Vision Res. 48(12), 1391–1408 (2008)CrossRefGoogle Scholar
  83. 83.
    Summerfield, C., Egner, T.: Expectation (and attention) in visual cognition. Trends Cogn. Sci. 13(9), 403–409 (2009)CrossRefGoogle Scholar
  84. 84.
    Tanaka, K.: Neuronal mechanisms of object recognition. Science 262, 685–688 (1993)CrossRefGoogle Scholar
  85. 85.
    Tanaka, K.: Representation of Visual Features of Objects in the Inferotemporal Cortex. Neural Netw. 9(8), 1459–1475 (1996)zbMATHCrossRefGoogle Scholar
  86. 86.
    Taylor, S.F., Phan, K.L., et al.: Subjective rating of emotionally salient stimuli modulates neural activity. Neuroimage 18(3), 650–659 (2003)CrossRefGoogle Scholar
  87. 87.
    Tobler, P.N., O’Doherty, J.P., et al.: Human neural learning depends on reward prediction errors in the blocking paradigm. J. Neurophysiol. 95(1), 301–310 (2006)CrossRefGoogle Scholar
  88. 88.
    Torralba, A.: Contextual priming for object detection. International Journal of Computer Vision 53(2), 153–167 (2003)CrossRefGoogle Scholar
  89. 89.
    Tovee, M.J., Rolls, E.T., et al.: Translation invariance in the responses to faces of single neurons in the temporal visual cortical areas of the alert macaque. J. Neurophysiol. 72(3), 1049–1060 (1994)Google Scholar
  90. 90.
    Ullman, S.: Sequence seeking and counter streams: A computational model for bidirectional information flow in the visual cortex. Cerebral Cortex 1, 1–11 (1995)MathSciNetCrossRefGoogle Scholar
  91. 91.
    Wig, G.S., Grafton, S.T., et al.: Reductions in neural activity underlie behavioral components of repetition priming. Nat. Neurosci. 8(9), 1228–1233 (2005)CrossRefGoogle Scholar
  92. 92.
    Zago, L., Fenske, M.J., et al.: The rise and fall of priming: How visual exposure shapes cortical representations of objects. Cereb. Cortex 15, 1655–1665 (2005)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Helena Yardley
    • 1
  • Leonid Perlovsky
    • 2
  • Moshe Bar
    • 3
  1. 1.Martinos Center for Biomedical Imaging at Massachusetts General HospitalUSA
  2. 2.Harvard University and Air Force Research Laboratory, Hanscom AFBUSA
  3. 3.Martinos Center Biomedical Imaging at Massachusetts General HospitalHarvard Medical SchoolCharlestownUSA

Personalised recommendations