How Is Scene Recognition in a Convolutional Network Related to that in the Human Visual System?

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9886)


This study is an analysis of scene recognition in a pre-trained convolutional network, to evaluate the information the network uses to distinguish scene categories. We are particularly interested in how the network is related to various areas in the human brain that are involved in different modes of scene recognition. Results of several experiments suggest that the convolutional network relies heavily on objects and fine features, similar to the lateral occipital complex (LOC) in the brain, but less on large-scale scene layout. This suggests that future scene-processing convolutional networks might be made more brain-like by adding parallel components that are more sensitive to arrangement of simple forms.


Convolutional neural networks (CNNs) Scene recognition Human visual system 



Supported by CFI & OIT infrastructure funds, the Canada Research Chairs program, NSERC Discovery grants 261453 and 296878, ONR grant N000141310419, AFOSR grant FA8655-13-1-3084 and OGS.


  1. 1.
    Epstein, R., Kanwisher, N.: A cortical representation of the local visual environment. Nature 392(6676), 598–601 (1998)CrossRefGoogle Scholar
  2. 2.
    Fei-Fei, L., Iyer, A., Koch, C., Perona, P.: What do we perceive in a glance of a real-world scene? J. Vis. 7(1), 10–10 (2007)CrossRefGoogle Scholar
  3. 3.
    Greene, M.R., Oliva, A.: The briefest of glances: the time course of natural scene understanding. Psychol. Sci. 20(4), 464–472 (2009)CrossRefGoogle Scholar
  4. 4.
    Greene, M.R., Oliva, A.: Recognition of natural scenes from global properties: seeing the forest without representing the trees. Cogn. Psychol. 58(2), 137–176 (2009)CrossRefGoogle Scholar
  5. 5.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. Adv. Neural Inf. Process. Syst. 25, 1097–1105 (2012)Google Scholar
  6. 6.
    MacEvoy, S.P., Epstein, R.A.: Constructing scenes from objects in human occipitotemporal cortex. Nature Neurosci. 14(10), 1323–1329 (2011)CrossRefGoogle Scholar
  7. 7.
    Oliva, A.: Scene perception. In: Werner, J.S., Chalupa, L.M. (eds.) The New Visual Neurosciences, pp. 725–732. MIT Press, Cambridge (2014)Google Scholar
  8. 8.
    Park, S., Brady, T.F., Greene, M.R., Oliva, A.: Disentangling scene content from spatial boundary: complementary roles for the parahippocampal place area and lateral occipital complex in representing real-world scenes. J. Neurosci. 31(4), 1333–1340 (2011)CrossRefGoogle Scholar
  9. 9.
    Park, S., Chun, M.M.: Different roles of the parahippocampal place area (PPA) and retrosplenial cortex (RSC) in panoramic scene perception. Neuroimage 47(4), 1747–1756 (2009)CrossRefGoogle Scholar
  10. 10.
    Zhou, B., Lapedriza, A., Xiao, J., Torralba, A., Oliva, A.: Learning deep features for scene recognition using places database. Adv. Neural Inf. Process. Syst. 27, 487–495 (2014)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  1. 1.Centre for Theoretical NeuroscienceUniversity of WaterlooWaterlooCanada

Personalised recommendations