Skip to main content

How Is Scene Recognition in a Convolutional Network Related to that in the Human Visual System?

  • Conference paper
  • First Online:
Artificial Neural Networks and Machine Learning – ICANN 2016 (ICANN 2016)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 9886))

Included in the following conference series:

  • 2761 Accesses


This study is an analysis of scene recognition in a pre-trained convolutional network, to evaluate the information the network uses to distinguish scene categories. We are particularly interested in how the network is related to various areas in the human brain that are involved in different modes of scene recognition. Results of several experiments suggest that the convolutional network relies heavily on objects and fine features, similar to the lateral occipital complex (LOC) in the brain, but less on large-scale scene layout. This suggests that future scene-processing convolutional networks might be made more brain-like by adding parallel components that are more sensitive to arrangement of simple forms.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others


  1. Epstein, R., Kanwisher, N.: A cortical representation of the local visual environment. Nature 392(6676), 598–601 (1998)

    Article  Google Scholar 

  2. Fei-Fei, L., Iyer, A., Koch, C., Perona, P.: What do we perceive in a glance of a real-world scene? J. Vis. 7(1), 10–10 (2007)

    Article  Google Scholar 

  3. Greene, M.R., Oliva, A.: The briefest of glances: the time course of natural scene understanding. Psychol. Sci. 20(4), 464–472 (2009)

    Article  Google Scholar 

  4. Greene, M.R., Oliva, A.: Recognition of natural scenes from global properties: seeing the forest without representing the trees. Cogn. Psychol. 58(2), 137–176 (2009)

    Article  Google Scholar 

  5. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. Adv. Neural Inf. Process. Syst. 25, 1097–1105 (2012)

    Google Scholar 

  6. MacEvoy, S.P., Epstein, R.A.: Constructing scenes from objects in human occipitotemporal cortex. Nature Neurosci. 14(10), 1323–1329 (2011)

    Article  Google Scholar 

  7. Oliva, A.: Scene perception. In: Werner, J.S., Chalupa, L.M. (eds.) The New Visual Neurosciences, pp. 725–732. MIT Press, Cambridge (2014)

    Google Scholar 

  8. Park, S., Brady, T.F., Greene, M.R., Oliva, A.: Disentangling scene content from spatial boundary: complementary roles for the parahippocampal place area and lateral occipital complex in representing real-world scenes. J. Neurosci. 31(4), 1333–1340 (2011)

    Article  Google Scholar 

  9. Park, S., Chun, M.M.: Different roles of the parahippocampal place area (PPA) and retrosplenial cortex (RSC) in panoramic scene perception. Neuroimage 47(4), 1747–1756 (2009)

    Article  Google Scholar 

  10. Zhou, B., Lapedriza, A., Xiao, J., Torralba, A., Oliva, A.: Learning deep features for scene recognition using places database. Adv. Neural Inf. Process. Syst. 27, 487–495 (2014)

    Google Scholar 

Download references


Supported by CFI & OIT infrastructure funds, the Canada Research Chairs program, NSERC Discovery grants 261453 and 296878, ONR grant N000141310419, AFOSR grant FA8655-13-1-3084 and OGS.

Author information

Authors and Affiliations


Corresponding author

Correspondence to Bryan Tripp .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this paper

Cite this paper

Sharma, S., Tripp, B. (2016). How Is Scene Recognition in a Convolutional Network Related to that in the Human Visual System?. In: Villa, A., Masulli, P., Pons Rivero, A. (eds) Artificial Neural Networks and Machine Learning – ICANN 2016. ICANN 2016. Lecture Notes in Computer Science(), vol 9886. Springer, Cham.

Download citation

  • DOI:

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-44777-3

  • Online ISBN: 978-3-319-44778-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics