International Journal of Computer Vision

, Volume 42, Issue 3, pp 145–175

Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope

  • Aude Oliva
  • Antonio Torralba

DOI: 10.1023/A:1011139631724

Cite this article as:
Oliva, A. & Torralba, A. International Journal of Computer Vision (2001) 42: 145. doi:10.1023/A:1011139631724


In this paper, we propose a computational model of the recognition of real world scenes that bypasses the segmentation and the processing of individual objects or regions. The procedure is based on a very low dimensional representation of the scene, that we term the Spatial Envelope. We propose a set of perceptual dimensions (naturalness, openness, roughness, expansion, ruggedness) that represent the dominant spatial structure of a scene. Then, we show that these dimensions may be reliably estimated using spectral and coarsely localized information. The model generates a multidimensional space in which scenes sharing membership in semantic categories (e.g., streets, highways, coasts) are projected closed together. The performance of the spatial envelope model shows that specific information about object shape or identity is not a requirement for scene categorization and that modeling a holistic representation of the scene informs about its probable semantic category.

scene recognition natural images energy spectrum principal components spatial layout 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Copyright information

© Kluwer Academic Publishers 2001

Authors and Affiliations

  • Aude Oliva
    • 1
  • Antonio Torralba
    • 2
  1. 1.Harvard Medical School and the Brigham and Women's HospitalBoston
  2. 2.Department of Brain and Cognitive Sciences, MITCambridge

Personalised recommendations