International Journal of Computer Vision

, Volume 108, Issue 1, pp 59–81

The SUN Attribute Database: Beyond Categories for Deeper Scene Understanding

Article

DOI: 10.1007/s11263-013-0695-z

Cite this article as:
Patterson, G., Xu, C., Su, H. et al. Int J Comput Vis (2014) 108: 59. doi:10.1007/s11263-013-0695-z

Abstract

In this paper we present the first large-scale scene attribute database. First, we perform crowdsourced human studies to find a taxonomy of 102 discriminative attributes. We discover attributes related to materials, surface properties, lighting, affordances, and spatial layout. Next, we build the “SUN attribute database” on top of the diverse SUN categorical database. We use crowdsourcing to annotate attributes for 14,340 images from 707 scene categories. We perform numerous experiments to study the interplay between scene attributes and scene categories. We train and evaluate attribute classifiers and then study the feasibility of attributes as an intermediate scene representation for scene classification, zero shot learning, automatic image captioning, semantic image search, and parsing natural images. We show that when used as features for these tasks, low dimensional scene attributes can compete with or improve on the state of the art performance. The experiments suggest that scene attributes are an effective low-dimensional feature for capturing high-level context and semantics in scenes.

Keywords

Scene understanding Crowdsourcing Attributes Image captioning Scene parsing 

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  • Genevieve Patterson
    • 1
  • Chen Xu
    • 1
  • Hang Su
    • 1
  • James Hays
    • 1
  1. 1.Department of Computer ScienceBrown UniversityProvidenceUSA

Personalised recommendations