Article

International Journal of Computer Vision

, Volume 77, Issue 1, pp 291-330

First online:

Describing Visual Scenes Using Transformed Objects and Parts

  • Erik B. SudderthAffiliated withComputer Science Division, University of California Email author 
  • , Antonio TorralbaAffiliated withElectrical Engineering & Computer Science, Massachusetts Institute of Technology
  • , William T. FreemanAffiliated withElectrical Engineering & Computer Science, Massachusetts Institute of Technology
  • , Alan S. WillskyAffiliated withElectrical Engineering & Computer Science, Massachusetts Institute of Technology

Rent the article at a discount

Rent now

* Final gross prices may vary according to local VAT.

Get Access

Abstract

We develop hierarchical, probabilistic models for objects, the parts composing them, and the visual scenes surrounding them. Our approach couples topic models originally developed for text analysis with spatial transformations, and thus consistently accounts for geometric constraints. By building integrated scene models, we may discover contextual relationships, and better exploit partially labeled training images. We first consider images of isolated objects, and show that sharing parts among object categories improves detection accuracy when learning from few examples. Turning to multiple object scenes, we propose nonparametric models which use Dirichlet processes to automatically learn the number of parts underlying each object category, and objects composing each scene. The resulting transformed Dirichlet process (TDP) leads to Monte Carlo algorithms which simultaneously segment and recognize objects in street and office scenes.

Keywords

Object recognition Dirichlet process Hierarchical Dirichlet process Transformation Context Graphical models Scene analysis