International Journal of Computer Vision

, Volume 91, Issue 3, pp 328-346

First online:

Recovering Occlusion Boundaries from an Image

  • Derek HoiemAffiliated withDepartment of Computer Science, University of Illinois at Urbana-Champaign Email author 
  • , Alexei A. EfrosAffiliated withRobotics Institute, Carnegie Mellon University
  • , Martial HebertAffiliated withRobotics Institute, Carnegie Mellon University

Rent the article at a discount

Rent now

* Final gross prices may vary according to local VAT.

Get Access


Occlusion reasoning is a fundamental problem in computer vision. In this paper, we propose an algorithm to recover the occlusion boundaries and depth ordering of free-standing structures in the scene. Rather than viewing the problem as one of pure image processing, our approach employs cues from an estimated surface layout and applies Gestalt grouping principles using a conditional random field (CRF) model. We propose a hierarchical segmentation process, based on agglomerative merging, that re-estimates boundary strength as the segmentation progresses. Our experiments on the Geometric Context dataset validate our choices for features, our iterative refinement of classifiers, and our CRF model. In experiments on the Berkeley Segmentation Dataset, PASCAL VOC 2008, and LabelMe, we also show that the trained algorithm generalizes to other datasets and can be used as an object boundary predictor with figure/ground labels.


Image segmentation Occlusion boundaries Figure/ground labeling Image interpretation Scene understanding 3D reconstruction Depth from image Edge detection