Shapecollage: Occlusion-Aware, Example-Based Shape Interpretation

  • Forrester Cole
  • Phillip Isola
  • William T. Freeman
  • Frédo Durand
  • Edward H. Adelson
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7574)


This paper presents an example-based method to interpret a 3D shape from a single image depicting that shape. A major difficulty in applying an example-based approach to shape interpretation is the combinatorial explosion of shape possibilities that occur at occluding contours. Our key technical contribution is a new shape patch representation and corresponding pairwise compatibility terms that allow for flexible matching of overlapping patches, avoiding the combinatorial explosion by allowing patches to explain only the parts of the image they best fit. We infer the best set of localized shape patches over a graph of keypoints at multiple scales to produce a discontinuous shape representation we term a shape collage. To reconstruct a smooth result, we fit a surface to the collage using the predicted confidence of each shape patch. We demonstrate the method on shapes depicted in line drawing, diffuse and glossy shading, and textured styles.


  1. 1.
    Roberts, L.: Machine perception of three-dimensional solids. (1963)
  2. 2.
    Durou, J., Falcone, M., Sagona, M.: Num. methods for shape-from-shading: A new survey with benchmarks. Comp. Vis. and Img. Understanding 109, 22–43 (2008)CrossRefGoogle Scholar
  3. 3.
    Malik, J., Rosenholtz, R.: Computing local surface orientation and shape from texture for curved surfaces. IJCV 23, 149–168 (1997)CrossRefGoogle Scholar
  4. 4.
    Hassner, T., Basri, R.: Example based 3d reconstruction from single 2d images. In: CVPR Workshops: Beyond Patches, pp. 1–8 (2006)Google Scholar
  5. 5.
    Saxena, A., Sun, M., Ng, A.: Make3d: Learning 3d scene structure from a single still image. PAMI 31, 824–840 (2009)CrossRefGoogle Scholar
  6. 6.
    Malik, J.: Interpreting line drawings of curved objects. IJCV (1987)Google Scholar
  7. 7.
    Wang, Y., Chen, Y., Liu, J., Tang, X.: 3d reconstruction of curved objects from single 2d line drawings. In: CVPR, pp. 1834–1841 (2009)Google Scholar
  8. 8.
    Xue, T., Liu, J., Tang, X.: Symmetric piecewise planar object reconstruction from a single image. In: CVPR, pp. 2577–2584 (2011)Google Scholar
  9. 9.
    Saund, E.: Logic and mrf circuitry for labeling occluding and thinline visual contours. In: Neural Information Processing Systems, NIPS (2005)Google Scholar
  10. 10.
    Ouyang, T., Davis, R.: Learning from neighboring strokes: combining appearance and context for multi-domain sketch recognition. In: NIPS (2009)Google Scholar
  11. 11.
    Freeman, W., Pasztor, E., Carmichael, O.: Learning low-level vision. International Journal of Computer Vision 40, 25–47 (2000)zbMATHCrossRefGoogle Scholar
  12. 12.
    Fitzgibbon, A.W., Wexler, Y., Zisserman, A.: Image-based rendering using image-based priors. In: Intl. Conf. Computer Vision, ICCV (2003)Google Scholar
  13. 13.
    Shotton, J., Fitzgibbon, A., Cook, M., Sharp, T., Finocchio, M., Moore, R., Kipman, A., Blake, A.: Real-time human pose recognition in parts from a single depth image. In: CVPR (2011)Google Scholar
  14. 14.
    Harris, C., Stephens, M.: A combined corner and edge detector. In: Alvey Vision Conference, vol. 15, p. 50 (1988)Google Scholar
  15. 15.
    Lowe, D.: Object recognition from local scale-invariant features. In: ICCV (1999)Google Scholar
  16. 16.
    Supplemental Material (2012),
  17. 17.
    DeCarlo, D., Finkelstein, A., Rusinkiewicz, S., Santella, A.: Suggestive contours for conveying shape. ACM Trans. Graph. 22, 848–855 (2003)CrossRefGoogle Scholar
  18. 18.
    Hays, J., Efros, A.: Im2gps: estimating geographic information from a single image. In: CVPR, pp. 1–8 (2008)Google Scholar
  19. 19.
    Tappen, M.: Recovering shape from a single image of a mirrored surface from curvature constraints. In: CVPR, pp. 2545–2552 (2011)Google Scholar
  20. 20.
    Cole, F., Golovinskiy, A., Limpaecher, A., Barros, H., Finkelstein, A., Funkhouser, T., Rusinkiewicz, S.: Where do people draw lines? In: SIGGRAPH (2008)Google Scholar
  21. 21.
    Barron, J.T., Malik, J.: High-frequency shape and albedo from shading using natural image statistics. In: CVPR (2011)Google Scholar
  22. 22.
    Kaneva, B., Torralba, A., Freeman, W.: Evaluation of image features using a photorealistic virtual world. In: ICCV (2011)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Forrester Cole
    • 1
  • Phillip Isola
    • 1
  • William T. Freeman
    • 1
  • Frédo Durand
    • 1
  • Edward H. Adelson
    • 1
  1. 1.Massachusetts Institute of TechnologyUSA

Personalised recommendations