Virtual Visual Hulls: Example-Based 3D Shape Inference from Silhouettes

  • Kristen Grauman
  • Gregory Shakhnarovich
  • Trevor Darrell
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3247)


We present a method for estimating the 3D visual hull of an object from a known class given a single silhouette or sequence of silhouettes observed from an unknown viewpoint. A non-parametric density model of object shape is learned for the given object class by collecting multi-view silhouette examples from calibrated, though possibly varied, camera rigs. To infer a 3D shape from a single input silhouette, we search for 3D shapes which maximize the posterior given the observed contour. The input is matched to component single views of the multi-view training examples. A set of viewpoint-aligned virtual views are generated from the visual hulls corresponding to these examples. The most likely visual hull for the input is then found by interpolating between the contours of these aligned views. When the underlying shape is ambiguous given a single view silhouette, we produce multiple visual hull hypotheses; if a sequence of input images is available, a dynamic programming approach is applied to find the maximum likelihood path through the feasible hypotheses over time. We show results of our algorithm on real and synthetic images of people.


Object Class Single View Virtual View Virtual Camera Visual Hull 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Baumberg, A., Hogg, D.: An adaptive eigenshape model. In: BMVC (1995)Google Scholar
  2. 2.
    Brand, M.: Shadow puppetry. In: ICCV (1999)Google Scholar
  3. 3.
    Cheung, G.K.M., Baker, S., Kanade, T.: Shape-From-Silhouette of articulated objects and its use for human body kinematics estimation and motion capture. In: CVPR (2003)Google Scholar
  4. 4.
    Cipolla, R., Blake, A.: Surface shape from the deformation of apparent contours. IJCV 9(2) (1992)Google Scholar
  5. 5.
    Cootes, T., Taylor, C.: A mixture model for representing shape variation. In: BMVC (1997)Google Scholar
  6. 6.
    Curious Labs, Egisys Co. Poser 5: The ultimate 3D character solutionGoogle Scholar
  7. 7.
    Forsyth, D., Ponce, J.: Computer Vision: A Modern Approach, pp. 552–554 (2003)Google Scholar
  8. 8.
    Gavrila, D., Philomin, V.: Real-time object detection for smart vehicles. In: ICCV (1999)Google Scholar
  9. 9.
    Grauman, K., Darrell, T.: Fast contour matching using approximate earth mover’s distance. In: CVPR (2004)Google Scholar
  10. 10.
    Grauman, K., Shakhnarovich, G., Darrell, T.: Inferring 3D structure with a statistical image-based shape model. In: ICCV (2003)Google Scholar
  11. 11.
    Grauman, K., Shakhnarovich, G., Darrell, T.: A Bayesian approach to image-based visual hull reconstruction. In: CVPR (2003)Google Scholar
  12. 12.
    Huttenlocher, D., Klanderman, G., Rucklidge, W.: Comparing images using the Hausdorff distance. In: PAMI (1993)Google Scholar
  13. 13.
    Jones, M., Poggio, T.: Multidimensional morphable models. In: ICCV (1998)Google Scholar
  14. 14.
    Laurentini, A.: The visual hull concept for silhouette-based image understanding. PAMI 16(2) (1994)Google Scholar
  15. 15.
    Lazebnik, S., Boyer, E., Ponce, J.: On computing exact visual hulls of solids bounded by smooth surfaces. In: CVPR (2001)Google Scholar
  16. 16.
    Matusik, W., Buehler, C., McMillan, L.: Polyhedral visual hulls for real-time rendering. In: EGWR (2001)Google Scholar
  17. 17.
    Mori, G., Malik, J.: Estimating human body configurations using shape context matching. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2352, pp. 666–680. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  18. 18.
    Rosales, R., Sclaroff, S.: Specialized mappings and the estimation of body pose from a single image. In: HUMO (2000)Google Scholar
  19. 19.
    Szeliski, R., Weiss, R.: Robust shape recovery from occluding contours using a linear smoother. IJCV 28(1) (1998)Google Scholar
  20. 20.
    Toyama, K., Blake, A.: Probabilistic Exemplar-based tracking in a metric space. In: ICCV (2001)Google Scholar
  21. 21.
    Wong, K.-Y.K., Cipolla, R.: Structure and motion from silhouettes. In: ICCV (2001)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2004

Authors and Affiliations

  • Kristen Grauman
    • 1
  • Gregory Shakhnarovich
    • 1
  • Trevor Darrell
    • 1
  1. 1.Computer Science and Artificial Intelligence LaboratoryMassachusetts Institute of TechnologyCambridgeUSA

Personalised recommendations