The Visual Computer

, Volume 26, Issue 12, pp 1435–1450 | Cite as

Volumetric stereo and silhouette fusion for image-based modeling

Original Article

Abstract

This paper presents a volumetric stereo and silhouette fusion algorithm for acquiring high quality models from multiple calibrated photographs. Our method is based on computing and merging depth maps. Different from previous methods of this category, the silhouette information is also applied in our algorithm to recover the shape information on the textureless and occluded areas. The proposed algorithm starts by computing visual hull using a volumetric method in which a novel projection test method is proposed for visual hull octree construction. Then, the depth map of each image is estimated by an expansion-based approach that returns a 3D point cloud with outliers and redundant information. After generating an oriented point cloud from stereo by rejecting outlier, reducing scale, and estimating surface normal for the depth maps, another oriented point cloud from silhouette is added by carving the visual hull octree structure using the point cloud from stereo to restore the textureless and occluded surfaces. Finally, Poisson Surface Reconstruction approach is applied to convert the oriented point cloud both from stereo and silhouette into a complete and accurate triangulated mesh model. The proposed approach has been implemented and the performance of the approach is demonstrated on several real data sets, along with qualitative comparisons with the state-of-the-art image-based modeling techniques according to the Middlebury benchmark.

Keywords

Multi-view stereo Depth map Oriented point cloud Visual hull 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Supplementary material

371_2010_429_MOESM1_ESM.wmv (10.6 mb)
(WMV 10.832 kb)

References

  1. 1.
    Prados, E., Faugeras, O.: Perspective shape from shading and viscosity solutions. In: ICCV 2, pp. 826–831 (2003) Google Scholar
  2. 2.
    Szeliski, R.: Rapid octree construction from range sequences. Comput. Vis. Graph. Image Process. 58(1), 23–32 (1993) Google Scholar
  3. 3.
    Yemez, Y., Schmitt, F.: 3D reconstruction of real objects with high resolution shape and texture. Image Vis. Comput. 22(13), 1137–1153 (2004) CrossRefGoogle Scholar
  4. 4.
    Kazhdan, M., Bolithp, M., Hoppe, H.: Poisson surface reconstruction. In: Eurographics Symposium on Geometry Processing, pp. 61–70 (2006) Google Scholar
  5. 5.
    Seitz, S.M., Curless, B., Diebel, J., Scharstein, D., Szeliski, R.: A comparison and evaluation of multi-view stereo reconstruction algorithms. In: CVPR 1, pp. 519–526 (2006) Google Scholar
  6. 6.
    Vogiatzis, G., Torr, P., Cipolla, R.: Multi-view stereo via volumetric graph-cuts. In: CVPR, pp. 391–398 (2005) Google Scholar
  7. 7.
    Sinha, S.N., Mordohai, P., Pollefeys, M.: Multi-view stereo via graph cuts on the dual of an adaptive tetrahedral mesh. In: ICCV (2007) Google Scholar
  8. 8.
    Vogiatzis, G., Hernandez, C., Torr, P.H.S., Cipolla, R.: Multi-view stereo via volumetric graph-cuts and occlusion robust photo-consistenc. IEEE Trans. Pattern Anal. Mach. Intell. 29(12), 2241–2246 (2007) CrossRefGoogle Scholar
  9. 9.
    Kutulakos, K., Seitz, S.M.: A theory of shape by space carving. Int. J. Comput. Vis. 38(3), 199–218 (2000) MATHCrossRefGoogle Scholar
  10. 10.
    Hernandez, C., Schmitt, F.: Silhouette and stereo fusion for 3d object modeling. Comput. Vis. Image Underst. 96(3), 367–392 (2004) CrossRefGoogle Scholar
  11. 11.
    Pons, J., Keriven, R., Faugeras, O.: Modelling dynamic scenes by registering multi-view image sequences. In: CVPR, pp. 822–827 (2005) Google Scholar
  12. 12.
    Furukawa, Y., Ponce, J.: Accurate, dense, and robust multi-view stereopsis. In: CVPR (2007) Google Scholar
  13. 13.
    Habbecke, M., Kobbelt, L.: A surface-growing approach to multi-view stereo reconstruction. In: CVPR (2007) Google Scholar
  14. 14.
    Goesele, M., Snavely, N., Curless, B., Hoppe, H., Seitz, S.M.: Multi-view stereo for community photo collections. In: ICCV (2007) Google Scholar
  15. 15.
    Narayanan, P., Rander, P., Kanade, T.: Constructing virtual worlds using dense stereo. In: ICCV, pp. 3–10 (1998) Google Scholar
  16. 16.
    Goesele, M., Curless, B., Seitz, S.M.: Multi-view stereo revisited. In: CVPR, pp. 2402–2409 (2006) Google Scholar
  17. 17.
    Bradley, D., Boubekeur, T., Heidrich, W.: Accurate multi-view reconstruction using robust binocular stereo and surface meshing. In: CVPR (2008) Google Scholar
  18. 18.
    Liu, Y., Cao, X., Dai, Q., Xu, W.: Continuous depth estimation for multi-view stereo. In: CVPR, pp. 2121–2128 (2009) Google Scholar
  19. 19.
    Laurentini, A.: The visual hull concept for silhouette based image understanding. IEEE Trans. Pattern Anal. Mach. Intell. 16(2), 150–162 (1994) CrossRefGoogle Scholar
  20. 20.
    Matusik, W., Buehler, C., McMillan, L.: Polyhedral visual hulls for real-time rendering. In: Proc. 12th Eurographics Workshop on Rendering, pp. 115–125 (2001) Google Scholar
  21. 21.
    Tarini, M., Callieri, M., Montani, C., Rocchini, C., Olsson, K., Persson, T.: Marching intersections: An efficient approach to shape-from-silhouette. In: Proceedings of VMV, pp. 283–290 (2002) Google Scholar
  22. 22.
    Lorensen, W.E., Cline, H.E.: Marching cubes: A high resolution 3d surface construction algorithm. In: SIGGRAPH 21, pp. 163–169 (1987) Google Scholar
  23. 23.
    Song, P., Wu, X., Wang, M.Y.: A robust and accurate method for visual hull computation. In: IEEE International Conference on Information and Automation, pp. 784–789 (2009) Google Scholar
  24. 24.
    Potmesil, M.: Generating octree models of 3d objects from their silhouettes in a sequence of images. Comput. Vis. Graph. Image Process. 40, 1–29 (1987) CrossRefGoogle Scholar
  25. 25.
    Borgefors, G.: Distance transformations in digital images. Comput. Vis. Graph. Image Process. 34, 344–371 (1986) CrossRefGoogle Scholar
  26. 26.
    Jarvis, R.A.: On the identification of the convex hull of a finite set of points in the plane. Inf. Process. Lett. 2, 18–21 (1973) MATHCrossRefGoogle Scholar
  27. 27.
    Hernandez, C., Schmitt, F.: Multi-stereo 3d object reconstruction. In: 3DPVT, pp. 159–166 (2002) Google Scholar
  28. 28.
    Hoppe, H., DeRose, T., Duchamp, T., McDonald, J., Stuetzle, W.: Surface reconstruction from unorganized points. In: SIGGRAPH, pp. 71–78 (1992) Google Scholar
  29. 29.
    Yemez, Y., Wetherilt, C.J.: A volumetric fusion technique for surface reconstruction from silhouette and range data. Comput. Vis. Image Underst. 105(1), 30–41 (2007) CrossRefGoogle Scholar
  30. 30.
    Poisson Surface Reconstruction package. http://www.cs.jhu.edu/misha/Code/PoissonRecon/
  31. 31.
  32. 32.
  33. 33.
    Dino and Temple data sets. http://vision.middlebury.edu/mview/
  34. 34.
    Furukawa, Y., Ponce, J.: Carved visual hulls for imaged-based modeling. In: ECCV, vol. 1, pp. 564–577 (2006) Google Scholar
  35. 35.
    Vu, H., Keriven, R., Labatut, P., Pons, J.P.: Towards high-resolution large-scale multi-view stereo. In: CVPR (2009) Google Scholar
  36. 36.
    Lhuillier, M., Quan, L.: Match propagation for image-based modeling and rendering. IEEE Trans. Pattern. Anal. Mach. Intell. 24(8), 1140–1146 (2002) CrossRefGoogle Scholar

Copyright information

© Springer-Verlag 2010

Authors and Affiliations

  1. 1.Division of Control and Mechatronics EngineeringHarbin Institute of Technology Shenzhen Graduate SchoolShenzhenChina
  2. 2.Department of Mechanical and Automation EngineeringChinese University of Hong KongShatin, N.T.Hong Kong, China

Personalised recommendations