Advertisement

Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

Embedded Voxel Colouring with Adaptive Threshold Selection Using Globally Minimal Surfaces

Abstract

Image-based 3D reconstruction remains a competitive field of research as state-of-the-art algorithms continue to improve. This paper presents a voxel-based algorithm that adapts the earliest space-carving methods and utilises a minimal surface technique to obtain a cleaner result. Embedded Voxel Colouring is built in two stages: (a) progressive voxel carving is used to build a volume of embedded surfaces and (b) the volume is processed to obtain a surface that maximises photo-consistency data in the volume. This algorithm combines the strengths of classical carving techniques with those of minimal surface approaches. We require only a single pass through the voxel volume, this significantly reduces computation time and is the key to the speed of our approach. We also specify three requirements for volumetric reconstruction: monotonic carving order, causality of carving and water-tightness. Experimental results are presented that demonstrate the strengths of this approach.

This is a preview of subscription content, log in to check access.

References

  1. Appleton, B. (2004). Globally minimal contours and surfaces for image segmentation. Ph.D. thesis, University of Queensland.

  2. Appleton, B., & Talbot, H. (2003). Globally optimal surfaces by continuous maximal flows. In C. Sun, H. Talbot, S. Ourselin, & T. Adriaansen (Eds.), Digital image computing: techniques and applications (Vol. 2, pp. 987–996). Sydney, Australia.

  3. Appleton, B., & Talbot, H. (2006). Globally minimal surfaces by continuous maximal flows. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(1), 106–118.

  4. Beardsley, P., Torr, P., & Zisserman, A. (1996). 3D model acquisition from extended image sequences. In European conference on computer vision (pp. 683–695).

  5. Bhotika, R., Fleet, D., & Kutulakos, K. (2002). A probabilistic theory of occupancy and emptiness. In European conference on computer vision (Vol. 3, pp. 112–132). Copenhagen, Denmark.

  6. Broadhurst, A. (2001). A probabilistic framework for space carving. Ph.D. thesis, Dept. of Engineering, University of Cambridge.

  7. Broadhurst, A., Drummond, T., & Cipolla, R. (2001). A probabilistic framework for space carving. In IEEE international conference of computer vision (pp. 388–393). Vancouver, Canada.

  8. Campbell, N., Vogiatzis, G., Hernandez, C., & Cipolla, R. (2008). Using multiple hypotheses to improve depth-maps for multi-view stereo. In Proceedings of European conference on computer vision (pp. 766–779). Marseille, France.

  9. Carceroni, R., & Kutulakos, K. (2001). Multi-view scene capture by surfel sampling: From video streams to non-rigid 3D motion, shape and reflectance. In IEEE international conference on computer vision (pp. 60–67). Vancouver, Canada.

  10. Caselles, V., Kimmel, R., Sapiro, G., & Sbert, C. (1997). Minimal surfaces based object segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(4), 394–398.

  11. Cohen, L. D., & Kimmel, R. (1997). Global minimum for active contour models: A minimal path approach. International Journal of Computer Vision, 24(1), 57–78.

  12. Colosimo, A., Sarti, A., & Tubaro, S. (2001). Image-based object modeling: A multiresolution level-set approach. In IEEE international conference on image processing (pp. 181–184). Thessaloniki, Greece.

  13. Culbertson, B., Malzbender, T., & Slabaugh, G. (1999). Generalized voxel coloring. In B. Triggs, A. Zisserman, & R. Szeliski (Eds.), Lecture notes in computer science: Vol. 1883. Vision algorithms: Theory and practice (pp. 100–115). Corfu, Greece. Berlin: Springer.

  14. Curless, B., & Levoy, M. (1996). A volumetric method for building complex models from range images. In SIGGRAPH, computer graphics proceedings (pp. 303–312). New Orleans, USA.

  15. Dyer, C. R. (2001). Volumetric scene reconstruction from multiple views. In Foundations of image understanding (pp. 469–489). Boston: Kluwer. Chap. 16.

  16. Faugeras, O., & Keriven, R. (1998). Variational principles, surface evolution, PDE’s, level set methods and the stereo problem. IEEE Transactions on Image Processing, 7(3), 336–344. Special Issue on Partial Differential Equations and Geometry-Driven Diffusion in Image Processing and Analysis.

  17. Freund, J. E. (1992). Mathematical statistics. Englewood Cliffs: Prentice Hall.

  18. Fua, P. (1997). From multiple stereo views to multiple 3D surfaces. International Journal of Computer Vision, 24(1), 19–35.

  19. Fua, P., & Leclerc, Y. (1995). Object-centered surface reconstruction: Combining multi-image stereo and shading. International Journal of Computer Vision, 16(1), 35–56.

  20. Furukawa, Y., & Ponce, J. (2010). Accurate, dense, and robust multiview stereopsis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(8), 1362–1376.

  21. Goldluecke, B., & Magnor, M. (2004). Space-time isosurface evolution for temporally coherent 3D reconstruction. In IEEE conference on computer vision and pattern recognition (Vol. 1, pp. 350–355). Washington D.C., USA.

  22. Graham, R. (1972). An efficient algorithm for determining the convex hull of a finite point set. Information Processing Letters, 1(4), 132–133.

  23. Kimura, M., Saito, H., & Kanade, T. (1999). 3D voxel construction based on epipolar geometry. In IEEE international conference on image processing (Vol. 3, pp. 135–139).

  24. Koch, R. (1995). 3-D surface reconstruction from stereoscopic image sequences. In IEEE international conference on computer vision (pp. 109–114). Cambridge, MA, USA.

  25. Kutulakos, K. N., & Seitz, S. M. (2000). A theory of shape by space carving. International Journal of Computer Vision, 38(3), 199–218.

  26. Laurentini, A. (1994). The visual hull concept for silhouette-based image understanding. IEEE Transactions on Pattern Analysis and Machine Intelligence, 16(2), 150–162.

  27. Leung, C. (2006). Efficient methods for 3D reconstruction from multiple images. Ph.D. thesis, University of Queensland.

  28. Leung, C., Appleton, B., & Sun, C. (2003). Embedded voxel colouring. In C. Sun, H. Talbot, S. Ourselin, & T. Adriaansen (Eds.), Digital image computing: techniques and applications (Vol. 2, pp. 623–632). Sydney, Australia.

  29. Lhuillier, M., & Quan, L. (2005). A quasi-dense approach to surface reconstruction from uncalibrated images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(3), 418–433.

  30. Martin, W. N., & Aggarwal, J. K. (1983). Volumetric descriptions of objects from multiple views. IEEE Transactions on Pattern Analysis and Machine Intelligence, 5(2), 150–158.

  31. MeshLab (2010). http://meshlab.sourceforge.net/.

  32. Moezzi, S., Katkere, A., Kuramura, D. Y., & Jain, R. (1996). Reality modeling and visualization from multiple video sequences. IEEE Computer Graphics and Applications, 16(6), 58–63.

  33. Moezzi, S., Tai, L.-C., & Gerard, P. (1997). Virtual view generation for 3D digital video. IEEE Multimedia, 4(1), 18–26.

  34. Niem, W. (1994). Robust and fast modelling of 3D natural objects from multiple views. In SPIE proceedings image and video processing II (Vol. 2182, pp. 388–397). San Jose.

  35. O’Rouke, J. (1998). Convex hulls in two dimensions. In Computational geometry in C (2nd ed.). Cambridge: Cambridge University Press. Chap. 3.

  36. Pollefeys, M., Koch, R., Vergauwen, M., & Van Gool, L. (1999). Hand-held acquisition of 3D models with a video camera. In International conference on 3D digital imaging and modeling (pp. 14–23). Ottawa, Canada.

  37. Pons, J.-P., Keriven, R., & Faugeras, O. D. (2005). Modelling dynamic scenes by registering multi-view image sequences. In Proceedings of computer vision and pattern recognition (pp. 822–827). San Diego, CA.

  38. Prock, A., & Dyer, C. R. (1998). Towards real-time voxel coloring. In DARPA image understanding workshop (pp. 315–321).

  39. Roy, S., & Cox, I. J. (1998). A maximum-flow formulation of the N-camera correspondence problem. In IEEE international conference on computer vision (pp. 492–499). Bombay, India.

  40. Seitz, S. M., & Dyer, C. R. (1997). Photorealistic scene reconstruction by voxel coloring. In IEEE conference on computer vision and pattern recognition (pp. 1067–1073). Puerto Rico.

  41. Shade, J., Gortler, S., He, L.-W., & Szeliski, R. (1998). Layered depth images. In Proceedings of the 25th annual conference on computer graphics and interactive techniques, SIGGRAPH ’98 (pp. 231–242).

  42. Slabaugh, G. G., Culbertson, W. B., Malzbender, T., & Schafer, R. (2001). A survey of volumetric scene reconstruction methods from photographs. In K. Mueller & A. Kaufman (Eds.), Springer computer science. Volume graphics 2001, joint IEEE TCVG and Eurographics workshop (pp. 81–100). Stony Brook, New York, USA.

  43. Slabaugh, G. G., Schafer, R. W., & Hans, M. C. (2002). Multi-resolution space carving using level sets methods. In IEEE international conference on image processing (Vol. 2, pp. 545–548). Rochester, USA.

  44. Slabaugh, G. G., Culbertson, W. B., Malzbender, T., Stevens, M. R., & Schafer, R. W. (2004). Methods for volumetric reconstruction of visual scenes. International Journal of Computer Vision, 57(3), 179–199.

  45. Snow, D., Viola, P., & Zabih, R. (2000). Exact voxel occupancy with graph cuts. In IEEE conference on computer vision and pattern recognition (pp. 345–352).

  46. Stevens, M., Culbertson, B., & Malzbender, T. (2002). A histogram-based color consistency test for voxel coloring. In International conference on pattern recognition (Vol. 4, pp. 118–121). Quebec, Canada.

  47. Sun, C. (2002). Fast stereo matching using rectangular subregioning and 3D maximum-surface techniques. International Journal of Computer Vision, 47(1/2/3), 99–117.

  48. Vedula, S., Baker, S., Seitz, S., & Kanade, T. (2000). Shape and motion carving in 6D. In IEEE conference on computer vision and pattern recognition (Vol. 2, pp. 592–598). Hilton Head Island, SC, USA.

  49. Vu, H. H., Keriven, R., Labatut, P., & Pons, J.-P. (2009). Towards high-resolution large-scale multi-view stereo. In IEEE conference on computer vision and pattern recognition (pp. 1430–1437). Miami, FL.

  50. Weghorst, H., Hooper, G., & Greenberg, D. P. (1984). Improved computational methods for ray tracing. ACM Transactions on Graphics, 3(1), 52–69.

  51. Yezzi, A., & Soatto, S. (2003). Stereoscopic segmentation. International Journal of Computer Vision, 53(1), 31–43.

  52. Yezzi, A., Slabaugh, G., Broadhurst, A., Cipolla, R., & Schafer, R. (2002). A surface evolution approach to probabilistic space carving. In International symposium on 3D processing, visualization, and transmission (pp. 612–621). Padova, Italy.

  53. Zeng, G., Paris, S., Quan, L., & Sillion, F. (2007). Accurate and scalable surface representation and reconstruction from images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(1), 141–158.

Download references

Author information

Correspondence to Changming Sun.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Leung, C., Appleton, B., Buckley, M. et al. Embedded Voxel Colouring with Adaptive Threshold Selection Using Globally Minimal Surfaces. Int J Comput Vis 99, 215–231 (2012). https://doi.org/10.1007/s11263-012-0525-8

Download citation

Keywords

  • Volumetric 3D reconstruction
  • Embedded voxel colouring
  • Globally minimal surfaces
  • Monotonic carving order
  • Causality of carving
  • Water-tightness