International Journal of Computer Vision

, Volume 99, Issue 2, pp 215–231 | Cite as

Embedded Voxel Colouring with Adaptive Threshold Selection Using Globally Minimal Surfaces

  • Carlos Leung
  • Ben Appleton
  • Mitchell Buckley
  • Changming Sun
Article

Abstract

Image-based 3D reconstruction remains a competitive field of research as state-of-the-art algorithms continue to improve. This paper presents a voxel-based algorithm that adapts the earliest space-carving methods and utilises a minimal surface technique to obtain a cleaner result. Embedded Voxel Colouring is built in two stages: (a) progressive voxel carving is used to build a volume of embedded surfaces and (b) the volume is processed to obtain a surface that maximises photo-consistency data in the volume. This algorithm combines the strengths of classical carving techniques with those of minimal surface approaches. We require only a single pass through the voxel volume, this significantly reduces computation time and is the key to the speed of our approach. We also specify three requirements for volumetric reconstruction: monotonic carving order, causality of carving and water-tightness. Experimental results are presented that demonstrate the strengths of this approach.

Keywords

Volumetric 3D reconstruction Embedded voxel colouring Globally minimal surfaces Monotonic carving order Causality of carving Water-tightness 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Appleton, B. (2004). Globally minimal contours and surfaces for image segmentation. Ph.D. thesis, University of Queensland. Google Scholar
  2. Appleton, B., & Talbot, H. (2003). Globally optimal surfaces by continuous maximal flows. In C. Sun, H. Talbot, S. Ourselin, & T. Adriaansen (Eds.), Digital image computing: techniques and applications (Vol. 2, pp. 987–996). Sydney, Australia. Google Scholar
  3. Appleton, B., & Talbot, H. (2006). Globally minimal surfaces by continuous maximal flows. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(1), 106–118. CrossRefGoogle Scholar
  4. Beardsley, P., Torr, P., & Zisserman, A. (1996). 3D model acquisition from extended image sequences. In European conference on computer vision (pp. 683–695). Google Scholar
  5. Bhotika, R., Fleet, D., & Kutulakos, K. (2002). A probabilistic theory of occupancy and emptiness. In European conference on computer vision (Vol. 3, pp. 112–132). Copenhagen, Denmark. Google Scholar
  6. Broadhurst, A. (2001). A probabilistic framework for space carving. Ph.D. thesis, Dept. of Engineering, University of Cambridge. Google Scholar
  7. Broadhurst, A., Drummond, T., & Cipolla, R. (2001). A probabilistic framework for space carving. In IEEE international conference of computer vision (pp. 388–393). Vancouver, Canada. Google Scholar
  8. Campbell, N., Vogiatzis, G., Hernandez, C., & Cipolla, R. (2008). Using multiple hypotheses to improve depth-maps for multi-view stereo. In Proceedings of European conference on computer vision (pp. 766–779). Marseille, France. Google Scholar
  9. Carceroni, R., & Kutulakos, K. (2001). Multi-view scene capture by surfel sampling: From video streams to non-rigid 3D motion, shape and reflectance. In IEEE international conference on computer vision (pp. 60–67). Vancouver, Canada. Google Scholar
  10. Caselles, V., Kimmel, R., Sapiro, G., & Sbert, C. (1997). Minimal surfaces based object segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(4), 394–398. MathSciNetCrossRefGoogle Scholar
  11. Cohen, L. D., & Kimmel, R. (1997). Global minimum for active contour models: A minimal path approach. International Journal of Computer Vision, 24(1), 57–78. CrossRefGoogle Scholar
  12. Colosimo, A., Sarti, A., & Tubaro, S. (2001). Image-based object modeling: A multiresolution level-set approach. In IEEE international conference on image processing (pp. 181–184). Thessaloniki, Greece. Google Scholar
  13. Culbertson, B., Malzbender, T., & Slabaugh, G. (1999). Generalized voxel coloring. In B. Triggs, A. Zisserman, & R. Szeliski (Eds.), Lecture notes in computer science: Vol. 1883. Vision algorithms: Theory and practice (pp. 100–115). Corfu, Greece. Berlin: Springer. CrossRefGoogle Scholar
  14. Curless, B., & Levoy, M. (1996). A volumetric method for building complex models from range images. In SIGGRAPH, computer graphics proceedings (pp. 303–312). New Orleans, USA. Google Scholar
  15. Dyer, C. R. (2001). Volumetric scene reconstruction from multiple views. In Foundations of image understanding (pp. 469–489). Boston: Kluwer. Chap. 16. CrossRefGoogle Scholar
  16. Faugeras, O., & Keriven, R. (1998). Variational principles, surface evolution, PDE’s, level set methods and the stereo problem. IEEE Transactions on Image Processing, 7(3), 336–344. Special Issue on Partial Differential Equations and Geometry-Driven Diffusion in Image Processing and Analysis. MathSciNetMATHCrossRefGoogle Scholar
  17. Freund, J. E. (1992). Mathematical statistics. Englewood Cliffs: Prentice Hall. MATHGoogle Scholar
  18. Fua, P. (1997). From multiple stereo views to multiple 3D surfaces. International Journal of Computer Vision, 24(1), 19–35. CrossRefGoogle Scholar
  19. Fua, P., & Leclerc, Y. (1995). Object-centered surface reconstruction: Combining multi-image stereo and shading. International Journal of Computer Vision, 16(1), 35–56. CrossRefGoogle Scholar
  20. Furukawa, Y., & Ponce, J. (2010). Accurate, dense, and robust multiview stereopsis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(8), 1362–1376. CrossRefGoogle Scholar
  21. Goldluecke, B., & Magnor, M. (2004). Space-time isosurface evolution for temporally coherent 3D reconstruction. In IEEE conference on computer vision and pattern recognition (Vol. 1, pp. 350–355). Washington D.C., USA. Google Scholar
  22. Graham, R. (1972). An efficient algorithm for determining the convex hull of a finite point set. Information Processing Letters, 1(4), 132–133. MATHCrossRefGoogle Scholar
  23. Kimura, M., Saito, H., & Kanade, T. (1999). 3D voxel construction based on epipolar geometry. In IEEE international conference on image processing (Vol. 3, pp. 135–139). Google Scholar
  24. Koch, R. (1995). 3-D surface reconstruction from stereoscopic image sequences. In IEEE international conference on computer vision (pp. 109–114). Cambridge, MA, USA. Google Scholar
  25. Kutulakos, K. N., & Seitz, S. M. (2000). A theory of shape by space carving. International Journal of Computer Vision, 38(3), 199–218. MATHCrossRefGoogle Scholar
  26. Laurentini, A. (1994). The visual hull concept for silhouette-based image understanding. IEEE Transactions on Pattern Analysis and Machine Intelligence, 16(2), 150–162. CrossRefGoogle Scholar
  27. Leung, C. (2006). Efficient methods for 3D reconstruction from multiple images. Ph.D. thesis, University of Queensland. Google Scholar
  28. Leung, C., Appleton, B., & Sun, C. (2003). Embedded voxel colouring. In C. Sun, H. Talbot, S. Ourselin, & T. Adriaansen (Eds.), Digital image computing: techniques and applications (Vol. 2, pp. 623–632). Sydney, Australia. Google Scholar
  29. Lhuillier, M., & Quan, L. (2005). A quasi-dense approach to surface reconstruction from uncalibrated images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(3), 418–433. CrossRefGoogle Scholar
  30. Martin, W. N., & Aggarwal, J. K. (1983). Volumetric descriptions of objects from multiple views. IEEE Transactions on Pattern Analysis and Machine Intelligence, 5(2), 150–158. CrossRefGoogle Scholar
  31. Moezzi, S., Katkere, A., Kuramura, D. Y., & Jain, R. (1996). Reality modeling and visualization from multiple video sequences. IEEE Computer Graphics and Applications, 16(6), 58–63. CrossRefGoogle Scholar
  32. Moezzi, S., Tai, L.-C., & Gerard, P. (1997). Virtual view generation for 3D digital video. IEEE Multimedia, 4(1), 18–26. CrossRefGoogle Scholar
  33. Niem, W. (1994). Robust and fast modelling of 3D natural objects from multiple views. In SPIE proceedings image and video processing II (Vol. 2182, pp. 388–397). San Jose. Google Scholar
  34. O’Rouke, J. (1998). Convex hulls in two dimensions. In Computational geometry in C (2nd ed.). Cambridge: Cambridge University Press. Chap. 3. Google Scholar
  35. Pollefeys, M., Koch, R., Vergauwen, M., & Van Gool, L. (1999). Hand-held acquisition of 3D models with a video camera. In International conference on 3D digital imaging and modeling (pp. 14–23). Ottawa, Canada. Google Scholar
  36. Pons, J.-P., Keriven, R., & Faugeras, O. D. (2005). Modelling dynamic scenes by registering multi-view image sequences. In Proceedings of computer vision and pattern recognition (pp. 822–827). San Diego, CA. Google Scholar
  37. Prock, A., & Dyer, C. R. (1998). Towards real-time voxel coloring. In DARPA image understanding workshop (pp. 315–321). Google Scholar
  38. Roy, S., & Cox, I. J. (1998). A maximum-flow formulation of the N-camera correspondence problem. In IEEE international conference on computer vision (pp. 492–499). Bombay, India. Google Scholar
  39. Seitz, S. M., & Dyer, C. R. (1997). Photorealistic scene reconstruction by voxel coloring. In IEEE conference on computer vision and pattern recognition (pp. 1067–1073). Puerto Rico. Google Scholar
  40. Shade, J., Gortler, S., He, L.-W., & Szeliski, R. (1998). Layered depth images. In Proceedings of the 25th annual conference on computer graphics and interactive techniques, SIGGRAPH ’98 (pp. 231–242). CrossRefGoogle Scholar
  41. Slabaugh, G. G., Culbertson, W. B., Malzbender, T., & Schafer, R. (2001). A survey of volumetric scene reconstruction methods from photographs. In K. Mueller & A. Kaufman (Eds.), Springer computer science. Volume graphics 2001, joint IEEE TCVG and Eurographics workshop (pp. 81–100). Stony Brook, New York, USA. Google Scholar
  42. Slabaugh, G. G., Schafer, R. W., & Hans, M. C. (2002). Multi-resolution space carving using level sets methods. In IEEE international conference on image processing (Vol. 2, pp. 545–548). Rochester, USA. Google Scholar
  43. Slabaugh, G. G., Culbertson, W. B., Malzbender, T., Stevens, M. R., & Schafer, R. W. (2004). Methods for volumetric reconstruction of visual scenes. International Journal of Computer Vision, 57(3), 179–199. CrossRefGoogle Scholar
  44. Snow, D., Viola, P., & Zabih, R. (2000). Exact voxel occupancy with graph cuts. In IEEE conference on computer vision and pattern recognition (pp. 345–352). Google Scholar
  45. Stevens, M., Culbertson, B., & Malzbender, T. (2002). A histogram-based color consistency test for voxel coloring. In International conference on pattern recognition (Vol. 4, pp. 118–121). Quebec, Canada. Google Scholar
  46. Sun, C. (2002). Fast stereo matching using rectangular subregioning and 3D maximum-surface techniques. International Journal of Computer Vision, 47(1/2/3), 99–117. MATHCrossRefGoogle Scholar
  47. Vedula, S., Baker, S., Seitz, S., & Kanade, T. (2000). Shape and motion carving in 6D. In IEEE conference on computer vision and pattern recognition (Vol. 2, pp. 592–598). Hilton Head Island, SC, USA. Google Scholar
  48. Vu, H. H., Keriven, R., Labatut, P., & Pons, J.-P. (2009). Towards high-resolution large-scale multi-view stereo. In IEEE conference on computer vision and pattern recognition (pp. 1430–1437). Miami, FL. Google Scholar
  49. Weghorst, H., Hooper, G., & Greenberg, D. P. (1984). Improved computational methods for ray tracing. ACM Transactions on Graphics, 3(1), 52–69. CrossRefGoogle Scholar
  50. Yezzi, A., & Soatto, S. (2003). Stereoscopic segmentation. International Journal of Computer Vision, 53(1), 31–43. MathSciNetCrossRefGoogle Scholar
  51. Yezzi, A., Slabaugh, G., Broadhurst, A., Cipolla, R., & Schafer, R. (2002). A surface evolution approach to probabilistic space carving. In International symposium on 3D processing, visualization, and transmission (pp. 612–621). Padova, Italy. Google Scholar
  52. Zeng, G., Paris, S., Quan, L., & Sillion, F. (2007). Accurate and scalable surface representation and reconstruction from images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(1), 141–158. CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2012

Authors and Affiliations

  • Carlos Leung
    • 1
    • 2
  • Ben Appleton
    • 3
  • Mitchell Buckley
    • 4
    • 5
  • Changming Sun
    • 4
  1. 1.Intelligent Real-Time Imaging and Sensing Group, ITEEThe University of QueenslandBrisbaneAustralia
  2. 2.SuncorpMelbourneAustralia
  3. 3.Google Inc.SydneyAustralia
  4. 4.CSIRO Mathematics, Informatics and StatisticsNorth RydeAustralia
  5. 5.Macquarie UniversitySydneyAustralia

Personalised recommendations