Advertisement

International Journal of Computer Vision

, Volume 76, Issue 1, pp 13–29 | Cite as

A Theory of Refractive and Specular 3D Shape by Light-Path Triangulation

  • Kiriakos N. KutulakosEmail author
  • Eron Steger
Article

Abstract

We investigate the feasibility of reconstructing an arbitrarily-shaped specular scene (refractive or mirror-like) from one or more viewpoints. By reducing shape recovery to the problem of reconstructing individual 3D light paths that cross the image plane, we obtain three key results. First, we show how to compute the depth map of a specular scene from a single viewpoint, when the scene redirects incoming light just once. Second, for scenes where incoming light undergoes two refractions or reflections, we show that three viewpoints are sufficient to enable reconstruction in the general case. Third, we show that it is impossible to reconstruct individual light paths when light is redirected more than twice. Our analysis assumes that, for every point on the image plane, we know at least one 3D point on its light path. This leads to reconstruction algorithms that rely on an “environment matting” procedure to establish pixel-to-point correspondences along a light path. Preliminary results for a variety of scenes (mirror, glass, etc.) are also presented.

Keywords

3D photography 3D scanning Shape acquisition Multi-media photogrammetry Reflectance modeling Shape from refraction Shape from specularities Triangulation Stereo Transparency Specular reflection 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Agarwal, S., Mallick, S. P., Kriegman, D., & Belongie, S. (2004). On refractive optical flow. In Proceedings of the 8th European conference on computer vision (pp. 483–494). Google Scholar
  2. Baba, M., Ohtani, K., & Imai, M. (2001). New laser range finder for three-dimensional shape measurement of specular objects. Optical Engineering, 40(1), 53–60. CrossRefGoogle Scholar
  3. Ben-Ezra, M., & Nayar, S. (2003). What does motion reveal about transparency? In Proceedings of the 9th international conference on computer vision (pp. 1025–1032). Google Scholar
  4. Blake, A. (1985). Specular stereo. In Proceedings of the international joint conference on artificial intelligence (pp. 973–976). Google Scholar
  5. Bonfort, T., & Sturm, P. (2003). Voxel carving for specular surfaces. In Proceedings of the 9th international conference on computer vision (pp. 591–596). Google Scholar
  6. Bonfort, T., Sturm, P., & Gargallo, P. (2006). General specular surface triangulation. In Proceedings of the Asian conference on computer vision (pp. 872–881). Google Scholar
  7. Bouguet, J.-Y. (2001) MATLAB camera calibration toolbox. http://www.vision.caltech.edu/bouguetj/calib_doc/.
  8. Faugeras, O., & Keriven, R. (1998). Complete dense stereovision using level set methods. In Proceedings of the 5th European conference on computer vision (pp. 379–393). Google Scholar
  9. Fischler, M., & Bolles, R. (1981). Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 24, 381–395. CrossRefMathSciNetGoogle Scholar
  10. Glassner, A. S. (1995). Principles of digital image synthesis. Los Altos: Kaufmann. Google Scholar
  11. Gluckman, J. M., & Nayar, S. K. (1999) Planar catadioptric stereo: geometry and calibration. In Proceedings of the computer vision and pattern recognition conference (pp. 22–28). Google Scholar
  12. Grossberg, M. D., & Nayar, S. K. (2001). A general imaging model and a method for finding its parameters. In Proceedings of the 8th international conference on computer vision (Vol. 2, pp. 108–115). Google Scholar
  13. Guillemin, V., & Pollack, A. (1974). Differential topology. New York: Prentice-Hall. zbMATHGoogle Scholar
  14. Halstead, M. A., Barsky, B. A., Klein, S. A., & Mandell, R. B. (1996). Reconstructing curved surfaces from specular reflection patterns using spline surface fitting of normals. In Proceedings of ACM SIGGRAPH (pp. 335–342). Google Scholar
  15. Hasinoff, S., & Kutulakos, K. N. (2003). Photo-consistent 3d fire by flame sheet decomposition. In Proceedings of the 9th international conference on computer vision (pp. 1184–1191). Google Scholar
  16. Hertzman, A., & Seitz, S. M. (2003). Shape and materials by example: a photometric stereo approach. In Proceedings of the computer vision and pattern recognition conference (pp. 533–540). Google Scholar
  17. Höhle, J. (1971) Reconstruction of the underwater object. In Photogrammetric engineering (pp. 948–954). Google Scholar
  18. Ikeuchi, K. (1981). Determining surface orientations of specular surfaces by using the photometric stereo method. IEEE Transactions on Pattern Analysis and Machine Intelligence, 3(6), 661–669. Google Scholar
  19. Jähne, B., Klinke, J., & Waas, S. (1994). Imaging of short ocean wind waves: a critical theoretical review. Journal of the Optical Society of America A, 11(8), 2197–2209. Google Scholar
  20. Keller, W. C., & Gotwols, B. L. (1983). Two-dimensional optical measurement of wave slope. Applied Optics, 22(22), 3476–3478. CrossRefGoogle Scholar
  21. Koendering, J. J., & van Doorn, A. J. (1979). The structure of two-dimensional scalar fields with applications to vision. Biological Cybernetics, 33, 151–158. CrossRefGoogle Scholar
  22. Kutulakos, K. N., & Steger, E. (2005). A theory of refractive and specular 3d shape by light-path triangulation. In Proceedings of the 10th international conference on computer vision (pp. 1448–1455). Google Scholar
  23. Maas, H.-G. (1995) New developments in multimedia photogrammetry. In Optical 3D measurement techniques III. Karlsruhe: Wichmann. Google Scholar
  24. Matusik, W., Pfister, H., Ziegler, R., Ngan, A., & McMillan, L. (2002). Acquisition and rendering of transparent and refractive objects. In Proceedings of the 13th Eurographics workshop on rendering (pp. 267–278). Google Scholar
  25. Miyazaki, D., Kagesawa, M., & Ikeuchi, K. (2004). Transparent surface modeling from a pair of polarization images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26(1), 73–82. CrossRefGoogle Scholar
  26. Morris, N. J. W. (2004). Image-based water surface reconstruction with refractive stereo. Master’s thesis, Department of Computer Science, University of Toronto. http://www.dgp.toronto.edu/~nmorris/Water/thesis.pdf.
  27. Morris, N., & Kutulakos, K. N. (2005). Dynamic refraction stereo. In Proceedings of the 10th international conference on computer vision (pp. 1573–1580). Google Scholar
  28. Murase, H. (1990). Surface shape reconstruction of an undulating transparent object. In Proceedings of the 3rd international conference on computer vision (pp. 313–317). Google Scholar
  29. Nayar, S. K., Ikeuchi, K., & Kanade, T. (1991). Surface reflection: physical and geometrical perspectives. IEEE Transactions on Pattern Analysis and Machine Intelligence, 13(7), 611–634. CrossRefGoogle Scholar
  30. Nehab, D., Rusinkiewicz, S., Davis, J., & Ramamoorthi, R. (2005). Efficiently combining positions and normals for precise 3d geometry. In Proceedings of ACM SIGGRAPH (pp. 536–543). Google Scholar
  31. Oren, M., & Nayar, S. K. (1995). A theory of specular surface geometry. In Proceedings of the 5th international conference on computer vision (pp. 740–747). Google Scholar
  32. Pless, R. (2002). Two view discrete and differential constraints for generalized imaging systems. In Proceedings of the IEEE workshop on omnidirectional vision (pp. 53–59). Google Scholar
  33. Sanderson, A. C., Weiss, L. E., & Nayar, S. K. (1988). Structured highlight inspection of specular surfaces. IEEE Transactions on Pattern Analysis and Machine Intelligence, 10(1), 44–55. CrossRefGoogle Scholar
  34. Savarese, S., & Perona, P. (2002). Local analysis for 3d reconstruction of specular surfaces, part ii. In Proceedings of the 7th European conference on computer vision (pp. 759–774). Google Scholar
  35. Sharpe, J., Ahlgren, U., Perry, P., Hill, B., Ross, A., Hecksher-Sorensen, J., Baldock, R., & Davidson, D. (2002). Optical projection tomography as a tool for 3d microscopy and gene expression studies. Science, 296, 541–545. CrossRefGoogle Scholar
  36. Steger, E. (2006). Reconstructing transparent objects by refractive light-path triangulation. Master’s thesis, Department of Computer Science, University of Toronto. http://www.cs.toronto.edu/~esteger/thesis.pdf.
  37. Tarini, M., Lensch, H. P. A., Goesele, M., & Seidel, H. -P. (2005). 3D acquisition of mirroring objects using striped patterns. Graphical Models, 67(4), 233–259. CrossRefGoogle Scholar
  38. Trifonov, B., Bradley, D., & Heidrich, W. (2006). Tomographic reconstruction of transparent objects. In Proceedings of the 17th Eurographics symposium on rendering (pp. 51–60). Google Scholar
  39. Wang, J., & Dana, K. J. (2003). A novel approach for texture shape recovery. In Proceedings of the 9th international conference on computer vision (pp. 1374–1380). Google Scholar
  40. Zhang, X., & Cox, C. S. (1994). Measuring the two-dimensional structure of a wavy water surface optically: a surface gradient detector. Experiments in Fluids, 17, 225–237. CrossRefzbMATHGoogle Scholar
  41. Zheng, J. Y., & Murata, A. (2000). Acquiring a complete 3d model from specular motion under the illumination of circular-shaped light sources. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(8), 913–920. CrossRefGoogle Scholar
  42. Zickler, T., Belhumeur, P. N., & Kriegman, D. J. (2002). Helmholz stereopsis: exploiting reciprocity for surface reconstruction. In Proceedings of the 7th European conference on computer vision (pp. 869–884). Google Scholar
  43. Zongker, D. E., Werner, D. M., Curless, B., & Salesin, D. H. (1999). Environment matting and compositing. In Proceedings of ACM SIGGRAPH (pp. 205–214). Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2007

Authors and Affiliations

  1. 1.Department of Computer ScienceUniversity of TorontoTorontoCanada

Personalised recommendations