International Journal of Computer Vision

, Volume 49, Issue 2–3, pp 229–246 | Cite as

Relighting with the Reflected Irradiance Field: Representation, Sampling and Reconstruction

  • Zhouchen Lin
  • Tien-Tsin Wong
  • Heung-Yeung Shum
Article

Abstract

Image-based relighting (IBL) is a technique to change the illumination of an image-based object/scene. In this paper, we define a representation called the reflected irradiance field which records the light reflected from a scene as viewed at a fixed viewpoint as a result of moving a point light source on a plane. It synthesizes a novel image under a different illumination by interpolating and superimposing appropriate recorded samples. Furthermore, we study the minimum sampling problem of the reflected irradiance field, i.e., how many light source positions are needed. We find that there exists a geometry-independent bound for the sampling interval whenever the second-order derivatives of the surface BRDF and the minimum depth of the scene are bounded. This bound ensures that when the novel light source is on the plane, the error in the reconstructed image is controlled by a given tolerance, regardless of the geometry. We also analyze the bound of depth error so that the extra reconstruction error can also be governed when the novel light source is off-plane. Experiments on both synthetic and real surfaces are conducted to verify our analysis.

sampling BRDF light field Lumigraph plenoptic functions image-based rendering relighting 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Adelson, E.H. and Bergen, J.R. 1991. The plenoptic function and the elements of early vision. In Computational Models of Visual Processing, M.S. Landy and J.A. Movshon (Eds.). MIT Press: Cambridge, MA, Ch. 1, pp. 3–20.Google Scholar
  2. Amanatides, J. 1992. Algorithms for the detection and elimination of specular aliasing. In Proceedings of Graphics Interface'92, Canadian Information Processing Society, pp. 86–93.Google Scholar
  3. Belhumeur, P.N. and Kriegman, D.J. 1996. What is the set of images of an object under all possible lighting conditions? In IEEE Conference on Computer Vision and Pattern Recognition (CVPR'96), pp. 270–277.Google Scholar
  4. Bouguet, J.-Y. 2000. Camera calibration toolbox for Matlab. available at http://www.vision.caltech.edu/bouguetj/calib_doc/.Google Scholar
  5. Chai, J.X., Tong, X., Chan, C.-C., and Shum, H.-Y. 2000. Plenoptic sampling. In Computer Graphics, SIGGRAPH'2000 Proceedings, ACM SIGGRAPH, pp. 307–318.Google Scholar
  6. Chen, S.E. 1995. QuickTime VR-an image-based approach to virtual environment navigation. In Computer Graphics, SIGGRAPH'95 Proceedings, ACM SIGGRAPH, pp. 29–38.Google Scholar
  7. Cook, R.L., Porter, T., and Carpenter, L. 1984. Distributed ray tracing. In Computer Graphics, SIGGRAPH'84 Proceedings,ACM SIGGRAPH, pp. 137–145.Google Scholar
  8. Debevec, P.E., Hawkins, T., Tchou, C., Duiker, H.-P., Sarokin, W., and Sagar, M. 2000. Acquiring the reflectance field of a human face. In Computer Graphics, SIGGRAPH'2000 Proceedings, ACM SIGGRAPH, pp. 145–156.Google Scholar
  9. Debevec, P.E. and Malik, J. 1997. Recovering high dynamic range radiance maps from photographs. In Computer Graphics, SIGGRAPH'97 Proceedings, ACM SIGGRAPH, pp. 369–378.Google Scholar
  10. Ferwerda, J.A., Pattanaik, S.N., Shirley, P., and Greenberg, D.P. 1997. A model of visual masking for computer graphics. In Computer Graphics, SIGGRAPH'97 Proceedings, ACM SIGGRAPH, pp. 143–152.Google Scholar
  11. Fournier, A., Gunawan, A.S., and Romanzin, C. 1993. Common illumination between real and computer generated scenes. In Proceedings of Graphics Interface'93, Toronto, ON, Canada, pp. 254–262.Google Scholar
  12. Gibson, S. and Murta, A. 2000. Interactive rendering with real-world illumination. In Eurographics Workshop on Rendering (Rendering Techniques'2000), Brno, Czhech Republic, pp. 365–376.Google Scholar
  13. Golub, G. and Loanc, C. van 1989. Matrix Computations. The John Hopkins University Press.Google Scholar
  14. Gortler, S.J., Grzeszczuk, R., Szeliski, R., and Cohen, M.F. 1996. The lumigraph. In Computer Graphics, SIGGRAPH'96 Proceedings, ACM SIGGRAPH, pp. 43–54.Google Scholar
  15. Haeberli, P. 1992. Synthetic lighting for photography. available at http://www.sgi.com/grafica/synth/index.html.Google Scholar
  16. Hanrahan, P. and Krueger, W. 1993. Reflection from layered surfaces due to subsurface scattering. In Computer Graphics, SIGGRAPH'93 Proceedings, ACM SIGGRAPH, pp. 43–54.Google Scholar
  17. Horn, B. 1986. Robot Vision. MIT Press.Google Scholar
  18. Isaksen, A., McMillan, L., and Gortler, S.J. 2000. Dynamically reparameterized light fields. In Computer Graphics, SIGGRAPH'2000 Proceedings, ACM SIGGRAPH, pp. 297–306.Google Scholar
  19. Lafortune, E.P.F. 2000. Personal communications.Google Scholar
  20. Lafortune, E.P.F., Foo, S.-C., Torrance, K.E., and Greenberg, D.P. 1997. Non-linear approximation of reflectance functions. In Computer Graphics, SIGGRAPH'97 Proceedings, ACM SIGGRAPH, pp. 117–126.Google Scholar
  21. Levoy, M. and Hanrahan, P. 1996. Light field rendering. In Computer Graphics, SIGGRAPH'96 Proceedings, ACM SIGGRAPH, pp. 31–42.Google Scholar
  22. Lin, Z. and Shum, H.-Y. 2000. On the number of samples needed in light field rendering with constant-depth assumption. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR'2000), Hilton Head Island, South Carolina, USA, pp. 588–579.Google Scholar
  23. Lin, Z., Wong, T.-T., and Shum, H.-Y. 2001. Relighting with the reflected irradiance field: Representation, sampling and reconstruction. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR'01), Kauai, Hawaii, USA, pp. 561–567.Google Scholar
  24. Loscos, C., Drettakis, G., and Robert, L. 2000. Interactive virtual relighting of real scenes. IEEE Trans. on Visualization and Computer Graphics, 6(3):289–305.Google Scholar
  25. Magda, S., Zickler, T., Kriegman, D., and Belhumeur, P. 2001. Beyond Lambert: Reconstructing surfaces with arbitrary BRDFs. In Proceedings of the International Conference on Computer Vision (ICCV'01), Vancouver, Canada, pp. 391–398.Google Scholar
  26. McMillan, L. and Bishop, G. 1995. Plenoptic modeling: An image-based rendering system. In Computer Graphics, SIGGRAPH'95 Proceedings, ACM SIGGRAPH, pp. 39–46.Google Scholar
  27. Miller, G.S.P., Rubin, S., and Ponceleon, D. 1998. Lazy decompression of surface light fields for precomputed global illumination. In Eurographics Workshop on Rendering (Rendering Techniques'98), Vienna, Australia, pp. 281–2928.Google Scholar
  28. Nimeroff, J.S., Simoncelli, E., and Dorsey, J. 1994. Efficient re-rendering of naturally illuminated environments. In Eurographics Workshop on Rendering (Rendering Techniques'94), Darmstadt, Germany, pp. 359–373.Google Scholar
  29. Oren, M. and Nayar, S.K. 1994. Generalization of Lambert's reflectance model. In Computer Graphics, SIGGRAPH'94 Proceedings, ACM SIGGRAPH, pp. 239–246.Google Scholar
  30. Pattanaik, S.N, Ferwerda, J.A., Fairchild, M.D., and Greenberg, D.P. 1998. A multiscale model of adaptation and spatial vision for realistic image display. In Computer Graphics, SIGGRAPH'98 Proceedings, ACM SIGGRAPH, pp. 287–298.Google Scholar
  31. Pellacini, F., Ferwerda, J.A., and Greenberg, D.P. 2000. Toward a psychophysically-based light reflection model for image synthesis. In Computer Graphics, SIGGRAPH'2000 Proceedings,ACM SIGGRAPH, pp. 55–64.Google Scholar
  32. Press, W.H., Teukolsky, S.A., Vetterling, W.T., and Flannery, B.P. 1997. Numerical Recipies in C. Cambridge University Press.Google Scholar
  33. Sato, Y., Wheeler, M.D., and Ikeuchi, K. 1997. Object shape and reflectance modeling from observation. In Computer Graphics, SIGGRAPH'97 Proceedings, ACM SIGGRAPH, pp. 379–387.Google Scholar
  34. Shashua, A. 1997. On photometric issues in 3D visual recognition from a single 2D image. Int. Journal of Computer Vision, 21:99–122.Google Scholar
  35. Shum, H.-Y. and He, L.-W. 1999. Rendering with concentric mosaics. In Computer Graphics, SIGGRAPH'99 Proceedings,ACM SIGGRAPH, pp. 299–306.Google Scholar
  36. Wang, R.H. 1999. Numerical Approximation (in Chinese). Higher Education Press, Beijing.Google Scholar
  37. Ward, G.J. 1992. Measuring and modeling anisotropic reflection. In Computer Graphics, SIGGRAPH'92 Proceedings,ACM SIGGRAPH, pp. 265–272.Google Scholar
  38. Wong, T.-T. 2001. irrad demo (version 1.0). available at http://www.cse.cuhk.edu.hk/~ttwong/demo/irrad/irrad.html.Google Scholar
  39. Wong, T.-T., Fu, C.-W., Heng, P.-A., and Leung, C.-S. 2001. The plenoptic illumination function. IEEE Transactions on Multimedia, to appear.Google Scholar
  40. Wong, T.-T., Heng, P.A., Or, S.-H., and Ng, W.-Y. 1997. Image-based rendering with controllable illumination. In Eurographics Workshop on Rendering (Rendering Techniques'97), Saint Etienne, France, pp. 13–22.Google Scholar
  41. Yu, Y., Debevec, P., Malik, J., and Hawkins, T. 1999. Inverse global illumination: Recovering reflectance models of real scenes from photographs. In Computer Graphics, SIGGRAPH'99 Proceedings, ACM SIGGRAPH, pp. 215–224.Google Scholar
  42. Yu, Y. and Malik, J. 1998. Recovering photometric properties of architectural scenes from photographs. In Computer Graphics, SIGGRAPH'98 Proceedings. ACM SIGGRAPH, pp. 207–217.Google Scholar
  43. Zhang, Z. 1998. Modeling geometric structure and illumination variation of a scene from real images. In Proceedings of the International Conference on Computer Vision (ICCV'98), Bombay, India, pp. 1041–1046.Google Scholar
  44. Zhang, Z. 1999. Flexible camera calibration by viewing a plane from unknown orientations. In Proceedings of the International Conference on Computer Vision (ICCV'99), Kerkyra, Greece, pp. 666–673.Google Scholar

Copyright information

© Kluwer Academic Publishers 2002

Authors and Affiliations

  • Zhouchen Lin
    • 1
  • Tien-Tsin Wong
    • 2
  • Heung-Yeung Shum
    • 1
  1. 1.Microsoft Research, AsiaChina
  2. 2.The Chinese University of Hong KongPeople's Republic of China

Personalised recommendations