Skip to main content
SpringerLink
Log in

Inverse lighting and photorealistic rendering for augmented reality

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

We present a practical and robust photorealistic rendering pipeline for augmented reality. We solve the real world lighting conditions from observations of a diffuse sphere or a rotated marker. The solution method is based on l 1-regularized least squares minimization, yielding a sparse set of light sources readily usable with most rendering methods. The framework also supports the use of more complex light source representations. Once the lighting conditions are solved, we render the image using modern real-time rendering methods such as shadow maps with variable softness, ambient occlusion, advanced BRDF’s and approximate reflections and refractions. Finally, we perform post-processing on the resulting images in order to match the various aberrations and defects typically found in the underlying real-world video.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Agusanto, K., Li, L., Chuangui, Z., Sing, N.W.: Photorealistic rendering for augmented reality using environment illumination. In: Proceedings of ISMAR ’03, p. 208. IEEE Comput. Soc., Los Alamitos (2003)

    Google Scholar 

  2. Akenine-Möller, T., Haines, E., Hoffman, N.: Real-Time Rendering, 3rd edn. AK Peters, Natick (2008)

    Google Scholar 

  3. Annen, T., Dong, Z., Mertens, T., Bekaert, P., Seidel, H., Kautz, J.: Real-time, all-frequency shadows in dynamic scenes. In: SIGGRAPH ’08, pp. 1–8. ACM, New York (2008)

    Chapter  Google Scholar 

  4. Ashikhmin, M., Shirley, P.: An anisotropic Phong BRDF model. J. Graph. Tools 5(2), 25–32 (2000)

    Google Scholar 

  5. Azuma, R., Baillot, Y., Behringer, R., Feiner, S., Julier, S., Macintyre, B.: Recent advances in augmented reality. Comput. Graph. Appl. IEEE 21), 34–47 (2001)

    Article  Google Scholar 

  6. Bavoil, L., Sainz, M.: Image-space horizon-based ambient occlusion. In: ShaderX 7, pp. 425–444. Charles River Media (2009)

  7. Basri, R., Jacobs, D.: Lambertian reflectance and linear subspaces. IEEE Trans. Pattern Anal. Mach. Intell. 25(2), 218–233 (2003)

    Article  Google Scholar 

  8. Chen, S., Donoho, D., Saunders, M.: Atomic decomposition by basis pursuit. SIAM J. Sci. Comput. 20, 33–61 (1998)

    Article  MathSciNet  Google Scholar 

  9. Debevec, P.: Rendering synthetic objects into real scenes: bridging traditional and image-based graphics with global illumination and high dynamic range photography. In: Proceedings of SIGGRAPH ’98 pp. 189–198. ACM, New York (1998)

    Chapter  Google Scholar 

  10. Duda, R.O., Hart, P.E.: Use of the Hough transformation to detect lines and curves in pictures. Commun. ACM 15(1), 11–15 (1972)

    Article  Google Scholar 

  11. Franc, V., Hlavac, V., Navara, M.: Sequential coordinate-wise algorithm for the non-negative least squares problem. CAIP 2005(1), 407–414 (2005)

    Google Scholar 

  12. Gierlinger, T., Danch, D., Stork, A.: Rendering techniques for mixed reality. J. Real-Time Image Process. (2009)

  13. Golub, G.H., Van Loan, C.F.: Matrix Computations. Johns Hopkins Press, Baltimore (1996)

    MATH  Google Scholar 

  14. Gritz, L.: The importance of being linear. In: GPU Gems 3, pp. 529–542. Addison-Wesley, Reading (2007)

    Google Scholar 

  15. Haller, M., Drab, S., Hartmann, W.: A real-time shadow approach for an augmented reality application using shadow volumes. In: Proceedings of VRST ’03, pp. 56–65. ACM, New York (2003)

    Chapter  Google Scholar 

  16. Hensley, J., Scheuermann, T., Coombe, G., Singh, M., Lastra, A.: Fast summed-area table generation and its applications. Comput. Graph. Forum 24(3), 547–555 (2005)

    Article  Google Scholar 

  17. Jacobs, K., Loscos, C.: Classification of illumination methods for mixed reality. Comput. Graph. Forum 25(1), 29–51 (2006)

    Article  Google Scholar 

  18. Kajiya, J.T.: The rendering equation. SIGGRAPH Comput. Graph. 20(4), 143–150 (1986)

    Article  Google Scholar 

  19. Klein, G., Murray, D.: Compositing for small cameras. In: Proceedings of ISMAR ’08, pp. 57–60. IEEE Comput. Soc., Washington (2008)

    Google Scholar 

  20. Malgouyres, F., Zeng, T.: A predual proximal point algorithm solving a non negative basis pursuit denoising model. Int. J. Comput. Vis. 83, 294–311 (2009)

    Article  Google Scholar 

  21. Mallat, S., Zhang, Z.: Matching pursuits with time-frequency dictionaries. IEEE Trans. Signal Process. 41, 3397–3415 (1993)

    Article  MATH  Google Scholar 

  22. Marschner, S.R., Marschner, S.R., Greenberg, D.P.: Inverse lighting for photography. In: Fifth Color Imaging Conference, Society for Imaging Science and Technology, pp. 262–265 (1997)

  23. Mei, X., Ling, H., Jacobs, D.: Sparse representation of cast shadows via l1-regularized least squares. In: IEEE International Conference on Computer Vision (2009)

  24. Okumura, B., Kanbara, M., Yokoya, N.: Augmented reality based on estimation of defocusing and motion blurring from captured images. In: Proceedings of ISMAR ’06, pp. 219–225. IEEE Comput. Soc., Los Alamitos (2006)

    Google Scholar 

  25. Papagiannakis, G., Foni, A., Magnenat-Thalmann, N.: Practical precomputed radiance transfer for mixed reality. In: Proceedings of VSMM ’05, pp. 189–199, VSMM, Society (2005)

  26. Pilet, J., Geiger, A., Lagger, P., Lepetit, V., Fua, P.: An all-in-one solution to geometric and photometric calibration. In: Proceedings of ISMAR ’06, pp. 69–78. IEEE Comput. Soc., Los Alamitos (2006)

    Google Scholar 

  27. Porter, T., Duff, T.: Compositing digital images. In: Proceedings of SIGGRAPH ’84, pp. 253–259. ACM, New York (1984)

    Chapter  Google Scholar 

  28. Ramamoorthi, R., Hanrahan, P.: An efficient representation for irradiance environment maps. In: Proceedings of SIGGRAPH ’01, pp. 407–500. ACM, New York (2001)

    Google Scholar 

  29. Ramamoorthi, R., Hanrahan, P.: A signal-processing framework for inverse rendering. In: Proceedings of SIGGRAPH ’01, pp. 117–128. ACM, New York (2001)

    Chapter  Google Scholar 

  30. Sato, I., Sato, Y., Ikeuchi, K.: Acquiring a radiance distribution to superimpose virtual objects onto a real scene. IEEE Trans. Vis. Comput. Graph. 5(1), 1–12 (1999)

    Article  Google Scholar 

  31. Sato, I., Sato, Y., Ikeuchi, K.: Illumination from shadows. IEEE Trans. Pattern Anal. Mach. Intell. 25(3), 290–300 (2003)

    Article  Google Scholar 

  32. Schlick, C.: An inexpensive BRDF model for physically-based rendering. Comput. Graph. Forum 13, 233–246 (1994)

    Article  Google Scholar 

  33. Sloan, P., Kautz, J., Snyder, J.: Precomputed radiance transfer for real-time rendering in dynamic, low-frequency lighting environments. In: Proceedings of SIGGRAPH ’02, pp. 527–536. ACM, New York (2002)

    Chapter  Google Scholar 

  34. Sugano, N., Kato, H., Tachibana, K.: The effects of shadow representation of virtual objects in augmented reality. In: Proceedings of ISMAR ’03, p. 76. IEEE Comput. Soc., Los Alamitos (2003)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. VTT Technical Research Centre of Finland, Vuorimiehentie 3, Espoo, 02044, VTT, Finland

    Miika Aittala

Authors
  1. Miika Aittala
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Miika Aittala.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Aittala, M. Inverse lighting and photorealistic rendering for augmented reality. Vis Comput 26, 669–678 (2010). https://doi.org/10.1007/s00371-010-0501-7

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-010-0501-7

Keywords

Search

Navigation