Abstract
In augmented reality, it is essential that the rendered virtual objects are embedded harmonically into the view of the background scenes and their appearance should reflect the changing lighting condition of the real scene to ensure illumination consistency. In this paper, we propose a novel method to solve for the sunlight and skylight basis images of static outdoor scenes from a time-lapse image sequence. It is proved that the resulted basis images encapsulate the geometry and material reflectivity of the scene, correspond to the global illumination effects of the outdoor scene under a unit intensity of the sunlight and skylight. Our method is fully automatic. Unlike previous methods, it gets rid of the constraints that the reflectance of all objects in scenes should be ideal diffuse, or the weather condition should be overcast or sunshine. During decomposition, we first detect shadowed pixels by analyzing the time-lapse curve of each pixel through k-means clustering, the basis images of sunlight and skylight are then solved by an iterative procedure with the decomposition equation. The basis images are further optimized by exploiting their constraints and priors. Experimental results demonstrate the effectiveness and flexibility of the proposed method. Our method can also be applied in image understanding and compressing.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Andersen, M., Jensen, T., Madsen, C.: Estimation of dynamic light changes in outdoor scenes without the use of calibration objects. In: Proc. International Conference on Pattern Recognition, vol. 4, pp. 91–94 (2006)
Barrow, H., Tenenbaum, J.: Recovering intrinsic scene characteristics from images. In: Computer Vision Systems. Academic Press, San Diego (1978)
Debevec, P.: Rendering synthetic objects into real scenes: bridging traditional and image-based graphics with global illumination and high dynamic range photography. In: Proc. SIGGRAPH’98, pp. 189–198 (1998)
Debevec, P., Malik, J.: Recovering high dynamic range radiance maps from photographs. In: Proc. SIGGRAPH’97, pp. 369–378 (1997)
Kajiya, J.: The rendering equation. In: Proc. SIGGRAPH’86, pp. 143–150 (1986)
Kimmel, R., Elad, M., Shaked, D., Keshet, R., Sobel, I.: A variational framework for retinex. Int. J. Comput. Vis. 52(1), 7–23 (2003)
Lalonde, J., Efros, A., Narasimhan, S.: Estimating the natural illumination conditions from a single outdoor image. Int. J. Comput. Vis. 98(2), 123–145 (2012)
Li, Y., Lin, S., Lu, H., Shum, H.: Multiple-cue illumination estimation in textured scenes. In: Proc. International Conference on Computer Vision, pp. 1366–1373 (2003)
Liu, Y., Granier, X.: Online tracking of outdoor lighting variations for augmented reality with moving cameras. IEEE Trans. Vis. Comput. Graph. 18(4), 573–580 (2012)
Liu, Y., Qin, X., Xing, G., Peng, Q.: A new approach to illumination estimation based on statistical analysis for augmented reality. Comput. Animat. Virtual Worlds 21, 321–330 (2010)
Liu, Y., Qin, X., Xu, S., Nakamae, E., Peng, Q.: Light source estimation of outdoor scenes for mixed reality. Vis. Comput. 25(5–7), 637–646 (2009)
Madsen, C., Stöerring, M., Jensen, T., Andersen, M., Christensen, M.: Real-time illumination estimation from image sequences. In: Proc. 13th Danish Conference on Pattern Recognition and Image Analysis, DSAGM 2005 (2005)
Matsushita, Y., Lin, S., Kang, S., Shum, H.: Estimating instrinsic images from image sequences with biased illumination. In: Proc. European Conference on Computer Vision, pp. 274–286 (2004)
Moreno-noguer, F., Nayar, S., Belhumeur, P.: Optimal illumination for image and video relighting. In: Proc. IEEE European Conference on Visual Media Production (CVMP), pp. 201–210 (2005)
Qin, X., Zhang, R., Lin, L., Zhong, F., Xing, G., Peng, Q.: Decomposition equation of basis images with consideration of global illumination. In: Computational Visual Media Conference. Proc. Lecture Notes in Computer Science (2012)
Sato, I., Hayashida, M., Kai, F., Sato, Y., Ikeuchi, K.: Fast image synthesis of virtual objects in a real scene with natural shadings. Syst. Comput. Jpn. 36(14), 1864–1872 (2005)
Sunkavalli, K., Matusik, W., Pfister, H., Rusinkiewicz, S.: Factored time-lapse video. In: Proc. SIGGRAPH’07, pp. 101–111 (2007)
Sunkavalli, K., Romeiro, F., Matusik, W., Zickler, T., Pfister, H.: What do color changes reveal about an outdoor scene? In: Proc. IEEE International Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2008)
Tappen, M., Freeman, W., Adelson, E.: Recovering intrinsic images from a single image. In: Advances in Neural Information Processing Systems, vol. 15. MIT Press, Cambridge (2002)
Wang, Y., Samaras, D.: Estimation of multiple directional light sources for synthesis of augmented reality images. Graph. Models 65(4), 185–205 (2003)
Xing, G., Liu, Y., Qin, X., Peng, Q.: A practical approach for real-time illumination estimation of outdoor videos. Comput. Graph. 36(7), 857–865 (2012)
Yu, Y., Debevec, P., Malik, J., Hawkins, T.: Inverse global illumination: recovering reflectance models of real scenes from photographs. In: Proc. SIGGRAPH’99, pp. 215–224 (1999)
Acknowledgements
The authors gratefully acknowledge the anonymous reviewers for their comments to help us to improve our paper. This work is supported by 973 program of China (No. 2009CB320802), NSF of China (No. U1035004, No. 61173070, No. 61202149, No. 61272431), Natural Science Fund for Distinguished Young Scholars of Shandong Province (No. JQ200920).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Zhang, R., Zhong, F., Lin, L. et al. Basis image decomposition of outdoor time-lapse videos. Vis Comput 29, 1197–1210 (2013). https://doi.org/10.1007/s00371-013-0776-6
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00371-013-0776-6