Abstract
Systems for the creation of photorealistic models using range scans and digital photographs are becoming increasingly popular in a wide range of fields, from reverse engineering to cultural heritage preservation. These systems employ a range finder to acquire the geometry information and a digital camera to measure color detail. But bringing together a set of range scans and color images to produce an accurate and usable model is still an area of research with many unsolved problems. In this paper we address the problem of how to build illumination coherent integrated texture maps from images that were taken under different illumination conditions. To achieve this we present two different solutions. The first one is to align all the images to the same illumination, for which we have developed a technique that computes a relighting operator over the area of overlap of a pair of images that we then use to relight the entire image. Our proposed method can handle images with shadows and can effectively remove the shadows from the image, if required. The second technique uses the ratio of two images to factor out the diffuse reflectance of an image from its illumination. We do this without any light measuring device. By computing the actual reflectance we remove from the images any effects of the illumination, allowing us to create new renderings under novel illumination conditions.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Agathos, A., & Fisher, R. (2003). Colour texture fusion of multiple range images. In Proceedings of the 4th international conference on 3D digital imaging and modeling (pp. 139–146).
Allen, P. K., Troccoli, A., Smith, B., Murray, S., Stamos, I., & Leordeanu, M. (2003). New methods for digital modeling of historic sites. IEEE Computer Graphics and Applications Magazine, 23(6), 32–41.
Allen, P., Feiner, S., Troccoli, A., Benko, H., Ishak, E., & Smith, B. (2004). Seeing into the past: creating a 3D modeling pipeline for archaeological visualization. In Proceedings of 2nd international symposium on 3D data processing, visualization and transmission.
Bannai, N., Agathos, A., & Fisher, R. B. (2004). Fusing multiple color images for texturing models. In Proceedings of 2nd international symposium on 3D data processing, visualization and transmission (pp. 558–565). Los Alamitos: IEEE Computer Society.
Basri, R., & Jacobs, D. W. (2003). Lambertian reflectance and linear subspaces. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(2), 218–233.
Beauchesne, E., & Roy, S. (2003). Automatic relighting of overlapping textures of a 3D model. In Proceedings of computer vision and pattern recognition.
Buehler, C., Bosse, M., McMillan, L., Gortler, S., & Cohen, M. (2001). Unstructured lumigraph rendering. In Proceedings of the 28th annual conference on computer graphics and interactive techniques (pp. 425–432). New York: ACM.
Chuang, Y.-Y., Goldman, D. B., Curless, B., Salesin, D. H., & Szeliski, R. (2003). Shadow matting and compositing. ACM Transactions on Graphics, 22(3), 494–500.
Curless, B., & Levoy, M. (1996). A volumetric method for building complex models from range images. In Proceedings of the 23rd annual conference on computer graphics and interactive techniques (pp. 303–312). New York: ACM.
Debevec, P. E., Taylor, C. J., & Malik, J. (1996). Modeling and rendering architecture from photographs: a hybrid geometry- and image-based approach. In Proceedings of the 23rd annual conference on computer graphics and interactive techniques (pp. 11–20). New York: ACM.
Debevec, P., Hawkins, T., Tchou, C., Duiker, H.-P., Sarokin, W., & Sagar, M. (2000). Acquiring the reflectance field of a human face. In Proceedings of the 27th annual conference on computer graphics and interactive techniques (pp. 145–156). New York: ACM/Addison-Wesley.
Debevec, P., Tchou, C., Gardner, A., Hawkins, T., Stumpfel, J., Jones, A., et al. (2004). Estimating surface reflectance of a complex scene under natural captured illumination (Technical report). University of Souther Californian, Institute for Creative Technologies.
Frolova, D., Simakov, D., & Basri, R. (2004). Accuracy of spherical harmonic approximations for images of Lambertian objects under far and near lighting. In Proceedings of the European conference on computer vision (Vol. 1, pp. 574–587).
Funka-Lea, G., & Bajcsy, R. (1995). Combining color and geometry for the active, visual recognition of shadows. In ICCV ’95: proceedings of the fifth international conference on computer vision (p. 203). Washington: IEEE Computer Society.
Hertzmann, A., & Seitz, S. M. (2005). Example-based photometric stereo: Shape reconstruction with general, varying BRDFs. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(8), 1254–1264.
Ikeuchi, K., & Sato, K. (1991). Determining reflectance properties of an object using range and brightness images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 13(11), 1139–1153.
Lensch, H. P. A., Kautz, J., Goesele, M., Heidrich, W., & Seidel, H.-P. (2003). Image-based reconstruction of spatial appearance and geometric detail. ACM Transactions on Graphics, 22(2), 234–257.
Marschner, S. R., & Greenberg, D. P. (1997). Inverse lighting for photography. In Proceedings of the 5th color imaging conference.
Narasimhan, S. G., & Nayar, S. K. (2001). Removing weather effects from monochrome images. In IEEE conference on computer vision and pattern recognition (CVPR) (Vol. 2, p. 186).
Pulli, K., Cohen, M., Duchamp, T., Hoppe, H., Shapiro, L., & Stuetzle, W. (1997). View-based rendering: visualizing real objects from scanned range and color data. In Rendering techniques ’97 (pp. 23–34). New York: Springer Wien.
Ramamoorthi, R. (2002). Analytic PCA construction for theoretical analysis of lighting variability in images of a Lambertian object. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(10), 1322–1333.
Ramamoorthi, R., & Hanrahan, P. (2001a). On the relationship between radiance and irradiance: determining the illumination from images of a convex Lambertian object. Journal of the Optical Society of America A, 18(10), 2448–2459.
Ramamoorthi, R., & Hanrahan, P. (2001b). A signal-processing framework for inverse rendering. In SIGGRAPH ’01: proceedings of the 28th annual conference on computer graphics and interactive techniques (pp. 117–128). New York: ACM.
Reda, I., & Andreas, A. (2003). Solar position algorithm for solar radiation applications (Technical report). National Renewable Energy Laboratory, Golden, CO.
Salvador, E., Cavallaro, A., & Ebrahimi, T. (2004). Cast shadow segmentation using invariant color features. Computer Vision and Image Understanding, 95(2), 238–259.
Shashua, A., & Riklin-Raviv, T. (2001). The quotient image: class-based re-rendering and recognition with varying illuminations.. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(2), 129–139.
Tomasi, C., & Manduchi, R. (1998). Bilateral filtering for gray and color images. In ICCV ’98: proceedings of the sixth international conference on computer vision (p. 839). Washington: IEEE Computer Society.
Troccoli, A. (2006a). Animation of relighted image sequence of the church of St. Marie at Chappes, France. http://www.cs.columbia.edu/atroccol/ijcv07.
Troccoli, A. (2006b). New methods and tools for 3D-modeling of large scale outdoor scenes using range and color images. PhD thesis, Department of Computer Science, Columbia University.
Troccoli, A., & Allen, P. K. (2004). A shadow based method for image to model registration. In 2nd IEEE workshop on image and video registration (IVR 04).
Troccoli, A., & Allen, P. (2005). Relighting acquired models of outdoor scenes. In Proceedings of 3DIM’05.
Troccoli, A., & Allen, P. (2006). Illumination and textures factorization using ratio images of an object of known geometry. In Proceedings of the 3rd international symposium 3D data processing, visualization, and transmission (3DPVT 06). Los Alamitos: IEEE Computer Society.
Wang, L., Kang, S. B., Szeliski, R., & Shum, H.-Y. (2001). Optimal texture map reconstruction from multiple views. In Proceedings of the 2001 IEEE computer society conference on computer vision and pattern recognition (pp. 347–354).
Wang, H., Li, S. Z., & Wang, Y. (2004). Generalized quotient image. In IEEE conference on computer vision and pattern recognition (CVPR) (Vol. 2, pp. 498–505).
Xu, C., Georghiades, A., Rushmeier, H., & Dorsey, J. (2006). A system for reconstructing integrated texture maps for large structures. In Proceedings of 3rd international symposium on 3D data processing, visualization and transmission.
Yu, Y., & Malik, J. (1998). Recovering photometric properties of architectural scenes from photographs. In Proceedings of the 25th annual conference on computer graphics and interactive techniques (pp. 207–217). New York: ACM.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Troccoli, A., Allen, P. Building Illumination Coherent 3D Models of Large-Scale Outdoor Scenes. Int J Comput Vis 78, 261–280 (2008). https://doi.org/10.1007/s11263-007-0100-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11263-007-0100-x