Skip to main content
Log in

Data Processing Algorithms for Generating Textured 3D Building Facade Meshes from Laser Scans and Camera Images

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

In this paper, we develop a set of data processing algorithms for generating textured facade meshes of cities from a series of vertical 2D surface scans and camera images, obtained by a laser scanner and digital camera while driving on public roads under normal traffic conditions. These processing steps are needed to cope with imperfections and non-idealities inherent in laser scanning systems such as occlusions and reflections from glass surfaces. The data is divided into easy-to-handle quasi-linear segments corresponding to approximately straight driving direction and sequential topological order of vertical laser scans; each segment is then transformed into a depth image. Dominant building structures are detected in the depth images, and points are classified into foreground and background layers. Large holes in the background layer, caused by occlusion from foreground layer objects, are filled in by planar or horizontal interpolation. The depth image is further processed by removing isolated points and filling remaining small holes. The foreground objects also leave holes in the texture of building facades, which are filled by horizontal and vertical interpolation in low frequency regions, or by a copy-paste method otherwise. We apply the above steps to a large set of data of downtown Berkeley with several million 3D points, in order to obtain texture-mapped 3D models.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Ballester, C., Bertalmio, M., Caselles, V., Sapiro, G., and Verdera, J. 2001. Filling in by joint interpolation of vector fields and gray levels. In IEEE Transactions on Image Processing, pp. 1200-1211.

  • Ballester, C., Caselles, V., Verdera, J., Bertalmio, M., and Sapiro, G. A variational model for filling-in gray level and color images. In Proc. 8th IEEE Int’l Conference on Computer Vision, vol. 1, pp. 10-16.

  • Bertalmio, M., Sapiro, G., Ballester, C., and Caselles, V. 2000. Image inpainting. In Proc. SIGGRAPH 2000, pp. 417-424.

  • Chan, T. and Shen, J. 2001. Mathematical models for local nontexture inpaintings. SIAM Journal on Applied Mathematics 62(3):1019-1043.

    Article  Google Scholar 

  • Chang, N.L. and Zakhor, A. 1999. A multivalued representation for view synthesis. In Proc. Int’l Conference on Image Processing, Kobe, Japan, vol. 2, pp. 505-509.

    Google Scholar 

  • Curless, B. and Levoy, M. A volumetric method for building com-plex models from range images. In SIGGRAPIL, New Orleans, pp. 303-312.

  • Debevec, P.E., Taylor, C.J., and Malik, J. 1996. Modeling and ren-dering architecture from photographs. In Proc. ACM SIGGRAPH.

  • Dick, A., Torr, P., Ruffle, S., and Cipolla, R. 2001. Combining single view recognition and multiple view stereo for architectural scenes. In International Conference on Computer, Vision, Vancouver, Canada, pp. 268-274.

    Google Scholar 

  • Elfros, A. and Freeman W. 2001. Image quilting for texture synthesis and transfer. In Proc. SIGGRAPI, pp. 341-346.

  • Fox, D., Thrun, S, Dellaert, E. and Burgard, W. 2000. Particle filters for mobile robot localization. In Sequential Monte Carlo Methods in Practice, A. Doucet, N. de Freitas, and N. Gordon (Eds.), Springer Verlag, New York.

    Google Scholar 

  • Frere, D., Vandekerckhove, J., Moons, T., and Van Gool, L. 1998. Automatic modelling and 3D reconstruction of urban build-ings from aerial imagery. In IEEE International Geoscience and Remote Sensing Symposium Proceedings, Seattle, pp. 2593-2596.

  • Frueh, C. 2002. Automated 3D Model Generation for Urban Environments. Ph.D. Thesis, University of Karlsruhe.

  • Frueh, C., Flynn, J., Foroosh, HI., and Zakhor, A. 2001. Fast 3D model generation in urban environments. In Workshop on the Conver-gence of Graphics, Vision, and Video (CGW’OI), Berkeley, USA.

  • Frueh, C. and Zakhor, A. 2001a. Fast 3D model generation in urban environments. In IEEE Conf on Multisensor Fusion and Integra-tionfor Intelligent Systems, Baden-Baden, Gemany, pp. 165-170.

  • Frueh, C. and Zakhor, A. 2001b. 3D model generation of cities using aerial photographs and ground level laser scans. In Computer Vi-sion and Pattern Recognition, Hawaii, USA, vol. 2.2. pp. II-31-8.

    Google Scholar 

  • Frueh, C. and Zakhor, A. 2003. Constructing 3D city models by merging ground-based and airborne views. IEEE Computer Graphics and Applications, Special Issue Nov/Dlec. pp. 52-41.

  • Garland, M. and EIeckbert, P. 1997. Surface Simplification Using Quadric Error Metrics. In SIGGRAPH ‘97, Los Angeles, pp. 209-216.

  • Hlaala, N. and Brenner, C. 1997. Generation of 3D city models from airborne laser scanning data. In Proc. EARSEL Workshop on LIDAR Remnote Sensing on Land and Sea, Tallin, Esonia, pp. 105-112.

  • Kim, Z., Huertas, A., and Nevatia, R. 2001. Automatic description of Buildings with complex rooftops from multiple images. In Computer Vision ard Pattern Recognition, Kauai, pp. 272-279.

  • Koch, R., Pollefeys, M., and van Gool, L. 1999. Realistic 3D Scene modeling from uncalibrated image sequences. In ICIP’99, Kobe, Japan, Oct., pp. II: 500-504.

  • Maas, H.-G. 2001. The suitability of airborne laser scanner data for automatic 3D object reconstruction. 3. In Int’l Workshop on Automatic Extraction of Man-Made Objects, Ascona, Switzerland.

  • Masnou, S. and Morel, J. 1998. Level-lines based disocclusion. In 5th IEEE Int’l Conference on Image Processing, pp. 259-263.

  • Nitzberg, M., Mumford, D., and Shiota, T. 1993. Filtering, Segmentation and Depth. Springer-Verlag: Berlin.

    Google Scholar 

  • Stamos, I. and Allen, PK. 2002. Geometry and texture recov-ery of scenes of large scale. In Computer Vision and Image Understanding (CVIU), 88(2):94-118.

    Article  Google Scholar 

  • Stulp, F., Dell’Acqua, F., and Fisher, R.B. 2001. Reconstruction of surfaces behind occlusions in range images. In Proc. 3rd Int. Conf. on 3-D Digital Imaging and Modeling, Montreal, Canada, pp. 232-239.

  • Thrun, S., Burgard, W., and Fox, D. 2000. A real-time algorithm for mobile robot mapping with applications to multi-robot and 3D mapping. In Proc. of International Conference on Robotics and Automation, San Francisco, vol. 1. pp. 321-328.

    Google Scholar 

  • Wang, X., Totaro, S., Taillandier, IR, Hanson, A., and Teller, S. 2002. Recovering facade texture and microstructure from real-world images. In Proc. 2nd International Workshop on Texture Analysis and Synthesis in Conjunction with European Conference on Computer Vision, pp. 145-149.

  • Zhao, H. and Shibasaki, R. 1999. A system for reconstructing urban 3D objects using ground-based range and CCD images. In Proc. of International Workshop on Urban Multi-Media/3D Mapping, Tokyo.

  • Zhao, H. and Shibasaki, R. 2001. Reconstructing urban 3D model using vehicle-borne laser range scanners. In Proc. of the 3rd International Conference on 3D Digital Imaging and Modeling, Quebec, Canada.

Download references

Author information

Authors and Affiliations

Authors

Rights and permissions

Reprints and permissions

About this article

Cite this article

Frueh, C., Jain, S. & Zakhor, A. Data Processing Algorithms for Generating Textured 3D Building Facade Meshes from Laser Scans and Camera Images. Int J Comput Vision 61, 159–184 (2005). https://doi.org/10.1023/B:VISI.0000043756.03810.dd

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/B:VISI.0000043756.03810.dd

Navigation