Skip to main content
SpringerLink
Log in

Real-time visualization of 3D city models at street-level based on visual saliency

  • Research Paper
  • Published:
Science China Earth Sciences Aims and scope Submit manuscript

Abstract

Street-level visualization is an important application of 3D city models. Challenges to street-level visualization include the cluttering of buildings due to fine detail and visualization performance. In this paper, a novel method is proposed for street-level visualization based on visual saliency evaluation. The basic idea of the method is to preserve these salient buildings in a scene while removing those that are non-salient. The method can be divided into pre-processing procedures and real-time visualization. The first step in pre-processing is to convert 3D building models at higher Levels of Detail (LoDs) into LoD1 models with simplified ground plans. Then, a number of index viewpoints are created along the streets; these indices refer to both the position and the direction of each street site. A visual saliency value is computed for each building, with respect to the index site, based on a visual difference between the original model and the generalized model. We calculate and evaluate three methods for visual saliency: local difference, global difference and minimum projection area. The real-time visualization process begins by mapping the observer to its closest indices. The street view is then generated based on the building information stored in those indexes. A user study shows that the local visual saliency method performs better than do the global visual saliency, area and image-based methods and that the framework proposed in this paper may improve the performance of 3D visualization.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Anders K. 2005. Level of detail generation of 3D building groups by aggregation and typification. Proceedings of XXII International Cartographic Conference 2005, Spain

    Google Scholar 

  • Anter K. 2000. What Colour is the Red House? Perceived colour of painted facades. Doctoral Dissertation. Stockholm: Royal Institute of Technology

    Google Scholar 

  • Behr J, Eschler P, Jung Y, Zöllner M. 2009. X3DOM: A DOM-based HTML5/X3D integration model. Web3D’ 09, Darmstadt, Germany

    Google Scholar 

  • Bruce N, Tsotsos J. 2005. Saliency Based on information maximization. Proc Neural Inf Process Syst (NIPS), 2005

    Google Scholar 

  • International Commission on Illumination (CIE). 2007. CIE 1976 L*a*b* colour space draft standard. http://cie.co.at/index.php?i_ca_id=485. accessed Feb 13, 2014

    Google Scholar 

  • Cole G G, Kentridge R W, Heywood C A. 2004. Visual saliency in the change detection paradigm: The special role of object onset. J Exp Psychol-Human Percept Performance, 30: 464–477

    Article  Google Scholar 

  • Elias B. 2003. Extracting landmarks with data mining methods. In: Kuhn W, Worboys M, Timpf S, eds. Spatial Information Theory: Foundations of Geographic Information Science. Lecture Notes Comput Sci, 2825: 375–389

    Chapter  Google Scholar 

  • Elias B, Hampe M, Monika S. 2005. Adaptive visualisation of landmarks using an MRDB. In: Meng L Q, Zipf A, Reichenbacher T, eds. Map-based Mobile Services-Theories, Methods and Implementations. Heidelberg: Springer. 73–86

    Chapter  Google Scholar 

  • Fan H, Meng L, Jahnke M. 2009. Generalization of 3D buildings modeled by CityGML. Lecture notes in geoinformation and cartography. Adv GISci, 387–405

    Chapter  Google Scholar 

  • Forberg A. 2007. Generalization of 3D building data based on a scale-space approach. ISPRS-J Photogramm Remote Sens, 62: 104–111

    Article  Google Scholar 

  • Guercke R, Götzelmann T, Brenner C, et al. 2011. Aggregation of LoD 1 building models as an optimization problem. ISPRS-J Photogramm Remote Sens, 66: 209–222

    Article  Google Scholar 

  • Hara K, Le V, Froehlich J. 2013. Combining crowdsourcing and google street view to identify street-level accessibility problems. In: Proc of the SIGCHI Conference on Human Factors in Computing Syst (CHI’ 13). New York: ACM. 631–640

    Chapter  Google Scholar 

  • Harel J. 2013. A Saliency Implementation in MATLAB: http://www.klab.caltech.edu/~harel/share/gbvs.php. accessed Nov 12 2013

    Google Scholar 

  • Harel J, Koch C, Perona P. 2006. Graph-based visual saliency. In: Schölkopf B, Platt J C, Hoffman T, eds. Advances, Proceeding of Neural Information Process Systems 19 (NIPS 2006)

    Google Scholar 

  • Hitchcock F L. 1941. The distribution of a product from several sources to numerous localities. J Math Phys, 20: 224–230

    Google Scholar 

  • Hoffmann G. 2003. CIELAB Color Space, online at: http://www.fho-emden.de/~hoffmann/cielab03022003.pdf. accessed Nov 12, 2013

    Google Scholar 

  • Itti L. 2004. Automatic foveation for video compression using a neurobiological model of visual attention. IEEE Trans Image Process, 13: 1304–1318

    Article  Google Scholar 

  • Itti L, Koch C. 2000. A saliency-based search mechanism for overt and covert shifts of visual attention. Vision Res, 40: 1489–1506

    Article  Google Scholar 

  • Itti L, Koch C, Niebur E. 1998. A model of saliency-based visual attention for rapid scene analysis. IEEE Trans Pattern Anal Mach Intell, 20: 1254–1259

    Article  Google Scholar 

  • JTS. 2013. http://www.vividsolutions.com/jts/JTSHome.htm. accessed Nov 12, 2013

  • Kada M. 2002. Automatic generalization of 3D building models. Int Arch Photogramm Remote Sens Spatial Inf Sci, 34: 243–248

    Google Scholar 

  • Kim D H, Yun I D, Lee S U. 2004. A new attributed relational graph matching algorithm using the nested structure of earth mover’s distance. In: Proc of 17th Int Conf Pattern Recognition (ICPR’04)

    Google Scholar 

  • Kim Y, Varshney A. 2006. Saliency-guided enhancement for volume visualization. IEEE Trans Vis Comput Graph 12: 925–932

    Article  Google Scholar 

  • Kolbe T H. 2008. Representing and Exchanging 3D City Models with CityGML in 3D Geo-Information Sciences. Berlin: Springer

    Google Scholar 

  • Kopf J, Chen B, Szeliski R, et al. 2010. Street Slide: Browsing Street Level Imagery. ACM Trans Graph, 29:96: 1–8

    Google Scholar 

  • Lee C H, Kim Y, Varshney A. 2009. Saliency-guided lighting. IEICE Trans Inf Systems, E92.D: 369–373

    Article  Google Scholar 

  • Lee T. 2009. Robust 3D street-view reconstruction using sky motion estimation. In: Proceedings of IEEE Int Workshop on 3-D Digital Imaging and Modeling (3DIM2009) in Conjunction with ICCV

    Google Scholar 

  • Mao B, Fan H, Harrie L, et al. 2010. City model generalization similarity measurement using nested earth mover’s distance. In: Proceedings of 13th Workshop of the ICA commission on Generalisation and Multiple Representation, Zurich, Switzerland, 12–13 September

    Google Scholar 

  • Mao B, Ban Y. 2011. Visualization of 3D city model through the internet using CityGML and X3DOM. Cartographica, 46: 109–114

    Article  Google Scholar 

  • Mao B, Harrie L, Ban Y. 2011. Detection and typification of linear structures for dynamic visualisation of 3D city models. Computs Environ Urban Syst, 36: 233–244

    Article  Google Scholar 

  • Mao B, Ban Y. 2013. Generalization of 3D building texture using image compression and multiple representation data structure. ISPRS-J Photogramm Remote Sens, 79: 68–79

    Article  Google Scholar 

  • Menzel N, Guthe M. 2010. Towards perceptual simplification of models with arbitrary materials. Comput Graphics Forum, 29: 2261–2270

    Article  Google Scholar 

  • Miao Y, Feng J. 2010. Perceptual-saliency extremum lines for 3D shape illustration. Visual Comput, 26: 433–443

    Article  Google Scholar 

  • Micusik B, Kosecka J. 2009. Piecewise planar city 3D modeling from street view panoramic sequences. IEEE Conf Comput Vision Pattern Recognition (CVPR), Miami, USA

    Google Scholar 

  • Mortara M, Spagnuolo M. 2009. Semantics-driven best view of 3D shapes. Comput Graph, 33: 280–290

    Article  Google Scholar 

  • Nurminen A. 2008. Mobile 3D city maps. Comput Graphics, 28: 20–31

    Article  Google Scholar 

  • Parry M, Ribarsky W, Shaw C, et al. 2002. Organization and simplification of high-resolution 3D city facades. In: Proceedings of SPIE Aerosense Symposium, Visualization of Temporal and Spatial Data for Civilian and Defense Applications IV, 4744B

    Google Scholar 

  • Rakkolainen I, Vainio T. 2001. A 3D city information for mobile users. Comput Graphics, 25: 619–625

    Article  Google Scholar 

  • Raubal M, Winter S. 2002. Enriching way finding instructions with local landmarks. Lecture Notes Comput Sci, 2478: 243–259

    Article  Google Scholar 

  • Salmen J, Houben S, Schlipsing M. 2012. Google street view images support the development of vision-based driver assistance systems. Intell Vehicles Symposium. 891–895

    Google Scholar 

  • Schulte-Coerne T. 2005. Visualisierung von distanzfunktionen insbesondere der earth movers distance. GI Inf 2005, Schloss Birlinghoven: 8. bis 9. April 2005

    Google Scholar 

  • Sester M, Brenner C. 2000. Typification based on Kohonen feature nets. The 1st International Conference on Geographic Information Science (GIScience 2000) Savannah, Georgia, USA

    Google Scholar 

  • Sester M, Brenner C. 2004. Continuous generalization for fast and smooth visualization on small displays. Int Arch Photogramm Remote Sens, 34: 1293–1298

    Google Scholar 

  • Frintrop S, Jensfelt P, Christensen H. 2006. Attentional landmark selection for visual SLAM. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS’06), October 2006

    Google Scholar 

  • Sun X, Qing W H, Hua Y W. 2009. Design and implementation of global 3D visualization client program on intelligent mobile devices. Third Int Conf Multimedia Ubiquitous Eng, 459–464

    Google Scholar 

  • Thiemann F. 2002. Generalization of 3D building data. The Int Arch Photogramm Remote Sens Spatial Inf Sci, 34: 286–290

    Google Scholar 

  • Thompson K G, Bichot N P. 2005. A visual saliency map in the primate frontal eye field. Prog Brain Res, 147: 251–262

    Google Scholar 

  • Vincent L. 2007. Taking online maps down to street level. Compute, 40: 118–120

    Article  Google Scholar 

  • Wang M, Li J, Huang T, et al. 2010. Saliency detection based on 2D log-gabor wavelets and center bias. ACM Multimedia, 2010: 979–982

    Google Scholar 

  • Winter S. 2003. Route adaptive selection of salient features. COSIT, 2003: 349–361

    Google Scholar 

  • Xiao J, Fang T, Zhao P, et al. 2009. Image-based street-side city modeling. ACM Trans Graphics, 28: 114

    Article  Google Scholar 

  • Xj3D. 2013. http://www.xj3d.org/. accessed Nov 12, 2013

  • Yantis S. 2005. How visual salience wins the battle for awareness. Nat Neurosci 8: 975–977

    Article  Google Scholar 

  • Zhu Q, Li D, Zhang Y. 2002. Integration of DEMs, images and 3D models. Photogramm Eng Remote Sens, 68: 361–367

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Jiangsu Provincial Key Laboratory of E-Business, Nanjing University of Finance and Economics, Nanjing, 210003, China

    Bo Mao

  2. College of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, 210094, China

    Bo Mao

  3. Department of Urban Planning and Environment, KTH Royal Institute of Technology, SE-10044, Stockholm, Sweden

    YiFang Ban

  4. Department of Physical Geography and Ecosystem Science, Lund University, Box118, SE-223 62, Lund, Sweden

    Lars Harrie

Authors
  1. Bo Mao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. YiFang Ban
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lars Harrie
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bo Mao.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mao, B., Ban, Y. & Harrie, L. Real-time visualization of 3D city models at street-level based on visual saliency. Sci. China Earth Sci. 58, 448–461 (2015). https://doi.org/10.1007/s11430-014-4955-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11430-014-4955-8

Keywords

Search

Navigation