Multimedia Tools and Applications

, Volume 77, Issue 20, pp 27387–27404 | Cite as

Interactive 3D building modeling method using panoramic image sequences and digital map

  • Hyungki KimEmail author
  • Soonhung Han


This paper proposes a method of generating 3D building models with precise geospatial information and a photograph-based façade appearance from panoramic image sequences and digital maps. 3D building modeling research is actively being conducted in areas such as geographic information systems, virtual reality, and augmented reality. However, the generation of realistic 3D models from a ground-level viewpoint is still extremely costly in terms of labor of modeling experts, and collection of data. We have developed a method for 3D building modeling with high-resolution photograph-based appearance information using panoramic images captured at ground level with a mobile mapping system, and geospatial information obtained from a digital map. The proposed method includes 1) pre-processing for tilt correction and base 3D model generation, 2) geo-registration of panoramic images with minimal user input, and 3) building height and shape estimation. This paper presents the proposed method and the quantitative performance measure obtained from a developed test modeling system. In addition, modeling results from an experimental dataset are also presented.


3D building modeling Street view Geo-registration Shape estimation 


  1. 1.
    Alexakis E, Tsironis V, Petsa E, Karras G (2016) Automatic adjustment of wide-base google street view panoramas. Remote Sens Spat Inf Sci XLI-B1:639–645Google Scholar
  2. 2.
    Baillard C, Zisserman A (1999) Automatic reconstruction of piecewise planar models from multiple views. In: Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149), Fort Collins 2, 559–565Google Scholar
  3. 3.
    Barinova O, Konushin V, Yakubenko A, Lee K, Lim H, Konushin A (2008) Fast automatic single-view 3-d reconstruction of urban scenes. In: ECCV 2008, 12-18 October, Marseille, France: Springer, 100–113Google Scholar
  4. 4.
    Comaniciu D, Meer P (2002) Mean Shift: A Robust Approach Toward Feature Space Analysis. IEEE Trans Pattern Anal Mach Intell 24(5):603–619CrossRefGoogle Scholar
  5. 5.
    Debevec PE, Taylor CJ, Malik J (1996) Modeling and rendering architecture from photographs: a hybrid geometry- and image-based approach. In: Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques, 11–20Google Scholar
  6. 6.
    Gioi RG, Jakubowicz J, Morel J, Randall G (2012) LSD: a line segment detector. Image Process On Line 2:35–55CrossRefGoogle Scholar
  7. 7.
    Google earth (2014) Available from: Accessed 2 June 2016
  8. 8.
    Guo H, Li X, Wang W, Lv Z, Wu C, Xu W (2016) An event-driven dynamic updating method for 3D geo-databases. Geo-spatial Inf Sci 19(2):140–147CrossRefGoogle Scholar
  9. 9.
    Hartley R, Zisserman A (2004) Multiple View Geometry in Computer Vision, 2nd edn. Cambridge University Press, CambridgeCrossRefGoogle Scholar
  10. 10.
    Hoiem D, Efros AA, Hebert M (2005) Automatic photo pop-up. ACM Trans Graph 24(3):577–584CrossRefGoogle Scholar
  11. 11.
    Kada M, Klinec D, Haala N (2005) Facade texturing for rendering 3D city models. In: ASPRS Conference 2005, 78–85Google Scholar
  12. 12.
    Kim H, Han S (2016) Geo-registration of wide-baseline panoramic image sequences using a digital map reference. Multimed Tools Appl 1–19Google Scholar
  13. 13.
    Kim H, Kang Y, Han S (2014) Automatic 3D City Modeling Using a Digital Map and Panoramic Images from a Mobile Mapping System. Math Probl Eng 1–10Google Scholar
  14. 14.
    Kolbe T, Gröger G, Plümer L (2005) CityGML – interoperable access to 3D city models. In: Proc. of the 1st International Symposium on Geo-information for Disaster Management, 21-23 March Netherlands: Springer, 883–899CrossRefGoogle Scholar
  15. 15.
    Li X, Wu C, Zach C, Lazebnik S, Frahm J (2008) Modeling and recognition of landmark image collections using iconic scene graphs. In: ECCV 2008, 12-18 October, Marseille, France: Springer, 427–440Google Scholar
  16. 16.
    Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110CrossRefGoogle Scholar
  17. 17.
    Lu Z, Guerrero P, Mitra NJ, Steed A (2016) Open3D: crowd-sourced distributed curation of city models. In Proceedings of the 21st International Conference on Web3D Technology, 87–94Google Scholar
  18. 18.
    Lv Z, Li X, Zhang B, Wang W, Zhu Y, Hu J, Feng S (2016) Managing big city information based on WebVRGIS. IEEE Access 4:407–415CrossRefGoogle Scholar
  19. 19.
    Moslah O, Klee M, Grolleau A, Guitteny V, Couvet S, Philipp-Foliguet S (2008) Urban models texturing from un-calibrated photographs. In: IVCNZ, 26-28 November, Christchurch, New Zealand: IEEE, 1–6Google Scholar
  20. 20.
    Oh SH, Jung SK (2012) RANSAC-based Orthogonal Vanishing Point Estimation in the Equirectangular Images. J Korea MultimedSoc 15(12):1230–1441Google Scholar
  21. 21.
    Park J (2005) Interactive 3D reconstruction from multiple images: A primitive-based approach. Pattern Recogn Lett 26(16):2558–2571CrossRefGoogle Scholar
  22. 22.
    Rau J, Teo T, Chen L, Tsai F, Hsiao K, Hsu W (2006) Integration of GPS, GIS and photogrammetry for texture mapping in photo-realistic city modeling. In PSIVT, 10-13 December, Hsinchu, Taiwan: Springer, 1283–1292CrossRefGoogle Scholar
  23. 23.
    Sato T, Pajdla T, Yokoya N (2011) Epipolar geometry estimation for wide-baseline omnidirectional street view images. In: ICCV 2011, 6-13 November, Varcelona, Spain, 56–63Google Scholar
  24. 24.
    Sinha SN, Steedly D, Szeliski R, Agrawala M, Pollefeys M (2008) Interactive 3D architectural modeling from unordered photo collections. ACM Trans Graph 27(5):159 1–159:10CrossRefGoogle Scholar
  25. 25.
    Tsai F, Lin H-C (2007) Polygon-based texture mapping for cyber city 3D building models. Int J Geogr Inf Sci 21(9):965–981CrossRefGoogle Scholar
  26. 26.
    Ventura J, Hollerer T (2013) Structure and motion in urban environments using upright panoramas. Virtual Reality 17(2):147–156CrossRefGoogle Scholar
  27. 27.
    Vworld (2014) Available from: Accessed 8 Aug 2017
  28. 28.
    Wang G, Tsui H, Hu Z (2005) Reconstruction of structured scenes from two uncalibrated images. Pattern Recogn Lett 26(2):207–220CrossRefGoogle Scholar
  29. 29.
    Xiao J, Fang T, Zhao P, Lhuillier M, Quan L (2009) Image-based street-side city modelling. ACM Trans Graph 28(5):114 1–114:12CrossRefGoogle Scholar
  30. 30.
    Yoo B (2013) Rapid three-dimensional urban model production using bilayered displacement mapping. Int J Geogr Inf Sci 27(1):24–46CrossRefGoogle Scholar
  31. 31.
    Zebedin L, Klaus A, Gruber-Geymayer B, Karner K (2006) Towards 3D map generation from digital aerial images. ISPRS J Photogramm Remote Sens 60(6):413–427CrossRefGoogle Scholar
  32. 32.
    Zebedin L, Bauer J, Karner K, Bischof H (2008) Fusion of feature- and area-based information for urban buildings modeling from aerial imagery, In: ECCV 2008, 12-18 October, Marseille, France: Springer, 873–886Google Scholar
  33. 33.
    Zhang C, Wang L, Yang R (2010) Semantic Segmentation of Urban Scenes Using Dense Depth Maps. EVVC 2010:708–721Google Scholar
  34. 34.
    Zheng X, Zhang X, Guo P (2011) Building modeling from a single image applied in urban reconstruction. In: Proceedings of the 10th International Conference on Virtual Reality Continuum and Its Applications in Industry, Hongkong, Chana: ACM, 225–234Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.3rd Aero Systems Division, ADDDaejeonSouth Korea
  2. 2.Graduate Program of Ocean Systems, Department of Mechanical EngineeringKAISTDaejeonSouth Korea

Personalised recommendations