Texturizing and Refinement of 3D City Models with Mobile Devices

  • Ralf GutbellEmail author
  • Hannes Kuehnel
  • Arjan Kuijper
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10617)


Building recognition from images and video streams of mobile devices to texturize and refine an existing 3D city model is an open challenge, since such models most often do not completely represent the actual buildings. We present ways to extract buildings from images enabling improvement of the existing model. The approach is based on edge detection on images to detect walls, pure use of sensor data by creating an overlay to the video stream with the 3D model renderer from current position by a server, and the use of structure from motion algorithms to create point clouds and recognize a building via the support of the device’s sensors. We show that we are thus able to texturize and refine an existing 3D city model.


  1. 1.
    Ackermann, J., Goesele, M.: A survey of photometric stereo techniques. Found. Trends Comput. Graph. Vis. 9(3–4), 149–254 (2015)CrossRefzbMATHGoogle Scholar
  2. 2.
    Akbarzadeh, A., Frahm, J., Mordohai, P., Clipp, B., Engels, C., Gallup, D., Merrell, P., Phelps, M., Sinha, S.N., Talton, B., Wang, L., Yang, Q., Stewénius, H., Yang, R., Welch, G., Towles, H., Nistér, D., Pollefeys, M.: Towards urban 3D reconstruction from video. In: 3rd International Symposium on 3D Data Processing, Visualization and Transmission (3DPVT 2006), 14–16 June 2006, Chapel Hill, North Carolina, USA, pp. 1–8 (2006)Google Scholar
  3. 3.
    Arth, C., Pirchheim, C., Ventura, J., Schmalstieg, D., Lepetit, V.: Instant outdoor localization and SLAM initialization from 2.5D maps. IEEE Trans. Vis. Comput. Graph. 21(11), 1309–1318 (2015)CrossRefGoogle Scholar
  4. 4.
    Canny, J.: A computational approach to edge detection. IEEE Trans. Pattern Anal. Mach. Intell. 8(6), 679–698 (1986)CrossRefGoogle Scholar
  5. 5.
    von Gioi, R.G., Jakubowicz, J., Morel, J., Randall, G.: LSD: a fast line segment detector with a false detection control. IEEE Trans. Pattern Anal. Mach. Intell. 32(4), 722–732 (2010)CrossRefGoogle Scholar
  6. 6.
    Harris, C., Stephens, M.: A combined corner and edge detector. In: Proceedings of the Alvey Vision Conference, AVC 1988, Manchester, UK, pp. 1–6, September 1988Google Scholar
  7. 7.
    Kahn, S., Bockholt, U., Kuijper, A., Fellner, D.W.: Towards precise real-time 3D difference detection for industrial applications. Comput. Ind. 64(9), 1115–1128 (2013)CrossRefGoogle Scholar
  8. 8.
    Lucas, B.D., Kanade, T.: An iterative image registration technique with an application to stereo vision. In: Proceedings of the 7th International Joint Conference on Artificial Intelligence, IJCAI 1981, Vancouver, BC, Canada, pp. 674–679, 24–28 August 1981Google Scholar
  9. 9.
    Martinovic, A., Knopp, J., Riemenschneider, H., Gool, L.J.V.: 3D all the way: semantic segmentation of urban scenes from start to end in 3D. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2015, pp. 4456–4465 (2015)Google Scholar
  10. 10.
    Nistér, D., Naroditsky, O., Bergen, J.R.: Visual odometry for ground vehicle applications. J. Field Robot. 23(1), 3–20 (2006)CrossRefzbMATHGoogle Scholar
  11. 11.
    Pollefeys, M., Gool, L.J.V., Vergauwen, M., Verbiest, F., Cornelis, K., Tops, J., Koch, R.: Visual modeling with a hand-held camera. Int. J. Comput. Vis. 59(3), 207–232 (2004)CrossRefGoogle Scholar
  12. 12.
    Pollefeys, M., Nistér, D., Frahm, J., Akbarzadeh, A., Mordohai, P., Clipp, B., Engels, C., Gallup, D., Kim, S.J., Merrell, P., Salmi, C., Sinha, S.N., Talton, B., Wang, L., Yang, Q., Stewénius, H., Yang, R., Welch, G., Towles, H.: Detailed real-time urban 3D reconstruction from video. Int. J. Comput. Vis. 78(2–3), 143–167 (2008)CrossRefGoogle Scholar
  13. 13.
    Snavely, N., Simon, I., Goesele, M., Szeliski, R., Seitz, S.M.: Scene reconstruction and visualization from community photo collections. Proc. IEEE 98(8), 1370–1390 (2010)CrossRefGoogle Scholar
  14. 14.
    Szeliski, R.: Computer Vision - Algorithms and Applications. Springer, Heidelberg (2011). Texts in Computer SciencezbMATHGoogle Scholar
  15. 15.
    Tomasi, C., Kanade, T.: Shape and motion from image streams : a factorization method. part 3, detection and tracking of point features. Technical Report CMU-CS-91-132, Carnegie-Mellon University, Computer science, Pittsburgh (PA US) (1991)Google Scholar
  16. 16.
    Wientapper, F., Wuest, H., Kuijper, A.: Composing the feature map retrieval process for robust and ready-to-use monocular tracking. Compute. Graph. 35(4), 778–788 (2011)CrossRefGoogle Scholar
  17. 17.
    Wientapper, F., Wuest, H., Kuijper, A.: Reconstruction and accurate alignment of feature maps for augmented reality. In: 3DIMPVT 2011, pp. 140–147. IEEE (2011)Google Scholar
  18. 18.
    Xiao, J., Fang, T., Tan, P., Zhao, P., Ofek, E., Quan, L.: Image-based faÇade modeling. ACM Trans. Graph. 27(5), 161:1–161:10 (2008)CrossRefGoogle Scholar
  19. 19.
    Xiao, J., Fang, T., Zhao, P., Lhuillier, M., Quan, L.: Image-based street-side city modeling. ACM Trans. Graph. 28(5), 114:1–114:12 (2009)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.Fraunhofer IGDDarmstadtGermany
  2. 2.TU DarmstadtDarmstadtGermany

Personalised recommendations