Indoor-Outdoor 3D Reconstruction Alignment

  • Andrea CohenEmail author
  • Johannes L. SchönbergerEmail author
  • Pablo Speciale
  • Torsten Sattler
  • Jan-Michael Frahm
  • Marc Pollefeys
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9907)


Structure-from-Motion can achieve accurate reconstructions of urban scenes. However, reconstructing the inside and the outside of a building into a single model is very challenging due to the lack of visual overlap and the change of lighting conditions between the two scenes. We propose a solution to align disconnected indoor and outdoor models of the same building into a single 3D model. Our approach leverages semantic information, specifically window detections, in multiple scenes to obtain candidate matches from which an alignment hypothesis can be computed. To determine the best alignment, we propose a novel cost function that takes both the number of window matches and the intersection of the aligned models into account. We evaluate our solution on multiple challenging datasets.


Window Detection Outdoor Scene Model Alignment Voxel Grid Common Reference Frame 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



This project was funded by the CTI Switzerland grant #17136.1 Geometric and Semantic Structuring of 3D point clouds, and the European Union’s Horizon 2020 research and innovation programme under grant agreement #637221.

Supplementary material

Supplementary material 1 (mp4 23692 KB)

419975_1_En_18_MOESM2_ESM.pdf (5.1 mb)
Supplementary material 2 (pdf 5172 KB)


  1. 1.
    Bentley, J.: Programming pearls: algorithm design techniques. ACM Commun. 27(9), 856–873 (1984)CrossRefGoogle Scholar
  2. 2.
    Cabral, R., Furukawa, Y.: Piecewise planar and compact floorplan reconstruction from images. In: CVPR (2014)Google Scholar
  3. 3.
    Ceylan, D., Mitra, N.J., Zheng, Y., Pauly, M.: Coupled Structure-from-motion and 3D symmetry detection for urban facades. ACM Trans. Graph. 33(1), 2:1–2:15 (2013)zbMATHGoogle Scholar
  4. 4.
    Cohen, A., Sattler, T., Pollefeys, M.: Merging the unmatchable: stitching visually disconnected SfM models. In: ICCV (2015)Google Scholar
  5. 5.
    Cohen, A., Schwing, A.G., Pollefeys, M.: Efficient structured parsing of facades using dynamic programming. In: CVPR (2014)Google Scholar
  6. 6.
    Cohen, A., Zach, C., Sinha, S., Pollefeys, M.: Discovering and exploiting 3D symmetries in structure from motion. In: CVPR (2012)Google Scholar
  7. 7.
    Cosmas, J., Itegaki, T., Green, D., Joseph, N., Gool, L.V., Zalesny, A., Vanrintel, D., Leberl, F., Grabner, M., Schindler, K., Karner, K., Gervautz, M., Hynst, S., Waelkens, M., Vergauwen, M., Pollefeys, M., Cornelis, K., Vereenooghe, T., Sablatnig, R., Kampel, M., Axell, P., Meyns, E.: Providing multimedia tools for recording, reconstruction, visualisation and database storage/access of archaeological excavations. In: VAST (2003)Google Scholar
  8. 8.
    Fischler, M., Bolles, R.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981)MathSciNetCrossRefGoogle Scholar
  9. 9.
    Furukawa, Y., Curless, B., Seitz, S.M., Szeliski, R.: Reconstructing building interiors from images. In: ICCV (2009)Google Scholar
  10. 10.
    Furukawa, Y., Ponce, J.: Accurate, dense, and robust multi-view stereopsis. PAMI 32(8), 1362–1376 (2010)CrossRefGoogle Scholar
  11. 11.
    Häne, C., Zach, C., Cohen, A., Angst, R., Pollefeys, M.: Joint 3D scene reconstruction and class segmentation. In: CVPR (2013)Google Scholar
  12. 12.
    Heinly, J., Schönberger, J.L., Dunn, E., Frahm, J.M.: Reconstructing the world* in six days *(As captured by the Yahoo 100 million image dataset). In: CVPR (2015)Google Scholar
  13. 13.
    Ikehata, S., Yan, H., Furukawa, Y.: Structured indoor modeling. In: ICCV (2015)Google Scholar
  14. 14.
    Koch, T., Korner, M., Fraundorfer, F.: Automatic alignment of indoor and outdoor building models using 3D line segments. In: CVPR Workshops (2016)Google Scholar
  15. 15.
    Korč, F., Förstner, W.: eTRIMS image database for interpreting images of man-made scenes. Technical report TR-IGG-P-2009-01, Dept. of Photogrammetry, University of Bonn.
  16. 16.
    Kushal, A., Self, B., Furukawa, Y., Gallup, D., Hernandez, C., Curless, B., Seitz, S.: Photo tours. In: 3DIMPVT (2012)Google Scholar
  17. 17.
    Ladický, L., Russell, C., Kohli, P., Torr, P.: Associative hierarchical random fields. PAMI 36(6), 1056–1077 (2014)CrossRefGoogle Scholar
  18. 18.
    Li, Y., Snavely, N., Huttenlocher, D., Fua, P.: Worldwide pose estimation using 3D Point clouds. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7572, pp. 15–29. Springer, Heidelberg (2012). doi: 10.1007/978-3-642-33718-5_2 Google Scholar
  19. 19.
    Liu, C., Schwing, A.G., Kundu, K., Urtasun, R., Fidler, S.: Rent3D: floor-plan priors for monocular layout estimation. In: CVPR (2015)Google Scholar
  20. 20.
    Lynen, S., Sattler, T., Bosse, M., Hesch, J., Pollefeys, M., Siegwart, R.: Get out of my lab: large-scale, real-time visual-inertial localization. In: RSS (2015)Google Scholar
  21. 21.
    Martin-Brualla, R., He, Y., Russell, B.C., Seitz, S.M.: The 3D jigsaw puzzle: mapping large indoor spaces. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8691, pp. 1–16. Springer, Heidelberg (2014). doi: 10.1007/978-3-319-10578-9_1 Google Scholar
  22. 22.
    Middelberg, S., Sattler, T., Untzelmann, O., Kobbelt, L.: Scalable 6-DOF localization on mobile devices. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8690, pp. 268–283. Springer, Heidelberg (2014). doi: 10.1007/978-3-319-10605-2_18 Google Scholar
  23. 23.
    Russell, B.C., Martin-Brualla, R., Butler, D.J., Seitz, S.M., Zettlemoyer, L.: 3D Wikipedia: using online text to automatically label and navigate reconstructed geometry. In: SIGGRAPH Asia (2013)Google Scholar
  24. 24.
    Savinov, N., Ladicky, L., Häne, C., Pollefeys, M.: Discrete optimization of ray potentials for semantic 3D reconstruction. In: CVPR (2015)Google Scholar
  25. 25.
    Schönberger, J.L., Radenovic, F., Chum, O., Frahm, J.M.: From single image query to detailed 3D reconstruction. In: CVPR (2015)Google Scholar
  26. 26.
    Schönberger, J.L., Frahm, J.M.: Structure-from-motion revisited. In: CVPR (2016)Google Scholar
  27. 27.
    Snavely, N., Garg, R., Seitz, S.M., Szeliski, R.: Finding paths through the world’s photos. In: SIGGRAPH (2008)Google Scholar
  28. 28.
    Strecha, C., Krull, M., Betschart, S.: The chillon project: aerial/terrestrial and indoor integration. Technical report, Pix4D.
  29. 29.
    Xiao, J., Furukawa, Y.: Reconstructing the world’s museums. In: ECCV (2012)Google Scholar
  30. 30.
    Zeisl, B., Sattler, T., Pollefeys, M.: Camera pose voting for large-scale image-based localization. In: ICCV (2015)Google Scholar

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  • Andrea Cohen
    • 1
    Email author
  • Johannes L. Schönberger
    • 1
    Email author
  • Pablo Speciale
    • 1
  • Torsten Sattler
    • 1
  • Jan-Michael Frahm
    • 2
  • Marc Pollefeys
    • 1
    • 3
  1. 1.ETH ZürichZürichSwitzerland
  2. 2.UNC Chapel HillChapel HillUSA
  3. 3.MicrosoftRedmondUSA

Personalised recommendations