Dense 3D Reconstruction from Wide Baseline Image Sets

  • Helmut Mayer
  • Jan Bartelsen
  • Heiko Hirschmüller
  • Andreas Kuhn
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7474)


This paper describes an approach for Structure from Motion (SfM) for wide baselines image sets and its combination with the dense Semiglobal Matching (SGM) 3D reconstruction approach. Our approach for SfM relies on given information concerning image overlap, but can deal with large baselines and produces highly precise camera parameters and 3D points. At the core of our contribution is robust least squares adjustment with full exploitation of the covariance information from affine point matching to bundle adjustment. Reweighting for robust adjustment is based on covariance information for each individual residual. We use points detected based on Differences of Gaussians including scale and orientation information as well as a variant of the five point algorithm. A strategy similar to the Expectation Maximization (EM) algorithm is employed to extend partial solutions. The key characteristics of the approach is reliability obtained by aiming at a high precision in every step. The capabilities of our approach are demonstrated by presenting results for sets consisting of images from the ground and from small Unmanned Aircraft Systems (UASs).


Scale Invariant Feature Transform Intrinsic Parameter Camera Parameter Bundle Adjustment Epipolar Line 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Agarwal, S., Snavely, N., Simon, I., Seitz, S., Szeliski, R.: Building Rome in a Day. In: Twelfth International Conference on Computer Vision, pp. 72–79 (2009)Google Scholar
  2. 2.
    Bartelsen, J., Mayer, H.: Orientation of Image Sequences Acquired from UAVs and with GPS Cameras. Surveying and Land Information Science 70(3), 151–159 (2010)Google Scholar
  3. 3.
    Chum, O., Matas, J., Kittler, J.: Locally Optimized RANSAC. In: Michaelis, B., Krell, G. (eds.) DAGM 2003. LNCS, vol. 2781, pp. 236–243. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  4. 4.
    Fischler, M., Bolles, R.: Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography. Communications of the ACM 24(6), 381–395 (1981)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Frahm, J.-M., Fite-Georgel, P., Gallup, D., Johnson, T., Raguram, R., Wu, C., Jen, Y.-H., Dunn, E., Clipp, B., Lazebnik, S., Pollefeys, M.: Building Rome on a Cloudless Day. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part IV. LNCS, vol. 6314, pp. 368–381. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  6. 6.
    Goesele, M., Ackermann, J., Fuhrmann, S., Klowsky, R., Langguth, F., Muecke, P., Ritz, M.: Scene Reconstruction from Community Photo Collections. IEEE Computer 43(6), 48–53 (2010)CrossRefGoogle Scholar
  7. 7.
    Grün, A.: Adaptive Least Squares Correlation: A Powerful Image Matching Technique. South African Journal of Photogrammetry, Remote Sensing and Cartography 14(3), 175–187 (1985)Google Scholar
  8. 8.
    Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision, 2nd edn. Cambridge University Press, Cambridge (2004)zbMATHCrossRefGoogle Scholar
  9. 9.
    Hirschmüller, H.: Stereo Processing by Semiglobal Matching and Mutual Information. IEEE Transactions on Pattern Analysis and Machine Intelligence 30(2), 328–341 (2008)CrossRefGoogle Scholar
  10. 10.
    Hirschmüller, H., Scharstein, D.: Evaluation of Stereo Matching Costs on Images with Radiometric Differences. IEEE Transactions on Pattern Analysis and Machine Intelligence 31(9), 1582–1599 (2009)CrossRefGoogle Scholar
  11. 11.
    Huang, H., Mayer, H.: Generative Statistical 3D Reconstruction of Unfoliaged Trees from Terrestrial Images. Annals of GIS 15(2), 97–105 (2009)CrossRefGoogle Scholar
  12. 12.
    Huber, P.: Robust Statistics. John Wiley & Sons, Inc., New York (1981)zbMATHCrossRefGoogle Scholar
  13. 13.
    Jian, Y.D., Balcan, D., Dellaert, F.: Generalized Subgraph Preconditioners for Large-Scale Bundle Adjustment. In: Thirteenth International Conference on Computer Vision, pp. 295–302 (2011)Google Scholar
  14. 14.
    Leberl, F., Bischof, H., Pock, T., Irschara, A., Kluckner, S.: Aerial Computer Vision for a 3D Virtual Habitat. IEEE Computer 43(6), 24–31 (2010)CrossRefGoogle Scholar
  15. 15.
    Lowe, D.: Distinctive Image Features from Scale-Invariant Keypoints. International Journal of Computer Vision 60(2), 91–110 (2004)CrossRefGoogle Scholar
  16. 16.
    Mayer, H.: Efficiency and Evaluation of Markerless 3D Reconstruction from Weakly Calibrated Long Wide-Baseline Image Loops. In: 8th Conference on Optical 3-D Measurement Techniques, vol. II, pp. 213–219 (2007)Google Scholar
  17. 17.
    Mayer, H.: Issues for Image Matching in Structure from Motion. In: International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, vol. (37) B3a, pp. 21–26 (2008)Google Scholar
  18. 18.
    Mikolajczyk, K., Schmid, C.: A Performance Evaluation of Local Descriptors. IEEE Transactions on Pattern Analysis and Machine Intelligence 27(10), 1615–1630 (2005)CrossRefGoogle Scholar
  19. 19.
    Nistér, D.: An Efficient Solution to the Five-Point Relative Pose Problem. In: Computer Vision and Pattern Recognition, vol. II, pp. 195–202 (2003)Google Scholar
  20. 20.
    Pollefeys, M., Nistér, D., Frahm, J.M., Akbarzadeh, A., Mordohai, P., Clipp, B., Engels, C., Gallup, D., Kim, S.J., Merrell, P., Salmi, C., Sinha, S., Talton, B., Wang, L., Yang, Q., Stewénius, H., Yang, R., Welch, G., Towles, H.: Detailed Real-Time Urban 3D Reconstruction from Video. International Journal of Computer Vision 78(2-3), 143–167 (2008)CrossRefGoogle Scholar
  21. 21.
    Pollefeys, M., Van Gool, L., Vergauwen, M., Verbiest, F., Cornelis, K., Tops, J.: Visual Modeling with a Hand-Held Camera. International Journal of Computer Vision 59(3), 207–232 (2004)CrossRefGoogle Scholar
  22. 22.
    Pollefeys, M., Verbiest, F., Van Gool, L.: Surviving Dominant Planes in Uncalibrated Structure and Motion Recovery. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002, Part II. LNCS, vol. 2351, pp. 837–851. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  23. 23.
    Raguram, R., Frahm, J.M.: RECON: Scale-Adaptive Robust Estimation via Residual Consensus. In: Thirteenth International Conference on Computer Vision, pp. 1299–1306 (2011)Google Scholar
  24. 24.
    Reznik, S., Mayer, H.: Implicit Shape Models, Self Diagnosis, and Model Selection for 3D Facade Interpretation. Photogrammetrie – Fernerkundung – Geoinformation 3(08), 187–196 (2008)Google Scholar
  25. 25.
    Schaffalitzky, F., Zisserman, A.: Multi-view Matching for Unordered Image Sets, or How Do I Organize My Holiday Snaps? In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2350, pp. 414–431. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  26. 26.
    Strecha, C., von Hansen, W., Van Gool, L., Fua, P., Thoennessen, U.: On Benchmarking Camera Calibration and Multi-View Stereo for High Resolution Imagery. In: Computer Vision and Pattern Recognition, pp. 1–8 (2008)Google Scholar
  27. 27.
    Torr, P.: An Assessment of Information Criteria for Motion Model Selection. In: Computer Vision and Pattern Recognition, pp. 47–53 (1997)Google Scholar
  28. 28.
    Triggs, B., McLauchlan, P., Hartley, R., Fitzgibbon, A.: Bundle Adjustment – A Modern Synthesis. In: Workshop on Vision Algorithms in conjunction with ICCV 1999, pp. 298–372 (1999)Google Scholar
  29. 29.
    Wu, C.: SiftGPU: A GPU Implementation of Scale Invariant Feature Transform (SIFT) (2007),

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Helmut Mayer
    • 1
  • Jan Bartelsen
    • 1
  • Heiko Hirschmüller
    • 2
  • Andreas Kuhn
    • 1
    • 2
  1. 1.Institute of Applied Computer ScienceBundeswehr University MunichGermany
  2. 2.Institute for Robotics and MechatronicsGerman Aerospace Center (DLR)Germany

Personalised recommendations