Advertisement

Bundle Adjustment — A Modern Synthesis

  • Bill Triggs
  • Philip F. McLauchlan
  • Richard I. Hartley
  • Andrew W. Fitzgibbon
Conference paper
First Online:
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1883)

Abstract

This paper is a survey of the theory and methods of photogrammetric bundle adjustment, aimed at potential implementors in the computer vision community. Bundle adjustment is the problem of refining a visual reconstruction to produce jointly optimal structure and viewing parameter estimates. Topics covered include: the choice of cost function and robustness; numerical optimization including sparse Newton methods, linearly convergent approximations, updating and recursive methods; gauge (datum) invariance; and quality control. The theory is developed for general robust cost functions rather than restricting attention to traditional nonlinear least squares.

Keywords

Bundle Adjustment Scene Reconstruction Gauge Freedom Sparse Matrices Optimization 
This is a preview of subscription content, log in to check access.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. [1]
    F. Ackermann. Digital image correlation: Performance and potential applications in photogrammetry. Photogrammetric Record, 11(64):429–439, 1984.CrossRefGoogle Scholar
  2. [2]
    F. Amer. Digital block adjustment. Photogrammetric Record, 4:34–47, 1962.CrossRefGoogle Scholar
  3. [3]
    E. Anderson, Z. Bai, C. Bischof, S. Blackford, J. Demmel, J. Dongarra, J. Du Croz, A. Greenbaum, S. Hammarling, A. McKenney, and D. Sorensen. LAPACK Users’ Guide, Third Edition. SIAM Press, Philadelphia, 1999. LAPACK home page: http://www.netlib.org/lapack.Google Scholar
  4. [4]
    C. Ashcraft and J. W.-H. Liu. Robust ordering of sparse matrices using multisection. SIAM J. Matrix Anal. Appl., 19:816–832, 1998.CrossRefzbMATHMathSciNetGoogle Scholar
  5. [5]
    K. B. Atkinson, editor. Close Range Photogrammetry and Machine Vision. Whittles Publishing, Roseleigh House, Latheronwheel, Caithness, Scotland, 1996.Google Scholar
  6. [6]
    W. Baarda. S-transformations and criterion matrices. Netherlands Geodetic Commission, Publications on Geodesy, New Series, Vol. 5, No. 1 (168 pages), 1967.Google Scholar
  7. [7]
    W. Baarda. Statistical concepts in geodesy Netherlands Geodetic Commission Publications on Geodesy, New Series, Vol. 2, No. 4 (74 pages), 1967.Google Scholar
  8. [8]
    W. Baarda. A testing procedure for use in geodetic networks. Netherlands Geodetic Commission Publications on Geodesy, New Series, Vol. 2, No. 5 (97 pages), 1968.Google Scholar
  9. [9]
    E. P. Baltsavias. Multiphoto Geometrically Constrained Matching. PhD thesis, ETH-Zurich, 1992.Google Scholar
  10. [10]
    R. Barrett, M. W. Berry, T. F. Chan, J. Demmel, J. Donato, J. Dongarra, V. Eijkhout, R. Pozo, C. Romine, and H. van der Vorst. Templates for the Solution of Linear Systems: Building Blocks for Iterative Methods. SIAM Press, Philadelphia, 1993.zbMATHGoogle Scholar
  11. [11]
    Åke Björck. Numerical Methods for Least Squares Problems. SIAM Press, Philadelphia, PA, 1996.zbMATHGoogle Scholar
  12. [12]
    J. A. R. Blais. Linear least squares computations using Givens transformations. Canadian Surveyor, 37(4):225–233, 1983.Google Scholar
  13. [13]
    P. T. Boggs, R. H. Byrd, J. E. Rodgers, and R. B. Schnabel. Users reference guide for ODRPACK 2.01: Software for weighted orthogonal distance regression. Technical Report NISTIR 92-4834, NIST, Gaithersburg, MD, June 1992.Google Scholar
  14. [14]
    P. T. Boggs, R. H. Byrd, and R. B. Schnabel. A stable and efficient algorithm for nonlinear orthogonal regression. SIAM J. Set Statist. Comput, 8:1052–1078, 1987.CrossRefzbMATHMathSciNetGoogle Scholar
  15. [15]
    R. J. Boscovich. De litteraria expeditione per pontificiam ditionem, et synopsis amplioris operis, ac habenturplura ejus ex exemplaria etiam sensorum impressa. Bononiensi Scientarum et Artum Instituto Atque Academia Commentarii, IV:353–396, 1757.Google Scholar
  16. [16]
    D. C. Brown. A solution to the general problem of multiple station analytical stereotriangulation. Technical Report RCA-MTP Data Reduction Technical Report No. 43 (or AFMTC TR 58-8), Patrick Airforce Base, Florida, 1958.Google Scholar
  17. [17]
    D. C. Brown. Close range camera calibration. Photogrammetric Engineering, XXXVII(8), August 1971.Google Scholar
  18. [18]
    D. C. Brown. Calibration of close range cameras. Int. Archives Photogrammetry, 19(5), 1972. Unbound paper (26 pages).Google Scholar
  19. [19]
    D. C. Brown. The bundle adjustment — progress and prospects. Int. Archives Photogrammetry, 21(3), 1976. Paper number 3-03 (33 pages).Google Scholar
  20. [20]
    Q. Chen and G. Medioni. Efficient iterative solutions to m-view projective reconstruction problem. In Int. Conf. Computer Vision & Pattern Recognition, pages II:55–61. IEEE Press, 1999.Google Scholar
  21. [21]
    M. A. R. Cooper and P. A. Cross. Statistical concepts and their application in photogrammetry and surveying. Photo grammetric Record, 12(71):637–663, 1988.Google Scholar
  22. [22]
    M. A. R. Cooper and P. A. Cross. Statistical concepts and their application in photogrammetry and surveying (continued). Photo grammetric Record, 13(77):645–678, 1991.Google Scholar
  23. [23]
    D. R. Cox and D. V. Hinkley. Theoretical Statistics. Chapman & Hall, 1974.Google Scholar
  24. [24]
    P. J. de Jonge. A comparative study of algorithms for reducing the fill-in during Cholesky factorization. Bulletin Géodésique, 66:296–305, 1992.CrossRefGoogle Scholar
  25. [25]
    A. Dermanis. The photogrammetric inner constraints. J. Photogrammetry & Remote Sensing, 49(1):25–39, 1994.CrossRefGoogle Scholar
  26. [26]
    I. Duff, A. M. Erisman, and J. K. Reid. Direct Methods for Sparse Matrices. Oxford University Press, 1986.Google Scholar
  27. [27]
    O. Faugeras. What can be seen in three dimensions with an uncalibrated stereo rig? In G. Sandini, editor, European Conf. Computer Vision, Santa Margherita Ligure, Italy, May 1992. Springer-Verlag.Google Scholar
  28. [28]
    A. W. Fitzgibbon and A. Zisserman. Automatic camera recovery for closed or open image sequences. In European Conf. Computer Vision, pages 311–326, Freiburg, 1998.Google Scholar
  29. [29]
    R. Fletcher. Practical Methods of Optimization. John Wiley, 1987.Google Scholar
  30. [30]
    W. Förstner. Evaluation of block adjustment results. Int. Arch. Photogrammetry, 23-III, 1980.Google Scholar
  31. [31]
    W. Förstner. On the geometric precision of digital correlation. Int. Arch. Photogrammetry & Remote Sensing, 24(3): 176–189, 1982.Google Scholar
  32. [32]
    W. Förstner. A feature-based correspondence algorithm for image matching. Int. Arch. Photogrammetry & Remote Sensing, 26(3/3): 150–166, 1984.Google Scholar
  33. [33]
    W. Förstner. The reliability of block triangulation. Photogrammetric Engineering & Remote Sensing, 51(8): 1137–1149, 1985.Google Scholar
  34. [34]
    W. Förstner. Reliability analysis of parameter estimation in linear models with applications to mensuration problems in computer vision. Computer Vision, Graphics & Image Processing, 40:273–310, 1987.CrossRefGoogle Scholar
  35. [35]
    D. A. Forsyth, S. Ioffe, and J. Haddon. Bayesian structure from motion. In Int. Conf. Computer Vision, pages 660–665, Corfu, 1999.Google Scholar
  36. [36]
    C. F. Gauss. Werke. Königlichen Gesellschaft der Wissenschaften zu Göttingen, 1870–1928.Google Scholar
  37. [37]
    C. F. Gauss. Theoria Combinationis Observationum Erroribus Minimis Obnoxiae (Theory of the Combination of Observations Least Subject to Errors). SIAM Press, Philadelphia, PA, 1995. Originally published in Commentatines Societas Regiae Scientarium Gottingensis Recentiores 5, 1823 (Pars prior, Pars posterior), 6, 1828 (Supplementum). Translation and commentary by G. W. Stewart.Google Scholar
  38. [38]
    J. A. George. Nested dissection of a regular finite element mesh. SIAM J. Numer. Anal. 10:345–363, 1973.CrossRefzbMATHMathSciNetGoogle Scholar
  39. [39]
    J. A. George, M. T. Heath, and E. G. Ng. A comparison of some methods for solving sparse linear least squares problems. SIAM J. Set Statist. Comput., 4:177–187, 1983.CrossRefzbMATHMathSciNetGoogle Scholar
  40. [40]
    J. A. George and J. W.-H. Liu. Computer Solution of Large Sparse Positive Definite Systems. Prentice-Hall, 1981.Google Scholar
  41. [41]
    J. A. George and J. W.-H. Liu. Householder reflections versus Givens rotations in sparse orthogonal decomposition. Lin. Alg. Appl., 88/89:223–238, 1987.CrossRefMathSciNetGoogle Scholar
  42. [42]
    P. Gill, W. Murray, and M. Wright. Practical Optimization. Academic Press, 1981.Google Scholar
  43. [43]
    P. E. Gill, G. H. Golub, W. Murray, and M. Saunders. Methods for modifying matrix factorizations. Math. Comp., 28:505–535, 1974.CrossRefzbMATHMathSciNetGoogle Scholar
  44. [44]
    G. Golub and C. F. Van Loan. Matrix Computations. Johns Hopkins University Press, 3rd edition, 1996.Google Scholar
  45. [45]
    G. Golub and R. Plemmons. Large-scale geodetic least squares adjustment by dissection and orthogonal decomposition. Linear Algebra Appl., 34:3–28, 1980.CrossRefzbMATHMathSciNetGoogle Scholar
  46. [46]
    S. Granshaw. Bundle adjustment methods in engineering photogrammetry. Photogrammetric Record, 10(56):181–207, 1980.CrossRefGoogle Scholar
  47. [47]
    A. Greenbaum. Behaviour of slightly perturbed Lanczos and conjugate-gradient recurrences. Linear Algebra Appl., 113:7–63, 1989.CrossRefzbMATHMathSciNetGoogle Scholar
  48. [48]
    A. Greenbaum. Iterative Methods for Solving Linear Systems. SIAM Press, Philadelphia, 1997.zbMATHGoogle Scholar
  49. [49]
    A. Grün. Accuracy, reliability and statistics in close range photogrammetry. In Inter-Congress Symposium of ISP Commission V, page Presented paper. Unbound paper No.9 (24 pages), Stockholm, 1978.Google Scholar
  50. [50]
    A. Grün. Precision and reliability aspects in close range photogrammetry. Int. Arch. Photogrammetry, 11(23B):378–391, 1980.Google Scholar
  51. [51]
    A. Grün. An optimum algorithm for on-line triangulation. In Symposium of Commission III of the ISPRS, Helsinki, 1982.Google Scholar
  52. [52]
    A. Grün. Adaptive least squares correlation — concept and first results. Intermediate Research Report to Helava Associates, Ohio State University. 13 pages, March 1984.Google Scholar
  53. [53]
    A. Grün. Adaptive kleinste Quadrate Korrelation and geometrische Zusatzinformationen. Vermessung, Photogrammetrie, Kulturtechnik, 9(83):309–312, 1985.Google Scholar
  54. [54]
    A. Grün. Algorithmic aspects of on-line triangulation. Photogrammetric Engineering & Remote Sensing, 4(51):419–436, 1985.Google Scholar
  55. [55]
    A. Grün and E. P. Baltsavias. Adaptive least squares correlation with geometrical constraints. In SPIE Computer Vision for Robots, volume 595, pages 72–82, Cannes, 1985.Google Scholar
  56. [56]
    R. Gupta and R. I. Hartley. Linear pushbroom cameras. IEEE Trans. Pattern Analysis & Machine Intelligence, September 1997.Google Scholar
  57. [57]
    M. S. Gyer. The inversion of the normal equations of analytical aerotriangulation by the method of recursive partitioning. Technical report, Rome Air Development Center, Rome, New York, 1967.Google Scholar
  58. [58]
    R. Hartley. Euclidean reconstruction from multiple views. In 2 nd Europe-U.S. Workshop on Invariance, pages 237–56, Ponta Delgada, Azores, October 1993.Google Scholar
  59. [59]
    R. Hartley. An object-oriented approach to scene reconstruction. In IEEE Conf. Systems, Man & Cybernetics, pages 2475–2480, Beijing, October 1996.Google Scholar
  60. [60]
    R. Hartley. Lines and points in three views and the trifocal tensor. Int. J. Computer Vision, 22(2): 125–140, 1997.CrossRefGoogle Scholar
  61. [61]
    R. Hartley, R. Gupta, and T. Chang. Stereo from uncalibrated cameras. In Int. Conf. Computer Vision & Pattern Recognition, pages 761–4, Urbana-Champaign, Illinois, 1992.Google Scholar
  62. [62]
    R. Hartley and A. Zisserman. Multiple View Geometry in Computer Vision. Cambridge University Press, 2000.Google Scholar
  63. [63]
    R. I. Hartley and T. Saxena. The cubic rational polynomial camera model. In Image Understanding Workshop, pages 649–653, 1997.Google Scholar
  64. [64]
    F. Helmert. Die Mathematischen und Physikalischen Theorien der höheren Geodäsie, volume 1 Teil. Teubner, Leipzig, 1880.Google Scholar
  65. [65]
    B. Hendrickson and E. Rothberg. Improving the run time and quality of nested dissection ordering. SIAM J. Set Comput., 20:468–489, 1998.CrossRefMathSciNetGoogle Scholar
  66. [66]
    K. R. Holm. Test of algorithms for sequential adjustment in on-line triangulation. Photogrammetria, 43:143–156, 1989.CrossRefGoogle Scholar
  67. [67]
    M. Irani, P. Anadan, and M. Cohen. Direct recovery of planar-parallax from multiple frames. In Vision Algorithms: Theory and Practice. Springer-Verlag, 2000.Google Scholar
  68. [68]
    K. Kanatani and N. Ohta. Optimal robot self-localization and reliability evaluation. In European Conf. Computer Vision, pages 796–808, Freiburg, 1998.Google Scholar
  69. [69]
    H. M. Karara. Non-Topographic Photogrammetry. Americal Society for Photogrammetry and Remote Sensing, 1989.Google Scholar
  70. [70]
    G. Karypis and V. Kumar. Multilevel k-way partitioning scheme for irregular graphs. J. Parallel & Distributed Computing, 48:96–129, 1998.CrossRefGoogle Scholar
  71. [71]
    G. Karypis and V. Kumar. A fast and highly quality multilevel scheme for partitioning irregular graphs. SIAM J. Scientific Computing, 20(1):359–392, 1999. For Metis code see http://www-users.cs.umn.edu/karypis/.CrossRefzbMATHMathSciNetGoogle Scholar
  72. [72]
    I. P. King. An automatic reordering scheme for simultaneous equations derived from network systems. Int. J. Numer. Meth. Eng., 2:479–509, 1970.CrossRefGoogle Scholar
  73. [73]
    K. Kraus. Photogrammetry. Dummler, Bonn, 1997. Vol.1: Fundamentals and Standard Processes. Vol.2: Advanced Methods and Applications. Available in German, English & several other languages.Google Scholar
  74. [74]
    A. M. Legendre. Nouvelles méthodes pour la détermination des orbites des comètes. Courcier, Paris, 1805. Appendix on least squares.Google Scholar
  75. [75]
    R. Levy. Restructuring the structural stiffness matrix to improve computational efficiency. Jet Propulsion Lab. Technical Review, 1:61–70, 1971.Google Scholar
  76. [76]
    M. X. Li. Hierarchical Multi-point Matching with Simultaneous Detection and Location of Breaklines. PhD thesis, KTH Stockholm, 1989.Google Scholar
  77. [77]
    Q.-T. Luong, R. Deriche, O. Faugeras, and T. Papadopoulo. On determining the fundamental matrix: Analysis of different methods and experimental results. Technical Report RR-1894, INRIA, Sophia Antipolis, France, 1993.Google Scholar
  78. [78]
    S. Mason. Expert system based design of close-range photogrammetric networks. J. Photogrammetry & Remote Sensing, 50(5): 13–24, 1995.CrossRefGoogle Scholar
  79. [79]
    S.O. Mason. Expert System Based Design of Photogrammetric Networks. Ph.D. Thesis, Institut für Geodäsie und Photogrammetrie, ETH Zürich, May 1994.Google Scholar
  80. [80]
    B. Matei and P. Meer. Bootstrapping a heteroscedastic regression model with application to 3D rigid motion evaluation. In Vision Algorithms: Theory and Practice. Springer-Verlag, 2000.Google Scholar
  81. [81]
    P. F. McLauchlan. Gauge independence in optimization algorithms for 3D vision. In Vision Algorithms: Theory and Practice, Lecture Notes in Computer Science, Corfu, September 1999. Springer-Verlag.Google Scholar
  82. [82]
    P. F. McLauchlan. Gauge invariance in projective 3D reconstruction. In Multi-View Modeling and Analysis of Visual Scenes, Fort Collins, CO, June 1999. IEEE Press.Google Scholar
  83. [83]
    P. F. McLauchlan. The variable state dimension filter. Technical Report VSSP 5/99, University of Surrey, Dept of Electrical Engineering, December 1999.Google Scholar
  84. [84]
    P. F. McLauchlan. A batch/recursive algorithm for 3D scene reconstruction. In Int. Conf. Computer Vision & Pattern Recognition, Hilton Head, South Carolina, 2000.Google Scholar
  85. [85]
    P. F. McLauchlan and D. W. Murray. A unifying framework for structure and motion recovery from image sequences. In E. Grimson, editor, Int. Conf. Computer Vision, pages 314–20, Cambridge, MA, June 1995.Google Scholar
  86. [86]
    P. F. McLauchlan and D. W. Murray. Active camera calibration for a Head-Eye platform using the Variable State-Dimension filter. IEEE Trans. Pattern Analysis & Machine Intelligence, 18(1): 15–22, 1996.CrossRefGoogle Scholar
  87. [87]
    P. Meissl. Die innere Genauigkeit eines Punkthaufens. Österreichische Zeitschrift für Vermessungswesen, 50(5): 159–165 and 50(6): 186–194, 1962.Google Scholar
  88. [88]
    E. Mikhail and R. Helmering. Recursive methods in photogrammetric data reduction. Photogrammetric Engineering, 39(9):983–989, 1973.Google Scholar
  89. [89]
    E. Mittermayer. Zur Ausgleichung freier Netze. Zeitschrift für Vermessungswesen, 97(11):481–489, 1962.Google Scholar
  90. [90]
    J. J. Moré and S. J. Wright. Optimization Software Guide. SIAM Press, Philadelphia, 1993.zbMATHGoogle Scholar
  91. [91]
    D. D. Morris and T. Kanade. A unified factorization algorithm for points, line segments and planes with uncertainty. In Int. Conf. Computer Vision, pages 696–702, Bombay, 1998.Google Scholar
  92. [92]
    D. D. Morris, K. Kanatani, and T. Kanade. Uncertainty modelling for optimal structure and motion. In Vision Algorithms: Theory and Practice. Springer-Verlag, 2000.Google Scholar
  93. [93]
    J. Nocedal and S.J. Wright. Numerical Optimization. Springer-Verlag, 1999.Google Scholar
  94. [94]
    M. Okutomi and T. Kanade. A multiple-baseline stereo. IEEE Trans. Pattern Analysis & Machine Intelligence, 15(4):353–363, 1993.CrossRefGoogle Scholar
  95. [95]
    D. W. Proctor. The adjustment of aerial triangulation by electronic digital computers. Photogrammetric Record, 4:24–33, 1962.CrossRefGoogle Scholar
  96. [96]
    B. D. Ripley. Pattern Recongition and Neural Networks. Cambridge University Press, 1996.Google Scholar
  97. [97]
    D. Rosenholm. Accuracy improvement of digital matching for elevation of digital terrain models. Int. Arch. Photogrammetry & Remote Sensing, 26(3/2):573–587, 1986.Google Scholar
  98. [98]
    S. Roy and I. Cox. A maximum-flow formulation of the n-camera stereo correspondence problem. In Int. Conf. Computer Vision, Bombay, 1998.Google Scholar
  99. [99]
    Y. Saad. On the rates of convergence of Lanczos and block-Lanczos methods. SIAMJ. Numer. Anal., 17:687–706, 1980.CrossRefzbMATHMathSciNetGoogle Scholar
  100. [100]
    C. C. Slama, editor. Manual of Photo grammetry. American Society of Photogrammetry and Remote Sensing, Falls Church, Virginia, USA, 1980.Google Scholar
  101. [101]
    R. A. Snay. Reducing the profile of sparse symmetric matrices. Bulletin Géodésique, 50:341–352, 1976. Also NOAA Technical Memorandum NOS NGS-4, National Geodetic Survey, Rockville, MD.CrossRefMathSciNetGoogle Scholar
  102. [102]
    R. Szeliski, S.B. Kang, and H. Y. Shum. A parallel feature tracker for extended image sequences. Technical Report CRL 95/2, DEC Cambridge Research Labs, May 1995.Google Scholar
  103. [103]
    R. Szeliski and S.B. Kang. Shape ambiguities in structure from motion. In European Conf. Computer Vision, pages 709–721, Cambridge, 1996.Google Scholar
  104. [104]
    R. Szeliski and H.Y Shum. Motion estimation with quadtree splines. In Int. Conf. Computer Vision, pages 757–763, Boston, 1995.Google Scholar
  105. [105]
    B. Triggs. A new approach to geometric fitting. Available from http://www.inrialpes.fr/movi/people/Triggs, 1997.
  106. [106]
    B. Triggs. Optimal estimation of matching constraints. In R. Koch and L. Van Gool, editors, 3D Structure from Multiple Images of Large-scale Environments SMILE’98, Lecture Notes in Computer Science. Springer-Verlag, 1998.Google Scholar
  107. [107]
    G. L. Strang van Hees. Variance-covariance transformations of geodetic networks. Manuscripta Geodaetica, 7:1–20, 1982.zbMATHGoogle Scholar
  108. [108]
    X. Wang and T. A. Clarke. Separate adjustment of close range photogrammetric measurements. Int. Symp. Photogrammetry & Remote Sensing, XXXII, part 5:177–184, 1998.Google Scholar
  109. [109]
    P.R. Wolf and C.D. Ghilani. Adjustment Computations: Statistics and Least Squares in Surveying and GIS. John Wiley & Sons, 1997.Google Scholar
  110. [110]
    B.P. Wrobel. Facets stereo vision (FAST vision) — a new approach to computer stereo vision and to digital photogrammetry. In ISPRS Intercommission Conf. Fast Processing of Photogrammetric Data, pages 231–258, Interlaken, Switzerland, June 1987.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2000

Authors and Affiliations

  • Bill Triggs
    • 1
  • Philip F. McLauchlan
    • 2
  • Richard I. Hartley
    • 3
  • Andrew W. Fitzgibbon
    • 4
  1. 1.INRIA Rhône-AlpesMontbonnotFrance
  2. 2.School of Electrical Engineering, Information Technology & MathematicsUniversity of SurreyGuildfordUK
  3. 3.General Electric CRDSchenectady
  4. 4.Dept of Engineering ScienceUniversity of OxfordUK

Personalised recommendations

Cite paper

Buy options