Skip to main content
SpringerLink
Log in

Exploiting Redundancy for Large Scale Bundle Adjustment: In Partial Defense of Minimization by Alternation

  • Conference paper
  • First Online:
Image Analysis (SCIA 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13886))

Included in the following conference series:

  • 591 Accesses

Abstract

This paper presents a new approach to accelerate large-scale bundle adjustment (BA). As starting point, we empirically demonstrate that there exist a significant amount of redundant information that has yet to be leveraged in many real-world BA datasets. We propose an adaptive algorithm that builds on this redundancy and further utilizes the bipartite dependency structure of BA. Our algorithm maintains a small set of “training” 3D landmarks that is used to update the camera parameters, while the remaining landmarks are updated using faster point iterations (PI) in an inexact manner. This training set of landmarks is extended as necessary as indicated by a suitable “overfitting” criterion. The proposed algorithm also works gracefully for BA instances with robustified costs (such as the robust Cauchy and Geman-McClure costs). Experimental results on several large-scale BA datasets show that our method achieves faster convergence rates compared to several existing standard approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    as verified in the supplementary material.

  2. 2.

    In principle, the theory such as [35] suggests the necessary accuracy for solving the linear system, but the underlying theory only holds for instances with zero optimal objective value.

  3. 3.

    We observe similar behavior when using only the block-diagonal part in the augmented normal equation.

  4. 4.

    https://github.com/chzach/SSBA.

  5. 5.

    https://github.com/zlthinker/STBA.

  6. 6.

    http://www.maths.lth.se/matematiklth/personal/calle/dataset/dataset.html.

  7. 7.

    C++ code made available by the authors.

References

  1. Agarwal, S., Snavely, N., Seitz, S.M., Szeliski, R.: Bundle adjustment in the large. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6312, pp. 29–42. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-15552-9_3

    Chapter  Google Scholar 

  2. Bergou, E., Gratton, S., Vicente, L.N.: Levenberg-marquardt methods based on probabilistic gradient models and inexact subproblem solution, with application to data assimilation. SIAM/ASA J. Uncertainty Quantification 4(1), 924–951 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  3. Bollapragada, R., Nocedal, J., Mudigere, D., Shi, H.J., Tang, P.T.P.: A progressive batching L-BFGS method for machine learning. In: International Conference on Machine Learning, pp. 620–629 (2018)

    Google Scholar 

  4. Byrd, R.H., Hansen, S.L., Nocedal, J., Singer, Y.: A stochastic quasi-newton method for large-scale optimization. SIAM J. Optim. 26(2), 1008–1031 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  5. Byröd, M., Åström, K.: Bundle adjustment using conjugate gradients with multiscale preconditioning

    Google Scholar 

  6. Byröd, M., Åström, K.: Conjugate gradient bundle adjustment. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6312, pp. 114–127. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-15552-9_9

    Chapter  Google Scholar 

  7. Curtis, F.E., Shi, R.: A fully stochastic second-order trust region method. arXiv preprint arXiv:1911.06920 (2019)

  8. Dai, A., Nießner, M., Zollhöfer, M., Izadi, S., Theobalt, C.: BundleFusion: real-time globally consistent 3D reconstruction using on-the-fly surface reintegration. ACM Trans. Graph. (ToG) 36(4), 1 (2017)

    Article  Google Scholar 

  9. Dan, H., Yamashita, N., Fukushima, M.: Convergence properties of the inexact levenberg-marquardt method under local error bound conditions. Optim. Methods Softw. 17(4), 605–626 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  10. Davis, T.A., Gilbert, J.R., Larimore, S.I., Ng, E.G.: Algorithm 836: colamd, a column approximate minimum degree ordering algorithm. ACM Trans. Math. Softw. (TOMS) 30(3), 377–380 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  11. Dellaert, F., Carlson, J., Ila, V., Ni, K., Thorpe, C.E.: Subgraph-preconditioned conjugate gradients for large scale slam. In: 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 2566–2571. IEEE (2010)

    Google Scholar 

  12. Dolan, E.D., Moré, J.J.: Benchmarking optimization software with performance profiles. Math. Program. 91(2), 201–213 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  13. Enqvist, O., Kahl, F., Olsson, C.: Non-sequential structure from motion. In: 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops), pp. 264–271. IEEE (2011)

    Google Scholar 

  14. Eriksson, A., Bastian, J., Chin, T.J., Isaksson, M.: A consensus-based framework for distributed bundle adjustment. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1754–1762 (2016)

    Google Scholar 

  15. Facchinei, F., Kanzow, C.: A nonsmooth inexact newton method for the solution of large-scale nonlinear complementarity problems. Math. Program. 76(3), 493–512 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  16. Golub, G.H., Pereyra, V.: The differentiation of pseudo-inverses and nonlinear least squares problems whose variables separate. SIAM J. Numer. Anal. 10(2), 413–432 (1973)

    Article  MathSciNet  MATH  Google Scholar 

  17. Jeong, Y., Nister, D., Steedly, D., Szeliski, R., Kweon, I.S.: Pushing the envelope of modern methods for bundle adjustment. IEEE Trans. Pattern Anal. Mach. Intell. 34(8), 1605–1617 (2011)

    Article  Google Scholar 

  18. Katyan, S., Das, S., Kumar, P.: Two-grid preconditioned solver for bundle adjustment. In: The IEEE Winter Conference on Applications of Computer Vision, pp. 3599–3606 (2020)

    Google Scholar 

  19. Konolige, T., Brown, J.: Multigrid for bundle adjustment. arXiv preprint arXiv:2007.01941 (2020)

  20. Kushal, A., Agarwal, S.: Visibility based preconditioning for bundle adjustment. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1442–1449. IEEE (2012)

    Google Scholar 

  21. Le, H., Zach, C., Rosten, E., Woodford, O.J.: Progressive batching for efficient non-linear least squares (2020)

    Google Scholar 

  22. Liu, D.C., Nocedal, J.: On the limited memory BFGS method for large scale optimization. Math. Program. 45(1–3), 503–528 (1989)

    Article  MathSciNet  MATH  Google Scholar 

  23. Lourakis, M., Argyros, A.A.: Is levenberg-marquardt the most efficient optimization algorithm for implementing bundle adjustment? In: Tenth IEEE International Conference on Computer Vision (ICCV 2005) Volume 1. vol. 2, pp. 1526–1531. IEEE (2005)

    Google Scholar 

  24. Mitra, K., Chellappa, R.: A scalable projective bundle adjustment algorithm using the l infinity norm. In: 2008 Sixth Indian Conference on Computer Vision, Graphics & Image Processing, pp. 79–86. IEEE (2008)

    Google Scholar 

  25. Moré, J.J.: The Levenberg-marquardt algorithm: implementation and theory. In: Watson, G.A. (ed.) Numerical Analysis. LNM, vol. 630, pp. 105–116. Springer, Heidelberg (1978). https://doi.org/10.1007/BFb0067700

    Chapter  Google Scholar 

  26. Mur-Artal, R., Montiel, J.M.M., Tardos, J.D.: ORB-SLAM: a versatile and accurate monocular slam system. IEEE Trans. Robot. 31(5), 1147–1163 (2015)

    Article  Google Scholar 

  27. Ni, K., Steedly, D., Dellaert, F.: Out-of-core bundle adjustment for large-scale 3D reconstruction. In: 2007 IEEE 11th International Conference on Computer Vision, pp. 1–8. IEEE (2007)

    Google Scholar 

  28. Nocedal, J., Wright, S.: Numerical optimization. Springer Science & Business Media (2006)

    Google Scholar 

  29. Parra, Á., Chin, T.J., Eriksson, A., Reid, I.: Visual slam: Why bundle adjust? arXiv preprint arXiv:1902.03747 (2019)

  30. Saad, Y.: Iterative methods for sparse linear systems. SIAM (2003)

    Google Scholar 

  31. Schonberger, J.L., Frahm, J.M.: Structure-from-motion revisited. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4104–4113 (2016)

    Google Scholar 

  32. Tran-Dinh, Q., Pham, N.H., Nguyen, L.M.: Stochastic gauss-newton algorithms for nonconvex compositional optimization. arXiv preprint arXiv:2002.07290 (2020)

  33. Transtrum, M.K., Machta, B.B., Sethna, J.P.: Geometry of nonlinear least squares with applications to sloppy models and optimization. Phys. Rev. E 83(3), 036701 (2011)

    Article  Google Scholar 

  34. Triggs, B., McLauchlan, P.F., Hartley, R.I., Fitzgibbon, A.W.: Bundle adjustment — a modern synthesis. In: Triggs, B., Zisserman, A., Szeliski, R. (eds.) IWVA 1999. LNCS, vol. 1883, pp. 298–372. Springer, Heidelberg (2000). https://doi.org/10.1007/3-540-44480-7_21

    Chapter  Google Scholar 

  35. Wright, S., Holt, J.N.: An inexact levenberg-marquardt method for large sparse nonlinear least squares. ANZIAM J. 26(4), 387–403 (1985)

    MathSciNet  MATH  Google Scholar 

  36. Wu, C., Agarwal, S., Curless, B., Seitz, S.M.: Multicore bundle adjustment. In: CVPR 2011, pp. 3057–3064. IEEE (2011)

    Google Scholar 

  37. Zhang, F.: The Schur Complement and its Applications. vol. 4. Springer Science & Business Media (2006). https://doi.org/10.1007/b105056

  38. Zhang, R., Zhu, S., Fang, T., Quan, L.: Distributed very large scale bundle adjustment by global camera consensus. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 29–38 (2017)

    Google Scholar 

  39. Zhou, L., et al.: Stochastic bundle adjustment for efficient and scalable 3D reconstruction. arXiv preprint arXiv:2008.00446 (2020)

Download references

Acknowledgement

This work was partially supported by the Wallenberg AI, Autonomous Systems and Software Program (WASP) funded by the Knut and Alice Wallenberg Foundation.

Author information

Authors and Affiliations

  1. Chalmers University of Technology, Gothenburg, Sweden

    Christopher Zach & Huu Le

  2. Now with Tobii, Stockholm, Sweden

    Huu Le

Authors
  1. Christopher Zach
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Huu Le
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Christopher Zach .

Editor information

Editors and Affiliations

  1. Aalborg University, Aalborg, Denmark

    Rikke Gade

  2. Linköping University, Linköping, Sweden

    Michael Felsberg

  3. Tampere University, Tampere, Finland

    Joni-Kristian Kämäräinen

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zach, C., Le, H. (2023). Exploiting Redundancy for Large Scale Bundle Adjustment: In Partial Defense of Minimization by Alternation. In: Gade, R., Felsberg, M., Kämäräinen, JK. (eds) Image Analysis. SCIA 2023. Lecture Notes in Computer Science, vol 13886. Springer, Cham. https://doi.org/10.1007/978-3-031-31438-4_33

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-31438-4_33

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-31437-7

  • Online ISBN: 978-3-031-31438-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships

Search

Navigation