Stochastic Bundle Adjustment for Efficient and Scalable 3D Reconstruction

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 12360)


Current bundle adjustment solvers such as the Levenberg-Marquardt (LM) algorithm are limited by the bottleneck in solving the Reduced Camera System (RCS) whose dimension is proportional to the camera number. When the problem is scaled up, this step is neither efficient in computation nor manageable for a single compute node. In this work, we propose a stochastic bundle adjustment algorithm which seeks to decompose the RCS approximately inside the LM iterations to improve the efficiency and scalability. It first reformulates the quadratic programming problem of an LM iteration based on the clustering of the visibility graph by introducing the equality constraints across clusters. Then, we propose to relax it into a chance constrained problem and solve it through sampled convex program. The relaxation is intended to eliminate the interdependence between clusters embodied by the constraints, so that a large RCS can be decomposed into independent linear sub-problems. Numerical experiments on unordered Internet image sets and sequential SLAM image sets, as well as distributed experiments on large-scale datasets, have demonstrated the high efficiency and scalability of the proposed approach. Codes are released at


Stochastic bundle adjustment Clustering 3D reconstruction 



This work is supported by Hong Kong RGC GRF16206819 & 16203518 and T22-603/15N.

Supplementary material

504470_1_En_22_MOESM1_ESM.pdf (1.9 mb)
Supplementary material 1 (pdf 1899 KB)


  1. 1.
    Agarwal, S., Mierle, K., et al.: Ceres solver.
  2. 2.
    Agarwal, S., Snavely, N., Seitz, S.M., Szeliski, R.: Bundle adjustment in the large. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6312, pp. 29–42. Springer, Heidelberg (2010). Scholar
  3. 3.
    Amestoy, P.R., Davis, T.A., Duff, I.S.: An approximate minimum degree ordering algorithm. SIAM J. Matrix Anal. Appl. 17(4), 886–905 (1996)MathSciNetCrossRefGoogle Scholar
  4. 4.
    Bertsekas, D.P.: Parallel and Distributed Computation: Numerical Methods, vol. 3 (1989)Google Scholar
  5. 5.
    Bertsekas, D.P.: Constrained Optimization and Lagrange Multiplier Methods. Academic Press, Cambridge (2014)zbMATHGoogle Scholar
  6. 6.
    Blondel, V.D., Guillaume, J.L., Lambiotte, R., Lefebvre, E.: Fast unfolding of communities in large networks. J. Stat. Mech. Theor. Exp. 2008(10), P10008 (2008)CrossRefGoogle Scholar
  7. 7.
    Calafiore, G., Campi, M.C.: Uncertain convex programs: randomized solutions and confidence levels. Math. Program. 102(1), 25–46 (2005)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Campi, M.C., Garatti, S.: A sampling-and-discarding approach to chance-constrained optimization: feasibility and optimality. J. Optim. Theor. Appl. 148(2), 257–280 (2011)MathSciNetCrossRefGoogle Scholar
  9. 9.
    Chatterjee, A., Madhav Govindu, V.: Efficient and robust large-scale rotation averaging. In: ICCV (2013)Google Scholar
  10. 10.
    Combettes, P.L., Pesquet, J.C.: Proximal splitting methods in signal processing. In: Bauschke, H., Burachik, R., Combettes, P., Elser, V., Luke, D., Wolkowicz, H. (eds.) Fixed-Point Algorithms for Inverse Problems in Science and Engineering. Springer Optimization and Its Applications, vol. 49. Springer, New York (2011). Scholar
  11. 11.
    Darmaillac, Y., Loustau, S.: MCMC Louvain for online community detection. arXiv preprint arXiv:1612.01489 (2016)
  12. 12.
    Davis, T.A., Gilbert, J.R., Larimore, S.I., Ng, E.G.: Algorithm 836: COLAMD, a column approximate minimum degree ordering algorithm. TOMS 30(3), 377–380 (2004)MathSciNetCrossRefGoogle Scholar
  13. 13.
    Dellaert, F., Carlson, J., Ila, V., Ni, K., Thorpe, C.E.: Subgraph-preconditioned conjugate gradients for large scale SLAM. In: IROS (2010)Google Scholar
  14. 14.
    Dolan, E.D., Moré, J.J.: Benchmarking optimization software with performance profiles. Math. Program. 91(2), 201–213 (2002)MathSciNetCrossRefGoogle Scholar
  15. 15.
    Engels, C., Stewénius, H., Nistér, D.: Bundle adjustment rulesGoogle Scholar
  16. 16.
    Eriksson, A., Bastian, J., Chin, T.J., Isaksson, M.: A consensus-based framework for distributed bundle adjustment. In: CVPR (2016)Google Scholar
  17. 17.
    Fang, M., Pollok, T., Qu, C.: Merge-SfM: merging partial reconstructions. In: BMVC (2019)Google Scholar
  18. 18.
    Geiger, A., Lenz, P., Urtasun, R.: Are we ready for autonomous driving? The KITTI vision benchmark suite. In: CVPR (2012)Google Scholar
  19. 19.
    Hestenes, M.R., et al.: Methods of conjugate gradients for solving linear systems. J. Res. Natl. Bur. Stan. 49(6), 409–436 (1952)MathSciNetCrossRefGoogle Scholar
  20. 20.
    Jeong, Y., Nister, D., Steedly, D., Szeliski, R., Kweon, I.S.: Pushing the envelope of modern methods for bundle adjustment. PAMI 34(8), 1605–1617 (2011)CrossRefGoogle Scholar
  21. 21.
    Jian, Y.-D., Balcan, D.C., Dellaert, F.: Generalized subgraph preconditioners for large-scale bundle adjustment. In: Dellaert, F., Frahm, J.-M., Pollefeys, M., Leal-Taixé, L., Rosenhahn, B. (eds.) Outdoor and Large-Scale Real-World Scene Analysis. LNCS, vol. 7474, pp. 131–150. Springer, Heidelberg (2012). Scholar
  22. 22.
    Konolige, K., Garage, W.: Sparse sparse bundle adjustment. In: BMVC (2010)Google Scholar
  23. 23.
    Kushal, A., Agarwal, S.: Visibility based preconditioning for bundle adjustment. In: CVPR (2012)Google Scholar
  24. 24.
    Li, P., Arellano-Garcia, H., Wozny, G.: Chance constrained programming approach to process optimization under uncertainty. Comput. Chem. Eng. 32(1–2), 25–45 (2008)CrossRefGoogle Scholar
  25. 25.
    Lourakis, M.I., Argyros, A.A.: SBA: a software package for generic sparse bundle adjustment. TOMS 36(1), 2 (2009)MathSciNetCrossRefGoogle Scholar
  26. 26.
    Lourakis, M., Argyros, A.A.: Is Levenberg-Marquardt the most efficient optimization algorithm for implementing bundle adjustment? In: ICCV (2005)Google Scholar
  27. 27.
    Marquardt, D.W.: An algorithm for least-squares estimation of nonlinear parameters. J. Soc. Ind. Appl. Math. 11(2), 431–441 (1963)MathSciNetCrossRefGoogle Scholar
  28. 28.
    Mur-Artal, R., Tardós, J.D.: ORB-SLAM2: an open-source SLAM system for monocular, stereo and RGB-D cameras. IEEE Trans. Robot. 33(5), 1255–1262 (2017)CrossRefGoogle Scholar
  29. 29.
    Ni, K., Steedly, D., Dellaert, F.: Out-of-core bundle adjustment for large-scale 3D reconstruction. In: ICCV (2007)Google Scholar
  30. 30.
    Rotkin, V., Toledo, S.: The design and implementation of a new out-of-core sparse Cholesky factorization method. TOMS 30(1), 19–46 (2004)MathSciNetCrossRefGoogle Scholar
  31. 31.
    Schaeffer, S.E.: Survey: graph clustering. Comput. Sci. Rev. 1(1), 27–64 (2007)CrossRefGoogle Scholar
  32. 32.
    Schönberger, J.L., Frahm, J.M.: Structure-from-motion revisited. In: CVPR (2016)Google Scholar
  33. 33.
    Wilson, K., Snavely, N.: Robust global translations with 1DSfM. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8691, pp. 61–75. Springer, Cham (2014). Scholar
  34. 34.
    Wu, C., Agarwal, S., Curless, B., Seitz, S.M.: Multicore bundle adjustment. In: CVPR (2011)Google Scholar
  35. 35.
    Zach, C.: Robust bundle adjustment revisited. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8693, pp. 772–787. Springer, Cham (2014). Scholar
  36. 36.
    Zhang, R., Zhu, S., Fang, T., Quan, L.: Distributed very large scale bundle adjustment by global camera consensus. In: ICCV (2017)Google Scholar
  37. 37.
    Zhu, S., et al.: Parallel structure from motion from local increment to global averaging. arXiv preprint arXiv:1702.08601 (2017)
  38. 38.
    Zhu, S., et al.: Very large-scale global SfM by distributed motion averaging. In: CVPR (2018)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Hong Kong University of Science and TechnologyHong KongChina
  2. 2.Everest Innovation TechnologyHong KongChina

Personalised recommendations