Advertisement

Calibration-Free Structure-from-Motion with Calibrated Radial Trifocal Tensors

Conference paper
  • 755 Downloads
Part of the Lecture Notes in Computer Science book series (LNCS, volume 12350)

Abstract

In this paper we consider the problem of Structure-from-Motion from images with unknown intrinsic calibration. Instead of estimating the internal camera parameters through some self-calibration procedure, we propose to use a subset of the reprojection constraints that is invariant to radial displacement. This allows us to recover metric 3D reconstructions without explicitly estimating the cameras’ focal length or radial distortion parameters. The weaker projection model makes initializing the reconstruction especially difficult. To handle this additional challenge we propose two novel minimal solvers for radial trifocal tensor estimation. We evaluate our approach on real images and show that even for extreme optical systems, such as fisheye or catadioptric, we are able to get accurate reconstructions without performing any calibration.

Notes

Acknowledgements

Viktor Larsson was supported by an ETH Zurich Postdoctoral Fellowship.

Supplementary material

504441_1_En_23_MOESM1_ESM.zip (90.7 mb)
Supplementary material 1 (zip 92894 KB)

References

  1. 1.
    Agarwal, S., Mierle, K., Others: Ceres solver. http://ceres-solver.org
  2. 2.
    Angst, R., Zach, C., Pollefeys, M.: The generalized trace-norm and its application to structure-from-motion problems. In: International Conference on Computer Vision (ICCV) (2011)Google Scholar
  3. 3.
    Aranda, M., López-Nicolás, G., Sagüés, C.: Omnidirectional visual homing using the 1D trifocal tensor. In: International Conference on Robotics and Automation (ICRA) (2010)Google Scholar
  4. 4.
    Åström, K., Oskarsson, M.: Solutions and ambiguities of the structure and motion problem for 1D retinal vision. J. Math. Imaging Vis. (JMIV) 12(2), 121–135 (2000)MathSciNetzbMATHCrossRefGoogle Scholar
  5. 5.
    Camposeco, F., Sattler, T., Pollefeys, M.: Non-parametric structure-based calibration of radially symmetric cameras. In: International Conference on Computer Vision (ICCV) (2015)Google Scholar
  6. 6.
    Chandraker, M., Agarwal, S., Kahl, F., Nistér, D., Kriegman, D.: Autocalibration via rank-constrained estimation of the absolute quadric. In: Computer Vision and Pattern Recognition (CVPR) (2007)Google Scholar
  7. 7.
    Chatterjee, A., Govindu, V.M.: Robust relative rotation averaging. Trans. Pattern Anal. Mach. Intell. (PAMI) 40(4), 958–972 (2017)CrossRefGoogle Scholar
  8. 8.
    Cui, H., Gao, X., Shen, S., Hu, Z.: HSFM: hybrid structure-from-motion. In: Computer Vision and Pattern Recognition (CVPR) (2017)Google Scholar
  9. 9.
    Cui, Z., Tan, P.: Global structure-from-motion by similarity averaging. In: International Conference on Computer Vision (ICCV) (2015)Google Scholar
  10. 10.
    Dai, Y., Li, H., He, M.: Projective multiview structure and motion from element-wise factorization. Trans. Pattern Anal. Mach. Intell. (PAMI) 35(9), 2238–2251 (2013)Google Scholar
  11. 11.
    Dellaert, F., Stroupe, A.W.: Linear 2D localization and mapping for single and multiple robot scenarios. In: International Conference on Robotics and Automation (ICRA) (2002)Google Scholar
  12. 12.
    Demazure, M.: Sur deux problemes de reconstruction. Technical report, RR-0882, INRIA, July 1988. https://hal.inria.fr/inria-00075672
  13. 13.
    Duff, T., Kohn, K., Leykin, A., Pajdla, T.: PLMP-point-line minimal problems in complete multi-view visibility. In: International Conference on Computer Vision (ICCV) (2019)Google Scholar
  14. 14.
    Eriksson, A., Olsson, C., Kahl, F., Chin, T.J.: Rotation averaging and strong duality. In: Computer Vision and Pattern Recognition (CVPR) (2018)Google Scholar
  15. 15.
    Fabbri, R., et al.: Trifocal relative pose from lines at points and its efficient solution. arXiv preprint arXiv:1903.09755 (2019)
  16. 16.
    Faugeras, O., Quan, L., Strum, P.: Self-calibration of a 1D projective camera and its application to the self-calibration of a 2D projective camera. Trans. Pattern Anal. Mach. Intell. (PAMI) 22(10), 1179–1185 (2000)Google Scholar
  17. 17.
    Hartley, R., Schaffalitzky, F.: Reconstruction from projections using grassmann tensors. Int. J. Comput. Vis. (IJCV) 83(3), 274–293 (2009)zbMATHCrossRefGoogle Scholar
  18. 18.
    Hartley, R., Trumpf, J., Dai, Y., Li, H.: Rotation averaging. Int. J. Comput. Vis. (IJCV) 103(3), 267–305 (2013)MathSciNetzbMATHCrossRefGoogle Scholar
  19. 19.
    Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press, Cambridge (2003)zbMATHGoogle Scholar
  20. 20.
    Hong, J.H., Zach, C., Fitzgibbon, A., Cipolla, R.: Projective bundle adjustment from arbitrary initialization using the variable projection method. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 477–493. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-46448-0_29CrossRefGoogle Scholar
  21. 21.
    Hyeong Hong, J., Zach, C.: pose: Pseudo object space error for initialization-free bundle adjustment. In: Computer Vision and Pattern Recognition (CVPR) (2018)Google Scholar
  22. 22.
    Kileel, J.: Minimal problems for the calibrated trifocal variety. SIAM J. Appl. Algebra Geom. 1(1), 575–598 (2017)MathSciNetzbMATHGoogle Scholar
  23. 23.
    Kim, J.H., Dai, Y., Li, H., Du, X., Kim, J.: Multi-view 3D reconstruction from uncalibrated radially-symmetric cameras. In: International Conference on Computer Vision (ICCV) (2013)Google Scholar
  24. 24.
    Kozuka, K., Sato, J.: Multiple view geometry for mixed dimensional cameras. In: International Conference on Computer Vision Theory and Applications (VISAPP) (2008)Google Scholar
  25. 25.
    Kukelova, Z., Bujnak, M., Pajdla, T.: Real-time solution to the absolute pose problem with unknown radial distortion and focal length. In: International Conference on Computer Vision (ICCV) (2013)Google Scholar
  26. 26.
    Kukelova, Z., Heller, J., Bujnak, M., Fitzgibbon, A., Pajdla, T.: Efficient solution to the epipolar geometry for radially distorted cameras. In: International Conference on Computer Vision (ICCV) (2015)Google Scholar
  27. 27.
    Larsson, V., Astrom, K., Oskarsson, M.: Efficient solvers for minimal problems by syzygy-based reduction. In: Computer Vision and Pattern Recognition (CVPR) (2017)Google Scholar
  28. 28.
    Larsson, V., Kukelova, Z., Zheng, Y.: Making minimal solvers for absolute pose estimation compact and robust. In: International Conference on Computer Vision (ICCV) (2017)Google Scholar
  29. 29.
    Larsson, V., Kukelova, Z., Zheng, Y.: Camera pose estimation with unknown principal point. In: Computer Vision and Pattern Recognition (CVPR) (2018)Google Scholar
  30. 30.
    Larsson, V., Sattler, T., Kukelova, Z., Pollefeys, M.: Revisiting radial distortion absolute pose. In: International Conference on Computer Vision (ICCV) (2019)Google Scholar
  31. 31.
    Lebeda, K., Matas, J., Chum, O.: Fixing the locally optimized RANSAC. In: British Machine Vision Conference (BMVC) (2012)Google Scholar
  32. 32.
    Leonardos, S., Tron, R., Daniilidis, K.: A metric parametrization for trifocal tensors with non-colinear pinholes. In: Computer Vision and Pattern Recognition (CVPR) (2015)Google Scholar
  33. 33.
    Locher, A., Havlena, M., Van Gool, L.: Progressive structure from motion. In: European Conference on Computer Vision (ECCV) (2018)Google Scholar
  34. 34.
    Magerand, L., Del Bue, A.: Revisiting projective structure for motion: a robust and efficient incremental solution. Trans. Pattern Anal. Mach. Intell. (PAMI) 42(2), 430–443 (2018)Google Scholar
  35. 35.
    Martyushev, E.: On some properties of calibrated trifocal tensors. J. Math. Imaging Vis. (JMIV) 58(2), 321–332 (2017)MathSciNetzbMATHGoogle Scholar
  36. 36.
    Molana, R., Daniilidis, K.: A single-perspective novel panoramic view from radially distorted non-central images. In: British Machine Vision Conference (BMVC) (2007)Google Scholar
  37. 37.
    Moulon, P., Monasse, P., Marlet, R.: Global fusion of relative motions for robust, accurate and scalable structure from motion. In: International Conference on Computer Vision (ICCV) (2013)Google Scholar
  38. 38.
    Olsson, C., Enqvist, O.: Stable structure from motion for unordered image collections. In: Scandinavian Conference on Image Analysis (SCIA) (2011)Google Scholar
  39. 39.
    Quan, L., Kanade, T.: Affine structure from line correspondences with uncalibrated affine cameras. Trans. Pattern Anal. Mach. Intell. (PAMI) 19(8), 834–845 (1997)Google Scholar
  40. 40.
    Sagues, C., Murillo, A., Guerrero, J.J., Goedemé, T., Tuytelaars, T., Van Gool, L.: Localization with omnidirectional images using the radial trifocal tensor. In: International Conference on Robotics and Automation (ICRA) (2006)Google Scholar
  41. 41.
    Sattler, T., Sweeney, C., Pollefeys, M.: On sampling focal length values to solve the absolute pose problem. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8692, pp. 828–843. Springer, Cham (2014).  https://doi.org/10.1007/978-3-319-10593-2_54CrossRefGoogle Scholar
  42. 42.
    Schonberger, J.L., Frahm, J.M.: Structure-from-motion revisited. In: Computer Vision and Pattern Recognition (CVPR) (2016)Google Scholar
  43. 43.
    Schönberger, J.L., Zheng, E., Frahm, J.-M., Pollefeys, M.: Pixelwise view selection for unstructured multi-view stereo. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9907, pp. 501–518. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-46487-9_31CrossRefGoogle Scholar
  44. 44.
    Schops, T., Larsson, V., Pollefeys, M., Sattler, T.: Why having 10,000 parameters in your camera model is better than twelve. In: Computer Vision and Pattern Recognition (CVPR) (2020)Google Scholar
  45. 45.
    Snavely, N., Seitz, S.M., Szeliski, R.: Modeling the world from internet photo collections. Int. J. Comput. Vis. (IJCV) 80(2), 189–210 (2008)Google Scholar
  46. 46.
    Sweeney, C., Holynski, A., Curless, B., Seitz, S.M.: Structure from motion for panorama-style videos. arXiv preprint arXiv:1906.03539 (2019)
  47. 47.
    Sweeney, C., Sattler, T., Hollerer, T., Turk, M., Pollefeys, M.: Optimizing the viewing graph for structure-from-motion. In: International Conference on Computer Vision (ICCV) (2015)Google Scholar
  48. 48.
    Thirthala, S., Pollefeys, M.: Radial multi-focal tensors. Int. J. Comput. Vis. (IJCV) 96(2), 195–211 (2012)zbMATHGoogle Scholar
  49. 49.
    Triggs, B.: Matching constraints and the joint image. In: International Conference on Computer Vision (ICCV) (1995)Google Scholar
  50. 50.
    Triggs, B.: Factorization methods for projective structure and motion. In: Computer Vision and Pattern Recognition (CVPR) (1996)Google Scholar
  51. 51.
    Tsai, R.: A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses. J. Robot. Autom. 3(4), 323–344 (1987)Google Scholar
  52. 52.
    Ventura, J.: Structure from motion on a sphere. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9907, pp. 53–68. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-46487-9_4CrossRefGoogle Scholar
  53. 53.
    Wu, C.: Towards linear-time incremental structure from motion. In: International Conference on 3D Vision (3DV) (2013)Google Scholar
  54. 54.
    Wu, C.: P3.5P: pose estimation with unknown focal length. In: Computer Vision and Pattern Recognition (CVPR) (2015)Google Scholar
  55. 55.
    Zhang, Z., et al.: Flexible camera calibration by viewing a plane from unknown orientations. In: International Conference on Computer Vision (ICCV) (1999)Google Scholar
  56. 56.
    Zhu, S., et al.: Very large-scale global SFM by distributed motion averaging. In: Computer Vision and Pattern Recognition (CVPR) (2018)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Department of Computer ScienceETH ZürichZürichSwitzerland
  2. 2.Department of Information Technology and Electrical EngineeringETH ZürichZürichSwitzerland
  3. 3.KTH Royal Institute of TechnologyStockholmSweden
  4. 4.Microsoft Mixed Reality & AI Zurich LabZürichSwitzerland

Personalised recommendations