Minimal Solvers for Rectifying from Radially-Distorted Scales and Change of Scales

Abstract

This paper introduces the first minimal solvers that jointly estimate lens distortion and affine rectification from the image of rigidly-transformed coplanar features. The solvers work on scenes without straight lines and, in general, relax strong assumptions about scene content made by the state of the art. The proposed solvers use the affine invariant that coplanar repeats have the same scale in rectified space. The solvers are separated into two groups that differ by how the equal scale invariant of rectified space is used to place constraints on the lens undistortion and rectification parameters. We demonstrate a principled approach for generating stable minimal solvers by the Gröbner basis method, which is accomplished by sampling feasible monomial bases to maximize numerical stability. Synthetic and real-image experiments confirm that the proposed solvers demonstrate superior robustness to noise compared to the state of the art. Accurate rectifications on imagery taken with narrow to fisheye field-of-view lenses demonstrate the wide applicability of the proposed method. The method is fully automatic.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

References

  1. Ahmad, S., & Cheong, L. F. (2018). Robust detection and affine rectification of planar homogeneous texture for scene understanding. International Journal of Computer Vision, 126, 1–33.

    Article  Google Scholar 

  2. Aiger, D., Cohen-Or, D., & Mitra, N. J. (2012). Repetition maximization based texture rectification. Computer Graphics Forum, 31(2pt2), 439–448.

    Article  Google Scholar 

  3. Antunes, M., Barreto, J. P., Aouada, D., & Ottersten, B. (2017). Unsupervised vanishing point detection and camera calibration from a single manhattan image with radial distortion. In CVPR.

  4. Arandjelović, R., & Zisserman, A. (2012). Three things everyone should know to improve object retrieval. In CVPR.

  5. Barath, D., & Hajder, L. (2017). A theory of point-wise homography estimation. Pattern Recognition Letters, 94, 7–14.

    Article  Google Scholar 

  6. Brown, D. C. (1966). Decentering distortion of lenses. Photometric Engineering, 32(3), 444–462.

    Google Scholar 

  7. Bukhari, F., & Dailey, M. N. (2013). Automatic radial distortion estimation from a single image. Journal of Mathematical Imaging and Vision, 45(1), 31–45.

    MathSciNet  Article  Google Scholar 

  8. Camposeco, F., Cohen, A., Pollefeys, M., & Sattler, T. (2018). Hybrid camera pose estimation. In CVPR.

  9. Chum, O., & Matas, J. (2010). Planar affine rectification from change of scale. In ACCV.

  10. Chum, O., Matas, J., & Obdržálek, V. (2004). Enhancing RANSAC by generalized model optimization. In ACCV.

  11. Conrady, A. (1919). Decentering lens systems. Monthly Notices of the Royal Astronomical Society, 79, 384–390.

    Article  Google Scholar 

  12. Criminisi, A., & Zisserman, A. (2000). Shape from texture: Homogeneity revisited. In BMVC.

  13. Devernay, F., & Faugeras, O. (2001). Straight lines have to be straight. Machine Vision and Applications, 13(1), 14–24.

    Article  Google Scholar 

  14. Fischler, M. A., & Bolles, R. C. (1981). Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 24(6), 381–395.

    MathSciNet  Article  Google Scholar 

  15. Fitzgibbon, A. W. (2001). Simultaneous linear estimation of multiple view geometry and lens distortion. In CVPR.

  16. Funk, C., Lee, S., Oswald, M. R., Tsogkas, S., Shen, W., Cohen, A., Dickinson, S., & Liu, Y. (2017). 2017 ICCV challenge: Detecting symmetry in the wild. In ICCV workshop.

  17. Hartley, R. I., & Zisserman, A. W. (2004). Multiple view geometry in computer vision (2nd ed.). Cambridge: Cambridge University Press.

    Book  Google Scholar 

  18. Köser, K., & Koch, R. (2008). Differential spatial resection-pose estimation using a single local image feature. In ECCV.

  19. Köser, K., Beder, C., & Koch, R. (2008). Conjugate rotation: Parameterization and estimation from an affine feature correspondence. In CVPR.

  20. Kuang, Y., & Åström, K. (2012). Numerically stable optimization of polynomial solvers for minimal problems. In ECCV.

  21. Kukelova, Z., Bujnak, M., & Pajdla, T. (2008). Automatic generator of minimal problem solvers. In ECCV.

  22. Kukelova, Z., Heller, J., Bujnak, M., & Pajdla, T. (2015). Radial distortion homography. In CVPR.

  23. Larsson, V., Åström, K., & Oskarsson, M. (2017a). Efficient solvers for minimal problems by syzygy-based reduction. In CVPR.

  24. Larsson, V., Åström, K., & Oskarsson, M. (2017b). Polynomial solvers for saturated ideals. In ICCV.

  25. Larsson, V., Oskarsson, M., Astrom, K., Wallis, A., Kukelova, Z., & Pajdla, T. (2018). Beyond grobner bases: Basis selection for minimal solvers. In CVPR.

  26. Lowe, D. G. (2004). Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 60(2), 91–110.

    Article  Google Scholar 

  27. Lukáč, M., Sýkora, D., Sunkavalli, K., Shechtman, E., Jamriška, O., Carr, N., et al. (2017). Nautilus: Recovering regional symmetry transformations for image editing. ACM Transactions on Graphics, 36(4), 108:1–108:11.

    Article  Google Scholar 

  28. Matas, J., Chum, O., Urban, M., & Pajdla, T. (2002). Robust wide baseline stereo from maximally stable extremal regions. In BMVC.

  29. Mikolajczyk, K., & Schmid, C. (2004). Scale & affine invariant interest point detectors. International Journal of Computer Vision, 60(1), 63–86.

    Article  Google Scholar 

  30. Mikolajczyk, K., & Schmid, C. (2005). A performance evaluation of local descriptors. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(10), 1615–1630.

    Article  Google Scholar 

  31. Mishkin, D., Radenovic, F., & Matas, J. (2018). Repeatability is not enough: Learning affine regions via discriminability. In Proceedings of ECCV.

  32. Obdržálek, S., & Matas, J. (2002). Object recognition using local affine frames on distinguished regions. In BMVC.

  33. Ohta, T., Maenobu, K., & Sakai, T. (1981). Obtaining surface orientation from texels under perspective projection. In IJCAI.

  34. Pritts, J., Chum, O., & Matas, J. (2014). Detection, rectification and segmentation of coplanar repeated patterns. In CVPR.

  35. Pritts, J., Kukelova, Z., Larsson, V., & Chum, O. (2018). Radially-distorted conjugate translations. In CVPR.

  36. Pritts, J., Rozumnyi, D., Kumar, M. P., & Chum, O. (2016). Coplanar repeats by energy minimization. In BMVC.

  37. Strand, R., & Hayman, E. (2005). Correcting radial distortion by circle fitting. In BMVC.

  38. Vedaldi, A., & Fulkerson, B. (2008). VLFeat: An open and portable library of computer vision algorithms. In Proceedings of the 18th ACM international conference on Multimedia (pp. 1469–1472).

  39. Wang, A., Qiu, T., & Shao, L. (2009). A simple method of radial distortion correction with centre of distortion estimation. Journal of Mathematical Imaging and Vision, 35(3), 165–172.

    MathSciNet  Article  Google Scholar 

  40. Wildenauer, H., & Micusík, B. (2013). Closed form solution for radial distortion estimation from a single vanishing point. In BMVC.

  41. Wu, C., Frahm, J. M., & Pollefeys, M. (2011). Repetition-based dense single-view reconstruction. In CVPR.

  42. Zhang, Z., Ganesh, A., Liang, X., & Ma, Y. (2012). TILT: Transform invariant low-rank textures. International Journal of Computer Vision, 99(1), 1–24.

    MathSciNet  Article  Google Scholar 

Download references

Acknowledgements

James Pritts acknowledges the European Regional Development Fund under the project Robotics for Industry 4.0 (Reg. No. CZ.02.1.01/0.0/0.0/15_003/0000470) and Grant SGS17/185/OHK3/3T/13; Zuzana Kukelova the ESI Fund, OP RDE programme under the project International Mobility of Researchers MSCA-IF at CTU No. CZ.02.2.69/0.0/0.0/17_050/0008025; and Ondřej Chum Grant OP VVV funded project CZ.02.1.01/0.0/0.0/16_019/0000765 “Research Center for Informatics”. Viktor Larsson received funding from the ETH Zurich Postdoctoral Fellowship program and the Marie Sklodowska-Curie Actions COFUND program. Yaroslava Lochman was also supported by Robotics for Industry 4.0 as well as ELEKS Ltd.

Author information

Affiliations

Authors

Corresponding author

Correspondence to James Pritts.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Communicated by C.V. Jawahar, Hongdong Li, Greg Mori, Konrad Schindler.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Pritts, J., Kukelova, Z., Larsson, V. et al. Minimal Solvers for Rectifying from Radially-Distorted Scales and Change of Scales. Int J Comput Vis 128, 950–968 (2020). https://doi.org/10.1007/s11263-019-01216-x

Download citation

Keywords

  • Rectification
  • Radial lens distortion
  • Minimal solvers
  • Repeated patterns
  • Symmetry
  • Local features