Skip to main content
SpringerLink
Log in

A Reliable Online Method for Joint Estimation of Focal Length and Camera Rotation

  • Conference paper
  • First Online:
Computer Vision – ECCV 2022 (ECCV 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13661))

Included in the following conference series:

Abstract

Linear perspective cues deriving from regularities of the built environment can be used to recalibrate both intrinsic and extrinsic camera parameters online, but these estimates can be unreliable due to irregularities in the scene, uncertainties in line segment estimation and background clutter. Here we address this challenge through four initiatives. First, we use the PanoContext panoramic image dataset [27] to curate a novel and realistic dataset of planar projections over a broad range of scenes, focal lengths and camera poses. Second, we use this novel dataset and the YorkUrbanDB [4] to systematically evaluate the linear perspective deviation measures frequently found in the literature and show that the choice of deviation measure and likelihood model has a huge impact on reliability. Third, we use these findings to create a novel system for online camera calibration we call \(f\textbf{R}\), and show that it outperforms the prior state of the art, substantially reducing error in estimated camera rotation and focal length. Our fourth contribution is a novel and efficient approach to estimating uncertainty that can dramatically improve online reliability for performance-critical applications by strategically selecting which frames to use for recalibration.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://github.com/ElderLab-York-University/OnlinefR.

References

  1. Almazan, E.J., Tal, R., Qian, Y., Elder, J.H.: MCMLSD: a dynamic programming approach to line segment detection. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2031–2039 (2017)

    Google Scholar 

  2. Collins, R.T., Weiss, R.S.: Vanishing point calculation as a statistical inference on the unit sphere. In: International Conference on Computer Vision, vol. 90, pp. 400–403. Citeseer (1990)

    Google Scholar 

  3. Coughlan, J.M., Yuille, A.L.: Manhattan world: compass direction from a single image by Bayesian inference. In: International Conference on Computer Vision, vol. 2, pp. 941–947. IEEE (1999)

    Google Scholar 

  4. Denis, P., Elder, J.H., Estrada, F.J.: Efficient edge-based methods for estimating manhattan frames in urban imagery. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008. LNCS, vol. 5303, pp. 197–210. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-88688-4_15

    Chapter  Google Scholar 

  5. Deutscher, J., Isard, M., MacCormick, J.: Automatic camera calibration from a single manhattan image. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2353, pp. 175–188. Springer, Heidelberg (2002). https://doi.org/10.1007/3-540-47979-1_12

    Chapter  Google Scholar 

  6. Faugeras, O.: Three-Dimensional Computer Vision: A Geometric Viewpoint. MIT Press, Cambridge (1993)

    Google Scholar 

  7. von Gioi, R.G., Jakubowicz, J., Morel, J.M., Randall, G.: Lsd: a fast line segment detector with a false detection control. IEEE Trans. Pattern Anal. Mach. Intell. 4, 722–732 (2008)

    Google Scholar 

  8. Hold-Geoffroy, Y., et al.: A perceptual measure for deep single image camera calibration. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2354–2363 (2018)

    Google Scholar 

  9. Košecká, J., Zhang, W.: Video compass. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) European Conference on Computer Vision, pp. 476–490. Springer, Berlin Heidelberg, Berlin, Heidelberg (2002)

    Google Scholar 

  10. Lee, D.C., Hebert, M., Kanade, T.: Geometric reasoning for single image structure recovery. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2136–2143. IEEE (2009)

    Google Scholar 

  11. Lee, J., Go, H., Lee, H., Cho, S., Sung, M., Kim, J.: CTRL-C: camera calibration transformer with line-classification. In: International Conference on Computer Vision, pp. 16228–16237 (2021)

    Google Scholar 

  12. Lee, J., Sung, M., Lee, H., Kim, J.: Neural geometric parser for single image camera calibration. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12357, pp. 541–557. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58610-2_32

    Chapter  Google Scholar 

  13. Li, H., Chen, K., Kim, P., Yoon, K.J., Liu, Z., Joo, K., Liu, Y.H.: Learning icosahedral spherical probability map based on bingham mixture model for vanishing point estimation. In: International Conference on Computer Vision, pp. 5661–5670 (2021)

    Google Scholar 

  14. Li, H., Zhao, J., Bazin, J.C., Chen, W., Liu, Z., Liu, Y.H.: Quasi-globally optimal and efficient vanishing point estimation in Manhattan world. In: International Conference on Computer Vision, pp. 1646–1654 (2019)

    Google Scholar 

  15. Lin, Y., Pintea, S.L., van Gemert, J.C.: Deep hough-transform line priors. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12367, pp. 323–340. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58542-6_20

    Chapter  Google Scholar 

  16. Rother, C.: A new approach to vanishing point detection in architectural environments. Image Vis. Comput. 20(9–10), 647–655 (2002)

    Article  Google Scholar 

  17. Schindler, G., Dellaert, F.: Atlanta world: An expectation maximization framework for simultaneous low-level edge grouping and camera calibration in complex man-made environments. In: Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004. vol. 1, pp. I-I. IEEE (2004)

    Google Scholar 

  18. Simon, G., Fond, A., Berger, M.O.: A Simple and Effective Method to Detect Orthogonal Vanishing Points in Uncalibrated Images of Man-Made Environments. In: Bashford-Rogers, T., Santos, L.P. (eds.) EuroGraphics - Short Papers. The Eurographics Association (2016). https://doi.org/10.2312/egsh.20161008

  19. Tal, R., Elder, J.H.: An accurate method for line detection and manhattan frame estimation. In: Park, J.-I., Kim, J. (eds.) ACCV 2012. LNCS, vol. 7729, pp. 580–593. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-37484-5_47

    Chapter  Google Scholar 

  20. Tardif, J.P.: Non-iterative approach for fast and accurate vanishing point detection. In: IEEE International Conference on Computer Vision, pp. 1250–1257. IEEE (2009)

    Google Scholar 

  21. Wildenauer, H., Hanbury, A.: Robust camera self-calibration from monocular images of Manhattan worlds. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2831–2838. IEEE (2012)

    Google Scholar 

  22. Wu, Z., Radke, R.: Keeping a pan-tilt-zoom camera calibrated. Pattern Analysis and Machine Intelligence, IEEE Transactions on 35(8), 1994–2007 (2013). https://doi.org/10.1109/TPAMI.2012.250

    Article  Google Scholar 

  23. Xian, W., Li, Z., Fisher, M., Eisenmann, J., Shechtman, E., Snavely, N.: UprightNet: geometry-aware camera orientation estimation from single images. In: International Conference on Computer Vision, pp. 9974–9983 (2019)

    Google Scholar 

  24. Xiao, J., Ehinger, K.A., Oliva, A., Torralba, A.: Recognizing scene viewpoint using panoramic place representation. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2695–2702. IEEE (2012)

    Google Scholar 

  25. Xu, Y., Oh, S., Hoogs, A.: A minimum error vanishing point detection approach for uncalibrated monocular images of man-made environments. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1376–1383 (2013)

    Google Scholar 

  26. Xu, Z., Shin, B., Klette, R.: A statistical method for line segment detection. Comput. Vis. Image Underst. 138, 61–73 (2015)

    Article  Google Scholar 

  27. Zhang, Y., Song, S., Tan, P., Xiao, J.: PanoContext: a whole-room 3D context model for panoramic scene, pp. 668–686 (2014)

    Google Scholar 

Download references

Acknowledgement

This work was supported by the University of Toronto, NSERC, the Ontario Research Fund, the York University VISTA and Research Chair programs (Canada), and the Agency for Science, Technology and Research (A*STAR) under its RIE2020 Health and Biomedical Sciences (HBMS) Industry Alignment Fund Pre-Positioning (IAF-PP) Grant No. H20c6a0031 and AI3 HTPO Seed Fund (C211118014) (Singapore).

Author information

Authors and Affiliations

  1. A*star, Institute of High Performance Computing, Singapore, Singapore

    Yiming Qian

  2. Centre for Vision Research, York University, Toronto, Canada

    James H. Elder

Authors
  1. Yiming Qian
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. James H. Elder
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to James H. Elder .

Editor information

Editors and Affiliations

  1. Tel Aviv University, Tel Aviv, Israel

    Shai Avidan

  2. University College London, London, UK

    Gabriel Brostow

  3. Google AI, Accra, Ghana

    Moustapha Cissé

  4. University of Catania, Catania, Italy

    Giovanni Maria Farinella

  5. Facebook (United States), Menlo Park, CA, USA

    Tal Hassner

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 3260 KB)

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Qian, Y., Elder, J.H. (2022). A Reliable Online Method for Joint Estimation of Focal Length and Camera Rotation. In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T. (eds) Computer Vision – ECCV 2022. ECCV 2022. Lecture Notes in Computer Science, vol 13661. Springer, Cham. https://doi.org/10.1007/978-3-031-19769-7_15

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-19769-7_15

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-19768-0

  • Online ISBN: 978-3-031-19769-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation