Skip to main content

Fast and Accurate Self-calibration Using Vanishing Point Detection in Manmade Environments

Abstract

Interests of auto-calibration have been increased in several camera systems. This paper presents a novel self-calibration method using fast and accurate vanishing point detection algorithm that works in manmade environments. The proposed algorithm estimates focal length assuming that the principal point is the center of an image to satisfy the orthogonality of three vanishing points. By using proposed vanishing point detection algorithm and minimization of the proposed objective function, the proposed system detects accurate vanishing points with focal length outperforming other methods. The proposed vanishing point detection algorithm detects vanishing points by using J-linkage based method that is more delicate by fragmentation and re-merging strategies. The proposed objective function finally detects vanishing points that meets orthogonality among estimated hypotheses for vanishing points by checking several geometric relationships. We believe that the proposed method can be used for automatic camera calibration, localization of a camera in an autonomous navigation system, and three-dimensional reconstruction of a single-view image.

This is a preview of subscription content, access via your institution.

References

  1. [1]

    S. Hutchinson, G. D. Hager, and P. I. Corke, “A tutorial on visual servo control,” IEEE Transactions on Robotics and Automation, vol. 12, no. 5, pp. 651–670, 1996.

    Article  Google Scholar 

  2. [2]

    G. Zhang and I. H. Suh, “A vertical and floor line-based monocular SLAM system for corridor environments,” International Journal of Control, Automation, and Systems, vol. 10, no. 3, pp. 547–557, 2012.

    Article  Google Scholar 

  3. [3]

    V. Hoang and K. Jo, “Automatic calibration of camera and LRF based on morphological pattern and optimal angular back-projection error,” International Journal of Control, Automation, and Systems, vol. 13, no. 6, pp. 1436–1445, 2015.

    Article  Google Scholar 

  4. [4]

    Y. Shin, J.-H. Park, J.-H. Bae, and M.-H. Baeg, “A study on reliability enhancement for laser and camera calibration,” International Journal of Control, Automation, and Systems, vol. 10, no. 1, pp. 109–116, 2012.

    Article  Google Scholar 

  5. [5]

    R. Hartley and A. Zisserman, Multiple View Geometry n Computer Vision, Cambridge University Press, 2nd ed, New York, NY, USA, 2003.

    MATH  Google Scholar 

  6. [6]

    J. M. Coughlan and A. L. Yuille, “The Manhattan world assumption: regularities in scene statistics which enable Bayesian inference,” Proc. NIPS, Denver, CO, USA, pp.809–815, 2000.

    Google Scholar 

  7. [7]

    S. T. Barnard, “Interpreting perspective images,” Artificial Intelligence, vol. 21, no. 4, pp. 435–462, Nov. 1983.

    Article  Google Scholar 

  8. [8]

    J. P. Tardif, “Non-iterative approach for fast and accurate vanishing point detection,” Proc. of 12th IEEE Conf. International Conference on Computer Vision, Kyoto, Japan, pp. 1250–1257, 2009.

    Google Scholar 

  9. [9]

    O. Barinova, V. Lempitsky, E. Tretiak, and P. Kohli, “Geometric image parsing in man-made environments,” Proc. of European Conference on Computer Vision, Springer, Berlin, Heidelberg, Germany, pp. 57–70, 2010.

    Google Scholar 

  10. [10]

    H. Wildenauer and A. Hanbury, “Robust camera self-calibration from monocular images of Manhattan worlds,” Proc. of IEEE Conf. on Computer Vision and Pattern Recognition, pp. 2831–2838, 2012.

    Google Scholar 

  11. [11]

    R. Toldo and A. Fusiello, “Robust multiple structures estimation with j-linkage,” Proc. of European Conference on Computer Vision, Berlin, Heidelberg, Germany, pp. 537–547, 2008.

    Google Scholar 

  12. [12]

    J. K. Jana and W. Zhang, “Video compass,” Proc. of European Conference on Computer Vision, Berlin, Heidelberg, Germany, pp. 476–490, 2002.

    Google Scholar 

  13. [13]

    M. E. Antone and S. Teller, “Automatic recovery of relative camera rotations for urban scenes,” Proc. of IEEE Conf on Computer Vision and Pattern Recognition, Hilton Head, SC, USA, pp. 282–289, 2000.

    Google Scholar 

  14. [14]

    Y. Xu, S. Oh, and A. Hoogs, “A minimum error vanishing point detection approach for uncalibrated monocular images of man-made environments,” Proc. of IEEE Conf. on IEEE, Computer Vision and Pattern Recognition (CVPR), Portland, Oregon, USA, pp. 1376–1383, 2013.

    Google Scholar 

  15. [15]

    G. Nebehay and R. Pflugfelder, “A self-calibration method for smart video cameras,” Proc. of 12th IEEE International Conference on Computer Vision, Kyoto, Japan, 2009.

    Google Scholar 

  16. [16]

    M. A. Fischler and R. C. Bolles, “Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography,” Comm. ACM, vol. 24, no. 6, pp. 381–395, Jun. 1981.

  17. [17]

    S. J. Lee and S. S. Hwang, “Fast and robust vanishing point detection on uncalibrated images,” Proc. of IEEE International Conference on Image Processing, Athens, Greece, 2018.

    Google Scholar 

  18. [18]

    J. C. Bazin, Y. Seo, C. Demonceaux, P. Vasseur, K. Ikeuchi, I. Kweon, and M. Pollefeys, “Globally optimal line clustering and vanishing point estimation in manhattan world,” Proc. of IEEE Conf. on Computer Vision and Pattern Recognition, Rhode Island, USA, pp. 638–645, 2012.

    Google Scholar 

  19. [19]

    R. G. Von Gioi, J. Jakubowicz, J.-M. Morel, G. Randall, “LSD: a line segment detector,” Image Processing On Line, vol. 2, pp. 35–55, Mar. 2012.

    Article  Google Scholar 

  20. [20]

    R. Fletcher, Practical Methods of Optimization, 2nd ed., John Wiley & Sons, New York, Jul. 2000.

    Book  Google Scholar 

  21. [21]

    A. V. Lazo and P. Rathie, “On the entropy of continuous probability distributions,” IEEE Transactions on Information Theory, vol. 24, no. 1, pp. 120–122, Jan. 1978.

    MathSciNet  Article  Google Scholar 

  22. [22]

    E. A. Nadaraya, “On estimating regression,” Theory of Probability and its Applications, vol. 9, no. 1, pp. 141–142, Sep. 1963.

    Article  Google Scholar 

  23. [23]

    S. J. Lee and S. S. Hwang, “A robust line merge method using Gaussian mapping,” Proc. of Conf. Image Processing and Image Understanding, Jeju, Korea, 2018.

    Google Scholar 

  24. [24]

    T. H. Cormen, C. E. Leiserson, R. L. Rivest, and C. Stein, Introduction to Algorithms, 2nd ed., MIT Press and McGraw-Hill, Cambridge, Massachusetts, Sec. 22.5, pp. 552–557, USA, 2001.

    MATH  Google Scholar 

  25. [25]

    P. S. Mann, Introductory Statistics, 9th ed., John Wiley & Sons, Hoboken, New Jersey, USA, 2016.

    MATH  Google Scholar 

  26. [26]

    B. Caprile and V. Torre, “Using vanishing points for camera calibration,” International Journal of Computer Vision, vol. 4, no. 2, pp. 127–139, Mar. 1990.

  27. [27]

    W. Feller, An Introduction to Probability Theory and Its Applications, 2nd ed., John Wiley & Sons Inc., New York, pp. 704, Jan. 1991.

    Google Scholar 

  28. [28]

    J. C. Bazin, C. Demonceaux, P. Vasseur, and I. Kweon, “Rotation estimation and vanishing point extraction by omnidirectional vision in urban environment,” The International Journal of Robotics Research, vol. 31, no. 1, pp. 63–81, Jan. 2012.

    Article  Google Scholar 

  29. [29]

    R. Fletcher, Practical Methods of Optimization, 2nd ed., John Wiley & Sons, New York, Jul. 2000.

    Book  Google Scholar 

  30. [30]

    S. Lee, J. Kim, J. S. Yoon, S. Shin, O. Bailo, N. Kim, T.-H. Lee, H. S. Hong, S.-H. Han, and I. S. Kweon, “VPGNet: Vanishing point guided network for lane and road marking detection and recognition,” Proc. of IEEE International Conference on Computer Vision (ICCV), pp. 1965–1973, 2017.

    Google Scholar 

  31. [31]

    S. B. Aruoba and J. F. Villaverde, “A comparison of programming languages in macroeconomics,” Journal of Economic Dynamics and Control, vol. 58, pp. 265–273, Sep. 2015.

    MathSciNet  Article  Google Scholar 

  32. [32]

    K. Sirisantisamrid, K. Tirasesth, and T. Matsuura, “A determination method of initial camera parameters for coplanar calibration,” International Journal of Control, Automation, and Systems, vol. 7, no. 5, pp. 777–787, 2009

    Article  Google Scholar 

  33. [33]

    D. Kang, J. Ha, and M. Jeong, “Detection of calibration patterns for camera calibration with irregular lighting and complicated backgrounds,” International Journal of Control, Automation, and Systems, vol. 6, no. 5, pp. 746–754, 2008.

    Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Sung Soo Hwang.

Additional information

Publisher’s Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Recommended by Associate Editor DaeEun Kim under the direction of Editor Euntai Kim. This journal was supported by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education [No. 2016R1D1A3B03934808].

Sang Jun Lee received his B.S. degree in Computer Science and Engineering from Handong Global University, Pohang-si, Korea, in 2017. He is currently pursuing an M.S. degree in the Dept. of Information Technology at the Handong Global University. His research interests include the SLAM system for the localization of self-driving cars, robotics, or augmented reality, and optimization of these technologies using machine learning.

Sung Soo Hwang received his B.S. degree in Electrical Engineering and Computer Science from Handong Global Unveristy, Pohang, Korea in 2008, and his M.S and a Ph.D. degrees in Korea Advanced Institute of Science and Technology, Daejeon, Korea, in 2010 and 2015, respectively. From 2015 to now, he has been an Assistant Professor with School of Computer Science and Electrical Engineering, Handong Global University, Pohang, Korea. His research interests include image-based 3D modeling, 3D data compression, augmented reality.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Lee, S.J., Hwang, S.S. Fast and Accurate Self-calibration Using Vanishing Point Detection in Manmade Environments. Int. J. Control Autom. Syst. 18, 2609–2620 (2020). https://doi.org/10.1007/s12555-019-0284-1

Download citation

Keywords

  • Auto camera calibration
  • line clustering
  • single view geometry
  • vanishing point detection