Abstract
In this paper, we study the effects of windshield refraction for autonomous driving applications. These distortion effects are surprisingly large and can not be explained by traditional camera models. Instead of using a generalized camera approach, we propose a novel approach to jointly optimize a traditional camera model, and a mathematical representation of the windshield’s surface. First, using the laws of geometric optics, the refraction is modeled using a local spherical approximation to the windshield’s geometry. Next, a spline-based model is proposed as a refinement to better adapt to deviations from the ideal shape and manufacturing variations. By jointly optimizing refraction and camera parameters, the projection error can be significantly reduced. The proposed models are validated on real windshield observations and custom setups to compare recordings with and without windshield, with accurate laser scan measurements as 3D ground truth.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Agarwal, S., Mierle, K., Others: Ceres solver. http://ceres-solver.org
Agrawal, A., Ramalingam, S., Taguchi, Y., Chari, V.: A theory of multilayer flat refractive geometry. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3346–3353. IEEE (2012) (2012)
Chari, V., Sturm, P.: Multiple-view geometry of the refractive plane. In: 20th British Machine Vision Conference, pp. 1–11 (2009). In BMVA
Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981)
Gregson, J., Krimerman, M., Hullin, M.B., Heidrich, W.: Stochastic tomography and its applications in 3D imaging of mixing fluids. ACM Trans. Graph. (Proc. SIGGRAPH 2012) 31(4), 52:1–52:10 (2012)
Grossberg, M.D., Nayar, S.K.: The raxel imaging model and ray-based calibration. Int. J. Comput. Vis. 61(2), 119–137 (2005)
Hanel, A., Hoegner, L., Stilla, U.: Towards the influence of a car windshield on depth calculation with a stereo camera system. In: ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences XLI-B5, pp. 461–468 (2016)
Hartley, R.I., Zisserman, A.: Multiple View Geometry in Computer Vision, Second edn. Cambridge University Press, Cambridge (2004)
Huber, P.J.: Robust Statistic. In: Lovric, M. (ed.) International Encyclopedia of Statistical Science, pp. 1248-1251. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-04898-2_594
Kukelova, Z., Bujnak, M., Pajdla, T.: Real-time solution to the absolute pose problem with unknown radial distortion and focal length. In: Proceedings of the 2013 IEEE International Conference on Computer Vision (ICCV 2013), Washington, DC, USA, pp. 2816–2823 (2013)
Kunz, C., Singh, H.: Hemispherical refraction and camera calibration in underwater vision. In: OCEANS2008, pp. 1–7 (2008)
Miraldo, P., Araujo, H.: Calibration of smooth camera models. IEEE Trans. Pattern Anal. Mach. Intell. 35(9), 2091–2103 (2013)
Narasimhan, S., Nayar, S., Sun, B., Koppal, S.: Structured light in scattering media. In: Proceedings of ICCV, pp. 420–427 (2005)
Nishimura, M., Nobuhara, S., Matsuyama, T., Shimizu, S., Fujii, K.: A linear generalized camera calibration from three intersecting reference planes. In: Proceedings of ICCV, pp. 2354–2362 (2015)
Pável, S., Sándor, C., Csató, L.: Distortion estimation through explicit modeling of the refractive surface. In: Tetko, I.V., Kůrková, V., Karpov, P., Theis, F. (eds.) ICANN 2019. LNCS, vol. 11729, pp. 17–28. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-30508-6_2
Pedersen, M., Bengtson, S.H., Gade, R., Madsen, N., Moeslund, T.B.: Camera calibration for underwater 3D reconstruction based on ray tracing using Snell’s law. In: CVPR Workshops (CVPRW), pp. 1410–1417 (2018)
Ramalingam, S., Lodha, S., Sturm, P.: A generic structure-from-motion framework. Comput. Vis. Image Understand. 103, 218–228 (2006)
Ramalingam, S., Sturm, P., Lodha, S.: Towards complete generic camera calibration. IEEE Conference on Computer Vision and Pattern Recognition, San Diego, USA, pp. 1093–1098 (2005)
Rousseeuw, P.J., Croux, C.: Alternatives to the median absolute deviation. J. Am. Stat. Assoc. 88(424), 1273–1283 (1993)
Sturm, P., Ramalingam, S.: A generic concept for camera calibration. In: Pajdla, T., Matas, J. (eds.) ECCV 2004. LNCS, vol. 3022, pp. 1–13. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-24671-8_1
Sturm, P., Ramalingam, S., Tardif, J.P., Gasparini, S., Barreto, J.: Camera models and fundamental concepts used in geometric computer vision. Found. Trends ®Comput. Graph. Vis. 6(1–2), 1–183 (2011)
Trifonov, B., Bradley, D., Heidrich, W.: Tomographic reconstruction of transparent objects. In: Eurographics Conference on Rendering Techniques (2006)
Wikipedia: False position method – Wikipedia, the free encyclopedia (2018). https://en.wikipedia.org/wiki/False_position_method
Acknowledgement
This work was supported by the TRACE project with Toyota Motors Europe (TME).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Supplementary material 1 (mp4 528 KB)
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Verbiest, F., Proesmans, M., Van Gool, L. (2020). Modeling the Effects of Windshield Refraction for Camera Calibration. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, JM. (eds) Computer Vision – ECCV 2020. ECCV 2020. Lecture Notes in Computer Science(), vol 12351. Springer, Cham. https://doi.org/10.1007/978-3-030-58539-6_24
Download citation
DOI: https://doi.org/10.1007/978-3-030-58539-6_24
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-58538-9
Online ISBN: 978-3-030-58539-6
eBook Packages: Computer ScienceComputer Science (R0)