Radial Lens Distortion Correction Using Convolutional Neural Networks Trained with Synthesized Images

  • Jiangpeng Rong
  • Shiyao Huang
  • Zeyu Shang
  • Xianghua YingEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10113)


Radial lens distortion often exists in images taken by common cameras, which violates the assumption of pinhole camera model. Estimating the radial lens distortion of an image is an important preprocessing step for many vision applications. This paper intends to employ CNNs (Convolutional Neural Networks), to achieve radial distortion correction. However, the main issue hinder its progress is the scarcity of training data with radial distortion annotations. Inspired by the growing availability of image dataset with non-radial distortion, we propose a framework to address the issue by synthesizing images with radial distortion for CNNs. We believe that a large number of images of high variation of radial distortion is generated, which can be well exploited by deep CNN with a high learning capacity. We present quantitative results that demonstrate the ability of our technique to estimate the radial distortion with comparisons against several baseline methods, including an automatic method based on Hough transforms of distorted line images.


Training Image Convolutional Neural Network Distorted Image Transfer Learning Target Task 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



This work was supported in part by State Key Development Program Grand No. 2016YFB1001001, NNSFC Grant No. 61322309, and NNSFC Grant No. 61273283.


  1. 1.
    Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press, Cambridge (2003)zbMATHGoogle Scholar
  2. 2.
    Kuang, Y., Solem, J.E., Kahl, F., Åström, K.: Minimal solvers for relative pose with a single unknown radial distortion. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition, pp. 33–40. IEEE (2014)Google Scholar
  3. 3.
    Kukelova, Z., Pajdla, T.: A minimal solution to radial distortion autocalibration. IEEE Trans. Pattern Anal. Mach. Intell. 33, 2410–2422 (2011)CrossRefGoogle Scholar
  4. 4.
    Ying, X., Mei, X., Yang, S., Wang, G., Rong, J., Zha, H.: Imposing differential constraints on radial distortion correction. In: Cremers, D., Reid, I., Saito, H., Yang, M.-H. (eds.) ACCV 2014. LNCS, vol. 9003, pp. 384–398. Springer, Cham (2015). doi: 10.1007/978-3-319-16865-4_25 Google Scholar
  5. 5.
    Ying, X., Mei, X., Yang, S., Wang, G., Zha, H.: Radial distortion correction from a single image of a planar calibration pattern using convex optimization. In: 2014 IEEE International Conference on Image Processing (ICIP), pp. 3440–3443. IEEE (2014)Google Scholar
  6. 6.
    Zhang, Z.: Flexible camera calibration by viewing a plane from unknown orientations. In: The Proceedings of the Seventh IEEE International Conference on Computer Vision, vol. 1, pp. 666–673. IEEE (1999)Google Scholar
  7. 7.
    Basu, A., Licardie, S.: Alternative models for fish-eye lenses. Pattern Recogn. Lett. 16, 433–441 (1995)CrossRefGoogle Scholar
  8. 8.
    Claus, D., Fitzgibbon, A.W.: A rational function lens distortion model for general cameras. In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2005), vol. 1, pp. 213–219. IEEE (2005)Google Scholar
  9. 9.
    Mei, X., Yang, S., Rong, J., Ying, X., Huang, S., Zha, H.: Radial lens distortion correction using cascaded one-parameter division model. In: 2015 IEEE International Conference on Image Processing (ICIP), pp. 3615–3619. IEEE (2015)Google Scholar
  10. 10.
    Ying, X., Hu, Z.: Can we consider central catadioptric cameras and fisheye cameras within a unified imaging model. In: Pajdla, T., Matas, J. (eds.) ECCV 2004. LNCS, vol. 3021, pp. 442–455. Springer, Heidelberg (2004). doi: 10.1007/978-3-540-24670-1_34 CrossRefGoogle Scholar
  11. 11.
    Melo, R., Antunes, M., Barreto, J.P., Falcao, G., Goncalves, N.: Unsupervised intrinsic calibration from a single frame using a “plumb-line” approach. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 537–544 (2013)Google Scholar
  12. 12.
    Fitzgibbon, A.W.: Simultaneous linear estimation of multiple view geometry and lens distortion. In: Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2001), vol. 1, pp. 1–125. IEEE (2001)Google Scholar
  13. 13.
    Devernay, F., Faugeras, O.: Straight lines have to be straight. Mach. Vis. Appl. 13, 14–24 (2001)CrossRefGoogle Scholar
  14. 14.
    Bukhari, F., Dailey, M.N.: Automatic radial distortion estimation from a single image. J. Math. Imaging Vis. 45, 31–45 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  15. 15.
    Hughes, C., Denny, P., Glavin, M., Jones, E.: Equidistant fish-eye calibration and rectification by vanishing point extraction. IEEE Trans. Pattern Anal. Mach. Intell. 32, 2289–2296 (2010)CrossRefGoogle Scholar
  16. 16.
    Rosten, E., Loveland, R.: Camera distortion self-calibration using the plumb-line constraint and minimal hough entropy. Mach. Vis. Appl. 22, 77–85 (2011)CrossRefGoogle Scholar
  17. 17.
    Alemán-Flores, M., Alvarez, L., Gomez, L., Santana-Cedrés, D.: Line detection in images showing significant lens distortion and application to distortion correction. Pattern Recognit. Lett. 36, 261–271 (2014)CrossRefGoogle Scholar
  18. 18.
    Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: a large-scale hierarchical image database. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2009), pp. 248–255. IEEE (2009)Google Scholar
  19. 19.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)Google Scholar
  20. 20.
    Fernandes, L.A., Oliveira, M.M.: Real-time line detection through an improved Hough transform voting scheme. Pattern Recognit. 41, 299–314 (2008)CrossRefzbMATHGoogle Scholar
  21. 21.
    Alemán-Flores, M., Alvarez, L., Gomez, L., Santana-Cedrés, D.: Automatic lens distortion correction using one-parameter division models. Image Process. Line 4, 327–343 (2014)CrossRefGoogle Scholar
  22. 22.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arxiv:1409.1556 (2014)
  23. 23.
    Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.: Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–9 (2015)Google Scholar
  24. 24.
    Juneja, M., Vedaldi, A., Jawahar, C., Zisserman, A.: Blocks that shout: distinctive parts for scene classification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 923–930 (2013)Google Scholar
  25. 25.
    Le, Q.V.: Building high-level features using large scale unsupervised learning. In: 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 8595–8598. IEEE (2013)Google Scholar
  26. 26.
    Le, Q.V., Zou, W.Y., Yeung, S.Y., Ng, A.Y.: Learning hierarchical invariant spatio-temporal features for action recognition with independent subspace analysis. In: 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3361–3368. IEEE (2011)Google Scholar
  27. 27.
    Donahue, J., Jia, Y., Vinyals, O., Hoffman, J., Zhang, N., Tzeng, E., Darrell, T.: DeCAF: a deep convolutional activation feature for generic visual recognition. In: ICML, pp. 647–655 (2014)Google Scholar
  28. 28.
    Pan, S.J., Yang, Q.: A survey on transfer learning. IEEE Trans. Knowl. Data Eng. 22, 1345–1359 (2010)CrossRefGoogle Scholar
  29. 29.
    Aytar, Y., Zisserman, A.: Tabula rasa: model transfer for object category detection. In: 2011 International Conference on Computer Vision, pp. 2252–2259. IEEE (2011)Google Scholar
  30. 30.
    Tommasi, T., Orabona, F., Caputo, B.: Safety in numbers: learning categories from few examples with multi model knowledge transfer. In: 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3081–3088. IEEE (2010)Google Scholar
  31. 31.
    LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86, 2278–2324 (1998)CrossRefGoogle Scholar
  32. 32.
    Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., Darrell, T.: Caffe: convolutional architecture for fast feature embedding. In: Proceedings of the 22nd ACM International Conference on Multimedia, pp. 675–678. ACM (2014)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Jiangpeng Rong
    • 1
  • Shiyao Huang
    • 1
  • Zeyu Shang
    • 1
  • Xianghua Ying
    • 1
    Email author
  1. 1.Key Laboratory of Machine Perception (Ministry of Education), School of Electronic Engineering and Computer SciencePeking UniversityBeijingPeople’s Republic of China

Personalised recommendations