Abstract
Camera parameter estimation can be used in visual odometry, robot vision, SLAM, 3D reconstruction and other directions. It is also the main research content of computer vision. Based on the deep learning strategy, we propose a secondary encoding network for camera parameters (SECPNet), which can predict the camera parameters and recover the camera pose according to a single RGB image. Based on the three-dimensional dataset ShapeNet40 (Chang et al. in An information-rich 3D model repository, 2015. arXiv:1512.03012), we build a varifocal multi-viewpoint image dataset for camera parameter estimation. Experimental results show that our method has state-of-the-art performance in camera parameter estimation.
Similar content being viewed by others
References
Abdel-Aziz, Y.I., Karara, H.M., Hauck, M.: Direct linear transformation from comparator coordinates into object space coordinates in close-range photogrammetry. Photogramm. Eng. Remote Sens. 81(2), 103–107 (2015)
Ardakani, H.K., Mousavinia, A., Safaei, F.: Four points: one-pass geometrical camera calibration algorithm. Visual Comput. 36(2), 413–424 (2020)
Bock, B., Allen, M.: Blender, June 8 2010. US Patent App. 29/350,912
Brachmann, E., Krull, A., Michel, F., Gumhold, S., Shotton, J., Rother, C.: Learning 6d object pose estimation using 3d object coordinates. In: European Conference on Computer Vision, pp. 536–551. Springer (2014)
Brahmbhatt, S., Gu, J., Kim, K., Hays, J., Kautz, J.: Geometry-aware learning of maps for camera localization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2616–2625 (2018)
Cai, B., Wang, Y., Jiajun, W., Wang, M., Li, F., Ma, M., Chen, X., Wang, K.: An effective method for camera calibration in defocus scene with circular gratings. Opt. Lasers Eng. 114, 44–49 (2019)
Chang, A.X., Funkhouser, T., Guibas, L., Hanrahan, P., Huang, Q., Li, Z., Savarese, S., Savva, M., Song, S., Su, H., et al.: Shapenet: an information-rich 3d model repository (2015). arXiv:1512.03012
Charco, J.L., Vintimilla, B.X, Sappa, A.D: Deep learning based camera pose estimation in multi-view environment. In: 2018 14th International Conference on Signal-Image Technology & Internet-Based Systems (SITIS), pp. 224–228. IEEE (2018)
Chen, B., Pan, B.: Camera calibration using synthetic random speckle pattern and digital image correlation. Opt. Lasers Eng. 126, 105919 (2020)
De Ma, S.: A self-calibration technique for active vision systems. IEEE Trans. Robot. Autom. 12(1), 114–120 (1996)
Do, T.-T., Cai, M., Pham, T., Reid, I.: Deep-6dpose: recovering 6d object pose from a single rgb image (2018). arXiv:1802.10367
Duong, N.-D., Kacete, A., Sodalie, C., Richard, P.-Y., Royan, J.: xyznet: towards machine learning camera relocalization by using a scene coordinate prediction network. In: 2018 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct), pp. 258–263. IEEE (2018)
Fang, Q., Zhao, K., Tang, D., Zhou, Z., Zhou, Y., Hu, T., Zhou, H.: Euler angles based loss function for camera localization with deep learning. In: 2018 IEEE 8th Annual International Conference on CYBER Technology in Automation, Control, and Intelligent Systems (CYBER), pp. 61–66. IEEE (2018)
Faugeras, O.D., Luong, Q.T., Maybank, S.J.: Camera self-calibration: Theory and experiments. In: Lecture Notes in Computer Science, vol. 588 (1998)
Frosio, I., Turrini, C., Alzati, A.: Camera re-calibration after zooming based on sets of conics. Visual Comput. 32(5), 663–674 (2016)
Glorot, X., Bengio, Y.: Understanding the difficulty of training deep feedforward neural networks. J. Mach. Learn. Res. 9, 249–256 (2010)
Grabner, A., Roth, P.M , Lepetit, V.: Gp2c: geometric projection parameter consensus for joint 3d pose and focal length estimation in the wild. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2222–2231 (2019)
Guan, W., Li, W., Xi, J.: Improved particle swarm optimization algorithm based nonlinear calibration of camera. In: 2017 29th Chinese Control And Decision Conference (CCDC), pp. 5217–5221. IEEE (2017)
Guo, F., He, Y., Guan, L.: Rgb-d camera pose estimation using deep neural network. In: 2017 IEEE Global Conference on Signal and Information Processing (GlobalSIP), pp. 408–412. IEEE (2017)
Hinterstoisser, S., Lepetit, V., Ilic, S., Holzer, S., Bradski, G., Konolige, K., Navab, N.: Model based training, detection and pose estimation of texture-less 3d objects in heavily cluttered scenes. In: Asian Conference on Computer Vision, pp. 548–562. Springer (2012)
Kehl, W., Manhardt, F., Tombari, F., Ilic, S., Navab, N.: Ssd-6d: making rgb-based 3d detection and 6d pose estimation great again. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1521–1529 (2017)
Kendall, A., Cipolla, R.: Geometric loss functions for camera pose regression with deep learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5974–5983 (2017)
Kendall, A., Grimes, M., Cipolla, R.: Posenet: a convolutional network for real-time 6-dof camera relocalization. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2938–2946 (2015)
Li, S., Harada, T., Zou, W.: Estimating relative pose between nonoverlapping cameras by four laser pointers based on general camera model. In: 2017 4th IAPR Asian Conference on Pattern Recognition (ACPR), pp. 168–172. IEEE (2017)
Liu, Y., Chen, X., Tianlun, G., Zhang, Y., Xing, G.: Real-time camera pose estimation via line tracking. Visual Comput. 34(6–8), 899–909 (2018)
Lu, L., Li, H.: Study of camera calibration algorithm based on spatial perpendicular intersect. In: 2010 2nd International Conference on Signal Processing Systems, vol. 3, pp. V3–V125. IEEE (2010)
Matei, B.C., Meer, P.: Estimation of nonlinear errors-in-variables models for computer vision applications. IEEE Trans. Pattern Anal. Mach. Intell. 28(10), 1537–1552 (2006)
Mottaghi, R., Xiang, Y., Savarese, S.: A coarse-to-fine model for 3d pose estimation and sub-category recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 418–426 (2015)
Mousavian, A., Anguelov, D., Flynn, J., Kosecka, J.: 3d bounding box estimation using deep learning and geometry. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7074–7082 (2017)
Naseer, T., Burgard, W.: Deep regression for monocular camera-based 6-dof global localization in outdoor environments. In: 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 1525–1530. IEEE (2017)
Peng, S., Liu, Y., Huang, Q., Zhou, X., Bao, H.: Pvnet: pixel-wise voting network for 6dof pose estimation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4561–4570 (2019)
Sahin, C., Garcia-Hernando, G., Sock, J. Kim, T.-K.: Instance- and Category-Level 6D Object Pose Estimation, pp. 243–265 (2019)
Sattler, T., Maddern, W., Toft, C., Torii, A., Hammarstrand, L., Stenborg, E., Safari, D., Okutomi, M., Pollefeys, M., Sivic, J., et al.: Benchmarking 6dof outdoor visual localization in changing conditions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8601–8610 (2018)
Sattler, T., Sweeney, C., Pollefeys, M.: On sampling focal length values to solve the absolute pose problem. In: European Conference on Computer Vision, pp. 828–843. Springer (2014)
Sattler, T., Zhou, Q., Pollefeys, M., Leal-Taixe, L.: Understanding the limitations of cnn-based absolute camera pose regression. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3302–3312 (2019)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition (2014). arXiv:1409.1556
Sun, J., Wang, P., Qin, Z., Qiao, H.: Overview of camera calibration for computer vision. In: Proceeding of the 11th World Congress on Intelligent Control and Automation, pp. 86–92. IEEE (2014)
Sun, X., Wu, J., Zhang, X., Zhang, Z., Zhang, C., Xue, T., Tenenbaum, J.B., Freeman, W.T.: Pix3d: dataset and methods for single-image 3d shape modeling. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2974–2983 (2018)
Sundermeyer, M., Marton, Z.-C., Durner, M., Brucker, M., Triebel, R.: Implicit 3d orientation learning for 6d object detection from rgb images. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 699–715 (2018)
Tekin, B., Sinha, S.N., Fua, P.: Real-time seamless single shot 6d object pose prediction. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 292–301 (2018)
Triggs, B.: Autocalibration and the absolute quadric. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 609–614. IEEE (1997)
Wang, C., Martín-Martín, R., Xu, D., Lv, J., Lu, C., Fei-Fei, L., Savarese, S., Zhu, Y.: 6-pack: category-level 6d pose tracker with anchor-based keypoints (2019). arXiv:1910.10750
Workman, S., Greenwell, C., Zhai, M., Baltenberger, R., Jacobs, N.: Deepfocal: a method for direct focal length estimation. In: 2015 IEEE International Conference on Image Processing (ICIP), pp. 1369–1373. IEEE (2015)
Wu, C.: P3. 5p: pose estimation with unknown focal length. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2440–2448 (2015)
Xiang, Y., Schmidt, T., Narayanan, V., Fox, D.: Posecnn: a convolutional neural network for 6d object pose estimation in cluttered scenes (2017). arXiv:1711.00199
Xu, G.Y., Chen, L.P., Gao, F.: Study on binocular stereo camera calibration method. In: 2011 International Conference on Image Analysis and Signal Processing, pp. 133–137. IEEE (2011)
Zhang, Z.: Flexible camera calibration by viewing a plane from unknown orientations. In:Proceedings of ICCV (1999)
Zhang, Z.: A flexible new technique for camera calibration. IEEE Trans. Pattern Anal. Mach. Intell. 22, 1330–1334 (2000)
Zhang, Z., Zhao, R., Liu, E., Yan, K., Ma, Y.: A single-image linear calibration method for camera. Measurement 130, 298–305 (2018)
Zheng, Y., Kneip, L.: A direct least-squares solution to the pnp problem with unknown focal length. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1790–1798 (2016)
Zheng, Y., Sugimoto, S., Sato, I., Okutomi, M.: A general and simple method for camera pose and focal length determination. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 430–437 (2014)
Zhu, Z., Wang, X., Liu, Q., Zhang, F.: Camera calibration method based on optimal polarization angle. Opt. Lasers Eng. 112, 128–135 (2019)
Funding
This study was funded by Postgraduate Research & Practice Innovation Program of Jiangsu Province Grant Number SJCX20 _ 0775.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Liu, D., Chen, L. SECPNet—secondary encoding network for estimating camera parameters. Vis Comput 38, 1689–1702 (2022). https://doi.org/10.1007/s00371-021-02098-2
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00371-021-02098-2