Abstract
This paper presents a unified approach for the relative pose estimation of a spectral camera - 3D Lidar pair without the use of any special calibration pattern or explicit point correspondence. The method works without specific setup and calibration targets, using only a pair of 2D-3D data. Pose estimation is formulated as a 2D-3D nonlinear shape registration task which is solved without point correspondences or complex similarity metrics. The registration is then traced back to the solution of a non-linear system of equations which directly provides the calibration parameters between the bases of the two sensors. The method has been extended both for perspective and omnidirectional central cameras and was tested on a large set of synthetic lidar-camera image pairs as well as on real data acquired in outdoor environment.
This research was partially supported by the European Union and the State of Hungary, co-financed by the European Social Fund through project TAMOP-4.2.2.A-11/1/KONV-2012-0073 (Telemedicine-focused research activities in the fields of Mathematics, Informatics and Medical sciences); as well as by Domus MTA Hungary. The laser data of the Bremen Cog was provided by Amandine Colson from the German Maritime Museum, Bremerhaven, Germany. The authors gratefully acknowledge the help of Csaba Benedek from DEVA Lab., SZTAKI in providing us with preprocessed Velodyne Lidar scans. The catadioptric camera was provided by the Multimedia Technologies and Telecommunications Research Center of UTCN with the help of Camelia Florea.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Boochs, F., Bentkowska-Kafel, A., Degrigny, C., Karaszewski, M., Karmacharya, A., Kato, Z., Picollo, M., Sitnik, R., Trémeau, A., Tsiafaki, D., Tamas, L.: Colour and space in cultural heritage: Key questions in 3D optical documentation of material culture for conservation, study and preservation. In: Ioannides, M., Magnenat-Thalmann, N., Fink, E., Žarnić, R., Yen, A.-Y., Quak, E. (eds.) EuroMed 2014. LNCS, vol. 8740, pp. 11–24. Springer, Heidelberg (2014)
Bouaziz, S., Pauly, M.: Dynamic 2D/3D registration for the kinect. In: ACM SIGGRAPH 2013 Courses, SIGGRAPH 2013, pp. 21:1–21:14. ACM, New York (2013)
Cao, T., Zach, C., Modla, S., Czymmek, D.P.K., Niethammer, M.: Multi-modal registration for correlative microscopy using image analogies. Medical Image Analysis 18(6), 914–926 (2014)
Domokos, C., Nemeth, J., Kato, Z.: Nonlinear Shape Registration without Correspondences. IEEE Transactions on Pattern Analysis and Machine Intelligence 34(5), 943–958 (2012)
Geiger, A., Lauer, M., Wojek, C., Stiller, C., Urtasun, R.: 3D Traffic Scene Understanding From Movable Platforms. IEEE Transactions on Pattern Analysis and Machine Intelligence 36(5), 1012–1025 (2014)
Geiger, A., Moosmann, F., Car, O., Schuster, B.: Automatic camera and range sensor calibration using a single shot. In: International Conference on Robotics and Automation, pp. 3936–3943. IEEE (2012)
Geyer, C., Daniilidis, K.: A unifying theory for central panoramic systems and practical implications. In: Vernon, D. (ed.) ECCV 2000. LNCS, vol. 1843, pp. 445–461. Springer, Heidelberg (2000)
Gong, X., Lin, Y., Liu, J.: Extrinsic calibration of a 3D lidar and a camera using a trihedron. Optics and Lasers in Engineering 51(4), 394–401 (2013)
Herrera, C.D., Kannala, J., Heikkila, J.: Joint depth and color camera calibration with distortion correction. IEEE Transactions on Pattern Analysis and Machine Intelligence 34(10), 1–8 (2012)
Kannala, J., Brandt, S.S.: A Generic Camera Model and Calibration Method for Conventional, Wide-Angle, and Fish-Eye Lenses. IEEE Transactions on Pattern Analysis and Machine Intelligence 28(8), 1335–1340 (2006)
Li, Y., Ruichek, Y., Cappelle, C.: Optimal extrinsic calibration between a stereoscopic system and a lidar. IEEE Transactions on Instrumentation and Measurement 62(8), 2258–2269 (2013)
Markelj, P., Tomaževič D., Likar, B., Pernuš, F.: A review of 3D/2D registration methods for image-guided interventions. Medical Image Analysis 16(3), 642–661 (2012). Computer Assisted Interventions
Mastin, A., Kepner, J., Fisher III, J.: Automatic registration of lidar and optical images of urban scenes. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 2639–2646. IEEE, Miami (June 2009)
Mei, C., Rives, P.: Single view point omnidirectional camera calibration from planar grids. In: International Conference on Robotics and Automation, pp. 3945–3950. Roma, Italy (April 2007)
Mičušík, B., Pajdla, T.: Para-catadioptric camera auto-calibration from epipolar geometry. In: Asian Conference on Computer Vision, Seoul, Korea South, vol. 2, pp. 748–753 (January 2004)
Mirzaei, F.M., Kottas, D.G., Roumeliotis, S.I.: 3D LIDAR-camera intrinsic and extrinsic calibration: Identifiability and analytical least-squares-based initialization. International Journal of Robotics Research 31(4), 452–467 (2012)
Mishra, R., Zhang, Y.: A review of optical imagery and airborne lidar data registration methods. The Open Remote Sensing Journal 5, 54–63 (2012)
Nunez, P., Drews, P., Rocha, R., Dias, J.: Data fusion calibration for a 3D laser range finder and a camera using inertial data. In: European Conference on Mobile Robots, Dubrovnik, Croatia, pp. 31–36 (September 2009)
Pandey, G., McBride, J.R., Savarese, S., Eustice, R.M.: Automatic targetless extrinsic calibration of a 3D lidar and camera by maximizing mutual information. In: AAAI National Conference on Artificial Intelligence, pp. 2053–2059. Toronto, Canada (July 2012)
Pluim, J., Maintz, J., Viergever, M.: Mutual-information-based registration of medical images: a survey. IEEE Transactions on Medical Imaging 22(8), 986–1004 (2003)
Scaramuzza, D., Harati, A., Siegwart, R.: Extrinsic self calibration of a camera and a 3D laser range finder from natural scenes. In: International Conference on Intelligent Robots and Systems, pp. 4164–4169. San Diego, USA (October 2007)
Scaramuzza, D., Martinelli, A., Siegwart, R.: A flexible technique for accurate omnidirectional camera calibration and structure from motion. In: International Conference on Computer Vision Systems, Washington, USA, pp. 45–51 (January 2006)
Schoenbein, M., Strauss, T., Geiger, A.: Calibrating and centering quasi-central catadioptric cameras. In: International Conference on Robotics and Automation, Hong-Kong, China, pp. 1253–1256 (June 2014)
Stamos, I.: Automated registration of 3D-range with 2D-color images: an overview. In: 2010 44th Annual Conference on Information Sciences and Systems (CISS), pp. 1–6 (March 2010)
Tamas, L., Frohlich, R., Kato, Z.: Relative pose estimation and fusion of omnidirectional and lidar cameras. In: ECCV Workshop on Computer Vision for Road Scene Understanding and Autonomous Driving (ECCV-CVRSUAD). Lecture Notes in Computer Science, pp. 1–12, Zurich, Switzerland (September 2014)
Tamas, L., Kato, Z.: Targetless calibration of a lidar - perspective camera pair. In: International Conference on Computer Vision, Bigdata3dcv Workshops, Sydney, Australia, pp. 668–675 (December 2013)
Taylor, Z., Nieto, J.: A mutual information approach to automatic calibration of camera and lidar in natural environments. In: Australian Conference on Robotics and Automation, Wellington, Australia, pp. 3–8 (December 2012)
Unnikrishnan, R., Hebert, M.: Fast extrinsic calibration of a laser rangefinder to a camera. Tech. rep. Carnegie Mellon University (2005)
Zhang, Q.: Extrinsic calibration of a camera and laser range finder. In: International Conference on Intelligent Robots and Systems, pp. 2301–2306. IEEE, Sendai (September 2004)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Kato, Z., Tamas, L. (2015). Relative Pose Estimation and Fusion of 2D Spectral and 3D Lidar Images. In: Trémeau, A., Schettini, R., Tominaga, S. (eds) Computational Color Imaging. CCIW 2015. Lecture Notes in Computer Science(), vol 9016. Springer, Cham. https://doi.org/10.1007/978-3-319-15979-9_4
Download citation
DOI: https://doi.org/10.1007/978-3-319-15979-9_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-15978-2
Online ISBN: 978-3-319-15979-9
eBook Packages: Computer ScienceComputer Science (R0)