Registration of 3D data is a key problem in many applications in computer vision, computer graphics and robotics. This paper provides a family of minimal solutions for the 3D-to-3D registration problem in which the 3D data are represented as points and planes. Such scenarios occur frequently when a 3D sensor provides 3D points and our goal is to register them to a 3D object represented by a set of planes. In order to compute the 6 degrees-of-freedom transformation between the sensor and the object, we need at least six points on three or more planes. We systematically investigate and develop pose estimation algorithms for several configurations, including all minimal configurations, that arise from the distribution of points on planes. We also identify the degenerate configurations in such registrations. The underlying algebraic equations used in many registration problems are the same and we show that many 2D-to-3D and 3D-to-3D pose estimation/registration algorithms involving points, lines, and planes can be mapped to the proposed framework. We validate our theory in simulations as well as in three real-world applications: registration of a robotic arm with an object using a contact sensor, registration of planar city models with 3D point clouds obtained using multi-view reconstruction, and registration between depth maps generated by a Kinect sensor.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Tax calculation will be finalised during checkout.
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
Tax calculation will be finalised during checkout.
Besl, P. J., & McKay, N. D. (1992). A method for registration of 3-D shapes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 14(2), 239–256.
Chen, H. H. (1991). Pose determination from line-to-plane correspondences: existence condition and closed-form solutions. IEEE Transactions on Pattern Analysis and Machine Intelligence, 13(6), 530–541.
Chen, Y., & Medioni, G. (1991). Object modeling by registration of multiple range images. In Proc. IEEE int’l conf. robotics automation (ICRA) (Vol. 3, pp. 2724–2729).
Dhome, M., Richetin, M., Lapresté, J. T., & Rives, G. (1989). Determination of the attitude of 3D objects from a single perspective view. IEEE Transactions on Pattern Analysis and Machine Intelligence, 11(12), 1265–1278.
Drost, B., Ulrich, M., Navab, N., & Ilic, S. (2010). Model globally, match locally: efficient and robust 3D object recognition. In Proc. IEEE conf. computer vision and pattern recognition (CVPR) (pp. 998–1005).
Enqvist, O., Josephson, K., & Kahl, F. (2009). Optimal correspondences from pairwise constraints. In Proc. IEEE int’l conf. computer vision (ICCV) (pp. 1295–1302).
Fischler, M. A., & Bolles, R. C. (1981). Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 24(6), 381–395.
Fitzgibbon, A. W. (2003). Robust registration of 2D and 3D point sets. Image and Vision Computing, 21(13–14), 1145–1153.
Furukawa, Y., & Ponce, J. (2010). Accurate, dense, and robust multi-view stereopsis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(8), 1362–1376.
Gao, X. S., Hou, X. R., Tang, J., & Cheng, H. F. (2003). Complete solution classification for the perspective-three-point problem. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(8), 930–943.
Geyer, C., & Stewenius, H. (2007). A nine-point algorithm for estimating para-catadioptric fundamental matrices. In Proc. IEEE conf. computer vision and pattern recognition (CVPR).
Grimson, W. E. L., & Lozano-Pérez, T. (1983). Model-based recognition and localization from sparse range or tactile data. MIT AI Lab, A. I. Memo, 738.
Haralick, R. M., Lee, C. N., Ottenberg, K., & Nolle, M. (1994). Review and analysis of solutions of the three point perspective pose estimation problem. International Journal of Computer Vision, 13(3), 331–356.
Horn, B. K. P. (1987). Closed-form solution of absolute orientation using unit quaternions. Journal of the Optical Society of America. A, Online, 4(4), 629–642.
Kukelova, Z., Bujnak, M., & Pajdla, T. (2008). Automatic generator of minimal problem solvers. In Proc. European conf. computer vision (ECCV) (pp. 302–315).
Kukelova, Z., Bujnak, M., & Pajdla, T. (2010). Closed-form solutions to the minimal absolute pose problems with known vertical direction. In Proc. Asian conf. computer vision (ACCV) (pp. 216–229).
Kukelova, Z., & Pajdla, T. (2007). A minimal solution to the autocalibration of radial distortion. In Proc. IEEE conf. computer vision and pattern recognition (CVPR).
Li, H., & Hartley, R. (2005). A non-iterative method for correcting lens distortion from nine-point correspondences. In Proc. OMNIVIS.
Li, H., & Hartley, R. (2007). The 3D-3D registration problem revisited. In Proc. IEEE int’l conf. computer vision (ICCV) (pp. 1–8).
Naroditsky, O., Patterson, A., & Daniilidis, K. (2011). Automatic alignment of a camera with a line scan LIDAR system. In Proc. IEEE int’l conf. robotics automation (ICRA) (pp. 3429–3434).
Naroditsky, O., Zhou, X. S., Gallier, J., Roumeliotis, S. I., & Daniilidis, K. (2012). Two efficient solutions for visual odometry using directional correspondence. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(4), 818–824.
Nistér, D. (2003). An efficient solution to the five-point relative pose problem. In Proc. IEEE conf. computer vision and pattern recognition (CVPR) (pp. 195–202).
Nistér, D. (2004). A minimal solution to the generalized 3-point pose problem. In Proc. IEEE conf. computer vision and pattern recognition (CVPR) (Vol. 1, pp. 560–567).
Nistér, D., & Schaffalitzky, F. (2006). Four points in two or three calibrated views: theory and practice. International Journal of Computer Vision, 67(2), 211–231.
Olsson, C., Kahl, F., & Oskarsson, M. (2006). The registration problem revisited: optimal solutions from points, lines and planes. In Proc. IEEE conf. computer vision and pattern recognition (CVPR) (Vol. 1, pp. 1206–1213).
Raguram, R., Frahm, J. M., & Pollefeys, M. (2008). A comparative analysis of RANSAC techniques leading to adaptive real-time random sample consensus. In Proc. European conf. computer vision (ECCV) (pp. 500–513).
Ramalingam, S., & Lodha, S. K. (2003). Adaptive enhancement of 3D scenes using hierarchical registration of texture-mapped 3D models. In Proc. int’l conf. 3-D digital imaging and modeling (3DIM) (pp. 203–210).
Ramalingam, S., Taguchi, Y., Marks, T. K., & Tuzel, O. (2010). P2Π: a minimal solution for registration of 3D points to 3D planes. In Proc. European conf. computer vision (ECCV) (Vol. 5, pp. 436–449).
Rusinkiewicz, S., & Levoy, M. (2001). Efficient variants of the ICP algorithm. In Proc. int’l conf. 3-D digital imaging and modeling (3DIM) (pp. 145–152).
Shotton, J., Fitzgibbon, A., Cook, M., Sharp, T., Finocchio, M., Moore, R., Kipman, A., & Blake, A. (2011). Real-time human pose recognition in parts from single depth images. In Proc. IEEE conf. computer vision and pattern recognition (CVPR) (pp. 1297–1304).
Snavely, N., Seitz, S. M., & Szeliski, R. (2006). Photo tourism: exploring image collections in 3D. ACM Transactions on Graphics, 25(3), 835–846.
Stewenius, H., Nistér, D., Kahl, F., & Schaffalitzky, F. (2005a). A minimal solution for relative pose with unknown focal length. In Proc. IEEE conf. computer vision and pattern recognition (CVPR) (Vol. 2, pp. 789–794).
Stewenius, H., Nistér, D., Oskarsson, M., & Astrom, K. (2005b). Solutions to minimal generalized relative pose problems. In Proc. OMNIVIS.
Tu, P., Saxena, T., & Hartley, R. (1999). Recognizing objects using color-annotated adjacency graphs. In Lecture notes in computer science: shape, contour and grouping in computer vision (pp. 246–263).
Special thanks to Tim K. Marks and Oncel Tuzel in helping us in the conference submission. We would like to thank Jay Thornton, Keisuke Kojima, John Barnwell, and Haruhisa Okuda for their valuable feedback, help and support. We would also like to thank the anonymous reviewers and AC’s comments in ECCV 2010.
About this article
Cite this article
Ramalingam, S., Taguchi, Y. A Theory of Minimal 3D Point to 3D Plane Registration and Its Generalization. Int J Comput Vis 102, 73–90 (2013). https://doi.org/10.1007/s11263-012-0576-x
- 3D-to-3D registration
- Pose estimation
- Point-to-plane registration
- Minimal solution
- Correspondence problem