A Theory of Minimal 3D Point to 3D Plane Registration and Its Generalization

Abstract

Registration of 3D data is a key problem in many applications in computer vision, computer graphics and robotics. This paper provides a family of minimal solutions for the 3D-to-3D registration problem in which the 3D data are represented as points and planes. Such scenarios occur frequently when a 3D sensor provides 3D points and our goal is to register them to a 3D object represented by a set of planes. In order to compute the 6 degrees-of-freedom transformation between the sensor and the object, we need at least six points on three or more planes. We systematically investigate and develop pose estimation algorithms for several configurations, including all minimal configurations, that arise from the distribution of points on planes. We also identify the degenerate configurations in such registrations. The underlying algebraic equations used in many registration problems are the same and we show that many 2D-to-3D and 3D-to-3D pose estimation/registration algorithms involving points, lines, and planes can be mapped to the proposed framework. We validate our theory in simulations as well as in three real-world applications: registration of a robotic arm with an object using a contact sensor, registration of planar city models with 3D point clouds obtained using multi-view reconstruction, and registration between depth maps generated by a Kinect sensor.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Notes

  1. 1.

    http://cmp.felk.cvut.cz/minimal/.

References

  1. Besl, P. J., & McKay, N. D. (1992). A method for registration of 3-D shapes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 14(2), 239–256.

    Article  Google Scholar 

  2. Chen, H. H. (1991). Pose determination from line-to-plane correspondences: existence condition and closed-form solutions. IEEE Transactions on Pattern Analysis and Machine Intelligence, 13(6), 530–541.

    Article  Google Scholar 

  3. Chen, Y., & Medioni, G. (1991). Object modeling by registration of multiple range images. In Proc. IEEE int’l conf. robotics automation (ICRA) (Vol. 3, pp. 2724–2729).

    Google Scholar 

  4. Dhome, M., Richetin, M., Lapresté, J. T., & Rives, G. (1989). Determination of the attitude of 3D objects from a single perspective view. IEEE Transactions on Pattern Analysis and Machine Intelligence, 11(12), 1265–1278.

    Article  Google Scholar 

  5. Drost, B., Ulrich, M., Navab, N., & Ilic, S. (2010). Model globally, match locally: efficient and robust 3D object recognition. In Proc. IEEE conf. computer vision and pattern recognition (CVPR) (pp. 998–1005).

    Google Scholar 

  6. Enqvist, O., Josephson, K., & Kahl, F. (2009). Optimal correspondences from pairwise constraints. In Proc. IEEE int’l conf. computer vision (ICCV) (pp. 1295–1302).

    Google Scholar 

  7. Fischler, M. A., & Bolles, R. C. (1981). Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 24(6), 381–395.

    MathSciNet  Article  Google Scholar 

  8. Fitzgibbon, A. W. (2003). Robust registration of 2D and 3D point sets. Image and Vision Computing, 21(13–14), 1145–1153.

    Article  Google Scholar 

  9. Furukawa, Y., & Ponce, J. (2010). Accurate, dense, and robust multi-view stereopsis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(8), 1362–1376.

    Article  Google Scholar 

  10. Gao, X. S., Hou, X. R., Tang, J., & Cheng, H. F. (2003). Complete solution classification for the perspective-three-point problem. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(8), 930–943.

    Article  Google Scholar 

  11. Geyer, C., & Stewenius, H. (2007). A nine-point algorithm for estimating para-catadioptric fundamental matrices. In Proc. IEEE conf. computer vision and pattern recognition (CVPR).

    Google Scholar 

  12. Grimson, W. E. L., & Lozano-Pérez, T. (1983). Model-based recognition and localization from sparse range or tactile data. MIT AI Lab, A. I. Memo, 738.

  13. Haralick, R. M., Lee, C. N., Ottenberg, K., & Nolle, M. (1994). Review and analysis of solutions of the three point perspective pose estimation problem. International Journal of Computer Vision, 13(3), 331–356.

    Article  Google Scholar 

  14. Horn, B. K. P. (1987). Closed-form solution of absolute orientation using unit quaternions. Journal of the Optical Society of America. A, Online, 4(4), 629–642.

    MathSciNet  Article  Google Scholar 

  15. Kukelova, Z., Bujnak, M., & Pajdla, T. (2008). Automatic generator of minimal problem solvers. In Proc. European conf. computer vision (ECCV) (pp. 302–315).

    Google Scholar 

  16. Kukelova, Z., Bujnak, M., & Pajdla, T. (2010). Closed-form solutions to the minimal absolute pose problems with known vertical direction. In Proc. Asian conf. computer vision (ACCV) (pp. 216–229).

    Google Scholar 

  17. Kukelova, Z., & Pajdla, T. (2007). A minimal solution to the autocalibration of radial distortion. In Proc. IEEE conf. computer vision and pattern recognition (CVPR).

    Google Scholar 

  18. Li, H., & Hartley, R. (2005). A non-iterative method for correcting lens distortion from nine-point correspondences. In Proc. OMNIVIS.

    Google Scholar 

  19. Li, H., & Hartley, R. (2007). The 3D-3D registration problem revisited. In Proc. IEEE int’l conf. computer vision (ICCV) (pp. 1–8).

    Google Scholar 

  20. Naroditsky, O., Patterson, A., & Daniilidis, K. (2011). Automatic alignment of a camera with a line scan LIDAR system. In Proc. IEEE int’l conf. robotics automation (ICRA) (pp. 3429–3434).

    Google Scholar 

  21. Naroditsky, O., Zhou, X. S., Gallier, J., Roumeliotis, S. I., & Daniilidis, K. (2012). Two efficient solutions for visual odometry using directional correspondence. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(4), 818–824.

    Article  Google Scholar 

  22. Nistér, D. (2003). An efficient solution to the five-point relative pose problem. In Proc. IEEE conf. computer vision and pattern recognition (CVPR) (pp. 195–202).

    Google Scholar 

  23. Nistér, D. (2004). A minimal solution to the generalized 3-point pose problem. In Proc. IEEE conf. computer vision and pattern recognition (CVPR) (Vol. 1, pp. 560–567).

    Google Scholar 

  24. Nistér, D., & Schaffalitzky, F. (2006). Four points in two or three calibrated views: theory and practice. International Journal of Computer Vision, 67(2), 211–231.

    Article  Google Scholar 

  25. Olsson, C., Kahl, F., & Oskarsson, M. (2006). The registration problem revisited: optimal solutions from points, lines and planes. In Proc. IEEE conf. computer vision and pattern recognition (CVPR) (Vol. 1, pp. 1206–1213).

    Google Scholar 

  26. Raguram, R., Frahm, J. M., & Pollefeys, M. (2008). A comparative analysis of RANSAC techniques leading to adaptive real-time random sample consensus. In Proc. European conf. computer vision (ECCV) (pp. 500–513).

    Google Scholar 

  27. Ramalingam, S., & Lodha, S. K. (2003). Adaptive enhancement of 3D scenes using hierarchical registration of texture-mapped 3D models. In Proc. int’l conf. 3-D digital imaging and modeling (3DIM) (pp. 203–210).

    Google Scholar 

  28. Ramalingam, S., Taguchi, Y., Marks, T. K., & Tuzel, O. (2010). P2Π: a minimal solution for registration of 3D points to 3D planes. In Proc. European conf. computer vision (ECCV) (Vol. 5, pp. 436–449).

    Google Scholar 

  29. Rusinkiewicz, S., & Levoy, M. (2001). Efficient variants of the ICP algorithm. In Proc. int’l conf. 3-D digital imaging and modeling (3DIM) (pp. 145–152).

    Chapter  Google Scholar 

  30. Shotton, J., Fitzgibbon, A., Cook, M., Sharp, T., Finocchio, M., Moore, R., Kipman, A., & Blake, A. (2011). Real-time human pose recognition in parts from single depth images. In Proc. IEEE conf. computer vision and pattern recognition (CVPR) (pp. 1297–1304).

    Google Scholar 

  31. Snavely, N., Seitz, S. M., & Szeliski, R. (2006). Photo tourism: exploring image collections in 3D. ACM Transactions on Graphics, 25(3), 835–846.

    Article  Google Scholar 

  32. Stewenius, H., Nistér, D., Kahl, F., & Schaffalitzky, F. (2005a). A minimal solution for relative pose with unknown focal length. In Proc. IEEE conf. computer vision and pattern recognition (CVPR) (Vol. 2, pp. 789–794).

    Google Scholar 

  33. Stewenius, H., Nistér, D., Oskarsson, M., & Astrom, K. (2005b). Solutions to minimal generalized relative pose problems. In Proc. OMNIVIS.

    Google Scholar 

  34. Tu, P., Saxena, T., & Hartley, R. (1999). Recognizing objects using color-annotated adjacency graphs. In Lecture notes in computer science: shape, contour and grouping in computer vision (pp. 246–263).

    Chapter  Google Scholar 

Download references

Acknowledgements

Special thanks to Tim K. Marks and Oncel Tuzel in helping us in the conference submission. We would like to thank Jay Thornton, Keisuke Kojima, John Barnwell, and Haruhisa Okuda for their valuable feedback, help and support. We would also like to thank the anonymous reviewers and AC’s comments in ECCV 2010.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Srikumar Ramalingam.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Ramalingam, S., Taguchi, Y. A Theory of Minimal 3D Point to 3D Plane Registration and Its Generalization. Int J Comput Vis 102, 73–90 (2013). https://doi.org/10.1007/s11263-012-0576-x

Download citation

Keywords

  • 3D-to-3D registration
  • Pose estimation
  • Point-to-plane registration
  • Minimal solution
  • Correspondence problem