Advertisement

A Consistently Fast and Globally Optimal Solution to the Perspective-n-Point Problem

Conference paper
  • 2.1k Downloads
Part of the Lecture Notes in Computer Science book series (LNCS, volume 12346)

Abstract

An approach for estimating the pose of a camera given a set of 3D points and their corresponding 2D image projections is presented. It formulates the problem as a non-linear quadratic program and identifies regions in the parameter space that contain unique minima with guarantees that at least one of them will be the global minimum. Each regional minimum is computed with a sequential quadratic programming scheme. These premises result in an algorithm that always determines the global minima of the perspective-n-point problem for any number of input correspondences, regardless of possible coplanar arrangements of the imaged 3D points. For its implementation, the algorithm merely requires ordinary operations available in any standard off-the-shelf linear algebra library. Comparative evaluation demonstrates that the algorithm achieves state-of-the-art results at a consistently low computational cost.

Keywords

Perspective-n-point problem Pose estimation Non-linear quadratic program Sequential quadratic programming Global optimality 

Notes

Acknowledgements

M. Lourakis has been funded by the EU H2020 Programme under Grant Agreement No. 826506 (sustAGE).

Supplementary material

500725_1_En_28_MOESM1_ESM.pdf (557 kb)
Supplementary material 1 (pdf 556 KB)

References

  1. 1.
    Andersen, E.D., Roos, C., Terlaky, T.: On implementing a primal-dual interior-point method for conic quadratic optimization. Math. Program. 95(2), 249–277 (2003)MathSciNetCrossRefGoogle Scholar
  2. 2.
    Bardet, M., Faugere, J.C., Salvy, B.: On the complexity of Gröbner basis computation of semi-regular overdetermined algebraic equations. In: Proceedings of the International Conference on Polynomial System Solving, pp. 71–74 (2004)Google Scholar
  3. 3.
    Beck, A., Eldar, Y.C.: Strong duality in nonconvex quadratic optimization with two quadratic constraints. SIAM J. Optim. 17(3), 844–860 (2006)MathSciNetCrossRefGoogle Scholar
  4. 4.
    Boggs, P.T., Tolle, J.W.: Sequential quadratic programming. Acta Numerica 4, 1–51 (1995)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Buchberger, B., Kauers, M.: Groebner basis. Scholarpedia 5(10), 7763 (2010)CrossRefGoogle Scholar
  6. 6.
    Bujnak, M., Kukelova, Z., Pajdla, T.: A general solution to the P4P problem for camera with unknown focal length. In: Conference on Computer Vision and Pattern Recognition, pp. 1–8. IEEE (2008)Google Scholar
  7. 7.
    Bujnak, M., Kukelova, Z., Pajdla, T.: New efficient solution to the absolute pose problem for camera with unknown focal length and radial distortion. In: Kimmel, R., Klette, R., Sugimoto, A. (eds.) ACCV 2010. LNCS, vol. 6492, pp. 11–24. Springer, Heidelberg (2011).  https://doi.org/10.1007/978-3-642-19315-6_2CrossRefGoogle Scholar
  8. 8.
    Cayley, A.: Sur quelques propriétés des déterminants gauches. J. für die reine und angewandte Mathematik 32, 119–123 (1846)MathSciNetGoogle Scholar
  9. 9.
    Ferraz, L., Binefa, X., Moreno-Noguer, F.: Very fast solution to the PnP problem with algebraic outlier rejection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 501–508 (2014)Google Scholar
  10. 10.
    Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Floudas, C.A., Visweswaran, V.: Quadratic optimization. In: Horst, R., Pardalos, P.M. (eds.) Handbook of Global Optimization. Nonconvex Optimization and Its Applications, vol. 2, pp. 217–269. Springer, Boston (1995).  https://doi.org/10.1007/978-1-4615-2025-2_5CrossRefzbMATHGoogle Scholar
  12. 12.
    Forst, W., Hoffmann, D.: Optimization - Theory and Practice. Springer, Heidelberg (2010).  https://doi.org/10.1007/978-0-387-78977-4CrossRefzbMATHGoogle Scholar
  13. 13.
    Fraleigh, J., Beauregard, R.: Linear Algebra. Addison-Wesley, Boston (1995)zbMATHGoogle Scholar
  14. 14.
    Gao, X.S., Hou, X.R., Tang, J., Cheng, H.F.: Complete solution classification for the perspective-three-point problem. IEEE Trans. Pattern Anal. Mach. Intell. 25(8), 930–943 (2003)CrossRefGoogle Scholar
  15. 15.
    Grunert, J.: Das pothenotische Problem in erweiterter Gestalt nebst über seine Anwendungen in Geodäsie. Grunerts Archiv für Mathematik und Physik (1841)Google Scholar
  16. 16.
    Haralick, R.M., Joo, H., Lee, C.N., Zhuang, X., Vaidya, V.G., Kim, M.B.: Pose estimation from corresponding point data. IEEE Trans. Syst. Man Cybern. 19(6), 1426–1446 (1989)CrossRefGoogle Scholar
  17. 17.
    Hesch, J.A., Roumeliotis, S.I.: A direct least-squares (DLS) method for PnP. In: International Conference on Computer Vision, pp. 383–390. IEEE (2011)Google Scholar
  18. 18.
    Higham, N.J.: Functions of Matrices: Theory and Computation. Society for Industrial and Applied Mathematics, Philadelphia (2008)CrossRefGoogle Scholar
  19. 19.
    Hmam, H.: Quadratic optimisation with one quadratic equality constraint. Technical Report 2416, Defence Science and Technology Organisation, Australia (2010)Google Scholar
  20. 20.
    Horn, B.K., Hilden, H.M., Negahdaripour, S.: Closed-form solution of absolute orientation using orthonormal matrices. JOSA A 5(7), 1127–1135 (1988)MathSciNetCrossRefGoogle Scholar
  21. 21.
    Johnson, D.S., Garey, M.R.: Computers and Intractability: A Guide to the Theory of NP-Completeness. WH Freeman, New York (1979)zbMATHGoogle Scholar
  22. 22.
    Kim, S., Kojima, M.: Second order cone programming relaxation of nonconvex quadratic optimization problems. Optim. Methods Softw. 15(3–4), 201–224 (2001)MathSciNetCrossRefGoogle Scholar
  23. 23.
    Klein, G., Murray, D.: Parallel tracking and mapping on a camera phone. In: IEEE International Symposium on Mixed and Augmented Reality, pp. 83–86. IEEE (2009)Google Scholar
  24. 24.
    Kneip, L., Li, H., Seo, Y.: UPnP: an optimal O(n) solution to the absolute pose problem with universal applicability. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8689, pp. 127–142. Springer, Cham (2014).  https://doi.org/10.1007/978-3-319-10590-1_9CrossRefGoogle Scholar
  25. 25.
    Kneip, L., Scaramuzza, D., Siegwart, R.: A novel parametrization of the perspective-three-point problem for a direct computation of absolute camera position and orientation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2969–2976. IEEE (2011)Google Scholar
  26. 26.
    Kukelova, Z., Bujnak, M., Pajdla, T.: Polynomial eigenvalue solutions to minimal problems in computer vision. IEEE Trans. Pattern Anal. Mach. Intell. 34(7), 1381–1393 (2012)CrossRefGoogle Scholar
  27. 27.
    Kukelova, Z., Bujnak, M., Pajdla, T.: Automatic generator of minimal problem solvers. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008. LNCS, vol. 5304, pp. 302–315. Springer, Heidelberg (2008).  https://doi.org/10.1007/978-3-540-88690-7_23CrossRefGoogle Scholar
  28. 28.
    Lepetit, V., Moreno-Noguer, F., Fua, P.: EPnP: an accurate O(n) solution to the PnP problem. Int. J. Comput. Vis. 81(2), 155 (2009)CrossRefGoogle Scholar
  29. 29.
    Li, S., Xu, C., Xie, M.: A robust O(n) solution to the perspective-n-point problem. IEEE Trans. Pattern Anal. Mach. Intell. 34(7), 1444–1450 (2012)CrossRefGoogle Scholar
  30. 30.
    Lourakis, M.: An efficient solution to absolute orientation. In: International Conference on Pattern Recognition (ICPR), pp. 3816–3819 (2016)Google Scholar
  31. 31.
    Lourakis, M., Terzakis, G.: Efficient absolute orientation revisited. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 5813–5818 (2018)Google Scholar
  32. 32.
    Lourakis, M., Zabulis, X.: Model-based pose estimation for rigid objects. In: Chen, M., Leibe, B., Neumann, B. (eds.) ICVS 2013. LNCS, vol. 7963, pp. 83–92. Springer, Heidelberg (2013).  https://doi.org/10.1007/978-3-642-39402-7_9CrossRefGoogle Scholar
  33. 33.
    Lourakis, M.I.A., Argyros, A.A.: Efficient, causal camera tracking in unprepared environments. Comput. Vis. Image Underst. 99(2), 259–290 (2005)CrossRefGoogle Scholar
  34. 34.
    Lu, C.P., Hager, G.D., Mjolsness, E.: Fast and globally convergent pose estimation from video images. IEEE Trans. Pattern Anal. Mach. Intell. 22(6), 610–622 (2000)CrossRefGoogle Scholar
  35. 35.
    Mora, T.: Solving Polynomial Equation Systems II: Macaulay’s Paradigm and Gröbner Technology, vol. 2. Cambridge University Press, Cambridge (2003)CrossRefGoogle Scholar
  36. 36.
    Mur-Artal, R., Montiel, J.M.M., Tardós, J.D.: ORB-SLAM: a versatile and accurate monocular SLAM system. IEEE Trans. Rob. 31(5), 1147–1163 (2015)CrossRefGoogle Scholar
  37. 37.
    Nakano, G.: Globally optimal DLS method for PnP problem with Cayley parameterization. In: British Machine Vision Conference, pp. 78.1-78.11 (2015)Google Scholar
  38. 38.
    Nocedal, J., Wright, S.J.: Sequential Quadratic Programming, pp. 529–562. Springer, New York (2006).  https://doi.org/10.1007/978-0-387-40065-5_18CrossRefGoogle Scholar
  39. 39.
    Nousias, S., Lourakis, M., Bergeles, C.: Large-scale, metric structure from motion for unordered light fields. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3292–3301 (2019)Google Scholar
  40. 40.
    Ohayon, S., Rivlin, E.: Robust 3D head tracking using camera pose estimation. In: International Conference on Pattern Recognition (ICPR), vol. 1, pp. 1063–1066. IEEE (2006)Google Scholar
  41. 41.
    Pardalos, P.M., Vavasis, S.A.: Quadratic programming with one negative eigenvalue is NP-hard. J. Glob. Optim. 1(1), 15–22 (1991)MathSciNetCrossRefGoogle Scholar
  42. 42.
    Romea, A.C., Torres, M.M., Srinivasa, S.: The MOPED framework: object recognition and pose estimation for manipulation. Int. J. Rob. Res. 30(10), 1284–1306 (2011)CrossRefGoogle Scholar
  43. 43.
    Rosten, E., Reitmayr, G., Drummond, T.: Improved RANSAC performance using simple, iterative minimal-set solvers. CoRR abs/1007.1432 (2010)Google Scholar
  44. 44.
    Schönberger, J.L., Frahm, J.M.: Structure-from-motion revisited. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4104–4113 (2016)Google Scholar
  45. 45.
    Schweighofer, G., Pinz, A.: Globally optimal O(n) solution to the PnP problem for general camera models. In: British Machine Vision Conference, pp. 1–10 (2008)Google Scholar
  46. 46.
    Shuster, M.: A survey of attitude representations. J. Astronaut. Sci. 41(4), 439–517 (1993)MathSciNetGoogle Scholar
  47. 47.
    Terzakis, G., Lourakis, M., Ait-Boudaoud, D.: Modified Rodrigues parameters: an efficient representation of orientation in 3D vision and graphics. J. Math. Imaging Vis. 60(3), 422–442 (2018)MathSciNetCrossRefGoogle Scholar
  48. 48.
    Urban, S., Leitloff, J., Hinz, S.: MLPnP - a real-time maximum likelihood solution to the perspective-n-point problem. ISPRS Ann. Photogram. Remote Sens. Spat. Inf. Sci. 3, 131–138 (2016)CrossRefGoogle Scholar
  49. 49.
    Wang, P., Xu, G., Cheng, Y., Yu, Q.: A simple, robust and fast method for the perspective-n-point problem. Pattern Recogn. Lett. 108, 31–37 (2018)CrossRefGoogle Scholar
  50. 50.
    Zheng, E., Wu, C.: Structure from motion using structure-less resection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2075–2083 (2015)Google Scholar
  51. 51.
    Zheng, Y., Kuang, Y., Sugimoto, S., Åström, K., Okutomi, M.: Revisiting the PnP problem: a fast, general and optimal solution. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2344–2351 (2013)Google Scholar
  52. 52.
    Zheng, Y., Sugimoto, S., Okutomi, M.: ASPnP: an accurate and scalable solution to the perspective-n-point problem. IEICE Trans. Inf. Syst. 96(7), 1525–1535 (2013)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Rovco, The QuorumBristolUK
  2. 2.Foundation for Research and Technology – HellasHeraklionGreece

Personalised recommendations