Advertisement

Journal of Mathematical Imaging and Vision

, Volume 61, Issue 7, pp 1051–1068 | Cite as

A Differential–Algebraic Projective Framework for the Deformable Single-View Geometry of the 1D Perspective Camera

  • Adrien BartoliEmail author
Article
  • 200 Downloads

Abstract

Single-View Geometry (SVG) studies the world-to-image mapping or warp, which is the relationship that exists between a body’s model and its image. For a rigid body observed by a projective camera, the warp is described by the usual camera matrix and its properties. However, it is clear that for a body whose deformation state changes between the body’s model and its image, the ‘simple,’ globally parameterized warp described solely by the camera matrix, breaks down. Existing work has exploited deformation to reconstruct the deformed body from its image, but did not establish the properties of the deformable warp. Studying these properties is part of deformable SVG and forms a recent research topic. Because deformations may take place anywhere in the object’s body, and because they may be uncorrelated, the warp is local in nature. Using a differential framework is thus an obvious choice. We propose a differential–algebraic projective framework based on modeling the body’s surface by a locally rational projective embedding and on the 1D projective camera. We show that this leads, via the study of univariate rational functions, to differential invariants that the warp must satisfy. It may seem surprising, given the generic hypothesis made on the observed body, hardly stronger than mere local smoothness, that constraints can still be found. Our framework generalizes the Schwarzian derivative, the first-order projective differential invariant, which holds under the assumption that the body’s shape is locally linear. Our invariants may be used to construct regularizers to be used in warp estimation. We report experimental results of two types on simulated and real data. The first type shows that the proposed invariants hold well for an independently estimated warp. The second type shows that the proposed regularizers improve warp estimation from point correspondences compared to the classical derivative-penalizing regularizers.

Keywords

Visual geometry Deformation Rational function Differential invariant 

Notes

Acknowledgements

This research has received funding from the EU’s FP7 through the ERC research grant 307483 FLEXABLE. We thank the authors of [11] for the real dataset and Yan Gérard for his kind feedback on the paper.

Supplementary material

References

  1. 1.
    Bartoli, A., Gérard, Y., Chadebecq, F., Collins, T., Pizarro, D.: Shape-from-template. IEEE Trans. Pattern Anal. Mach. Intell. 37(10), 2099–2118 (2015)CrossRefGoogle Scholar
  2. 2.
    Bishop, C.M.: Pattern Recognition and Machine Learning. Springer, Berlin (2006)zbMATHGoogle Scholar
  3. 3.
    Bookstein, F .L.: Principal warps: thin-plate splines and the decomposition of deformations. IEEE Trans. Pattern Anal. Mach. Intell. 11(6), 567–585 (1989)CrossRefzbMATHGoogle Scholar
  4. 4.
    Duchon, J.: Interpolation des fonctions de deux variables suivant le principe de la flexion des plaques minces. RAIRO Anal. Numér. 10, 5–12 (1976)MathSciNetGoogle Scholar
  5. 5.
    Fabbri, R., Kimia, B.B.: Multiview differential geometry of curves. Int. J. Comput. Vis. 117(3), 1–23 (2016)MathSciNetzbMATHGoogle Scholar
  6. 6.
    Faugeras, O.: Three-Dimensional Computer Vision. MIT Press, Cambridge (1993)Google Scholar
  7. 7.
    Faugeras, O., Mourrain, B.: On the geometry and algebra of the point and line correspondences between \(n\) images. In: International Conference on Computer Vision (1995)Google Scholar
  8. 8.
    Faugeras, O., Quan, L., Sturm, P.: Self-calibration of a 1d projective camera and its application to the self-calibration of a 2d projective camera. IEEE Trans. Pattern Anal. Mach. Intell. 22(10), 1179–1185 (2000)CrossRefGoogle Scholar
  9. 9.
    Forsyth, D., Ponce, J.: Computer Vision—A Modern Approach, International edn. Pearson, London (2012)Google Scholar
  10. 10.
    Fortun, D., Bouthemy, P., Kervrann, C.: Optical flow modeling and computation: a survey. Comput. Vis. Image Underst. 134, 1–21 (2015)CrossRefzbMATHGoogle Scholar
  11. 11.
    Gallardo, M., Pizarro, D., Bartoli, A., Collins, T.: Shape-from-template in flatland. In: International Conference on Computer Vision and Pattern Recognition (2015)Google Scholar
  12. 12.
    Gumerov, N.A., Zandifar, A., Duraiswami, R., Davis, L.S.: 3D structure recovery and unwarping surfaces applicable to planes. Int. J. Comput. Vis. 66(3), 261–281 (2006)CrossRefGoogle Scholar
  13. 13.
    Hartley, R .I., Zisserman, A.: Multiple View Geometry in Computer Vision, 2nd edn. Cambridge University Press, Cambridge (2003)zbMATHGoogle Scholar
  14. 14.
    Kaminski, J., Shashua, A.: Multiple view geometry of general algebraic curves. Int. J. Comput. Vis. 56(3), 195–219 (2004)CrossRefGoogle Scholar
  15. 15.
    Ovsienko, V., Tabachnikov, S.: Projective Differential Geometry Old and New. Cambridge University Press, Cambridge (2005)zbMATHGoogle Scholar
  16. 16.
    Ovsienko, V., Tabachnikov, S.: What is... the schwarzian derivative? Not. AMS 56(1), 34–36 (2009)MathSciNetzbMATHGoogle Scholar
  17. 17.
    Parashar, S., Pizarro, D., Bartoli, A.: Isometric non-rigid shape-from-motion with riemannian geometry solved in linear time. IEEE Trans. Pattern Anal. Mach. Intell. 40(10), 2442–2454 (2018)CrossRefGoogle Scholar
  18. 18.
    Perriollat, M., Hartley, R., Bartoli, A.: Monocular template-based reconstruction of inextensible surfaces. Int. J. Comput. Vis. 95(2), 124–137 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  19. 19.
    Piegl, L., Tiller, W.: The NURBS Book. Monographs in Visual Communication, 2nd edn. Springer, Berlin (1997)zbMATHGoogle Scholar
  20. 20.
    Pizarro, D., Khan, R., Bartoli, A.: Schwarps: Locally projective image warps based on 2D schwarzian derivatives. Int. J. Comput. Vis. 119(2), 93–109 (2016)MathSciNetCrossRefzbMATHGoogle Scholar
  21. 21.
    Press, W .H., Teukolsky, S .A., Vetterling, W .T., Flannery, B.P.: Numerical Recipes—The Art of Scientific Computing, 3rd edn. Cambridge University Press, Cambridge (2007)zbMATHGoogle Scholar
  22. 22.
    Quan, L., Kanade, T.: Affine structure from line correspondences with uncalibrated affine cameras. IEEE Trans. Pattern Anal. Mach. Intell. 19(8), 834–845 (1997)CrossRefGoogle Scholar
  23. 23.
    Salzmann, M., Pilet, J., Ilic, S., Fua, P.: Surface deformation models for nonrigid 3D shape recovery. IEEE Trans. Pattern Anal. Mach. Intell. 29(8), 1–7 (2007)CrossRefGoogle Scholar
  24. 24.
    Schmid, C., Zisserman, A.: The geometry and matching of lines and curves over multiple views. Int. J. Comput. Vis. 40(3), 199–234 (2000)CrossRefzbMATHGoogle Scholar
  25. 25.
    Sendra, J.R., Winkler, F., Prez-Daz, S.: Rational Algebraic Curves A Computer Algebra Approach. Springer, Berlin (2008)CrossRefGoogle Scholar
  26. 26.
    Walker, R.J.: Algebraic Curves. Springer, Berlin (1978)CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.ENCOV, Institut Pascal UMR6602 CNRSUniversité Clermont Auvergne, SIGMAAubièreFrance

Personalised recommendations