Advertisement

Self-Calibrating Isometric Non-Rigid Structure-from-Motion

  • Shaifali ParasharEmail author
  • Adrien Bartoli
  • Daniel Pizarro
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11205)

Abstract

We present self-calibrating isometric non-rigid structure-from-motion (SCIso-NRSfM), the first method to reconstruct a non-rigid object from at least three monocular images with constant but unknown focal length. The majority of NRSfM methods using the perspective camera simply assume that the calibration is known. SCIso-NRSfM leverages the recent powerful differential approaches to NRSfM, based on formulating local polynomial constraints, where local means correspondence-wise. In NRSfM, the local shape may be solved from these constraints. In SCIso-NRSfM, the difficulty is to also solve for the focal length as a global variable. We propose to eliminate the shape using resultants, obtaining univariate polynomials for the focal length only, whose sum of squares can then be globally minimized. SCIso-NRSfM thus solves for the focal length by integrating the constraints for all correspondences and the whole image set. Once this is done, the local shape is easily recovered. Our experiments show that its performance is very close to the state-of-the-art methods that use a calibrated camera.

Keywords

NRSfM Self-calibration Uncalibrated camera Differential geometry Metric tensor Christoffel symbols Resultants 

Notes

Acknowledgments

This research has received funding from the EU’s FP7 through the ERC research grant 307483 FLEXABLE, the Spanish Ministry of Economy, Industry and Competitiveness under project ARTEMISA (TIN2016-80939- R) and the University of Alcalá, Spain under the project SEQUENCE (CCGP2017-EXP/048).

References

  1. 1.
    Bartoli, A., Gérard, Y., Chadebecq, F., Collins, T., Pizarro, D.: Shape-from-template. IEEE Trans. Pattern Anal. Mach. Intell. 37(10), 2099–2118 (2015)CrossRefGoogle Scholar
  2. 2.
    Bartoli, A., Pizarro, D., Collins, T.: A robust analytical solution to isometric shape-from-template with focal length calibration. In: ICCV (2013)Google Scholar
  3. 3.
    Bregler, C., Hertzmann, A., Biermann, H.: Recovering non-rigid 3D shape from image streams. In: CVPR (2000)Google Scholar
  4. 4.
    Chhatkuli, A., Pizarro, D., Bartoli, A.: Non-rigid shape-from-motion for isometric surfaces using infinitesimal planarity. In: BMVC (2014)Google Scholar
  5. 5.
    Chhatkuli, A., Pizarro, D., Collins, T., Bartoli, A.: Inextensible non-rigid structure-from-motion by second-order cone programming. IEEE Trans. Pattern Anal. Mach. Intell. 1 (2017)Google Scholar
  6. 6.
    Dai, Y., Li, H., He, M.: A simple prior-free method for non-rigid structure-from-motion factorization. Int. J. Comput. Vis. 107(2), 101–122 (2014)MathSciNetCrossRefGoogle Scholar
  7. 7.
    Del Bue, A., Smeraldi, F., Agapito, L.: Non-rigid structure from motion using non-parametric tracking and non-linear optimization. In: CVPRW (2004)Google Scholar
  8. 8.
    Faugeras, O.D., Luong, Q.-T., Maybank, S.J.: Camera self-calibration: theory and experiments. In: Sandini, G. (ed.) ECCV 1992. LNCS, vol. 588, pp. 321–334. Springer, Heidelberg (1992).  https://doi.org/10.1007/3-540-55426-2_37CrossRefGoogle Scholar
  9. 9.
    Gotardo, P., Martinez, A.: Kernel non-rigid structure from motion. In: ICCV (2011)Google Scholar
  10. 10.
    Gurdjos, P., Sturm, P.: Methods and geometry for plane-based self-calibration. In: CVPR (2003)Google Scholar
  11. 11.
    Hartley, R.I.: Self-calibration from multiple views with a rotating camera. In: Eklundh, J.-O. (ed.) ECCV 1994. LNCS, vol. 800, pp. 471–478. Springer, Heidelberg (1994).  https://doi.org/10.1007/3-540-57956-7_52CrossRefGoogle Scholar
  12. 12.
    Hartley, R.I., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press, New York (2000). ISBN: 0521623049zbMATHGoogle Scholar
  13. 13.
    Henrion, D., Lasserre, J.B.: GloptiPoly: global optimization over polynomials with Matlab and SeDuMi. ACM Trans. Math. Softw. 29(2), 165–194 (2003)MathSciNetCrossRefGoogle Scholar
  14. 14.
    Heyden, A., Astrom, K.: Euclidean reconstruction from constant intrinsic parameters. In: ICPR (1996)Google Scholar
  15. 15.
    Kahl, F., Triggs, B., Astrom, A.: Critical motions for auto-calibration when some intrinsic parameters can vary. J. Math. Image Vis. 13(2), 131–146 (2000)MathSciNetCrossRefGoogle Scholar
  16. 16.
    Lee, J.: Riemannian manifolds : an introduction to curvature. Springer, New York (1997).  https://doi.org/10.1007/b98852CrossRefzbMATHGoogle Scholar
  17. 17.
    Lladó, X., Del Bue, A., Agapito, L.: Non-rigid metric reconstruction from perspective cameras. Image Vis. Comput. 28(9), 1339–1353 (2010)CrossRefGoogle Scholar
  18. 18.
    Parashar, S., Pizarro, D., Bartoli, A.: Isometric non-rigid shape-from-motion with Riemannian geometry solved in linear time. IEEE Trans. Pattern Anal. Mach. Intell. (2017)Google Scholar
  19. 19.
    Pizarro, D., Khan, R., Bartoli, A.: Schwarps: locally projective image warps based on 2D schwarzian derivatives. Int. J. Comput. Vis. 119(2), 93–109 (2016)MathSciNetCrossRefGoogle Scholar
  20. 20.
    Pollefeys, M., Koch, R., Van Gool, L.: Self-calibration and metric reconstruction in spite of varying and unknown internal camera parameters. In: ICCV (1998)Google Scholar
  21. 21.
    Pollefeys, M., Van Gool, L.: Stratified self-calibration with the modulus constraint. IEEE Trans. Pattern Anal. Mach. Intell. 21(8), 707–724 (1999)CrossRefGoogle Scholar
  22. 22.
    Ramachandran, M., Veeraraghavan, A., Chellappa, R.: A fast bilinear structure from motion algorithm using a video sequence and inertial sensors. IEEE Trans. Pattern Anal. Mach. Intell. 33(1), 186–193 (2011)CrossRefGoogle Scholar
  23. 23.
    Ramalingam, S., Lodha, S., Sturm, P.: A generic structure-from-motion framework. Comput. Vis. Image Underst. 103(1), 218–228 (2006)CrossRefGoogle Scholar
  24. 24.
    Russell, C., Yu, R., Agapito, L.: Video pop-up: monocular 3D reconstruction of dynamic scenes. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8695, pp. 583–598. Springer, Cham (2014).  https://doi.org/10.1007/978-3-319-10584-0_38CrossRefGoogle Scholar
  25. 25.
    Salzmann, M., Fua, P.: Linear local models for monocular reconstruction of deformable surfaces. IEEE Trans. Pattern Anal. Mach. Intell. 33(5), 931–944 (2011)CrossRefGoogle Scholar
  26. 26.
    Snavely, N., Seitz, S.M., Szeliski, R.: Modeling the world from internet photo collections. Int. J. Comput. Vis. 80(1), 189–210 (2008)CrossRefGoogle Scholar
  27. 27.
    Sturm, P.: Critical motion sequences for monocular self- calibration and uncalibrated Euclidean reconstruction. In: CVPR (1997)Google Scholar
  28. 28.
    Taylor, J., Jepson, A.D., Kutulakos, K.N.: Non-rigid structure from locally-rigid motion. In: CVPR (2010)Google Scholar
  29. 29.
    Tomasi, C., Kanade, T.: Shape and motion from image streams under orthography: a factorization method. Int. J. Comput. Vis. 9(2), 137–154 (1992)CrossRefGoogle Scholar
  30. 30.
    Torresani, L., Hertzmann, A., Bregler, C.: Nonrigid structure-from-motion: estimating shape and motion with hierarchical priors. IEEE Trans. Pattern Anal. Mach. Intell. 30(5), 878–892 (2008)CrossRefGoogle Scholar
  31. 31.
    Triggs, B.: Autocalibration and the absolute quadric. In: CVPR (1997)Google Scholar
  32. 32.
    Varol, A., Salzmann, M., Fua, P., Urtasun, R.: A constrained latent variable model. In: CVPR (2012)Google Scholar
  33. 33.
    Varol, A., Salzmann, M., Tola, E., Fua, P.: Template-free monocular reconstruction of deformable surfaces. In: ICCV (2009)Google Scholar
  34. 34.
    Vicente, S., Agapito, L.: Soft inextensibility constraints for template-free non-rigid reconstruction. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7574, pp. 426–440. Springer, Heidelberg (2012).  https://doi.org/10.1007/978-3-642-33712-3_31CrossRefGoogle Scholar
  35. 35.
    Van der Waerden, B.L.: Modern Algebra. Springer, New York (2003)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Shaifali Parashar
    • 1
    Email author
  • Adrien Bartoli
    • 1
  • Daniel Pizarro
    • 1
    • 2
  1. 1.Institut Pascal - CNRS/Université Clermont AuvergneClermont-FerrandFrance
  2. 2.GEINTRAUniversidad de AlcaláAlcalá de HenaresSpain

Personalised recommendations