Advertisement

International Journal of Computer Vision

, Volume 119, Issue 2, pp 93–109 | Cite as

Schwarps: Locally Projective Image Warps Based on 2D Schwarzian Derivatives

  • Daniel Pizarro
  • Rahat KhanEmail author
  • Adrien Bartoli
Article

Abstract

Image warps -or just warps- capture the geometric deformation existing between two images of a deforming surface. The current approach to enforce a warp’s smoothness is to penalize its second order partial derivatives (Bookstein in IEEE Trans Pattern Anal Mach Intell 11:567–585, 1989; Rueckert et al. in IEEE Trans Med Imaging 18:712–721, 1999). Because this favors locally affine warps, this fails to capture the local projective component of the image deformation. This may have a negative impact on applications such as image registration and deformable 3D reconstruction. We propose a novel penalty designed to smooth the warp while capturing the deformation’s local projective structure. Our penalty is based on equivalents to the Schwarzian derivatives, which are projective differential invariants exactly preserved by homographies. We propose a methodology to derive a set of partial differential equations with only homographies as solutions. We call this system the Schwarzian equations and we explicitly derive them for 2D functions using differential properties of homographies. We name as Schwarp a warp which is estimated by penalizing the residual of Schwarzian equations. Experimental evaluation shows that Schwarps outperform existing warps in modeling and extrapolation power, and lead to better results in three deformable reconstruction methods, namely, shape reconstruction in shape-from-template, camera calibration in Shape-from-Template and Non-Rigid Structure-from-Motion.

Keywords

Schwarzian penalizer Bending energy Projective differential invariants Image warps 

Notes

Acknowledgments

This research has received funding from the EUs FP7 ERC research Grant 307483 FLEXABLE.

References

  1. Bartoli, A., & Collins, T. (2013a). Template-based isometric deformable 3D reconstruction with sampling-based focal length self-calibration. In Computer Vision and Pattern Recognition.Google Scholar
  2. Bartoli, A., & Collins, T. (2013b). Template-based isometric deformable 3D reconstruction with sampling-based focal length self-calibration. In Computer Vision and Pattern Recognition.Google Scholar
  3. Bartoli, A., Perriollat, M., & Chambon, S. (2010). Generalized thin-plate spline warps. International Journal of Computer Vision, 88(1), 85–110.CrossRefGoogle Scholar
  4. Bartoli, A., Gérard, Y., Chadebecq, F., & Collins, T. (2012). On template-based reconstruction from a single view: Analytical solutions and proofs of well-posedness for developable, isometric and conformal surfaces. In Computer Vision and Pattern Recognition.Google Scholar
  5. Bookstein, F. L. (1989). Principal warps: Thin-plate splines and the decomposition of deformations. IEEE Transaction on Pattern Analysis Machine Intelligence, 11(6), 567–585.CrossRefzbMATHGoogle Scholar
  6. Bregler, C., Hertzmann, A., & Biermann, H. (2000). Recovering non-rigid 3D shape from image streams. In Computer Vision and Pattern Recognition.Google Scholar
  7. Bruckstein, A. M., & Netravali, A. N. (1995). On differential invariants of planar curves and recognizing partially occluded planar shapes. Annals of Mathematics and Artificial Intelligence, 13(3–4), 227–250.MathSciNetCrossRefzbMATHGoogle Scholar
  8. Brunet, F., Bartoli, A., Navab, N., & Malgouyres, R. (2009). NURBS warps. In British Machine Vision Conference.Google Scholar
  9. Calabi, E., Olver, P. J., Shakiban, C., Tannenbaum, A., & Haker, S. (1998). Differential and numerically invariant signature curves applied to object recognition. International Journal of Computer Vision, 26(2), 107–135.CrossRefGoogle Scholar
  10. Carlsson, S. (1996). Projectively invariant decomposition and recognition of planar shapes. International Journal of Computer Vision, 17(2), 193–209.CrossRefGoogle Scholar
  11. Carlsson, S., Mohr, R., Moons, T., Morin, L., Rothwell, C., Van Diest, M., et al. (1996). Semi-local projective invariants for the recognition of smooth plane curves. International Journal of Computer Vision, 19(3), 211–236.CrossRefGoogle Scholar
  12. Cayley, A. (1880). On the Schwarzian derivatives and the polyhedral functions. In Transaction of Cambridge Philosophical Society (vol. 13).Google Scholar
  13. Coddington, E. A. (2012). An introduction to ordinary differential equations. New York: Courier Dover Publications.zbMATHGoogle Scholar
  14. Faugeras, O. D., Luong, Q.-T., & Papadopoulo, T. (2001). The geometry of multiple images: The laws that govern the formation of multiple images of a scene and some of their applications. Cambridge: MIT Press.zbMATHGoogle Scholar
  15. Hartley, R. I., & Zisserman, A. (2003). Multiple view geometry in computer vision. Cambridge University Press, ISBN: 0521623049.Google Scholar
  16. Horn, B., & Schunck, B. (1981). Determining optical flow. Artificial Intelligence, 17, 185–203.CrossRefGoogle Scholar
  17. Khan, R., Pizarro, D., & Bartoli, A. (2014). Schwarps: Locally projective image warps based on 2d schwarzian derivatives. In ECCV.Google Scholar
  18. Kummer, E. E. (1836). Über die hypergeometrische reihe. Journal Fur Die Reine Und Angewandte Mathematik, 1836(15), 39–83.MathSciNetCrossRefGoogle Scholar
  19. Lazebnik, S., & Ponce, J. (2005). The local projective shape of smooth surfaces and their outlines. International Journal of Computer Vision, 63, 65–83.CrossRefGoogle Scholar
  20. Lei, G. (1990). Recognition of planar objects in 3D space from single perspective views using cross ratio. IEEE Transactions on Robotics and Automation, 6(4), 432–437.CrossRefGoogle Scholar
  21. Lowe, D. G. (2004). Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 60(2), 91–110.CrossRefGoogle Scholar
  22. Matsumoto, K., Sasaki, T., & Yoshida, M. (1993). Recent progress of Gauss-Schwartz theory and related geometric structures. Memoirs of the faculty of Science, Kyushu University, 47–2, 283–381.MathSciNetCrossRefzbMATHGoogle Scholar
  23. Molzon, R., & Mortensen, K. P. (1996). The Schwarzian derivative for maps between manifolds with complex projective connections. Transactions of the American Mathematical Society, 348(8), 3015–3036.MathSciNetCrossRefzbMATHGoogle Scholar
  24. Moons, T., Pauwels, E. J., Van Gool, L. J., & Oosterlinck, A. (1995). Foundations of semi-differential invariants. International Journal of Computer Vision, 14(1), 25–47.CrossRefzbMATHGoogle Scholar
  25. Mundy, J. L., & Zisserman, A. (1992). Geometric invariance in computer vision. Cambridge: MIT Press.Google Scholar
  26. Oda, T. (1974). On schwarzian derivatives in several variables (in Japanese). Kokyuroku Research Institute for Mathematical Sciences, 226, 82–85.Google Scholar
  27. Olver, P. J. (2007). Generating differential invariants. Journal of Mathematical Analysis and Applications, 333(1), 450–471.MathSciNetCrossRefzbMATHGoogle Scholar
  28. Osgood, B., & Stowe, D. (1992). The Schwarzian derivative and conformal mapping of Riemannian manifolds. Duke Mathematical Journal, 67(1), 57–99.MathSciNetCrossRefzbMATHGoogle Scholar
  29. Ovsienko, V. (1989). Lagrange Schwarzian derivative. Moscow University Mechanics Bulletin, 44(6), 8–13.zbMATHGoogle Scholar
  30. Ovsienko, V., & Tabachnikov, S. (2005). Projective differential geometry old and new: From the Schwarzian derivative to the cohomology of diffeomorphism groups. Cambridge: Cambridge University Press.Google Scholar
  31. Ovsienko, V., & Tabachnikov, S. (2009). What is the Schwarzian derivative? North American Mathematical Society, 56, 34–36.MathSciNetzbMATHGoogle Scholar
  32. Perriollat, M., Hartley, R., & Bartoli, A. (2011). Monocular template-based reconstruction of inextensible surfaces. International Journal of Computer Vision, 95, 124–137.Google Scholar
  33. Pilet, J., Lepetit, V., & Fua, P. (2007). Fast non-rigid surface detection, registration and realistic augmentation. International Journal of Computer Vision, 76(2), 109–122.CrossRefGoogle Scholar
  34. Rueckert, D., Sonoda, L. I., Hayes, C., Hill, D. L. G., Leach, M. O., & Hawkes, D. J. (1999). Nonrigid registration using free-form deformations: Application to breast MR images. IEEE Transactions on Medical Imaging, 18, 712–721.CrossRefGoogle Scholar
  35. Salzmann, M., Pilet, J., Ilic, S., & Fua, P. (2007). Surface deformation models for nonrigid 3D shape recovery. Transactions on Pattern Analysis and Machine Intelligence, 29(8), 1481–1487.CrossRefGoogle Scholar
  36. Sato, J., & Cipolla, R. (1998). Quasi-invariant parameterisations and matching of curves in images. International Journal of Computer Vision, 28(2), 117–136.CrossRefGoogle Scholar
  37. Singer, D. (1978). Stable orbits and bifurcation of maps of the interval. SIAM Journal of Applied Mathematics, 35(2), 260–267.MathSciNetCrossRefzbMATHGoogle Scholar
  38. Szeliski, R. (2006). Image alignment and stitching: A tutorial. Foundations and Trends in Computer Graphics and Computer Vision, 2(1), 1–104.MathSciNetCrossRefzbMATHGoogle Scholar
  39. Tanner, C. (2005). Registration and lesion classiffication of contrast-enhanced magnetic resonance breast images. PhD thesis, University of London.Google Scholar
  40. Torr, P. (2000). MLESAC: A new robust estimator with application to estimating image geometry. Computer Vision and Image Understanding, 78, 138–156.CrossRefGoogle Scholar
  41. Torresani, L., Hertzmann, A., & Bregler, C. (2008). Non-rigid structure-from-motion: Estimating shape and motion with hierarchical priors. IEEE Transactions on Pattern Analysis and Machine Intelligence., 30(5), 878–892.CrossRefGoogle Scholar
  42. Van Gool, L., Moons, T., Pauwels, E., & Oosterlinck, A. (1995). Vision and lie’s approach to invariance. Image and vision computing, 13(4), 259–277.CrossRefGoogle Scholar
  43. Varol, A., Salzmann, M., Tola, E., & Fua, P. (2009). Template-free monocular reconstruction of deformable surfaces. In International conference on computer vision.Google Scholar
  44. Weiss, I. (1993). Geometric invariants and object recognition. International journal of computer vision, 10(3), 207–231.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2016

Authors and Affiliations

  1. 1.ISIT, UMR 6284 CNRS-UdAClermont-FerrandFrance

Personalised recommendations