Advertisement

Implicit Non-Rigid Structure-from-Motion with Priors

  • S. I. OlsenEmail author
  • A. Bartoli
Article

Abstract

This paper describes an approach to implicit Non-Rigid Structure-from-Motion based on the low-rank shape model. The main contributions are the use of an implicit model, of matching tensors, a rank estimation procedure, and the theory and implementation of two smoothness priors. Contrarily to most previous methods, the proposed method is fully automatic: it handles a substantial amount of missing data as well as outlier contaminated data, and it automatically estimates the degree of deformation. A major problem in many previous methods is that they generalize badly. Although the estimated model fits the visible training data well, it often predicts the missing data badly. To improve generalization a temporal smoothness prior and a surface shape prior are developed. The temporal smoothness prior constrains the camera trajectory and the configuration weights to behave smoothly. The surface shape prior constrains consistently close image point tracks to have similar implicit structure. We propose an algorithm for achieving a Maximum A Posteriori (map) solution and show experimentally that the map-solution generalizes far better than the prior-free Maximum Likelihood (ml) solution.

Keywords

Computer vision Structure-from-motion Non-rigid Low-rank shape 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Aanæs, H., Kahl, F.: Estimation of deformable structure and motion. In: The Vision and Modeling of Dynamic Scenes Workshop (2002) Google Scholar
  2. 2.
    Bartoli, A., Olsen, S.: A batch algorithm for implicit non-rigid shape and motion recovery. In: Workshop on Dynamical Vision at ICCV’05 (2005) Google Scholar
  3. 3.
    Brand, M.: Morphable 3D models from video. In: Conf. on Computer Vision and Pattern Recognition, pp. 456–463 (2001) Google Scholar
  4. 4.
    Brand, M.: A direct method for 3D factorization of nonrigid motion observed in 2D. In: Conf. on Computer Vision and Pattern Recognition, pp. 122–128 (2005) Google Scholar
  5. 5.
    Bregler, C., Hertzmann, A., Biermann, H.: Recovering non-rigid 3D shape from image streams. In: Conf. on Computer Vision and Pattern Recognition, pp. 690–696 (2000) Google Scholar
  6. 6.
    Buchanan, A.M., Fitzgibbon, A.W.: Damped Newton algorithms for matrix factorization with missing data. In: Conf. on Computer Vision and Pattern Recognition, pp. 316–322 (2005) Google Scholar
  7. 7.
    Del Bue, A., Lladó, X., de Agapito, L.: Non-rigid metric shape and motion recovery from uncalibrated images using priors. In: Conf. on Computer Vision and Pattern Recognition, pp. 1191–1198 (2006) Google Scholar
  8. 8.
    Hartley, R.I., Zisserman, A.: Multiple View Geometry in Computer Vision, 2nd edn. Cambridge University Press, Cambridge (2003) Google Scholar
  9. 9.
    Jacobs, D.W.: Linear fitting with missing data for structure-from-motion. Comput. Vis. Image Underst. 82(1), 57–81 (2001) zbMATHCrossRefGoogle Scholar
  10. 10.
    Martinec, D., Pajdla, T.: 3D reconstruction by fitting low-rank matrices with missing data. In: Conf. on Computer Vision and Pattern Recognition, pp. 198–205 (2005) Google Scholar
  11. 11.
    Olsen, S., Bartoli, A.: Using priors for improving generalization in non-rigid structure-from-motion. In: Proceedings of the British Machine Vision Conference, pp. 1050–1059 (2007) Google Scholar
  12. 12.
    Roy-Chowdhury, A.K.: A measure of deformability of shapes, with application to human motion analysis. In: Conf. on Computer Vision and Pattern Recognition, pp. 398–404 (2005) Google Scholar
  13. 13.
    Tomasi, C., Kanade, T.: Shape and motion from image streams under orthography: A factorization method. Int. J. Comput. Vis. 9(2), 137–154 (1992) CrossRefGoogle Scholar
  14. 14.
    Torr, P.H.S.: Bayesian model estimation and selection for epipolar geometry and generic manifold fitting. Int. J. Comput. Vis. 50(1), 27–45 (2002) CrossRefGoogle Scholar
  15. 15.
    Torr, P.H.S., Zisserman, A.: MLESAC: A new robust estimator with application to estimating image geometry. Comput. Vis. Image Underst. 78, 138–156 (2000) CrossRefGoogle Scholar
  16. 16.
    Torresani, L., Bregler, C.: Space-time tracking. In: European Conference on Computer Vision, pp. 801–812 (2002) Google Scholar
  17. 17.
    Torresani, L., Hertzmann, A.: Automatic non-rigid 3D modeling from video. In: European Conference on Computer Vision, pp. 299–312 (2004) Google Scholar
  18. 18.
    Torresani, L., Hertzmann, A., Bregler, C.: Non-rigid structure-from-motion: Estimating shape and motion with hierarchical priors. In: IEEE PAMI (2007) Google Scholar
  19. 19.
    Torresani, L., Yang, D.B., Alexander, E.J., Bregler, C.: Tracking and modeling non-rigid objects with rank constraints. In: Conf. on Computer Vision and Pattern Recognition, pp. 493–500 (2001) Google Scholar
  20. 20.
    Triggs, B.: Linear projective reconstruction from matching tensors. Image Vis. Comput. 15(8), 617–625 (1997) CrossRefGoogle Scholar
  21. 21.
    Vidal, R., Abretske, D.: Nonrigid shape and motion from multiple perspective views. In: European Conference on Computer Vision, pp. 205–218 (2006) Google Scholar
  22. 22.
    Xiao, J., Chai, J.-X., Kanade, T.: A closed-form solution to non-rigid shape and motion recovery. In: European Conference on Computer Vision, pp. 573–587 (2004) Google Scholar
  23. 23.
    Xiao, J., Kanade, T.: Non-rigid shape and motion recovery: Degenerate deformations. In: International Conference on Computer Vision and Pattern Recognition, pp. 668–675 (2004) Google Scholar
  24. 24.
    Yan, J., Pollefeys, M.: Articulated motion segmentation using RANSAC with priors. In: Workshop on Dynamical Vision (2005) Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2008

Authors and Affiliations

  1. 1.Department of Computer ScienceUniversity of CopenhagenCopenhagen ØDenmark
  2. 2.LASMEA (CNRS / UBP)Clermont-FerrandFrance

Personalised recommendations