International Journal of Computer Vision

, Volume 76, Issue 2, pp 109–122 | Cite as

Fast Non-Rigid Surface Detection, Registration and Realistic Augmentation

Article

Abstract

We present a real-time method for detecting deformable surfaces, with no need whatsoever for a priori pose knowledge.

Our method starts from a set of wide baseline point matches between an undeformed image of the object and the image in which it is to be detected. The matches are used not only to detect but also to compute a precise mapping from one to the other. The algorithm is robust to large deformations, lighting changes, motion blur, and occlusions. It runs at 10 frames per second on a 2.8 GHz PC.We demonstrate its applicability by using it to realistically modify the texture of a deforming surface and to handle complex illumination effects.

Combining deformable meshes with a well designed robust estimator is key to dealing with the large number of parameters involved in modeling deformable surfaces and rejecting erroneous matches for error rates of more than 90%, which is considerably more than what is required in practice.

Keywords

Non-rigid detection Non-rigid augmented reality Real-time deformable registration 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Baker, S., Matthews, I., Xiao, J., Gross, R., Kanade, T., and Ishikawa, T. 2004. Real-time non-rigid driver head tracking for driver mental state estimation. In World Congress on Intelligent Transportation Systems.Google Scholar
  2. Bartoli, A. and Zisserman, A. 2004. {Direct estimation of non-rigid registration}. In British Machine Vision Conference, Kingston, UK.Google Scholar
  3. Belongie, S., Malik, J., and Puzicha, J. 2002. {Shape matching and object recognition using shape contexts}. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(24):509–522.CrossRefGoogle Scholar
  4. Chui, H. and Rangarajan, A. 2003. A new point matching algorithm for non-rigid registration. Computer Vision and Image Understanding, 89(2–3):114–141.MATHCrossRefGoogle Scholar
  5. Cootes, T.F., Edwards, G.J., and Taylor, C.J. 2001. {Active appearance models}. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(6).Google Scholar
  6. DeCarlo, D. and Metaxas, D. 1998. {Deformable model-based shape and motion analysis from images using motion residual error}. In International Conference on Computer Vision, Bombay, India, pp. 113–119.Google Scholar
  7. Dewaele, G., Devernay, F., and Horaud, R. 2004. Hand motion from 3d point trajectories and a smooth surface model. In European Conference on Computer Vision, pp 495–507.Google Scholar
  8. Ferrari, V., Tuytelaars, T., and Van Gool, L. 2004. {Simultaneous object recognition and segmentation by image exploration}. In European Conference on Computer Vision.Google Scholar
  9. Fischler, M.A. and Bolles, R.C. 1981. {Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography}. Communications ACM, 24(6):381–395.CrossRefMathSciNetGoogle Scholar
  10. Fua, P. 1997. {RADIUS: Image Understanding for Intelligence Imagery}. In Chapter Model-Based Optimization: An Approach to Fast, Accurate, and Consistent Site Modeling from Imagery, M. Kaufmann, O. Firschein, and T.M. Strat, (Eds.).Google Scholar
  11. Granger, S. and Pennec, X. 2002. Multi-scale em-icp: A fast and robust approach for surface registration. In European Conference on Computer Vision, Copenhagen, Denmark, pp. 418–432Google Scholar
  12. Gumerov, N.A., Zandifar, A., Duraiswami, R., and Davis, L.S. 2004. {Structure of Applicable Surfaces from Single Views}. In European Conference on Computer Vision, Prague.Google Scholar
  13. Hartley, R. and Zisserman, A. 2000. {Multiple View Geometry in Computer Vision}. Cambridge University Press.Google Scholar
  14. Kass, M., Witkin, A., and Terzopoulos, D. 1988. {Snakes: Active contour models}. International Journal of Computer Vision, 1(4):321–331.CrossRefGoogle Scholar
  15. Lepetit, V. and Fua, P. 2005. Monocular model-based 3d tracking of rigid objects: A survey. Foundations and Trends in Computer Graphics and Vision, 1(1):1–89.CrossRefGoogle Scholar
  16. Lepetit, V., Lagger, P., and Fua, P. 2005. {Randomized trees for real-time keypoint recognition}. In Conference on Computer Vision and Pattern Recognition, San Diego, CA.Google Scholar
  17. Lin, W.-C. and Liu, Y. 2006, May. Tracking dynamic near-regular textures under occlusion and rapid movements. In European Conference on Computer Vision.Google Scholar
  18. Ling, H. and Jacobs, D.W. 2005. Deformation invariant image matching. In International Conference on Computer Vision, Beijing, China, pp. 1466–1473.Google Scholar
  19. Lowe, D.G. 2004. {Distinctive image features from scale-invariant keypoints}. International Journal of Computer Vision, 20(2):91–110.CrossRefGoogle Scholar
  20. Rosten, E. and Drummond, T. 2005. Fusing points and lines for high performance tracking. In International Conference on Computer Vision, Beijing, China.Google Scholar
  21. Sclaroff, S. and Isidoro, J. 2003. Active blobs: region-based, deformable appearance models. Computer Vision and Image Understanding, 89(2–3).Google Scholar
  22. White, R. and Forsyth, D.A. 2006. Retexturing single views using texture and shading. In European Conference on Computer Vision, volume LNCS 3954, pp. 70–81.Google Scholar
  23. Wills, J. and Belongie, S. 2004. A feature-based approach for determining long range optical flow. In European Conference on Computer Vision, Prague, Czech Republic.Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2007

Authors and Affiliations

  1. 1.Computer Vision LaboratoryÉcole Polytechnique Fédérale de Lausanne (EPFL)LausanneSwitzerland

Personalised recommendations