International Journal of Computer Vision

, Volume 38, Issue 2, pp 153–171 | Cite as

Regularized Bundle-Adjustment to Model Heads from Image Sequences without Calibration Data

Article

Abstract

We address the structure-from-motion problem in the context of head modeling from video sequences for which calibration data is not available. This task is made challenging by the fact that correspondences are difficult to establish due to lack of texture and that a quasi-euclidean representation is required for realism.

We have developed an approach based on regularized bundle-adjustment. It takes advantage of our rough knowledge of the head's shape, in the form of a generic face model. It allows us to recover relative head-motion and epipolar geometry accurately and consistently enough to exploit a previously-developed stereo-based approach to head modeling. In this way, complete and realistic head models can be acquired with a cheap and entirely passive sensor, such as an ordinary video camera, with minimal manual intervention.

We chose to demonstrate and evaluate our technique mainly in the context of head-modeling. We do so because it is the application for which all the tools required to perform the complete reconstruction are available to us. We will, however, argue that the approach is generic and could be applied to other tasks, such as body modeling, for which generic facetized models exist.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Baratoff, G. and Aloimonos, Y. 1998. Changes in surface convexity and topography caused by distorsions of stereoscopic visual space. In European Conference on Computer Vision, Freiburg, Germany, pp. 188–202.Google Scholar
  2. Beardsley, P.A., Zissermam, A., and Murray, D.W. 1997. Sequential update of projective and affine structure from motion. International Journal of Computer Vision, 23(3):235–259.Google Scholar
  3. Beaton, A.E. and Turkey, J.W. 1974. The fitting of power series, meaning polynomials, illustrated on band-spectroscopic data. Technometric, 16:147–185.Google Scholar
  4. Blanz, V. and Vetter, T. 1999. A morphable model for the synthesis of 3-D faces. In Computer Graphics, SIGGRAPH Proceedings, Log Angeles, CA, pp. 71–78.Google Scholar
  5. Boulic, R., Capin, T., Huang, Z., Moccozet, L., Molet, T., Kalra, P., Lintermann, B., Magnenat-Thalmann, N., Pandzic, I., Saar, K., Schmitt, A., Shen, J., and Thalmann, D. 1995. Environment for interactive animation of multiple deformable human characters. In Eurographics, Maastricht, Netherlands, pp. 337–348.Google Scholar
  6. DeCarlo, D. and Metaxas, D. 1998. Deformable model-based shape and motion analysis from images using motion residual error. In International Conference on Computer Vision, Bombay, India, pp. 113–119.Google Scholar
  7. Devernay, F. and Faugeras, O.D. 1994. Computing differential properties of 3-D shapes from stereoscopic images without 3-D models. In Conference on Computer Vision and Pattern Recognition, Seattle, WA, pp. 208–213.Google Scholar
  8. Faugeras, O.D., Luong, Q.-T., and Maybank, S.J. 1992. Camera selfcalibration: theory and experiments. In European Conference on Computer Vision, Santa-Margerita, Italy, pp. 321–334.Google Scholar
  9. Fitzgibbon, A.W. and Zisserman, A. 1998. Automatic camera recovery for closed or open image sequences. In European Conference on Computer Vision, Freiburg, Germany, pp. 311–326.Google Scholar
  10. Fua, P. 1993. A parallel stereo algorithm that produces dense depth maps and preserves image features. Machine Vision and Applications, 6(1):35–49.Google Scholar
  11. Fua, P. and Leclerc, Y.G. 1995. Object-centered surface reconstruction: Combining multi-image stereo and shading. International Journal of Computer Vision, 16:35–56.Google Scholar
  12. Fua, P. and Miccio C. 1998. From regular images to animated heads: A least squares approach. In European Conference on Computer Vision, Freiburg, Germany, pp. 188–202.Google Scholar
  13. Fua, P. and Miccio, C. 1999. Animated heads from ordinary images: A least squares approach. Computer Vision and Image Understanding, 75(3):247–259.Google Scholar
  14. Gruen, A. and Beyer, H.A. 1992. System calibration through selfcalibration. In Calibration and Orientation of Cameras in Computer Vision, Washington D.C.Google Scholar
  15. Hartley, R.I., Gupta, R., and Chang, T. 1992. Stereo from uncalibrated cameras. In Conference on Computer Vision and Pattern Recognition, pp. 761–764.Google Scholar
  16. Jebara, T.S. and Pentland, A. 1997. Parametrized structure from motion for 3D adaptive feedback tracking of faces. In Conference on Computer Vision and Pattern-Recognition, Porto Rico, pp. 144–150.Google Scholar
  17. Kalra, P., Mangili, A., Magnenat Thalmann, N., and Thalmann, D. 1992. Simulation of facial muscle actions based on rational free form deformations. In Eurographics.Google Scholar
  18. Kang, S.B. 1997. A structure from motion approach using constrained deformable models and apperance prediction. Technical Report CRL 97/6, Digial, Cambridge Research Laboratory.Google Scholar
  19. Lanitis, A., Taylor, C.J., and Cootes, T.F. 1995. A united approach of coding and interpreting face images. In International Conference on Computer Vision, Cambridge, MA, pp. 975–980.Google Scholar
  20. Leclerc, Y.G. and Bobick, A.F. 1991. The direct computation of height from shading. In Conference on Computer Vision and Pattern Recognition, Lahaina, Maui, Hawaii.Google Scholar
  21. Lee, W.S. and Magnenat Thalmann, N. 1998. From real faces to virtual faces: Problems and solutions. In 31A, Limoges, France.Google Scholar
  22. Lee, Y. Terzopoulos, D., and Waters, K. 1995. Realistic modeling for facial animation. In Computer Graphics, SIGGRAPH Proceedings, Los Angeles, CA, pp. 191–198.Google Scholar
  23. Luong, Q.-T. and Viéville, T. 1996. Canonical representations for the geometries of multiple projective views. Computer Vision and Image Understanding, 64(2):193–229.Google Scholar
  24. Pighin, F., Hecker, J., Lischinski, D., Szeliski, R., and Salesin, D.H. 1998. Synthesizing realistic facial expressions from photographs. In Computer Graphics, SIGGRAPH Proceedings, Vol. 26, pp. 75–84.Google Scholar
  25. Pollefeys, M., Koch, R., and VanGool, L. 1998. Self-calibration amd metric reconstruction in spite of varying and unknown internal camera parameters. In International Conference on Computer Vision.Google Scholar
  26. Press, W.H., Flannery, B.P., Teukolsky, S.A., and Vetterling, W.T. 1986. Numerical Recipes, the Art of Scientific Computing. Cambridge U. Press: Cambridge, MA.Google Scholar
  27. Proesmans, M., Van, Gool, L., and Oosterlinck, A. 1996. Active acquisition of 3D shape for moving objects. In International Conference on Image Processing, Lausanne, Switzerland.Google Scholar
  28. Samaras, D. and Metaxas, D. 1998. Incorporating illumination constraints in deformable models. In Conference on Computer Vision and Pattern Recognition, Santa Barbara, pp. 322–329.Google Scholar
  29. Sturm, P. 1997. Critical motion sequences for monocular self-calibration and uncalibrated euclidean reconstruction. In Conference on Computer Vision and Pattern Recognition, Puerto Rico, pp. 1100–1105.Google Scholar
  30. Tang, L. and Huang, T.S. 1996. Analysis-based facial expression synthesis. ICIP-III, 94:98–102.Google Scholar
  31. Tarel, J.P. and Vezien, J.M. 1996. Camcal manual: A complete software solution for camera calibration. Technical Report 0196, INRIA.Google Scholar
  32. Thalmann, D., Shen, J., and Chauvineau, E. 1996. Fast realistic human body deformations for animation and VR applications. In Computer Graphics International, Pohang, Korea.Google Scholar
  33. Triggs, B. 1997. Autocalibration and the absolute quadric. In Conference on Computer Vision and Pattern Recognition, pp. 609–614.Google Scholar
  34. Zhang, Z., Deriche, R., Faugeras, O., and Luong, Q. 1995. A robust technique for matching two uncalibrated images through the recovery of the unknown epipolar geometry. Artificial Intelligence, 78:87–119.Google Scholar
  35. Zienkiewicz, O.C. 1989. The Finite Element Method. McGraw-Hill.Google Scholar
  36. Zisserman, A., Liebowitz, D., and Armstrong, M. 1998. Resolving ambiguities in auto-calibration. Philosophical Transactions: Mathematical, Physical and Engineering Sciences (A), 356(1740):11931–1211.Google Scholar

Copyright information

© Kluwer Academic Publishers 2000

Authors and Affiliations

  • P. Fua
    • 1
  1. 1.Computer Graphics Lab (LIG)Swiss Federal Institute of Technology (EPLausanneSwitzerland

Personalised recommendations